Dirichlet negative multinomial distribution

The Dirichlet distribution is a conjugate distribution to the negative multinomial distribution. This fact leads to an analytically tractable compound distribution. For a random vector of category counts ${\displaystyle \mathbf {x} =(x_{1},\dots ,x_{m})}$, distributed according to a negative multinomial distribution, the compound distribution is obtained by integrating on the distribution for p which can be thought of as a random vector following a Dirichlet distribution:

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})=\int _{\mathbf {p} }\mathrm {NegMult} (\mathbf {x} \mid x_{0},\mathbf {p} )\mathrm {Dir} (\mathbf {p} \mid \alpha _{0},{\boldsymbol {\alpha }}){\textrm {d}}\mathbf {p} }$

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {\Gamma \left(\sum _{i=0}^{m}{x_{i}}\right)}{\Gamma (x_{0})\prod _{i=1}^{m}x_{i}!}}{\frac {1}{\mathrm {B} ({\boldsymbol {\alpha }}_{+})}}\int _{\mathbf {p} }\prod _{i=0}^{m}p_{i}^{x_{i}+\alpha _{i}-1}{\textrm {d}}\mathbf {p} }$

which results in the following formula:

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {\Gamma \left(\sum _{i=0}^{m}{x_{i}}\right)}{\Gamma (x_{0})\prod _{i=1}^{m}x_{i}!}}{\frac {{\mathrm {B} }(\mathbf {x_{+}} +{\boldsymbol {\alpha }}_{+})}{\mathrm {B} ({\boldsymbol {\alpha }}_{+})}}}$

where ${\displaystyle \mathbf {x_{+}} }$ and ${\displaystyle {\boldsymbol {\alpha }}_{+}}$ are the ${\displaystyle m+1}$ dimensional vectors created by appending the scalars ${\displaystyle x_{0}}$ and ${\displaystyle \alpha _{0}}$ to the ${\displaystyle m}$ dimensional vectors ${\displaystyle \mathbf {x} }$ and ${\displaystyle {\boldsymbol {\alpha }}}$ respectively and ${\displaystyle \mathrm {B} }$ is the multivariate version of the beta function. We can write this equation explicitly as

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})=x_{0}{\frac {\Gamma (\sum _{i=0}^{m}x_{i})\Gamma (\sum _{i=0}^{m}\alpha _{i})}{\Gamma (\sum _{i=0}^{m}(x_{i}+\alpha _{i}))}}\prod _{i=0}^{m}{\frac {\Gamma (x_{i}+\alpha _{i})}{\Gamma (x_{i}+1)\Gamma (\alpha _{i})}}.}$

Alternative formulations exist. One convenient representation[1] is

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {\Gamma (x_{\bullet })}{\Gamma (x_{0})\prod _{i=1}^{m}\Gamma (x_{i}+1)}}\times {\frac {\Gamma (\alpha _{\bullet })}{\prod _{i=0}^{m}\Gamma (\alpha _{i})}}\times {\frac {\prod _{i=0}^{m}\Gamma (x_{i}+\alpha _{i})}{\Gamma (x_{\bullet }+\alpha _{\bullet })}}}$

where ${\displaystyle x_{\bullet }=x_{0}+x_{1}+\cdots +x_{m}}$ and ${\displaystyle \alpha _{\bullet }=\alpha _{0}+\alpha _{1}+\cdots +\alpha _{m}}$.

This can also be written

${\displaystyle \Pr(\mathbf {x} \mid x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {\mathrm {B} (x_{\bullet },\alpha _{\bullet })}{\mathrm {B} (x_{0},\alpha _{0})}}\prod _{i=1}^{m}{\frac {\Gamma (x_{i}+\alpha _{i})}{x_{i}!\Gamma (\alpha _{i})}}.}$

To obtain the marginal distribution over a subset of Dirichlet negative multinomial random variables, one only needs to drop the irrelevant ${\displaystyle \alpha _{i}}$'s (the variables that one wants to marginalize out) from the ${\displaystyle {\boldsymbol {\alpha }}}$ vector. The joint distribution of the remaining random variates is ${\displaystyle \mathrm {DNM} (x_{0},\alpha _{0},{\boldsymbol {\alpha _{(-)}}})}$ where ${\displaystyle {\boldsymbol {\alpha _{(-)}}}}$ is the vector with the removed ${\displaystyle \alpha _{i}}$'s. The univariate marginals are said to be beta negative binomially distributed.

If m-dimensional x is partitioned as follows

${\displaystyle \mathbf {x} ={\begin{bmatrix}\mathbf {x} ^{(1)}\\\mathbf {x} ^{(2)}\end{bmatrix}}{\text{ with sizes }}{\begin{bmatrix}q\times 1\\(m-q)\times 1\end{bmatrix}}}$

and accordingly ${\displaystyle {\boldsymbol {\alpha }}}$

${\displaystyle {\boldsymbol {\alpha }}={\begin{bmatrix}{\boldsymbol {\alpha }}^{(1)}\\{\boldsymbol {\alpha }}^{(2)}\end{bmatrix}}{\text{ with sizes }}{\begin{bmatrix}q\times 1\\(m-q)\times 1\end{bmatrix}}}$

then the conditional distribution of ${\displaystyle \mathbf {X} ^{(1)}}$ on ${\displaystyle \mathbf {X} ^{(2)}=\mathbf {x} ^{(2)}}$ is ${\displaystyle \mathrm {DNM} (x_{0}^{\prime },\alpha _{0}^{\prime },{\boldsymbol {\alpha }}^{(1)})}$ where

${\displaystyle x_{0}^{\prime }=x_{0}+\sum _{i=1}^{m-q}x_{i}^{(2)}}$

and

${\displaystyle \alpha _{0}^{\prime }=\alpha _{0}+\sum _{i=1}^{m-q}\alpha _{i}^{(2)}}$.

That is,

${\displaystyle \Pr(\mathbf {x} ^{(1)}\mid \mathbf {x} ^{(2)},x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {\mathrm {B} (x_{\bullet },\alpha _{\bullet })}{\mathrm {B} (x_{0}^{\prime },\alpha _{0}^{\prime })}}\prod _{i=1}^{q}{\frac {\Gamma (x_{i}^{(1)}+\alpha _{i}^{(1)})}{(x_{i}^{(1)}!)\Gamma (\alpha _{i}^{(1)})}}}$

The conditional distribution of a Dirichlet negative multinomial distribution on ${\displaystyle \sum _{i=1}^{m}x_{i}=n}$ is Dirichlet-multinomial distribution with parameters ${\displaystyle n}$ and ${\displaystyle {\boldsymbol {\alpha }}}$. That is

${\displaystyle \Pr(\mathbf {x} \mid \sum _{i=1}^{m}x_{i}=n,x_{0},\alpha _{0},{\boldsymbol {\alpha }})={\frac {n!\Gamma \left(\sum _{i=1}^{m}\alpha _{i}\right)}{\Gamma \left(n+\sum _{i=1}^{m}\alpha _{i}\right)}}\prod _{i=1}^{m}{\frac {\Gamma (x_{i}+\alpha _{i})}{x_{i}!\Gamma (\alpha _{i})}}}$.

Notice that the expression does not depend on ${\displaystyle x_{0}}$ or ${\displaystyle \alpha _{0}}$.

If

${\displaystyle X=(X_{1},\ldots ,X_{m})\sim \operatorname {DNM} (x_{0},\alpha _{0},\alpha _{1},\ldots ,\alpha _{m})}$

then, if the random variables with positive subscripts i and j are dropped from the vector and replaced by their sum,

${\displaystyle X'=(X_{1},\ldots ,X_{i}+X_{j},\ldots ,X_{m})\sim \operatorname {DNM} \left(x_{0},\alpha _{0},\alpha _{1},\ldots ,\alpha _{i}+\alpha _{j},\ldots ,\alpha _{m}\right).}$

For ${\displaystyle \alpha _{0}>2}$ the entries of the correlation matrix are

${\displaystyle \rho (X_{i},X_{i})=1.}$

${\displaystyle \rho (X_{i},X_{j})={\frac {\operatorname {cov} (X_{i},X_{j})}{\sqrt {\operatorname {var} (X_{i})\operatorname {var} (X_{j})}}}={\sqrt {\frac {\alpha _{i}\alpha _{j}}{(\alpha _{0}+\alpha _{i}-1)(\alpha _{0}+\alpha _{j}-1)}}}.}$

The Dirichlet negative multinomial is a heavy tailed distribution. It does not have a finite mean for ${\displaystyle \alpha _{0}\leq 1}$ and it has infinite covariance matrix for ${\displaystyle \alpha _{0}\leq 2}$. Therefore the moment generating function does not exist.

In the case when the ${\displaystyle m+2}$ parameters ${\displaystyle x_{0},\alpha _{0}}$ and ${\displaystyle {\boldsymbol {\alpha }}}$ are positive integers the Dirichlet negative multinomial can also be motivated by an urn model - or more specifically a basic Pólya urn model. Consider an urn initially containing ${\displaystyle \sum _{i=0}^{m}{\alpha _{i}}}$ balls of ${\displaystyle m+1}$ various colors including ${\displaystyle \alpha _{0}}$ red balls (the stopping color). The vector ${\displaystyle {\boldsymbol {\alpha }}}$ gives the respective counts of the other balls of various ${\displaystyle m}$ non-red colors. At each step of the model, a ball is drawn at random from the urn and replaced, along with one additional ball of the same color. The process is repeated over and over, until ${\displaystyle x_{0}}$ red colored balls are drawn. The random vector ${\displaystyle \mathbf {X} }$ of observed draws of the other ${\displaystyle m}$ non-red colors are distributed according to a ${\displaystyle \mathrm {DNM} (x_{0},\alpha _{0},{\boldsymbol {\alpha }})}$. Note, at the end of the experiment, the urn always contains the fixed number ${\displaystyle x_{0}+\alpha _{0}}$ of red balls while containing the random number ${\displaystyle \mathbf {X} +{\boldsymbol {\alpha }}}$ of the other ${\displaystyle m}$ colors.