# Normal-inverse-Wishart distribution

In probability theory and statistics, the normal-inverse-Wishart distribution (or Gaussian-inverse-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and covariance matrix (the inverse of the precision matrix).

## Definition

Suppose
    ${\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Sigma }}\sim {\mathcal {N}}\left({\boldsymbol {\mu }}{\Big |}{\boldsymbol {\mu }}_{0},{\frac {1}{\lambda }}{\boldsymbol {\Sigma }}\right)$
has a multivariate normal distribution with mean ${\boldsymbol {\mu }}_{0}$ and covariance matrix ${\tfrac {1}{\lambda }}{\boldsymbol {\Sigma }}$, where
    ${\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu \sim {\mathcal {W}}^{-1}({\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu )$
has an inverse Wishart distribution. Then $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})$ has a normal-inverse-Wishart distribution, denoted as
    $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})\sim \mathrm {NIW} ({\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu ).$

## Characterization
### Probability density function

    $f({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )={\mathcal {N}}\left({\boldsymbol {\mu }}{\Big |}{\boldsymbol {\mu }}_{0},{\frac {1}{\lambda }}{\boldsymbol {\Sigma }}\right){\mathcal {W}}^{-1}({\boldsymbol {\Sigma }}|{\boldsymbol {\Psi }},\nu )$
The full version of the PDF is as follows:
$f({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )={\frac {\lambda ^{D/2}|{\boldsymbol {\Psi }}|^{\nu /2}|{\boldsymbol {\Sigma }}|^{-{\frac {\nu +D+2}{2}}}}{(2\pi )^{D/2}2^{\frac {\nu D}{2}}\Gamma _{D}({\frac {\nu }{2}})}}{\text{exp}}\left\{-{\frac {1}{2}}Tr({\boldsymbol {\Psi \Sigma }}^{-1})-{\frac {\lambda }{2}}({\boldsymbol {\mu }}-{\boldsymbol {\mu }}_{0})^{T}{\boldsymbol {\Sigma }}^{-1}({\boldsymbol {\mu }}-{\boldsymbol {\mu }}_{0})\right\}$
Here $\Gamma _{D}[\cdot ]$ is the multivariate gamma function and $Tr({\boldsymbol {\Psi }})$ is the Trace of the given matrix.

## Properties
### Marginal distributions

By construction, the marginal distribution over ${\boldsymbol {\Sigma }}$ is an inverse Wishart distribution, and the conditional distribution over ${\boldsymbol {\mu }}$ given ${\boldsymbol {\Sigma }}$ is a multivariate normal distribution. The marginal distribution over ${\boldsymbol {\mu }}$ is a multivariate t-distribution.

## Posterior distribution of the parameters

Suppose the sampling density is a multivariate normal distribution
    ${\boldsymbol {y_{i}}}|{\boldsymbol {\mu }},{\boldsymbol {\Sigma }}\sim {\mathcal {N}}_{p}({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})$
where ${\boldsymbol {y}}$ is an $n\times p$ matrix and ${\boldsymbol {y_{i}}}$ (of length $p$) is row $i$ of the matrix .
With the mean and covariance matrix of the sampling distribution is unknown, we can place a Normal-Inverse-Wishart prior on the mean and covariance parameters jointly
    $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})\sim \mathrm {NIW} ({\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu ).$
The resulting posterior distribution for the mean and covariance matrix will also be a Normal-Inverse-Wishart
    $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}|y)\sim \mathrm {NIW} ({\boldsymbol {\mu }}_{n},\lambda _{n},{\boldsymbol {\Psi }}_{n},\nu _{n}),$
where
    ${\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\bar {\boldsymbol {y}}}}{\lambda +n}}$
    $\lambda _{n}=\lambda +n$
    $\nu _{n}=\nu +n$
    ${\boldsymbol {\Psi }}_{n}={\boldsymbol {\Psi +S}}+{\frac {\lambda n}{\lambda +n}}({\boldsymbol {{\bar {y}}-\mu _{0}}})({\boldsymbol {{\bar {y}}-\mu _{0}}})^{T}~~~\mathrm {with} ~~{\boldsymbol {S}}=\sum _{i=1}^{n}({\boldsymbol {y_{i}-{\bar {y}}}})({\boldsymbol {y_{i}-{\bar {y}}}})^{T}$.
To sample from the joint posterior of $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})$, one simply draws samples from ${\boldsymbol {\Sigma }}|{\boldsymbol {y}}\sim {\mathcal {W}}^{-1}({\boldsymbol {\Psi }}_{n},\nu _{n})$, then draw ${\boldsymbol {\mu }}|{\boldsymbol {\Sigma ,y}}\sim {\mathcal {N}}_{p}({\boldsymbol {\mu }}_{n},{\boldsymbol {\Sigma }}/\lambda _{n})$. To draw from the posterior predictive of a new observation, draw ${\boldsymbol {\tilde {y}}}|{\boldsymbol {\mu ,\Sigma ,y}}\sim {\mathcal {N}}_{p}({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})$ , given the already drawn values of ${\boldsymbol {\mu }}$ and ${\boldsymbol {\Sigma }}$.

## Generating normal-inverse-Wishart random variates

Generation of random variates is straightforward:
* Sample ${\boldsymbol {\Sigma }}$ from an inverse Wishart distribution with parameters ${\boldsymbol {\Psi }}$ and $\nu$
* Sample ${\boldsymbol {\mu }}$ from a multivariate normal distribution with mean ${\boldsymbol {\mu }}_{0}$ and variance ${\boldsymbol {\tfrac {1}{\lambda }}}{\boldsymbol {\Sigma }}$

## Related distributions

* The normal-Wishart distribution is essentially the same distribution parameterized by precision rather than variance. If $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }})\sim \mathrm {NIW} ({\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }},\nu )$ then $({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}^{-1})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Psi }}^{-1},\nu )$ .
* The normal-inverse-gamma distribution is the one-dimensional equivalent.
* The multivariate normal distribution and inverse Wishart distribution are the component distributions out of which this distribution is made.