# Statistics/Multivariate Data Analysis
## Distributions
### Multivariate Normal

The multivariate normal is just an extension of the normal distribution to the multivariate case. The simplest definition of the multivariate normal distribution can be given as follows:
Definition (Multivariate Normal Distribution):
A random vector $\mathbf {X}$ of dimension $p$ is said to follow a multivariate normal distribution with mean $\mu$ and covariance matrix $\Sigma$ if $\forall \mathbf {a} \in \mathbb {R} ^{p},\ \mathbf {a} ^{T}\mathbf {X} \sim {\mathcal {N}}(\mathbf {a} ^{T}\mu ,\mathbf {a} ^{T}\Sigma \mathbf {a} )$. It is denoted by $\mathbf {X} \sim {\mathcal {N}}_{p}(\mu ,\Sigma )$.
At first glance, the definition seems rather abstract and esoteric. After all, the univariate normal distribution has a specific form of density and a specific characteristic function, both of which are mathematically valid characterisations of any probability distribution. However, this kind of definition is necessary to deal with the case where $\Sigma$ is not strictly positive definite. In the case where $\Sigma$ is positive definite, it can be shown via Gauss-Markov theorem that the density function of $\mathbf {X} ,\ f_{\mathbf {X} }(\mathbf {x} )={\frac {1}{{\sqrt {2\pi }}|\Sigma |^{\frac {1}{2}}}}e^{-{\frac {1}{2}}(\mathbf {x} -\mu )^{T}\Sigma ^{-1}(\mathbf {x} -\mu )}$. However, this will not be true when $\Sigma$ is singular, as in that case the density function will not exist. But a definition based on the characteristic function will still work. A piecewise density function can still be derived based on the eigenvalues of $\Sigma$, but it is not a true density.

### Matrix-variate Normal

We will first need to develop some notation. Let $X_{m\times n}$ be a matrix with columns $c_{(1)},c_{(2)},\ldots ,c_{(n)}$. Then we define the column vector vec(X):=[c$_{(1)}$c$_{(2)}$⋮c$_{(n)}$]{\textstyle vec(X):={\begin{bmatrix}c_{(1)}\\c_{(2)}\\\vdots \\c_{(n)}\end{bmatrix}}}, and we call it the vectorisation of $X$.
Definition (Matrix-variate Normal):
We say $X_{m\times n}$ follows a matrix-variate normal distribution with mean matrix $\mu _{m\times n}$ and covariance matrix $\Sigma _{mn\times mn}$ if $vec(X)\sim {\mathcal {N}}_{mn}(vec(\mu ),\Sigma )$
The reader here should notice that this is simply imposing a normal distribution on the vectorisation of $X$. Thus, many of the results that are true for multivariate normal random vector will also be true for the vectorisation of matrix variate normal random variable.
Now that we have a definition of the multivariate and matrix-variate normal distribution, our next aim should be to find a similar analogue of the univariate $\chi _{(p)}^{2}$ distribution with $p$ degrees of freedom and Student's $t$ distribution, both of which are very closely related to the univariate normal distribution. We know that if $X_{i}\sim {\mathcal {N}}(\mu _{i},\sigma _{i}^{2})\ \forall i\in \{1,2,\ldots n\}$ then $\sum _{i=1}^{n}{\frac {(X_{i}-\mu _{i})^{2}}{\sigma _{i}^{2}}}\sim \chi _{(n)}^{2}$. What would be an analogue of this for the multivariate case?

### Wishart Distribution

Definition (Wishart Distribution):
If $\mathbf {X} _{i}{\overset {iid}{\sim }}{\mathcal {N}}_{p}(\mu ,\Sigma )$ for $i\in \{1,2,\ldots n\}$, then $S=\sum _{i=1}^{n}(\mathbf {X_{i}} -\mu )(\mathbf {X_{i}} -\mu )^{T}$ is said to have a Wishart distribution with $n$ degrees of freedom and associated matrix $\Sigma$. It is denoted by $S\sim W_{p}(n,\Sigma )$.
Although there does exist a form of density for the Wishart distribution, it is not necessary to prove most of the results we will require. An important thing to note, however, is that if $S$ follows a Wishart distribution, then ${\frac {\mathbf {a} ^{T}S\mathbf {a} }{\mathbf {a} ^{T}\Sigma \mathbf {a} }}\sim \chi _{(n)}^{2}$. This result can be easily proved by multiplying $S$ on the left and right by $\mathbf {a} ^{T}$and $\mathbf {a}$, and then using the fact that $\mathbf {a} ^{T}\mathbf {X} \sim {\mathcal {N}}(\mathbf {a} ^{T}\mu ,\mathbf {a} ^{T}\Sigma \mathbf {a} )$.

## Methodology

* Principal Component Analysis
* Canonical Correlation Analysis