Repository: pytorch
Issue: Add Wishart and inverse-Wishart distributions
Labels: module: distributions,feature,triaged

## 🚀 Feature

Implement the positive-definite-matrix-valued Wishart and inverse-Wishart distributions, currently missing from `torch.distributions`.

## Motivation

These distributions often pop up in Bayesian analysis as the conjugate priors for the covariance or precision matrix of the multivariate Gaussian. It would be great to have an official, GPU-accelerated, `torch.distributions`-compatible implementation available to everyone. This feature already exists in [TensorFlow Probability](https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/Wishart), for example.
<!-- Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too -->

## Pitch

I already have a working implementation I wrote for a personal project (involving Bayesian mixture models), extending [`ExponentialFamily`](https://pytorch.org/docs/stable/distributions.html#exponentialfamily). Other than the basic methods, it currently includes:
- reparametrised sampling (`rsample`), based on some background research (some refs. below);
- efficient computations using Cholesky factors;
- unit tests for output shapes, for different combinations of batch and sample shapes;
- unit tests for correctness of `log_prob` and `entropy` vs. [`scipy.stats.wishart`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wishart.html);
- statistical tests show sample statistics (e.g. means, determinants) are indistinguishable vs. [`scipy.stats.wishart`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wishart.html).

Would there be any interest in integrating this into the `torch.distributions` codebase? I'd be happy to make the necessary changes to harmonise style, documentation, and unit-test format.
<!-- A clear and concise description of what you want to happen. -->

## Alternatives

Currently the way to use Wishart distributions in a PyTorch project would be to go through NumPy and use [`scipy.stats.wishart`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wishart.html). This has a couple of big limitations:
- need to copy data between CPU and GPU;
- no support for batched parameters or arguments (e.g. to `wishart.logpdf()`).
<!-- A clear and concise description of any alternative solutions or features you've considered, if any. -->

## References
- Sawyer, S. (2007). Wishart Distributions and Inverse-Wishart Sampling. https://www.math.wustl.edu/~sawyer/hmhandouts/Wishart.pdf
- Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis (3rd ed.). John Wiley & Sons, Inc.
- Odell, P. L. & Feiveson, A. H. (1966). A Numerical Procedure to Generate a Sample Covariance Matrix. Journal of the American Statistical Association, 61(313):199-203.
- Ku, Y.-C. & Blomfield, P. (2010). Generating Random Wishart Matrices with Fractional Degrees of Freedom in OX.

cc `torch.distribution` maintainers @fritzo @neerajprad @alicanb @vishwakftw 

cc @vincentqb @fritzo @neerajprad @alicanb @vishwakftw