Repository: scipy/scipy

Issue:
scipy.stats: Allow specifying inverse-variance matrix to multivariate_normal
**Describe the solution you'd like**

Currently, `scipy.stats.multivariate_normal` accepts a mean and covariance matrix, and computes the inverse variance matrix internally and explicitly. But I often have covariance matrices with inverses that can be computed through other linear algebra tricks, e.g., the woodbury matrix identity. It would therefore be great if `scipy.stats.multivariate_normal` would allow optionally passing in an inverse-variance matrix, which is sometimes useful for (a) performance, or (b) numerical precision issues with covariance matrices with large values. 

I thought I would raise an issue here to discuss before actually implementing anything, but I would be happy to work on a PR.


**Describe alternatives you've considered**

I have either implemented my own `custom_multivariate_normal` distribution, or used the [implementation in astroML](https://github.com/astroML/astroML/blob/master/astroML/utils/utils.py#L22) to compute the log-pdf, but it would be nice to have this included in scipy as well.


Discussion:
I am not opposed to this proposal. However, that would change things in many signatures so probably want to get a broad agreement. Usually these discussions happen on the dev mailing list, I recommend sending the proposal there with an idea of how you would implement. Not the whole code just something more specific than the proposal. 

To be clear, your proposal would *not* include SMW theorem type inversions inside the mVN methods, correct?

I understood that you would do the inversion on user side and pass the inverted matrix into the mVN method(s). 

-Lucas Roberts 

> On Nov 11, 2019, at 2:38 PM, Adrian Price-Whelan <notifications@github.com> wrote:
> 
> ﻿
> Describe the solution you'd like
> 
> Currently, scipy.stats.multivariate_normal accepts a mean and covariance matrix, and computes the inverse variance matrix internally and explicitly. But I often have covariance matrices with inverses that can be computed through other linear algebra tricks, e.g., the woodbury matrix identity. It would therefore be great if scipy.stats.multivariate_normal would allow optionally passing in an inverse-variance matrix, which is sometimes useful for (a) performance, or (b) numerical precision issues with covariance matrices with large values.
> 
> I thought I would raise an issue here to discuss before actually implementing anything, but I would be happy to work on a PR.
> 
> Describe alternatives you've considered
> 
> I have either implemented my own custom_multivariate_normal distribution, or used the implementation in astroML to compute the log-pdf, but it would be nice to have this included in scipy as well.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

That's right, I was imagining just adding a new argument to accept the inverted matrix into the mVN methods — it would be up to the user to do the inversion. I'll open a thread on the mailing list - thanks.
So this would change
```
pdf(x, mean=None, cov=1, allow_singular=False)
```
to
```
pdf(x, mean=None, cov=1, allow_singular=False, inverse_cov=None)
```
and the same for `cdf`, `entropy`, `logcdf`, `logpdf` and `rvs`?

Seems reasonable.
Elsewhere in statistics, I think the inverse covariance matrix is usually called the ["precision" matrix.](https://en.wikipedia.org/wiki/Precision_(statistics)) Other packages use either "precision" ([stan](https://mc-stan.org/docs/2_20/functions-reference/multivariate-normal-distribution-precision-parameterization.html)) or "tau" ([PyMC3](https://docs.pymc.io/api/distributions/multivariate.html#pymc3.distributions.multivariate.MvNormal)) as a separate argument that supersedes the default cov. "Precision" might be the more generally used term? 
I would prefer the inverse instead of a domain specific terminology. But I am also curious what inverse tricks can be done meant here. 
> So this would change
> 
> ```
> pdf(x, mean=None, cov=1, allow_singular=False)
> ```
> 
> to
> 
> ```
> pdf(x, mean=None, cov=1, allow_singular=False, inverse_cov=None)
> ```
> 
> and the same for `cdf`, `entropy`, `logcdf`, `logpdf` and `rvs`?
> 
> Seems reasonable.

I think entropy works with ***covariance*** matrix whereas the others work with inverse covariance matrix, so for `entropy` method you'd still need to invert if you passed in inverse covariance. 

Another thing to consider is a flag for removing PSD checks. I've found in simulations that PSD checks can take a fair bit of overhead, even if you pass in the same covariance (or inverse covariance) matrix every time, or are (say) drawing from a wishart/inverse wishart which is PSD by construction. 

It sounds like one of the goals here is speed/performance so wanted to point this out too. 
@adrn thanks for this suggestion! What was the verdict of the mailing list discussion?
@rgommers expressed support, but I don't think anyone else chimed in! I forgot about this but could take another crack at it if you think it'd be useful :) 
Mailing list link: https://mail.python.org/pipermail/scipy-dev/2019-November/023843.html

> @rgommers expressed support, but I don't think anyone else chimed in! 

That's not uncommon. Discussion typically happens here, the mailing list message gets seen by many more people though so it's a good data point that no one was opposed (or had a better alternative).
Yes, makes sense - not a criticism, just reporting :) 
Thanks @adrn. Based on what I see here and from the limited response on the mailing list, sounds like a PR would be welcome. (How long it takes to get a full review depends on how complicated it is!)
I would like to see an inverse_cov option. In my MCMC algorithms I often have to compute the log probability many times using the same cov matrix. It does not make sense for me to recompute the inverse at each iteration if i am using the same cov matrix and therefore it would be convenient to have inverse_cov as an option. Until this becomes an option I have to define my own multivariate normal log pdf function that allows inverse_cov which is annoying
Here is a similar discussion at numpy: https://github.com/numpy/numpy/issues/17158

For those interested, I wrote a cython function to do just this some time ago. It samples from a `MVN(A^(-1) b, A^(-1))` using the precision `A` as input.:
https://github.com/zoj613/OccuSpytial/blob/e599d515b5377cb8f7a16af828f39b115a04bc66/occuspytial/distributions.pyx#L42-L110


Resolving patch:
diff --git a/scipy/stats/__init__.py b/scipy/stats/__init__.py
--- a/scipy/stats/__init__.py
+++ b/scipy/stats/__init__.py
@@ -173,6 +173,14 @@
    multivariate_t         -- Multivariate t-distribution
    multivariate_hypergeom -- Multivariate hypergeometric distribution
 
+`scipy.stats.multivariate_normal` methods accept an instance of
+any one of the following classes to represent the covariance.
+
+.. autosummary::
+   :toctree: generated/
+
+   CovViaPrecision        -- Covariance represented via the precision matrix
+
 Discrete distributions
 ----------------------
 
@@ -488,6 +496,7 @@
 from ._page_trend_test import page_trend_test
 from ._mannwhitneyu import mannwhitneyu
 from ._fit import fit
+from ._covariance import CovViaPrecision
 
 # Deprecated namespaces, to be removed in v2.0.0
 from . import (
diff --git a/scipy/stats/_covariance.py b/scipy/stats/_covariance.py
new file mode 100644
--- /dev/null
+++ b/scipy/stats/_covariance.py
@@ -0,0 +1,194 @@
+from functools import cached_property
+
+import numpy as np
+from scipy import linalg
+
+
+__all__ = ["CovViaPrecision"]
+
+
+class Covariance:
+    """
+    Representation of a covariance matrix
+    """
+
+    def whiten(self, x):
+        """
+        Perform a whitening transformation on data.
+
+        "Whitening" ("white" as in "white noise", in which each frequency has
+        equal magnitude) transforms a set of random variables into a new set of
+        random variables with unit-diagonal covariance. When a whitening
+        transform is applied to a sample of points distributed according to
+        a multivariate normal distribution with zero mean, the covariance of
+        the transformed sample is approximately the identity matrix.
+
+        Parameters
+        ----------
+        x : array_like
+            An array of points. The last dimension must correspond with the
+            dimensionality of the space, i.e., the number of columns in the
+            covariance matrix.
+
+        Returns
+        -------
+        x_ : array_like
+            The transformed array of points.
+
+        References
+        ----------
+        .. [1] "Whitening Transformation". Wikipedia.
+               https://en.wikipedia.org/wiki/Whitening_transformation
+
+        Examples
+        --------
+        >>> import numpy as np
+        >>> from scipy import stats
+        >>> rng = np.random.default_rng()
+        >>> n = 3
+        >>> A = rng.random(size=(n, n))
+        >>> cov_array = A @ A.T  # make matrix symmetric positive definite
+        >>> cov_object = stats.CovViaPrecision(np.linalg.inv(cov_array))
+        >>> x = rng.multivariate_normal(np.zeros(n), cov_array, size=(10000))
+        >>> x_ = cov_object.whiten(x)
+        >>> np.cov(x_, rowvar=False)  # near-identity covariance
+        array([[0.97862122, 0.00893147, 0.02430451],
+               [0.00893147, 0.96719062, 0.02201312],
+               [0.02430451, 0.02201312, 0.99206881]])
+
+        """
+        return self._whiten(np.asarray(x))
+
+    @property
+    def log_pdet(self):
+        """
+        Log of the pseudo-determinant of the covariance matrix
+        """
+        return np.array(self._log_pdet, dtype=float)[()]
+
+    @property
+    def rank(self):
+        """
+        Rank of the covariance matrix
+        """
+        return np.array(self._rank, dtype=int)[()]
+
+    @property
+    def covariance(self):
+        """
+        Explicit representation of the covariance matrix
+        """
+        return self._covariance
+
+    @property
+    def dimensionality(self):
+        """
+        Dimensionality of the vector space
+        """
+        return np.array(self._dimensionality, dtype=int)[()]
+
+    @property
+    def shape(self):
+        """
+        Shape of the covariance array
+        """
+        return self._shape
+
+    def _validate_matrix(self, A, name):
+        A = np.atleast_2d(A)
+        m, n = A.shape[-2:]
+        if m != n or A.ndim != 2 or not (np.issubdtype(A.dtype, np.integer) or
+                                         np.issubdtype(A.dtype, np.floating)):
+            message = (f"The input `{name}` must be a square, "
+                       "two-dimensional array of real numbers.")
+            raise ValueError(message)
+        return A
+
+    def _validate_vector(self, A, name):
+        A = np.atleast_1d(A)
+        if A.ndim != 1 or not (np.issubdtype(A.dtype, np.integer) or
+                               np.issubdtype(A.dtype, np.floating)):
+            message = (f"The input `{name}` must be a one-dimensional array "
+                       "of real numbers.")
+            raise ValueError(message)
+        return A
+
+
+class CovViaPrecision(Covariance):
+    r"""
+    Representation of a covariance provided via its precision matrix.
+
+    Notes
+    -----
+    Let the covariance matrix be :math:`A`, its precision matrix be
+    :math:`P = A^{-1}`, and :math:`L` be the lower Cholesky factor such that
+    :math:`L L^T = P`.
+    Whitening of a data point :math:`x` is performed by computing
+    :math:`x^T L`. :math:`\log\det{A}` is calculated as :math:`-2tr(\log{P})`,
+    where the :math:`\log` operation is performed element-wise.
+
+    This `Covariance` class does not support singular covariance matrices
+    because the precision matrix does not exist for a singular covariance
+    matrix.
+
+    """
+
+    def __init__(self, precision, covariance=None):
+        """
+        Parameters
+        ----------
+        precision : array_like
+            The precision matrix; that is, the inverse of a square, symmetric,
+            positive definite covariance matrix.
+        covariance : array_like, optional
+            The square, symmetric, positive definite covariance matrix. If not
+            provided, this may need to be calculated (e.g. to evaluate the
+            cumulative distribution function of
+            `scipy.stats.multivariate_normal`) by inverting `precision`.
+        """
+        precision = self._validate_matrix(precision, 'precision')
+        if covariance is not None:
+            covariance = self._validate_matrix(covariance, 'covariance')
+            message = "`precision.shape` must equal `covariance.shape`."
+            if precision.shape != covariance.shape:
+                raise ValueError(message)
+
+        self._chol_P = np.linalg.cholesky(precision)
+        self._log_pdet = -2*np.log(np.diag(self._chol_P)).sum(axis=-1)
+        self._rank = precision.shape[-1]  # must be full rank if invertible
+        self._precision = precision
+        self._cov_matrix = covariance
+        self._dimensionality = self._rank
+        self._shape = precision.shape
+        self._allow_singular = False
+
+    def _whiten(self, x):
+        return x @ self._chol_P
+
+    @cached_property
+    def _covariance(self):
+        n = self._dimensionality
+        return (linalg.cho_solve((self._chol_P, True), np.eye(n))
+                if self._cov_matrix is None else self._cov_matrix)
+
+
+class CovViaPSD(Covariance):
+    """
+    Representation of a covariance provided via an instance of _PSD
+    """
+
+    def __init__(self, psd):
+        self._LP = psd.U
+        self._log_pdet = psd.log_pdet
+        self._rank = psd.rank
+        self._covariance = psd._M
+        self._dimensionality = psd._M.shape[-1]
+        self._shape = psd._M.shape
+        self._psd = psd
+        self._allow_singular = False  # by default
+
+    def _whiten(self, x):
+        return x @ self._LP
+
+    def _support_mask(self, x):
+        return self._psd._support_mask(x)
diff --git a/scipy/stats/_multivariate.py b/scipy/stats/_multivariate.py
--- a/scipy/stats/_multivariate.py
+++ b/scipy/stats/_multivariate.py
@@ -14,7 +14,8 @@
 from scipy.linalg.lapack import get_lapack_funcs
 
 from ._discrete_distns import binom
-from . import _mvn
+from . import _mvn, _covariance
+
 
 __all__ = ['multivariate_normal',
            'matrix_normal',
@@ -152,6 +153,8 @@ class _PSD:
 
     def __init__(self, M, cond=None, rcond=None, lower=True,
                  check_finite=True, allow_singular=True):
+        self._M = np.asarray(M)
+
         # Compute the symmetric eigendecomposition.
         # Note that eigh takes care of array conversion, chkfinite,
         # and assertion that the matrix is square.
@@ -247,18 +250,19 @@ def random_state(self, seed):
 _mvn_doc_default_callparams = """\
 mean : array_like, default: ``[0]``
     Mean of the distribution.
-cov : array_like, default: ``[1]``
+cov : array_like or `Covariance`, default: ``[1]``
     Symmetric positive (semi)definite covariance matrix of the distribution.
 allow_singular : bool, default: ``False``
-    Whether to allow a singular covariance matrix.
+    Whether to allow a singular covariance matrix. This is ignored if `cov` is
+    a `Covariance` object.
 """
 
 _mvn_doc_callparams_note = """\
 Setting the parameter `mean` to `None` is equivalent to having `mean`
 be the zero-vector. The parameter `cov` can be a scalar, in which case
 the covariance matrix is the identity times that value, a vector of
-diagonal entries for the covariance matrix, or a two-dimensional
-array_like.
+diagonal entries for the covariance matrix, a two-dimensional array_like,
+or a `Covariance` object.
 """
 
 _mvn_doc_frozen_callparams = ""
@@ -309,7 +313,11 @@ class multivariate_normal_gen(multi_rv_generic):
     -----
     %(_mvn_doc_callparams_note)s
 
-    The covariance matrix `cov` must be a symmetric positive semidefinite
+    The covariance matrix `cov` may be an instance of a subclass of
+    `Covariance`, e.g. `scipy.stats.CovViaPrecision`. If so, `allow_singular`
+    is ignored.
+
+    Otherwise, `cov` must be a symmetric positive semidefinite
     matrix when `allow_singular` is True; it must be (strictly) positive
     definite when `allow_singular` is False.
     Symmetry is not checked; only the lower triangular portion is used.
@@ -385,11 +393,42 @@ def __call__(self, mean=None, cov=1, allow_singular=False, seed=None):
                                           allow_singular=allow_singular,
                                           seed=seed)
 
-    def _process_parameters(self, dim, mean, cov):
+    def _process_parameters(self, mean, cov, allow_singular=True):
         """
         Infer dimensionality from mean or covariance matrix, ensure that
         mean and covariance are full vector resp. matrix.
         """
+        if isinstance(cov, _covariance.Covariance):
+            return self._process_parameters_Covariance(mean, cov)
+        else:
+            # Before `Covariance` classes were introduced,
+            # `multivariate_normal` accepted plain arrays as `cov` and used the
+            # following input validation. To avoid disturbing the behavior of
+            # `multivariate_normal` when plain arrays are used, we use the
+            # original input validation here.
+            dim, mean, cov = self._process_parameters_psd(None, mean, cov)
+            # After input validation, some methods then processed the arrays
+            # with a `_PSD` object and used that to perform computation.
+            # To avoid branching statements in each method depending on whether
+            # `cov` is an array or `Covariance` object, we always process the
+            # array with `_PSD`, and then use wrapper that satisfies the
+            # `Covariance` interface, `CovViaPSD`.
+            psd = _PSD(cov, allow_singular=allow_singular)
+            cov_object = _covariance.CovViaPSD(psd)
+            return dim, mean, cov_object
+
+    def _process_parameters_Covariance(self, mean, cov):
+        dim = cov.dimensionality
+        mean = np.array([0.]) if mean is None else mean
+        message = (f"`cov` represents a covariance matrix in {dim} dimensions,"
+                   f"and so `mean` must be broadcastable to shape {(dim,)}")
+        try:
+            mean = np.broadcast_to(mean, dim)
+        except ValueError as e:
+            raise ValueError(message) from e
+        return dim, mean, cov
+
+    def _process_parameters_psd(self, dim, mean, cov):
         # Try to infer dimensionality
         if dim is None:
             if mean is None:
@@ -463,7 +502,7 @@ def _process_quantiles(self, x, dim):
 
         return x
 
-    def _logpdf(self, x, mean, prec_U, log_det_cov, rank):
+    def _logpdf(self, x, mean, cov_object):
         """Log of the multivariate normal probability density function.
 
         Parameters
@@ -473,13 +512,8 @@ def _logpdf(self, x, mean, prec_U, log_det_cov, rank):
             density function
         mean : ndarray
             Mean of the distribution
-        prec_U : ndarray
-            A decomposition such that np.dot(prec_U, prec_U.T)
-            is the precision matrix, i.e. inverse of the covariance matrix.
-        log_det_cov : float
-            Logarithm of the determinant of the covariance matrix
-        rank : int
-            Rank of the covariance matrix.
+        cov_object : Covariance
+            An object representing the Covariance matrix
 
         Notes
         -----
@@ -487,8 +521,12 @@ def _logpdf(self, x, mean, prec_U, log_det_cov, rank):
         called directly; use 'logpdf' instead.
 
         """
+        log_det_cov, rank = cov_object.log_pdet, cov_object.rank
         dev = x - mean
-        maha = np.sum(np.square(np.dot(dev, prec_U)), axis=-1)
+        if dev.ndim > 1:
+            log_det_cov = log_det_cov[..., np.newaxis]
+            rank = rank[..., np.newaxis]
+        maha = np.sum(np.square(cov_object.whiten(dev)), axis=-1)
         return -0.5 * (rank * _LOG_2PI + log_det_cov + maha)
 
     def logpdf(self, x, mean=None, cov=1, allow_singular=False):
@@ -510,12 +548,12 @@ def logpdf(self, x, mean=None, cov=1, allow_singular=False):
         %(_mvn_doc_callparams_note)s
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
+        params = self._process_parameters(mean, cov, allow_singular)
+        dim, mean, cov_object = params
         x = self._process_quantiles(x, dim)
-        psd = _PSD(cov, allow_singular=allow_singular)
-        out = self._logpdf(x, mean, psd.U, psd.log_pdet, psd.rank)
-        if allow_singular and (psd.rank < dim):
-            out_of_bounds = ~psd._support_mask(x-mean)
+        out = self._logpdf(x, mean, cov_object)
+        if np.any(cov_object.rank < dim):
+            out_of_bounds = ~cov_object._support_mask(x-mean)
             out[out_of_bounds] = -np.inf
         return _squeeze_output(out)
 
@@ -538,12 +576,12 @@ def pdf(self, x, mean=None, cov=1, allow_singular=False):
         %(_mvn_doc_callparams_note)s
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
+        params = self._process_parameters(mean, cov, allow_singular)
+        dim, mean, cov_object = params
         x = self._process_quantiles(x, dim)
-        psd = _PSD(cov, allow_singular=allow_singular)
-        out = np.exp(self._logpdf(x, mean, psd.U, psd.log_pdet, psd.rank))
-        if allow_singular and (psd.rank < dim):
-            out_of_bounds = ~psd._support_mask(x-mean)
+        out = np.exp(self._logpdf(x, mean, cov_object))
+        if np.any((cov_object.rank < dim)):
+            out_of_bounds = ~cov_object._support_mask(x-mean)
             out[out_of_bounds] = 0.0
         return _squeeze_output(out)
 
@@ -631,10 +669,10 @@ def logcdf(self, x, mean=None, cov=1, allow_singular=False, maxpts=None,
         .. versionadded:: 1.0.0
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
+        params = self._process_parameters(mean, cov, allow_singular)
+        dim, mean, cov_object = params
+        cov = cov_object.covariance
         x = self._process_quantiles(x, dim)
-        # Use _PSD to check covariance matrix
-        _PSD(cov, allow_singular=allow_singular)
         if not maxpts:
             maxpts = 1000000 * dim
         cdf = self._cdf(x, mean, cov, maxpts, abseps, releps, lower_limit)
@@ -676,10 +714,10 @@ def cdf(self, x, mean=None, cov=1, allow_singular=False, maxpts=None,
         .. versionadded:: 1.0.0
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
+        params = self._process_parameters(mean, cov, allow_singular)
+        dim, mean, cov_object = params
+        cov = cov_object.covariance
         x = self._process_quantiles(x, dim)
-        # Use _PSD to check covariance matrix
-        _PSD(cov, allow_singular=allow_singular)
         if not maxpts:
             maxpts = 1000000 * dim
         out = self._cdf(x, mean, cov, maxpts, abseps, releps, lower_limit)
@@ -706,7 +744,8 @@ def rvs(self, mean=None, cov=1, size=1, random_state=None):
         %(_mvn_doc_callparams_note)s
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
+        dim, mean, cov_object = self._process_parameters(mean, cov)
+        cov = cov_object.covariance
 
         random_state = self._get_random_state(random_state)
         out = random_state.multivariate_normal(mean, cov, size)
@@ -729,9 +768,8 @@ def entropy(self, mean=None, cov=1):
         %(_mvn_doc_callparams_note)s
 
         """
-        dim, mean, cov = self._process_parameters(None, mean, cov)
-        _, logdet = np.linalg.slogdet(2 * np.pi * np.e * cov)
-        return 0.5 * logdet
+        dim, mean, cov_object = self._process_parameters(mean, cov)
+        return 0.5 * (cov_object.rank * (_LOG_2PI + 1) + cov_object.log_pdet)
 
 
 multivariate_normal = multivariate_normal_gen()
@@ -781,11 +819,10 @@ def __init__(self, mean=None, cov=1, allow_singular=False, seed=None,
         array([[1.]])
 
         """
-        self.allow_singular = allow_singular
         self._dist = multivariate_normal_gen(seed)
-        self.dim, self.mean, self.cov = self._dist._process_parameters(
-                                                            None, mean, cov)
-        self.cov_info = _PSD(self.cov, allow_singular=allow_singular)
+        self.dim, self.mean, self.cov_object = (
+            self._dist._process_parameters(mean, cov, allow_singular))
+        self.allow_singular = allow_singular or self.cov_object._allow_singular
         if not maxpts:
             maxpts = 1000000 * self.dim
         self.maxpts = maxpts
@@ -794,10 +831,9 @@ def __init__(self, mean=None, cov=1, allow_singular=False, seed=None,
 
     def logpdf(self, x):
         x = self._dist._process_quantiles(x, self.dim)
-        out = self._dist._logpdf(x, self.mean, self.cov_info.U,
-                                 self.cov_info.log_pdet, self.cov_info.rank)
-        if self.allow_singular and (self.cov_info.rank < self.dim):
-            out_of_bounds = ~self.cov_info._support_mask(x-self.mean)
+        out = self._dist._logpdf(x, self.mean, self.cov_object)
+        if np.any(self.cov_object.rank < self.dim):
+            out_of_bounds = ~self.cov_object._support_mask(x-self.mean)
             out[out_of_bounds] = -np.inf
         return _squeeze_output(out)
 
@@ -814,12 +850,14 @@ def logcdf(self, x, *, lower_limit=None):
 
     def cdf(self, x, *, lower_limit=None):
         x = self._dist._process_quantiles(x, self.dim)
-        out = self._dist._cdf(x, self.mean, self.cov, self.maxpts, self.abseps,
-                              self.releps, lower_limit)
+        out = self._dist._cdf(x, self.mean, self.cov_object.covariance,
+                              self.maxpts, self.abseps, self.releps,
+                              lower_limit)
         return _squeeze_output(out)
 
     def rvs(self, size=1, random_state=None):
-        return self._dist.rvs(self.mean, self.cov, size, random_state)
+        return self._dist.rvs(self.mean, self.cov_object.covariance, size,
+                              random_state)
 
     def entropy(self):
         """Computes the differential entropy of the multivariate normal.
@@ -830,8 +868,8 @@ def entropy(self):
             Entropy of the multivariate normal distribution
 
         """
-        log_pdet = self.cov_info.log_pdet
-        rank = self.cov_info.rank
+        log_pdet = self.cov_object.log_pdet
+        rank = self.cov_object.rank
         return 0.5 * (rank * (_LOG_2PI + 1) + log_pdet)
 
 


Test patch:
diff --git a/scipy/stats/tests/test_multivariate.py b/scipy/stats/tests/test_multivariate.py
--- a/scipy/stats/tests/test_multivariate.py
+++ b/scipy/stats/tests/test_multivariate.py
@@ -26,6 +26,7 @@
                          beta, wishart, multinomial, invwishart, chi2,
                          invgamma, norm, uniform, ks_2samp, kstest, binom,
                          hypergeom, multivariate_t, cauchy, normaltest)
+from scipy.stats import _covariance
 
 from scipy.integrate import romb
 from scipy.special import multigammaln
@@ -35,6 +36,130 @@
 from unittest.mock import patch
 
 
+def assert_close(res, ref, *args, **kwargs):
+    assert_allclose(res, ref, *args, **kwargs)
+    assert_equal(res.shape, ref.shape)
+
+
+class TestCovariance:
+    def test_input_validation(self):
+        message = "The input `precision` must be a square, two-dimensional..."
+        with pytest.raises(ValueError, match=message):
+            _covariance.CovViaPrecision(np.ones(2))
+
+        message = "`precision.shape` must equal `covariance.shape`."
+        with pytest.raises(ValueError, match=message):
+            _covariance.CovViaPrecision(np.eye(3), covariance=np.eye(2))
+
+    _covariance_preprocessing = {"CovViaPrecision": np.linalg.inv,
+                                 "CovViaPSD": lambda x:
+                                     _PSD(x, allow_singular=True)}
+    _all_covariance_types = np.array(list(_covariance_preprocessing))
+    _matrices = {"diagonal full rank": np.diag([1, 2, 3]),
+                 "general full rank": [[5, 1, 3], [1, 6, 4], [3, 4, 7]],
+                 "diagonal rank-deficient": np.diag([1, 0, 3]),
+                 "general rank-deficient": [[5, -1, 0], [-1, 5, 0], [0, 0, 0]]}
+    _cov_types = {"diagonal full rank": _all_covariance_types,
+                  "general full rank": _all_covariance_types,
+                  "diagonal rank-deficient": _all_covariance_types[[1]],
+                  "general rank-deficient": _all_covariance_types[[1]]}
+
+    @pytest.mark.parametrize("matrix_type", list(_matrices))
+    @pytest.mark.parametrize("cov_type_name", _all_covariance_types)
+    def test_covariance(self, matrix_type, cov_type_name):
+        message = f"{cov_type_name} does not support {matrix_type} matrices"
+        if cov_type_name not in self._cov_types[matrix_type]:
+            pytest.skip(message)
+
+        A = self._matrices[matrix_type]
+        cov_type = getattr(_covariance, cov_type_name)
+        preprocessing = self._covariance_preprocessing[cov_type_name]
+
+        psd = _PSD(A, allow_singular=True)
+
+        # test properties
+        cov_object = cov_type(preprocessing(A))
+        assert_close(cov_object.log_pdet, psd.log_pdet)
+        assert_close(cov_object.rank, np.int64(psd.rank))
+        assert_close(cov_object.dimensionality,
+                     np.int64(np.array(A).shape[-1]))
+        assert_close(cov_object.covariance, np.asarray(A))
+
+        # test whitening 1D x
+        rng = np.random.default_rng(5292808890472453840)
+        x = rng.random(size=3)
+        res = cov_object.whiten(x)
+        ref = x @ psd.U
+        # res != ref in general; but res @ res == ref @ ref
+        assert_close(res @ res, ref @ ref)
+
+        # test whitening 3D x
+        x = rng.random(size=(2, 4, 3))
+        res = cov_object.whiten(x)
+        ref = x @ psd.U
+        assert_close((res**2).sum(axis=-1), (ref**2).sum(axis=-1))
+
+    @pytest.mark.parametrize("size", [tuple(), (2, 4, 3)])
+    @pytest.mark.parametrize("matrix_type", list(_matrices))
+    @pytest.mark.parametrize("cov_type_name", _all_covariance_types)
+    def test_mvn_with_covariance(self, size, matrix_type, cov_type_name):
+        message = f"{cov_type_name} does not support {matrix_type} matrices"
+        if cov_type_name not in self._cov_types[matrix_type]:
+            pytest.skip(message)
+
+        A = self._matrices[matrix_type]
+        cov_type = getattr(_covariance, cov_type_name)
+        preprocessing = self._covariance_preprocessing[cov_type_name]
+
+        mean = [0.1, 0.2, 0.3]
+        cov_object = cov_type(preprocessing(A))
+        mvn = multivariate_normal
+        dist0 = multivariate_normal(mean, A, allow_singular=True)
+        dist1 = multivariate_normal(mean, cov_object, allow_singular=True)
+
+        rng = np.random.default_rng(5292808890472453840)
+        x = rng.multivariate_normal(mean, A, size=size)
+        rng = np.random.default_rng(5292808890472453840)
+        x1 = mvn.rvs(mean, cov_object, size=size, random_state=rng)
+        rng = np.random.default_rng(5292808890472453840)
+        x2 = mvn(mean, cov_object, seed=rng).rvs(size=size)
+        assert_close(x1, x)
+        assert_close(x2, x)
+
+        assert_close(mvn.pdf(x, mean, cov_object), dist0.pdf(x))
+        assert_close(dist1.pdf(x), dist0.pdf(x))
+        assert_close(mvn.logpdf(x, mean, cov_object), dist0.logpdf(x))
+        assert_close(dist1.logpdf(x), dist0.logpdf(x))
+        assert_close(mvn.entropy(mean, cov_object), dist0.entropy())
+        assert_close(dist1.entropy(), dist0.entropy())
+
+    @pytest.mark.parametrize("size", [tuple(), (2, 4, 3)])
+    @pytest.mark.parametrize("cov_type_name", _all_covariance_types)
+    def test_mvn_with_covariance_cdf(self, size, cov_type_name):
+        # This is split from the test above because it's slow to be running
+        # with all matrix types, and there's no need because _mvn.mvnun
+        # does the calculation. All Covariance needs to do is pass is
+        # provide the `covariance` attribute.
+        matrix_type = "diagonal full rank"
+        A = self._matrices[matrix_type]
+        cov_type = getattr(_covariance, cov_type_name)
+        preprocessing = self._covariance_preprocessing[cov_type_name]
+
+        mean = [0.1, 0.2, 0.3]
+        cov_object = cov_type(preprocessing(A))
+        mvn = multivariate_normal
+        dist0 = multivariate_normal(mean, A, allow_singular=True)
+        dist1 = multivariate_normal(mean, cov_object, allow_singular=True)
+
+        rng = np.random.default_rng(5292808890472453840)
+        x = rng.multivariate_normal(mean, A, size=size)
+
+        assert_close(mvn.cdf(x, mean, cov_object), dist0.cdf(x))
+        assert_close(dist1.cdf(x), dist0.cdf(x))
+        assert_close(mvn.logcdf(x, mean, cov_object), dist0.logcdf(x))
+        assert_close(dist1.logcdf(x), dist0.logcdf(x))
+
+
 def _sample_orthonormal_matrix(n):
     M = np.random.randn(n, n)
     u, s, v = scipy.linalg.svd(M)
@@ -132,7 +257,7 @@ def test_rank(self):
             s = np.random.randn(n, expected_rank)
             cov = np.dot(s, s.T)
             distn = multivariate_normal(mean, cov, allow_singular=True)
-            assert_equal(distn.cov_info.rank, expected_rank)
+            assert_equal(distn.cov_object.rank, expected_rank)
 
     def test_degenerate_distributions(self):
 
@@ -163,9 +288,9 @@ def test_degenerate_distributions(self):
                                                allow_singular=True)
                 distn_rr = multivariate_normal(np.zeros(n), cov_rr,
                                                allow_singular=True)
-                assert_equal(distn_kk.cov_info.rank, k)
-                assert_equal(distn_nn.cov_info.rank, k)
-                assert_equal(distn_rr.cov_info.rank, k)
+                assert_equal(distn_kk.cov_object.rank, k)
+                assert_equal(distn_nn.cov_object.rank, k)
+                assert_equal(distn_rr.cov_object.rank, k)
                 pdf_kk = distn_kk.pdf(x[:k])
                 pdf_nn = distn_nn.pdf(x)
                 pdf_rr = distn_rr.pdf(y)
@@ -546,6 +671,25 @@ def test_cdf_signs(self):
         cdf = multivariate_normal.cdf(b, mean, cov, lower_limit=a)
         assert_allclose(cdf, cdf[0]*expected_signs)
 
+    def test_mean_cov(self):
+        # test the interaction between a Covariance object and mean
+        P = np.diag(1 / np.array([1, 2, 3]))
+        cov_object = _covariance.CovViaPrecision(P)
+
+        message = "`cov` represents a covariance matrix in 3 dimensions..."
+        with pytest.raises(ValueError, match=message):
+            multivariate_normal.entropy([0, 0], cov_object)
+
+        with pytest.raises(ValueError, match=message):
+            multivariate_normal([0, 0], cov_object)
+
+        x = [0.5, 0.5, 0.5]
+        ref = multivariate_normal.pdf(x, [0, 0, 0], cov_object)
+        assert_equal(multivariate_normal.pdf(x, cov=cov_object), ref)
+
+        ref = multivariate_normal.pdf(x, [1, 1, 1], cov_object)
+        assert_equal(multivariate_normal.pdf(x, 1, cov=cov_object), ref)
+
 
 class TestMatrixNormal: