Repository: scipy/scipy

Issue:
ENH: Faster inverse-Wishart sampling
### Is your feature request related to a problem? Please describe.

Currently inverse-Wishart sampling ([here](https://github.com/scipy/scipy/blob/d7f12d38b171f3a6ae841937bb6c2444a1e04b8d/scipy/stats/_multivariate.py#L2957-L3004)) Cholesky decomposes the scale matrix to invert it, then Cholesky decomposes its inverse, samples a Cholesky factor from the Wishart distribution, then inverts it to form an inverse-Wishart matrix.

Algorithm 5 in https://arxiv.org/abs/2310.15884v1 generates a Cholesky factor of an inverse-Wishart matrix directly. Table 1 shows that for the $\Psi$ parameterization used in `invwishart`, this algorithm eliminates 1 triangular product, 1 triangular solve, and 1 Cholesky factorization compared to the current algorithm, while adding no additional operations.

### Describe the solution you'd like.

I propose adopting the new algorithm.

From a cursory glance, it seems `invwishart_gen` might subclass `wishart_gen` primarily so they can share the same `_standard_rvs` method. The new algorithm would have a different `_standard_rvs` method, so I wonder if it would make sense for `invwishart_gen` to subtype `wishart_gen` anymore.

### Describe alternatives you've considered.

_No response_

### Additional context (e.g. screenshots, GIFs)

Here are the relevant sections of the paper:
<img src=https://github.com/scipy/scipy/assets/8673634/df12cd40-e673-455c-beee-5934393394b7 width=400>

<img src=https://github.com/scipy/scipy/assets/8673634/daab97bc-abc4-4852-a6f3-71e4617e1d52 width=400>

And here is a reference implementation: https://github.com/sethaxen/rand_invwishart/tree/main/RandomInverseWishart.jl
ENH: Faster inverse-Wishart sampling
### Is your feature request related to a problem? Please describe.

Currently inverse-Wishart sampling ([here](https://github.com/scipy/scipy/blob/d7f12d38b171f3a6ae841937bb6c2444a1e04b8d/scipy/stats/_multivariate.py#L2957-L3004)) Cholesky decomposes the scale matrix to invert it, then Cholesky decomposes its inverse, samples a Cholesky factor from the Wishart distribution, then inverts it to form an inverse-Wishart matrix.

Algorithm 5 in https://arxiv.org/abs/2310.15884v1 generates a Cholesky factor of an inverse-Wishart matrix directly. Table 1 shows that for the $\Psi$ parameterization used in `invwishart`, this algorithm eliminates 1 triangular product, 1 triangular solve, and 1 Cholesky factorization compared to the current algorithm, while adding no additional operations.

### Describe the solution you'd like.

I propose adopting the new algorithm.

From a cursory glance, it seems `invwishart_gen` might subclass `wishart_gen` primarily so they can share the same `_standard_rvs` method. The new algorithm would have a different `_standard_rvs` method, so I wonder if it would make sense for `invwishart_gen` to subtype `wishart_gen` anymore.

### Describe alternatives you've considered.

_No response_

### Additional context (e.g. screenshots, GIFs)

Here are the relevant sections of the paper:
<img src=https://github.com/scipy/scipy/assets/8673634/df12cd40-e673-455c-beee-5934393394b7 width=400>

<img src=https://github.com/scipy/scipy/assets/8673634/daab97bc-abc4-4852-a6f3-71e4617e1d52 width=400>

And here is a reference implementation: https://github.com/sethaxen/rand_invwishart/tree/main/RandomInverseWishart.jl


Discussion:
Don't know what this means on the statistics side but assuming that I understood correctly, you are solving $Z U_\Omega = U_b$ if you use `trsm` in the step 8 of alg 4. it would be more accurate and faster. Because trtri is essentially the same thing with an identity matrix. Or are you generating the inverse directly? Not clear why you would use trtri if you have the inverse already. 


 I'll leave the suitableness of the algo to our stats residents. 
Step 7 uses `trtri`, while Step 8 uses `trmm`. I chose this over combining to `trsm` because in benchmarks it was slightly faster. It also allows for the entire algorithm to be in-place; `trsm` would require allocating an additional matrix for storage.
That should not be the case [dtrsm](https://www.netlib.org/lapack/explore-html/d1/d54/group__double__blas__level3_ga6a0a7704f4a747562c1bd9487e89795c.html) combines both of those steps without explicitly forming the multiplication and does it in-place overwriting $U_\Omega$. No allocation should be necessary.
Yes, but in Algorithm 4, $U_\Omega$ is a user-provided parameter and should probably not be overwritten. However, in Algorithm 5, if $\Omega$ is provided, then $U_\Omega$ once computed could be overwritten. I just benchmarked, and using `trsm` in that case is indeed slightly faster than `trtri` and `trmm`. Probably not relevant for scipy, since either way it would make sense to use `trsm` here, but relevant for the paper.

I don't have a great sense for how relevant the improved accuracy due to `trsm` would be when `Z` is random.
Could you benchmark your algorithm against what is currently implemented in scipy? Reducing the number of linear algebra operations sounds in principle good although I do not know the LAPACK functions.
Don't know what this means on the statistics side but assuming that I understood correctly, you are solving $Z U_\Omega = U_b$ if you use `trsm` in the step 8 of alg 4. it would be more accurate and faster. Because trtri is essentially the same thing with an identity matrix. Or are you generating the inverse directly? Not clear why you would use trtri if you have the inverse already. 


 I'll leave the suitableness of the algo to our stats residents. 
Step 7 uses `trtri`, while Step 8 uses `trmm`. I chose this over combining to `trsm` because in benchmarks it was slightly faster. It also allows for the entire algorithm to be in-place; `trsm` would require allocating an additional matrix for storage.
That should not be the case [dtrsm](https://www.netlib.org/lapack/explore-html/d1/d54/group__double__blas__level3_ga6a0a7704f4a747562c1bd9487e89795c.html) combines both of those steps without explicitly forming the multiplication and does it in-place overwriting $U_\Omega$. No allocation should be necessary.
Yes, but in Algorithm 4, $U_\Omega$ is a user-provided parameter and should probably not be overwritten. However, in Algorithm 5, if $\Omega$ is provided, then $U_\Omega$ once computed could be overwritten. I just benchmarked, and using `trsm` in that case is indeed slightly faster than `trtri` and `trmm`. Probably not relevant for scipy, since either way it would make sense to use `trsm` here, but relevant for the paper.

I don't have a great sense for how relevant the improved accuracy due to `trsm` would be when `Z` is random.
Could you benchmark your algorithm against what is currently implemented in scipy? Reducing the number of linear algebra operations sounds in principle good although I do not know the LAPACK functions.

Resolving patch:
diff --git a/scipy/stats/_multivariate.py b/scipy/stats/_multivariate.py
--- a/scipy/stats/_multivariate.py
+++ b/scipy/stats/_multivariate.py
@@ -3,15 +3,12 @@
 #
 import math
 import numpy as np
-from numpy import asarray_chkfinite, asarray
 import scipy.linalg
 from scipy._lib import doccer
 from scipy.special import (gammaln, psi, multigammaln, xlogy, entr, betaln,
                            ive, loggamma)
 from scipy._lib._util import check_random_state, _lazywhere
-from scipy.linalg.blas import drot
-from scipy.linalg._misc import LinAlgError
-from scipy.linalg.lapack import get_lapack_funcs
+from scipy.linalg.blas import drot, get_blas_funcs
 from ._continuous_distns import norm
 from ._discrete_distns import binom
 from . import _mvn, _covariance, _rcont
@@ -2558,69 +2555,6 @@ def entropy(self):
     method.__doc__ = doccer.docformat(method.__doc__, wishart_docdict_params)
 
 
-def _cho_inv_batch(a, check_finite=True):
-    """
-    Invert the matrices a_i, using a Cholesky factorization of A, where
-    a_i resides in the last two dimensions of a and the other indices describe
-    the index i.
-
-    Overwrites the data in a.
-
-    Parameters
-    ----------
-    a : array
-        Array of matrices to invert, where the matrices themselves are stored
-        in the last two dimensions.
-    check_finite : bool, optional
-        Whether to check that the input matrices contain only finite numbers.
-        Disabling may give a performance gain, but may result in problems
-        (crashes, non-termination) if the inputs do contain infinities or NaNs.
-
-    Returns
-    -------
-    x : array
-        Array of inverses of the matrices ``a_i``.
-
-    See Also
-    --------
-    scipy.linalg.cholesky : Cholesky factorization of a matrix
-
-    """
-    if check_finite:
-        a1 = asarray_chkfinite(a)
-    else:
-        a1 = asarray(a)
-    if len(a1.shape) < 2 or a1.shape[-2] != a1.shape[-1]:
-        raise ValueError('expected square matrix in last two dimensions')
-
-    potrf, potri = get_lapack_funcs(('potrf', 'potri'), (a1,))
-
-    triu_rows, triu_cols = np.triu_indices(a.shape[-2], k=1)
-    for index in np.ndindex(a1.shape[:-2]):
-
-        # Cholesky decomposition
-        a1[index], info = potrf(a1[index], lower=True, overwrite_a=False,
-                                clean=False)
-        if info > 0:
-            raise LinAlgError("%d-th leading minor not positive definite"
-                              % info)
-        if info < 0:
-            raise ValueError('illegal value in %d-th argument of internal'
-                             ' potrf' % -info)
-        # Inversion
-        a1[index], info = potri(a1[index], lower=True, overwrite_c=False)
-        if info > 0:
-            raise LinAlgError("the inverse could not be computed")
-        if info < 0:
-            raise ValueError('illegal value in %d-th argument of internal'
-                             ' potrf' % -info)
-
-        # Make symmetric (dpotri only fills in the lower triangle)
-        a1[index][triu_rows, triu_cols] = a1[index][triu_cols, triu_rows]
-
-    return a1
-
-
 class invwishart_gen(wishart_gen):
     r"""An inverse Wishart random variable.
 
@@ -2690,6 +2624,10 @@ class invwishart_gen(wishart_gen):
     inverse Gamma distribution with parameters shape = :math:`\frac{\nu}{2}`
     and scale = :math:`\frac{1}{2}`.
 
+    Instead of inverting a randomly generated Wishart matrix as described in [2],
+    here the algorithm in [4] is used to directly generate a random inverse-Wishart
+    matrix without inversion.
+
     .. versionadded:: 0.16.0
 
     References
@@ -2702,6 +2640,8 @@ class invwishart_gen(wishart_gen):
     .. [3] Gupta, M. and Srivastava, S. "Parametric Bayesian Estimation of
            Differential Entropy and Relative Entropy". Entropy 12, 818 - 843.
            2010.
+    .. [4] S.D. Axen, "Efficiently generating inverse-Wishart matrices and
+           their Cholesky factors", :arXiv:`2310.15884v1`. 2023.
 
     Examples
     --------
@@ -2743,7 +2683,7 @@ def __call__(self, df=None, scale=None, seed=None):
         """
         return invwishart_frozen(df, scale, seed)
 
-    def _logpdf(self, x, dim, df, scale, log_det_scale):
+    def _logpdf(self, x, dim, df, log_det_scale, C):
         """Log of the inverse Wishart probability density function.
 
         Parameters
@@ -2755,10 +2695,10 @@ def _logpdf(self, x, dim, df, scale, log_det_scale):
             Dimension of the scale matrix
         df : int
             Degrees of freedom
-        scale : ndarray
-            Scale matrix
         log_det_scale : float
             Logarithm of the determinant of the scale matrix
+        C : ndarray
+            Cholesky factorization of the scale matrix, lower triagular.
 
         Notes
         -----
@@ -2766,20 +2706,18 @@ def _logpdf(self, x, dim, df, scale, log_det_scale):
         called directly; use 'logpdf' instead.
 
         """
+        # Retrieve tr(scale x^{-1})
         log_det_x = np.empty(x.shape[-1])
-        x_inv = np.copy(x).T
+        tr_scale_x_inv = np.empty(x.shape[-1])
+        trsm = get_blas_funcs(('trsm'), (x,))
         if dim > 1:
-            _cho_inv_batch(x_inv)  # works in-place
+            for i in range(x.shape[-1]):
+                Cx, log_det_x[i] = self._cholesky_logdet(x[:, :, i])
+                A = trsm(1., Cx, C, side=0, lower=True)
+                tr_scale_x_inv[i] = np.linalg.norm(A)**2
         else:
-            x_inv = 1./x_inv
-        tr_scale_x_inv = np.empty(x.shape[-1])
-
-        for i in range(x.shape[-1]):
-            C, lower = scipy.linalg.cho_factor(x[:, :, i], lower=True)
-
-            log_det_x[i] = 2 * np.sum(np.log(C.diagonal()))
-
-            tr_scale_x_inv[i] = np.dot(scale, x_inv[i]).trace()
+            log_det_x[:] = np.log(x[0, 0])
+            tr_scale_x_inv[:] = C[0, 0]**2 / x[0, 0]
 
         # Log PDF
         out = ((0.5 * df * log_det_scale - 0.5 * tr_scale_x_inv) -
@@ -2810,8 +2748,8 @@ def logpdf(self, x, df, scale):
         """
         dim, df, scale = self._process_parameters(df, scale)
         x = self._process_quantiles(x, dim)
-        _, log_det_scale = self._cholesky_logdet(scale)
-        out = self._logpdf(x, dim, df, scale, log_det_scale)
+        C, log_det_scale = self._cholesky_logdet(scale)
+        out = self._logpdf(x, dim, df, log_det_scale, C)
         return _squeeze_output(out)
 
     def pdf(self, x, df, scale):
@@ -2954,6 +2892,60 @@ def var(self, df, scale):
         out = self._var(dim, df, scale)
         return _squeeze_output(out) if out is not None else out
 
+    def _inv_standard_rvs(self, n, shape, dim, df, random_state):
+        """
+        Parameters
+        ----------
+        n : integer
+            Number of variates to generate
+        shape : iterable
+            Shape of the variates to generate
+        dim : int
+            Dimension of the scale matrix
+        df : int
+            Degrees of freedom
+        random_state : {None, int, `numpy.random.Generator`,
+                        `numpy.random.RandomState`}, optional
+
+            If `seed` is None (or `np.random`), the `numpy.random.RandomState`
+            singleton is used.
+            If `seed` is an int, a new ``RandomState`` instance is used,
+            seeded with `seed`.
+            If `seed` is already a ``Generator`` or ``RandomState`` instance
+            then that instance is used.
+
+        Returns
+        -------
+        A : ndarray
+            Random variates of shape (`shape`) + (``dim``, ``dim``).
+            Each slice `A[..., :, :]` is lower-triangular, and its
+            inverse is the lower Cholesky factor of a draw from
+            `invwishart(df, np.eye(dim))`.
+
+        Notes
+        -----
+        As this function does no argument checking, it should not be
+        called directly; use 'rvs' instead.
+
+        """
+        A = np.zeros(shape + (dim, dim))
+
+        # Random normal variates for off-diagonal elements
+        tri_rows, tri_cols = np.tril_indices(dim, k=-1)
+        n_tril = dim * (dim-1) // 2
+        A[..., tri_rows, tri_cols] = random_state.normal(
+            size=(*shape, n_tril),
+        )
+
+        # Random chi variates for diagonal elements
+        rows = np.arange(dim)
+        chi_dfs = (df - dim + 1) + rows
+        A[..., rows, rows] = random_state.chisquare(
+            df=chi_dfs, size=(*shape, dim),
+        )**0.5
+
+        return A
+
     def _rvs(self, n, shape, dim, df, C, random_state):
         """Draw random samples from an inverse Wishart distribution.
 
@@ -2978,29 +2970,23 @@ def _rvs(self, n, shape, dim, df, C, random_state):
 
         """
         random_state = self._get_random_state(random_state)
-        # Get random draws A such that A ~ W(df, I)
-        A = super()._standard_rvs(n, shape, dim, df, random_state)
+        # Get random draws A such that inv(A) ~ iW(df, I)
+        A = self._inv_standard_rvs(n, shape, dim, df, random_state)
 
         # Calculate SA = (CA)'^{-1} (CA)^{-1} ~ iW(df, scale)
-        eye = np.eye(dim)
-        trtrs = get_lapack_funcs(('trtrs'), (A,))
+        trsm = get_blas_funcs(('trsm'), (A,))
+        trmm = get_blas_funcs(('trmm'), (A,))
 
         for index in np.ndindex(A.shape[:-2]):
-            # Calculate CA
-            CA = np.dot(C, A[index])
-            # Get (C A)^{-1} via triangular solver
             if dim > 1:
-                CA, info = trtrs(CA, eye, lower=True)
-                if info > 0:
-                    raise LinAlgError("Singular matrix.")
-                if info < 0:
-                    raise ValueError('Illegal value in %d-th argument of'
-                                     ' internal trtrs' % -info)
+                # Calculate CA
+                # Get CA = C A^{-1} via triangular solver
+                CA = trsm(1., A[index], C, side=1, lower=True)
+                # get SA
+                A[index] = trmm(1., CA, CA, side=1, lower=True, trans_a=True)
             else:
-                CA = 1. / CA
-            # Get SA
-            A[index] = np.dot(CA.T, CA)
-
+                A[index][0, 0] = (C[0, 0] / A[index][0, 0])**2
+                
         return A
 
     def rvs(self, df, scale, size=1, random_state=None):
@@ -3027,12 +3013,8 @@ def rvs(self, df, scale, size=1, random_state=None):
         n, shape = self._process_size(size)
         dim, df, scale = self._process_parameters(df, scale)
 
-        # Invert the scale
-        eye = np.eye(dim)
-        L, lower = scipy.linalg.cho_factor(scale, lower=True)
-        inv_scale = scipy.linalg.cho_solve((L, lower), eye)
-        # Cholesky decomposition of inverted scale
-        C = scipy.linalg.cholesky(inv_scale, lower=True)
+        # Cholesky decomposition of scale
+        C = scipy.linalg.cholesky(scale, lower=True)
 
         out = self._rvs(n, shape, dim, df, C, random_state)
 
@@ -3080,20 +3062,13 @@ def __init__(self, df, scale, seed=None):
         )
 
         # Get the determinant via Cholesky factorization
-        C, lower = scipy.linalg.cho_factor(self.scale, lower=True)
-        self.log_det_scale = 2 * np.sum(np.log(C.diagonal()))
-
-        # Get the inverse using the Cholesky factorization
-        eye = np.eye(self.dim)
-        self.inv_scale = scipy.linalg.cho_solve((C, lower), eye)
-
-        # Get the Cholesky factorization of the inverse scale
-        self.C = scipy.linalg.cholesky(self.inv_scale, lower=True)
+        self.C = scipy.linalg.cholesky(self.scale, lower=True)
+        self.log_det_scale = 2 * np.sum(np.log(self.C.diagonal()))
 
     def logpdf(self, x):
         x = self._dist._process_quantiles(x, self.dim)
-        out = self._dist._logpdf(x, self.dim, self.df, self.scale,
-                                 self.log_det_scale)
+        out = self._dist._logpdf(x, self.dim, self.df,
+                                 self.log_det_scale, self.C)
         return _squeeze_output(out)
 
     def pdf(self, x):


Test patch:
diff --git a/scipy/stats/tests/test_multivariate.py b/scipy/stats/tests/test_multivariate.py
--- a/scipy/stats/tests/test_multivariate.py
+++ b/scipy/stats/tests/test_multivariate.py
@@ -18,7 +18,6 @@
 import scipy.linalg
 from scipy.stats._multivariate import (_PSD,
                                        _lnB,
-                                       _cho_inv_batch,
                                        multivariate_normal_frozen)
 from scipy.stats import (multivariate_normal, multivariate_hypergeom,
                          matrix_normal, special_ortho_group, ortho_group,
@@ -1483,6 +1482,50 @@ def test_frozen(self):
             assert_equal(w.entropy(), wishart.entropy(df, scale))
             assert_equal(w.pdf(x), wishart.pdf(x, df, scale))
 
+    def test_wishart_2D_rvs(self):
+        dim = 3
+        df = 10
+
+        # Construct a simple non-diagonal positive definite matrix
+        scale = np.eye(dim)
+        scale[0,1] = 0.5
+        scale[1,0] = 0.5
+
+        # Construct frozen Wishart random variables
+        w = wishart(df, scale)
+
+        # Get the generated random variables from a known seed
+        np.random.seed(248042)
+        w_rvs = wishart.rvs(df, scale)
+        np.random.seed(248042)
+        frozen_w_rvs = w.rvs()
+
+        # Manually calculate what it should be, based on the Bartlett (1933)
+        # decomposition of a Wishart into D A A' D', where D is the Cholesky
+        # factorization of the scale matrix and A is the lower triangular matrix
+        # with the square root of chi^2 variates on the diagonal and N(0,1)
+        # variates in the lower triangle.
+        np.random.seed(248042)
+        covariances = np.random.normal(size=3)
+        variances = np.r_[
+            np.random.chisquare(df),
+            np.random.chisquare(df-1),
+            np.random.chisquare(df-2),
+        ]**0.5
+
+        # Construct the lower-triangular A matrix
+        A = np.diag(variances)
+        A[np.tril_indices(dim, k=-1)] = covariances
+
+        # Wishart random variate
+        D = np.linalg.cholesky(scale)
+        DA = D.dot(A)
+        manual_w_rvs = np.dot(DA, DA.T)
+
+        # Test for equality
+        assert_allclose(w_rvs, manual_w_rvs)
+        assert_allclose(frozen_w_rvs, manual_w_rvs)
+
     def test_1D_is_chisquared(self):
         # The 1-dimensional Wishart with an identity scale matrix is just a
         # chi-squared distribution.
@@ -1787,7 +1830,7 @@ def test_1D_is_invgamma(self):
             # entropy
             assert_allclose(iw.entropy(), ig.entropy())
 
-    def test_wishart_invwishart_2D_rvs(self):
+    def test_invwishart_2D_rvs(self):
         dim = 3
         df = 10
 
@@ -1796,72 +1839,69 @@ def test_wishart_invwishart_2D_rvs(self):
         scale[0,1] = 0.5
         scale[1,0] = 0.5
 
-        # Construct frozen Wishart and inverse Wishart random variables
-        w = wishart(df, scale)
+        # Construct frozen inverse-Wishart random variables
         iw = invwishart(df, scale)
 
         # Get the generated random variables from a known seed
-        np.random.seed(248042)
-        w_rvs = wishart.rvs(df, scale)
-        np.random.seed(248042)
-        frozen_w_rvs = w.rvs()
-        np.random.seed(248042)
+        np.random.seed(608072)
         iw_rvs = invwishart.rvs(df, scale)
-        np.random.seed(248042)
+        np.random.seed(608072)
         frozen_iw_rvs = iw.rvs()
 
-        # Manually calculate what it should be, based on the Bartlett (1933)
-        # decomposition of a Wishart into D A A' D', where D is the Cholesky
-        # factorization of the scale matrix and A is the lower triangular matrix
-        # with the square root of chi^2 variates on the diagonal and N(0,1)
-        # variates in the lower triangle.
-        np.random.seed(248042)
+        # Manually calculate what it should be, based on the decomposition in
+        # https://arxiv.org/abs/2310.15884 of an invers-Wishart into L L',
+        # where L A = D, D is the Cholesky factorization of the scale matrix,
+        # and A is the lower triangular matrix with the square root of chi^2
+        # variates on the diagonal and N(0,1) variates in the lower triangle.
+        # the diagonal chi^2 variates in this A are reversed compared to those
+        # in the Bartlett decomposition A for Wishart rvs.
+        np.random.seed(608072)
         covariances = np.random.normal(size=3)
         variances = np.r_[
-            np.random.chisquare(df),
-            np.random.chisquare(df-1),
             np.random.chisquare(df-2),
+            np.random.chisquare(df-1),
+            np.random.chisquare(df),
         ]**0.5
 
         # Construct the lower-triangular A matrix
         A = np.diag(variances)
         A[np.tril_indices(dim, k=-1)] = covariances
 
-        # Wishart random variate
+        # inverse-Wishart random variate
         D = np.linalg.cholesky(scale)
-        DA = D.dot(A)
-        manual_w_rvs = np.dot(DA, DA.T)
-
-        # inverse Wishart random variate
-        # Supposing that the inverse wishart has scale matrix `scale`, then the
-        # random variate is the inverse of a random variate drawn from a Wishart
-        # distribution with scale matrix `inv_scale = np.linalg.inv(scale)`
-        iD = np.linalg.cholesky(np.linalg.inv(scale))
-        iDA = iD.dot(A)
-        manual_iw_rvs = np.linalg.inv(np.dot(iDA, iDA.T))
+        L = np.linalg.solve(A.T, D.T).T
+        manual_iw_rvs = np.dot(L, L.T)
 
         # Test for equality
-        assert_allclose(w_rvs, manual_w_rvs)
-        assert_allclose(frozen_w_rvs, manual_w_rvs)
         assert_allclose(iw_rvs, manual_iw_rvs)
         assert_allclose(frozen_iw_rvs, manual_iw_rvs)
 
-    def test_cho_inv_batch(self):
-        """Regression test for gh-8844."""
-        a0 = np.array([[2, 1, 0, 0.5],
-                       [1, 2, 0.5, 0.5],
-                       [0, 0.5, 3, 1],
-                       [0.5, 0.5, 1, 2]])
-        a1 = np.array([[2, -1, 0, 0.5],
-                       [-1, 2, 0.5, 0.5],
-                       [0, 0.5, 3, 1],
-                       [0.5, 0.5, 1, 4]])
-        a = np.array([a0, a1])
-        ainv = a.copy()
-        _cho_inv_batch(ainv)
-        ident = np.eye(4)
-        assert_allclose(a[0].dot(ainv[0]), ident, atol=1e-15)
-        assert_allclose(a[1].dot(ainv[1]), ident, atol=1e-15)
+    def test_sample_mean(self):
+        """Test that sample mean consistent with known mean."""
+        # Construct an arbitrary positive definite scale matrix
+        df = 10
+        sample_size = 20_000
+        for dim in [1, 5]:
+            scale = np.diag(np.arange(dim) + 1)
+            scale[np.tril_indices(dim, k=-1)] = np.arange(dim * (dim - 1) / 2)
+            scale = np.dot(scale.T, scale)
+
+            dist = invwishart(df, scale)
+            Xmean_exp = dist.mean()
+            Xvar_exp = dist.var()
+            Xmean_std = (Xvar_exp / sample_size)**0.5  # asymptotic SE of mean estimate
+
+            X = dist.rvs(size=sample_size, random_state=1234)
+            Xmean_est = X.mean(axis=0)
+
+            ntests = dim*(dim + 1)//2
+            fail_rate = 0.01 / ntests  # correct for multiple tests
+            max_diff = norm.ppf(1 - fail_rate / 2)
+            assert np.allclose(
+                (Xmean_est - Xmean_exp) / Xmean_std,
+                0,
+                atol=max_diff,
+            )
 
     def test_logpdf_4x4(self):
         """Regression test for gh-8844."""