SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Examples 2, 3 and 4 have been provided solely for intuition. Their PIDs can be inferred directly from Equations (1)-(3). We next describe three one-dimensional examples that each have two non-zero PID components. For lack of space, we only provide a brief description and defer details to App. B. We estimate the  $\sim_{G}$ -PID (as well as the  $\delta_{G}$ -PID [12] and the ground-truth MMI-PID [13]) for these examples and show that all three are equal (see Fig. 1).
Example 5 (Unique and redundant information). Let  $X$  be a noisy representation of  $M$ , and let  $Y$  be a noisy representation of  $X$  with standard deviation  $\sigma_{Y|X}$ . When  $Y = X$  (zero noise), this example reduces to Example 3. As  $\sigma_{Y|X} \to \infty$ ,  $RI$  reduces while  $UI_{X}$  approaches  $I(M;X)$ .
Example 6 (Unique and synergistic information). Let  $M \sim \mathcal{N}(0,1)$ ,  $H_{X} = 1$ ,  $H_{Y} = 0$  and  $N_{X}, N_{Y} \sim \mathcal{N}(0, \sigma^{2})$  such that their correlation is  $\rho$ . When  $\sigma^2$  is finite and  $\rho = 0$ , this example reduces to Example 2, since there can be no synergy between  $X$  and  $Y$ . As  $\rho \to 1$ ,  $X - Y \to M$ ; so the total mutual information  $I(M; (X,Y)) \to \infty$ , driven by synergy growing unbounded, while the unique component remains constant at  $I(M; X)$ .
Example 7 (Redundant and synergistic information). Let  $M \sim \mathcal{N}(0,1)$ ,  $H_{X} = H_{Y} = 1$  and  $N_{X}, N_{Y} \sim \mathcal{N}(0,1)$  such that their correlation is  $\rho$ . When  $\rho < 1$ ,  $I(M;X)$  and  $I(M;Y)$  are both equal by symmetry, and thus equal to  $RI$  (see Def. 3 for the MMI-PID, which is ground truth here). As  $\rho$  reduces, the two channels  $X$  and  $Y$  have noisy representations of  $M$  with increasingly independent noise terms. Averaging the two,  $(X + Y) / 2$ , will provide more information about  $M$  than either one of them individually (i.e., synergy), and thus  $SI$  increases as  $\rho$  reduces.
Figure 3: The first five plots on the left show PID values for Example 10. Different shadings represent different values of gain in  $X_{1}$  ( $\alpha$ ) from Example 8. The  $\sim_{G}$ -PID doubles every time  $d$  doubles as seen by the constant  $45^{\circ}$  slope on the base-2 log-log plot, even when  $d_{M} = d_{X} = d_{Y} = 1024$ . The right-most plot shows a timing comparison between the  $\sim_{G}$ - and the  $\delta_{G}$ -PIDs on this example. The  $\delta_{G}$ -PID failed for  $d > 64$ ; the  $\sim_{G}$ -PID's timing performance up to  $d = 1024$  is shown in App. B.4.
The next set of examples will use the examples presented above in different combinations. This ensures that, where possible, the ground truth remains known in accordance with Property 1. These examples are also designed to reveal the differences between the  $\sim$ -PID, the MMI-PID and the  $\delta$ -PID: in particular, they show how the MMI-PID and the  $\delta$ -PID fail where the  $\sim$ -PID does not. These examples use two-dimensional  $M$ ,  $X$  and  $Y$ , i.e.,  $(d_M, d_X, d_Y) = (2, 2, 2)$ . A diagrammatic representation of Examples 8 and 9 is given in App. B.2.
Example 8. Let  $X_{1} = \alpha M_{1} + N_{X,1}$ ,  $Y_{1} = M_{1} + N_{Y,1}$ ,  $X_{2} = M_{2} + N_{X,2}$  and  $Y_{2} = 3M_{2} + N_{Y,2}$ , where  $M_{1}, M_{2}, N_{X,i}, N_{Y,i} \sim \mathrm{i.i.d.} \mathcal{N}(0,1)$ ,  $i = 1,2$ . Here,  $(M_1,X_1,Y_1)$  is independent of  $(M_2,X_2,Y_2)$ , therefore using Property 1, we can add the PID values from their individual decompositions (which each have known ground truth via the MMI-PID since  $M_{1}$  and  $M_{2}$  are scalar). Fig. 2(l) compares the  $\sim_G$ -PID, the  $\delta_G$ -PID and the MMI-PID for the joint decomposition of  $I(M;(X,Y))$ , at different values of  $\alpha$ , the gain in  $X_{1}$ . Only the  $\sim_G$ -PID matches the ground truth, as it is the only definition here that is additive.
Example 9. Let  $M$  and  $Y$  be as in Example 8. Suppose  $X = H_{X} R(\theta)M$ , where  $H_{X}$  is a diagonal matrix with diagonal entries 3 and 1, and  $R(\theta)$  is a  $2 \times 2$  rotation matrix that rotates  $M$  by an angle  $\theta$ . When  $\theta = 0$ ,  $X$  has higher gain for  $M_{1}$  while  $Y$  has higher gain for  $M_{2}$ . When  $\theta$  increases to  $\pi / 2$ ,  $X$  and  $Y$  have equal gains for both  $M_{1}$  and  $M_{2}$  (barring a difference in sign). Since  $(M_{1}, X_{1}, Y_{1})$  is not independent of  $(M_{2}, X_{2}, Y_{2})$  for all  $\theta$ , we know the ground truth only at the end-points. Nonetheless, the example shows a difference between the three definitions, as shown in Fig. 2(r).
Example 10. In this example, we test the stability of the  $\sim_G$ -PID as the dimensionality,  $d \coloneqq d_M = d_X = d_Y$  increases. By Property 1, if we take two i.i.d. systems of variables  $(M, X, Y)$  at dimensionality  $d$  and concatenate their respective variables, every PID component of the composite system of dimensionality  $2d$  should be double that of the original. This process can be repeated, taking two independent  $2d$ -dimensional systems and concatenating them to create a  $4d$ -dimensional system. Fig. 3 shows precisely this process starting with the system in Example 8 with  $d = 2$ , and continually doubling its size until  $d = 1024$ . The  $\sim_G$ -PID accurately matches ground truth by doubling in value, and remains stable with small relative errors (shown in App. B.5). The  $\sim_G$ -PID also runs much faster than the  $\delta_G$ -PID on this example, providing a speed-up of more than  $1000\times$  at  $d = 64$  (see right-most plot in Fig. 3, and extended results in App. B.4).
Remark 2. In the above examples, the ground truth referred to the  $\sim$ -PID of Def. 1, since it was inferred using Def. 3 and additivity (Property 1). Since our method for computing the  $\sim_G$ -PID recovers the ground truth  $\sim$ -PID, we can infer that the distribution of the optimal  $Q_{MXY}$  for the  $\sim$ -PID is in fact Gaussian in these examples. This provides empirical evidence in support of the conjecture stated in Sec. 3.
5 Estimation and Bias-correction for the  $\sim_G$ -PID
Having discussed how to compute the  $\sim_G$ -PID and shown that it agrees well with ground truth in several canonical examples, we discuss how the  $\sim_G$ -PID may be estimated from data. Given a sample
Stability of the  $\sim_{G}$ -PID over increasing dimensionality
Figure 4: Empirical evaluation of bias-corrected PID estimates with increasing sample size for two configurations with  $d_{M} = d_{X} = d_{Y} = 10$ , as described in Section 5. Solid horizontal lines represent ground truth (as computed from the true covariance matrix); dark colored bars represent biased PID components and light colored bars represent bias-corrected PID components, as estimated from the sample covariance matrix. Overlaid box plots indicate results from 100 random draws. Empirically, we find our bias-corrected estimates are both unbiased and consistent.
of  $n$  realizations of  $M$ ,  $X$  and  $Y$  drawn from  $P_{MXY}$ , we may estimate the sample joint covariance matrix  $\hat{\Sigma}_{MXY}$ . We therefore use the straightforward "plug-in" estimator for the  $\sim_G$ -PID, by using  $\hat{\Sigma}_{MXY}$  in place of  $\Sigma_{MXY}$  in the optimization problem in equation (9).
However, it is well-known that estimators of information-theoretic quantities suffer from large biases for moderate sample sizes [37]. Cai et al. [38] characterized the bias in the entropy of a  $d$ -dimensional Gaussian random vector, for a fixed sample size  $n$ .
Proposition 3 (Bias in Gaussian entropy [38]). Suppose  $M \in \mathbb{R}^{d_M}$  has an auto-covariance matrix  $\Sigma_M$ . The entropy of  $M$  is  $H(M) = \frac{1}{2}\log \det(2\pi e\Sigma_M)$  when  $\Sigma_M$  is known [24]. For the sample covariance matrix  $\hat{\Sigma}_M$ , the bias is given by:
$$
\operatorname {B i a s} \left[ \hat {H} (M) \right] = \sum_ {k = 1} ^ {d _ {M}} \log (1 - k / n). \tag {15}
$$
For a proof, we refer the reader to [38, Corollary 2]. This result may be naturally extended to compute the bias of each of the mutual information quantities in the LHS of equations (1)-(3):
Corollary 4 (Bias in Gaussian mutual information). For the joint mutual information  $I(M; (X, Y))$ ,
$$
\operatorname {B i a s} \left[ \hat {I} (M; (X, Y)) \right] = \sum_ {k = 1} ^ {d _ {M}} \log (1 - k / n) + \sum_ {k = 1} ^ {d _ {X} + d _ {Y}} \log (1 - k / n) - \sum_ {k = 1} ^ {d _ {M} + d _ {X} + d _ {Y}} \log (1 - k / n) \tag {16}
$$
This follows directly from the fact that  $I(M; (X, Y)) = H(M) + H(X, Y) - H(M, X, Y)$ . Similarly, we can compute the bias of  $\hat{I}(M; X)$  and  $\hat{I}(M; Y)$ . But this does not uniquely determine the bias in the individual PID components, and as with defining PIDs, we need to decompose the bias in Corollary 4 across the four PID components such that they agree with these constraints. We solve this problem by defining a bias-corrected version of the union information from Proposition 1.
Definition 4 (Bias-corrected Union Information). We assign the bias in the union information to be the same fraction as the bias in the joint mutual information  $I(M; (X, Y))$ . This gives rise to a bias-corrected estimate of the union information:
$$
\left. \widetilde {I _ {G} ^ {\cup}} \right| _ {\text {b i a s - c o r r}} (M: X; Y) := \widetilde {I _ {G} ^ {\cup}} (M: X; Y) \left(1 - \frac {\text {B i a s} [ \hat {I} (M ; (X , Y)) ]}{\hat {I} (M ; (X , Y))}\right). \tag {17}
$$
We do not analyze theoretically whether the bias-corrected union information is consistent and unbiased, thus, it may still have some residual bias relative to the true union information. However, we find empirically that this bias correction process works reasonably well and appears both consistent and unbiased, in a number of examples. Figure 4 shows biased and bias-corrected PID values for
PID values (left) and PID values normalized by  $I(M; (X, Y))$  (right) in a multivariate Poisson spike-count simulation
Figure 5: A comparison of the  $\sim_{G}$ -PID, the  $\delta_{G}$ -PID and the MMI-PID for a multivariate Poisson system. The ground truth is a discrete  $\sim$ -PID computed using the package of Banerjee et al. [30]. The  $\sim_{G}$ -PID comes closest to the ground truth (possibly because they compute the same PID definition), despite the fact that the  $\sim_{G}$ -PID only uses the covariance matrix of the Poisson distribution, whereas the ground truth uses knowledge of the distribution itself.
Figure 6: Bias-corrected redundancy estimates for information about VISp activity that is shared between VIS1 and VISal: redundancy in bits (left) and redundancy as a fraction of total mutual information (right). Data spread is across 40 mice. Statistical comparisons use a two-sided Mann-Whitney-Wilcoxon test. Observe that there is greater and more sustained redundancy on flashes corresponding to image changes, which are behaviorally relevant and linked to rewards in this task.
100 runs of two configurations called "Bit-of-all" (with a little bit of each PID component) and "Fully-redundant" (which has predominantly redundancy), with  $d_{M} = d_{X} = d_{Y} = 10$  (details and additional setups in App. C). We find that bias correction brings the PID values closer to their true values even at small sample sizes. In App. C.3, we also include a preliminary analysis of the variance of PID estimates using bootstrap [39, Ch. 8].
6 Application to Simulated and Real Neural Data
So far, we have only considered the  $\sim_G$ -PID when applied to Gaussian  $P_{MXY}$ . Although Def. 2 strictly applies only to Gaussian  $P_{MXY}$ , the estimation process in Sec. 5 relies only on a sample covariance matrix, which is well-defined for a wide variety of non-Gaussian distributions. Many applications where PIDs could be useful have non-Gaussian data. For instance, there is great interest in applying PIDs in neuroscience (e.g., to understand how multiple brain regions jointly encode and communicate information [6, 40]), but spike-count distributions are non-Gaussian.
Simulated neural data. To show that our  $\sim_{G}$ -PID estimates provide reasonable results on non-Gaussian spiking neural data, we first simulate spike-count data using Poisson random variables (following [12]; described in App. D.1). We evaluate the ground truth  $\sim$ -PID for this distribution using the discrete PID estimator of Banerjee et al. [20]. The  $\sim_{G}$ -PID is estimated from a sample covariance matrix using  $10^{6}$  realizations of  $M$ ,  $X$  and  $Y$ . We find that the  $\sim_{G}$ -PID closely matches
the ground truth for a range of parameter values, despite the fact that the  $\sim_{G}$ -PID is effectively computed on a Gaussian approximation of a Poisson distribution (Fig. 5). More examples of the  $\sim_{G}$ -PID applied to non-Gaussian data are provided in App. D.2 (also see App. A.2). We conclude that it is reasonable to use and interpret the  $\sim_{G}$ -PID on non-Gaussian spike count data.
Real neural data. We then applied our bias-corrected  $\sim_{G}$ -PID estimator to the Visual Behavior Neuropixels dataset collected by us at the Allen Institute [41]. We recorded over 80 mice using six neuropixels probes targeting various regions of visual cortex, while the mice were engaged in a visual change-detection task. In the task, images from a set of 8 natural scenes were presented in  $250~\mathrm{ms}$  flashes, at intervals of  $750~\mathrm{ms}$ ; the image would stay the same for a variable number of flashes after which it would change to a new image. The mouse had to lick to receive a water reward when the image changed. Thus, a given image flash could be a behaviorally relevant target if the previous image was different, or not, if the previous image was the same.
We used our bias-corrected PID estimator to understand how information is processed along the visual hierarchy during this task. We estimated the  $\sim_G$ -PID to understand how information contained in the spiking activity of primary visual cortex (VISp) was represented in two higher-order visual cortical areas, VIS1 and VISal. We aligned trials to the onset of a stimulus flash, binned spikes in 50 ms intervals and considered the top 20 principal components (to achieve reasonable estimates at these sample sizes; explained further in App. D.5) from each region in each time bin. We computed the  $\sim_G$ -PID on the sample covariance matrix of these principal components (shown in Fig. 6). We found that there was a significantly larger amount of redundant information about VISp activity between VIS1 and VISal for stimulus flashes corresponding to an image change, compared to flashes that were not changes (Fig. 6(l)). The larger redundancy was also sustained slightly longer for flashes corresponding to changes, than non-change flashes. Both of these effects were maintained even when the redundancy was normalized by the joint mutual information, suggesting that the effect was not purely due to an increase in the total amount of information (Fig. 6(r)). Our results suggest that the visual cortex propagates information throughout the hierarchy more robustly when such information is relevant for behavior.
In this paper, we proposed a new and efficient method for estimating the  $\sim_G$ -PID for Gaussian distributions. We showed that our method recovers the ground truth and suitably corrects for bias through a series of examples. In particular, Fig. 4 showed how large the biases can be at small sample sizes, which makes bias correction particularly important in neuroscientific settings where sample sizes are often small.
We focused on Gaussian distributions, as they have historically been a starting point for many estimators (e.g., correlation is used as a measure of dependence, but zero correlation implies independence only in the Gaussian case). We were able to show ground-truth validation for our  $\sim_{G}$ -PID estimator at high dimensionalities only thanks to the existence of closed-form results on scalar Gaussians.
While our central claims and results applied to Gaussian distributions, our method for computing PIDs did not immediately break down for distributions close to Gaussian in some limit (e.g., Poisson). The effective spiking rate used in our multivariate Poisson distribution was also not particularly large (see App. D.1); we would expect our estimates to improve for higher firing rates, since the Poisson distribution will then be more Gaussian.
Limitations. Our work has several limitations that require further theory and simulations to resolve, the most important of which are: (1) Our estimator is technically a bound on the PID values because we assume Gaussian optimality in Definition 2; (2) Our bias-correction method is heuristic: we do not provide a rigorous theoretical characterization of the bias of PID values.
Broader impacts. Our work is mainly methodological, so the scope for negative impacts depends on how the methods might be used. For example, incorrect interpretations drawn from the use of our PID estimators may affect scientific conclusions. In particular, the PID is inherently a correlational quantity and carries the same caveats: it may not be appropriate to make causal interpretations on the basis of observed PID values. Also, despite our best efforts to explore a variety of systems, we cannot tell how accurate our bias-correction method will be in novel configurations.
We thank Lukasz Kusmierz for providing a valuable reference on the bias of Shannon entropy estimates. We also thank Gabe Schamberg and Christof Koch for helpful discussions.
We wish to thank the Allen Institute founder, Paul G. Allen, for his vision, encouragement, and support. Praveen Venkatesh was supported by a Shanahan Family Foundation Fellowship at the Interface of Data and Neuroscience, supported in part by the Allen Institute. Stefan Mihalas was in part supported by NSF 2223725, NIH R01EB029813, and RF1DA055669 grants.
Praveen Venkatesh, Corbett Bennett, Sam Gale, Tamina K. Ramirez, Greggory Heller, Severine Durand, Shawn Olsen, Stefan Mihalas
A Supplementary Material for Sections 2 and 3
A.1 Implications of the Additivity Property
As mentioned in the main text, additivity states that the PID values of an isolated system should not depend on the PID values of another isolated system. Without additivity, it is not possible to examine independent systems in isolation, since broadening your view to include a different isolated system could change the PID values of the first system.
Hypothetically, suppose we have two separate individuals (labeled 1 and 2) receiving completely independent stimuli, and we examine the PID between the activity in brain regions  $M$ ,  $X$  and  $Y$  in each of their brains. Then the total unique information that  $X_{1}$  and  $X_{2}$  have about  $M_{1}$  and  $M_{2}$  with respect to  $Y_{1}$  and  $Y_{2}$  should be equal to the sum of the unique information in each of the individuals taken separately.
As another example, suppose we are trying to examine visual information flow and auditory information flow in a multi-sensory integration task. We may want to understand the degree to which the activity in the sub-regions of the visual and auditory systems depend on each other. A reasonable null model of independence between the two systems would be that the PID values of the joint system will be equal to the sum of the PID values in the two individual systems. Then, measuring the actual degree to which the joint PID value is not equal to the sum will be a meaningful measure of dependence between the systems. This would only be possible with a PID definition that guaranteed additivity of independent sub-systems. However, since the  $\delta$ -PID does not satisfy the additivity (Property 1), we cannot guarantee that the aforementioned null would be the correct null model, and we would not be able to perform such an analysis.
A.2 Explaining the Exponential Reduction in the Number of Optimization Variables
In Def. 2, we restrict  $Q_{MXY}$  to be jointly Gaussian. This reduces the number of optimization variables from being exponential in the dimensionality to quadratic, as we show here.
Previous discrete  $\sim$ -PID estimators [20, 21] have found efficient methods to solve the optimization problem of Def. 1 without reducing the number of optimization variables. If  $P_{MXY}$  is discrete in Def. 1, and each dimension of  $M$ ,  $X$  and  $Y$  has a support of size  $K$  (i.e.,  $M_i$ ,  $X_i$  and  $Y_i$  can each take one of  $K$  discrete values), then the total number of degrees of freedom in  $\Delta_P$  is  $\mathcal{O}(K^{(d_M + d_X + d_Y)})$ . This corresponds to the total support of a discrete  $Q_{MXY}$ , which can be an arbitrarily complex discrete distribution.
In contrast, the  $\sim_G$ -PID in Def. 2 reduces the dimensionality of the optimization space by assuming that  $Q_{MXY}$  is Gaussian.  $Q_{MXY}$  is then completely parameterized by its covariance matrix,  $\Sigma_{MXY}^Q$ . In fact, given the additional constraints in Def. 2, the only part of  $\Sigma_{MXY}^Q$  that is variable is  $\Sigma_{X,Y}^Q$ , as explained in Sec. 3.2. This matrix has only  $d_X\times d_Y$  variables. Thus, by restricting the optimization space to jointly Gaussian distributions, we have reduced the number of variables from exponential in the dimensionality to quadratic.
As a concrete example, consider a neuroscientific setting (as in Sec. 6) where  $M$ ,  $X$  and  $Y$  represent the spiking activity of neurons in three different brain regions, so that  $d_{M}$ ,  $d_{X}$  and  $d_{Y}$  represent the number of neurons, and  $K$  represents the maximum number of spikes. Makkeh et al. [21] demonstrated their method for a maximum support size of  $K = 18$  with  $d_{M} = d_{X} = d_{Y} = 1$ , for a total support of approximately  $18^{3} - 1 = 5831$  in  $Q_{MXY}$ . If we assume a single neuron can have at most  $K = 10$  spikes, then we get an effective dimensionality of  $\log_{10}(5831) < 4$ . In other words,
these discrete  $\sim$ -PID estimators can handle around 4 neurons across all three regions  $M$ ,  $X$  and  $Y$  (which is also what we use in the simulation of Fig. 5 in Sec. 6), whereas our  $\sim_{G}$ -PID estimator can handle several hundreds if not thousands of neurons (as seen in Example 10).
A.3 Proofs of Propositions 1 and 2
Proof of Proposition 1. Firstly, the differential entropy of a Gaussian random variable  $M$  with covariance matrix  $\Sigma_{M}$  is given by [43, Thm. 8.4.1]:
$$
h (M) = \frac {1}{2} \log \det  \left(2 \pi e \Sigma_ {M}\right). \tag {18}
$$
Secondly, for a joint Gaussian distribution  $P_{MXY}$  parameterized by a covariance matrix  $\Sigma_{MXY}$ , the conditional covariance matrix can be written as [44, Sec. 8.1.3]:
$$
\Sigma_ {X Y \mid M} = \Sigma_ {X Y} - \Sigma_ {X Y, M} \Sigma_ {M} ^ {- 1} \Sigma_ {X Y, M} ^ {\mathsf {T}} \tag {19}
$$
$$
\Rightarrow \quad \Sigma_ {X Y} = \Sigma_ {X Y | M} + \Sigma_ {X Y, M} \Sigma_ {M} ^ {- 1} \Sigma_ {X Y, M} ^ {\mathsf {T}} \tag {20}
$$
Using these two equations, we can derive the mutual information between  $M$  and  $(X,Y)$  as follows:
$$
I (M; (X, Y)) = h (X, Y) - h (X, Y \mid M) \tag {21}
$$
$$
\stackrel {(a)} {=} \frac {1}{2} \log \det  (2 \pi e \Sigma_ {X Y}) - \frac {1}{2} \log \det  (2 \pi e \Sigma_ {X Y | M}) \tag {22}
$$
$$
= \frac {1}{2} \log \left((2 \pi e) ^ {d _ {M}} \det (\Sigma_ {X Y})\right) - \frac {1}{2} \log \left((2 \pi e) ^ {d _ {M}} \det (\Sigma_ {X Y | M})\right) \tag {23}
$$
$$
= \frac {1}{2} \log \left(\frac {\det (\Sigma_ {X Y})}{\det (\Sigma_ {X Y | M})}\right) \tag {24}
$$
$$
\stackrel {(b)} {=} \frac {1}{2} \log \left(\frac {\det  \left(\Sigma_ {X Y \mid M} + \Sigma_ {X Y , M} \Sigma_ {M} ^ {- 1} \Sigma_ {X Y , M} ^ {\top}\right)}{\det  \left(\Sigma_ {X Y \mid M}\right)}\right) \tag {25}
$$
$$
= \frac {1}{2} \log \left(\frac {\det \left(\Sigma_ {X Y \mid M}\right) \det \left(I + \Sigma_ {X Y \mid M} ^ {- 1} \Sigma_ {X Y , M} \Sigma_ {M} ^ {- 1} \Sigma_ {X Y , M} ^ {\top}\right)}{\det \left(\Sigma_ {X Y \mid M}\right)}\right) \tag {26}
$$
$$
= \frac {1}{2} \log \det  \left(I + \Sigma_ {X Y | M} ^ {- 1} \Sigma_ {X Y, M} \Sigma_ {M} ^ {- 1} \Sigma_ {X Y, M} ^ {\mathsf {T}}\right) \tag {27}
$$