SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Praveen Venkatesh $^{1,2,*}$ , Corbett Bennett $^{1}$ , Sam Gale $^{1}$ , Tamina K. Ramirez $^{3}$ , Greggory Heller $^{4}$ , Severine Durand $^{1}$ , Shawn Olsen $^{1}$ , Stefan Mihalas $^{1,\dagger}$
$^{1}$ Allen Institute;  $^{2}$ University of Washington;  $^{3}$ Columbia University;
4Massachusetts Institute of Technology
* praveen.venkatesh@alleninstitute.org; †stefanm@alleninstitute.org
Recent advances in neuroscientific experimental techniques have enabled us to simultaneously record the activity of thousands of neurons across multiple brain regions. This has led to a growing need for computational tools capable of analyzing how task-relevant information is represented and communicated between several brain regions. Partial information decompositions (PIDs) have emerged as one such tool, quantifying how much unique, redundant and synergistic information two or more brain regions carry about a task-relevant message. However, computing PIDs is computationally challenging in practice, and statistical issues such as the bias and variance of estimates remain largely unexplored. In this paper, we propose a new method for efficiently computing and estimating a PID definition on multivariate Gaussian distributions. We show empirically that our method satisfies an intuitive additivity property, and recovers the ground truth in a battery of canonical examples, even at high dimensionality. We also propose and evaluate, for the first time, a method to correct the bias in PID estimates at finite sample sizes. Finally, we demonstrate that our Gaussian PID effectively characterizes inter-areal interactions in the mouse brain, revealing higher redundancy between visual areas when a stimulus is behaviorally relevant.
Neuroscientific experiments are increasingly collecting large-scale datasets with simultaneous recordings of multiple brain regions with single-unit resolution [1-3]. These experimental advances call for new computational tools that can allow us to probe how multiple brain regions jointly process relevant information in a behaving animal.
Partial Information Decompositions (PIDs) offer a new method for studying how different brain regions carry task-relevant information: they provide measures to quantify the amount of unique, redundant and synergistic information that one region has with respect to another. The information itself could pertain to task-relevant variables such as stimuli, behavioral responses, or information contained in a third region. For example, we may be interested in how much information about a stimulus is communicated or shared (i.e., redundantly present) between two brain regions over time. Or, we might be interested in the extent to which one region's activity uniquely explains that of another, while excluding information corresponding to spontaneous behaviors.
Ideas such as redundancy and synergy have a long history in neuroscience, having been proposed for understanding noise correlations [4] and to understand differences in encoding complexity between different brain regions [5]. PIDs have also been suggested for quantifying how much sensory information is used to execute behaviors [6] and for tracking stimulus-dependent information flows between brain regions [7, 8]. Outside of neuroscience, PID has been used to understand interactions
between different variables in financial markets [9], to quantify the relevance of different features for the purpose of feature selection in machine learning [10], and to define and quantify bias in the field of fair Machine Learning [11].
An important constraint that has limited the broader adoption of PIDs in neuroscience is the computational difficulty of estimating PIDs for high-dimensional data. Many PID definitions that are operationally well-motivated involve solving an optimization problem over a space of probability distributions: the number of optimization variables can thus be exponential in the number of neurons [12]. This has led to the use of poorly motivated PID definitions that are easy to compute (such as the "MMI-PID" of [13], in works such as [9, 14-16]), or limited analyses to very few dimensions [17]. Furthermore, due to the limited exploration of estimators for PIDs, issues such as the bias and variance of estimates have received no attention so far, to our knowledge.
In this paper, we make the following contributions:
1. We provide a new and efficient method for computing and estimating a well-known PID definition called the  $\sim$ -PID or the BROJA-PID [18] on Gaussian distributions (Section 3). By restricting our attention to Gaussian distributions, we are able to significantly reduce the number of optimization variables, so that this is just quadratic in the number of neurons, rather than exponential.
2. We present a set of canonical examples for Gaussian distributions where ground truth is known, and show that our method outperforms others (Section 4).
3. We also raise (for what we believe is the first time) the issue of bias in PID estimates, propose a method for correcting the bias, and empirically evaluate its performance (Section 5).
4. Finally, we show that our Gaussian PID estimator closely agrees with ground truth, even on non-Gaussian distributions, and show an example of its use on real neural data (Section 6).
Our method is based on our earlier work [12], where we also examined PIDs for Gaussian distributions. Our current work differs in a few key aspects: (i) we estimate the PID of a different PID definition, the  $\sim$ -PID rather than the  $\delta$ -PID, because the  $\delta$ -PID does not satisfy a basic property called additivity [19] (defined in Sec. 2); (ii) our current method provides an exact upper bound to the PID definition being computed, rather than an approximate upper bound; (iii) we now consider the problem of estimation, not just computation, and explore the issue of the bias of PID estimates; and (iv) our current method is much faster, and we demonstrate agreement with ground truth at much higher dimensionality.
Several other studies have also considered methods for efficiently estimating PIDs: Banerjee et al. [20] and Makkeh et al. [21] address computing discrete PIDs, but their method does not scale to higher dimensions; Pakman et al. [17] estimate PIDs using copulas, but their method would also potentially be computationally prohibitive at high dimensionalities; Liang et al. [22] use convex optimization to directly estimate the  $\sim$ -PID for general high-dimensional distributions, but they do not compare with ground truth at high dimensionality or examine bias in their estimates.
2 Background: An Introduction to PIDs and the  $\sim$ -PID
In this section, we provide an introduction to the concept of partial information decomposition along with an illustrative example. Let  $M$ ,  $X$  and  $Y$  be three random variables with joint distribution  $P_{MXY}$ . A PID decomposes the total mutual information between the message  $M$  and two constituent random variables  $X$  and  $Y$  into a sum of four non-negative components that satisfy [18, 23]:
$$
\begin{array}{l} I (M; (X, Y)) = U I (M: X \backslash Y) + U I (M: Y \backslash X) + R I (M: X; Y) + S I (M: X; Y) (1) \\ I (M; X) = U I (M: X \backslash Y) + R I (M: X; Y) (2) \\ I (M; Y) = U I (M: Y \backslash X) + R I (M: X; Y) (3) \\ \end{array}
$$
Here,  $I(A;B)$  is the Shannon mutual information [24] between the random variables  $A$  and  $B$ , and the four terms in the RHS of (1) are respectively the information about  $M$  that is (i) uniquely present in  $X$  and not in  $Y$ ; (ii) uniquely present in  $Y$  and not in  $X$ ; (iii) redundantly present in both  $X$  and  $Y$  and can be extracted from either; and (iv) synergistically present in  $X$  and in  $Y$ , i.e., information which cannot be extracted from either of them individually, but can be extracted from their interaction. For the sake of brevity, we may also refer to these partial information components as  $UI_{X}$ ,  $UI_{Y}$ ,  $RI$  and  $SI$  respectively. Notwithstanding notation, they should all properly be understood to be functions of the joint distribution  $P_{MXY}$ .
Now,  $UI_{X}, UI_{Y}, RI$  and  $SI$  consist of four undefined quantities, subject to the three equations in (1)-(3). In addition, they are typically assumed to be non-negative,  $RI$  and  $SI$  are each constrained to be symmetric in  $X$  and  $Y$ , and the functional forms of  $UI_{X}$  and  $UI_{Y}$  should be identical when exchanging  $X$  for  $Y$ . Despite the number of constraints, many definitions satisfy all of them, each differing in its motivation and interpretation [18, 23, 25-27] (see [27, 28] for a review), and we need to formally define one of these partial information components to determine the other three.
Example 1. Before we jump into a specific definition, we provide an intuition into what these terms mean using a simple example. Suppose  $M = [A,B,C]$ ,  $X = [A,B,C\oplus Z]$ , and  $Y = [B,Z]$ , where  $A,B,C,Z\sim \mathrm{i.i.d.}\operatorname {Ber}(0.5)$ . Then,  $X$  has 1 bit of unique information about  $M$ , i.e.,  $A;Y$  has no unique information;  $X$  and  $Y$  both have 1 bit of redundant information, i.e.,  $B$ , since it can be obtained from either  $X$  or  $Y$ ; and  $X$  and  $Y$  have 1 bit of synergistic information, i.e.,  $C$ , which cannot be obtained from either  $X$  or  $Y$  individually (since  $C\oplus Z\perp C$ ), but can only be recovered when both  $X$  and  $Y$  are known. For more examples on binary variables, we refer the reader to [18].
In this manuscript, we consider a definition that we refer to as the  $\sim$ - $\mathrm{PID}^2$  [18, 25], which is defined below. We chose to build an estimator for this definition for two reasons: (i) it is a Blackwellian PID definition, i.e., it has well-defined operational interpretations based on concepts from statistical decision theory (e.g., see [18, 27, 29] for details); and (ii) it satisfies many desirable properties (e.g., see [18, 30]), and in particular, a property that we call additivity of independent components.
Definition 1 ( $\sim$ -PID [18]). The unique information about  $M$  present in  $X$  and not in  $Y$  is given by
$$
\widetilde {U I} (M: X \backslash Y) := \min  _ {Q \in \Delta_ {P}} I _ {Q} (M; X \mid Y), \tag {4}
$$
where  $\Delta_P\coloneqq \{Q_{MXY}:Q_{MX} = P_{MX},Q_{MY} = P_{MY}\}$  and  $I_Q(\cdot ;\cdot |\cdot)$  is the conditional mutual information over the joint distribution  $Q_{MXY}$ . The remaining  $\sim$  -PID components,  $\widetilde{U} I(M:Y\setminus X)$ ,  $\widetilde{R} I(M:X;Y)$  and  $\widetilde{S} I(M:X;Y)$ , follow from equations (1)-(3).
Property 1 (Additivity of independent components). Suppose  $M = [M_1, M_2]$ ,  $X = [X_1, X_2]$ , and  $Y = [Y_1, Y_2]$ , such that  $(M_1, X_1, Y_1) \perp (M_2, X_2, Y_2)$ . Then, additivity implies that
$$
U I (M: X \backslash Y) = U I \left(M _ {1}: X _ {1} \backslash Y _ {1}\right) + U I \left(M _ {2}: X _ {2} \backslash Y _ {2}\right), \tag {5}
$$
and similarly for the other three partial information components,  $U I_{Y}$ ,  $RI$  and  $SI$ .
Property 1 stipulates that we should be able to compute the PIDs of two independent systems separately, and then add the components across both systems. In effect, additivity implies that the PID of an isolated system should not depend on the PID of another isolated system, making it an intuitive and highly desirable property (see App. A.1 for concrete examples). Of the many PID definitions examined by Rauh et al. [19], only the  $\sim$ -PID satisfied additivity (as proved in [18]).
3 Computing the  $\sim$ -PID for Gaussian Distributions
The first contribution of this paper is a method to efficiently compute bounds on the  $\sim$ -PID for jointly Gaussian random vectors  $M, X$  and  $Y$ . To be precise, our method computes an upper bound for  $\widetilde{U}I_X$  and  $\widetilde{U}I_Y$ , and lower bounds for  $\widetilde{R}I$  and  $\widetilde{S}I$ . Similar to our earlier work [12], we present a new PID definition that we call the  $\sim_G$ -PID, which characterizes an upper bound on the unique information of the  $\sim$ -PID by restricting the optimization space to jointly Gaussian  $Q_{MXY}$ :
Definition 2 ( $\sim_G$ -PID). Let  $P_{MXY}$  be jointly Gaussian. Then, the unique information about  $M$  present in  $X$  and not in  $Y$  is given by
$$
\widetilde {U I} _ {G} (M: X \backslash Y) := \min  _ {Q \in \Delta_ {P}} I _ {Q} (M; X \mid Y), \tag {6}
$$
where  $\Delta_P \coloneqq \{Q_{MXY} : Q_{MXY}$  jointly Gaussian,  $Q_{MX} = P_{MX}$ ,  $Q_{MY} = P_{MY}\}$  and  $I_Q$  is the conditional mutual information over the joint distribution  $Q_{MXY}$ .
If the optimal  $Q_{MXY}$  in the unrestricted optimization of Definition 1 happens to be Gaussian for some  $P_{MXY}$ , then the  $\sim_G$ -PID would be identical to the  $\sim$ -PID for that  $P_{MXY}$ . We conjecture
i.i.d. stands for "independent and identically distributed";  $X \perp Y$  means  $X$  and  $Y$  are independent.
2This PID is also sometimes referred to as the BROJA PID (after the authors of [18]), or the minimum-synergy PID in the literature. We prefer to use an author-agnostic nomenclature as introduced in our earlier work [12], because this PID was also introduced contemporaneously by [25].
that this happens whenever  $P_{MXY}$  is Gaussian: for example, in a similar optimization problem for computing the information bottleneck [31], the optimal distribution is Gaussian whenever  $P$  is Gaussian [32, 33]. We provide empirical evidence in favor of this conjecture through a sequence of examples with Gaussian  $P$  in Sec. 4, where we recover the ground truth even when  $Q$  is restricted to be Gaussian. However, we leave a theoretical examination of this conjecture for future work.
In practical terms, restricting the search space to Gaussian  $Q_{MXY}$  reduces the number of optimization variables from being exponential in the dimensionality to quadratic (see Appendix A.2), allowing us to compute the  $\sim_G$ -PID for much higher dimensionalities of  $M$ ,  $X$  and  $Y$ . In what follows, we show how the optimization problem for the  $\sim_G$ -PID can be written out in closed-form and then solved using projected gradient descent.
3.1 Notation and Preliminaries
Suppose  $M, X$  and  $Y$  are jointly Gaussian random vectors of dimensions  $d_{M}, d_{X}$  and  $d_{Y}$  respectively, with a joint covariance matrix given by  $\Sigma_{MXY}$ . We will make extensive use of the submatrices of  $\Sigma_{MXY}$ , so we explain their notation here:
-  $\Sigma_{XY}$  will denote the  $(d_X + d_Y)\times (d_X + d_Y)$  joint (auto-)covariance matrix of the vector  $[X^{\mathsf{T}},Y^{\mathsf{T}}]^{\mathsf{T}}$
-  $\Sigma_{X,Y}$  (note the comma) will denote the  $d_X\times d_Y$  cross-covariance matrix between  $X$  and  $Y$ .
-  $\Sigma_{XY,M}$  will denote the  $(d_X + d_Y)\times d_M$  cross-covariance matrix between the concatenated vector  $[X^{\mathsf{T}},Y^{\mathsf{T}}]^{\mathsf{T}}$  and the vector  $M$ .
In general, groupings of vectors without commas represent joint covariance, while a comma represents a cross-covariance between the groups on either side of the comma. The same notation will also be used for conditional covariance matrices: for example,  $\Sigma_{XY|M}$  is the conditional joint covariance of  $(X,Y)$  given  $M$ , while  $\Sigma_{X,Y|M}$  is the conditional cross-covariance between  $X$  and  $Y$  given  $M$ .
We will also use an equivalent notation for the joint distribution [12], where  $P_{MXY}$  is parameterized as a "broadcast channel" [24, Ch. 15.6] from  $M$  to  $X$  and  $Y$ :
$$
X = H _ {X} M + N _ {X} \quad \text {a n d} \quad Y = H _ {Y} M + N _ {Y}. \tag {7}
$$
Here,  $H_{X} \coloneqq \Sigma_{X,M}$  and  $H_{Y} \coloneqq \Sigma_{Y,M}$  represent channel gain matrices, while  $N_{X}$  and  $N_{Y}$  represent additive noise and are not necessarily independent of each other:  $[N_{X}^{\mathsf{T}}, N_{Y}^{\mathsf{T}}]^{\mathsf{T}} \sim \mathcal{N}(0, \Sigma_{XY|M})$ .
Remark 1. Without loss of generality, we can assume that  $M$ ,  $X$  and  $Y$  are all zero-mean, and that  $\Sigma_{M} = I$ . Further, we explicitly assume that the  $X$  and  $Y$  channels are individually whitened, i.e., that  $\Sigma_{X|M} = I$  and  $\Sigma_{Y|M} = I$ . This assumption precludes deterministic relationships between  $M$  and  $X$  or  $Y$ , and is required to ensure that information quantities remain finite [12].
3.2 Optimizing the Union Information
Bertschinger et al. [18] showed that the minimizer for the unique information is also the minimizer for the "union information",  $I^{\cup}(M:X;Y) \coloneqq UI_X + UI_Y + RI$ . In other words, we can also solve the following optimization problem, which yields simpler expressions for the objective and gradient:
$$
\widetilde {I ^ {\cup}} (M: X; Y) := \min  _ {Q _ {M X Y}} I _ {Q} (M; X, Y) \quad \text {s . t .} \quad Q _ {M X} = P _ {M X}, Q _ {M Y} = P _ {M Y} \tag {8}
$$
Now, suppose  $P_{MXY}$  is Gaussian with covariance  $\Sigma_{MXY}^{P}$  and the solution  $Q_{MXY}$  is also assumed to be Gaussian with covariance  $\Sigma_{MXY}^{Q}$ . Then, the constraint in (8) implies that  $\Sigma_{MX}^{Q} = \Sigma_{MX}^{P}$  and  $\Sigma_{MY}^{Q} = \Sigma_{MY}^{P}$ . In other words,  $\Sigma_{M}, \Sigma_{X}, \Sigma_{Y}$ , and  $\Sigma_{M,XY}$  are all constant across  $P$  and  $Q$ . Therefore, the only part of  $\Sigma_{MXY}^{Q}$  that is variable is  $\Sigma_{X,Y}^{Q}$ , or equivalently,  $\Sigma_{X,Y|M}^{Q}$ . In what follows, we will drop the superscripts denoting the distribution, as this will be clear from context. Generally speaking, we will discuss the optimization problem and thus the distribution will be  $Q$ .
Proposition 1. The union information for the  $\sim_G$ -PID of Definition 2 is given by
$$
\widetilde {I _ {G} ^ {\cup}} := \min  _ {\Sigma_ {X, Y | M}} \frac {1}{2} \log \det  \left(I + \Sigma_ {M} ^ {- 1} \Sigma_ {X Y, M} ^ {\mathsf {T}} \Sigma_ {X Y | M} ^ {- 1} \Sigma_ {X Y, M}\right) \quad s. t. \quad \Sigma_ {X Y | M} \succcurlyeq 0 \tag {9}
$$
<sup>3</sup>We can use  $\Sigma_{X,Y|M}$  in place of  $\Sigma_{X,Y}$  because they differ by a constant:  $\Sigma_{X,Y|M} - \Sigma_{X,Y}$  is an off-diagonal block in  $\Sigma_{XY} - \Sigma_{XY|M}$ , which is equal to  $\Sigma_{XY,M}\Sigma_M^{-1}\Sigma_{XY,M}^\top$ , which is constant across  $P$  and  $Q$ .
where the optimization variable  $\Sigma_{X,Y|M}$  is an off-diagonal block embedded within  $\Sigma_{XY|M}$ ; all other matrices in the objective are constants that are derived from  $\Sigma_{MXY}^P$ .
We solve the above optimization problem using projected gradient descent: we analytically derive the gradient and the projection operator for the constraint set as shown below. Then, we use the RProp algorithm [34, 35] for gradient descent, which independently adjusts the learning rates for each optimization parameter (derivations, and details of implementation and complexity are in App. A). Code for our implementation is available on GitHub [36], and details of the compute configuration are given in App. E.
Proposition 2. The objective in Proposition 1 can be simplified to
$$
f \left(\Sigma_ {X, Y \mid M}\right) = \frac {1}{2} \log \det  \left(I + H _ {Y} ^ {\top} H _ {Y} + B ^ {\top} S ^ {- 1} B\right), \tag {10}
$$
where  $B\coloneqq \big(H_X - \Sigma_{X,Y|M}H_Y\big)$  and  $S\coloneqq (I - \Sigma_{X,Y|M}\Sigma_{X,Y|M}^{\top})$
The gradient of the objective with respect to  $\Sigma_{X,Y|M}$  is given by
$$
\nabla f \left(\Sigma_ {X, Y \mid M}\right) = S ^ {- 1} B \left(I + H _ {Y} ^ {\top} H _ {Y} + B ^ {\top} S ^ {- 1} B\right) ^ {- 1} \left(B ^ {\top} S ^ {- 1} \Sigma_ {X, Y \mid M} - H _ {Y} ^ {\top}\right). \tag {11}
$$
A projection operator on to the constraint set  $\Sigma_{XY|M} \succcurlyeq 0$  can be obtained as follows: let  $\Sigma_{XY|M} = : V\Lambda V^{\top}$  be the eigenvalue decomposition of  $\Sigma_{XY|M}$ , with  $\Lambda = : \mathrm{diag}(\lambda_i)$ . Let  $\overline{\lambda}_i := \max(0, \lambda_i)$  represent the rectified eigenvalues, and  $\overline{\Lambda} := \mathrm{diag}(\overline{\lambda}_i)$ . Then, define
$$
\bar {\Sigma} _ {X Y | M} := V \bar {\Lambda} V ^ {\intercal}, \tag {12}
$$
$$
\Sigma_ {X, Y \mid M} ^ {\text {p r o j}} := \bar {\Sigma} _ {X \mid M} ^ {- 1 / 2} \bar {\Sigma} _ {X, Y \mid M} \bar {\Sigma} _ {Y \mid M} ^ {- 1 / 2}, \tag {13}
$$
where  $\overline{\Sigma}_{X|M}$ ,  $\overline{\Sigma}_{Y|M}$  and  $\overline{\Sigma}_{X,Y|M}$  are submatrices of  $\overline{\Sigma}_{XY|M}$ .
4 Canonical Gaussian Examples
In this section, we show how well our  $\sim_G$ -PID estimator performs on a series of Gaussian examples of increasing complexity, which have known ground truth. Barrett [13] showed that, for Gaussian distributions, the  $\sim$ -PID reduces to the MMI-PID (defined below), whenever  $M$  is scalar. These also happen to be cases when the optimal distribution  $Q_{MXY}$  is Gaussian [12], and thus the  $\sim_G$ -PID should recover the ground truth. We then leverage additivity (Property 1) to combine two or more simple examples into complex ones, where ground truth continues to be known.
Definition 3 (Minimum Mutual Information (MMI) PID). Let the redundant information be defined as the minimum of the two mutual informations:
$$
R I _ {M M I} (M: X; Y) = \min  \left\{I (M; X), I (M; Y) \right\}. \tag {14}
$$
The remaining MMI-PID components,  $UI_{MMI}(M : X \setminus Y)$ ,  $UI_{MMI}(M : Y \setminus X)$  and  $SI_{MMI}(M : X; Y)$ , follow from equations (1)-(3).
We first provide a Gaussian analog of Example 1 in Examples 2-4 (for  $d_M = d_X = d_Y = 1$ ). We will use the channel notation described in Equation (7). Complete derivations for these examples (and some nuances that are omitted here) are presented in App. B.
Example 2 (Pure uniqueness: variable  $A$  from Example 1). Suppose  $M \sim \mathcal{N}(0,1)$ ,  $H_{X} = 1$  and  $H_{Y} = 0$ , with  $N_{X}, N_{Y} \sim$  i.i.d.  $\mathcal{N}(0,1)$ . Here, only  $X$  receives information about  $M$ , while  $Y$  is pure noise. Thus,  $X$  has unique information about  $M$  ( $UI_{X} = I(M;X) > 0$ ), with no unique information in  $Y$ , and no redundancy or synergy ( $UI_{Y} = RI = SI = 0$ ).
Example 3 (Pure redundancy: variable  $B$  from Example 1). Ideally, we would set  $M \sim \mathcal{N}(0,1)$ ,  $X = M$  and  $Y = M$ . However, for continuous random variables,  $I(M;X) = \infty$  when  $M = X$ . So instead, we set  $M \sim \mathcal{N}(0,1)$ ,  $H_{X} = 1$  and  $H_{Y} = 1$ , with  $N_{X} \sim \mathcal{N}(0,1)$  while  $N_{Y} = N_{X}$  (i.e.,  $X = Y$ , so they are both the same noisy version of  $M$ ). In this case,  $X$  and  $Y$  are fully redundant since they both contain exactly the same information about  $M$ . Thus,  $RI = I(M; (X,Y)) > 0$ , while  $UI_{X} = UI_{Y} = SI = 0$ .
Example 4 (Pure synergy: variable  $C$  from Example 1). We cannot replicate pure synergy for Gaussian variables, but we can approach it in a limit. Let  $M \sim \mathcal{N}(0,1)$ ,  $H_{X} = 1$  and  $H_{Y} = 0$ , with  $N_{X} \sim \mathcal{N}(0,\sigma^{2})$  and  $N_{Y} = N_{X}$  (i.e.,  $X = M + Y$ ). Further, let  $\sigma^2 \to \infty$ . In this case,  $I(M;Y) = 0$
PIDs for Canonical Gaussian Examples
Figure 1: PID values for Examples 5, 6 and 7. The  $\sim_{G}$ -PID and the  $\delta_{G}$ -PID agree exactly with the MMI-PID, which is known to be the ground truth, since  $M$  is scalar [13].
Figure 2: Left and right panels respectively show PID values for Examples 8 and 9, which combine two scalar examples with known ground truth, using Property 1. The middle panel shows the absolute error between each PID definition and the ground truth. The  $\sim_{G}$ -PID diverges from the  $\delta_{G}$ - and MMI-PIDs, and is the only one that agrees with the ground truth. The  $\sim_{G}$ -PID maintains an error less than  $10^{-7}$  bits, whereas other definitions have errors greater than 0.1 bits.
and  $I(M;X) \to 0$  as  $\sigma^2 \to \infty$ , so  $X$  and  $Y$  individually convey little to no information about  $M$ . However, we can recover information about  $M$  from  $X$  and  $Y$  together by taking their difference, since  $X - Y = M$ . Thus,  $SI > 0$ , while  $UI_{Y} = RI = 0$  and  $UI_{X} \to 0$ .