# Functional Connectome of the Human Brain with Total Correlation
## 2 Total Correlation as Neural Connectivity Descriptor  
### Definitions and Preliminaries  
**Mutual Information:** Given two multivariate random variables \(X_{1}\) and \(X_{2}\), the mutual information between them, \(\mathrm{I}(X_{1};X_{2})\), can be calculated as the difference between the sum of individual entropies, \(H(X_{i})\), and the entropy of the variables considered jointly as a single system, \(H(X_{1},X_{2})\)[34]:  
\[\mathrm{I}(X_{1};X_{2})=\mathrm{H}(X_{1})+\mathrm{H}(X_{2})-\mathrm{H}(X_{1},X_{ 2}) \tag{1}\]  
where, for each (multivariate) random variable \(\
u\), the entropy is \(\mathrm{H}(\
u)=\langle-\log_{2}\mathrm{p}(\
u)\rangle\) and the brackets represent expectation values spanning random variables. The mutual information also can be seen as the information shared by the two variables or the reduction of uncertainty in one variable given the information about the other [35]. **Mutual information is better than linear correlation:** For Gaussian sources, mutual information reduces to linear correlation because the entropy factors in Equation (1) just depend on \(|\langle X_{1}\cdot X_{2}^{\top}\rangle|\). However, for more general (non-Gaussian) sources, mutual information cannot be reduced to covariance and cross-covariance matrices. In these (more realistic) situations, I is better than the linear correlation because I captures nonlinear relations that are ruled out by \(|\langle X_{1}\cdot X_{2}^{\top}\rangle|\). For an illustration of the qualitative differences between I and linear correlation, see the examples in Section 2.2 of [24]. As a result, mutual information has been proposed as a good alternative to linear correlation for estimating functional connectivity [8; 21]. However, mutual information cannot capture dependencies beyond pairs of nodes. This may be a limitation in complex networks [36]. **Total Correlation:** This magnitude describes the dependence among \(n\) variables, and it is a generalization of the mutual information concept from two parties to \(n\) parties. The Venn diagram in Figure 1 qualitatively illustrates this for three variables. The definition of Total Correlation from [PERSON] [26] can be denoted as:  
\[\mathrm{TC}(\mathrm{X}_{1},\ldots,\mathrm{X}_{n})\equiv\sum_{i=1}^{n}\mathrm{ H}(\mathrm{X}_{i})-\mathrm{H}(\mathrm{X}_{1},\ldots,\mathrm{X}_{n})=\mathrm{D}_{ \mathrm{KL}}\Bigg{(}\mathrm{p}(\mathrm{X}_{1},\ldots,\mathrm{X}_{n})\|\prod _{i=1}^{n}\mathrm{p}(\mathrm{X}_{i})\Bigg{)} \tag{2}\]  
where \(\mathrm{X}\equiv(\mathrm{X}_{1},\ldots,\mathrm{X}_{n})\) and \(\mathrm{TC}\) can also be expressed as the Kullback-Leibler divergence, \(D_{KL}\), between the joint probability density and the product of the marginal densities. From these definitions, if all variables are independent, then \(\mathrm{TC}\) will be zero. For the conditional Total Correlation, which is similar to the definition of Total Correlation, but with a condition appended to each term, the Kullback-Leibler divergence of the two conditional probability distributions can also be used to define the conditional Total Correlation. The estimation method used in this work (CorEx presented in the next  
Figure 1: Conceptual scheme of information-theoretic measures of neural information flow. The left circle areas represent the amounts of information, and intersections represent shared information among the corresponding variables, \(X_{0},X_{1},X_{2}\). Examples of entropy, \(H(X_{0}),H(X_{1}),H(X_{2})\), Total Correlation (red color), and \(TC[X_{0},X_{1},X_{2}]\) are given. The middle figures show some neural time series extracted from brain regions, which correspond to the nodes in the right figure. The right figures illustrate large-scale time series in the brain and how the coupled information is transmitted among the brain regions. The blue and green lines show Linear Correlation (CC) and Mutual Information (I), respectively, between different parts of the brain. The modules represent the lobes of the human brain. Each module has specific brain regions, and each module works with the others.