# Functional Connectome of the Human Brain with Total Correlation
## 2 Total Correlation as Neural Connectivity Descriptor
### Total Correlation Estimated from CorEx  
Straightforward application of the direct definition of TC is not feasible in high-dimensional scenarios, and alternatives are required [28; 29]. A practical approach to estimate Total Correlation is via _latent factor modeling_. A latent factor model is a statistical model that relates a set of observable variables to a set of latent variables. The idea is to explicitly construct latent factors, \(Y\), that somehow capture the dependencies in the data. If we measure dependencies via Total Correlation, \(TC(X)\), then we say that the latent factors _explain_ the dependencies if \(TC(X|Y)=0\). We can measure the extent to which \(Y\) explains the correlations in \(X\) by looking at how much Total Correlation is reduced:  
\[TC(X)-TC(X|Y)=\sum_{i=1}^{n}I(X_{i};Y)-I(X;Y) \tag{5}\]  
Total Correlation is always non-negative, and the decomposition on the right in terms of mutual information can be verified directly from the definitions. Any latent factor model can be used to lower-bound Total Correlation, and the terms on the right-hand side of Equation (5) can be further lower-bounded with tractable estimators using variational methods; Variational Autoencoders (VAEs) are a popular example [38]. Although latent factor models do not give a direct Total Correlation estimation as the Rotation-based Iterative Gaussianization (RBIG) [28; 29] and the matrix-based Renyi entropy [39] did, the approach can be complementary because the construction of latent factors can help in dealing with the curse of dimensionality and for interpreting the dependencies in the data. Compared to CorEx, the main goal of (RBIG [[https://isp.uv.es/RBIG4](https://isp.uv.es/RBIG4) IT.htm]([https://isp.uv.es/RBIG4](https://isp.uv.es/RBIG4) IT.htm) (accessed on 12 October 2022)) is to convert any non-Gaussian-distributed data into a Gaussian distribution through marginal Gaussianization and rotation to obtain TC. The matrix-based Renyi entropy ([[http://www.cnel.ufl.edu/people/people.php?name=shujian](http://www.cnel.ufl.edu/people/people.php?name=shujian)]([http://www.cnel.ufl.edu/people/people.php?name=shujian](http://www.cnel.ufl.edu/people/people.php?name=shujian)) (accessed on 12 October 2022)) is mainly used for estimating multivariate information based on [PERSON]'s entropy, which is [PERSON]'s \(\alpha\)-order entropy [40]. With these goals in mind, we now describe a particular latent factor approach known as Total Correlation Explanation (CorEx ([[https://github.com/gregversteeg/CorEx](https://github.com/gregversteeg/CorEx)]([https://github.com/gregversteeg/CorEx](https://github.com/gregversteeg/CorEx))) (accessed on 12 October 2022)) [32]. CorEx constructs a factor model by reconstructing latent factors using a factorized probabilistic function of the input data, \(p(y|x)=\prod_{j=1}^{m}p(y_{j}|x)\), with \(m\) discrete latent factors, \(Y_{j}\). This function is optimized to give the tightest lower bound possible for Equation (5). \[TC(X)\geq\max_{p(Y_{j}|x)}\sum_{i=1}^{n}I(X_{i};Y)-I(X;Y)=\sum_{j=1}^{m}\left( \sum_{i=1}^{n}\alpha_{i,j}I(X_{i};Y_{j})-I(Y_{j};X)\right) \tag{6}\]  
The factorization of the latent factors leads to the terms \(I(X;Y)=\sum_{j}I(Y_{j};X)\), which can be directly calculated. The term \(I(X_{i};Y)\) is still intractable and is decomposed using the chain rule into \(I(X_{i};Y)\approx\sum\alpha_{i,j}I(X_{i};Y_{j})\). Each \(I(X_{i};Y_{j})\) can then be tractably estimated [32; 33]. There are free parameters \(\alpha_{i,j}\) that must be updated while searching for latent factors and achieving objective functions. When \(t=0\), the \(\alpha_{i,j}\) initializes and then updates according to:  
\[\alpha_{i,j}^{t+1}=(1-\lambda)\alpha_{i,j}^{t}+\lambda\alpha_{i,j}^{**} \tag{7}\]  
The second term \(\alpha_{i,j}^{**}=\exp\bigl{(}\gamma\bigl{(}I\bigl{(}X_{i}:Y_{j}\bigr{)}-\max_ {j}I\bigl{(}X_{i}:Y_{j}\bigr{)}\bigr{)}\bigr{)}\), and \(\lambda\) and \(\gamma\) are constant parameters.