# Functional Connectome of the Human Brain with Total Correlation
## 4 Experiment 2: Clustering by Total Correlation for Dependent and Independent Mixtures  
To evaluate the performance of CorEx in clustering tasks. The elements in group \(X\) include \(X1\), \(X2\), and \(X3\), which satisfy Gaussian distributions and are completely independent of each other and of group \(Y\), and the variables in group \(Y\) include \(Y1\), \(Y2\) from \(Y1\)  
Figure 3: The estimated Total Correlation values for three independent variables. The various Total Correlation estimators are compared with the ground truth value (red line), for example matrix-based Rényi entropy (black line), Shannon discrete entropy (cyan line), RBIG (magenta line), and CorEx (green line). See the main text for more information.  
Figure 2: CorEx learns a hierarchical latent factor as illustrated above. Edge thickness indicates strength of the relationship between factors, and node thickness indicates how much Total Correlation is explained by each latent factor.  
and \(Y3\) from \(Y2\), which are connected to each other. Then, we compared the CorEx cluster results with the pairwise Pearson correlation, pairwise mutual information, and partial correlation, which consider confounding effects to find the groups.  
In Figure 4, we find that CorEx based on Total Correlation has high accuracy in estimating their dependencies (Figure 4e) compared to pairwise Pearson correlation (Figure 4b), pairwise mutual information (Figure 4c), and partial correlation (Figure 4d). As we established in this experiment, the elements in group \(Y\) should be clustered together, and the elements in group \(X\) should be completely independent of each other and of group \(Y\). The ground truth is presented in Figure 4a. Then, we estimated the cluster result with the pairwise Pearson correlation with a threshold of 0.1, pairwise mutual information with a threshold of 0.4, and partial correlation without a threshold. Obviously, we found that pairwise approaches have high errors in accurately estimating their statistical dependencies, and pairwise mutual information is better than pairwise Pearson correlation, but still has high errors in correctly clustering tasks. When we considered the confounding effect of the third variables, we still did not obtain a better clustering result compared to TC. Therefore, the clustering results with CorEx by Total Correlation obtain the best performance compared to pairwise approaches. Moreover, we used _purity_ as a criterion of clustering quality to qualify the performance of clustering because it is a straightforward and transparent evaluation metric [43]. To calculate purity, each cluster is allocated to the class that occurs most frequently within it, and the accuracy of this assignment is determined by counting the number of correctly assigned elements and dividing by \(N(N=6)\). Formally:  
\[\text{Purity}(X,Y)=\frac{1}{N}\sum_{i}\max_{j}\abs{X_{i}\cap Y_{j}} \tag{9}\]  
where \(X=\{X1,X2,X3\}\) is the set of clusters and \(Y=\{Y1,Y2,Y3\}\) is the set of classes. Figure 4f presents the clustering performance of pairwise approaches and CorEx with purity as a criterion. Poor clusters have near-zero purity ratings (lower bound). A perfect cluster possesses a purity of one (maximum value). Based on Equation (9), we obtain purity values of 0.17 and 0.33 for pairwise approaches and partial correlation, and the purity value for CorEx is 0.83. All in all, we show that CorEx based on Total Correlation has the best performance compared to pairwise approaches.  
Figure 4: Clustering performance for dependent and independent mixtures. The top row: (**a**) displays the ground truth of variable clustering in two groups. (**f**) shows the purity value of each approach. The second row: (**b**) shows the clustering result based on Pearson correlation. (**c**) shows the clustering result by pairwise mutual information. (**d**) shows the clustering result by partial correlation. (**e**) shows clustering results by CorEx based on Total Correlation.