Learning Robust Hierarchical Patterns of Human Brain across Many fMRI Studies

1 Introduction . Multi-site fMRI studies have gained a lot of interest over the last decade [ 1 , 2 ] . One reason for this is the necessity to evaluate a hypothesis in multiple settings/sites and make the hypothesis result generalizable to a diverse population . Also , the pooling of data is essential when studying rare disorders or neurological conditions where the aim is to generalize the results to diverse populations [ 3 , 4 ] . However , the data pooling often results in the introduction of non-biological systematic variance due to differences in scanner hardware and imaging acquisition parameters [ 5 ] . This additional variability can lead to spurious results and a decrease in statistical power . The variability can also result in hindrance in the estimation of true biological changes or in inferring non-biological differences as biological because of the correlation between site effects and biological predictors . Many studies working with multi-site data fMRI have reported considerable variability due site or scanner effects [ 6 , 7 , 1 ] . The non-biological variability introduced due to inter-scanner and inter-protocol variations can affect the estimation of the common features derived from fMRI [ 8 ] , such as functional connectivity [ 5 ] or sparse hierarchical factors ( this paper ) . These features were used for the study of the brain ’ s function during aging [ 9 ] , of various neurological disorders [ 10 , 11 ] , and tasks [ 12 ] . The non-biological variability can considerably reduce these features ’ reproducibility across different datasets and hence their utility as biomarkers for diseases that disrupt functional connectivity . Thus the removal of non-biological variance introduced by pooling of the data is essential for many neuroimaging studies . Independent Component Analysis [ 13 ] , Non Negative Matrix Factorization [ 14 ] , Sparse Dictionary Learning [ 15 ] and Sparse factor Modeling [ 16 , 17 ] are some of the commonly used methods for estimating interpretable components that capture the complexity of the human brain function at rest . However , these methods do not account for the hierarchical organization of the human brain [ 18 , 19 ] . Recently , deep learning based methods [ 20 , 21 ] have been developed to estimate hierarchical networks . However , they fail to capture overlapping components and the existence of positively and negatively correlated nodes in a component that are shown to capture the underlying structure better [ 22 , 23 ] . Hierarchical Sparse Connectivity Patterns ( hSCPs ) overcome these limitations by 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) capturing interpretable hierarchical , sparse , and overlapping components using a linear deep matrix factorization approach . The method not only captures shared patterns but also the subject-specific weights of these patterns , thus capturing the heterogeneity of brain activity patterns across individuals . In this paper , we focus on robust estimation of hSCPs [ 23–25 ] in a multi-site regime . Many existing methods to reduce site effects are based on an empirical Bayes method ComBat [ 26 ] , which was developed to remove ‘ batch effects ’ in genetics and has been applied for harmonizing different measures derived from structural [ 27 , 28 ] and functional MRI [ 8 ] . But these harmonization methods can not be directly applied to the hSCP method because of the loss in the structure of captured heterogeneity ( more details in section 2.1 ) . See Appendix A for extended discussion on related work . Contribution . In this paper , we develop a new model that is robust to site-effects in the estimation of sparse hierarchical connectivity pattern components ( rshSCP ) . For this , the method learns site-specific features and global space , storing the information about the scanner and site , and uses these features to reduced site effects in the components . We also use an existing adversarial learning approach [ 25 ] on top of our method to improve the reproducibility and generalizability of the components across components from the same site . We formulate the method as a non-convex optimization problem which is solved using stochastic gradient descent . Experiments on simulated and real datasets show that our method can improve the split-sample and leave one site reproducibility of the components while retaining age-related biological variability in the data , thus capturing informative heterogeneity . 2 Method . In this section , we first present a brief overview of the hierarchical Sparse Connectivity Pattern ( hSCP ) and its adversarial formulation . We then describe our main method and the joint formulation incorporating adversarial learning . We follow the notation in [ 24 ] and [ 25 ] . The set of symmetric positive definite matrices of size P × P is denoted by SP×P++ . Matrix A with all the elements greater than or equal to 0 is denoted by A ≥ 0 . JP denotes P × P matrix with all elements equal to one . P × P identity matrix is denoted by IP and element-wise product between two matrices A and B is denoted by A ◦B . 2.1 Introduction to hierarchical Sparse Connectivity Patterns . Hierarchical Sparse Connectivity Patterns ( hSCP ) [ 24 ] is a hierarchical extension of Sparse Connectivity Patterns ( SCPs ) , first defined by [ 23 ] to estimate sparse functional patterns in the human brain using fMRI data . Let there be N number of subjects or participants , and each subject ’ s BOLD fMRI time series has T time points and P nodes representing regions of interest . The input to hSCP are correlation matrices Θn ∈ SP×P++ where ith and jth element of the matrix is the correlation between time series of ith and jth node . hSCP then outputs a set of shared hierarchical patterns following the below equations : Θn ≈W1Λn1W > 1 , . . . Θn ≈W1W2 . . .WKΛnKW > KW > K−1 . . .W > 1 , where Λnk is a diagonal matrix having positive elements storing relative contribution of the components for the nth subject at kth level , K is the depth of hierarchy and P > k1 > . . . > kK i.e . each successive level in the hierarchy has less number of components than the previous one . In the above formulation , W1 ∈ RP×k1 stores k1 components at the bottom most level , and each successive multiplication by W2 , W3 , . . . , WK linearly transforms to a lower dimensional space of k2 , k3 , . . . , kK dimension . LetW = { Wr | r = 1 , . . . , K } be the set storing sparse components shared across all subjects and D = { Λnr | r = 1 , . . . , K ; n = 1 . . . , N } be set storing subject specific diagonal matrix with Λnr ≥ 0 . The hierarchical components are estimated by solving the below optimization problem : min W , D H ( W , D , C ) = N∑ n=1 K∑ r=1 ‖Θn − ( r∏ j=1 Wj ) Λ n r ( r∏ j=1 Wj ) > ‖2F s.t . ‖wrl ‖1 < τr , ‖wrl ‖∞ ≤ 1 , trace ( Λnr ) = 1 , Λnr ≥ 0 , ∀n , r , l ; Wj ≥ 0 , j = 2 , . . . , K , ( 1 ) where C = { Θn | n = 1 , . . . , N } , l = 1 , . . . , kr , n = 1 , . . . , N and r = 1 , . . . , K. L1 , L∞ and trace constraints help the problem to identify a decomposition which can provide reproducible components . More details can be found in the original paper [ 24 ] . We will be denoting above constraint set as ΩW = { W | ‖wrl ‖1 < τr , ‖wrl ‖∞ ≤ 1 , Wj ≥ 0 , j = 2 , . . . , K } and Ψ = { Λ | trace ( Λnr ) = 1 , Λ n r ≥ 0 } . A detailed description of the method can be found in [ 24 ] . Can we use standard harmonization approaches ? These methods reduce site effects by adjusting for additive and multiplicative effects for each feature in data separately and use emperical Bayes estimates the model parameters . These methods can be used in the case of hSCP in two ways . First , harmonization can be directly applied to each element of the correlation matrices , which is the input of hSCP . This will reduce site effects from each element of the correlation matrix , thus from the complete input , but the final matrix that does not necessarily follow the essential property of a correlation matrix i.e. , positive definiteness . For similar reasons , COMBAT can not be directly applied to time series ; if applied , it can change the inference derived from the correlation matrix . Second , harmonization can be directly applied to Λ to remove site effects . To understand this , we look at the hSCP formulation at one level : Θn ≈ k∑ l=1 dnl wlw > l ≈WΛnW > , where dnl are non-zero elements storing the subject-specific information , which can be affected by the variability introduced by the site . In this model , harmonizing each feature across different sites will change the relative contribution of the components in each subject ’ s functional structure , which will remove the interpretability of the subject-wise weights , which is not desirable . Instead , a two step optimization procedure can be used to incorporate ComBat with hSCP ( ComBat hSCP ) . We first run hSCP and use ComBat on the extracted Λn to get harmonized subject specific information ∆n ∈ Rk1×k1 for each subject . We then re-fitted W using the below decomposition- Θn ≈W ( ∆n + S ) W > . We added a diagonal shift matrix S ∈ Rk1×k1 such that ∆n + S is positive for each subject and performed the optimization to estimate W and S. We show through experiments that this baseline two step optimization procedure is not optimal and performs worse than vanilla hSCP . 2.2 Adversarial Learning in hSCP . Adversarial learning has shown to achieve state-of-the-art performance of various matrix factorization approaches [ 29 , 30 ] . Recently , Sahoo and Davatzikos [ 25 ] demonstrated that incorporating adversarial learning in the estimation problem of hSCP can improve the reproducibility of the hierarchical components . The method is based on perturbation of input data Θn to learn stable components robust to adversaries . First the input data is perturbed to generate new data Γn = Θn + σJP where σ is the standard deviation of the data . We consider this perturbation as rank one perturbation which will transform eigenvalues of each subject ’ s correlation matrix differently . Since the addition of ones does not change the symmetric property of the matrix , we just scale the matrix such that the diagonal matrix contains 1 and by Weyl ’ s inequality about perturbation [ 31 ] it will be positive definite . We have experimented with randomly positively scaled rank one perturbation , but the results were worse than the vanilla hSCP . Also , using rank one perturbation , it is easier to control eigenvalues ( and noise added ) of the modified matrix than using rank k perturbation . The perturbed set of components W̃1 are then estimated using the new data and are ensured to be close to W1 by solving the below minimization problem : A ( Ŵ1 ) = α‖W̃1 −W1‖2F + N∑ n=1 ‖Γn − W̃1ΛnW̃ > 1 ‖2F . ( 2 ) In the above optimization problem , the first term constraints W̃1 to be close to W1 and second term is used for learning the components using the perturbed data . Aim of the learner is to estimate Λn and W1 by minimizing the below cost function : D ( W , Λ ) = N∑ n=1 ‖Θn − W̃1ΛnW̃ > 1 ‖2F + β N∑ n=1 ‖Θn −W1ΛnW > 1 ‖2F , ( 3 ) for a fixed W̃1 . Learner first estimates subject specific information Λn using perturbed weight matrix and then use it to learn W1 . Below is the complete optimization problem for adversarial learning of hSCP : min W , D H ( W̃ , D , C ) + βH ( W , D , C ) s.t . W̃r = arg min Ŵr α‖Ŵr −Wr‖2F +H ( W̃ , D , P ) r = 1 , . . . , K W̃r , Wr ∈ Ω , D ∈ Ψ , ( 4 ) where W̃ = { W̃r | r = 1 , . . . , K } and P = { Γn | n = 1 , . . . , N } . We next discuss our main method which aims to reduce the site effects .

Summary: This paper introduces the method site-robust hierarchical sparse connectivity patterns (rshscp) and its adversarial extension. It builds on sparse connectivity patterns and its adversarial extension introduced in references 22 and 23. These propose a factorization of correlation matrices extracted for functional connectivity analysis. The factorization consists of a symmetric product of  sparse factors around a central diagonal weighting matrix. The present contribution introduces a site-dependent component into the optimization problem and adds a classifier to predict site, which is then adversarially optimized to make the learned structure lose information about site. In synthetic and real-data experiments it is shown that the method is indeed capable of discarding site information while retaining other relevant information.