Learning Robust Hierarchical Patterns of Human Brain across Many fMRI Studies

1 Introduction . Multi-site fMRI studies have gained a lot of interest over the last decade [ 1 , 2 ] . One reason for this is the necessity to evaluate a hypothesis in multiple settings/sites and make the hypothesis result generalizable to a diverse population . Also , the pooling of data is essential when studying rare disorders or neurological conditions where the aim is to generalize the results to diverse populations [ 3 , 4 ] . However , the data pooling often results in the introduction of non-biological systematic variance due to differences in scanner hardware and imaging acquisition parameters [ 5 ] . This additional variability can lead to spurious results and a decrease in statistical power . The variability can also result in hindrance in the estimation of true biological changes or in inferring non-biological differences as biological because of the correlation between site effects and biological predictors . Many studies working with multi-site data fMRI have reported considerable variability due site or scanner effects [ 6 , 7 , 1 ] . The non-biological variability introduced due to inter-scanner and inter-protocol variations can affect the estimation of the common features derived from fMRI [ 8 ] , such as functional connectivity [ 5 ] or sparse hierarchical factors ( this paper ) . These features were used for the study of the brain ’ s function during aging [ 9 ] , of various neurological disorders [ 10 , 11 ] , and tasks [ 12 ] . The non-biological variability can considerably reduce these features ’ reproducibility across different datasets and hence their utility as biomarkers for diseases that disrupt functional connectivity . Thus the removal of non-biological variance introduced by pooling of the data is essential for many neuroimaging studies . Independent Component Analysis [ 13 ] , Non Negative Matrix Factorization [ 14 ] , Sparse Dictionary Learning [ 15 ] and Sparse factor Modeling [ 16 , 17 ] are some of the commonly used methods for estimating interpretable components that capture the complexity of the human brain function at rest . However , these methods do not account for the hierarchical organization of the human brain [ 18 , 19 ] . Recently , deep learning based methods [ 20 , 21 ] have been developed to estimate hierarchical networks . However , they fail to capture overlapping components and the existence of positively and negatively correlated nodes in a component that are shown to capture the underlying structure better [ 22 , 23 ] . Hierarchical Sparse Connectivity Patterns ( hSCPs ) overcome these limitations by 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) capturing interpretable hierarchical , sparse , and overlapping components using a linear deep matrix factorization approach . The method not only captures shared patterns but also the subject-specific weights of these patterns , thus capturing the heterogeneity of brain activity patterns across individuals . In this paper , we focus on robust estimation of hSCPs [ 23–25 ] in a multi-site regime . Many existing methods to reduce site effects are based on an empirical Bayes method ComBat [ 26 ] , which was developed to remove ‘ batch effects ’ in genetics and has been applied for harmonizing different measures derived from structural [ 27 , 28 ] and functional MRI [ 8 ] . But these harmonization methods can not be directly applied to the hSCP method because of the loss in the structure of captured heterogeneity ( more details in section 2.1 ) . See Appendix A for extended discussion on related work . Contribution . In this paper , we develop a new model that is robust to site-effects in the estimation of sparse hierarchical connectivity pattern components ( rshSCP ) . For this , the method learns site-specific features and global space , storing the information about the scanner and site , and uses these features to reduced site effects in the components . We also use an existing adversarial learning approach [ 25 ] on top of our method to improve the reproducibility and generalizability of the components across components from the same site . We formulate the method as a non-convex optimization problem which is solved using stochastic gradient descent . Experiments on simulated and real datasets show that our method can improve the split-sample and leave one site reproducibility of the components while retaining age-related biological variability in the data , thus capturing informative heterogeneity . 2 Method . In this section , we first present a brief overview of the hierarchical Sparse Connectivity Pattern ( hSCP ) and its adversarial formulation . We then describe our main method and the joint formulation incorporating adversarial learning . We follow the notation in [ 24 ] and [ 25 ] . The set of symmetric positive definite matrices of size P × P is denoted by SP×P++ . Matrix A with all the elements greater than or equal to 0 is denoted by A ≥ 0 . JP denotes P × P matrix with all elements equal to one . P × P identity matrix is denoted by IP and element-wise product between two matrices A and B is denoted by A ◦B . 2.1 Introduction to hierarchical Sparse Connectivity Patterns . Hierarchical Sparse Connectivity Patterns ( hSCP ) [ 24 ] is a hierarchical extension of Sparse Connectivity Patterns ( SCPs ) , first defined by [ 23 ] to estimate sparse functional patterns in the human brain using fMRI data . Let there be N number of subjects or participants , and each subject ’ s BOLD fMRI time series has T time points and P nodes representing regions of interest . The input to hSCP are correlation matrices Θn ∈ SP×P++ where ith and jth element of the matrix is the correlation between time series of ith and jth node . hSCP then outputs a set of shared hierarchical patterns following the below equations : Θn ≈W1Λn1W > 1 , . . . Θn ≈W1W2 . . .WKΛnKW > KW > K−1 . . .W > 1 , where Λnk is a diagonal matrix having positive elements storing relative contribution of the components for the nth subject at kth level , K is the depth of hierarchy and P > k1 > . . . > kK i.e . each successive level in the hierarchy has less number of components than the previous one . In the above formulation , W1 ∈ RP×k1 stores k1 components at the bottom most level , and each successive multiplication by W2 , W3 , . . . , WK linearly transforms to a lower dimensional space of k2 , k3 , . . . , kK dimension . LetW = { Wr | r = 1 , . . . , K } be the set storing sparse components shared across all subjects and D = { Λnr | r = 1 , . . . , K ; n = 1 . . . , N } be set storing subject specific diagonal matrix with Λnr ≥ 0 . The hierarchical components are estimated by solving the below optimization problem : min W , D H ( W , D , C ) = N∑ n=1 K∑ r=1 ‖Θn − ( r∏ j=1 Wj ) Λ n r ( r∏ j=1 Wj ) > ‖2F s.t . ‖wrl ‖1 < τr , ‖wrl ‖∞ ≤ 1 , trace ( Λnr ) = 1 , Λnr ≥ 0 , ∀n , r , l ; Wj ≥ 0 , j = 2 , . . . , K , ( 1 ) where C = { Θn | n = 1 , . . . , N } , l = 1 , . . . , kr , n = 1 , . . . , N and r = 1 , . . . , K. L1 , L∞ and trace constraints help the problem to identify a decomposition which can provide reproducible components . More details can be found in the original paper [ 24 ] . We will be denoting above constraint set as ΩW = { W | ‖wrl ‖1 < τr , ‖wrl ‖∞ ≤ 1 , Wj ≥ 0 , j = 2 , . . . , K } and Ψ = { Λ | trace ( Λnr ) = 1 , Λ n r ≥ 0 } . A detailed description of the method can be found in [ 24 ] . Can we use standard harmonization approaches ? These methods reduce site effects by adjusting for additive and multiplicative effects for each feature in data separately and use emperical Bayes estimates the model parameters . These methods can be used in the case of hSCP in two ways . First , harmonization can be directly applied to each element of the correlation matrices , which is the input of hSCP . This will reduce site effects from each element of the correlation matrix , thus from the complete input , but the final matrix that does not necessarily follow the essential property of a correlation matrix i.e. , positive definiteness . For similar reasons , COMBAT can not be directly applied to time series ; if applied , it can change the inference derived from the correlation matrix . Second , harmonization can be directly applied to Λ to remove site effects . To understand this , we look at the hSCP formulation at one level : Θn ≈ k∑ l=1 dnl wlw > l ≈WΛnW > , where dnl are non-zero elements storing the subject-specific information , which can be affected by the variability introduced by the site . In this model , harmonizing each feature across different sites will change the relative contribution of the components in each subject ’ s functional structure , which will remove the interpretability of the subject-wise weights , which is not desirable . Instead , a two step optimization procedure can be used to incorporate ComBat with hSCP ( ComBat hSCP ) . We first run hSCP and use ComBat on the extracted Λn to get harmonized subject specific information ∆n ∈ Rk1×k1 for each subject . We then re-fitted W using the below decomposition- Θn ≈W ( ∆n + S ) W > . We added a diagonal shift matrix S ∈ Rk1×k1 such that ∆n + S is positive for each subject and performed the optimization to estimate W and S. We show through experiments that this baseline two step optimization procedure is not optimal and performs worse than vanilla hSCP . 2.2 Adversarial Learning in hSCP . Adversarial learning has shown to achieve state-of-the-art performance of various matrix factorization approaches [ 29 , 30 ] . Recently , Sahoo and Davatzikos [ 25 ] demonstrated that incorporating adversarial learning in the estimation problem of hSCP can improve the reproducibility of the hierarchical components . The method is based on perturbation of input data Θn to learn stable components robust to adversaries . First the input data is perturbed to generate new data Γn = Θn + σJP where σ is the standard deviation of the data . We consider this perturbation as rank one perturbation which will transform eigenvalues of each subject ’ s correlation matrix differently . Since the addition of ones does not change the symmetric property of the matrix , we just scale the matrix such that the diagonal matrix contains 1 and by Weyl ’ s inequality about perturbation [ 31 ] it will be positive definite . We have experimented with randomly positively scaled rank one perturbation , but the results were worse than the vanilla hSCP . Also , using rank one perturbation , it is easier to control eigenvalues ( and noise added ) of the modified matrix than using rank k perturbation . The perturbed set of components W̃1 are then estimated using the new data and are ensured to be close to W1 by solving the below minimization problem : A ( Ŵ1 ) = α‖W̃1 −W1‖2F + N∑ n=1 ‖Γn − W̃1ΛnW̃ > 1 ‖2F . ( 2 ) In the above optimization problem , the first term constraints W̃1 to be close to W1 and second term is used for learning the components using the perturbed data . Aim of the learner is to estimate Λn and W1 by minimizing the below cost function : D ( W , Λ ) = N∑ n=1 ‖Θn − W̃1ΛnW̃ > 1 ‖2F + β N∑ n=1 ‖Θn −W1ΛnW > 1 ‖2F , ( 3 ) for a fixed W̃1 . Learner first estimates subject specific information Λn using perturbed weight matrix and then use it to learn W1 . Below is the complete optimization problem for adversarial learning of hSCP : min W , D H ( W̃ , D , C ) + βH ( W , D , C ) s.t . W̃r = arg min Ŵr α‖Ŵr −Wr‖2F +H ( W̃ , D , P ) r = 1 , . . . , K W̃r , Wr ∈ Ω , D ∈ Ψ , ( 4 ) where W̃ = { W̃r | r = 1 , . . . , K } and P = { Γn | n = 1 , . . . , N } . We next discuss our main method which aims to reduce the site effects .

Summary: This paper proposes a method called rshSCP (robust-to-site hSCP) to estimate hierarchical sparse connectivity patterns (hSCPs) from multi-site fMRI data while reducing the impact of site-specific variation due to scanner and experiment details rather than biological effects. Their approach is applied on top of the previous hSCP method as well as an extension that uses adversarial learning to improve reproducibility of estimated patterns (Adv. hSCP). The proposed method includes an additive component to the decomposition of the correlation matrix in hSCP, which models site effects as a matrix factorization with diagonal site-specific and full shared (across-site) matrix components. The method then additionally applies an adversarial classifier (a two hidden-layer feed-forward neural network) to predict the site from the subject-specific components of the hSCP decomposition. The entire optimization is formulated as a minimax problem, where the classifier maximizes its ability to predict site information while the remaining parameters are optimized to reconstruct the correlation matrices while minimizing the classifier's accuracy. The authors also propose an SVD-based initialization which improves reproducibility of the estimated components. Experiments on simulated data as well as fMRI data from multi-center imaging studies show that rshSCP improves accuracy (on simulated data) as well as reproducibility (on simulated and real data) of the estimated components, both when applied with hSCP and Adv. hSCP. The proposed method also leads to a reduction in accuracy when predicting site from the estimated subject-specific components using an SVM, as desired. Predicting age from the estimated subject-specific components with a random forest and visualizing the shared components shows that rshSCP preserves relevant biologically-related variation.