Shared Independent Component Analysis for Multi-Subject Neuroimaging
Authors: Hugo Richard
Pierre Ablin
Bertrand Thirion
Alexandre Gramfort
Aapo Hyvarinen
Venue: NeurIPS 2021 Poster (2021)
Keywords: neuroimaging
fMRI
MEG
shared response modeling
component analysis
independent component analysis
multi-view learning

Abstract: We consider shared response modeling, a multi-view learning problem where one wants to identify common components from multiple datasets or views. We introduce Shared Independent Component Analysis (ShICA) that models each
view as a linear transform of shared independent components contaminated by additive Gaussian noise. We show that this model is identifiable if the components are either non-Gaussian or have enough diversity in noise variances. We then show that in some cases multi-set canonical correlation analysis can recover the correct unmixing matrices, but that even a small amount of sampling noise makes Multiset CCA fail. To solve this problem, we propose to use joint diagonalization after Multiset CCA, leading to a new approach called ShICA-J. We show via simulations that ShICA-J leads to improved results while being very fast to fit. While ShICA-J is based on second-order statistics, we further propose to leverage non-Gaussianity of the components using a maximum-likelihood method, ShICA-ML, that is both more accurate and more costly. Further, ShICA comes with a principled method for shared components estimation. Finally, we provide empirical evidence on fMRI and MEG datasets that ShICA yields more accurate estimation of the components
than alternatives.

Shared Independent Component Analysis for Multi-Subject Neuroimaging
Hugo Richard
Pierre Ablin DMA CNRS and ENS Paris, France
Bertrand Thirion
Alexandre Gramfort Inria Université Paris-
Inria
Inria
Université Paris-Saclay
Université Paris-Saclay
Palaiseau, France
Palaiseau, France
Palaiseau, France
Aapo Hyvärinen
Department of Computer Science
University of Helsinki Helsinki, Finland
Abstract
We consider shared response modeling, multi-view learning problem where one wants to identify common components from multiple datasets or views. We introduce Shared Independent Component Analysis (ShICA) that models each view as a linear transform of shared independent components contaminated by additive Gaussian noise. We show that this model is identifiable if the components are either non-Gane an-Gaussian or have enough diversity in noise variances. We then show cases multi-set canonical correlation analysis
that in unmixing matrices, but that even a small amount of sampling noise makes Multiset propose to joint diagonalization after
recover the correct
CCA fail. To solve this problem, Multiset CCA, leading to a new approach called ShICA-J. We show via simulations that ShICA-J leads to improved results while being very fast fit. While ShICA-J is based on second-order statistics, we further propose to leverage non-Gaussianity of the components using maximun-likelhhodd method, ShICA-ML, that is both more accurate and more costly. Further, ShICA comes with principled method for shared components estimation. Finally, we provide empirical evidence on fMRI and MEG datasets that ShICA yields more accurate estimation of the components than alternatives.
Introduction
In many data science problems, data are available through different views. Generally, the views represent different measurement modalities such as audio and video, or the same text that may be available in different languages. Our main interest here is neuroimaging where recordings from multiple subjects. In particular, it is of interest to find common patterns or responses that are shared between subjects when they receive the same stimulation or perform the same cognitive task 16 51]
made
popular line of work to perform such shared response modeling is group Independent Component Analysis (ICA) methods. The fastest methods [14 are among the most popular, yet they are not grounded on principled probabilistic models for the multiview setting. More principled approaches exist [5]] 27 but they do not model subject-specific deviations from the shared response. However, such deviations expected in most neuroimaging settings, the magnitude of the response may differ from subject to subject 46), as may any noise due to heartheats, respiratory artefacts or head
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

movements [39]. Furthermore, most GroupICA methods are typically unable to separate components whose density close to a Gaussian.
Independent vector analysis (IVA) [6 5 is powerful framework where components are independent within views but each component of given view depend the corresponding component in other views. However, current implementations such as IVA-L [36], IVA-G [5], IVA-L-SOS ([]] IVA-GGD
IVA with Kotz distribution [6] estimate only the view-specific components, and do not model or extract a shared response which is the main focus in this work.
On the other hand, the shared response model [16] is popular approach to perform shared response modeling, yet it imposes orthogonality constrains that are restrictive and not biologically plausible.
In this work we introduce Shared ICA (ShICA), where each view is modeled as a linear transform of shared independent components contaminated by additive Gaussian noise. ShICA allows the principled extraction of the shared components (or responses) in addition to view-specific components. Since is based on a statistically sound noise model, it enables optimal inference (minimum mean square error, MMSE) of the shared responses.
Let us note that ShICA is longer the method of choice when the concept of common response is either not useful or not applicable. Nevertheless, we believe that the ability to extract common response an important feature in most contexts because highlights stereotypical brain response to stimulus. Moreover, finding commonality between subjects reduces often unwanted inter-subject variability.
The paper is organized as follows. We first analyse the theoretical properties of the ShICA model, before providing inference algorithms. We exhibit necessary and sufficient conditions for the ShICA model to be identifiable (previous work only shows local identifiability (7). in the presence of Gaussian or non-Gaussian components. We then use Multiset CCA to fit the model when all the components are assumed to be Gaussian. We exhibit necessary and sufficient conditions for Multiset CCA to be able recover the unmixing matrices (previous work only gives sufficient conditions (38). In addition, we provide instances of the problem where Multiset CCA cannot recover the mixing matrices while the model is identifiable. We next point out practical problem even small sampling noise can lead to large error in the estimation of unmixing matrices when Multiset CCA used. To address this issue and recover the correct unmixing matrices, we propose to apply joint diagonalization to the result of Multiset CCA yielding new method called ShICA-J. We further introduce ShICA-MI, a maximum likelihood estimator of ShICA that models non-Gaue non-Gaussian components using Gaussian mixture model. While ShICA-ML yields more accurate components, ShICA-J is significantly faster and offers great initialization to ShICA-ML. Experiments on fMRI and MEG data demonstrate that the method outperforms existing GroupICA and IVA methods.
2 Shared ICA (ShICA): an identifiable multi-view model
Notation We write vectors in bold letter v and scalars in lower case a. Upper case letters M are used denote matrices. We denote M the absolute value of the determinant of M. x ~ NN E) means that x I² follows multivariate normal distribution of mean μ E IR and covariance E R The j.j entry of a diagonal matrix Σ, is denoted Eij. the entry of y: is denoted Yij- Lastly, 8 is the Kronecker delta.
Model Definition In the following, X1 XI,...,X,, Xm E RR denote the m observed random vectors obtained from the m different views. We posit the following generative model, called Shared ICA (ShICA): f
x, As
(1)
where TRP contains the latent variables called shared components, A1,..., Am Rpxp the invertible mixing matrices, and n, E TR are individual noises. The individual noises model both the deviations of a view from the mean -i.e. individual differences and measurement noise. Importantly, we explicitly model both the shared components and the individual differences in probabilistic framework enable an optimal inference of the parameters and the responses.
We assume that the shared components are statistically independent, and that the individual noises Gaussian and independent from the shared components: p(s) = IT p(sy) and n, (0, E,). where the matrices E; are assumed diagonal and positive. Without loss of generality, components

---

are assumed to have unit variance E(ss" = Ig. We further assume that there are at least 3 views: m 2-
In contrast to almost all existing works, we assume that some components (possibly all of them) may be Gaussian, and denote G the set of Gaussian components: 8j (0,,1) for G. The other components are non-Gaussian: for & 9, Sy is non-Gaussian.
Identifiability The parameters of the model θ (A1, m. E1y...; Em)) We are inter- ested in the identifiability of this model: given observations Xi...., Xm generated with parameters O, are there some other O' that may generate the same observations? Let us consider the following assumption that requires that the individual noises for Gaussian components are sufficiently diverse:
Assumption (Noise diversity in Gaussian components). For all j:j' E G.j + j', the sequences (Et))== and )=1.))= different where Eig is the j.j entry of E4
It is readily seen that there is one trivial set of indeterminacies in the problem: if P E RPxP is sign and permutation matrix (i.e. a matrix which has one coefficient on each row and column, and 0's elsewhere) the parameters [AIP.... Am P. ET P..... P Em P) also generate X1,. Xm- The following theorem shows that under the above assumption, these are the only indeterminacies of the problem.
Theorem 1 (Identifiabilityy. We make Assumption We let θ' = -AA,.... A' Ey..... )))) another set of parameters, and assume that they also generate x1...., ,,m- Then, there exists sign and permutation matrix P such that for all i. A/ A,P, and Σ = P.PP
The proof is in Appendix A. Identifiability
Gaussian case is a consequence of the identifiability results in [59 and in the general case, local identifiability results can be derived from the work of 77] However local identifiability only shows that for given set of parameters there exists neighborhood in which other set of parameters can generate the same observations [52 In contrast, the proof of
Theorem shows global identifiability.
Theorem shows that the task of recovering the parameters from the observations is well-posed problem, under the sufficient condition of Assumption [ We also note that Assumption i necessary for identifiability. For instance, if and j' are two Gaussian components such that Dij Eny for all i, then global rotation of the components j.j' yields the same covariance matrices. The current work assumes 2 3, in appendix B we give an identifiability result for 2, under stronger conditions.
Estimation of components with noise diversity via joint-diaggnallzation
We now consider the computational problem of efficient parameter inference. This section considers components with noise diversity, while the next section deals with non-Gaussian components.
3.1 Parameter estimation with Multiset CCA
If assume that the components are all Gaussian, the covariance of the observations given by
Cjj = [ A. p + 6G Σ, A are sufficient statistics and methods using only second order information, like Multiset CCA, are candidates to estimate the parameters of the model. Consider the matrix R containing m x blocks of size p x such that the block i, is given by Cyz. Consider the matrix D identical C excepts that the non-diagonal blocks are filled with zeros:
Cim
C
C
D
:
(2)
Cml
Crem
Omm
Generalized CCA consists of the following generalized eigenvalue problem:
Cu XDu,
A > 0. E Rm
(3)
Consider the matrix = [u' ....,... u²] E mpp formed by concatenating the p leading eigenvectors of the previous problem ranked in decreasing eigenvalue order. Then, consider U to be formed of m

---

blocks of size stacked vertically and define (W*)T to be the i-th block. These m matrices are the output of Multiset CCA. We also denote A - ... - Ap the p leading eigenvalues of the problem.
An application of the results of (38 shows that Multiset CCA recovers the mixing matrices of ShICA under some assumptions.
Proposition 1 (Sufficient condition for solving ShICA via Multiset CCA (8]). Let Tijk (1 $. Assume that (Fajk)) is non-increasing. Assume that the maximum eigenvalue Vy of matrix R( of general element (Taske) is such that Vv = Axp- Assume that x1...Ap are distinct. Then, there exists scale matrices I such that W. = T;A,' for all i.
= + =,
This proposition gives a sufficient condition for solving ShICA with Multiset CCA. It needs particular structure for the noise covariances as well as specific ordering for the eigenvalues. The next theorem shows that only need Åι ...Ap A to be distinct for Multiset CCA to solve ShICA:
Assumption 2 (Unique eigenvalues). Ap.A.pp are distinct.
Theorem 2. We only make Assumption [ Then, there exists permutation matrix P and scale matrices T3 such that W; = AT, for all
The proof is in Appendix A.2 This theorem means that solving the generalized eigenvalue problem 3 allows to recover the mixing matrices up to a scaling and permutation: this form of generalized CCA recovers the parameters of the statistical model. Note that Assumption 2 is also necessary condition. Indeed, if two eigenvalues are identical, the eigenvalue problem is not uniquely determined.
We have two different Assumptions, and 2 the first of which guarantees theoretical identifiability as per Theorem D and the second guarantees consistent estimation by Multiset CCA as per Theorem Next we will discuss their connections, and show some limitations of the Multiset CCA approach. To begin with, we have the following result about the eigenvalues of the problem
(3) and the Σij.
Proposition 2. For j S p, let XÚ the largest solution of Ex (114 1. Then, X Ai...., are the p largest eigenvalues of problem
It is easy to see that we then have Åy..... Xp greater than 1, while the remaining cigenvalues are lower than 1. From this proposition, two things appear clearly. First, Assumption 2 implies Assumption [ Indeed, if the Xy 's are distinct, then the sequences (Eij): must also be different from the previous proposition. This is expected as from Theorem 2 Assumption [ implies identifiability, which in turn implies Assumption
Prop. also allows derive cases where Assumption I holds but not Assumption 2 The following Proposition gives a simple case where the model is identifiable but cannot be solved using Multiset CCA:
Proposition 3.
Assume that for two integers j.J', Jr, the sequence (Eig): ) is permutation of (Eay) ) Lee. that there exists a permutation of (1....,P). n, such that for all i, Eij- = Em(O)). Then, >3 = Ayr.
In this setting, Assumption [holds so ShICA is identifiable, while A does does not hold, so Multiset CCA cannot recover the unmixing matrices.
3.2 Sampling noise and improved estimation with joint diagonalization
The consistency theory for Multiset CCA developed above is conducted under the assumption that the covariances Cij are the true covariances of the model, and not approximations obtained from observed samples. practice, however, a serious limitation of Multiset CCA is that even a slight error of estimation on the covariances, due to "sampling noise", can yield large error in the estimation of the unmixing matrices, as will be shown next.
We begin with an empirical illustration. We take m = 3, p 2, and Σ; such that A 2+ and A2 2 for 8 > O. In this way, we can control the eigen-gap of the problem, E. We take W; the outputs of Multiset CCA applied to the true covariances Ciz. Then, we generate a perturbation A = 8- S, where is a random positive symmetric pm pm matrix of norm 1, and 8 > 0 controls the scale of the pertur- bation. We take Ai the p x p block of A in position (i. j). and W; the output of Multiset CCA applied to the covariances Ci + Aij. We finally compute the sum of the Amari distance between the W; and We the Amari distance measures how close the two matrices are, up to scale and permutation (4)

---

displays the median Amari distance over 100 random repetitions, as the perturbation scale increases. The different curves correspond to different values of the eigen- gap s. We see clearly that the robustness of Multiset CCA critically depends on the eigen-gap, and when it is small, even a small perturbation of the input (due, for instance, to sampling noise) leads to large estimation errors.
10-1
Amari distance
10
10
10-7
10
10
10-10
10
This problem is very general and well studied [53]: the mapping from matrices to (generalized) eigenvectors is highly non-smooth. However, the gist of our method is that the span of the leading p eigenvectors is smooth, long as there is large enough gap between Ap and Ap+1. For our specific problem we have the following bounds, derived from Prop.2 2
10 10-4
10
100
Perturbation scale
Figure 1: Amari distance between true mixing matrices and estimates of Multi- set CCA when covariances are perturbed. Different solid curves correspond dif- ferent eigen- gaps. The black dotted line shows the chance level. When the gap is small, small perturbation can lead to complete mixing. doint-diagonalization (colored dotted lines) fixes the problem.
Proposition 4. We let maax maxij Zij and amin minsy Eng- Then, Ap 2 1+ 140max'
m-1 while Apt1 1
1+0
As consequence, we have Ap
Ap+1
140min I+omax the gap between these cigenvalues increases with m, and decreases with the noise power.
In this setting, when the magnitude of the perturbation is smaller than Ap - Ap+1. 153 indicates
that Span(W Wm [ Span(W,,... Wm. [m]T), where [W,,... W., E R is the
vertical concatenation of the W,'s. s. turn, this shows that there exists a matrix Q E Roxp such that
Wi QW, for all
We propose to joint-iiagonalizztion to recover the matrix Q. Given the WP', con- sider the of symmetric matrices K; W.C.W.WT where Cú is the contaminated covari- ance of x,. Following Eq. 44. we have QK,Q" QK,QT = PT,A, T.PT. Since Č₁ is close to Cn = Ai + E.)AT the matrix almost diagonal. In other words, the matrix Q is an approximate diago- nalizer of the K; ss, and we approximate Q by diagonalaaaninnoionnn of the K, 's In Fig | we see that this procedure mitigates the problems of Calltiset-CCAA, and gets uniformly better performance regardless of the eigen- gap. In practice, we use fast joint-diagonalization algorithm [ to minimize a joint-diagonalization criterion for positive symmetric matrices
W,C,W,T. and using Theorem 2 have
CTA,
48 The estimated unmixing
matrices U, QW, correspond to the true unmixing matrices only up some scaling which may be different from subject to subject: the information that the components are of unit variance is lost. As a consequence, naive averaging of the recovered components may lead inconsistant estimation. We now describe a procedure to recover the correct scale of the individual components across subjects.
Scale estimation We form the matrices Tej U.C.,UT. In order to estimate the scalings, we solve min Eijs [ (Tt)) Il where the &; are diagonal matrices. This function is readily min- imized with respect to of the +; by the formula
Algorithm 1 ShICA-J
Input: Covariances Cif = E[x,x]]
(W:): + Mu(tise)CCA((Cuuaa)
4, Eies 4 diag((.,))
Eins (Ydiag(Y,))* (derivations in Appendix 20 We then iterate the previous formula over until con- vergence. The final estimates of the unmixing trices are given by (6,U,)?11 The full procedure, called ShICA-J, is summarized in Algorithm
Q - (W..Diigg(WWW,,.....
Tú < QW.C. WT WTQQ
(#e); + Scaling(( (Tann)((TT))))
Return: Unmixing matrices ($:QW)),
3.3 Estimation of noise covariances
practice, it is important to estimate noise covari- ances E; in order to take advantage of the fact that
5

---

some views are noisier than others. As it is well known in classical factor analysis, modelling noise variances allows the model virtually discard variables, or subjects, that are particularly noisy.
Using the ShICA model with Gaussian components, we derive an estimate for the noise covariances directly from maximum likelihood. We use especially fast because noise updates are in closed-form. Following derivations given in appendix D.I the sufficient statistics in the E-step are given by
oppectaioonmmaxiiizziion (EM) algorithm, which is
m
-1
m
m
E[s|x]
Σ
E y:)
V
1)-1
(5)
Incorporating the M-step we get the following updates that only depend on the covariance matrices:
Σ, < diag(C; (Cts - 2V/ss|x شØ= b C3+ + V[s دリ= Σx= (1 C
V(s/x] + V[s|x)
4 ShICA-ML: Maximum likelihood for non- Gau-Gaussian components
ShICA-J only uses second order statistics. However, the ShICA model (I) allows for non- non-Gaussian components. We now propose an algorithm for fitting the ShICA model that combines covariance information with non-Gaussianity in the estimation to optimally separate both Gaussian and non- Gaussian components. We estimate the parameters by maximum likelihood. Since most non-Gaussian components in real data are super-Gassiian have the super-Gaussian density p(8,) (N(s;; 0, () 0,,;;
21 13
we assume that the non-Gaussian components
We propose to maximize the log- log-likelihood using generalized EM 41 22 Derivations are available
in Appendix EE] Like in the previous section, the E-step is in closed-form yielding the following sufficient statistics:
Eee(t Oa og and V[sg[x]
Ece
Oa
E att
E Ece Oa
E[sy|x] xX
Eae(0.5,1.5) Oa
where 0, NW;;; 0, Es + a). Dj
E.
and ΣS (,, 2g 1 with y; Wxx, Noise Σ,
updates are in closed- form and given by: Σ, + (ong((y-Elss>> (y, E + V|s However, no closed-fore form is available for the updates of unmixing matrices. We therefore perform quasi- quasi-Newton updates given by W; + (I- )(? W. W,, where م E IR is chosen by backtracking line-
search, W.W,, abb..d Suddbe dad ( is approximation of the Hessian of the negative complete Ena likelihood and gW: = - + (E,)
(y:) the gradient.
We alternate between computing the statistics E[s]x], V/s|x] (E-step) and updates of parameters E; ...... m (M-step). Let us highlight that our EM algorithm and particular the E-step
and W; for 11...M resembles the one used in [40]. However because they assume noise on the sensors and not on the components, their formula for E[s[x] involves a sum with 2 terms whereas we have only terms. The resulting method is called ShICA-ML.
Minimum mean square error estimates in ShICA In ShICA-J well as in ShICA-ML. we have a closed-form for the expected components given the data TE[s) [x], shown in equation (5) and (6) respectively. This provides minimum mean square estimates of the shared components, and is an important benefit of explicitly modelling shared components in a probabilistic framework.
Related Work
ShICA combines theory and methods coming from different branches of "component analysis". can be viewed as a GroupICA method, as an extension of Multiset CCA, as an Independent Vector Analysis method or, crucially, as an extension of the shared response model. the setting studied here, ShICA improves upon all existing methods.

---

Gr GroupICA methods extract independent components from multiple datasets. In its original form [4] views are concatenated and then a PCA is applied yielding reduced data on which ICA is applied. One can also reduce the data using Multiset CCA instead of PCA, giving method called CanICA [58. Other works [24 32 apply ICA separately on the datasets and attempt to match the decompositions afterwards. Although these works provide very fast methods, they do not rely on a well defined model like ShICA. Other GroupICA methods impose some structure on the mixing matrices such as the tensorial method of [10 or the group tensor model in (27] (which assumes identical mixing matrices up to a scaling) or (54] (which assumes identical mixing matrices but different components). In ShICA the mixing matrices are only constrained to be invertible. Lastly, maximum- maximum-ikellhood based methods exist such as Multi Viewiii [51] (MVICA) or the full model of [27]. These methods are weaker than ShICA as they the same noise covariance across views and lack a principled method for shared response inference.
Multiset CCA In its basic formulation, CCA identifies shared space between two datasets. The extension to more than two datasets is ambiguous, and many different generalized CCA methods have been proposed. [33] introduces 6 objective functions that reduce to CCA when = 2 and [2] considered 4 different possible constrains leading to 24 different formulations of Multiset CCA. The formulation used in ShICA-J is refered to in 42 SUMCORR with constraint 4 which is one of the fastest as it reduces to solving a generalized eigenvalue problem. The fact that CCA solves a well defined probabilistic model has first been studied in [9] where it is shown that CCA is identical to multiple battery factor analysis [12 (restricted to 2 views). This latter formulation differs from our model in that the noise is added on the sensors and not the components which makes the model unidentifiable. Identifiable variants and generalizations can be obtained by imposing sparsity on the mixing matrices such as in [8 34 [2] or non-negativity (20) The work in [38] exhibits set of sufficient (but not necessary) conditions under which a well defined model can be learnt by the formulation of Multiset CCA used in ShICA-J. The set of conditions we exhibit in this work are necessary and sufficient. We further emphasize that basic Multiset CCA provides a poor estimator as explained in Section 3.2
Independent vector analysis Independent vector analysis 36 (IVA) models the data as a linear mixture of independent components x; A₁S; where each component Sty of given view can depend on the corresponding component in other views ((st)) ) are not independent). Practical implementations of this very general idea assume distribution for p((s:))(= In IVA-L (36),
((ss)))=1) x exp(- --y E [st))² (so the variance of each component in each view is assumed to be the same), in IVA-G 5 or in [60]. (((s;) y ~ NO, R.s) and 23 proposed normal averre-Gamma density. Let us also mention IVA-L-SOS [I. IVA-GGD
and IVA with Kotz distribution 6 that assume a non-Gaussian density general enough so that they can use both second and higher order statistics to extract view-specific components. The model of ShICA can be seen as instance of IVA which specifically enables extraction of shared components from the subject specific components, unlike previous versions of IVA. In fact, ShICA comes with minimum mean square error estimates for the shared components that is often the quantity of interest. The IVA theory provides global identifiability conditions in the Gaussian case (IVA-G) [59] and local identifiability conditions in the general case 7 from which local identifiability conditions of ShICA could be derived. However, in this work, we provide global identifiability conditions for ShICA. Lastly, IVA can be performed using joint diagonalization of cross covariances [37 [9] although multiple matrices have to be learnt and cross cross-covariances are not necessarily symmetric positive definite which makes the algorithm slower and less principled.
Shared response model ShICA extracts shared components from multiple datasets, which is also the goal of the shared response model (SRM) [66. The robust SRM (57] also allows to capture subject specific noise. However these models impose orthogonality constraints on the mixing matrices while ShICA does not. Deep variants of SRM exist such as [17] but while they release the orthogonality constrain, they are not very easy to train or interpret and have many hyper-parameters to tune. ShICA leverages ICA theory to provide much more powerful model of shared responses.
The main limitation of this work is that the model cannot reduce the dimension inside each view: : there are as many estimated sources sensors. This might be problematic when the number of sensors is very high. In line with other methods, view-specific dimension reduction has to be done by some external method, typically view-specific PCA. Using specialized methods
Limitations
7

---

ShICA-ML
ShICA-J
Multiset CCA
CanICA
MVICA
Amari distance
10a
10-5
(a)
(c)
10a
10a
10
10a
10
10*
10a
10a
10
Number of samples
Figure 2: Separation performance: Algorithms are fit on data following model (a) Gaussian
components with noise diversity (b) Non-Gaussian components without noise diversity (c) Half of the components are Gaussian with noise diversity, the other half is non-Gaussian without noise diversity.
for the estimation of covariances should also be of interest for ShICA-J, where it only relies on sample covariances. Finally, ShICA-ML uses simple model of Super-Gausian distribution, while modelling the non-gaussianities in more detail in ShICA-MI should improve the performance.
Experiments
Experiments used Nilearn 3 and MNE [26 for fMRI and MEG data processing respectively, as well
as the scientific Python ecosystem: Matplotlib 31, Scikit-leam 455. Numpy (29 and Scipy [61]. We
use the Picard algorithm for non-Gaussian ICA [2] and mvlearn for multi-view ICA 47). The above
libraries use open-source licenses. fMRI experiments used the following datasets: sherlock [[5)
forrest (28 raiders [49] and gallant [4) The data we use do not contain offensive content or
identifiable information and consent was obtained before data collection. Computations were run on
a large server using up 100 GB of RAM and 20 CPUs in parallel.
Separation performance In the following synthetic experiments, data are generated according to model II with p = 4 components and m = 5 views and mixing matrices are generated by sampling coefficients from standardized Gaussian. Gaussian components are generated from standardized
Gaussian and their noise has standard deviation Σ (obtained by sampling from uniform density between and 1) while non non-Gaussian components are generated from a Laplace distribution and their noise standard deviations components are non-Gause Gaussian or half of the components are Gaussian and half are non- non-Gausiian. We vary the number of samples between 102 and 10 and display in Fig 2 the mean Amari distance across subjects between the true unmixing matrices and estimates of algorithms as function of n. The experiment is repeated 100 times using different seeds. We report the median result and error bars represent the first and last deciles.
equal. We study 3 cases where either all components are Gaussian, all
When all components Gaussian (Fig. (a)), CanICA cannot separate the components all. In contrast ShICA-J, ShICA-MI, Multiset CCA and MVICA are able to separate them, but Multiset CCA needs many shows that correcting for the rotation due to sampling noise improves the results. Looking at error bars, we also that the performance of Multiset CCA varies quite lot with the random seeds: this shows that depending on the sampling noise, the rotation can be very different from identity. MVICA sample than Multiset CCA to reach low amari distance but still outperforms
samples than ShICA-J or ShICA-ML to reach low amari distance, which
needs even CanICA.
When none of the components are Gaussian (Fig. (b)), only CanICA, ShICA-ML and MVICA able to separate the components, as other methods do not make use of non- -ou-Gaussiantty. Finally, in the hybrid case (Fig. 2 (c)), ShICA-ML is able to separate the components as it can make use of both non-Gaussianity and noise diversity. MVICA is a lot less reliable than ShICA-ML, is uniformly worse and error bars are very large showing that for some seeds it gives poor results. CanICA, ShICA-J and MultisetCCA cannot separate the components at all, Additional experiments illustrating the separation powers of algorithms
available in Appendix H.I
As we can see, MVICA can separate Gaussian components to some extent and therefore does not completely fail when Gaussian and non-Gaussian components present. However MVICA is lot

---

SHICA-ML
Multisct OCA
MVICA
10
ShiCA-J
CanICA
8 moem
Pitting time (in s)
10a
10"2
(In
109
10-1
SMCA
SMFiA-J
Moltinn CGA
MVICA
CarCA
10-4
10-3
10-2
10-3
10P
Amari distance
(a)
Figure 3: Left: Computation time. Algorithms fit on data generated from model with Gaper-Gaussian density. For different values of the number of samples, we plot the Amari distance and the fitting time. Thick lines link median values across seeds. Right: Robustness war. intra-subject variability in MEG. (top) 22 distance between shared components corresponding to the same stimuli in different trials. (bottom) Fitting time.
less reliable than ShICA-ML: MVICA is uniformly worse than ShICA-ML and the error bars are very large showing that for some seeds it gives poor results.
Computation time We generate components using slightly super Gaussian density: 8, d(x) with d(xx) = _r/=:0.2 and x (0, 1). We vary the number of samples between 102 and 10 We compute the mean Amari distance across subjects and record the computation time. The experiment repeated 40 times. We plot the Amari distance as function of the computation time in Fig Ga] Each point corresponds to the Amari distance/computation time for a given number of samples and a given seed. We then consider for given number of samples, the median Amari distance and computation time across seeds and plot them the form thick line. From Fig [a we see that ShICA-J the method of choice when speed is a concern while ShICA-ML yields the best performance in terms of Amari distance at the cost of an increased computation time. The thick lines for ShICA-J and Multiset CCA are quasi-flat indicating that the number of samples does not have strong impact on the fitting time as these methods only work with covariances. On the other hand CanICA or MVICA computation time is more sensitive the number of samples.
In the following experiments we consider the Cam-CAN dataset [56). We use the magnetometer data from the MEG m 100 subjects chosen randomly among 496. In appendix F we give more information about Cam-CAN dataset. Each subject is repeatedly presented three audio-visual stimuli. For each stimulus, we divide the trials into two sets and within each set, the MEG signal is averaged across trials to isolate the evoked response. This procedure yields chunks of individual data per stimulus). We study the similarity between shared components corresponding to repetitions of the same stimulus. This gives measure of robustness of each ICA algorithm with respect to intra-subject variability. Data are first reduced using a subject-speciff PCA with 10 components. The initial dimensionality of the data before PCA 102 as we only use the 102 magnetometers. Algorithms are run times with different seeds on the 6 chunks of data, and shared components to repetitions of the same stimulus they should yield similar components. For each component and for each stimulus, we therefore measure the & distance between the two repetitions of the stimulus. This yields 300 distances per algorithm that are plotted on Fig 3b
Robustness w.r.t intra-subject variability in MEG
extracted. When two chunks of data correspond
The components recovered by ShICA-ML have much lower variability than other approaches. The performance of ShICA-J competitive with MVICA while being much faster fit. Multiset CCA yields satisfying results compared with ShICA-J. However we see that the number of components that do not match at all across trials is greater Multiset CCA. Additional experiments on MEG data are available in Appendix H.3

---

Reconstructing the BOLD signal of missing subjects We reproduce the experimental pipeline of [5T] bench- mark GroupICA methods using their ability to recon- struet fMRI data of a left-out subject. The preprocessing involves a dimension reduction step performed using the shared response model [16]. Detailed preprocessing We call unmix-
ShiCA-J
CanICA
Forrest
gallarnt
R s00m
0.20
6.126
6.300
0.15
6.076
pipeline is described in Appendix F ing operator the product of the dimension reduction operator and unmixing matrix and mixing operator its pseudoinverse. There is one unmixing operator and one mixing operator per view. The unmixing operators are learned using all subjects and 80% of the runs. Then they applied the remaining 20% of the runs using 80% the subjects yielding unmixed data from which shared components
Faiders
sberlock
5c0re
0.15
Number componestts
sherlock
forrest
extracted. The unmixed data combined by averaging (for SRM and other baselines) or using the MMSE estimate for ShICA-J and ShICA- ML. We then apply the mixing operator of the remaining 20% subjects on the shared components reconstruct their data. Reconstruction accuracy is measured via the coefficient of determination, a.k.a. R² score, that yields for each voxel the relative discrepancy between the true time course and the predicted one. For each compared algorithm, the experiment is run 25 times with different seeds to obtain error bars. We report the mean R² score across voxels in region of interest (see Appendix [+ for details) and display the results in Fig 4 The error bars represent a 95% confidence interval. The chance level is given by the R² score of an algorithm that samples the coefficients its unmixing matrices and dimension reduction operators from standardized Gaussian. The median chance level is below 10" on all datasets. ShICA-ML yields the best R² score all datasets and for any number of components. ShICA-J yields competitive results with respect to MVICA while being much faster to fit. A popular benchmark especially in the SRM community is the time-segment matching experiment [16] we include such experiments in Appendix [.2)] In appendix G we give the performance of ShICA-ML, ShICA-J and MVICA in form of a table.
107
Fis time 6)
10P
z01-
Faiders
gallant
107-
102
50
Namber of componenta
Figure 4: Reconstructing the BOLD sig- nal of missing subjects. (top) Mean Rt score between reconstructed data and true data. (bottom) Fitting time.
7 Conclusion, Future work and Societal impact
We introduced the ShICA model as a principled unifying solution the problems of shared response modelling and GroupICA. ShICA is able to use both the diversity of Gaussian variances and non- Gaussianity for optimal estimation. We presented two algorithms to fit the model: ShICA-J, a fast algorithm that uses noise diversity, and ShICA-ML, a maximum likelihood approach that can use non-Gan Con--aassiantty on top of noise diversity. ShICA algorithms come with principled procedures for shared components estimation, as well as adaptation and estimation of noise levels in each view (subject) and component. On simulated data, ShICA clearly outperforms all competing methods in terms of the trade-off between statistical accuracy and computation time. On brain imaging data, ShICA gives more stable decompositions for comparable computation times, and more accurately predicts the data of one subject from the data of other subjects, making it good candidate to perform transfer learning. Our code is available at hhtpps:///// / giuh........ com/hugort caaggorichar//Sh....
'Regarding the ethical aspects of this work, we think this work presents exactly the same issues as any brain imaging analysis method related to ICA.
10


Reviews:
[{"review_id": "wCm9XlihPH", "reviewer": "Reviewer_GxXa", "summary": "The paper considers a multi view (or multi subject) ICA model and provides a few methodological extensions and results. The Gaussian source assumption, EM optimization, and connections to mCCA + joint diagonalization for efficient initialization are the main contributions relative to the previously published model [41]. This work also supports different noise variances in different sources and different views, and presents a stronger (global) identifiability proof. The work is largely inspired by the temporal group ICA and is presented in that paradigm: shared timecourses, assumed shared task stimuli across subjects etc.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## The main concerns:\n\n1.  Overall, the paper heavily builds on reference [41] and the previous work is not discussed enough thus misrepresenting the context of the current work.\n2.  The papers expects the underlying system to consist of shared sources (here time courses) well modeled by ShICA. This feature is used to justify invalidity of IVA for ALL of fMRI analysis. Even under identical stimuli, a shared (identical) response over subjects is all but a fictional (though useful) construct, which cannot be the basis for establishing the utility of a method. Consider that the entire field of Spatial ICA for fMRI is fervently moving AWAY from shared sources in order to better model subject variability. Furthermore, it is not inconceivable that certain brain networks are highly variable from subject to subject to the point they may not even correlate at all and, even more, they can be dynamic (changing) over space AND time. The more you think about it, the less likely it is that a shared source model is true, even in task-based settings, when we consider mind wondering, task engagement issues, and various mental illnesses. It would be more reasonable to not criticize IVA for \"lack\" of a shared construct, and instead assess it on the basis of captured source variability at the subject level, i.e., compare s<sub>ij</sub> from IVA to (s<sub>j</sub> + n<sub>ij</sub>) of ShICA.\n3.  Related to the above comparisons are to the outdated IVA-L (no correlation estimation) and IVA-G (no HOS estimation). It is not unreasonable to apply the same joint diagonalization trick from ShICA-J to “fix” IVA-G, for example. And IVA has evolved since 2012.\n    1.  <https://ieeexplore.ieee.org/document/6638257?arnumber=6638257>\n    2.  see also IVA-L-SOS and IVA-GGD in <http://mlsp.umbc.edu/jointBSS_introduction.html>\n4.  Note a contradiction in the claims. First the paper claims that \"… These methods are weaker than ShICA as they assume non-Gaussianity, the same noise covariance across views and lack a principled method for shared response inference\" Then it adds: \"… ShICA-ML uses a simple model of a super-Gaussian distribution, while modelling the non-gaussianities in more detail in ShICA-ML should improve the performance\" But ShICA-ML is the best model, even though it uses non-gaussianity: \"We further introduce ShICA-ML, a maximum likelihood estimator of ShICA that models non-Gaussian components using a Gaussian mixture model. While ShICA-ML yields more accurate components, ShICA-J is significantly faster and offers a great initialization to ShICA-ML\".\n5.  Experimental results are incomplete and not just for lacking comparison with the latest IVA algorithms. The MultiView ICA [41] is not compared against. Some of the figures list MVICA, which be it, but the acronym is neither defined not properly discussed. Additionally if one was to compare Figure 5 in the current submission with Figure 2 from [41] - MultiView ICA looks extremely competitive if not better. The MVICA lines for the same datasets in Figure 5 are drawn lower than the model, if they are indeed the same models, was capable of in [41]. Additionally, the accuracy plot is missing in Figure 5 and that was the most discriminative plot of Figure 2 in 41.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "no limitations", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "oT3oK328-5A", "reviewer": "Reviewer_LARb", "summary": "This paper presents a method, called shared ICA, for finding shared independent components across multi-view neuroimaging data. ShICA models each dataset as a linear transform of shared independent components with Gaussian noise. Necessary and sufficient conditions for the identifiability of ShICA are first discussed. Then two variants of the implementation are introduced, where ShICA-ML is more robust than ShICA-J in terms of deriving unmixing matrices under perturbation. Experiments on MEG and fMRI show that ShICA shows potential improvement in reproducibility, time segment matching, and data reconstruction. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper provides necessary and sufficient conditions for the identifiability of the independent components under certain assumptions, which seems to be an improvement over prior works. The maximum-likelihood solution based on EM looks reasonable. The method section is drafted in a rather clear fashion. I now list my concerns in the following:\n\n1. Formulation: the authors name the generative process of Eq 1 as ShICA, but this is the same formulation as in [41]. The difference, as the authors mention, is the non-Gaussianity, noise model, and inference method. This limits the novelty and impact of the work. Moreover, the practice of putting both view-specific difference and measurement noise into a big blender $n_i$ worries me as these two sources of noise clearly entail different structures. Modeling measurement noise before mixing rather than after mixing  (outside the parenthesis) is unnatural to me.\n\n2. Validation: \na. ICA-based multi-modal analysis is a widely explored area, which goes beyond the SRM papers cited herein. The review paper by https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4917230/ lists 10+ practices for multi-modal neuroimaging analysis. jICA, pICA, and linked ICA are all conceivably good counterparts to compare with. Although the experimental setup highly overlaps with [41], the present baselines are a just subset of the baselines in [41]. \n\nb. I'm a bit confused why different baselines are used for different figures, e.g., MVICA has a runtime measure in Figure 3, but does not have an accuracy measure in Fig. 2. \n\nc. More details should be given for the MEG and fMRI experiments, at least in the supplement (study cohort description, data spatial/temporal resolution/size, experimental task design in fMRI, etc). \n\nd. I totally do not understand the last three figures in the supplement, no captions given.\n\n3. Impact on neuroimaging: the present experiments are all from a signal-processing perspective, e.g., reconstruction and segment mathcing. It is unclear how this method can eventually be used in neuroimaging studies. Well-established methods like groupICA can efficiently analyze hundreds of subjects in modern neuroimaging studies, which is unlikely to be feasible for ShICA due to the EM inference. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See point 3 of the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5 hours"}, "extra_text": {}}, {"review_id": "WaDOQxX1smM", "reviewer": "Reviewer_qGdn", "summary": "This paper introduces a new generative model and inference algorithms for shared Independent Component Analysis (ShICA). The model incorporates an explicit noise term on the components, which allows them to be identifiable up to a sign and permutation. The authors provide conditions to guarantee identifiability and introduce two solution algorithms. ShICA-J assumes Gaussian components and relies on multiset CCA coupled with a joint diagonalization. ShICA-ML assumes a specific non-Gaussianity and relies on the EM algorithm for component estimation. The authors demonstrate the robustness of their algorithms on simulated data and do an exploratory evaluation on MEG and fMRI.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this is a solid paper that presents an interesting new twist on ICA. The authors complement the model with theoretical guarantees for identifiability, assuming specific noise characteristics. With that said, the assumptions seem overly restrictive, and the performance gain is marginal on real-world data. Specific comments on the paper are as follows:\n\nStrengths\n\n+ The framework imposes an explicit noise model on the components, which allows for cross-subject fusion and confidence estimation. The noise model also provides identifiability of the components.\n+ The authors introduce two estimation algorithms that tradeoff efficiency and generalization.\n+ Theoretical guarantees are provided for each optimization procedure.\n+ The simulated experiments are quite extensive and clearly demonstrate the performance regimes for which each method (introduced and baselines) is well suited.\n+ The MEG also demonstrates the robustness of the SHICA-ML method over the others.\n\nWeaknesses\n\n- Assumption 1 (noise diversity) seems fairly restrictive, and it is unclear from a modeling standpoint, why one would assume this for real-world data. Taking fMRI as an example, the components would roughly correspond to different functional networks in the brain. Why would we assume a priori that one network is “noisier” than another? Even if this were the case empirically, to strictly require it seems presumptive.\n- As far as I can tell, there is no requirement that the mixing matrices A_i be similar across the views. This seems problematic, as I would assume the different “views” in an MEG or fMRI experiment correspond to different subjects. Thus, one would want to identify similar activations patterns across the group.\n- While Section 3 (ShICA-J) is interesting from an algorithmic standpoint, it is unclear how useful this algorithm is in practice. Specifically, ShICA-J assumes only Gaussian components, which is contrary to one of the main advantages of ICA in general (non-Gaussianity). At a high level, ShICA-J is closer to a PCA type analysis than the standard ICA. I would have liked to see a comparison with common PCA.\n- It is unclear why the authors claim that most non-Gaussian components in real data are super-Gaussian, or why they assumed the particular density in Section 4. At the very least, it seems that the mixture weights and variances of the non-Gaussian density should be estimated via EM.\n- The BOLD reconstruction results are very similar across methods. From a practical standpoint, there is little difference between the reported R^2 values. Plus, the confidence intervals are highly overlapping.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors acknowledge that their framework cannot perform dimensionality reduction and relies on a simple non-Gaussianity. However, they do not discuss other potential limitations (both theoretical and empirical) described above in the “Main Review”.\n\nWhile this work has natural application to functional neuroimaging, the performance gains are not particularly impressive, particularly for fMRI. The societal impact sentence is generic and overly broad. Given that decades of research involving PCA, ICA, dictionary learning, deep learning, etc. has made a minimal impact on “reducing the human and societal burden of brain disease”, it is unlikely that, yet another variant of ICA will accomplish this feat.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "5PRe2A1vt_T", "reviewer": "Reviewer_9MQr", "summary": "The paper introduces Shared Independent Component Analysis (ShICA), a multi-view ICA method which assumes each view as a linear transformation of a single set of shared independent components with additive gaussian noise. It can be viewed as a generalization of a slew of \"component analysis\" methods such as GroupICA, Multiset CCA, Independent Vector Analysis, and SRM. Unlike GroupICA methods, ShICA allows for Gaussian (as well as non-Gaussian) components, different noise covariance and extraction of shared components. While under certain conditions Multiset CCA can recover the mixing matrices in ShICA formulation, the paper proves conditions where that is not the case, and demonstrates the lack of robustness of Multiset CCA in certain situations. ShICA is also closely related to IVA, but allows for extraction of shared components. Similarly, it can be viewed as a relaxation of (biologically infeasible) orthogonality constraint on SRM.\n\nThe paper provides assumptions and theorems laying out the theoretical framework for ShICA and the inference algorithms used. It also theoretically demonstrates the connection between ShICA and Multiset CCA, and uses joint diagonalization of multiset CCA solution to recover unmixing matrices in situations where multiset CCA can't. For situations with gaussian and non-gaussian components the paper introduces ShICA-ML a maximum likelihood general estimator for ShICA. Experiments included simulations as well as fMRI and MEG datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall the paper is well-written and does a good job backing up claims with theory as well experiments. While the problem and formulation is related to previous work, the paper clearly lays out where it differs and improves upon the existing work. I think this is a significant contribution to the independent component analysis literature. I'd however like to see responses to the following concerns. please note that these are ordered in the order they appear in the paper, I will mark the major concerns as such, but would appreciate if the authors can respond to all/ as many of these as possible.\n\n- (Major) Title: the paper is titled \"Shared Independent Component Analysis for Multi-Subject Neuroimaging\". While I'm all for interdisciplinary work that is theoretical but also attack specific applications. I found this title to be somewhat misleading. While it's true that the paper uses fMRI data and MEG data in the experiments there is really no significant discussion of why this method is specifically geared towards neuroimaging. For example, the words neuroimaging and neuroscience appear literally once in the main paper.  One possible solution is to expand upon the neuroscientific implications a little bit more in the first paragraph (as opposed to just the one line that's there), or add more discussion in the conclusion section.\n\n-line 39 \"each dataset is modelled as...\" somewhat confusing use of the word dataset here. is x_1,...,x_m being referred to as one dataset, or x_1 is one dataset?\n\n- (Major) lines 47-54. The line starting \"we then introduce an algorithm called ShICA-J...\" and the line starting \"We next point out the practical problem...\" make it sound like these are two different contributions. From reading the paper (and from Algorithm 1) what I gathered is that ShICA-J is the joint-diagonalization of the W_i's obtained from Multiset CCA and is the solution to the sampling noise leading to large rotations in Multiset CCA solution. The way these lines are written right now makes it sound like ShICA-J is separate from joint diagonalization of the result of multiset CCA.\n\n- (Minor) line 91. The theorem/assumptions/propositions are all being counted in a single series. I think it will improve readability if you can separate out the numbering for theorems from numbering for assumptions etc... The way it is right now I went looking for Theorem 1 (which of course doesn't exist)\n\n- line 113. \"Generalized CCA consists in\" --> \"Generalized CCA consists of\"  . \n\n- line 224. Citation required for \"most non-Gaussian components in real data are super-Gaussian\"\n\nfollowing concerns regarding experiments, while not very major, will definitely go a long way in convincing me to possibly increase my score for this paper:\n\n- Figure 1: I had somewhat of a hard time with the figure. It might just need a clearer legend. What is the black dashed line at 10^-1? just like there are 4 solid colored lines, shouldn't there be 4 dashed colored lines corresponding to the different eigen gaps with joint diagonalization solution?\n\n- line 299. \"Deep variants of SRM... are much more computationally demanding\". Would be nice to see some experiments showing the truth of this statement. Or if this is established based on experiments in some other publication please provide citation.\n\n- Figure 2: why are results for MVICA not included in this figure?\n\n- line 373: Please include the original dimensionality of the data as well and why was p=10 components choosen?\n- line 387: what was the number of factors used in SRM preprocessing and why?, what was the original dimensionality?\n- (Major) Figure 5: This connects to my comment about the title. These results don't seem all that much of an improvement. In terms of R^2 score except for \"gallant\" dataset all methods are pretty much behaving in a very similar manner while CanICA seems to have a lower fit time in most cases. Now if this were just one of the many applications shown in experiments that would have been okay, but since the title is gearing me to expect good performance \"specifically\" on neuroimaging data, these results seem somewhat underwhelming. I'm happy to be convinced otherwise, else the title/intro/conclusion changes become all the more important.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "are addressed appropriately", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces Shared Independent Component Analysis (ShICA), a method for shared response modeling in which shared components are independent. While building on previous work, this paper has clear novelty in model specification and related theory, using  multiple estimation algorithm with different noise specifications, and convincing experimental results.

The reviewers have proposed multiple changes that would improve the paper and that the authors have consented to do. Namely, a change in the framing, including a better discussion of the relevance to neuroscience and the relation to prior work, multiple clarifications, including in respect to model formulation and assumptions. I recommend acceptance for the paper and ask the authors to make these changes that will further improve its quality.
Decision: Accept (Poster)