Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper examines a multi-view learning problem in which one wants to identify common shared components from multiple datasets. The proposed method, Shared Independent Component Analysis (ShICA) has identifiability if components express a sufficient level of non-Gaussianity as well as enough diversity in noise variation. The key to ShICA that makes it different from the common multiset CCA is that it incorporates joint diagonalization across the views. The paper additionally introduces a maximum likelihood version of the proposed method, ShICA-ML, which is more accurate but less computationally efficient.
main_review: The paper does an excellent job of clearly identifying the current methods used in independent component extraction from multiple views (in section 5, Related work). Although the reviewer sees why the related methods are added right before the results section (to provide benchmarks), he was not certain as to the main contributions of the model until reading this section. It would be useful to add a clearer description in the introduction as to the main limitations of the current methods and how ShICA addresses these. A short paragraph explaining this would do. The reviewer also appreciates the candid explanation of limitations for the current ShICA method.

The presentation of the results are clear, which show that ShICA has a clear advantage in regards to separation performance, computation time, robustness with respect to intra-subject variability in MEG, reconstructing the BOLD signal of missing subjects.

The paper is very excellent and clearly presents the methodology and results in a way that convince the reader the advantages of the method. The main suggestion from the reviewer is to make some of the advantages of ShICA more clear in the paper (add a paragraph in the introduction to explain the main contributions of ShICA and maybe even ShICA-ML).

However, after reading some other reviewer comments, I have some concerns about the methodology and originality of the paper. Therefore, I must give a lower score.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposed a Shared Independent Component Analysis (ShICA) that models each dataset as a linear transform of shared independent components contaminated by additive Gaussian noise.
main_review: The author proposed a Shared ICA for multi-subject data which can extract both Gaussian and non-Gaussian components. Some major concerns:
a.	In simulation, the author only simulated 4 components and 5 subjects’ data to do the comparisons, which is too small to test the performance of the proposed method. At least the subject number N>100, and component number around 10.  
b.	The well-known GICA and IVA are not compared with the proposed method.
c.	There is no independence constraint in mCCA, so it is obviously that mCCA can’t deal with noise as stated in abstract. It is unfair to compare mCCA with other ICA related method with respect to noise. 
d.	Apart from simulation, no real fMRI data decomposition was compared. 

limitations_and_societal_impact: In term of novelty and performance improvement, the current paper is not enough. The overall quality is poor, and it does not meet the publication requirement.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper develops Shared Independent Component Analysis (ShICA) that learns a linear transform of shared independent components from a multi-view dataset. The ShICA is identifiable when the components are non-Gaussian or if the noise variances show “enough diversity”. A number of empirical studies have demonstrated that the proposed method outperforms the classic CCA on fMRI and MEG datasets.
main_review: The following are the major concerns and minor comments:

1)  In this paper, the empirical studies are reported using Amari distance and R2 score. It makes the produced results unclear. Indeed, we cannot compare the performance of the proposed method with the previous studies (e.g., Chen et al. 2015) that evaluated the related methods with the same datasets. Moreover, I cannot find clear information about the experiment scheme, including how cross-validation is applied (across subjects or instances), how the neural responses are learned after mapping to the shared space (in the same order, or in shuffled form), etc. I would like to compare the performance of the proposed method with other approaches using the regular metrics like classification accuracy and in the form of different schemes proposed in other related studies, including Chen et al. 2015, etc.

2) In this paper, performances of the various techniques are reported by manually selecting the hyperparameters --- e.g., number of components in Figure 5. The optimum values for each hyperparameter should be learned using internal cross-validation or a grid search. In the current format, we cannot have any inference from the empirical studies. Here are two problems: imagine the number of components. There is no guarantee that different techniques will produce the same performance with the same number of components. You must evaluate each technique with its best performance, obviously using the optimum hyperparameters. In addition, it seems that the performances of most techniques are increasing when the number of components is increasing. You have to show us when this tendency is not consistent anymore. Then we can compare the best of other techniques (e.g., SRM) with the best of yours.

3) In this paper, handling non-Gaussian noise is described as a novelty of the proposed method in comparison with CCA-based approaches such as SRM. However, I did not find an explanation to see why it is important? In fMRI datasets, each voxel shows the BOLD signal as a proxy to trace the oxygen usage of around 10000 neurons. We can assume that the distribution of one voxel (the BOLD signal) is related to the aggregation of all distributions corresponding to the oxygen usage of the associated neurons. Although we obviously do not have each of these distributions, the central limit theorem clearly explains that the Gaussian distribution may be the best assumption on the voxel (aggregated) distribution. To keep the procedure conjugate during the factor analysis, assuming the Gaussian noise in the other techniques makes more sense --- for instance, it can even help us to control SNR.

4) There are several related works that should be compared with the proposed method. For instance, SRM has been published in 2015 and there are several new extensions of SRM that can produce better performances in comparison with SRM, including MN-SRM (AISTAT 2018) and RSRM (ICASSP 2018).

limitations_and_societal_impact: Empirical studies and scientific evidence are my primary concerns. The proposed method was not compared with the related state-of-the-art techniques. Furthermore, the empirical studies did not follow the same scheme as other studies for evaluating learned models in a trustable manner. For instance, hyperparameters (such as the number of components) are not tuned to produce the best performance for each technique. As another example, they did not report the prevalent evaluation metrics such as the classification accuracy that is used in other studies to compare the performances. It is hard to compare the produced results with other studies that are even using the same datasets.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose the shared independent component analysis which can account for shared components $\mathbf{s}$ and individual differences through the noise signal $\mathbf{n}$. Notably the noise can vary across subjects and views in magnitude whereas the shared components can both be Gaussian or non-Gaussian (through an invoked super-gaussian two component fixed gaussian mixture). The model is optimized through an associated EM procedure in which mean and variance of the shared sources are estimated in the E step and the associated mixing matrices and noise variances estimated in the M step. As no closed form solution for the M step exists when considering super-Gaussian noise sources an associated Newton optimization procedure is here invoked. The procedure is demonstrated on synthetic and real datasets to provide superior recovery when compared to existing component models including canICA, multiview ICA and multiset CCA whereas mathematical properties in regards to model identifiability are carefully derived.
main_review: The manuscript is well written and considers a well studied problem with an extensive review of the existing approaches well positioning the current approach within the relevant literature on independent vector analysis, group ICA, multiview ICA and multiset canonical correlation analysis.

$\textbf{Originality}$
The approach is rather straightforward however the mathematical derivations of properties appear non-trivial. In particular, the result of identifiability even facing Gaussian sources through properties of the individual noise covariance matrices $\Sigma_i$ is useful and differentiates this approach in general from conventional methods. Furthermore, the experimental evaluation is solid and the approach found to be meritable providing enhanced recovery. All in all the mathematical rigour and the convincing experimental section can warrant publication of this manuscript.

$\textbf{Quality}$
The manuscript is well written and sound. The experimentation is convincing and includes an extensive synthetic analysis as well as an analysis of real magneto-encephalography (MEG) data. The results are promising and the proposed approach appears both efficient and accurate in terms of component recovery.

$\textbf{Clarity}$
The manuscript is well written and clear in its presentation. 

In proof of theorem 2 note:that $s + n^i$ should be $s + n_i$

It is unclear to me how the approach relates and differ to the approach of 
Klami, Arto, Seppo Virtanen, and Samuel Kaski. "Bayesian Canonical Correlation Analysis." Journal of Machine Learning Research 14.4 (2013).
In particular, this approach also accounts for view specific covariances and seems very related to the current approach when not including also individual factors but only shared factors and not imposing the mixture of two Gaussian prior. The authors thus need to relate their approach to this work.

$\textbf{Significance}$
The manuscript falls within a large literature on decomposition approaches accounting for individual variability while preserving shared component extraction. The work is carefully positioned within this literature and forms a promising although rather straightforward framework. However, the merits in terms of model identifiability are interesting and admits Gaussian component extraction providing efficiency and favorable results as demonstrated in the experimental validation which warrents publication.


Response rebuttal:
I have read all reviews and rebuttals. I am satisfied with the authors’ response to my concerns and still find the manuscript marginally above the acceptance threshold. I find the experimental section sufficiently convincing and although the approach is rather straightforward it has favorable merits whereas non-trivial derivations of properties are provided which I find can warrant publication. I am thus inclined to keep my marginal accept score.

limitations_and_societal_impact: Limitations and societal impact is sufficiently discussed.
review_title: null

Meta-review: Two reviewers recommend rejection and two reviewers recommend acceptance. After reading the reviews, the rebuttal, the internal discussion among reviewers and after my own reading of the paper, I believe this work provides a genuine contribution to ICA for multiple views and therefore I recommend Accept. The method provides identifiable components (under certain restrictions) which makes it robust, and the empirical section shows this advantage of the method and its usefulness in downstream tasks, e.g. reconstructions of left-out subjects in fMRI data. Two reviewers that recommend rejection have mainly asked for comparisons against other baselines. However, I believe that the authors have included suitable baselines when testing their model. I recommend the authors follow the recommendations of the reviewers regarding the presentation and fixing any typos.
Decision: Accept (Poster)