Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a batch normalization framework that is suitable for problems of unsupervised domain adaptation (UDA) between symmetric positive definite (SPD) manifolds. Their framework- the SPD domain-specific momentum batch normalization or SPDDSMBN is designed to transform domain-specific SPD input matrices into domain-invariant SPD outputs by automatically inferring relevant batch statistics. This can be inserted a parameterized layer transformation within a geometric deep learning model in multi-(domain) source and target settings. The authors also provide a theoretical analysis on the correctness of recovered batch statistics. The authors perform experiments with 6 different EEG brain-computer interface (BCI) datasets and demonstrate that their framework provides improved performance within two transfer learning setups (inter-session and inter-subject) against baseline comparisons and ablation studies

strengths_and_weaknesses: STRENGTHS: 

1. The paper is very well written and easy to follow.  The relevant technical details are presented concisely and cover the relevant preliminaries making the paper fairly self contained. The main contributions are clearly articulated with dedicated theoretical analysis and thorough experimental evaluations provided to support the claims. 

2. The main contributions of the paper, extending momentum based batch-normalization to geometric deep learning on SPD manifolds as well as the application to domain adaptation in SPD manifolds is interesting and relevant to applications beyond the EEG BCI domain. Although not a major contribution of the work, the discussion on interpretability in the context of the potential healthcare application is a plus.

WEAKNESSES:

There are very few details provided for generating the interpretability maps in Fig. 3 making this section a bit hard to parse. 

(a) As per my understanding, the contribution of each extracted source to the target class (spatial mixing coefficient) is being plotted on the head model. I am a bit unclear on how to actually interpret the results being displayed, for example, are the darker regions more discriminative/important (the figure is also missing a colourbar)? This needs a bit of clarification
(b) I am unclear on what the axes of the plot on the top left of 3 a and 3 b are plotting. Specifically, what is the y axis "contribution" measuring?
questions: 1. A suggestion would be to include more details on generating the interpretability maps from the reference [17] in the supplementary for completeness. Adding a few more sentences of explanation to the plots in Fig. 3 would also help improve the clarity here. 
2. There are many acronyms used in the paper for each discussed variation of the framework. A suggestion would be to include a list of these for easy lookup within the supplementary document so a potential reader can avoid back and forth within the main text.


limitations: The main limitations of the framework, scalability to larger sized input data (eg. fMRI) is discussed briefly in Section 6. I agree that this is a fair criticism of all of the methods. Perhaps a relevant future direction would be to utilize techniques that can improve the scalability of the bottleneck eigenvalue decomposition.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors present a new batch normalization module designed to improve performance of neural networks in scenarios where domain adaptation is critical, such as classification of inter-session and inter-subject EEG data. This is achieved by combining the ideas of momemtum BN, domain-specific BN, while respecting the manifold of SPD matrices.

On top of theoretical results supporting their approach, the authors provide compelling empirical evidence that the proposed SPDDSMBN module, when inserted into an SPD-aware deep neural network architecture, helps improve cross-domain performance compared to existing state-of-the-art approaches, as shown on 6 EEG datasets encompassing BCI and mental workload estimation classification tasks. An ablation study further confirms the importance of the proposed mechanisms for achieving these results.
strengths_and_weaknesses: The proposed SPDDSMBN method is original. While a logical extension of existing batch normalization ideas, it combines multiple important ideas that together provide significant performance improvements over existing state-of-the-art methods. Moreover, the interpretability of the proposed approach makes it particularly interesting for neuroscience-minded audiences who might want to explore correlates of the different classes in an EEG dataset.

The paper is of high quality and includes compelling theoretical proofs, experiments and results on multiple datasets. The manuscript is clearly written and well organized, as is the provided code.

Overall, I believe the manuscript has particular relevance to the NeurIPS community, even beyond the EEG and neuroimaging communities.
questions: A single point that would make this manuscript even more convincing for me, though not necessary, would be if authors had provided results on a wider range of classification tasks, e.g. sleep staging, seizure detection, emotion classification and/or pathology detection. These tasks might present different inter-session and inter-subject variability characteristics. I believe including results on the mental workload estimation task was a first step in that direction. Whether results can extend to these types of tasks and even to other types of multivariate time series would be very interesting to see. 
limitations: The authors have adequately addressed the limitations and negative impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: **Update after rebuttal**: I thank the authors for their responses and must say that I'm very happy with their rebuttal. Their submission has improved in this new version and I will, therefore, increase my score from borderline-accept (5) to accept (7).

---
This work presents a pipeline for classification with EEG recordings based on three steps. Firstly, the statistical features of the EEG time series are encoded via a symmetric positive definite (SPD) matrix. Then, they go through a neural network specially tailored for handling data points defined in the SPD manifold. Finally, the transformed SPD matrices are projected onto the tangent space of their manifold where a linear classifier with softmax activations is applied. 

The authors were particularly interested in the case where the statistics of the training dataset is different from that of the testing set. This happens quite often in EEG, for instance when working with brain-computer interfaces trained on different subjects. The differences in these statistics is an important challenge in BCI research and the paper tackles it with their domain-specific version of the momentum batch normalization for SPD matrices (SPDDSMBN), which is a layer in their neural network. The authors also include two interesting features to classification pipeline: (1) the ability of learning from the data which SPD features should encode the time series, as opposed to the traditional approach of fixing them to be the spatial covariance matrix. (2) interpretability of the linear weights used to classify the data points, directly relating them to the spatial distribution of the EEG recordings on different channels.
strengths_and_weaknesses: As an expert on the domain of EEG classification with SPD matrices, I must say that I enjoyed seeing the many new ideas presented in this paper. There are several small gems that tackle different practical challenges of modern BCI systems and I congratulate the authors for getting interested in them. However, the sequence of the presentation of these ideas is quite confusing and it was not particularly easy to follow through the logic of the text, even for someone with some experience in the field. 

For instance, it feels a bit odd to talk so much about batch normalization (and its variants) without saying why we need it for this application in the first place. Also, while I understand the structure of the architecture of TSMNet with SPDDSMBN, I still don't get why the theoretical results presented in Proposition 1 and Proposition 2 were necessary. Indeed, it feels much more like a math result that the authors wanted to include in the text than something that had a motivation for being part of the work (Please correct me if I'm mistaken). 

In line 161, the authors say "we believe that the performance can be improved by extending MBN to the SPD manifold". This is a very strange and imprecise of presenting something that is supposed to be one of your main contributions. Also, I'm uncertain of what you mean by proposing a "theory-based" machine learning framework.

Also, the text can be a bit hard to read due to its (way too) many acronyms. Are the authors certain that all of them are really necessary? Things like MSMTUDA and SPDDSMBN can be a bit disturbing to read.

In all, the content of the paper is interesting and worth of being shared with the community, but I am not satisfied with the structure of the presentation of ideas and think that the authors should improve it before publication.
questions: **(Q)** In the case where you have several source datasets, you simply pool all of their data points into a single big source dataset post transformations via SPDDSMBN? Does your procedure learn somehow to give more importance to data from one subject as compared to another one? In fact, your TSMNet is the same for all possible target subjects? Or you train one for each possible target?

**(Q)** Is there any theoretical result showing the limitations of SPDDSMBN to correct the dissimilarities between the statistics of the source and target dataset? In other words, do you have any mathematical model showing how a source and a target dataset may be different and whether your procedure is capable of correcting it?

**(Q)** What is the SOA domain-specific method used as reference for comparisons in Section 5.1?

**(Q)** In Equation 17 you present a layer depends on parameter $\phi$ and modulates the dispersion of the data points at its output. How 
does your training procedure avoid making $v_\phi \to 0$ and decreasing the variance of its output as small as possible? 

**(Q)** Saying "latent observations" in Line 172 is an oxymore. Do you observe these data points or not?

**(Q)** I am more accustomed to seeing the term "intra-domain" than "domain-specific". Do the authors have any specific reason for preferring this term?

**(Q)** Is Equation 7 really necessary for the rest of the paper?
limitations: The authors show the limitations of their method with applications to real data and show that while it does not always attain the same performance as a domain-specific SOA method, it does beat all other concurrent methods from the literature. However, I would have appreciated seeing a simulated example with controlled dissimilarities between the datasets and checking which class of transforms the method proposed by the authors is able to correct.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors proposes to learn domain invariant tangent space mapping models in an end to end fashion.
This mapping is suited to problems on data matrices that come from different domains and thus have possible different distributions.
These domain shifts occur in EEG when data are recorded on different people (structural and functional differences in brain networks) or several sessions (differences in the positioning of the electrodes).
The proposed differentiable pipeline is composed of 3 steps.
1) Data matrices, associated with different domains, are transformed to covariance matrices. The latter belong to the set of symmetric positive definite (SPD) matrices which is viewed as a Riemannian manifold with an affine invariant Riemannian metric.
2) The first and second order statistics of these matrices (grouped by domain) are normalized using a proposed Algorithm called "SPD domain-specific momentum batch normalization" (SPDDSMBN). Then the obtained matrices are transformed to vectors by lifting them to the tangent space at the identity using the logarithmic mapping.
3) Obtained vectors in the tangent space (linear space) are classified using a linear classifier.

The main contribution is the Algorithm SPDDSMBN. Covariance matrices of each domain are assumed to follow a Gaussian distribution, on the SPD manifold, parametrized by a mean covariance matrix (Fréchet mean) and a positive scalar (Fréchet variance).
SPDDSMBN estimates these parameters iteratively using batchwise estimates. The authors claim that the estimate of the Fréchet mean converges to its true value as the number of iterations tends to the infinite (Proposition 2). Then, the covariance matrices are transformed so that they have new learned first and second order moments. The interest is when the learned moments are enforced to be equal: all the covariance matrices from the different domains share the same statistics. This step is developed to improve the classification between different domains.

A second contribution is that the full pipeline is differentiable and thus is trained in a end-to-end manner. By choosing linear operators in the pipeline, the authors are able to derive a log-linear relationship between the targets and the variances of the sources the classical EEG generative model. Thus the discriminative sources are identifiable which makes it an interpretable method. 

The proposed method is applied on 7 EEG datasets of mental imagery and mental workload estimation shows overall great performance compared to other classifiers.

strengths_and_weaknesses: - Strengths:

Originality:

The proposed method is novel to me and leverages recent advances made on the SPD manifold such as the Gaussian distribution, the geometric law of large numbers, and parallel transport of SPD matrices. Recalibrating the second order moment is rarely done whereas it is easy to implement as shown in this paper.
Also, the proposed method leverages the recent advances on geometric deep learning (SPDnet) by training a full pipeline in an end-to-end manner. This should improves the performance compared to classical methods that perform sequential operations associated with different optimization problems.
The combination of these contributions is original.

Quality:

The numerical experiments are numerous and of great quality. The authors perform experiments on 7 EEG datasets of mental imagery and mental workload estimation and the proposed pipeline is compared to 10 other methods. They achieve state of the art performance on several of them and are on-par with best models on the others. Also, p-values of statistical tests are reported to ensure the significance of the reported results. An ablation study is performed and shows the importance of using all the proposed ingredients (momentum and domain-specific batchnorn). An additional good point is the Figure 3 which gives an interpretation of the learned model on two datasets.

Clarity:

The experimental part is clear: datasets are presented and the proposed method  used in practice is presented in Figure 4 and Table 3.


- Weaknesses:

Quality and clarity:

The theoretical part about the convergence of the running mean to the Fréchet mean is poorly written.
First of all, it seems that there is a confusion around
$Var_{\theta_k}(\mathcal{T}_{\theta_k})$

which is sometimes referred as $Var_{\theta}(\mathcal{T}_{\theta_k})$.

It seems that the latter is never defined.
Then, it is not very clear why the Proposition 1 (which is the only proven theoretical result) is mentioned. It is not interpreted and is never really used in the paper.
Furthermore, to prove this result the assumption (14) is made. The authors mention that it is met when $f_\theta$ is $L$-smooth. To me, this statement is not trivial and should be proven.
Also, the proof of the Proposition 1 is questionable. First of all, it is not clear why proving this induction proves that the mapping $l \mapsto Var_{\theta_k}(G_l)$ is non-increasing.
Then, it is not obvious how the equation (29) is obtained. The authors leverage the equation (19), but how does (19) (which relates variances of $\mathcal{T}_{\theta_k}$

between two successive iterates)
implies inequalities on the variances of $B_{k-1}$ and $B_k$. It is non-trivial since $B_k$ is an estimator (1-step Karcher flow) of the Fréchet mean.
The assumptions of the Proposition 2 are not very clear. Why do the previous assumptions such as the equation (14) are necessary ? Also, it seems to me that $\theta_k$ converges to a stationary point does not imply that $||\theta_k - \theta^\star|| \leq 1/k^\beta$. Furthermore, why does the convergence rate of $\theta$ must be in $1/k^\beta$ ? If after a certain amount of iterations, the $\theta_k$ stay in a small ball around $\theta^\star$, the covariance matrices $f_{\theta_k}(x)$ can be considered fixed and thus the Theorem of large numbers on the SPD manifold directly applies.


Finally, too many different notations are introduced in the paper which make it hard to read.

questions: The disappointing part of this paper is about the convergence of the running mean to the Fréchet mean.

1) Can you clarify the confusion around $Var_{\theta_k}(\mathcal{T}_{\theta_k})$

which is sometimes referred as $Var_{\theta}(\mathcal{T}_{\theta_k})$ ?

Are they the same thing ? If not, $Var_{\theta}(\mathcal{T}_{\theta_k})$ should be defined.

2) Can you prove that if $f_\theta$ is $L$-smooth then (14) is respected ?

3) A comment should be added on why the induction in the proof of the Proposition 1 implies that the mapping $l \mapsto Var_{\theta_k}(G_l)$ is non-increasing.

4) Can you detail how (19) gives (29) ? The variance of $B_k$ should be compared with the variance of $\mathcal{T}_{\theta_k}$.

5) A comment on why the Proposition 1 is interesting/useful should be added. Is  it useful to state the Proposition 2 ?

6) The assumptions of the Proposition 2 should be stated explicitly (and not "If we additionally ...") and made clearer. If the Proposition 2 is new (versus the work of [50]), then a proof should be added. Otherwise, it can be transformed to a remark on how the Theorem of large numbers of [50] applies to the pipeline.

7) The introduction of many different notations makes the paper hard to read,

- e.g. the estimated batch mean is denoted $\hat{G}_\mathcal{B}, B_k$,

- the estimated Fréchet mean $G_k, G_\mathcal{T}(k)$,

- the Fréchet mean $G_{\theta_k}, \text{FM}(\mathcal{Z})$,

- the Riemannian distance $\delta, \delta_\text{AIRM}$ ... This should be simplified.

8) Minor remarks:
- In the section "Riemannian geometry on $\mathcal{S}^+_D$: the inner product $\langle ., . \rangle_Z$ is wrong .
- Equation (2): the logarithmic mapping should be written between two different matrices.
- After equation (3) $\mathcal{S}^+_d$ is written instead of $\mathcal{S}^+_D$.
- The parallel transport is used many times and thus should be presented in the section "Riemannian geometry on $\mathcal{S}^+_D$.
- Just before the equation (23): $G_{k-1}$ is written instead of $B_{k-1}$.
limitations: The authors mention the potential of the EEG applications to aid the society. Some references or one/two sentences could be added to justify.
review_title: null

Meta-review: metareview: The reviewers have now unanimously acknowledged the quality of the contribution both on the theory and experimental sides. This paper can be endorsed for publication at NeurIPS 2022.
confidence: Certain
Decision: Accept