SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG
Authors: Reinmar J Kobler
Jun-ichiro Hirayama
Qibin Zhao
Motoaki Kawanabe
Venue: NeurIPS 2022 Accept (2022)
Keywords: neurophysiology
electroencephalography
brain-computer interface
unsupervised domain adaptation
geometric deep learning
interpretability

Abstract: Electroencephalography (EEG) provides access to neuronal dynamics non-invasively with millisecond resolution, rendering it a viable method in neuroscience and healthcare. However, its utility is limited as current EEG technology does not generalize well across domains (i.e., sessions and subjects) without expensive supervised re-calibration. Contemporary methods cast this transfer learning (TL) problem as a multi-source/-target unsupervised domain adaptation (UDA) problem and address it with deep learning or shallow, Riemannian geometry aware alignment methods. Both directions have, so far, failed to consistently close the performance gap to state-of-the-art domain-specific methods based on tangent space mapping (TSM) on the symmetric, positive definite (SPD) manifold.
Here, we propose a machine learning framework that enables, for the first time, learning domain-invariant TSM models in an end-to-end fashion. To achieve this, we propose a new building block for geometric deep learning, which we denote  SPD domain-specific momentum batch normalization (SPDDSMBN). A SPDDSMBN layer can transform domain-specific SPD inputs into domain-invariant SPD outputs, and can be readily applied to multi-source/-target and online UDA scenarios. In extensive experiments with 6 diverse EEG brain-computer interface (BCI) datasets, we obtain state-of-the-art performance in inter-session and -subject TL with a simple, intrinsically interpretable network architecture, which we denote TSMNet. Code: https://github.com/rkobler/TSMNet

SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG
Reinmar Kobler Jun-ichiro Hirayama', Qibin Zhao', Motoaki Kawanabe
RIKEN Center for Advanced Intelligence Project (RIKEN AIP), Tokyo, Japan
2 "dvanncdd Telecommunication: Research Institute International (ATR), Kyoto, Japan
kobler. reinmar@gmaill com, qibin. zhaoorikee,r,e. jun-iching aum-ichiro.hiraymeeee rikan. jp, kawanabe@atr.]
Abstract
Eleetroencephalograapyy (EEG) provides access to neuronal dynamics non- invasively with millisecond resolution, rendering it a viable method in neuroscience and healtheare. However, its utility limited as current EEG technology does not generalize well across domains (i.e., sessions and subjects) without expensive supervised re-calibration. Contemporary methods cast this transfer learning (TL) problem muult-souurce/targee unsupervised domain adaptation (UDA) prob- lem and address with deep learning or shallow, Riemannian geometry aware alignment methods. Both directions have, far, failed consistently close the performance gap to state-of-the space mapping (TSM) the symmetric, positive definite (SPD) manifold. Here, we propose a machine learning framework that enables, for the first time, learn- ing domain-invariant TSM models in end-to-end fashion. To achieve this, we propose a new building block for geometric deep learning, which we denote SPD domain-specific momentum batch normalization (SPDDSMBN). A SPDDSMBN layer can transform domain-specific SPD inputs into domain-invariant SPD outputs, and can readily applied multi-source/-targe and online UDA scenarios. In extensive experiments with 6 diverse EEG brain-computer interface (BCI) datasets, we obtain of-the-r---e-ttttt performance in inter-session and -subject TL with a simple, intrinsically interpretable network architecture, which we denote TSMNet. Code: httpps://githuu.com/// htt.g/thww..ooooooooooo////eeea
Nooainn-speciio methods based on tangent
Introduction
Electroenceephalograahy (EEG) measures multi-channel electric brain activity from the human scalp with millisecond precision [.. Transient modulations in the rhythmic brain activity can reveal cognitive processes (2), affective states [3] and a person's health status [4]] Unfortunately, these modulations exhibit low signal-to-noise ratio (SNR), domain shifts (i.e., changes in the data
distribution) and have low specificity, rendering statistical learning a challenging task particularly in the context of brain-compute interfaces (BCI) [5] where the goal is to predict a target from short segment of multi-channel EEG data real-time.
Under domain shifts, domain adaptation (DA), defined as learning a model from source domain that performs well related target domain, offers principled statistical learning approaches with DA in the BCI field mainly distinguishes inter-session and -subject transfer learning (TL) In inter-session TL. domain shifts, expected across sessions mainly due to mental drifts (low specificity) as well as differences in the relative positioning of the electrodes and their impedances. Inter-subject TL more difficult, as domain shifts are additionally driven by structural and functional differences in brain networks as well as variations in the performed task M. These domain shifts are traditionally circumvented by recording labeled calibration data and fitting domain-specific models IT]] 11 As recording calibration data is costly, models that are robust to
theoretical guarantees 6 7
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

domain-specfii:
b components TSMNet
example outout of (BNCI2015001 datasel)
langenl space mapping (TSM)
domain index
w/o SPDDSMBN
with SPDDSMEN
m
SPDDSMEN
proposed
C
D
R
layers
X
SPDMBNT
fo
m
LogEig
e.g. EEG
SD
R
ISNE 2D projection
target domeins
classes
(sessions 1,2 of subject 7)
domain-specific Frechet mean origin
Figure Visualization of the proposed framework around (a) SPD domain-specifte momentum
batch normalization (SPDDSMBN) that (b) learns parameters t = {0, o, 3 of typical tangent space mapping (TSM) models end-to-end to crack multi-sourcef-tareet unsupervised domain adaptation on ST for EEG data (c. illustrative example). For EEG data, we propose simple, intrinsically interpretable parametrization of f and g. denoted TSMNet, and obtain SoA performance.
scarce data with low SNR perform well in practice. Currently, tangent space mapping (TSM) models 12 operating with symmetric, positive definite (SPD) covariance descriptors of preprocessed data are considered state-of-thes
(SoA) I0 14 [5]. They are well suited for EEG data they exhibit invariances to linear mixing of latent sources [16]. and are consistent [33 and intrinsically
interpretable [17] estimators for generative models that encode label information with log- log-linear relationship in source power modulations.
Competitive, supervised calibration-fre methods are one of the long-lasting grand challenges in EEG neurotechnology research [0
[9 [S]. Among the applied transfer learning techniques, including multi-task learning 20 and Nomain-invariant learning 233 unsupervised domain [C 18
adaptation (UDA) 2 is considered as key to overcome this challenge [10 [9]. Contemporary
methods cast the problem multi source and target UDA problem and address it with deep learning 22 28 or shallow, Riemannian geometry aware alignment methods [2/ Successful methods must cope with notoriously small and heterogeneous datasets (i.e., dozens of domains with a few dozens observations per domain and class). In a recent, relatively large scale inter-subject and -dataset TI. competition with few labeled examples per target domain [19] deep learning approaches that or latent space 64 obtained 15 with smaller dataset, Riemannian
aligned the first and second order statistics either in input 33
27
the highest scores. Whereas, in pure UDA competition geometry aware approaches dominated. With the increasing popularity of geometric deep learning [35]. Ju and Guan (36) proposed an architecture based on SPD neural networks 7 to align SPD features in latent space and attained SoA scores, Despite the tremendous advances in recent years, the field still lacks methods that can consistently close the performance gap to state-of-the-aa Nomain-specifia methods. To close this gap, we propose a machine learning framework around domain-specific batch normaliza- tion on the SPD manifold (Figure The proposed framework is used to implement domain-specific TSM (F) which requires tracking the domains' Fréchet means in latent space they are changing during training typical TSM model in an end-to-end fashion (Figure[b), After review- ing some preliminaries
section we extend momentum batch normalization (MBN) B3 to SPDMBN that controls the Fréchet mean and variance of SPD data in section 3. In a theoretical analysis, we show under reasonable assumptions that SPDMBN can track and converge to the data's true Fréchet mean, enabling, for the first time, end-to-end learning of feature extractors, TSM and tangent space classifiers. Building upon this insight, we combine SPDMBN with domain-specific batch normalization (DSBN) 38 to form SPDDSMBN (p). A SPDDSMBN layer can SPD outputs (Figurelt). Dt). Like DSBN,
transform domain-specific SPD inputs into Nomain-invarian SPDDSMBN easily extends to multi-source, multi-target and online UDA scenarios. In section 4, we briefly review the generative model of EEG, before the proposed methods are combined in a simple, intrinsically interpretable network architecture, denoted TSMNet (Figure We obtain state-of- the- art performance in netersession and -sujject UDA on small and large scale EEG BCI datasets, and show in ablation study that the performance increase is primarily driven by performing DSBN on the SPD manifold.
2 Preliminaries
Multi-source multi-target unsupervised domain adaptation Let X' denote the space of input features, y a label space, and Ia N an index set that contains unique domain identifiers. In

---

the multi-source, source, multi- target unsupervised domain adaptation scenario considered here, we are given a set Tsouree {Tili E Igguree C Id} with [ N domains. Each domain T; Pxy contains M observations of feature (X E X)) and label (y )) tuples sampled from a joint distribution P While the joint distributions can be different (but related) across domains, we assume that the class priors are the same (i.c.. P = Py). The goal is to learn predictive function h X x Ia + y/ that, once fitted to Tsource, can generalize to unseen target domains Tharget {Till E I C Ia. Iaarget n Igouree = 0} merely based on unsupervised adaptation of h to each target domain Ti once its label i and features (X1,) P are revealed.
((Xi).B1))) b1y)
Riemannian geometry on ST We start with recalling notions of geometry on the space of real D x D symmetric positive definite (SPD) matrices S {Z E RDDD z = Z, Z > 0}. The space S forms a cone shaped Riemannian manifold in R Dx 39 A Riemannian manifold M is smooth manifold equipped with an inner product on the tangent space TzM at each point Z E M. Tangent spaces have Euclidean structure with easy to compute distances Tz.M x Tz.M - R which locally approximate Riemannian distances on M induced by an inner product [0) Logarithmic
Logz: .M- + Tz.M and exponential Expz .z.M- - . mappings project points to and from tangent spaces.
Using the inner product (2,,S2)z = (i.e., the space of real symmetric D D matrices) results in a globally defined affine invariant Riemannian metric on S 41 39 which can be computed in closed form:
for points S1, S2 in the tangent space T
ÓAIRM (Z, Z2) = I|log(2, 2 ZZZ )[FF
(I)
where Z₁ and Zg are two SPD matrices, log(-) denotes the matrix logarithm || ||r the Frobe- nius norm, and Tr(.) in the inner product the trace operator. Due to affine invariance, we have 8(AZ, A", AZzA") = ö(Z,, Σ2) for any invertible D x D transformation matrix A. The expo- nential and logarithmic mapping are also globally defined in closed form as
Logz (Z1) Z² log ZZ
Z Z
Expz(S1) Z 2 exp(Z s $ jz
(3)
For set of SPD points 2 = =Z, Si)js.M will use the notion of Fréchet mean Gz E S
and Fréchet variance v2 E RR+ The Fréchet mean is defined as the minimizer of the average squared distances
1 M
G2 = arg min GESP M
Σ (G
Z,)
(4)
J=1
For M = 2, there is closed form solution expressed as
) 7 z Gz(7) = Z, # Ze z ( Z2
(5)
with weight 7 = 0.5. Choosing 7 E [0, 1] computes weighted means along the geodesic (i.e., the shortest curve) that connects both points. For M > 2, 4 can be solved using the Karcher flow algorithm 42), which iterates between projecting the data to the tangent space 2 at the current estimate, arithmetic averaging, and projecting the result back B to obtain new estimate. The Fréchet variance v² is defined as the attained value at the minimizer Gz:
M
2/ = Varz(Gz)= =
1
Σ
ATRR (G2,Z,)
M
J=1
To shift set of tangent space points to vary around a parametrized mean Go, parallel transport on S can be used 4
(S)
E = =GZ 3
While, parallel transport is generally defined for tangent space vectors S 40 on Sp the same operations also apply directly to points on the manifold (i.e., Z E Z) BI
I For ease of notation, although not required hy our method, we assume that M the same for each domain.
For SPD matrices, powers, logarithms and exponentials can be computed via eigen decomposition.
3

---

Table 1: Overview and differences of relevant batch normalization algorithms. The last column, denoted normalization, sumarizes which statisties are used to normalize the batch data during training.
Acronym
domain -peaii-
momentum Y
normalization
MBN
32
n10
no adaptive
running stats running stats
SPDBN
yes
no fixed adaptive
SPDMBN (algorithm proposed
yes
no
running stats
DSBN
R
yes
fixed
batch stats
SPDDSMBN
18 proposed
yes
yes
adaptive
running stats
3
Domain-specific batch normalization on S
In this section, we review relevant batch normalization (BN) 45 variants with focus on S We then present SPDMBN and show in a theoretical analysis that the running estimate converges to the true Fréchet mean under reasonable assumptions. At last, we combine the idea of domain-speciff batch normalization (DSBN) (38] with SPDMBN to form SPDDSMBN layer. Aableeprooidee brief overview of related and proposed methods.
Batch normalization Batch normalization (BN) 45 is widely used training technqiue in deep learning as BN layers speed up convergence and improve generalization via smoothing of the engery A standard BN layer applies slightly different transformations during training and testing to independent and identically distributed (iid) observations xj E IR within the k-th minibatch 8& of size M drawn from dataset T. During training, the data normalized using the batch mean by and variance si. and then scaled and shifted to have parametrized mean gú and variance a- Internally, the layer updates running estimates of the dataset's 's statistics (gx, o during each training step k; the updates are computed via exponential smoothing with momentum parameter y. During testing, the running estimates are used.
landscape 47 [2].
Using batch statistics to normalize data during training rather than running estimates introduces noise whose level depends on the batch size [32]; smaller batch sizes raise the noise level. The introduced noise regularizes the training process, which can help to escape poor local minima initial learning but also lead to underfitting. Momentum BN (MBN) 32 allows small batch sizes while avoiding underfitting. Like batch renormalization +
MBN uses running estimates during training and testing. The key difference is that MBN keeps two sets of rumning statistics; one for training and one for testing. The latter are updated conventionally, while the former are updated with momentum parameter "ttrin (k) that decays over training steps k. MBN can, therefore, quickly escape poor local minima during initial learning and avoid underfitting at later stages 32.
Batch normalization on SD is intractable to compute the Fréchet mean Gs, for each minibatch B2 (z, E 1 as there is no efficient algorithm to solve (4). Brooks et al. 44! proposed Riemannian Batch Normalization (RBN) tractable approximation. RBN approximateley solves 4 by aborting the iterative Karcher flow algorithm after one iteration. To transform Zj E Bg with estimated mean B, to vary around G parallel transport 7 is used. The RBN input output transformation is then expressed as
RBN Go, 7) TBB.-G. -
GB)
vZ, BB
(8)
Using 5). the rumning estimate of the dataset's Fréchet mean can be updated in closed form
G , B
(9)
In 46 we proposed an extension to RBN, denoted SPD batch renormalization (SPDBN) that controls both Fréchet mean and variance. Like batch renormalization 48, SPDBN uses running estimates G and v during training and testing. To transform Z, E B3 to vary around Go with variance D3 each observation is first transported to vary around the identity matrix I, rescaled via computing matrix powers and finally transported vary around Gg. The sequence of operations can be expressed as
SPDEN(Z;: G vg. = TI-G, o (G.-1(ZZ))
AZ E B
(10)
The standard backpropagation framework with extensions for structured matrices 49 and manifold- constrained gradients [40) can be used to propagate gradients through RBN and SPDBN layers and leam the parameters (G V.)).

---

Algorithm 1: SPD momentum batch normalization (SPDMBN)
Input :batch B& {Z, E S training step
running mean -1 Go I and variance 02 Pn 1 for training,
running mean G Go I and variance 172 11, 1/5 = 1 for testing,
learnable parameters (G,,Vo) and momentum for training and testing Ytram (k) [0,
Output: noomatind batch {Z,- = SPDMBN(Z,) E SD Z, B&
training then
B₃ karcher_ffood (Bx, stepe = 1);
approx. batch Fréchet mean
G G Hemee (1) Be:
// update running stats
P (1 Praai (k)) + Terain (k) Vars, (Gx)
Gı Wr-I
o = (1 - 7)0k + Brii++Vaan Vars, (G&)
end
(Ge, [ = (Gr, if training else (G2, (22)
2, = TI-G, Ia,-)(Z,) u // parallel transport to whiten, rescale and rebias
Momentum batch normalization on SD SPDBN 46 suffers from the same limitations as batch
renormalization 48. Consequently, we propose to extend MBN 32 to S We list the pseudocode of proposed extension, which we denote SPDMBN, in algorithm I SPDMBN uses approximations of batch-specific Fréchet means to update two sets of running estimates of the dataset's Fréchet mean. As MBN 32 we decay Aroai (l) with a clamped exponential decay schedule
max kk.0)
Itrai (k)
7in
Ymin
(11)
where K defines the training step which Yimin [0, 1] should be attained.
The running mean in SPDMBN converges to the Fréchet mean Here, we consider models that apply a SPDMBN layer to latent representations generated by a feature extractor so X' + S with learnable parameters 8.
We define a dataset that contains the latent representations generated with feature set 0g as Zo,, {Se, (x)|x E T). and minibatch of M iid samples at training step k as Bg We denote the Fréchet mean of Zee as G the estimated Fréchet mean, defined in ()) as Gu, and the estimated batch mean as Bg. Since the batches are drawn randomly, we consider the batch and running means as random variables.
We assume that the variance Vare, (Ba-1) = ( (B Go.) } of the previous batch mean Bą-1 1 with respect the current Fréchet mean Gs. bounded by the current variance Varg, (Bş) and the norm of the difference in the parameters
Vare, (Bx-1) s (1 + ||0, 1/-1/))aar,, (Bs)
(12)
That is, across training steps & the parameter updates are required to change the first and second order moments of the distribution of Ze, gradually so that the expected distance between Ga, and the previous batch mean B bounded. We conjecture that this is the case for feature extractors fo that are smooth in the parameters and small learning rates, but leave the proof for future work.
Proposition (Error bound for G&). Consider the setting defined above, and assumption [2 true. Then, the variance of the running mean Vare, (Gx) is bounded by
holds
Vare, S Vare. (Bx)
(13)
over training steps lo if
5 1- 72
1 02 10-1l
(14)
(1 7)2
holds true.
The proof is provided in appendix A. of the supplementary material and relies on the proof of the geometric law of large numbers (50).
Proposition states that 12 and (1 are met, the expected distance between the true Fréchet mean
and the running mean is less equal to the one of the batch mean. Consequently, the introduced
5

---

noise level of SPDBN (equation TO and SPDMBN (algorithm DT which use Gş to normalize batches during training, is smaller or equal to RBN (equation which uses Bg-
Since "y controls the adaptation speed of Gg. proposition also states that if y converges to zero (=no adaptation), the parameter updates are required to converge to zero as well (=no learning). Hence, for a fixed y E (0, 1), as in the case of SPDBN (equation 10 proposition is fulfilled, if the learning rate for the parameters is chosen sufficiently small. This can substantially slow down initial learning for standard choices of (e.g.. 0.1 or 0.01). As a remedy, SPDMBN (algorithm I uses an adaptive momentum parameter, which allows larger parameter updates during initial training steps.
If we consider a late stage of learning, and in particular assume that after a certain number of iterations the parameters stay small ball with radius p around 9 (i.e., 00 11 S p A k > x) and the feature extractor is L-smooth in the parameters (i.e., 6(fo(x), (x)) S L| Ø|| Vx ET, V0,0) then the distances are bounded ((fe, (x); fo- (x)) - pL.
Remark 1 (Convergence of Gy for SPDMBN), If pL is neglibile compared to the dataset's variance, then the Fréchet mean and variance can be considered fixed, and the theorem of large numbers on Sh [50] applies directly. That is, if the momentum parameter is decayed exponentially Vk: > is the running mean Gj: converges to the Fréchet mean G in probability as k - Dc.
Taken together. Proposition D and Remark provide guidelines to update G2 in SPDMBN so that the introduced estimation error bounded during initial fast learning (large ?) and decays towards zero in late learning (small 7)-
SPDMBN to learn tangent space mapping at Fréchet means Typical TSM models for classifica- tion I2 and regression [13 first use Q to project Z T c S to the tangent space at the Fréchet mean GT, then use 7 to transport the result to vary around I, and finally extract elements in the up- per triangular par to reduce feature redundancy. The invertible mapping Par S - R D(D+1)/2
is expressed as:
Par (Z) = upper o [ o Loga, (Z) = uppor(log( G G
(15)
37 to compute a similar mapping mz (Figure b). A LogEig layer simply computes the matrix logarithm and vectorizes the result so that the norm is preserved. If the parametrized mean of SPDMBN is fixed to the identify matrix (G, = I), the composition computes
We propose to use SPDMBN layer followed by a LogEig layer
m (Z) LogEig e SPDMBN(Z) = -upper o log o
Vo Vo_log log (G.
upper
(16)
V/ +
where (Gk. v are the estimated Fréchet mean and variance of the dataset T at training step k and @ {vo} the learnable parameters. According to remark | G converges to G. and, in turn, ms to a scaled version of Pay, since upper is linear.
The mapping my offers several advantageous properties. First, the features are projected to a Euclidean vector space where standard layers can be applied and distances are cheap compute. Second, distances between the projected features locally approximate 8 and, therefore, inherit its invariance properties (e.g., affine mixing) 41 This improves upon LogEig layer (37] which projects features to the tangent space at the identity matrix. As result, distances between LogEig projected features correspond to distances measured with the log-Fuclle Fuclidean Riemannian metric (LERM) (51) which is not invariant to affine mixing. Third, controlling the Fréchet variance in 16 empirically speeds up learning and improves generalization 46
Domain-specifie batch normalization on ST Considering multi-source UDA scenario, Chang et al. [3 proposed domain-speciin BN (DSBN) layer which simply keeps multiple parallel BN layers and distributes observations according to the associated domains. Formally, we consider minibatches B, that form the union of NB. < [Za] domain-specific minibatches B; drawn from distinct domains i E IB2 C Ig. As before, each Bj, contains j = 1,..., M/NB, iid observations xj. DSBN layer mapping Rd x In + Edd can then be expressed as
DSBN(x, i) = BN, (xy: Bo. S E, 7, Vx, E B Vi E Is.
(17)
To preserve the norm, the off diagonal elements scaled by v2.

---

In practice, the batch size M is typically fixed. The particular choice is influenced by resource availability and the desired noise level introduced by minibatch based stochastic gradient descent. A drawback of DSBN is that for fixed batch size M and an increasing number of source domains NB. the effective batch size declines for the BN layers within DSBN. Since small batch sizes increase the noise level introduced by BN, increasing the number of domains per batch can lead to underfitting [22] To alleviate this effect, we use the previously introduced SPDMBN layer. The proposed domain-specifie BN layer S is then formally defined as
SPD = SPDMBN, (Zy; G Vol. E, Y, Itrain (krain(k)) VZ, E B C C VI E IB,
(18) The layer can be readily adapted to new domains, as new SPDMBN layers can be added on the fly. If the entire data domain becomes available, the domain-speciffe Fréchet mean and variance can be estimated by solving 4. otherwise, the update rules in algorithm |
can be used.
4 SPDDSMBN to crack interpretable multi-sourcce/targel UDA for EEG data
With SPDDSMBN introduced in the previous section, we focus on a specific application domain, namely, multi-source//ae -tareet UDA for EEG-based BCIs and propose an intrinsically interpretable architecture which we denote TSMNet.
Generative model of EEG EEG signals x(t) Rº capture voltage fluctuations on P channels. An EEG record (=domain) is uniquely identified by a subject and session identifier. After standard pre-processing steps, each domain contains Xiy E C PxT where T is the number of temporal samples. Due to linearity of Maxwell's equations and Ohmic conductivity of tissue layers in the frequency ranges relevant for EEG [52]. domain-specific linear instantaneous mixture of sources model is valid generative model:
1,...; M labeled observations with features
X S. + Noj
(19)
where Sij E RxT represents the activity of Q latent sources, A, E PxQ a domain-specific mixing matrix and Núj E PxT additive noise. Both A, and Siy are unknown which demands making assumptions A; (e.g., anatomical prior knowledge 533 and/or Sig (e.g., statistical independence to extract interesting sources.
(54)
Interpretable multi-source/-targe UDA for EEG data As label information is available for the source domains, our goal is identify discriminative oscillatory sources shared across domains. Our approach relies on TSM models with linear classifiers (12). as they are consistent [3 and intrinsically interpretable [17 estimators for generative models with log-linear relationships between
the target Vay and variance Var(s
(k) (1)) of k = 1,.., K < Q discriminative sources:
K
yA Σ
balog (Var(s! (K) (6)) Ei
(20)
where by E IR summarizes the coupling between the target Diy and the variance of the encoding source, and Esj additive noise. In [1 we showed that the encoding sources' coupling and their pattern p (columns of A,) can be recovered via solving generalized eigenvalue problem between the Fréchet mean Gy; and classifier patterns resulting eigenvectors are the patterns and the eigenvalues dς reflect the relative source contribution c::
(55) that were back projected to S with Pot The
Ck
A=1), λs = exp(bx/llb]p
(21)
To benefit from the intrinsie interpretability of TSM models, we constrain our hypothesis class H. to functions h X Ia + y that can be decomposed into composition of shared linear feature extractor with covariance pooling fe X' + ST. domain-specific tangent space mapping m. St x Ia 1(D+1)/2 and a shared linear classifier 9v R D(D+1)/2 + y with parameters θ (0, Φb 1b).
'Here, we use the entire dataset's Fréchet mean instead of the domain- specifc ones to compute patterns for the average domain.
7

---

TSMNet with SPDDSMBN Unlike previous approaches which learn fa. ma 9 sequentially we parametrize h Is ma o fo as a neural network and learn the entire model in an end-to-end fashion (Figure[Tb) Details of the proposed architecture, denoted TSMNet, are provided in appendix A.2.5. In nutshell, we parametrize fo as the composition of the first two linear convolutional layers of ShConvNet [56,, covariance pooling [57 BiMap 3v,, and ReEig Sr layers. A BiMap layer applies a linear subspace projection, and ReEig layer thresholds eigenvalues of symmetric matrices so that the output is SPD. We used the default threshold (10-4) and found that it was never active in the trained models. Hence, after training, fo fulfilled the hypothesis class constraints. In order for my to align the domain data and compute TSM, we use SPDDSMBN (18) with shared parameters (i.e., Go. = G = I. Ve, = Vo) in (I6). Finally, the classifier ge was parametrized as linear layer with softmax activations. We use the standard-cross entropy loss as training objective, and optimized the parameters with the Riemannian ADAM optimizer (58).
29 61
60.
5 Experiments with EEG data
In the following, we apply our method to classify target labels from short segments of EEG data. We consider two BCI applications, namely, mental imagery [59 5 and mental workload estimation (0) Both applications have high potential to aid society in rehabilitation and healthcare [1] 12 18 but have, currently, limited practical value because of poor generalization across sessions and subjects
19 15
Datasets and preprocessing The considered mental imagery datasets were BNC12014001 x
subjects/2 sessions/4 classes), BNC12015001 (64] (12/2-3/2),
Lee2019 65 (54/2/2), Lehner2020
666
(1/7/2). Stieger2021 [67 (62/4-8/4) and Hehnberger2021 68 (1/26/4). For mental workload
estimation, we used recent competition dataset [69 (12/2/3). A detailed description of the datasets
is provided in appendix [.2.I Altogether, we analyzed a total of 603 sessions of 158 human subjects
whose data acquired in previous studies that obtained the subjects' informed consent and the right to share anonymized data.
The python packages moabb [14] and mne 70 were used to preprocess the datasets. The applied steps comprise resampling the EEG signals to 250/256 Hz, applying temporal filters to extract oscillatory EEG activity in the 4 to 36 Hz range (spectrally resolved if required by method) and finally extract short segments (S 3s) associated to class label (details provided in appendix (A.2.2).
Evaluation We evaluated TSMNet against several baseline methods implementing direct transfer or multi-source (-target) UDA strategies. They can be broadly categorized as component based 771 68) Riemannian geometry aware 12
7 25 All models were fit and evaluated with a randomized leave 5% of the sessions (inter-session TL) or subjects (inter-subject TL) out cross-validation (CV) scheme. For inter-session TL. the models were only provided with data of the associated subject. When required, inner train/test splits (neural nets) or CV (shallow methods) were used to optimize hyper parameters (c.g.. early stopping, regularization parameters). The dataset Hehenberger2021 was used to fit the hyper parameters of TSMNet, and is, therefore, omitted in the presented results. Balanced accuracy (i.c., the average recall across classes) was used as scoring metric. As the discriminability of the data varies considerably across subjects, we decided report the results in the figures relative to the score of a SoA domain-speciff Riemannian geometry aware method [77, which was fitted and evaluated in 80% chronological train/test split (for details see appendix A.2.4).
L7
50 72 or deep learning 55
Soft- and hardware We either used publicly available python code or implemented the methods in python using the packages torch 774), scikit-learn 75). skorch 76), geoopt 77. mne 770,. pyriemann 1781. pymanopt [79]. We ran the experiments on standard computation PCs equipped with 32 core CPUs with 128 GB of RAM and used up to GPU (24 GRAM). Depending on the dataset size, fitting and evaluating TSMNet varied from a few seconds to minutes.
5.1 Mental imagery
TSMNet closes the gap to domain-specific methods Figure summarizes the mental imagery results. It displays test set scores of the considered TL methods relative to the score of a SoA domain-specific reference method. Combining the results of all subjects across datasets (Figure becomes apparent that TSMNet is the only method that can significantly reduce the gap to the reference method (inter-subject) or even exceed its performance (inter-session). Figure displays the results resolved across datasets (for details see appendix A.3.T. We make two important observations. First, concerning inter- session TL, TSMNet meets or exceeds the score of the reference method
8

---

a
inter-session
suter-subject
b
inter-session
Inter-subject
A balarceed accuracy (%)
A balanced accurary (%)
TOCSP+OSS+LDD
20
URPA MDM
NEG
proposed
-10-
2111
-213
-12
20
FBCSP+DSSH DDD
SPDOT- SPDOT-15M- TSMM+ SWM
H16
FBCSP DSS LDA
SPDOT - TSM+ SVM
ShCurvitee++ DANN
20
EBGNeL+ DANN
SiCon
EBGNet- DANN
FBCSP
1SW-SVM
TD=TSM-LR
EBGNet
Sh Carviieeee
URPA+ MDM
propodee
FBCSP SVM
TSK TSK -SK
FB+
EEGNet
ShCanvitee
URPA+MDN
proposed
-40
BNC12014001 Stiecer2021
BNC22050001
2021
Lec2019
BNC1201
5001 DNC12014001
Lee019
category
component
geametric
proposed
end-to-end
transfer
MS(MT)JDA
direct
50
training
70
130
570
observations per class)
1.1k
2.6k 10k 12k
Figure Mental imagery results (5 datasets). BCI test set score (balanced accuracy) for sesrrosession-
subject transfer learning methods relative to SoA domain-specific reference model (80%/20%
chronological train/test split: for details see appendix A.2.4). a. Barplots summarize the grand average (573 sessions, 138 subjects) results. Errorbars indicate bootstrapped (1e3 repetitions) 95% confidence intervals (over subjects). b, Box and scatter plots summarize the dataset-specific results for selected methods from each category. Datasets are ordered according to the training set size. Each dot summarizes the score for one subject. Lehner2021 is not displayed as it contains only subject.
motor imagery (feet vs. right hand)
mental workload (inw medium high)
C contribution
source
Index
magnitude (dB)
channnel
spectral
spectral 1 channhel 2 channnel 3 channnel
spectral spectral
4
1.1 1:3
source index
magnitude (cB) 3D
spectral channnel
spectral spectral spectral channnel
channnel
channnel
500 30
1000 1
50
50 frequency (Hz)
50
50
500
500 frequency (Hz)
500
50
source
magnitude (dB)
0
Peeonnzuo (dB)
-25
50
source
50 0 50
source
-2523
absmax Figure 3: Model interpretation results. Patterns extracted from TSMNet. a, Motor imagery dataset (BNCI2015001, inter- subjett TL). The top, left panel lists the contribution, defined in (2T). for each extracted source k 1.... 20 (x-axis) to the target class. Panels in the left column summarize the spectral patterns of extracted sources. For visualization purposes, only the 2 most discriminative sources are displayed. Panels in the top row summarize the frequency profile of each spectral channel (output of temporal convolution layers in fo). Topographic plots summarize how the source activity projected regions covered by the EEG channels (rows correspond to the source index; columns to spectral channels). EEG channels at darker blue or red areas capture more source activity and are, therefore, more discriminative. b, As in a for the mental workload estimation dataset and class low.
frequency (Hz)
-bssax
frequency (H2)
consistently across datasets. Second, concerning inter-subject TL., we found that all considered methods tend to reduce the performance gap as the dataset size (# subjects) increases, and that TSMNet is consistently the top or among the top methods. As a fitted TSMNet corresponds to a typical TSM model with linear classifier, we can transform the fitted parameters into interpretable patterns 177. FigureBp displays extracted patterns for the BNCI20 5501 dataset (inter-subject TL). It clearly visible that TSMNet infers the target label from neurophysiologicall plausible sources (rows in FigureBpu As expected (2]. the source with highest contribution has spectral peaks in the alpha and beta bands, and originates in contralateral and central sensorimotor cortex.
DSBN on S drives the success of TSMNet Since TSMNet combines several advances, we present the results of an ablation study in Table It summarizes the grand average inter-session TL test scores relative to TSMNet with SPDDSMBN for n 138 subjects. We observed three significant effects. The largest effect can be attributed to S as we observed the largest performance decline if the architecture would be modified to omit the SPD manifold (4.5% with DSBN, 3% w/o DSBN).
'We replaced the covariance pooling, BiMap. ReEig, SPD(DS)MBN, LogEig layers with variance pooling, elementwise log activations followed by (DS)MBN. Note that the resulting architecture similar to ShConvNet.

---

Table 2: Ablation study. Grand average (5 mental imagery datasets, 138 subjects, inter-session TL) score for the test data relative to the proposed method, and training fit time (50 epochs). Standard- deviation is used to report the variability across subjects. Permutation t-tests (1e4 perms, df=137, 4 tests with t-max adjustment) were used to identify significant effects.
A balanced accuracy (%) mean (std)
fit time (s) mean (std)
t-val (p-val)
DSBN [1]
BN method
yes
yes
SPDMBN (proposed)
16.9 ( 1.0)
yes
SPDBN [4]
-1.6
2.2)
-7.8 (0.0001)
20.3 ( 1.6)
SPDMBN (proposed)
-3.9
4.4)
-10.7 (0.0001)
11.3 0.5)
no
yes
MBN [2]
-45
3.8)
-10.1 (0.0001)
6.6 ( 0.2)
no MBN [2]
-6.9
4.8)
-13.4 (0.0001)
4.4 ( 0.1)
The performance gain comes at the cost of a 2.6x longer time to fit the parameters. The second largest effect can be attributed to DSBN; without DSBN the performance dropped by 3.9% (with S and 2.4% (w/o ST)) The smallest, yet significant effect can be attributed SPDMBN.
5.2 Mental workload estimation
Compared to the baseline methods, TSMNet obtained the highest average scores of 54.7% (7.3%) and 52.4% (8.8 (8.8%) ) in inter-session and -subject TL (for details see appendix [A..II] Interestingly, the inter-session TL score of TSMNet matches the score (54.3%) of the winning method in last year's competition
15 To shed light on the sources utilized by TSMNet, we show patterns for a fitted model in Figure For the low mental workload class, the top contributing source's 's activity peaked in the theta band and originated in pre-frontal areas. The second source's s activity originated in occipital cortex with non-focal spectral profile. Our results agree with the findings of previous research, as both areas and the theta band have been implicated in mind wandering and effort withdrawal [60]
6 Discussion
In this contribution, we proposed machine learning framework around (domain-specific) momentum batch normalization on S to learn tangent space mapping (TSM) and feature extractors in an end-to-end fashion. In a theoretical analysis, provided error bounds for the rumning estimate of the Fréchet mean as well as convergence guarantecs under reasonable assumptions. We then applicd the framework, to a multi-source multi-target unsupervised domain adaptation problem, namely, inter-session and -subject transfer learning for EEG data and obtained or attained state-of-the art performance with simple, intrinsically interpretable model, denoted TSMNet, in total of 6 diverse BCI datasets (138 human subjects, 573 sessions). In the case of mental imagery, we found that TSMNet significantly reduced (inter-subject TL) even exceeded (inter-session TL) the performance gap to a SoA domain-specific method.
Although our framework could be readily extended to online UDA for unseen target domains, limited this study to offline evaluations and leave actual BCI studies to future work. A limitation of framework, and also any other method that involves eigen decomposition, is the computational complexity, which limits its application to high-dimensional SPD features (e.g.. fMRI connectivity matrices with fine spatial granularity). Altogether, the presented results demonstrate the utility of our framework and in particular TSMNet as it not only achieves highly competitive results but is also intrinsically interpretable. While we do foresee any immediate negative societal impacts, we provide direct contributions towards the scalability and acceptability of EEG-based healthcare [5] and consumer [T8. 60 technologies. We expect future works to evaluate the impact of the proposed methods in clinical applications of EEG like sleep staging [0 seizure
32 or pathology
detection
Acknowledgments and Disclosure of Funding
This work was supported by the JSPS KAKENHI (Grants-in-Aid for Scientific Research) under Grant Numbers 20H04249, 20H04208, 203555 and 21K 12055, and the Agency for Medical Research and Development (AMED) under Grant Number JP22dm0307009.
10

---

References
[1] Donald L.. Schomer and Fernando H. Lopes da Silva. Niedermeyer's Elletreencephhlogrpphyy basic principles, clinical applications, and related fields. Lippincott Williams & Wilkins, edition, 2018. ISBN 978-0-19-002848-4, doi: 10. 0.1212/WNL.000113111222222
[2] G Pfurtscheller and F H Lopes. Event-related EEG MEG synchronization and. Clinical Neurophysiology. 1 10:105564799 1999.
[3] Josef Faller, Jennifer Cummings, Sameer Saproo, and Paul Sajda. Regulation of arousal via online neurofeedback improves human performance in a demanding sensory-motor task. Proceedings of the National Academy of Sciences, (6((1))6482-6490) 2019. doi: 10.1073
pnas. 10ss.181110 817207116
[4]
Yu Zhang, Wei Wu, Russell T. Toll, Sharon Naparstek, Adi Maron-Katz, Mallissa Watts, Joseph Gordon, Jisoo Jeong, Laura Astolfi, Emmanuel Shpigel, Parker Longwell, Kamron Sarhadi, Dawlat El-Said, Yuanqing Li, Crystal Cooper, Cherise Chin-Fatt, Martijn Arns, Madeleine Goodkind, Madhukar H. Trivedi, Charles R. Marmar, and Amit Etkin. Identification of psychi- atric disorder subtypes from functional connectivity patterns resting- state electroenceehaalggra phy. Nature Biomedical Engineering, 5(4):309-323, 2021. doi: 10.1038/s41551-0200 14.000064440 4-88.
[5] Jonathan R Wolpaw, Niels Birbaumer, Dennis J McFarland, Gert Pfurtscheller, and Theresa M Vaughan. Brain- Brain-computer interfaces for communication and control. Clinical neurophysiology official journal of the International Federation of Clinical Neurophysiology. 113:767-791, 2002. doi: doi: 10111001111888 11011000661338224700000001----70
[6] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79
11-2):151-175, 2010. doi: 10.1007/s109944000 -09-555---09-555--4..
[7] Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. Algorithms and theory for multiple- source adaptation. In S. Bengio, Wallach, Larochelle, Grauman, N. Cesa- Desa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
[8] Dongrui Wu, Yifan Xu, and Bao-Liang Lu. Transfer Learning for EEG-Based Brain-Computer Interfaces: A Review of Progress Made Since 2016. IEEE Transactions on Cognitive and Developmental Systems, pages 1-1, 2020. doi: 10. 10.100//CDS
[9] Christa Neuper, Reinhold Scherer, Miriam Reiner, and Gert Pfurtscheller. Imagery motor actions: Differential effects of kinesthetic and motor mode of imagery in single-tria EEG. Cognitive Brain Research, 25()):668-677 2005. doi: 10.10166f.oogbbrairee.20000 08.014.
[10] F Lotte, L Bougrain, A Cichocki, M Clere, M Congedo, A Rakotomamonjy, and F Yger. A review of classification algorithms for EBG-based brain-computer interfaces: a 10 year update. Journal of Neural Engineering, 15(3):031000 1005, 2018 8. doi: (..1088/1741-2552aaabl
[11] Karina Statthaler, Andreas Schwarz, David Steyrl, Reinmar J. Kobler, Maria Katharina Höller, Julia Brandstetter, Lea Hehenberger, Marvin Bigga, and Gernot Müller-Putz. Cybathlon experi- ences the Graz BCI racing team Mirage91 in the brain-computer interface discipline. Journal of NeuroEngineering and Rehabilitation, 14(1):129, 2017. doi: 10. 186/s 1984-017-03344449
[12] Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. Multiclass brain- computer interface classification by Riemannian geometry. IEEE transactions bio-medical engineering, 59(4))920-97) 928, 2012. doi: 10.1 10.11199/BME.2211.22222
[13] David Sabbagh, Pierre Ablin, Gaël Varoquaux, Alexandre Gramfort, and Denis A Engemann. Manifold-regressionn predict from MEG/EEG brain signals without source modeling. In Advances in Neural Information Processing Systems, pages 7323-7334, 2019.
[14] Vinay Jayaram and Alexandre Barachant. MOABB: trustworthy algorithm benchmarking for BCIs. Journal of Neural Engineering, 15(6)1066011, 2018. doi: 1e..188//741-2552/adde

---

[15] Raphaëlle N. Roy, Marcel F. Hinss, Ludovic Darmet, Simon Ladouce, Emilie S. Jahanpour, Bertille Somon, Xiaoqi Xu, Nicolas Drougard, Frédéric Dehais, and Fabien Lotte. Retrospec- tive on the First Passive Brain-Computer Interface Competition Cross-Session Workload Estimation. Frontiers in Neuroeronoomics, 3:838342, 2022. doi: 2..3889ffrrgo.2222.88880
[16] Marco Congedo, Alexandre Barachant, and Rajendra Bhatia, Riemannian geometry for EEG- based crain-computer interfaces; a primer and review. Brain Crain-Computer Interfaces. 4(3): 155-174, 2017.
doi: 10.1080/23222633.2117..299
[17] Reinmar J. Kobler. Jun-Ichiro Hirayama, Lea Hehenberger, Catarina Lopes-Dias, Gernot Müller- Putz, and Motoaki Kawanabe. On the interpretation of linear Riemannian tangent space model parameters in M/EEG. In Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2021. doi: 10. 16.1E994144441 1930146..
[18] Stephen H. Fairelough and Fabien Lotte. Grand Challenges in Neurotechnology and System Neuroergonomies. Frontiers in Neuroergonomics, 1:602504, 2020. doi: 10.3389/fnrgo.2020 602504.
[19] Xiaoxi Wei, A. Aldo Faisal, Moritz Grosse Grosse-Wentrup, Alexandre Gramfort, Sylvain Chevallier, Vinay Jayaram, Camille Jeunet, Stylianos Bakas, Siegfried Ludwig, Konstantinos Barmpas, Mehdi Bahri, Yannis Panagakis, Nikolaos Laskaris, Dimitrios A. Adamos, Stefanos Zafeiriou, William C. Duong, Stephen M. Gordon, Vernon J. Lawhern, Maciej Sliwowski. Vincent Rouanne, and Piotr Tempezyk. 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets. arXiv:2202. 12950 [cs, eess), 2022.
[20] Vinay Jayaram, Morteza Alamgir, Yasemin Altun, Bernhard Scholkopf, and Moritz Grosse- Wentrup. Transfer Learning in Brain-Computer Interfaces. IEEE Computational Intelligence Magazine, 11(1):20-31, 2016. doi: 10.1109/MC22011255001
[21] Siamac Fazli, Florin Popescu, Márton Danóezy, Benjamin Blankertz, Klaus-Rober Müller, and Cristian Grozea. subject-independen mental state classification in single trials. Neural Networks, 220)):330-11312 2009. doi: 10.101 (016/..neuuet.2000 0.000.60033...
[22] Wojciech Samek, Motoaki Kawanabe, and Klaus Robert Müller. Divergence- framework for common spatial patterns algorithms. IEEE Reviews in Biomedical Engineering, 7:50-72,
2014. doi: 10. 2009RRBM..001322900000
[23] O-Yeon Kwon, Min-Ho Lee, Cuntai Guan, and Seong-Whan Lee. Subject-Independent Brain-Comee Ceaip-Computer Interfaces Based on Deep Convolutional Neural Networks. IEEE Trans- actions on Neural Networks and Learning Systems, 31(10):3839-33) 3852, 2020. doi: 10.1109/ TNNLS.2019.2946866
[24] Sicheng Zhao, Bo Li, Colorado Reed, Pengfei Xu, and Kurt Keutzer. Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey, 2020.
[25] Ozan Ozdenizei, Ye Wang. Toshiaki Koike-Akino, and Deniz Erdogmus. Learning Invariant Representations From EEG via Adversarial Inference. IEEE Access, 8:27074-27085, 2020. doi: 1011099//CESSS.222.0.. 2971600.
[26] Lichao Xu, Zhen Ma, Jiayuan Meng, Minpeng Xu, Tzyy-Ping Jung, and Dong Ming. Improving Transfer Performance of Deep Learning with Adaptive Batch Normalization for Brain-computer Interfaces In 202/ 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC). pages 5800-5803, Mexico, 2021. IEEE. ISBN 778-1-72811-17941 doi: 10.1 1090EBB44CC (4..CCC116442222 64.2002..6.9999., 9629529.
[27] Lichao Xu, Minpeng Xu. Yufeng Ke, Xingwei An, Shuang Liu, and Dong Ming. Cross-Datare Variability Problem EEG Decoding With Deep Learning. Frontiers Human Neuroscience. 14:103, 2020. doi: 223899/fhum.2222000000
[28] Ravikiran Mane, Effie Chew, Karen Chua, Kai Keng Ang, Neethu Robinson, A. P. Vinod, Seong-Wher Lee, and Cuntai Guan. FBCNet: A Multi-view Convolutional Neural Network for Brain-Computer Interface. arXiv:2104.01233 [cs, eess], 2021.
12

---

[29] Paolo Zanini, Marco Congedo, Christian Jutten, Salem Said, and Yannick Berthoumieu. Trans- fer Learning: A Riemannian Geometry Framework With Applications to Brain- Computer Interfaces. IEEE Transactions on Biomedical Engineering, 65(5):1107-1116, 2018. doi: 10.1100/BBM:2211::7745555
[30] Pedro Luiz Coelho Rodrigues, Christian Jutten, and Marco Congedo. Riemannian Procrustes Analysis: Transfer Learning for Brain-Compunt Interfaces. IEEE Transactions Biomedical Engineering. 66(8):2390-2400 2019. doi: BLL.///TBE..11..2888..
[31] Or Yair, Mirela Ben-Chen, and Ronen Talmon. Parallel Transport on the Cone Manifold of SPD Matrices for Domain Adaptation. IEEE Transactions Signal Processing, 67(7):1797-1811, 2019. doi: 10. 109/TSP.... 28994801.
[32] Hongwei Yong. Jianqiang Huang, Deyu Meng. Xiansheng Hua, and Lei Zhang. Momentum Batch Normalization for Deep Learning with Small Batch Size. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision - ECCV 2020, volume 12357, pages 224-240. Springer International Publishing, Cham, 2020. ISBN 978-3- 030-58609-6 978- -7--9-73---55500 3-030-58610-2.
doi: 10.1007/97--3-000 00--0000066---1-46
[33] He He and Dongrui Wu. Transfer Learning for Brain Brain-Commy Computer Interfaces: A Euclidean Space Data Aligmment Approach. IEEE Transactions on Biomedical Engineering, 67(2):399 410,
2020. doi: 10. 109/TTEM.2019.999333
[34] Stylianos Bakas, Siegfried Ludwig, Konstantinos Barmpas, Mehdi Bahri, Yannis Panagakis, Nikolaos Laskaris, Dimitrios A. Adamos, and Stefanos Zafeiriou. Team Cogitat at NeurIPS 2021: Benchmarks for EEG Transfer Learning Competition, 2022.
[35] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric Deep Learning: Going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4):18 2017. doi: 10.1 D0.1000/SSP.211.26694444
[36] Ce Ju and Cuntai Guan. Deep Optimal Transport on SPD Manifolds for Domain Adaptation. arXiv:2201.05745 [cs, eess), 2022.
[37] Zhiwu Huang and Luc Van Gool. A Riemannian Network for SPD Matrix Learning. In Proceedings of the Thirty-Firsi AAAI Conference Artificial Intelligence, AAAI'I 17, pages 2046.2042.222 AAAI Press, 2017.
[38] Woong- Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domain- Specific Batch Normalization for Unsupervised Domain Adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[39] Rajendra Bhatia. Positive definite matrices. Princeton university press, 2009. ISBN 978-0-691- 16825-8.
[40] P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds. Princeton University Press, 2009.
[41] Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computing. International Journal of computer vision, 66(1):41-66 2006.
[42] Hermann Karcher. Riemannian center of mass and mollifier smoothing. Communications on pure and applied mathematics, 3005)550-, 1977.
[43] Shun-ichi Amari. Information geometry and its applications, volume 194, Springer, Tokyo, 1 edition, 2016. ISBN 978-4-431-55978-8.
[44] Daniel Brooks, Olivier Schwander, Frederic Barbaresco, Jean- Yves Schneider, and Matthieu Cord. Riemannian batch normalization for SPD neural networks. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[45] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift, International conference on machine learning, pages 448-456. PMLR, 2015.
13

---

[46] Reinmar J. Kobler, Jun-ichiro Hirayama, and Motoaki Kawanabe. Controlling The Fréchet Variance Improves Batch Normalization on the Symmetric Positive Definite Manifold. In ICASSP 2022 2022 IEEF International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3863-3867, Singapore, 2022. IEEE. ISBN 978-1-66550-540--9 doi: 10.1109/ (CASSP44992.2022.9446662
[47] Shibani Santurkar, Dimitris Tsipras, Andrew llyas, and Aleksander Mądry, How Does Batch Normalization Help Optimization? In Proceedings of the 32nd International Conference Red Hook, NY, USA,
Neural Information Processing Systems, NIPS'18 18, pages 2018. Curran Associates Inc.
[48] Sergey loffe, Batch Renormalization: Towards Reducing Minibatch Dependence in Batch- Normalized Models. Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pages 1942-1950, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 10-------1-6966666
[49] Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix Backpropagation for Deep Networks with Structured Layers. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 2965-2977 Santiago, Chile, 2015. IEEE. ISBN 978-1-4673-8391-22 doi: 10.11099/CCV22015.333.
[50] Jeffrey Ho, Guang Cheng, Hesamoddin Salehian, and Baba Vemuri. Recursive Karcher Ex- pectation Estimators And Geometric Law of Large Numbers. In Carlos M. Carvalho and Pradeep Ravikumar, editors, Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, volume 31 of Proceedings of Machine Learning Research, pages 325 332, Scottsdale, Arizona, USA, 2013. PMLR.
[51] Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices. SIAM Journal on Matrix Analysis and Applications, 29(1):328-34) 2007. doi: 10. 00.1137000637996.
[52] Paul L. Nunez and Ramesh Srinivasan. Electric Fields of the Brain. Oxford University Press, 000000000000000000000001
2006. ISBN 178-0-19-500038-7 doi: 00.003///coofoo/77711555555888
[53] Christoph M. Michel, Micah M. Murray, Göran Lantz, Sara Gonzalez, Laurent Spinelli. and Rolando Grave De Peralta. EEG source imaging. Clinical Neurophystology. 115(10), 2004. doi: 10.1016/j. (10b6:..linph.2204.66.0000. 2004.06.00..
[54] A. Hyvärinen and F. Oja. Independent component analysis: algorithms and applications. Neural Networks, 13(4-5):41 1-430, 2000. doi: 10. 00.006650933-00800000000------
(55] Stefan Haufe, Frank Meinecke, Kai Görgen, Sven Dähne, John Dylan Haynes, Benjamin Blankertz, and Felix Bießmann. On the interpretation of weight vectors of linear models in multivariate neuroimaging. Neurolmage, 87:96-110, 110, 2014. 067.
doi: ne00o/..euro
[56) Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for EEG decoding and visualization: Convolutional Neural Networks in EEG Analysis. Human Brain Mapping, 38 (11):5391-5420, 2017. doi: 10. 1..110000..2335550
[57] Dinesh Acharya, Zhiwu Huang, Danda Pani Paudel, and Luc Van Gool. Covariance Pooling for Facial Expression Recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR) Workshops, 2018
[58] Gary Becigneul and Octavian- Eugen Ganea. Riemannian Adaptive Optimization Methods. In International Conference on Learning Representations, 2019.
[59] G. Pfurtscheller and Neuper. Motor imagery and direct brain-computer communication. Proceedings of the IEEE, 89(7):1123-11344 2001. doi: 10.1199/5.999829
14

---

[60] Frédéric Dehais, Alex Lafont, Raphaëlle Roy, and Stephen Fairclough. A Neuroergonomics Ap- proach to Mental Workload, Engagement and Human Performance. Frontiers in Neuroscience, 14:268, 2020.
doi: 10.38.99/ninns22220006666.
[61] Ana R. C. Donati. Solaiman Shokur, Edgard Morya, Debora S. F. Campos, Renan C. Moioli, Claudia M. Gitti. Patricia B. Augusto. Sandra Tripodi, Cristhianc G. Pires, Gislaine A. Percira, Fabricio L. Brasil, Simone Gallo, Anthony A. Lin, Angelo K. Takigami, Maria A. Aratanha, Sanjay Joshi, Hannes Bleuler, Gordon Cheng. Alan Rudoiph, and Miguel A. L. Nicolelis. Long- Term Training with a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery Paraplegic Patients. Scientific Reports, 6(1):30383, 2016. doi: 10. 10.1038/srep30383.
[62] Domen Novak, Roland Sigrist, Nicolas J. Gerig, Dario Wyss, René Bauer, Ulrich Götz, and Robert Riener. Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cy- bathlon 2016. Frontiers in Neuroscience, 11:756, 2018. doi: 1.3389/fnins.22177.000
[63] Michael Tangermann, Rlburt Müller, Ad Aertsen, Niels Birbaumer, Christoph Braun, Clemens Brunner, Robert Leeb, Carsten Mehring, Kai J Miller, Gernot Müller-Putz, Guido Nolte, Gert Pfurtscheller, Hubert Preissl, Gerwin Schalk, Alois Schlögl, Carmen Vidaurre, Stephan Waldert, and Benjamin Blankertz. Review of the BCI Competition IV. Frontiers in Neuroscience, 6, 2012.
[64] Josef Faller, Carmen Vidaurre, Teodoro Dolis-Escalante Christa Neuper, and Reinhold Scherer. Autocalibration and recurrent adaptation: towards a plug and play online ERD-BCI. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 20(3)3313-31,, 2012.
[65] Min-Ho Lee, Y-on Kwon, Yong-Jeong Kim, Hong- Kyung Kim, Young- Young-Eun Lec, John Williamson, Siamac Fazli, and Seong-Whar Lee. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy. GigaScience, 8(5):giz002, 2019. doi: 10. 1093/gggasciennee/i/002
[66] Rea Lehner, Neethu Robinson, Tushar Chouhan, Mihelj Mihelj, Ernest, Kratka Kratka, Paulina, Frédéric Debraine, Cuntai Guan, and Nicole Wenderoth. Design considerations for long term non-invasive Brain Computer Interface training with tetraplegic CYBATHLON pilot: CYBATHLON 2020 -rointCompputr Interface Race Calibration Paradigms. 2020. doi: 10. URL http://hdl handle www.tt.teee.ee 500. 11850/458693
3929/ETHZ-B B- --000-56-0.00
[67] James R. Stieger, Stephen A. Engel, and Bin He. Continuous sensorimotor rhythm based brain computer interface learning in large population. Scientific Data, 8(1):98, 2021. doi: 10. 10.1088/<41597-0010008833
[68] Lea Hehenberger, Reinmar Kobler, Catarina Lopes-Dias Nitikorn Srisrisawang, Peter Tumfart, John B. Uroko, Paul R. Torke, and Gernot R. Müller-Putz. Long-term mutual training for the CYBATHLON BCI Race with a tetraplegic pilot: a case study on inter-session transfer and intra- session adaptation. Frontiers in Human Neuroscience, 2021. doi: 10.3389/fnhum.205 2tt.umm.221...77777
[69] Marcel F. Hinss, Ludovie Darmet, Bertille Somon, Emilie Jahanpour, Fabien Lotte, Simon Ladouce, and Raphaëlle N. Roy. An EEG dataset for cross-session mental workload estimation: Passive BCI competition of the Neuroergonomies Conference 2021. 2021. doi: 10.5281/ ZENODO.5055046.
[70] Alexandre Gramfort. MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience,
7. 2013. doi: 10.3389/iinns.213...m
[71] Kai Keng Ang, Zhang Yang Chin, Haihong Zhang, and Cuntai Guan. Filter Bank Common Spatial Pattern (FBCSP) in Brain- Cein-Commutter Interface. In 2008 IEEE International Joint Conference on Newral Networks (IEEE World Congress on Computational Intelligence), pages 2390-2397, Hong Kong. China, 2008. IEEE. ISBN 978-1-4244 118-1-4244-18200 1820-6.. doi: 10.1109/IJCNN, 460884634130..
[72] Or Yair, Felix Dietrich, Ronen Talmon, and Toannis G. Kevrekidis. Domain Adaptation with Optimal Transport the Manifold of SPD matrices. 1rXiv:1906.00611 [cs, stat), 2020.
15

---

[73] Vernon J Lawherm, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent Lance. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. Journal of Neural Engineering, 15(5):050013, 2018. doi: 10.1088. 1741-2552/aace8c.
[74] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 0024-8035. Curran Associates, Inc., 2019.
[75] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg. J. Vanderplas, A. Passos, D. Cournaeea, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 2:2825-28830 2011.
(76] Marian Tietz, Thomas J. Fan, Daniel Nouri, Benjamin Bossan, and skorch Developers. skorch:
A scikit-learn compatible neural network library that wraps PyTorch. 2017. URI. https
1o/ en/stable/
[77] Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian Optimization in PyTorc,, 2020. _eprint: 2005.02819.
[78] Barachant Alexandre. pyRiemann
pyRiemann, 2022. URL https:://githubb.comm/p/iiiie
[79] James Townsend, Niklas Koep, and Sebastian Weichwald. Pymanopt: A python toolbox for optimization on manifolds using automatic differentiation. Journal of Machine Learning Research, 17(137):1-5, 2016.
[80] Ian G. Campbell. EEG Recording and Analysis for Sleep Research. Current Protocols in Neuroscience, 49(1):100..1110222..5 2009. doi:
[81] Hubert Banville, Omar Chehab, Aapo Hyvärinen, Denis-Alexander Engemann, and Alexandre Gramfort. Uncovering the structure of clinical EEG signals with self-supervised learning. Journal of Neural Engineering, 2020. doi: 10. 10.1088/1741 255/7//4422252//aa1a
[82] U. Rajendra Acharya, S. Vinitha Sree, G. Swapna, Roshan Joy Martis, and Jasjit S. Suri. Automated EEG analysis of epilepsy: review. Knowledge- Based Systems, 45:147-165, 2013. doi: https://doi.or/// 10.
[83] Marc R. Nuwer, David A. Hovda, Lara M. Schrader, and Paul M. Vespa. Routine and quantitative eeg in mild traumatic brain injury. Clinical Neurophysiology, 1 116(9):2001-200 -025,, 2005. doi: 10. 10116//.ciiph.2000.... 20..000000.0........
[84] Lukas A.W. Gemein, Robin T. Schirrmeister, Patryk Chrabaszez, Daniel Wilson, Joschka Boedecker, Andreas Schulze-Bar Baallze-BBBhae,,, Frank Hutter, and Tonio Ball. Machine-lcarring-beer based diagnostics of EEG pathology. Neurolmage, 220:1 17021, 2020. doi: 10. ...1016/..nuurommage 2720.117021.
[85] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-Adversarin Training of Neural Networks. Journal of Machine Learning Research, 17(59):1-35, 2016.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See section 6
16

---

(c) Did you discuss any potential negative societal impacts of your work? [Yes] See section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See paragraph
4 in section
(b) Did you include complete proofs of all theoretical results? [Yes] See appendix A. in the supplementary material.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as URL)? [Yes] See Supple-
mentary Material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See paragraph in section 5 for brief summary and appendix
A for details.
(c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] We provide error bars with respect to the variability across sessions and subjects.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See paragraph 3 in section 5
4. If you are using existing assets (e.g., code, data, models) or curating/releusing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See paragraph in section 5
(b) Did you mention the license of the assets? [Yes] See appendix A.2. in the Supplemen- tary material.
(c) Did you include any new assets either in the supplemental material or as URL?? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] See paragraph in section (c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] See paragraph 1 in section 5
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
17


Reviews:
[{"review_id": "qBGduR8HZU_", "reviewer": "Reviewer_D8q8", "summary": "The authors present a new batch normalization module designed to improve performance of neural networks in scenarios where domain adaptation is critical, such as classification of inter-session and inter-subject EEG data. This is achieved by combining the ideas of momemtum BN, domain-specific BN, while respecting the manifold of SPD matrices.\n\nOn top of theoretical results supporting their approach, the authors provide compelling empirical evidence that the proposed SPDDSMBN module, when inserted into an SPD-aware deep neural network architecture, helps improve cross-domain performance compared to existing state-of-the-art approaches, as shown on 6 EEG datasets encompassing BCI and mental workload estimation classification tasks. An ablation study further confirms the importance of the proposed mechanisms for achieving these results.", "questions": "A single point that would make this manuscript even more convincing for me, though not necessary, would be if authors had provided results on a wider range of classification tasks, e.g. sleep staging, seizure detection, emotion classification and/or pathology detection. These tasks might present different inter-session and inter-subject variability characteristics. I believe including results on the mental workload estimation task was a first step in that direction. Whether results can extend to these types of tasks and even to other types of multivariate time series would be very interesting to see. ", "limitations": "The authors have adequately addressed the limitations and negative impact of their work.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposed SPDDSMBN method is original. While a logical extension of existing batch normalization ideas, it combines multiple important ideas that together provide significant performance improvements over existing state-of-the-art methods. Moreover, the interpretability of the proposed approach makes it particularly interesting for neuroscience-minded audiences who might want to explore correlates of the different classes in an EEG dataset.\n\nThe paper is of high quality and includes compelling theoretical proofs, experiments and results on multiple datasets. The manuscript is clearly written and well organized, as is the provided code.\n\nOverall, I believe the manuscript has particular relevance to the NeurIPS community, even beyond the EEG and neuroimaging communities.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "IkxGHnbuG5D", "reviewer": "Reviewer_P4Fn", "summary": "The authors proposes to learn domain invariant tangent space mapping models in an end to end fashion.\nThis mapping is suited to problems on data matrices that come from different domains and thus have possible different distributions.\nThese domain shifts occur in EEG when data are recorded on different people (structural and functional differences in brain networks) or several sessions (differences in the positioning of the electrodes).\nThe proposed differentiable pipeline is composed of 3 steps.\n1) Data matrices, associated with different domains, are transformed to covariance matrices. The latter belong to the set of symmetric positive definite (SPD) matrices which is viewed as a Riemannian manifold with an affine invariant Riemannian metric.\n2) The first and second order statistics of these matrices (grouped by domain) are normalized using a proposed Algorithm called \"SPD domain-specific momentum batch normalization\" (SPDDSMBN). Then the obtained matrices are transformed to vectors by lifting them to the tangent space at the identity using the logarithmic mapping.\n3) Obtained vectors in the tangent space (linear space) are classified using a linear classifier.\n\nThe main contribution is the Algorithm SPDDSMBN. Covariance matrices of each domain are assumed to follow a Gaussian distribution, on the SPD manifold, parametrized by a mean covariance matrix (Fréchet mean) and a positive scalar (Fréchet variance).\nSPDDSMBN estimates these parameters iteratively using batchwise estimates. The authors claim that the estimate of the Fréchet mean converges to its true value as the number of iterations tends to the infinite (Proposition 2). Then, the covariance matrices are transformed so that they have new learned first and second order moments. The interest is when the learned moments are enforced to be equal: all the covariance matrices from the different domains share the same statistics. This step is developed to improve the classification between different domains.\n\nA second contribution is that the full pipeline is differentiable and thus is trained in a end-to-end manner. By choosing linear operators in the pipeline, the authors are able to derive a log-linear relationship between the targets and the variances of the sources the classical EEG generative model. Thus the discriminative sources are identifiable which makes it an interpretable method. \n\nThe proposed method is applied on 7 EEG datasets of mental imagery and mental workload estimation shows overall great performance compared to other classifiers.\n", "questions": "The disappointing part of this paper is about the convergence of the running mean to the Fréchet mean.\n\n1) Can you clarify the confusion around $Var_{\\theta_k}(\\mathcal{T}_{\\theta_k})$\n\nwhich is sometimes referred as $Var_{\\theta}(\\mathcal{T}_{\\theta_k})$ ?\n\nAre they the same thing ? If not, $Var_{\\theta}(\\mathcal{T}_{\\theta_k})$ should be defined.\n\n2) Can you prove that if $f_\\theta$ is $L$-smooth then (14) is respected ?\n\n3) A comment should be added on why the induction in the proof of the Proposition 1 implies that the mapping $l \\mapsto Var_{\\theta_k}(G_l)$ is non-increasing.\n\n4) Can you detail how (19) gives (29) ? The variance of $B_k$ should be compared with the variance of $\\mathcal{T}_{\\theta_k}$.\n\n5) A comment on why the Proposition 1 is interesting/useful should be added. Is  it useful to state the Proposition 2 ?\n\n6) The assumptions of the Proposition 2 should be stated explicitly (and not \"If we additionally ...\") and made clearer. If the Proposition 2 is new (versus the work of [50]), then a proof should be added. Otherwise, it can be transformed to a remark on how the Theorem of large numbers of [50] applies to the pipeline.\n\n7) The introduction of many different notations makes the paper hard to read,\n\n- e.g. the estimated batch mean is denoted $\\hat{G}_\\mathcal{B}, B_k$,\n\n- the estimated Fréchet mean $G_k, G_\\mathcal{T}(k)$,\n\n- the Fréchet mean $G_{\\theta_k}, \\text{FM}(\\mathcal{Z})$,\n\n- the Riemannian distance $\\delta, \\delta_\\text{AIRM}$ ... This should be simplified.\n\n8) Minor remarks:\n- In the section \"Riemannian geometry on $\\mathcal{S}^+_D$: the inner product $\\langle ., . \\rangle_Z$ is wrong .\n- Equation (2): the logarithmic mapping should be written between two different matrices.\n- After equation (3) $\\mathcal{S}^+_d$ is written instead of $\\mathcal{S}^+_D$.\n- The parallel transport is used many times and thus should be presented in the section \"Riemannian geometry on $\\mathcal{S}^+_D$.\n- Just before the equation (23): $G_{k-1}$ is written instead of $B_{k-1}$.", "limitations": "The authors mention the potential of the EEG applications to aid the society. Some references or one/two sentences could be added to justify.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Strengths:\n\nOriginality:\n\nThe proposed method is novel to me and leverages recent advances made on the SPD manifold such as the Gaussian distribution, the geometric law of large numbers, and parallel transport of SPD matrices. Recalibrating the second order moment is rarely done whereas it is easy to implement as shown in this paper.\nAlso, the proposed method leverages the recent advances on geometric deep learning (SPDnet) by training a full pipeline in an end-to-end manner. This should improves the performance compared to classical methods that perform sequential operations associated with different optimization problems.\nThe combination of these contributions is original.\n\nQuality:\n\nThe numerical experiments are numerous and of great quality. The authors perform experiments on 7 EEG datasets of mental imagery and mental workload estimation and the proposed pipeline is compared to 10 other methods. They achieve state of the art performance on several of them and are on-par with best models on the others. Also, p-values of statistical tests are reported to ensure the significance of the reported results. An ablation study is performed and shows the importance of using all the proposed ingredients (momentum and domain-specific batchnorn). An additional good point is the Figure 3 which gives an interpretation of the learned model on two datasets.\n\nClarity:\n\nThe experimental part is clear: datasets are presented and the proposed method  used in practice is presented in Figure 4 and Table 3.\n\n\n- Weaknesses:\n\nQuality and clarity:\n\nThe theoretical part about the convergence of the running mean to the Fréchet mean is poorly written.\nFirst of all, it seems that there is a confusion around\n$Var_{\\theta_k}(\\mathcal{T}_{\\theta_k})$\n\nwhich is sometimes referred as $Var_{\\theta}(\\mathcal{T}_{\\theta_k})$.\n\nIt seems that the latter is never defined.\nThen, it is not very clear why the Proposition 1 (which is the only proven theoretical result) is mentioned. It is not interpreted and is never really used in the paper.\nFurthermore, to prove this result the assumption (14) is made. The authors mention that it is met when $f_\\theta$ is $L$-smooth. To me, this statement is not trivial and should be proven.\nAlso, the proof of the Proposition 1 is questionable. First of all, it is not clear why proving this induction proves that the mapping $l \\mapsto Var_{\\theta_k}(G_l)$ is non-increasing.\nThen, it is not obvious how the equation (29) is obtained. The authors leverage the equation (19), but how does (19) (which relates variances of $\\mathcal{T}_{\\theta_k}$\n\nbetween two successive iterates)\nimplies inequalities on the variances of $B_{k-1}$ and $B_k$. It is non-trivial since $B_k$ is an estimator (1-step Karcher flow) of the Fréchet mean.\nThe assumptions of the Proposition 2 are not very clear. Why do the previous assumptions such as the equation (14) are necessary ? Also, it seems to me that $\\theta_k$ converges to a stationary point does not imply that $||\\theta_k - \\theta^\\star|| \\leq 1/k^\\beta$. Furthermore, why does the convergence rate of $\\theta$ must be in $1/k^\\beta$ ? If after a certain amount of iterations, the $\\theta_k$ stay in a small ball around $\\theta^\\star$, the covariance matrices $f_{\\theta_k}(x)$ can be considered fixed and thus the Theorem of large numbers on the SPD manifold directly applies.\n\n\nFinally, too many different notations are introduced in the paper which make it hard to read.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "I48YOVKzHgH", "reviewer": "Reviewer_A1dj", "summary": "**Update after rebuttal**: I thank the authors for their responses and must say that I'm very happy with their rebuttal. Their submission has improved in this new version and I will, therefore, increase my score from borderline-accept (5) to accept (7).\n\n---\nThis work presents a pipeline for classification with EEG recordings based on three steps. Firstly, the statistical features of the EEG time series are encoded via a symmetric positive definite (SPD) matrix. Then, they go through a neural network specially tailored for handling data points defined in the SPD manifold. Finally, the transformed SPD matrices are projected onto the tangent space of their manifold where a linear classifier with softmax activations is applied. \n\nThe authors were particularly interested in the case where the statistics of the training dataset is different from that of the testing set. This happens quite often in EEG, for instance when working with brain-computer interfaces trained on different subjects. The differences in these statistics is an important challenge in BCI research and the paper tackles it with their domain-specific version of the momentum batch normalization for SPD matrices (SPDDSMBN), which is a layer in their neural network. The authors also include two interesting features to classification pipeline: (1) the ability of learning from the data which SPD features should encode the time series, as opposed to the traditional approach of fixing them to be the spatial covariance matrix. (2) interpretability of the linear weights used to classify the data points, directly relating them to the spatial distribution of the EEG recordings on different channels.", "questions": "**(Q)** In the case where you have several source datasets, you simply pool all of their data points into a single big source dataset post transformations via SPDDSMBN? Does your procedure learn somehow to give more importance to data from one subject as compared to another one? In fact, your TSMNet is the same for all possible target subjects? Or you train one for each possible target?\n\n**(Q)** Is there any theoretical result showing the limitations of SPDDSMBN to correct the dissimilarities between the statistics of the source and target dataset? In other words, do you have any mathematical model showing how a source and a target dataset may be different and whether your procedure is capable of correcting it?\n\n**(Q)** What is the SOA domain-specific method used as reference for comparisons in Section 5.1?\n\n**(Q)** In Equation 17 you present a layer depends on parameter $\\phi$ and modulates the dispersion of the data points at its output. How \ndoes your training procedure avoid making $v_\\phi \\to 0$ and decreasing the variance of its output as small as possible? \n\n**(Q)** Saying \"latent observations\" in Line 172 is an oxymore. Do you observe these data points or not?\n\n**(Q)** I am more accustomed to seeing the term \"intra-domain\" than \"domain-specific\". Do the authors have any specific reason for preferring this term?\n\n**(Q)** Is Equation 7 really necessary for the rest of the paper?", "limitations": "The authors show the limitations of their method with applications to real data and show that while it does not always attain the same performance as a domain-specific SOA method, it does beat all other concurrent methods from the literature. However, I would have appreciated seeing a simulated example with controlled dissimilarities between the datasets and checking which class of transforms the method proposed by the authors is able to correct.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "As an expert on the domain of EEG classification with SPD matrices, I must say that I enjoyed seeing the many new ideas presented in this paper. There are several small gems that tackle different practical challenges of modern BCI systems and I congratulate the authors for getting interested in them. However, the sequence of the presentation of these ideas is quite confusing and it was not particularly easy to follow through the logic of the text, even for someone with some experience in the field. \n\nFor instance, it feels a bit odd to talk so much about batch normalization (and its variants) without saying why we need it for this application in the first place. Also, while I understand the structure of the architecture of TSMNet with SPDDSMBN, I still don't get why the theoretical results presented in Proposition 1 and Proposition 2 were necessary. Indeed, it feels much more like a math result that the authors wanted to include in the text than something that had a motivation for being part of the work (Please correct me if I'm mistaken). \n\nIn line 161, the authors say \"we believe that the performance can be improved by extending MBN to the SPD manifold\". This is a very strange and imprecise of presenting something that is supposed to be one of your main contributions. Also, I'm uncertain of what you mean by proposing a \"theory-based\" machine learning framework.\n\nAlso, the text can be a bit hard to read due to its (way too) many acronyms. Are the authors certain that all of them are really necessary? Things like MSMTUDA and SPDDSMBN can be a bit disturbing to read.\n\nIn all, the content of the paper is interesting and worth of being shared with the community, but I am not satisfied with the structure of the presentation of ideas and think that the authors should improve it before publication.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Hd0pJCKvi7", "reviewer": "Reviewer_LpZK", "summary": "The paper proposes a batch normalization framework that is suitable for problems of unsupervised domain adaptation (UDA) between symmetric positive definite (SPD) manifolds. Their framework- the SPD domain-specific momentum batch normalization or SPDDSMBN is designed to transform domain-specific SPD input matrices into domain-invariant SPD outputs by automatically inferring relevant batch statistics. This can be inserted a parameterized layer transformation within a geometric deep learning model in multi-(domain) source and target settings. The authors also provide a theoretical analysis on the correctness of recovered batch statistics. The authors perform experiments with 6 different EEG brain-computer interface (BCI) datasets and demonstrate that their framework provides improved performance within two transfer learning setups (inter-session and inter-subject) against baseline comparisons and ablation studies\n", "questions": "1. A suggestion would be to include more details on generating the interpretability maps from the reference [17] in the supplementary for completeness. Adding a few more sentences of explanation to the plots in Fig. 3 would also help improve the clarity here. \n2. There are many acronyms used in the paper for each discussed variation of the framework. A suggestion would be to include a list of these for easy lookup within the supplementary document so a potential reader can avoid back and forth within the main text.\n\n", "limitations": "The main limitations of the framework, scalability to larger sized input data (eg. fMRI) is discussed briefly in Section 6. I agree that this is a fair criticism of all of the methods. Perhaps a relevant future direction would be to utilize techniques that can improve the scalability of the bottleneck eigenvalue decomposition.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS: \n\n1. The paper is very well written and easy to follow.  The relevant technical details are presented concisely and cover the relevant preliminaries making the paper fairly self contained. The main contributions are clearly articulated with dedicated theoretical analysis and thorough experimental evaluations provided to support the claims. \n\n2. The main contributions of the paper, extending momentum based batch-normalization to geometric deep learning on SPD manifolds as well as the application to domain adaptation in SPD manifolds is interesting and relevant to applications beyond the EEG BCI domain. Although not a major contribution of the work, the discussion on interpretability in the context of the potential healthcare application is a plus.\n\nWEAKNESSES:\n\nThere are very few details provided for generating the interpretability maps in Fig. 3 making this section a bit hard to parse. \n\n(a) As per my understanding, the contribution of each extracted source to the target class (spatial mixing coefficient) is being plotted on the head model. I am a bit unclear on how to actually interpret the results being displayed, for example, are the darker regions more discriminative/important (the figure is also missing a colourbar)? This needs a bit of clarification\n(b) I am unclear on what the axes of the plot on the top left of 3 a and 3 b are plotting. Specifically, what is the y axis \"contribution\" measuring?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept