Towards a General Attention Framework on Gyrovector Spaces for Matrix Manifolds
Authors: Rui Wang
Chen Hu
Xiaoning Song
Xiaojun Wu
Nicu Sebe
Ziheng Chen
Venue: NeurIPS 2025 poster ()
Keywords: Manifold Learning
Gyrovector Space
Attention Mechanism
Riemannian Manifold
Signal Classification
Primary area: deep_learning
TL;DR: A Gyro Attention Framework for Matrix Manifolds

Abstract: Deep neural networks operating on non-Euclidean geometries have recently demonstrated impressive performance across various machine-learning applications. Several studies have extended the attention mechanism to different manifolds. However, most existing non-Euclidean attention models are tailored to specific geometries, limiting their applicability. On the other hand, recent studies show that several matrix manifolds, such as Symmetric Positive Definite (SPD), Symmetric Positive Semi-Definite (SPSD), and Grassmannian manifolds, admit gyrovector structures, which extend vector addition and scalar product into manifolds. Leveraging these properties, we propose a Gyro Attention (GyroAtt) framework over general gyrovector spaces, applicable to various matrix geometries. Empirically, we manifest GyroAtt on three gyro structures on the SPD manifold, three on the SPSD manifold, and one on the Grassmannian manifold. Extensive experiments on four electroencephalography (EEG) datasets demonstrate the effectiveness of our framework.

Towards a General Attention Framework on Gyrovector Spaces for Matrix Manifolds
Rui Wang', Chen Hu', Xiaoning Song', Xiao-Jun Wu', Nicu Sebe?, Ziheng Chen
School of Artificial Intelligence and Computer Science, Jiangnan University
2 Department of Information Engineering and Computer Science, University of Trento
{cs_wr, 6233112017, x. song, mu_xiaojjm)@jiangnn edu. seelaeeseee@umita. it, riheng_ch@163, com
Abstract
Deep Neural Networks (DNNs) operating on non- non-Euclidean geometries have recently demonstrated impressive performance across various machine-learming ap- plications. Several studies have extended the attention mechanism to different man- ifolds. However, most existing non -uc-Euclidean attention models are tailored to spe- cific geometries, limiting their applicability. On the other hand, recent studies show that several matrix manifolds, such as Symmetric Positive Definite (SPD). Symmet- ric Positive Semi-Definite (SPSD), and Grassmannian manifolds, admit gyrovector structures, which extend vector addition and scalar product into manifolds. Lever- aging these properties, we propose Gyro Attention (GyroAtt) framework over general gyrovector spaces, applicable to various matrix geometries. Empirically, we manifest GyroAtt three gyro structures on the SPD manifold, three the SPSD manifold, and one on the Grassmannian manifold. Extensive experiments on four Eleetroenceppallggrapyy (EEG) datasets demonstrate the effectiveness of our frame- work. The code is available at https: // //gi thub. com/ Matennuu-MM/Grreeec ML.GyroAtt.
1 Introduction
Recently, DNNs over Riemannian manifolds, known as Riemannian neural networks, have garnered increasing attention in various applications [26, 11, 61, 41, 13, 73, 53, 72, 18]. Commonly encoun- tered manifolds include vector manifolds, such as hyperbolic [68] and spherical spaces [65], and matrix manifolds, such as SPD [5], SPSD [9, 10], and Grassmannian manifolds [1]. Among these non-Euclidean spaces, hyperbolic manifolds stand out due the rich algebraic structure of gyrovector spaces [67-69], which enables principled and convenient extensions of Euclidean deep learning to hyperbolic manifolds [26, 61, 7]. In contrast, matrix manifolds offer a compelling trade-off between structural expressiveness and computational feasibility [20]. As a result, neural networks defined on matrix manifolds have emerged as appealing alternatives to their hyperbolic counterparts across various applications [40, 51, 52, 14, 37]. Notably, recent studies [40, 50-52) have demonstrated that several matrix manifolds, including SPD, SPSD, and Grassmannian, admit gyro structures, facilitating the extension of existing neural network components these manifolds [53].
Inspired by the success of attention mechanisms in DNNs [70, 32, 22], researchers have explored their extensions to non-Euclidean geometries. Wherein, Gulcchre et al. [26] introduced attention to the hyperbolic spaces based on the hyperboloid and Klein models, while Pan et al. [55] extended attention to the SPD manifolds under the Log-Euch Euclidean Metric (LEM). Wang et al. [72] further adapted it to the Grassmannian manifolds using an extrinsic approach under the projection distance. However, existing manifold-atteninne designs exhibit two key challenges: i) these designs are tailored for specific manifolds and metries, limiting their generalizability; ii) although their network layers,
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

such as BiMap and ReEig in MAtt [55], and FrMap in GDLNet [72], preserve manifold constraints, they are fundamentally numerical and only partially respect the underlying Riemannian geometry.
As self-attention serves as the prototype of other attention variants, this work extends it to non- Euclidean settings. Leveraging the fact that several matrix manifolds admit gyro structures, propose GyroAtt, a general framework for self-attention over gyrovector spaces. Unlike previous manifold attention approaches handcrafted for specific geometries [26, 55, 72], GyroAtt provides a unified building paradigm for attention design across different matrix manifolds. Notably, by intrinsically respecting Riemannian geometry and gyro algebra, GyroAtt generalizes fundamental attention components, such as linear transformations, attention computation, and feature aggregation, to gyrovector spaces. Specifically, we introduce gyro homomorphisms, extending linear transforma- tions to gyrovector spaces. The attention scores are computed using gyro distance- -based function, while feature aggregation is performed via the Weighted Fréchet Mean (WFM) [25], the manifold analogue of the Euclidean weighted average. We instantiate AtroAtt on three gyro structures for the SPD) manifold, three for the SPSD manifold, and one for the Grassmannian manifold. Exten- sive experiments on four EEG benchmarks show consistent improvements over geometry-speciff manifold-attention baselines; e.g., GyroAtt-SPSD outperforms MAtt [55] by 3.2% % on the MAMEM, while GtroAtt-SPD achieves 8.9% and 7.8% gains in inter-subject and inter-session settings on the BNCI201 4001. Our main contributions are summarized below:
Generalizing attention to gyrovector spaces. This is the first attention framework that unifies operations across diverse matrix manifolds through a common gyrovector space formulation, enabling flexible changes in the underlying geometry within a shared network structure.
Implementation on seven matrix gyrovector spaces. We implement GyroAtt across three different matrix manifolds: three gyro structures on the SPD manifold, one on the Grassmannian manifold, and three on the SPSD manifold. To the best of our knowledge, we are the first to investigate attention mechanisms for the SPSD manifold.
Empirical validation under EEG signal classification. We validate the effectiveness of GyroAtt on four EEG benchmarks. Apart from its good performance, the optimal geometries vary across different EEG tasks, demonstrating its efficacy and flexibility.
2 Preliminaries
This section reviews gyrogroups, gyrovector spaces, and their realizations on the SPD, Grassmannian, and SPSD manifolds, together with WFM. For further details, please refer to [68, 69, 56, 5, 10, 8].
Gyregroups and gyrovector spaces. Gyrogroups and gyrovector spaces generalize groups and vector spaces, offering a powerful framework to analyze Euclidid geometries. A gyrogroup equips set (G, (b) with non-associative "addition" (i whose failure of associativity is controlled by a gyration gyrl-- -]. Adding a scalar multiplication @ that obeys vector-like axioms upgrades the structure to gyrovector space. A review of gyrogroups and gyrovector spaces is given in App. D.I.
SPD geometries. Let St++ denote the space of x d SPD matrices. When equipped with the Affine-Invariat Metric (AIM) [56], LEM [5], and Log-Cholesky Metric (LCM)
induces three gyrovector spaces with corresponding binary operations [52] Bai. Stc, and Bic. and associated gyro distances [19] deja (.), diga (.), and (.), given in Tab. 1. Given P, Q e S we denote by logm(-) and expm(() the matrix logarithm and exponential. LPP)) means the Cholesky decom-
position of P, with (.) denoting its inverse map. Besides, [P)( denotes the strietly lower triangular part, (PP) signifies the diagonal part, and then (P) = [P) lDg)(
Grassmannian geometries. The Grassmannian manifold consists of all q-dimensional linear sub- spaces within Rd. This study centers on the Orthonormal Basis (ONB) perspective, where subspace is represented by Y E IRdxq satisfying Y Y = lg. We denote the set of such ONB represen- tations Y E d(),d). A point in Ō(g. d) represents equivalence class of orthonormal bases: [Y] ( Ý = YO,O E O(g)). By abuse of notation, we use (Y] or Y interchangeably. Under the ONB perspective, the Grassmannian manifold induces a nonreductive gyrovector structure [52]. The binary operation Cge Sgr [52] and gyro distance (.) [19] for U, V Õ(q. d) are defined in Tab. 1.
SPSD geometries.
Let St denote the set of d x d SPSD matrices of rank g d. Each P E St admits canonical decomposition P UTSPUT. where Up ~(g. d) and Sp E S [9, 10].

---

Table 1: Summary of the gyro additions and gyro distances over different manifolds.
Manifold
Metrie
Gyro addition
Gyro distance
AIM
B.. Q = Pi Qi
logrn
Q
PP
SPD
LEM
exppm (P 1ogm/
logn(P)
logmr Q
P Sie Q = 2 ([2(P) [2 +22
||
vuo
arccoe(22] (23)
Grassmannian
ONB perspective
U U8 V = (UU Au,
UT v 2 OER
SPSD
(Jer. A
(U,.Sp) Spudy 8q)= S, 8,, Sq) (U U2) + Ad (Sp, (8,,8)
Following Nguyen et al. [53], we represent P in the product space G(g, d) x * as detailed in App. D.3. By equipping St+ with a Riemannian metric E {ai, le, Ic}, the space St. naturally inherits corresponding gyrovector structure [53]. The induced binary operation @psd.g [53] and the related gyro distance (-) [19] are defined in Tab. 1, where À >>> 0.
WFMs. The WFM [25] of points (Ps... is the point S E M that minimizes the weighted sum of squared distances to all points (.....N. J. Given weights (wi...) satisfying the convexity constraint, i.e., Vi, wy > 0 and 2, = 1, the WFM is expressed as;
Σn w; SP
WFM([ (P.)) = arg min SEM
where S) is the distance between S and P.. On Riemannian manifolds, WFMs uniquely exists when samples are locally distributed [2], which is detailed in App. D.2.1. In this paper, we always assume WFMs are well-defined.
Proposed method
Inspired by the success of attention mechanism [70, 32], recent studies have extended attention models to non-Euclideen settings, including hyperbolic [26]. SPD [55], and Grassmannian [72] manifolds.
However, these approaches typically rely on geometry-speciffc operations, such as BiMap layer [33] for SPD manifolds or FrMap layer [34] for Grassmannian manifolds, which limits their applicability and generalization across manifolds. In contrast, our proposed GyroAtt framework leverages gyro structures to provide a unified and principled formulation of attention across matrix manifolds. To motivate our design, we first revisit prior manifold-base attention mechanisms and summarize their core components in Tab. 2, before introducing our generalized gyro-based formulation.
3.1 Revisiting attention mechanisms on different geometries
Despite differences in underlying geometries, self-attention mechanisms generally follow a common three-stage pipeline: 1) feature transformations to compute query (q:), key (k;), and value (v:); 2) similarity computations, often based on distances or inner products; 3) weighted aggregations of the values (r,). Tab. 2 summarizes these components across geometry- attention models.
Euclidean. Standard Transformers [70] generate q, k, and v; through linear projections Linear(-). compute attention weights using scaled dot-product Softmax ((q; k,)/ VTy), and obtain outputs j=1 Aij where N signifies the number of value tokens.
Hyperbolic. HAN [26] extends attention to the hyperbolic space using the Klein (K3) and hyper- boloid (IH²' models. Euclidean features are first projected onto the manifold via TR-sK and тR- Attention scores are computed by d(; k,) - C, where 3 >> and e are learnable parameters. Value aggregations are performed using the Einstein midpoint [66].
SPD. MAtt [55] formulates attention on the SPD manifolds under LEM. Querics and keys are obtained via the BiMap function [33], attention scores measured using the LEM-based geodesic distance, and LEM-based WFM is employed for value aggregation.
Grassmannian. GDLNet [72] adapts attention the Grassmannian manifolds by employing FrMap and ReOrth layers for data transformation and activation [34]. Attention scores are computed based on the projection distance, while the extrinsic WFM [62] is employed for value aggregation.
In summary, the above manifold attention approaches are inherently tied to particular manifolds or metrics, limiting their direct generalization across different geometries.
3

---

Table 2: Summary of attention based on different geometries, where f&(-) denotes the softmax.
Method
Geomeriee
Tramsformatioss
diq k.)
Altenton Au
Aggrepation ryR,))
Transforer
[70]
Eaclidaan
Limsar(x,)
kylly
Arithmetic mean
>. Agy;
HAN [25]
Hyperbolic
uR- (Linws((s.))
ma-a (Linwr((x.])
Σ) 5> Auster) Y
Einstein midporet
MAi
[55]
under LEM
WX.WT
[(ogui(())) logas( lo
(1 + Augl1 (.)))) ')
expax
LEM-hased WFM
(E) A., logm/
GDLNE
[7X]
Grassmantiia under ONB
ReOath (WX.)
K K,K]/
(O + kull K,)))
Barrissi WPM
(x:) A,,
GyroAit (Ozss)
Gynovector spaces (SPD, SPSD, Gravamans)
Homemerphism Eq. (4)
Ciyo distance
1. (1 keel + k11,., ')
WFM
3.2
Attention mechanisms over gyrovector spaces
In this part, we extend the basic attention operations illustrated in Tab. 2 to gyrovector spaces: 1) transformation through gyro homomorphisms, which preserves the gyrovector structure; 2) distance- based similarity computation; 3) value aggregation via gyro distance- based WFM.
Definition 3.1 (Gyro Homomorphisms), Let (M, @M, @M) and @Bx &wv) be two (nonre- ductive) gyrovector spaces, AA, B E M, and Vt E R, the map hom(-) : (M, BM. @M) - (W, @Nr &x) is (nonreductive) gyrovector space homomorphism if it satisfies:
hom ( BM B) = hom (A) @x hom(B), hom(t @M A) = = &y hom(A).
(2)
If we only consider (nonreductive) gyrogroups, (M, and (W,ww) a map hom(.) (M, (BM) + (N, @x) satisfying Eq. (35) is called (nonreductive) gyrogroup homomorphism, which has been introduced by Suksumran and Wiboonton [63]. By abuse of notations, we call the above homomorphisms collectively gyro homomorphisms. The concept of gyro homomorphisms naturally generalizes linear maps from vector spaces to gyrovector spaces. Recall that a linear map Linear(.) R" + Rm is a homomorphism of vector spaces, meaning it preserves both vector addition and scalar multiplication: for any Z1, Zg E IR" and t E R. it satisfies:
Linear(z) + m2) = Linear(z:) + Linear(Z2), Linear((z)) t Linear(zi).
(3)
Therefore, we use hom(-) for feature transformation. While the above offers an algebraic definition. a natural question arises: how can such mappings be identified on manifolds?
Theorem 3.2 (Sufficient condition for gyro homomorphisms). [L] Let M be Riemannian homogeneous space with isometry group G. If (M, @ 8) forms a gyrovector space and an isometry I E G fixes the identity element, i.e., f(e e, then I is gyro homomorphism.
This theorem provides a way to identify gyro homomorphisms from homogeneous space isometries. Then, we calculate the correlation between Q; and K, using their gyro distance, and map di Qq; K,) to a valid attention score, as defined in Eq. (5). For feature aggregation, we resort to WFM based on the gyro distance. Specifically, given a set of input data points {X,...N M), the key operations of GtroAtt are listed below:
Q Nom((,)), K; =om((X,,, V: = hom(
(Transformation)
(4)
A.j Softmax (1 1og(11(1 + d(Q,, K))))=
(Attention)
(5)
R. = WFM (A,, V....).
(Aggregation)
(6)
Here, As denote the i-th row of A, and R, is the resulting data under the manifold-valued weighted average (i.e., WFM) between A; and V...N- While WFM offers a principled approach for aggregating manifold-valued data, its output R, is inherently restricted to the geodesic convex hull of the input points. This constraint may limit the expressiveness of the resulting representations. To mitigate this limitation, we introduce a bias term and nonlinear transformation following the aggregation step:
< a (B R.),
where B a learnable bias parameter, and ø represents a power- based activation function.
So far, all ingredients are in place to build attention over gyrovector spaces, shown in Alg, 4.

---

4
Gyro attention mechanisms on matrix manifolds
This section presents our GyroAtt framework (Alg. 4) across various matrix gyrovector spaces, including three SPD gyro spaces, one Grassmannian gyro space, and three SPSD gyro spaces.
4.1
Gyro attention mechanism over SPD manifolds
As summarized in Tab. 1, three SPD gyrovector spaces are induced by the AIM, LEM, and LCM, respectively. The corresponding gyro distance (used for attention calculation) and gyro addition (used for biasing) have been well studied in prior works [5, 56, 45]. Therefore, our focus is on defining the gyro homomorphisms, WFMs, and activation for these geometries. We begin by deriving the explicit forms of the gyro homomorphisms corresponding to each SPD geometry. Theorem 4.1 (SPD Homomorphisms). [J] Given P + let homg
S
St be defined as
OPOT, ho expm (opmMllom(PPP logm (P)
g ai, O E O(d), g = le, M E Raxd,
(8)
-- + cxp
g Ic, M E Raxd
For each metric g (ai,le,le) Ic), hom,()) a gyro homomorphism with respect to (Sj Gg 8g). Moreover, if M orthonormal, the homomorphism is identical to the AIM one.
where L(P) M(L(()) + [2()) ((2(P))
Transformation. As shown Thm. 4.1, the forms of h. and homza() follow from the flat structures of their geometries. When manifold is flat, its gyro homomorphism can be directly derived. The SPD manifold under LEM is flat in the logarithmic domain, where linear mappings act as valid homomorphisms. Orthogonal constraints improve network generalization by imposing implicit regularization [30]. Therefore, we impose orthogonality on M in both homze(-) and hom, (-).
WFMs. The WFMs under LEM and LCM admit closed-form solutions, whereas the one under AIM requires iterative computation via the Karcher flow algorithm [38]. These are reviewed in App. D.2.2.
Activation. As demonstrated by Chen et [17, Fig. 1] and Chen al. [15, Sec. 5.1], the matrix power can deform the latent SPD geometries. As a consequence, we use matrix power as the activation function to activate the underlying Riemannian geometry.
4.2
Gyro attention mechanisms over Grassmannian manifolds
We implement GyroAtt on the Grassmannian nonreductive gyrovector spaces under the ONB perspec- tive. The gyro distance and gyro addition are shown in Tab. 1. Similar the SPD gyro spaces, gyro homomorphism for transformation and WFM for aggregation. As shown by Nguyen and Yang [52, Sec. 2.3.2], the Grassmannian gyro addition can be viewed as a non-linear activation. Therefore. we do not use additional activation before the Grassmannian gyro biasing. In the following, we discuss gyro homomorphism and WFM over the Grassmannian manifold.
Theorem 4.2 (Grassmannian Homomorphisms). [L] Lei U E (9(4, d). Hyr. &gr), and O
O,
O Od-g E Rd..d where Og E Raxa and Oa-g E ((d-a)x(d-o) are orthonormal. The map homgr ()) (9(g, d), Sgr. Bgr) + (ā(g. d), $gr
gr. @gr) is gyro homomorphism defined by
homgr (U) OU.
(9)
Eq. (9) plays the role of Grassmannian feature transformation. For the weighted aggregation, since the WFM on the Grassmannian manifold does not admit a closed-form solution, we employ the Karcher flow algorithm [1, 38). More details are provided in App. D.2.3.
4.3 Gyro attention mechanisms over SPSD manifolds
As introduced in Sec. 2, any P E S can be canonically represented as a pair (UP.Sp) e G(g. d) x S within the structured product space. According to Tab. 1, both the distance and gyro addition operations in this space are defined component wiiely over the Grassmannian and SPD subspaces. Therefore, implementing GyroAtt in the SPSD gyrovector space reduces to specifying the gyro homomorphisms, WFMs, and activation for the corresponding product structure.
5

---

Table 3: Key operators of GyroAtt over different matrix gyrovector spaces.
Manifold
SPD
Grasomantian
SPSD
Metri
AIM
LEM
LCM
ONB perseetinee
Appell
Homomoophiso
OPO
expan PoMM) Se [[L(P)) eup(D(L(P))
OU
[hamge (Mpp) hoom,
WFM
Alg.
Bow
Eg (15)
Eq. (16)
Alg.2
(WPM WFM,))
Bias and Non-lincarity
(Bage Oy R,)"
By R,..
(Ba-0g. Dp. (Baye Baye Sm.)))
Manifild Modeling
Manifodd Flattening
FC & Softmax
Conv Block
D
GyroAu
EEG Signal
Figure I: The GyroAtt network consists of three modules: A feature extraction module that applies convolution and manifold modeling to transform EEG signals into manifold-valued features; a gyro attention module that captures long-range dependencies among features; and a classification module that implements decision making with the flattened manifold-vall vallud data.
Theorem 4.3 (SPSD Homomorphisms). [4] Let E lai,le,le), and (Ur,Sr) e (G(g, d) x
S Dped,g @pedig)- The map homped. (g(g, d) x S Dpsd. @ped,g) + (g(g, d) x
S Dpad, gpsd.g) is a gyro homomorphism defined by
homped, (Up,Sp) = (hom,r (Up). hom, (Sr)).
(10)
For aggregation, we use the WFM induced by the product geometry, detailed in App. D.2.4. The bias and non-linear activation are also formulated in the product space:
@ped ( R, RR,) = Og Un. (Bepd ① Oapd SR,)")
(11)
where Bgr GG(g. d) and B Bapd S
In summary, our GyroAtt framework comprises several basic operations. First, the mapping hom()) generates Qi K,, and V.. Then, the attention is computed using gyro distances between queries and keys, followed by the WF.M-based aggregation on the values V,. Finally, the bias and non- non-linear activation enhance model expressiveness. GyroAtt offers unified framework that supports multiple manifolds and metries, demonstrating its superior generality and flexibility. Tab. summarizes all the key ingredients for computing GyroAtt the SPD, Grassmannian, and SPSD manifolds.
5 Experiments
Following prior works [55, 41], we evaluate the proposed GyroAtt on EEG decoding across four benchmarking datasets: BNNC1201000 [24], SENC1201500 [64], MAMEM-SSVEP-II [54], and BCI-ERN [47] in this paper. For BNCI201 4001 and 50C12015001, we conduct both inter-session and inter-subject evaluations. In the inter-session evaluation, models trained exclusively data from the target subject. Additionally, the balanced accuracy, calculated as the average recall across all classes, is used as the primary metric [41]. For MAMEM-SSVEP-IL the overall accuracy is used, while for BCI-ERN, we report the Area Under the Curve (AUC) to address class imbalance. In the experiments, the first four sessions of each subject are used for training, with one for validation, and the fifth for testing. Further details regarding datasets and preprocessings are provided in App. E.
Implementation details. shown in Fig. 1, the GtroAtt network consists of three components: feature extraction module, a gyro attention module, and classification head. In the feature extraction module, two convolutional blocks are first applied to the EEG signals to extract low-rede redundduny features, followed by pyramid-like like temporal segmentation into non-overlapping subparts, each producing a covariance matrix. For GyroAut-SPD, these covariance matrices X, serve directly inputs to the subsequent layers. In SyPSA and GyroAtt-Grr, each covariance matrix is transformed into its canonical form (U S S' using Alg. 3, mapping them into the structure space G(g, d) x S.T Here, U is used as the input for GyroAtt-Grr while both Ut and S used

---

Table 4: The results of different methods on BNCI2014001 and BNCI2015001. attention models are highlighted in yellow. The top three results are marked in red, blue, and cyan.
Riemannian manifold
BNCI2014001
BNCI2015001
Manifold
Method
session
Inter subject
Inter-session
Inter-subject
FBCSP+SVM [4]
60.6 4.9
32.3 + 7.3
81.5 + 4.4
58.6 + 13.4
EEGNet [44]
41.8 + 5.8
43.3 + 17.0
72.4 8.4
59.2 9.5
Ruclidean
ShConvNet [59]
51.3 + 2.3
42.2 16.2
74.1 + 4.2
58.7 + 5.8
FBCSP+DSS+LDA [29]
71.3 + 1.8
48.3 + 14.3
84.6 4.8
67.7 + 14.3
TSM4SVM [6]
61.8 + 4.1
8.66
75.7
56.0
FB+TSM+LR [42]
69.8 1: 4.8
3.2
80.9
60.6 10.9
10.9
URPA+MDM [57]
59.5 2.7
46.8
1.6
70.3
SPD
SPDOT+TSM-SSM [74]
66.8
8.6
77.5
63.3
TSMNet [41]
3.6
$1.6 16.5
85,8 +
77.0 13.7
Graph-CSPNC [36]
71.9
9.3
79.8 + 14.6
64.2 13.4
SPD
MAtt [55]
45.3 + 11.3
80.8 14.8
63.1 +
Grassmann
GDLNet [72]
58.1 8.9
46.3 + 5.1
76.9 = 13.6
63.3 + 14.2
GyroAut-PD-AAIM
75.4
53.1 14.8
86.2 4.5
77.9 + 13.0
SPD
GyroAu-SPD-LEN
75.3 6.5
52.3 +: 14.1
85.7 1: 5.5
76.6 + 13.7
GyroAM-SPD-LLCM
74.2 7.8
52.4 + 15.6
84.7 + 6.6
75.5 + 13.8
Grassmann
GyroAut-Gr
72.5 1. 7.3
52.1 +:: 14.2
85.0 + 7.7
75.3 +: 13.7
GyroAtt-SPSD-A
72,9 = 7.1
52.4 + 15.6
84.7 6.6
75.5 + 13.8
SPSD
GyroAU-SPSD-LEM
72.8 6.9
50.5 + 13.2
85.3 4 5.3
76.0 + 14.1
SyroAn-SPSD-LCCE
72.9 6.7
51.7 + 13.1
85.1 + 4.8
74.9 = 12.6
for GSSDAn--PPDD. We employ the corresponding GyroAtt block, shown in Alg. 4, capture long-range dependencies between different feature regions on the manifolds. In the classification module, we first flatten the manifold-valued representations by projecting them onto tangent (or Euclidean) space, followed by vectorization. For GyroAu-SPP, we follow previous works [71, 16] to apply matrix power normalization
the output matrix P from the GyroAtt block, defined as she (P) } p with > 0 and P E S The scaling factor ă ensures gradient stability during optimization. For GyroAtt-Gr, we project each element Y, G(q. d) into Euclidean space using the operator & (Y,) Y.Y.T For Gyro both U and S processed accordingly within the classification module. Across all three models, the resulting matrices are vectorized, concatenated, and passed through fully connected layer followed by Sofimax function for classification. For parameter optimization and detailed implementatinns, please refer to Apps. E.2. and E.2.2.
Main results. We evaluated the proposed
Table 5: Results the MAMEM-SSVEPS (MAMEM) and BCI-ERN datasets. Manifold atten- tion models highlighted in yellow. The top three results are marked in red, blue, and cyan.
GyroAtt framework four EEG classifica- tion datasets, with 10-fold cross- volss-valiationn results summarized in Tabs. 4 and 5. Our models models-GGrooAtt-- SPDDS/DDDD///SSDDD are com- pared against state-of of-the-rrt baselines. The most effective manifold choice in GyroAtt varies by dataset. To be specific, GyroAu- SPSD-LCM achieves best accuracy on the MAMEM and BCI-ERN datasets, surpassing GDLNet by 3.2% % and 0.9%, respectively. On the BNC12014001 and BNC12015001 datasets, GyroAM--PD-AIM delivers the highest scores, outperforming TSMNet by 6.4%, 1.5%. 0.4%. and 0.9%, respectively across different evaluation settings. Notably. GyroAtt-Gr Gr consistently ranks second across all datasets and outperforms GDLNet ev- ery case. These experimental observations highlight the generality and effectiveness of
Manifold
Method
MAMEM
BCI-ERN
EBGNet [44]
33.7.772
74.3 71.9
ShallowCNet [59]
6.7
EBGiTCNNet [35]
55.5 4.75
77.1 + 2.5
Euclidean
FBCNet [46]
53.1
4..1
TCNet-Fusion [48]
2. 6.60
MBEEGSE [3]
5.35
2.3
SPD
MAu [55]
6.12
75.7 2.2
Grassmann
GDLNer [72]
65.5 15.5+-2.9
78.2 2.5
SyyoAu-SPD-AAe
66.3
75.8 3.3
GyIoAH-SPD-LEM
6.52
76.1
GynDAutSPD-LCM
6.51
75.4 3.7
Grassmann
GymoAt-Gr
67.1
78.4 1.4
GymaAn-SPSD-AA SPMD-AIM
66.5 1..5::20 2.3
78.2 1.9
SPSD
SrroAttSPSD-LEM
66.5 2.9
79.1 + 1.7
GyDDAR-SPSD-LCM
68.7
78.4
GtroAtt. The superior performance of GyroAtt be attributed to the underlying geometry-
7

---

Table 6: The impact of matrix power activation on the accuracy of GyroAtt-SPD/SPS
BNCI2014001
BNCI2015001
Geometry
Activation
MAMEM
Inter-session
Inter-subject
Inter-session
Inter-subject
w/o
74,8 + 6.7
51.2 + 15.7
85.5 + 5.0
75.4 + 12,9
64.3 + 2.4 66.3 + 2.2
SPD
w/
75.4 + 7.1
53.1 + 14.8
86.2 + 4.5
77.9 + 13.0
w/o
72.2 + 7.2
49.2 + 13.7
83,9 5.1
74.2 + 14.4
66.5 2.4
SPSD
w/
72,9 + 7.1
52.4 + 15.6
85.3 5.3
76.0 + 14.1
68.7 + 1.5
Table 8: Replacing hom() and power activation with BiMap and ReEig in GyroAH-SPD-LEM
BNCT2014001
BNCT2015001
Transformation
Activation
Inter-session
Inter-subject
Inter-session
Inter-subject
BiMap
Power
74.0 + 6.5
52.3 + 15.0
85.2 + 7.2
7.2
77.2 + 13.2
Homomorphisms
ReFig
75.1 + 6.3
52.6 + 14.2
85.9 +.3
76.4 + 12.8
BiMap
ReEig
73.6 1 6.8
52.2 +: 15.2
85.4
76.8 + 13.0
Homomorphismms
Power
75,4
53.1 1: 14.8
86.2 4.5
77.9 + 13.0
aware attention mechanism, which effectively captures long-range dependencies and spatiotemporal fluctuations inherent in EEG signals.
blations on the matrix power-based nonlinear activation (-). Tab. 6 illustrates the impact of the nonlinear activation (as defined in Tab. 3) on the accuracy of GyrroAtt-PDD/SPSS Generally, removing nonlinear activation results in performance degradation, e.g., GyroAtt experiences 2.0% and 2.4% accuracy drop on the MAMEM dataset. These experimental findings emphasize the importance of matrix power activation in enhancing model expressiveness by introducing nonlinearity into the metric space of the underlying feature manifold.
Gyro distance vs. gyro inner prod- uet. The Euclidean inner product is a natural similarity metric flat space, but global inner product is gener- ally unavailable on curved manifolds. Hence, most comifold-attention mod- els rely on distance- based similarity [26, 55, 72]. Gyrovector spaces an exception because a gyro inner prod- uct can be defined in the tangent space
Table 7: Accuracy comparison under different mcasures.
BNC12014001
BNC12015001 Inter-session
Similarity
MAMEM
GymAAn-SPD
Gyro inner product
74.7 =: 6.8
85.6 2: 5.4
63.9 :: 3.2
Gyro distance
75.4 = 7.1
86.2 = 4,5
66.3 1:
GyroAn-Gr
Gyro inne product
Gyro distance
72.5 = 7,3
72.4 = 7.3
N5.0 = 7.7
83.4 2:
67.1 = 1.6
65.7 :: 3.1
GyoAAn-SPSD
Gyro product
71.6
65.0
Gyro distance
72.9 = 6.2
85.3 2: 5.3
68.7 1: 1.5
at the identity [52][Defs. 2.9, 2.15]: (P, Q) (Logı (P), Logı (Q),, - To evaluate its suitability, replaced the distance-based similarity in Eq. (5) with gyro inner product and conducted an ablation study, as reported in Tab. 7. Across all datasets, the gyro distance consistently outperforms the inner product in accuracy. An essential reason is that the gyro inner product is defined within a single tangent space, which may not reflect the geometric structure faithfully. In contrast, gyro distance better preserves the intrinsic manifold geometry. The detailed implementations are found in App. E.
GyroAtt-SDD-LEM vs. MAtL. Although both GyroAM--PD-LEN and MAtt [55] implement attention mechanisms under the SPD manifold with LEM, GyroAtt-SPD SPD-LEM differs from MAtt in two key components: linear transformation hom(.), and the nonlinear activation (.)). While BiMap and ReEig used in MAtt preserve SPD property, they are loosely connected to the underlying Riemannian geometry. Hence, we replace hom(.) :) and a-)) with Bimap and ReEig and made an ablation study, reported in Tab. 8. Either replacement leads to a performance drop, while combining both further degrades accuracy. From a theoretical standpoint, gyro homomorphisms generalize linear maps and naturally reduce to them in the Euclidean setting. This indicates that our transformation layer is a principled extension of Euclidean attention. In contrast, the ReFig function merely rescales cigenvalues ensure positive definiteness, lacking geometric interpretability. As shown in [17][Fig. 1]. matrix power can deform any given Riemannian metric, effectively acting as a geometry-aware activation. These findings confirm that integrating garo- transformations and nonlinearities is not only theoretically justified but also empirically beneficial.
8

---

6,6689z
7.508tz
8:571
10.00Hz
1 12.000ttz
04-
PM
62
02
PO
00
0.3
3:34
6.7
L03
3.1
6.14
6.5
1.0
222
140
322
0.30
0.13
1.30
Time
Time
Time
Time
6,60H
505zz
8.57Hz
10,00Hz
12 12.00z
Electrddss
WIT
Figure 2: Spatial topomaps and gradient heatmaps of SSSD on the MAMEM (subject S11) dataset. Strong responses appear around the Oz electrode, especially within 0-0.5s. In heatmaps, the x-axis denotes time and the y-axis denotes EEG channels.
Correct
Emrr
Correct
Error
Electrodes
0.
0.005
6.05
0.40%
0.3
02325
1,025
0.0075
1.2
Time
Time
Figure 3: Spatial topomaps and gradient heatmaps of GyroAu-SPSDD, showing 'correct' vs. 'error' trials. Strong activations consistently emerge around the FCz electrode within 0. 0.1.0.4s.
Training efficiency. Gy- roAtt demonstrates superior or comparable efficiency to existing manifold attention networks such as MAtt and GDLNet. In particular, Au-DAU-SPD under LCM achieves efficiency on par with MAtt, benefiting from the computational advantages of the Cholesky decomposition. More detailed discussions and comparisons are provided in App. E.8.
Table 9: Running time (s/epoch) comparison.
BNC12014001
BNC12015001
Method
Inter-session
Inter-subject
Inter-session
Inter-subject
MAIL GDLNet
4.86
89.12
2.74
56.78
4.55
88.59
1.71
47.66
GyIoAM--PD-LCM
4.11
87.44
2.42
49.86
EEG interpretability analysis. For the MAMEM dataset, as shown in Fig. 2, GyroAtt shows strong gradient responses around the Oz electrode across five stimulus frequencies, especially within 0-0.5s time window. This pattern aligns well with the established findings the relationship between SSVEP signals and the Oz region [31, 28]. This is likely due to the Oz central location in the primary visual cortex, which leads to stronger evoked potentials and signal-to-noise ratio. On the BCI-ERN dataset, as illustrated in Fig. 3, gradient responses for both 'correct' and 'error' trials centered around the FCz electrode. This observation aligns with substantial empirical evidence that the anterior cingulate cortex, a central medial prefrontal cortex region connected to limbic and frontal areas, underlies ERN generation. Notably, consistent gradient responses around FCz were observed for both feedback types within the 0-140.4s interval, reinforcing Event- Relate- Potential (ERP) waveform differences between correct and error trials as reported in [27].
Conclusion
In this paper, we propose GyroAtt, a principled framework that generalizes the Euclidean attention mechanism to gyrovector spaces. Specifically, we adopt gyro homomorphism, gyro listance-based attention, and WFM as counterparts to the transformation, attention, and aggregation operations in Euclidean attention. Notably, we identify the concrete non-trivial expressions of gyro homomorphisms on different matrix gyro spaces. The principled construction of GyroAtt enables direct assessment of the impact of geometry on given task while keeping the network architecture constant. Extensive experiments and ablation studies on four EEG benchmarking datasets certify the effectiveness and flexibility of our proposed framework.

---

Acknowledgments and Disclosure of Funding
This work was supported in part by the National Natural Science Foundation of China (62306127, 62020106012, 62332008), the Natural Science Foundation of Jiangsu Province (BK20231040 BK20221535), the Fundamental Research Funds for the Central Universities (JUSRPI24015), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (SJCX25 (SICX25_1319) 1319), the Key Project of Wuxi Municipal Health Commission (Z202318). the EU Horizon projects ELIAS (No. 101120237), the ELLIOT (No. 101214398), and the National Key R&D Program of China (2023YFF1 2023YFF11051000
References
[1] P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematica, 80:199 199- 220, 2004.
[2] Bijan Afsari. Riemannian 1 center of mass: existence, uniqueness, and convexity. Proceedings of the American Mathematical Society, 2011.
[3] Ghadir Ali Altuwairri, Ghulam Muhammad, Hamdi Altaheri, and Mansour Alsulaiman. A multi-branch convolutional neural network with gqueeze-and-excitale attention blocks for EEG-based motor imagery signals classification. Diagnostics, page 995, 2022.
[4] Kai Keng Ang, Zheng Yang Chin, Haihong Zhang, and Cuntai Guan. Filter bank common spatial pattern (FBCSP) in brain-compute interface. In I/CNN, pages 2390-2399 IEEE, 2008.
[5] Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and simple computa- tions on tensors with log- Eugllucld metrics. PhD thesis, INRIA, 2005.
[6] Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. Multiclass brain- computer interface classification by Riemannian geometry. IEEE Transactions Biomedical Engineering. 59(4):920-92) 2011.
[7] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully hyperbolic convolutional neural networks for computer vision. In ICLR, 2024.
[8] Thomas Bendokat, Ralf Zimmermann, and P-A Absil. A Grassmann manifold handbook: Basic geometry and computational aspects. Adv. Comput. Math., pages 1-51, 2024.
[9] Silvere Bonnabel and Rodolphe Sepulchre. Riemannian metric and geometric mean for positive semidefinite matrices of fixed rank. SIAM Journal on Matrix Analysis and Applications, 31()): (055-1070, 2010.
[10] Silvere Bonnabel, Anne Collard, and Rodolphe Sepulchre. Rank-preserving geometric means of positive semi-definite matrices. Linear Algebru and its Applications, 438()):3202-3216, 2013.
[11] Daniel Brooks, Olivier Schwander, Frédéric Barbaresco, Jean- Yves Schneider, and Matthieu Cord. Riemannian batch normalization for SPD neural networks. In NeurIPS, pages 15463- 15474, 2019.
[12] Rudrasis Chakraborty, Jose Bouza, Jonathan H Manton, and Baba C Vemuri. Manifoldnet: A deep neural network for manifold-valued data with applications. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(2):799 -10, 2020.
[13] Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, and Josef Kittler. Rieman- nian local mechanism for SPD neural networks. In AAAI, 2023.
[14] Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, Xiao-Jun Wu. and Nicu Sebe. Riemannian multinomial logistics regression for SPD neural networks. In CVPR, 2024.
[15] Ziheng Chen, Yue Song, Yunmei Liu, and Nieu Sebe. Lie group approach to Riemannian batch normalization. In ICLR, 2024.
10

---

[16] Ziheng Chen, Yue Song, Xiao-Jun Wu, Gaowen Liu, and Nicu Sebe. Understanding matrix function normalizations covariance pooling through the lens of Riemannian geometry. arXiv preprini arXiv:2407. 10484, 2024.
[17] Ziheng Chen, Yue Song, Xiaojun, and Nicu Sebe. RMLR: Extending multinomial logistic regression into general geometries. arXiv preprint UrXiv:2409.1943: 2024.
[18] Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, and Nicu Sebe. Adaptive Log-Eucling Euclidean metrics for SPD matrix learning. IEEE TIP, 2024.
[19] Ziheng Chen, Yue Song, Xiaojun Wu, and Nicu Sebe. Gyrogroup batch normalization. In ICLR, 2025.
[20] Calin Cruceru, Gary Bécigneul, and Octavian- -ugeen Ganea. Computationally Tractable Rie- mannian Manifolds for Graph Embeddings. In AAAI, pages
2021.
[21] Thibault de Surrel, Sylvain Chevallier, Fabien Lotte, and Florian Yger. Geometry-ave aware visualization high dimensional symmetric positive definite matrices. Transactions on Machine Learning Research Journal, 2025.
[22] Alexey Dosovitskiy. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2020.
[23] Alan Edelman, Tomás Arias, and Steven T Smith. The geometry of algorithms with or- thogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2):303-353, 1998.
[24] Josef Faller, Carmen Vidaurre, Teodoro Dolis-Escalante, Christa Neuper, and Reinhold Scherer. Autocalibration and recurrent adaptation: towards plug and play online ERD-BCI Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 20(3):313-311 2012.
[25] Maurice Fréchet. Les éléments aléatoires de nature quelconque dans un espace distancié. In Annales de I' l'institut Henri Poincaré, 1948.
[26] Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, et al. Hyperbolic attention networks. arXiv preprint arXiv: 10fv::880.0099 1805.09786, 2018.
[27] Greg Hajcak. What we've learned from mistakes: Insights from error-related brain activity. Curr. Dir. Psychol., 2012.
[28] Chengeheng Han, Guanghua Xu, Jun Xie, Chaoyang Chen, and Sicong Zhang. Highly interac- tive brain-computer interface based on flicker-free steady-state motion visual evoked potential. Sci. Rep., 2018
[29] Lea Hehenberger, Reinmar Kobler, Catarina Lopes-Dias, Nitikorn Srisrisawang, Peter Tumfart, John B Uroko, Paul R Torke, and Germot R Müller-Putz. Long-tems term-term mutual training for the CYBATHLON BCI race with a tetraplegic pilot: a case study on inter-session transfer and intra-session adaptation. Frontiers in Human Neuroscience, 15:635777, 2021.
[30] Kyle Helfrich, Devin Willmott, and Qiang Ye. Orthogonal recurrent neural networks with scaled cayley transform. In International Conference on Machine Learning, pages 1969-1978, PMLR, 2018.
[31] Christoph S Herrmann. Human EEG responses to 1-100 hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp. Brain Res., 2001.
[32] Jie Hu, Shen, and Gang Sun. Squeeze-and-exciiatio networks. In CVPR, pages 7132-7141. 2018.
[33] Zhiwu Huang and Lue Van Gool. A Riemannian network for SPD matrix learning. In AAAI, pages 2036-2042, 2017.
[34] Zhiwu Huang, Jiqing Wu, and Lue Van Gool. Building deep networks on Grassmann manifolds. In AAAI, pages 1137-1145, 2018.

---

[35] Thorir Mar Ingolfsson. Michael Hersche, Xiaying Wang, Nobuaki Kobayashi, Lukas Cavigelli, and Luca Benini. EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces. In SMC, pages 2958-2965, 2020.
[36] Ce Ju and Cuntai Guan. Graph neural networks on spd manifolds for motor imagery classifica- tion: A perspective from the time-frequency analysis. JEEE Transactions on Neural Networks and Learning Systems, pages 1-15, 2023. doi: 10. 10.1109/TNNSS WWTTNNNSSS2022333000
[37] Ce Ju, Reinmar J Kobler, Liyao Tang, Cuntai Guan, and Motoaki Kawanabe. Deep geodesic canonical comrllation analysis for covariance- neuroimaging data. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
[38] Hermann Karcher. Riemannian center of mass and mollifier smoothing. Communicutions Pure and Applied Mathematics, 30:509-541, 1977.
[39] Hermann Karcher. Riemannian center of mass and mollifier smoothing, Communications on pure and applied mathematics, 30(5):509-555) 1977.
[40] Sejong Kim. Ordered Gyrovector Spaces. Symmetry, 12(6), 2020.
[41] Reinmar Kobler, Jun-ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD domain- specific batch normalization to crack interpretable unsupervised domain adaptation EEG. NeurIPS, 35:6219-62355 2022.
[42] Reinmar Kobler, Jun-Ichiro Hirayama, Lea Hehenberger, Catarina Lopes-Dias, Gernot R MüllerrPutz, and Motoaki Kawanabe. On the interpretation of linear Riemannian tangent space model parameters M/EEG. In EMBC, pages 5909-5915 5913. IEEE, 2021.
[43] Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian Optimization in PyTorch, 2020. _eprint: 2005.00 0905.0
[44] Vernon Lawhern, Amelia Solon, Nicholas R Waytowich, Stephen M Gordon, Chou Hung, and Brent J Lance. EEGNet: a compact convolutional neural network for EEG-based crmin-computer interfaces. J. Neural Eng., page 056013, 2018.
[45] Zhenhua Lin. Riemannian geometry of symmetric positive definite matrices via Cholesky decomposition. SIAM Journal on Matrix Analysis and Applications, 10(3):1353-133)) 2019.
[46] Ravikiran Mane, Effic Chew, Karen Chua, Kai Keng Ang, Neethu Robinson, A Prasad Vinod, Seong-Whar Lee, and Cuntai Guan. FBCNet: A multi-view convolutional neural network for brain-computer interface. arXiv preprint arXiv:2104.0123: 2021.
[47] Perrin Margaux, Maby Emmanuel, Daligault Sébastien, Bertrand Olivier, and Mattout Jérémie. Objective and subjective evaluation of online error correction during P300-based spelling. Adv. Hum-Comput. Interact., pages 1-13, 2012.
[48] Yazced K Musallam, Nasser AlFassam, Ghulam Muhammad, Syed Umar Amin, Mansour Alsulaiman, Wadood Abdul, Hamdi Altaheri, Mohamed A Bencherif, and Mohammed Al- gabri. Eleetroencephalogrrphhy based motor imagery classification using temporal convolutional network fusion. Biomed. Signal Process. Control, page 102826, 2021.
[49] Galileo Namata, Ben London, Lise Getoor, Bert Huang, and Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, 2012.
[50] Xuan Son Nguyen. A Gyrovector Space Approach for Symmetric Positive Semi-definite Matrix Learning. In ECCV, pages 52-68, 2022.
[51] Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds. In NeurlPS, pages 26618- 26630, 2022.
[52] Xuan Son Nguyen and Shuo Yang. Building Neural Networks on Matrix Manifolds: Gyrovector Space Approach. In ICML, volume 0atp223050045600 2023.
12

---

[53] Xuan Son Nguyen. Shuo Yang, and Aymeric Histace. Matrix manifold neural networks++. arXiv preprint arXiv:2405. 2024.
[54] Spiros Nikolopoulos. MAMEM EEG SSVEP Dataset (256 channels, 11 subjects, frequen- cies presented simultaneously). Dataset, 2016. URL https://apl. www.stisshoooialaa.... httppsss/ org/ CorpusID: 63262529.
[55] Yue-Ting Pan, Jing-Lun Chou, and Chun-Shu Wei. MAtt: A manifold attention network for EEG decoding. In NeurIPS, pages 31116-31129, 2022.
[56] Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1):411 2006.
[57] Pedro Luiz Coelho Rodrigues, Christian Jutten, and Marco Congedo. Riemannian procrustes analysis: transfer learning for brain computer interfaces. IEEE Transactions on Biomedical Engineering, 66(8):2390-2411, 2018.
[58] Takashi Sakai. Riemannian geometry, volume 149. American Mathematical Soc., 1996.
[59] Robin Tibor Schirreeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp.,, pages 5391-5420, 2017.
[60] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. magazine, 2008.
[61] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. arXiv preprint arXiv:2006.088: 082 10, 2020.
[62] Anuj Srivastava and Eric Klassen. Bayesian and geometric subspace tracking. Adv. Appl. Prob., pages 43-56, 2004.
[63] Teerapong Suksumran and Keng Wiboonton. Isomorphism theorems for gyrogroups and I-subgyrogroups. arXiv e-prints, pages arXiv-1406, 2014.
[64] Michael Tangermann, Klaus-Robent Müller, Ad Aertsen, Niels Birbaumer, Christoph Braun, Clemens Brunner, Robert Leeb, Carsten Mehring, Kai J Miller, Gernot Müller-Putz, Guido Nolte, Gert Pfurtscheller, Hubert Preissl, Gerwin Schalk, Alois Schlögl, Carmen Vidaurre, Stephan Waldert, and Benjamin Blankertz. Review of the BCI Competition IV. Frontiers in Neuroscience, 6, 2012.
[65] William P HG Thurston. Three-Dimensional Geometry and Topology, Volume 1: Volume 1. Princeton university press, 1997.
[66] Abraham A Ungar. Analytic hyperbolic geometry: Mathematical foundations foundatticaluuudaioonn and applications. World Scientific, 2005.
[67] Abraham Albert Ungar. Beyond the Finstein Addition Law and Its Gyroscopic Thomas Preces- sion: The Theory of Gyrogroups and Gyrovector Spaces. Fundamental Theories of Physics, vol. 117, Springer, Netherlands, 2002.
[68] Abraham Albert Ungar. Analytic Hyperbolic Geometry: Mathematical Foundations and Applications. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2005.
[69] Abraham Albert Ungar. Analytic Hyperbolic Geometry in N Dimensions: An Introduction. CRC Press, 2014.
[70] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[71] Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang, and Peihua Li. Deep CNNs meet global covariance pooling: Better representation and generalization. IEEE transactions on pattern analysis and machine intelligence, 43(8):2582-255 2020.
13

---

[72] Rui Wang, Chen Hu, Ziheng Chen, Xiao-Jun Wu, and Xiaoning Song. A Grassmannian manifold self-attention network for signal classification. In IJCAI, 2024.
[73] Rui Wang, Xiao-Jun Wu, Ziheng Chen, Cong Hu, and Josef Kittler. SPD manifold deep metric learning for image set classification. IEEE TNNLS, 2024.
[74] Or Yair, Felix Dietrich, Ronen Talmon, and Ioannis G Kevrekidis. Domain adaptation with optimal transport on the manifold of spd matrices. arXiv preprint arXiv:1906.0061: 2019.
[75] Wei Zhao, Federico Lopez, J Maxwell Riestenberg, Michael Strube, Diazeldin Taha, and Steve Trettel. Modeling graphs beyond hyperbolie: Graph neural networks in symmetric positive definite matrices. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2023.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims in the abstract and introduction consistent with the theoretical contributions and experimental validations.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in App. A.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
15

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The full assumptions and formal proofs for Thm. 4.1, Thm. 4.2, and Thm. 4.3 are provided in App.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
We provide sufficient details in Sec. 5 and App. to reproduce all main experiments, including datasets, model settings.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
16

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
We will release the code upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/ onPolicy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
httpe/guuideee/eeuubbubioiieeee Coweeoomissseseeeeccccuu ssisssolicyyyiiii for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe all necessary training details in Sec. 5. Additional implementation details are provided App. F.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
17

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report mean + standard deviation over 10 independent runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: All experiments were conducted on a machine with an Intel i9- 14900 CPU, 64GB RAM, and two NVIDIA RTX 4080 Super GPUs. Further details are provided in App. E.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: htt///nuriip.../ httpp////......c/.cccccccceeeeee
18

---

Answer: [Yes]
Justification: We have reviewed and adhered to the NeurIPS Code of Ethics. Our research involves publicly available datasets and does not pose foresecable risks related to privacy, safety.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work focuses on foundational research and is not tied to particular applications. Therefore, societal impacts are not directly applicable this stage.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not involve pretrained language models, generative models, or scraped datasets.
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets and code used in this work are publicly available and properly cited in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
20

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy ptttps://neuripee htt.:/ppps..c/ooeeeeeeeee////220 /LLM) for what should or should not be described.
21

---

Appendix Contents
Limitations
24
Notations
24
C Abbreviations
24
Preliminaries
25
D.1 Gyrogroups and gyrovector spaces
25
D.2 Weighted Fréchet Mean
25
D.2.1 Existence and uniqueness of the Weighted Fréchet Mean
25
D.2.2 Weighted Fréchet Mean on SPD manifolds
26
D.2.3 Weighted Fréchet Mean on Grassmannian manifolds
27
D.2.4 Weighted Fréchet Mean on SPSD manifolds
27
D.3 Canonical representation in SPSD manifolds..
28
Implementation details and additional experiments
28
E.1
Datasets and EEG preprocessing
28
E.2
Additional implementation details
30
E.2.1
Network architectures.
30
................................
E.2.2
Optimization
31
................................
E.3
Statistical Significance Analysis
31
Graph node classification and radar recognition tasks
32
E.5
Additional ablations studies
32
E.5.1
Ablations on the Riemannian metrics and matrix power- power-based based nonlinear
activation
32
E.5.2
Ablations on the matrix power normalization
33
E.5.3
Ablation on the number of GyroAtt blocks
34
E.6
Implementation details of gyro inner product.
34
E.7
Implementation details of replacing geometric components in GyroAtt
34
E.8
Computational complexity analysis and comparison
35
E.8.1
Number of matrix functions in manifold attention mechanisms
35
E.8.2
Experimental comparison of time efficiency..
36
E.9
Model compactness and parameter efficiency
37
E.10
Long-range dependencies.
37
E.
Visualizing learned representations via Rie-SNE
37
22

---

F Proofs of the Theorems in the Main Paper
37
F.1 Proof of the Thm. 3.2
37
F22 Proof of the Thm. 4.1
38
F3 Proof of the Thm. 4.2
40
F,4 Proof of the Thm. 4.3
41
23

---

A Limitations
While several widely used manifolds, such as hyperbolic, SPD, Grassmannian, and SPSD manifolds, admit gyrovector structures, this property does not hold universally across all manifolds. As result, our method not directly applicable to manifolds lacking such a structure. Extending GyroAtt to accommodate more general -onryyyvveetor manifolds constitutes promising direction for future research.
B Notations
To enhance readability, we summarize the main mathematical symbols and operators used throughout this paper in Tab. 10.
Table 10: Summary of notations.
Notations
Explanation
(C,@)
A gyrogroup G with binary operation ? Space of d SPD matrices Space of symmetric matrices Space of d x d SPSD matrices with rank S
S7
S*
St.
G(g.4)
Grassmannian the projector perspective
(gg.d)
Grassmannian the ONB perspective
Puis Sai
Binary, inverse, and scalar multiplication operations in Sj under AIM Binary, inverse,
Bier Ster Sie
multiplication operations S under LEM Binary, inverse, and scalar multiplication operations in Sj under LCM
Gtc Oiei Øte
Gan Bar. Ser
Binary, inverse, and scalar multiplication operations in G(g. Binary, inverse, and scalar multiplication operations in Gig. d) Binary, inverse, and scalar multiplication operations in Gig. d) S under metrics Inner product in Sj under metrics g
Pgr Ogn Dgr
pad Spoddo Spntss
IP. Q)
(U, Vier
Inner product in Sig. d)
(U Spi, (Ug. SygBnedo
Inner product in G(g. x S under metrics y
10 a Q 4M
the gyrodistance S under metrics the gyrodistance in Gly,d)
@ U
or vi
(6 U Bp Uq S o2 S
the gyrodistance Glg d) S under metries g
be] expmn-)) logau())
the matrix commutator
Matrix exponentiation logarithm
(),,, 2 (() D(-)
Cholesky decomposition and its inverse
diagonal matrix with diagonal elements from square matrix strictly triangular part of a square matrix
H
LogE' (Q)
Logarithmic map of Q at P in G(g.
M.N
Matrix manifold
WFM
weighted Fréchet mean
hom,: ()), homm)))) hom;a()
the maps in SJ+ under AIM, L.M,, and ICM satisfying gyro homomorphism
homgr
the maps Ōlg. d) satisfying gyro homomorphism
ho
the maps in Gig. d) x S5* under metrics satisfying gymo homomorphism
||=||p
The norm induced by the standard Frobenius inner product
O(d)
special orthogonal group
Expe'(A (A.)
Exponential map of A at P in S under AIM
Logg (Q2)
Logarithmic map of Q
in S under AIM
Expo (W)
Exponential map of W at G(g.
Expx (H)
ot
Exponential map of H X Gig. Logarithmic map Q P Glg.
Logp (Q)
C
Abbreviations
For completeness, the abbreviations appearing in this paper are listed below for easy reference.
List of Abbreviations
AIM
InvinanI Metric Area Under the Curve
2, 3, 5, 6
AUC
24

---

DNN
Deep Neural Network
EEG
Fleetroencephalography
1,2,6-9
ERP
Event-Related Potential
9
GyroAtt
Gyro Attention
1-7,9
LCM
Jog-Cholesky Metric
2, 3, 5,
LEM
Log-Euclidean Metric
1-6
ONB
Orthonormal Basis
2-6
SPD
Symmetric Positive Definite Symmetric Positive Semi-Definite
1-9
SPSD
1-9
WFM
Weighted Fréchet Mean
2-6
D Preliminaries
D.1
Gyrogroups and gyrovector spaces
Gyrogroups and gyrovector spaces generalize groups and vector spaces, offering powerful frame- work to analyze non-Fuclidean geometries. Below, we formally present their definitions.
Definition D.1 (Gyrogroups [69]). gyrogroup is generalization of groups. Let G be nonempty set with a binary operation () and an identity element E E G. A pair (G, (B) a gyrogroup satisfies the following axioms:
(GI) There exists an identity element G such that for all A G, E A AA
(G2) For each A E G, there exists a left inverse eA E G satisfying 2A = E.
(G3) For all A, B, C G, there exists automorphism gyr (A, () )) G G. satisfying
A B + C) = (A e B) + gyr A, (C).
(12)
Here, the map gyr AA B] called the gyroautomorphism,
the gyration of G generated by A, B.
(G4) For all A, B G. The map gyr[A, B] generated by each A, satisfies the left loop property: gyr[A, BJ gyr A + B, B).
Definition D.2 (Gyrocommutative Gyrogroups [69]). gyrogroup (G, @) gyrocommutative it satisfies the gyrocommutative law: A + B = gyr A, ( + A) for all A, G.
The following definition of gyrovector spaces derived from Nguyen [51, Def. 2.3], which is slightly different from Ungar [69, Def. 3.2].
Definition D.3 (Gyrovector Spaces [51]. A gyrocommutative gyrogroup (G, @) equipped with sealar multiplication @ RR x G G is gyrovector space if the following axioms satisfied:
(V1) 1 A
E E, and (-1) & A eA.
(V2) (8 + 8 A = K A + 2 A.
(V3) (st) 8 A @ & A).
(V4) gyr A, B (1 C) @ gyr (A, BJC.
(V5) gyr 8 A, f A 1d, where Id the identity map.
D.2 Weighted Fréchet Mean
D.2.1
Existence and uniqueness of the Weighted Fréchet Mean
The WFM is a central tool in manifold-based learning for aggregating features that reside on non- Fuclidean spaces. Formally, given set of points (x;) } on Riemannian manifold M with associated non-negative weights (w;) summing to one, the WFM defined as the minimizer of the weighted sum of squared distances.
25

---

Table 12: Uniqueness and Solution of WFM.
Goometry
SPD-AIM
SPD-LEM
SPD-LCM
Grassmann
SPSD
Uniqueness of WFM Solution of WFM Reference
Glohal
Global
Global
Within r <. 4V7
The Grissmann component within r 107
Karcher Flow
Closed Form
Closed Form
Karcher Flow
WFM WFM,
[56]
[45]
[2]
[10]
The uniqueness of WFM is generally guaranteed when all sample points {z;} lie within geodesically convex hull of M [2]. In our paper, we focus on several commonly used manifolds, including SPD manifolds with various metrics (AIM, LEM, LCM), the Grassmann manifold, and the SPSD manifold. As summarized in Tab. 12, the WFM is globally unique for SPD manifolds under AIM, LEM, and LCM. For the Grassmann manifold, uniqueness holds when the radius r of the geodesic ball satisfies < 4v2 [2]. The same condition applies to the Grassmann component of the SPSD manifold under its canonical form. Following prior work [12, Sec. 2],
Uniqueness.
assume that the WFM is well-defined in our settings.
Computation. In terms of solution methods, the WFM can be computed via iterative optimization (e.g., Karcher flow) or, in some cases, closed-form expressions. For instance, SPD-LEEM and SPD- LCM permit closed-form solutions [5, 45], while SPD-AIM and Grassmann manifolds typically require gradient-based optimization [56, 2, 39]. For SPSD, we follow the product formulation [10]. where the WFM is computed by combining the SPD and Grassmann components.
D.2.2
Weighted Fréchet Mean on SPD manifolds
Algorithm 1: Karcher Flow Algorithm on the SPD Manifold under AIM
Input
A set of SPD matrices X1...N E S++ A set of weights W1... with Σ, wi Number of iterations K TTe WFM G5 E S
Output Initialize Go I for is f to do
G ExPGA-1
ef
EEizz wa Logg,., (X:)
N
end
We begin by introducing the exponential and logarithmic maps under the affine-invariant metric (AIM), followed by the Karcher flow algorithm.
Affine-Invarian Metric.
On the manifold S endowed with AIM, the exponential map point P St is given by [1]:
Expp ai (A) P expm
[P AP A P
#)
(13)
where A E TpSt+ d exponential map, defined as
is tangent vector at P. The logarithmic map, which is the inverse of the
ni Logp' (Q) = P logm (p- P--QP-- p
(14)
for any Q E St
As shown in Alg. 1, the Karcher flow algorithm computes the weighted Fréchet mean (WFM) on the SPD manifold through an iterative process. In each iteration, the data points are projected onto the tangent space at the current estimate Gg-1 using the logarithmic map (Eq. (14)), a weighted average is calculated in this tangent space, and the result is mapped back to the manifold using the exponential map (Eq. (13)). This algorithm is guaranteed to converge on manifolds with non-positive curvatures, such S [38]. We initialize Go the identity matrix I and set the number of iterations K 1.
Log-Euclidean Metric. Under the log-Euclidean metric (LEM), the WFM has a closed-form expression provided by Chen et al. [15]:
we logmloXX (X.))
G = expm
(15)
where X SJ+, WI... 0, and E, wi =
26

---

Similarly, for the oggCChlesky metric (LCM), the WFM also admits closed-form solution as shown by Chen al. [15]:
Log-Choles Cheleskky Metric.
N
N
G = L--1
+ II (((())))
(16)
1=1
where X E ESt WI...N > and Σ, w2 w =1
D.2.3 Weighted Fréchet Mean on Grassmannian manifolds
We now present the exponential and logarithmic maps, as well as the parallel translation under the ONB perspective, followed by the project perspective.
For the Grassmannian manifold G(g. d) in the ONB perspective, the exponential map at X g(g. d) is defined as
gr (H) XV cos Σ + U sin Σ, Expx
(17)
where H is tangent vector X, and UEVT is the thin singular value decomposition (SVD) of H:
UΣV thinSVD(H).
(18)
The logarithmic map, which the inverse of the exponential map, given by
Logx (Y) U tan -1 EV
(19)
where X, Y e G(q. d), and
UEV = thinSVD (II ([--XX) XX XXX
(20)
As stated in Edelman et al. [23, Theorem 2.4]. let H and A be tangent vectors at point Y on the Grassmann manifold. The parallel transport of A along the geodesic in the direction Y = H is given by 7(()) (rV U) (- (oo((i)))
U + +a (()-v))) A.
(21)
Shifting to the projector perspective for the Grassmannian manifold G(g, d), let P E G(g, and A TpG(q, d). The exponential map is defined [8]
Exp3" (A) (I PI) expm(-(A,,P)
(22)
As shown by Sakai [58], two points are in each other's cut locus if there exists more than shortest geodesic connecting them. When the exponential map Expø" is restricted to the injectivity domain 1Dp, for any F E G(g, d) Cutp, there exists a unique tangent vector A E IDp C TpG(g, d) such that Exp3" or ( = F. For such a point F, the logarithmic map is given by
Logg or (Q) = [S2, PI.
(23)
where P, Q E Gra.p and calculated as
= log (II, (I, - -2P)).
(24)
2
As shown in Alg. 1, the Karcher flow algorithm computes the WFM on the Grassmannian manifold through iterative process. We initialize Go the identity matrix X; and the number of iterations K 1.
D.2.4 Weighted Fréchet Mean on SPSD manifolds
As demonstrated by Bonnabel and Sepulchre [9], the WFM for batch of points X E S can be expressed as (WFMg. (U')). WFM spd (S))). Here, WFMg denotes the WFM on the Grassmannian manifold, while WFM represents the WFM on the SPD manifold under metric The matrices U and S correspond to the canonical representation of X..
27

---

Algorithm 2:
Karcher Flow Algorithm on the Grassmannian Manifold under ONB Perspective
Input: A set of Grassmannian points X1 N E d) A set of weights w1...N > with E, w, =
Number of iterations K
Output: The WFM G & ((g,d) Initialize GD X for k 6 to do gr
G. ExpGa-1 (Ei=l
gr wi w;LogGr-1 (Xi)
end
Algorithm 3: Computation of Canonical Representation in SPSD manifold
Input: A batch SPSD matrices X1...N E S
A constant & E [0
Output: A batch of Canonical Representation (Ux, Sx)]=1,.... of SPSD manifold
U In.gi
(U, E, Vi)e=...... (V
(U.)....... + (U,): d))==I....NG
if training then
U f GrMean (U,) U t GrGeodesic UTh U,7)
end
for
i 1-+1 to N do
(U.) TUT (c(cos V (Ux, S (U,Y, V.Y,
end
D.3 Canonical representation in SPSD manifolds
Nguyen et al. [53] introduced a canonical representation of P in the structure space G d) x S++. As shown Alg. 3, we follow this approach to derive the canonical representation of each point in Si.g. Canonical Representation of SPSD matrices is obtained in three steps. This first is to impose decomposition on X,, i..,, X, = U,E,U,, where U, E Õ(g, d) and E; E St+,. Then the mean of U.).= the common subspace, and rotated (U*, Σ') to the identified common subspace, denoted as (U Sx Here, GrMean((U,))
computes the Fréchet mean of its arguments, described in Alg. 2, with weights set to WI....N N. GrGeodesic U, computes a point on a geodesic (Eq. (21)) from UT U at step 7 (7 = 0.1 in our experiments).
E Implementation details and additional experiments
E.1 Datasets and EEG preprocessing
MAMEM-SSVEP-IL. This dataset includes EEG recordings from 11 subjects performing SSVEP tasks. Participants focused on one five visual stimuli flickering at different frequencies for five seconds. Each subject completed five sessions, with five trials per stimulation frequency in each session. EEG signals were captured with 256 channels at a sampling rate 250 Hz.
This dataset involves 26 subjects in a P300-based spelling task to measure ERN. EEG data were recorded from 56 electrodes following the extended 10-20 system sampling rate of 600 Hz. Each subject underwent five sessions: the first four with 60 trials each and the fifth with 100 trials. We used data from 16 subjects available in the initial competition release.
BCI-ERN.
BNCI2014001. This dataset comprises EEG recordings from subjects performing four motor imagery tasks: imagining movements of the left hand, right hand, both feet, and tongue. Each subject participated in two sessions on different days, each containing runs. Each run included 48 trials-12 12 per class-totaling 288 trials per session.
28

---

Concat
Qu
B
FC Softmax
aG.
Class, imagine
R
left hand
Weighted Focchee Meen
CymAAi
R
ban()
K2
42
Manifold Modeling
R2
<dr.
((
ham[)
V/
Cony Block
Flattening
manifold
Gyro addition
EEG Skgnels
hom(-): Gyro Homomorphism
af Nonlinear activation
Figure The GyroAtt network architecture comprises three components: a feature extraction module that converts EEG signals into manifold-valuee data, Gyro Attention module that explicitly captures long-range dependencies among features, and a classification module that flattens manifold data for fully connected layer and softmax function.
BNCI2015001. EEG signals were recorded from electrodes centered around positions C3, Cz, and C4, according the International 10-20 System. Data collected using g.GAMMAsys active electrode system with a g. U.SBmmp amplifier, sampled at 512 Hz. with a bandpass filter between 0.5 and 100 Hz and notch filter 50 Hz.
EEG Preprocessing. For the MAMEM-SSVEP-I dataset, adhered the preprocessing protocol of Pan et al. [55]. The steps included: (1) band-pass filtering between 1-50 Hz; (2) selecting eight channels (PO7, PO3, PO, PO4, POS, 01, Oz, and O2) located in the occipital area corresponding to the visual cortex; and (3) segmenting each trial into four 1-second segments from to 5s after cue onset. This resulted in 500 trials of I-second, 8-channel SSVEP signals per subject, with each input EEG segment comprising 125 time points.
For the BCI-ERN dataset, we followed the preprocessing procedure outlined by Pan et al. [55]. The steps involved: (1) downsampling the signals from 600 Hz 128 Hz; (2) applying band-pass filter between 1-40 Hz. After preprocessing, each trial consisted of 56 channels with 160 time points.
For the BNC12014001 and BNC12015001 datasets, we followed the preprocessing steps described by Kobler al. [41]. Using the Python packages moabb and mne, we resampled the EEG signals to 250/256 Hz, applied temporal filters to extract oscillatory activity in the 4-36 Hz range, and extracted short segments seconds) associated with class labels.
Table 13: GyroAu-SPPD architectures across four datasets. The q is the rank the SPSD matrices.
MAMEM-SSVIP.II
BCI-ERN
BNC12014001
BNC12015001
Operation
Input data
1 125
X 56 160
1 x 750
X 13 768
TempConv
750
Convolution Convolution
SparCony
21 x 126
57 x 1 161
43 x 1 750
44 1 768
Split CovPool
x 21
3 19
6 13
3 41
Split Covariance
Alignment
SPDDSMAN
wlo
wio
8 43
x x
Kobler al. [4]]
SPSDCam
(19
g)
(43
18)
Alg. 3
SyPDAu-SPPD
2 (21 x
3 (19 x 9. x g)
6 [43
18
18)
3x (44
18
18)
Alg
R2F
2.x (21 21,9
3 x (19) 19.9
6 x
(43
43, 18
18)
3xx (44 x
18)
(.)) el(
Flat
(882, 162)
(1083, v²)
(11094,
1944)
(5547, 972)
Vectorization
Classifier
2
FC + Softmax

---

Table 14: GyroAtt-SPP SPD architectures across four datasets.
Block
MAMEM-SSVEP-I
BCI-ERN
ANC1201400)
BNC12015001
Operation
Input data
1 x 8 125
x 56 160
x 22
750
1X 13
768
TempConv
125 x 125
22 1 160
x x 750
5 13
768
Convolution
SpatCony
21 x 126
57 x x 161
43 x
750
44 x
768
Convolution
Split & CovPool
2 x x 21
x 19 19
6 x 43
43
3 44
44
Split + Covariance
SPDDSMBN
w/o x
w/o
6 43
43
3 x 44
44
Domain Alignment
GyroAm-SPD
x 19 19
6
3 3
44
Alg. 4
R2E
3 19 19
3
v(-)
882 5
1083
11094
5547
Vectorization
Classifier
2
FC Softmax
Table 15: GyroAtt-Grr Architectures across four datasets.
The is the dimension of the linear
subspaces.
Block
MAMEM-SSVEP.II
BCI-ERN
BNC1201 4001
BNC12015001
Operation
Input data
1 x x 125
x 56 160
1 x 22 x 750
X 13 X 768
TempConv
125 x 1 125
22 x 1
160
x 22 x 750
x 13 x 768
Convolution
SpatConv
21 x 1 126
57 x x 161
43 x 750
43 x X 768
Convolution
Split & CovPool
2 x
21
3 19 19
6
3
Split + Covariance
SPDDSMBN
w/o
w/o
6
3 x
Domain Alignment
GrCom
2 21 9
3 x x
3
Alg.
GyroAtt-Gr
2
3
Alg.
R2E
3 19 19
6
3 x 44
44
4b--)
Flat
882 5
1083 2
11094
5547
Vectorization FC Softmax
Classifier
4
2
E.2
Additional implementation details
E.2.1 Network architectures.
Fig. 4 and Tab. 13 provide summary of the specific network architectures of GyroAtt-SPSD across the four datasets, and summarize the forward pass of GyroAtt in Alg. 4. The network structures for GyroAtt-Gr (Tab, 15) and GyroAtt-SPD (Tab. 14) are identical to that of GyroAu-SPPDD, We just introduced SyroAtt-SPSD
example.
For the MAMEM-SSVEP-II and BCI-ERN datasets, the initial convolutional block consists of a convolutional layer, followed by batch normalization and ELU activation function. The subsequent convolutional block performs depthwise spatial convolution. A pointwise convolution, batch normal- ization, and another ELU activation follow this. In the MAMEM-SSVEP. dataset, features are split into two non-overlapping segments, followed by covariance pooling. For the BCI-ERN dataset, the second convolutional block repeated in two additional blocks. The outputs from these blocks are concatenated along the channel dimension. The data is then split along the channel dimension, and covariance pooling is applied, resulting in three covariance matrices.
For BNC12014001 and BNCI201 5001 datasets, the initial convolutional layer employs or 5 filters with a kernel size of (1, 25), performing temporal convolution while maintaining the same size through padding. The second convolutional layer applies spatial convolution with a kernel size of (22, 1) to integrate information from different channels. The output sequences undergo temporal pyramid partitioning, dividing each sequence into equal segments at the i-th level (with levels set to and 2, respectively). To address distribution shifts across subjects and runs, we incorporate subject- and run-specific batch normalization layers [41].
The attention module designed the gyrovector spaces is constituted by five operation layers, which the Gyro homomorphism layer (fhom) used to generate Q., K,, and V, for each input data, the similarity measurement layer (fsim) for computing the correlation between Q; and K,, the Softmax layer (famx) used normalize the obtained attention matrix along the row direction, the weighted Fréchet Mean layer (fwFM) ) for the implementation of weighted aggregation, and the power- based nonlinear activation layer (Fpac) used to improve the representational capacity of GyroAtt module by introducing nonlinearity to the underlying metric space.
For classification, our GyroAutSPD model employs matrix power normalization following Wang et al. [71] and Chen et al. [16]. Specifically, we apply the transformation (P)) ) to the i-th
30

---

Algorithm 4: Gyro Attention over gyrovector spaces
Input: A set of manifold-valued features 1N....
Output: A set of manifold-valued features (R
for i 4- 1 to N do
Queries: Q; hom X;),
Keys: K, = hom X )
Values: V; hom X;)
end for
--1 to N do
for j 1 to N do
Similarity: Sij (1 log(1 dQ, K,)))
end
Attention calculation: A.j Softmax(S.)
Aggregation: R. WFM {=
j=1: (V, j====
Bias and nonlinearity: R RR B)
end
output matrix P E St+ where > O. The coefficient 1 stabilizes the gradient flow during training and facilitates convergence. In GyroAtt-Gr,
ş transform elements Y, G(g, d) by applying projection operator ((;)) Y.Y. ! to map them into the corresponding flat space. In contrast, for GyrDAtt-SPSD, we project (U x x e G(q. S onto their respective manifolds. In all three GyroAtt, the transformed matrices are vectorized, concatenated, and fed into a fully connected layer followed by Softmax function. All experiments were conducted on an 19-14900 CPU with 64GB RAM and two NVIDIA RTX4080 Super GPUs.
E.2.2 Optimization
We address the optimization of parameters that are SPD matrices by modeling them within the space of symmetric matrices and applying the exponential map to the identity matrix.
For any parameter P E [dd, q). we parameterize it using matrix B Rudd-9 such that
0
B
[oog (PP (PP)),L -npl-
(25)
B
0
With this parameterization, the parameter P can be computed as
B B 0
) Inpp
exp
To optimize parameters O E (0)n), we start by generating parameter A E R then compute its skew-symmetric matrix S = A A. With this parameterization, the parameter P can be computed
0 - S) (I+
(26)
This approach enables us optimize all parameters within Euclidean spaces, eliminating the need to employ optimization techniques specific to Riemannian manifolds.
E.3 Statistical Significance Analysis
To assess the robustness of the reported improvements, we conducted paired 1-tests comparing GyroAtt with the baseline MAtt across all evaluation folds. Tab. 16 reports the average performance gains and corresponding significance levels, where denotes p-value < 0.05 and n.s. indicates "not significant".
Although some confidence intervals overlap, the observed improvements are consistent across benchmarks. In particular, the +1488 gain under the inter-subject condition-knoww to be the most challenging EEG setting-dee demmonstatssss the practical effectiveness and robustness of GyroAtt.
31

---

Table 16: Paired 1-test results comparing GyroAtt with MAtt. Asterisks (*) denote statistical significance (p 0.05).
BNC12014001
BNC12015001
BCI-CHA
MAMEM
Inter-session
Inter-subject
Inter-session
Inter-subject
Gains (%) Significance
7.8 8.9
5,4
14.8
3,4 *
3.2
*
n.S.
n.s.
E.4
Graph node classification and radar recognition tasks
While our primary evaluation focuses on EEG decoding, we further assess the generalization capability of GyroAtt on two additional domains: graph node classification and drone recognition.
Table 17: Accuracy (%) on graph node classification benchmarks.
Method
Pubmed
Cora
H-GAT
77.5 1 1.6
78.1 + 1.1
SPD-GAT
77.8 1
79.4 1.4...6 0.6
-AD-GAT-MAt
77.5
80.1 + 1.1
SPD-GAT AAro-Gy
78.5 = 0.4
81.6 + 1.0
Graph node classification. We conduct experiments on two standard graph node classification datasets: Pubmed [49] and Cora [60]. We adopt SPD-GAY [75] as the backbone and substitute its attention module with GyroAtt-SPD (denoted
and MAtt (SPD-GAT-MAII)
We also include H-GAT [75] for comparison. As shown in Tab. 17, GyroAtt- consistently outperforms all baselines on both datasets.
Table 18: Accuracy (%) Radar dataset for drone recognition.
Method
Radar
MAtt
96.8 1 1.1
GDLNet
94.7 1 0.9
SyP
98.5 = 0.8
GyroAtt-Gr
+ 1.2
GyroAtt-SPSD
96.9 + 0.9
Drone recognition. We evaluate GyroAtt on the Radar dataset [11], following the GDLNet [72] ar- chitecture. The dataset comprises 3,000 synthetic radar signals, evenly distributed across three classes. Tab. 18 shows that GyroAtt-SPP attains the highest accuracy, further validating the adaptability of method across different modalities.
F.5
Additional ablations studies
E.5.1 Ablations on the Riemannian metrics and matrix power-based nonlinear activation
Tab. 19 illustrates the impact of the different metries and power parameter p (as defined in Tab. 3) on the performance of GtroAtt based on two Riemannian matrix manifolds. The candidate values of metrics are AIM, LEM, and LCM, with p values to (0.25, 0.50, 0.75). As shown this table, for SPD-based architectures, GyroAtt under the SPD-AIM geometry with 0.5 achieves the highest accuracy on both the BNC120140M and BNCI2015001 datasets, while the SPD-LCM geometry with 0.75 records the second-hige higgest inter session accuracy (86.0%) on the BNCI201500 5001 dataset. For SPSD-based settings, GyroAtt under the SPSD-LCM geometry with p = 0.75 reaches the highest accuracy (68.7%) on the HAMEM-SSVED generally robust to variations importance of selecting the metric space of the underlying feature manifold and demonstrate that the proposed matrix power activation enhances model performance by introducing nonlinearity into the metric space.
dataset. Furthermore, it is evident that GyroAtt is across all experimental scenarios. These findings emphasize
32

---

Table 19: Ablations of GyroAtt on Riemannian metries and matrix power activation p. The best result under each geometry is highlighted in bold.
BNCI2014001
BNC12015001
Geometry
p
Inter-session
Inter-subject
Inter-session
Inter-subject
MAMEM-SSVEP-II
w/o
74.8 + 6.7
51.2 + 15.7
85.5 + 5.0
75.4 + 12.9
64.3 2.4
0.255
75.2 + 6.9
51.4 + 14.3
85.8 + 6.8
77.1 + 12.8
61.9 61.9+ 2.5
SPD-AIM
0.50
75.4 + 7.1
53.1 + 14.8
86.2 + 4.5
77.9 + 13.0
66.1 2.6
0.75
8.1
51.0 13.8
85.9 = 6.6
77.4 + 12.6
66.3 + 2.2
w/o
74.9
7 7.3
51.7 15.8
85.2 1 5.2
75.3 + 12.3
65.6 2.3
0.25
74.7 6.7
$2.3
14.1
85.6 6.7
75.6 = 13.0
63.7 + 2.5
SPD-LEM
0.50
75.3 6.5
51.4 14.1
85.7
76.6
2.7
0.75
75.1 7.3
52.3 15.0
85.4 7.0
75.5
12.8
66.2 2.5
w/o
73.2 6.7
$1.9 14.8
8.2
76.2 + 13.3
64.0 1:00::0 2.8
0.25
73.4 7.5
$2.4 13.4
85.6
75.4
14.0
2.5
SPD-LCM
0.50
74.0
8.2
52.7 + 13.6
85.9 6.7
77.4 + 13.2
64.1 3.2
0.75
74.2 + 7.8
51.7 + 14.6
86.0 16.0-00 6.8
76.3 = 13.2
65.1 = 2.5
w/o
72.2 7.2
49.2 13.7
84.1
73.6
2.6
0.255
72.4
6.8
50.7 + 14.8
84.0 6.8
75.5
13.8
66.4
SPSD-AIM
0,50
72.9
+ 7.1
52.4 + 15.6
74.2 + 14.2
66.5 + 2.9
0.75
72.5
+ 6.7
51.0 15.3
84.0 4.9
74.5
13.6
65.7 2
w/o
72.1 6.7
49.8 12.9
83.9 5.1
74.2 + 14.4
66.2 + 1.9
0.25
72.8 + 6,9
49.9 14.0
85.
76.0 = 14.1
66.5 2.3
SPSD-LEM
0.50
7 6.6
50.5 14.2
84.5
75.4 + 14.5
66.4 2.5
0.75
72.7
7.6
50.5 13.2
85.2 4.8
75.0
14.2
1.7
w/o
72.3 7.3
49.5 12.0
84.8 - 6.1
75.4 + 13.2
66.5 + 2.4
0.25
72.2
7.5
50.6
13,9
85.1 4.8
74,9
2.3
SPSD-LCM
0.50
72.9
6.7
48.4 13.3
84.9 = 6.1
74.5
13.6
66.2 3.6
0.75
72.8 + 6.3
51.7 + 13.1
85.1 = 5.8
73.9
= 15.4
68.7 1.5
Table 20: Ablations of GyroAtt on matrix power normalization 9 used in classification and Rieman- nian metrics. The best result under each geometry highlighted in bold.
BNC12014001
BNC12015001
Geometry
MAMEM-SSVEP-II
Inter-session
Inter-subject
Inter-session
Inter-subject
0,25
74.9 6,9
51.2 + 13.6
86.1 7,3
76.2 12.8
61.9 + 2.5
SPD-AIM
0,50
75.4
53.1 14.8
86.2 4.5
77.9 13.0
66.2 2.8
0.75
75.0 #.1 8.1
51.7 4: 14.5
86.0 6.5
77.1 14.3
66.3 1: 2.2
75.2 - 6.7
$2.7
5.7
76.9
60.7 2.4
SPD-LEM
0.50
75.3
6.5
51.4 14.1
85.7 5.5
76.6 13.7
66.1 2.8
0.75
75.1 7.3
52.3 - 13.3
85.8 6.3
76.4 + 13.1
66.2 + 2.5
0,25
74.2
52.1
5.9
13.4
64.5 2,9
SPD-LCM
0.50
8.2
52.7 + 13.6
85,9 6.7
77.4 +: 13.2
64.3 2.8 65.1 + 2.5
0.75
74.1 + 7.8
52.0 + 14.7
86.0 5.3
75.8 + 13.8
0.25
72.7
7.0
51.2 + 15.8
84.0 6.8
75.5
13.8
66.3 + 2,9
SPSD-AIM
0.50
72.9
+ 6.2
52.4 + 15.6
84.5 + 6.6
74.2 + 15.2
66.3 + 2.4 65.7 + 2.7
0.75
72.7
6.7
50.0 + 15.2
84.4 4.9
75.3 13.5
0.25
72.8
7.1
50.7 + 13.9
85.3 5.3
76.0
14.1
66.6 = 2.6
SPSD-LEM
0.50
72.5
+ 6.6
50.6 + 14.2
84.5
5.8
75.1 + 12.9
66.5 + 1.9
0.75
72.7
49.5 + 12.9
84.3 4.8
74.7 14.3
66.2 + 1.7
0.25
72.1
7.4
49.9 + 13.1
85.1 +: 4.8
74.9 + 12.6
67.6 + 2.1
SPSD-LCM
0,50
72.9
+ 6.7
48.4 + 13.3
84.1 5.6
74.4
13.7
68.1 + 1.6
71.6
50.1
84.1 5.7
75.0
12.9
68.7
E.5.2 Ablations on the matrix power normalization
We conduct ablation experiments to assess the impact of the power normalization parameter 0 on the performance of the proposed GyroAtt, as summarized Tab. 20. For each gyro structure, we let the parameter 0 vary within the set (0.25, 0.50, 0.75). Among the SPD-based configurations, our Gy- roAttNt under SPD-AIM geometry achieves the highest inter-session accuracy on BNC12014001 dataset and the best inter-subject accuracy the BNC1201500 5001 dataset at p 0.5. For the SPSD- based settings, SPSD-LEM geometry consistently performs well across multiple metrics, especially for the inter-session scenario in BNCI201500L, where it achieves top accuracy of 85.3%. It also
33

---

can be noted that smaller or larger values of (e.g.. 0.25 or 0.75) tend to yield lower accuracy in most cases. In contrast, moderate value of p 0.5 appears be more suitable for both SPD and SPSD geometries, it could maintain good normalization power. Besides, GyroAtt tends to be less sensitive changes across all experimental scenarios. In short, these results confirm the effectiveness of the introduced matrix power normalization in classification.
E.5.3 Ablation on the number of GyroAtt blocks
We perform an ablation study to assess the impact of stacking multiple GyroAtt blocks. Tab. 21 reports results on BNCI2014001 and BNC12015001 under inter-session settings, with the number of GyroAtt blocks varying from
For both SPD and SPSD variants, using two blocks slightly improves performance over one, but adding third block provides further gain and even causes mild degradation. These results suggest that a single GyroAtt block is sufficient to model EEG signals effectively, likely due to the low dimensionality and limited temporal complexity of the data. Extended results are included in the supplementary material.
Table 21: Ablation on the number of GyroAtt blocks. BNCI2015001 under inter-session settings.
Performance (%) on BNCI2014001 and
Method
Number of Blocks
BNC12014001
BNC12015001
GyroAtt-SPD
75.4 + 7.1
86.2 6.5
GyroAt-SPP
2
76,5 + 6.7
86.1 + 6.2
GyroAtt-SPD
3
76.0 + 6.3
86.0 6.1
GySDAu-SPSD
1
72.9 1 6.7
84.7 6.6
GyroAu-SPSD
73.1 6.2
84.9 5.2
GyrDAtt-SPSD
3
72.9 5.1
84.8 5.3
E.6 Implementation details of gyro inner product.
In Euclidean space, attention mechanisms commonly use the inner product similarity measure. Nguyen and Yang [52], Nguyen al. [53] extends this concept by defining the inner product on SPD, SPSD, and Grassmannian manifolds. The specific formulations are detailed as follows:
For P, Q St+ the SPD inner product is given by [52]:
(P.Q Q) = (Logf, (P), Logi, (Q)
(27)
For U, V Ö(g, d), the inner product given by:
(U. V) = (Logiz. or
(U), Logia, (V))...
(28)
For (UP.Sp) Sq) E G(g, d) x S the inner product is defined as:
((Up,Sp). (Uq, Sq) ped. XUUUU, U + (Sp,
(29)
We replaced the distance-basee based similarity computation in Eq. (5) with the inner product defined in follow and conducted ablation experiments on the MAMEM, BNC12014001, and BNC12015001 datasets under inter-session settings.
E.7 Implementation details of replacing geometric components in GyroAtt
conducted an ablation study to evaluate contributions of the Gyro Homomorphism and nonlinear activation in GyroAtt. Specifically, we replaced these components in GyroAttSPP and GyPDAI-SPPD with equivalent layers from SPDNet and GrNet, such as Bimap, Frmap, and ReFig, to assess their impact performance.
34

---

Implementation of component replacement on GyroAtt. We replaced components in GyroAtt with their equivalents from MAtt and GDLNet to assess their contributions. Specifically, in GyroAtt- SPD, we replaced the Gyro Homomorphism hom(.) with the Bimap layer and the matrix power-based nonlinear activation (()) with the ReFig layer. In GyroAu--PPDD, we replaced hom(.) with the Frmap layer and a()) with the ReEig layer. That is, we substituted homgr (Up) in hompsd.g (.) with the Frmap layer and replaced (SR,) in (UR, (SR,)P) with the ReEig layer.
The BiMap (bilinear transformation) layer is defined as:
X (1) w x-' W'"""
(30)
where (() E St ( Sh w gdddd with d1 d2 is a semi-orthogonal matrix. For the parameter W we use the geoopt [43] package to optimize. The FrMap layer is defined as:
X ( = w x(=1)
(31)
where x E G(22, v). ((1-1) E G(d1. v), and w Rdzxd is semi-orthogonal matrix with d1 d2. We optimized w using Geoopt.
The ReEig (rectified eigenvalues activation) layer is defined as:
X = max(m) eLaa U
(32)
with X = ( D0 where ( contains the eigenvalues of X and eLa is used to ensure numerical stability and set by le-4. Here, set the dimensions of the Bimap layer 21 18, 43 x 20, and 44 x 20 and the frmap layer to 21 x 18, 43 x 30, and 44 x 30 for the MAMEM- SIVEP-II, BNCI2014001, and BNCI2015001 datasets, respectively.
As shown in Tab. and Tab. 22, replacing hom(-) with the Bimap layer or o()) with the ReEig layer leads to significant performance degradation across the datasets. Similarly, for GSPDA,- replacing hom(-) with Frmap or (-)) with ReEig degrades performance. This occurs because hom and (-)) respect the gyro algebraic structure and underlying Riemannian geometry. The hom() function, as a Gyro homomorphism, preserves the Gyro algebraic structure of t and &, serving as a natural generalization of linear transformations in Euclidean spaces. In contrast, Bimap lacks these properties. Similarly, (.) introduces nonlinearity to SPD matrices and, more importantly, acts as an activation and deformation mechanism for the Riemannian metric, as discussed in Chen et al. [17]. On the other hand, to some extent, ReEig is primarily a numerical activation method, ensuring only St St + without addressing these deeper structural and geometric considerations.
Table 22: Ablations of GyroAtt- SPSSt Replacing Gyro Homomorphisms and Power Activations with SPDNet or GrNet methods (The Frmap and ReEig layers).
BNC12014001
BNC12015001
Transformation
Activation
Inter-session
Inter-subject
Inter-session
Inter-subject
Frmap Homomorphisms
Power
68.9 + 6,9
51.2 + 12.9
82.3 + 6.2
65.8 + 13.1
ReEig
72.3 = 6.9
49.6 = 13.3
84.9 = 6.2
74.1 = 12.3
Frmap
ReEig
68.8 7.2
50.7 13.8
81.6 6.1
72.9 13.3
Homomorphisms
Power
72.9 + 7.1
52.4 15.6
85.3 5.3
76.0 14.1
E.8
Computational complexity analysis and comparison
The computational complexity of GyroAtt, MAtt, and GDLNet primarily depends on operations such as WFM, gyro homomorphisms, and similarity measurements. The primary computational overhead arises from the types and quantities of matrix functions used in manifold-valued computations.
E.8.1 Number of matrix functions in manifold attention mechanisms
To analyze computational complexity, we examine the matrix functions employed in the attention mechanisms of MAtt, GDLNet, and our GyroAtt framework. These methods primarily use Singular Value Decomposition (SVD) and Cholesky decomposition, each with a computational complexity of (ad)) for matrices of dimension d. GyroAut-Gr Gr and GyroAIt-SPSD, SVD is applied to matrices sized d x g, resulting in complexity of Od42). Tab. 23 outlines the number of required matrix functions for each method, where is the number of inputs to the attention module (e.g., queries or keys), and g denotes the subspace dimension.
35

---

Table 23: Complexity comparison of different manifold attention models, where Chol denotes the
Cholesky decomposition.
SVD (d x Old")
Chol
SVD (d x
SVD (g ① O(g")
Chol (g x q) O
Methods
Metries
O(d))
Oldq²)
MAI
LEM
3C
0
GDI.Net
ONB
0
0
GyroAtt-Gr
ONB
0
Ca +
2C'
GyP
AIM
202 3C
0
0
0
GyroAtt-PD
LEM
GyroAtt-SPD
LCM
3C
0
GyroAtt-SPDD
AIM
0
C²
30 4C
GyPSAt-SPSD
LEM
C
C 6C 6C
GyroAm-SPSD
LCM
C²+C
C + 2C
3C
Table 24: Training time (seconds/epoch) comparison of different manifold attention networks on the BNCI201400 and BNC1201500 datasets.
BNC12014001 Inter-session
BNC12014001 Inter-subject
BNC12015001 Inter-session
BNC12015001 Inter-subject
Methods
Metrics
MAtt
LEM
4.86
89.12
2.74
56.78
GDLNet
ONB
4.55
88.59
1.71
47.66
GyroAtttGrr
ONB
5.28
100.20
1.98
54.07
GyroAtt-SPD
AIM
10.36
149.14
4.44
81.73
GyroAtt-SPD
LEM
7.89
138.59
4.28
80.66
GyroAu-SPD
LCM
4.11
87.44
2.42
49.86
GyrDAu-SPSD
AIM
6.78
123.44
2.37
65.30
GyroAu-SPSID
LEM
6.54
116.49
2.34
64.23
GyroAu-SPSD
LCM
5.71
103.57
2.16
58.50
E.8.2 Experimental comparison of time efficiency
For practical comparison, we measured the average training time per epoch (seconds) for our GyroAtt variants and baseline models MAtt and GDLNet the BNC12014001 and BNC12015001 datasets, as shown in Tab. 24. The results indicate that GyroAut-Gr runs slower than GDLNet, mainly because of more SVD operations in WFM, increasing computational complexity. As predicted by Tab. 23 and confirmed in Tab. 24, GyroAU-SPPDD is less computationally efficient than GDLNet and MAtt across all metries. In contrast, GyroAtt-SPD with the LCM metric slightly outperforms MAu in runtime. Additionally, Tab. 24 shows that any GyroAtt variant using the LCM metric runs faster than those using AIM and LEM metrics, primarily because the Cholesky decomposition (used in LCM) is generally more efficient than SVD due to smaller constant factors. Notably, AlM-based GyroAtt variants are slower than their LEM-based counterparts because AIM requires more eigenvalue computations than LEM.
Our computational complexity analysis and empirical runtime comparisons highlight key observa- tions:
GyroAtt-Gr: Runs slower than GDLNet due to more SVD operations.
GyroAtt-SPSD: Less efficient than GDLNet and MAtt across all metrics, involving extra SVD computations on d x q and
matrices.
GyroAtt-SPD with LCM: Achieves comparable or better runtime than MAtt, benefiting from the efficiency of the Cholesky decomposition.
Metric Impact: The LCM metric leads to lower training times due to the efficiency of SVD. AIM-based GyroAtt variants are slower than LEM- based ones because AIM requires more eigenvalue operations.
Cholesky decomposition
Although some GyroAtt variants introduce additional computational overhead compared to baseline methods, especially under certain metrics, the performance improvements justify the increased
36

---

complexity. Selecting efficient metrics like LCM can mitigate computational costs, making GyroAtt practical for real-world applications.
E.9
Model compactness and parameter efficiency
Table 25: Learnable parameters (in millions) of different methods across four EEG datasets.
Method
MAMEM-SSVEP-II BCI-ERN
BNC12014001
BNC12015001
MAtt
0.07M
0.03M
0.03M
0.01M
GDLNet
0.5IM
0.05M
0,03M
0.01M
SPD
0.11M
0.08M
0.04M
0.01M
GyroAtt-Gr
0.11M
0.07M
0.04M
0.01M
GyroAtt-SPSD
0.12M
0.09M
0.05M
0.01M
As shown in Tab. 25, all GyroAtt variants maintain comparable parameter count to MAtt, and are substantially more compact than GDLNet on MAMEM-SSVEP-II. Despite this, GyroAtt achieves stronger performance across all datasets.
E.10 Long-range dependencies
A central advantage of self-attention is its ability to capture long-range dependencies via position- independent pairwise interactions, where each token attends to all others in single step.
Our GyroAtt module inherits this property by extending self-attention to Riemannian manifolds. manifold-valued features using geodesic distances,
Specifically, it computes pairwise attention followed by aggregation via the weighted Fréchet mean. This allows any two positions to interact directly, independent of sequence length.
E.11 Visualizing learned representations via Rie-SNE
Train
Test
3325
6.002
0.00
-2523
-3.5
-5.047
2:30
Figure Ric-SNE visualization of SPD representations extracted by SyrroAt-SPD-LEN) on BNC12015001. We plot the manifold-valued features from the training (left) and test (right) sets. Each point corresponds to SPD matrices used for final classification, with colors denoting different classes. Clear separation between classes and structural consistency across splits highlight the model's capacity to learn discriminative geometric features.
We visualize the SPD matrices output from GyroAtt-SDD-LEM prior to classification using Rie- SNE [21], separately for the training and test sets on BNC12015001. shown in Fig. 5, the learned representations exhibit compact intra-class clustering and clear inter-class separation across both splits.
Proofs of the Theorems the Main Paper
F.1 Proof of the Thm. 3.2
Proof of Thm. 3.2 This theorem can be induced from Nguyen and Yang [52][Lem. 2.1-2.2]. Let the Riemannian metric tensor be g. We have
f (M, (M,9), with f(e) e,
37

---

where f is the pullback metric under the diffeomorphism f. As f is an isometry, we have f"g = 9. Applying Lem. 2.1-2.2 in Nguyen and Yang [52], one can immediately prove f(P $ Q) S(P) @ ((Q) and [(t @ Q) = 1 80 f(Q) for any & E IR and P. Q E M.
F.2 Proof of the Thm. 4.1
Proof of Thm. 4.1 For each g E {ai, le, lc), let (s ,rg 8g) be the gyrovector space specified in Tab. 1. We must show that the map
OPOT,
= ai, O E O(d),
homg / S P PH
expm (Mlogm(P)M))
9 le, M E Raxd, (33)
L --1 (-(() exp (T (L(P))))
lc, M E RRudd
with
T(P)=M ( [(P) [P]] T + ID ( M
(34)
a gyro homomorphism, i.e.
homg (P) Gg homg hom, (P 0g Q),
(35)
2g hom, (P) hom, (t 8g P) v PPQQ S >
(36)
AIM case (g ai).
The (Dat and Øoi are defined by:
P Gai Q = PPPPPQPP
(37)
t 8ai P
(38)
Let the isometry hom OPOT with O(d) (orthogonal group). Since the identity
element is I and homaa(e) 010T I= e, the sufficient condition in Theorem 3.2 applies:
an isometry fixing is gyro homomorphism.
For completeness, verify both operations explicitly.
We begin by showing that homai (-) satisfies Eq. (35). let homa: (P) OPOT, with any P,Q St+ then we have
homag (P) Oai hom.. (Q) ! (OPOT)* oqo (OPOT)*
OPto T
(39)
O
homgi P Caa Q)
The derivation of Eq. (39) follows.
(1) follow from Eqs. (8) and (37).
(2) follows from the fact that is an SPD matrix and O is orthogonal matrix.
Now, we proof that hom (-) satisfies Eq. (36). For the Sai we have
t Eas hom.; (P)
(OPOT)*
(40)
OP'O
-homaett @ai P).
The derivation of Eq. (40) follows.
(I) follow from Eqs. (8) and (38).
(2) follows from the fact that is an SPD matrix and O an orthogonal matrix.
LEM case (g le).
38

---

The Die and E/e are defined by:
P Dte Q expm(logm(P) + (Q))
(41)
t Bte P = P
(42)
We begin by showing that homme (-) satisfies Eq. (35). For the Bic. with any P, Q E S we have
(1) expm logg (P)M + Mogin (Q) MT)
home (P) Dt home (Q)
expm M (logm (P) + logm (Q))
(43)
home P Bte Q).
The derivation of Eq. (43) follows. (I) follow from Eqs. (8) and (41).
For Ste, we have
Ste homee (P) ( (expm (M (P) M ))
( expm (tM (P) M
(44)
homm, (1 01e P).
Reduction to AIM for orthogonal M =0 O(d).
For the Ble, with any P, Q S O E O(d) we have
home (P) Bie home,
(Q) expm (0 (logm (P) + logm OQ
(45)
O expm (llogm (P) + logm (Q)))
home (P @ox Q).
The derivation of Eq. (45) follows.
(I) follow from Eqs. (41) and (43).
(2) follows from the fact that is an SPD matrix and O an orthogonal matrix.
For the Ste, we have
(1)
Ste homee()) (P)
expm (10 logm (P)
(2) O expm (t logm (P)) O
(46)
home (t &te P).
The derivation of Eq. (46) follows.
(I) follow from Eqs. (42) and (44).
(2) follows from the fact that P is an SPD matrix and O is an orthogonal matrix.
LCM case (y lc).
The (tt and &te defined by:
Ote p= (1
(47)
P Q- I-1 (1
+ [2(Q]
(48)
We begin by showing that homie(.) satisfies Eq. (35). With any P. Q E S for Dlc we can rewrite
Bie and home as
P P Q = ---1 (exp D Dlog (L + log D (2)))))),
(49)
hom PP) L- (exp D (L(P)))
(50)
where L(P)- = P T M by
log DD (F) and exp D (F) are given
log (F) = F logmn
(51)
exp DD (F) F + expm DV F)),
(52)
39

---

Then we have
hom Ote hom (Q)
(1)
L-1 lexp DD (L(P) + (())))
2
L-1 (exp D (L(P + Q)))
(53)
homec (P
Ptcc Q)
The derivation of Eq. (53) follows.
(I) follow from Eqs. (8) and (48).
(2) follow from the properties of L(-).
Hence homg is gyro homomorphism for each g E {ai, le, Ic), completing the proof.
F.3
Proof of the Thm. 4.2
Proof of Thm. 4.2 The @ oor and 0gr are defined by:
U@g V expm (Logf, (UUT), I,
(54)
10 U expm [[Togl". Lag] ld.
(55)
Let the isometry homgr (U) = OU with O E O(d). The reference element Ia., denotes the canonical subspace spanned by the first basis vectors. Since Old, and Ld,q represent the same subspace on G(g, d), homgr fixes the identity element the quotient sense and satisfies the condition of Thm. 3.2.
For completeness, verify that it preserves both gyroaddition and gyromultiplication below.
we begin by showing that homgr (.) satisfies Eq. (35), For any U, V E G(g, d), we have
homgr (U) homgr (V)
expm([Log]] (OUUTO 0 l
expm Log 9r
(UU")OT O OI ,I, 00 OJoo
expm Oogg
(UU L jov
(56)
expm (Logg
(UU ) I I oTov
expm (Logi.. T (UU glVV
hom gr (U@
The derivation of Eq. (56) follows.
(1) follow from Eqs. (9) and (54).
(2) follows from the fact that Log (OUUTOT) O Log (UUT)OT, and for O
T 0 O = In.
Od-g
(3) follows from the fact that O is an orthogonal matrix.
Now, we proof that homgr(-) satisfies Eq. (36). The differential homomorphism & Ögg d) g(g. d), U + UUT exists between Č(g, d) and G(g. d), and &gr is derived from 8gr via this differential homomorphism. Thus, to prove that øgr satisfies Eq. (36), it suffices to show that &gg satisfies Eq. (36). The &gg is defined by:
t Bggr U expm ([oú,La,o) Ia,q expm ((-iU,
(57)
40

---

For & Err, we have
t &gr homge
- (U)) homgr(U))
@ expm(t[Logi, (OUU O
(OUU""
OT), Im.el)
exppm( [Logi (UU") II w)) "io'In.. In.q expommLLL (UU') In
expm (o.g.... or in..
(UU ), In. expm (l.of (UU) In.a)
homgr(t (t Øgr UU
(58)
Since 0g satisfies Eq. (36), we can proof 0rr satisfies Eq. (36).
F44
Proof of the Thm. 4.3
Proof of Thm. 4.3
The @psd,g and Bpsd. are defined by:
(Up, Sp) @psd.y (Uq, SQ) = (Up@gr Uq Sp By SQ).
(59)
Opsd. (Ur (10gr Ur, 0g Sp)
(60)
begin by showing that hompsd.g satisfies Eq. (35). As shown in Eq. (10) For any (Up,Sp), (Uq; SQ) E G(q. d) x St+ we have:
hom (UP,Sp) Bpsd. (Ug, So)) pad.o (U a U2. Sp Bg SQ)
(homgr (U @Up Uo), hom, (Sp tg Sa))
(homgr (Ur) homgr (U2), hom, (Sp) 3g hSm)(SQ))
(homar (Up). homg (Sp)) @pdd. (homgr (Uq). homg (Sq))
homped, ( Sr) &ped, homped.g (UQ, SQ).
(61)
The derivation of Eq. (61) follows.
(I) follow from Eqs. (10) and (59).
(2) and (3) follow from the fact that homgr and homy are gyro homomorphisms.
For scalar multiplication, we have:
hompsd. (t @psd. (Up,Sp)) (1) hom (18 Up, 8g Sp)
(hom (t U Up). homy (t 0g Sp))
(tāgr hom (Ur), t 8g hom, (Sp))
(62)
&ped, (hom (UP), hom, (Sp)
ppdd.g hompad. (Up, Sp).
The derivation of Eq. (62) follows.
(1) follow from Eqs. (10) and (60).
(2) and (3) follow from the fact that homgr (.) and hom,(-) are gyro homomorphisms.
41


Reviews:
[{"review_id": "D8TDxoWlSm", "reviewer": "Reviewer_5ZCS", "summary": "This paper proposes a new framework for computing transformer-like attention on general gyrovector spaces for data with underlying matrix geometry.  Their Gyro Attention (GyroAtt) framework is applied in three geometric settings, namely on the SPD manifold, on the SPSD manifold, and on the Grassmannian manifold. The real-world application under consideration is EEG signals classification using four separate datasets. Comparisons are performed against existing deep learning architectures designed for matrix geometry.", "questions": "1. Since the datasets are on the small size,  are all the comparisons reported statistically significant in terms of improvements offered?\n2. What is the wall clock time for optimising each attentional framework\n3. How many model parameters do each considered attentional spaces have; how about the baselines being compared against?\n4. The authors should consider including a glossary for abbreviations as it is currently difficult to go back and forth within the manuscript each time one encounters an abbreviation.\n5. In section C3., why do the authors not choose to run a common principal components algorithm for initialisation instead of disjoint decompositions?", "limitations": "Currently, within the datasets under consideration, the matrix sizes are <50x50. In the context of the questions on computation time and time to convergence, it is not clear whether Gyro-Attn would scale to larger matrices (such as those from fMRI brain-connectomes) for a broader range of applications", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "STRENGTHS:\n\n1. The Gyro-Attentional framework is an interesting and theoretically sound geometric extension of attention onto matrix geometries and manifold valued data, such as covariance matrices from EEG (or fMRI). \n2. The approach is principled and well motivated. The paper explains the relevant theory clearly with sufficient explanation and offers a unified framework applicable under different geometric assumptions on the manifold structure.\n3. Improved performance is observed for the EEG datasets for signal classification as compared against existing deep neural networks for matrix manifold data.\n\nWEAKNESSES:\n\n1. The authors gloss over how computationally expensive the optimisation is for their method, especially compared to the alternatives proposed. For example, the initialisation requires them to run multiple SVD like decompositions, running Karcher flow at every epoch incurs additional cost. \n\n2. There are very few details available on the baseline methods chosen, which makes it hard to evaluate their performance gains hard to contextualise.  Do the authors compare against graph neural networks (an alternate strategy for classification, but with potential for saving on computation with sparsity).\n\n3. How many model parameters do each have relative to the proposed framework? It is unclear whether this can be a contributing factor for the observed performance improvements.", "was_revised": true, "final_justification": "I have read through the comments and appreciate the work that the authors have put in to address the concerns brought up during the review process. I have increased my final score to an accept.", "rebuttal": "We thank Reviewer $\\textcolor{blue}{\\rm{5ZCS (R1)}}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😊\n\n## **1. Statistical Significance of Reported Improvements.**\nTable A: Comparison between GyroAtt and the baseline MAtt, where * donetes $p$-vale < 0.05 and n.s. denotes \"not significant\".\n||MAMEM|BCI-CHA|BNCI2014001 inter-session|BNCI2014001 inter-subject|BNCI2015001 inter-session|BNCI2015001 inter-subject|\n|-|-|-|-|-|-|-|\nGains|3.2%|3.4%|7.8%|8.9%|5.4%|14.8%|\nStatistical significance|*|*|*|n.s.|n.s.|*|\n\nWhile some confidence intervals overlap, the improvements are substantial, consistent across all benchmarks. Particularly in the inter-subject setting, which is inherently more challenging, a +14.8% gain highlights the practical effectiveness of GyroAtt. We will clarify the statistical testing procedure in the final version.\n\n\n## **2. Experimental comparison of time efficiency.**\nTable B: Training time (s/epoch) comparison of different manifold attention networks on the BNCI2014001 and BNCI2015001 datasets.\n|Methods|Metrics|BNCI2014001 inter-session|BNCI2014001 inter-subject|BNCI2015001 inter-session|BNCI2015001 inter-subject|\n|-|-|-|-|-|-|\n|MAtt|LEM|4.86|89.12|2.74|56.78|\n|GDLNet|ONB|4.55|88.59|1.71|47.66|\n|GyroAtt-Gr|ONB|5.28|100.20|1.98|54.07|\n|GyroAtt-SPD|AIM|10.36|149.14|4.44|81.73|\n|GyroAtt-SPD|LEM|7.89|138.59|4.28|80.66|\n|GyroAtt-SPD|LCM|4.11|87.44|2.42|49.86|\n|GyroAtt-SPSD|AIM|6.78|123.44|2.37|65.30|\n|GyroAtt-SPSD|LEM|6.54|116.49|2.34|64.23|\n|GyroAtt-SPSD|LCM|5.71|103.57|2.16|58.50|\n\nFor practical comparison, we measured the average training time per epoch (seconds) for our GyroAtt variants and baseline models MAtt and GDLNet on the BNCI2014001 and BNCI2015001 datasets. As shown in Table B, GyroAtt-Gr is slower than GDLNet, mainly due to additional SVD operations required by the WFM aggregator. Similarly, GyroAtt-SPSD is less efficient than both baselines across all metrics. In contrast, GyroAtt-SPD with the LCM metric achieves slightly better runtime than MAtt. In general, LCM-based GyroAtt variants are faster than their AIM- and LEM-based counterparts, since Cholesky decomposition (used in LCM) is more efficient than SVD-based operations. AIM is consistently slower than LEM due to the higher computational cost of eigenvalue computations. Although some GyroAtt variants introduce additional computational overhead compared to baseline methods, especially under certain metrics, the performance improvements justify the increased complexity. Selecting efficient metrics like LCM can mitigate computational costs, making GyroAtt practical for real-world applications. Please refer to Appendix D.7 for further implementation and profiling details.\n\n\n## **3. Analysis of Parameter Counts.**\nTable C: Learnable parameters (in millions) of different methods across four EEG datasets.\n|Method|MAMEM|BCI-CHA|BNCI2014001|BNCI2015001|\n|-|-|-|-|-|\n|MAtt|0.07M|0.03M|0.03M|0.01M|\n|GDLNet|0.51M|0.05M|0.03M|0.01M|\n|GyroAtt-SPD|0.11M|0.08M|0.04M|0.01M|\n|GyroAtt-Gr|0.11M|0.07M|0.04M|0.01M|\n|GyroAtt-SPSD|0.12M|0.09M|0.05M|0.01M|\n\nFor a manifold attention layer mapping from $S^d \\to S^d$, the number of parameters in GyroAtt-SPD is:\n$\\frac{3}{2}n(n-1) + \\frac{1}{2}n(n+1)=2n ^2-\\frac{1}{2}n$, where the first term corresponds to the transformation layers for $Q$, $K$, and $V$, and the second term represents the bias parameters. In comparison, the baseline **MAtt** uses $3n^2$ parameters per attention block.\n\nAs summarized in Table C, all GyroAtt variants have comparable or even fewer total learnable parameters than MAtt across four EEG datasets, and are significantly more compact than GDLNet, particularly on MAMEM-SSVEP-II. Importantly, GyroAtt consistently achieves better performance despite this compactness, indicating that the gains are not simply due to increased model capacity. \n\n\n\n## **4. The glossary for abbreviations.**\nWe have included a glossary of notations and abbreviations in Appendix B, specifically in Tables 11 and 12, to improve readability and reduce the need to reference earlier sections when encountering abbreviations.\n\n## **5. Clarification on Initialization Strategy in Section C.3**\n\nWe appreciate the reviewer’s suggestion regarding the use of a common principal components algorithm (CPCA) for initialization. We would like to clarify a potential misunderstanding:\n\nIn Line 2 of Algorithm 6, we initialize the common subspace $U^m$ with a fixed canonical basis $\\widetilde{I}_{n,q}$, not via decompositions of individual samples. This line is intended to indicate that the default initialization of the Grassmannian Fréchet mean starts at the identity point on the manifold. We acknowledge that this notation may have caused confusion and will revise it in the final version for clarity.\n\nImportantly, the estimation of the common subspace is learned dynamically in the subsequent steps. Specifically, we first compute the Grassmannian Fréchet mean of the individual subspaces $\\{ U_i \\}$ obtained via SVD of each sample, and then update $U^m$ via geodesic interpolation toward this mean, and the same choice is also adopted in [a].\n\n## **6. Comparison with SOTA Baselines.**\n\nOur main baselines are strong SOTA methods specifically designed for manifold-valued EEG data:\n\n* On the MAMEM-SSVEP-II and BCI-ERN datasets, our main baselines are GDLNet [b] and MAtt [c], both of which represent state-of-the-art approaches for SPD and Grassmannian manifolds, respectively.\n* On the BNCI2014001 and BNCI2015001 datasets, we include TSMNet [d] and Graph-CSPNet [e], both of which are recent SOTA models widely used in motor imagery tasks. Notably, Graph-CSPNet [e] is a recent GNN-based approach tailored for motor imagery classification.\n\nTo ensure a fair comparison, we also follow the exact same data preprocessing and evaluation protocol as used in the original baseline papers: For MAMEM-SSVEP-II and BCI-ERN datasets, our setting matches that of MAtt [c]; For BNCI2014001 and BNCI2015001 datasets, we follow the exact pipeline from TSMNet [d].\n\n\n## **7. Comparison with Graph Neural Networks.**\n\n### **7.1 Comparison with Graph-Based EEG Methods.**\nTable D: Accuracy (\\%) comparison between Graph-CSPNet and GyroAtt on EEG datasets.\n|Method|BNCI2014001 inter-session|BNCI2014001 inter-subject|BNCI2015001 inter-session|BNCI2015001 inter-subject|\n|-|-|-|-|-|\n|Graph-CSPNet|71.9 ± 13.3|45.2 ± 9.3|79.8 ± 14.6|64.2 ± 13.4|\n|GyroAtt-SPD-AIM|**75.4 ± 7.1**|**53.1 ± 14.8**|**86.2 ± 4.5**|**77.9 ± 13.0**|\n|GyroAtt-SPD-LEM|75.3 ± 6.5|52.3 ± 14.1|85.7 ± 5.5|76.6 ± 13.7|\n|GyroAtt-SPD-LCM|74.2 ± 7.8|52.4 ± 15.6|84.7 ± 6.6|75.5 ± 13.8|\n\nWe compare GyroAtt against Graph-CSPNet [e], a recent SPD-based GNN method designed for motor imagery classification. As shown in the table below, all variants of GyroAtt outperform Graph-CSPNet across both inter-session and inter-subject settings on BNCI2014001 and BNCI2015001.\n\n### **7.2. Additional Comparison on Graph Attention Neural Networks.**\n\nTable E: Accuracy (\\%) comparison of SPD-GAT-GyroAtt under graph node classification.\nMethod|Pubmed|Cora|\n|-|-|-|\nH-GAT|77.5±1.6|78.1±1.1|\nSPD-GAT|77.8±0.6|79.4±0.6|\nSPD-GAT-MAtt|77.5±0.6|80.1±1.1|\nSPD-GAT-GyroAtt|**78.5±0.4**|**81.6±1.0**|\n\n\nTo further validate GyroAtt, we compare GyroAtt against hyerpbolic and SPD attention in standard GNN settings using graph node classification tasks. \n\nWe focus on the graph attention (GAT) setting. H-GAT [f] serves as a hyperbolic graph attention baseline, while SPD-GAT [g] is the SPD counterpart. By replacing SPD-GAT’s attention layer with MAtt and GyroAtt, we demonstrate that our attention module consistently improves performance. This highlights the versatility of GyroAtt across both manifold-valued and graph-structured domains.\n\n\n## **8. Computation Time and Convergence on Large Matrices.**\n\nTable F: Scalability of GyroAtt-SPD on BNCI2015001 with varying input matrix sizes.\n|Matrix Size|Time per epoch (s)|Accuracy|\n|-|-|-|\n|43×43|**0.98**|**85.7±5.5**|\n|50×50|1.13|85.21±5.8|\n|60×60|1.37|85.01±6.1|\n|70×70|1.86|84.91±5.8|\n|80×80|2.34|84.46±5.1|\n|90×90|3.82|84.08±5.6|\n|100×100|5.10|84.35±5.9|\n\nWe further conducted scalability experiments using GyroAtt-SPD on the BNCI2015001 dataset by adjusting the output dimensionality of the feature extraction layer, thereby increasing the size of the input SPD matrices from $43 \\times 43$ to $100 \\times 100$. As shown in Table F, the per-epoch time increases moderately with matrix size, and convergence is consistently achieved within a practical timeframe. These results confirm that GyroAtt remains computationally feasible even for relatively large matrices, in agreement with theoretical expectations.\n\nIn terms of accuracy, we observe that increasing the matrix size from 43 to 100 results in only marginal variations in performance. This suggests a saturation effect often seen in low-dimensional EEG tasks, where expanding the representational capacity beyond a certain point offers little additional discriminative power. The relevant signal information is likely already captured at smaller sizes, which accounts for the stable test accuracy across the entire range.\n\n\n**References**\n\n> [a] Matrix Manifold Neural Networks++.\n> \n> [b] A Grassmannian Manifold Self-Attention Network for Signal Classification.\n> \n> [c] MAtt: A manifold attention network for EEG decoding.\n> \n> [d] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG.\n> \n> [e] Graph neural networks on SPD manifolds for motor imagery classification: A perspective from the time-frequency analysis.\n> \n> [f] Hyperbolic graph attention network.\n> \n> [g] Modeling graphs beyond hyperbolic: graphneural networks in symmetric positive definite matrices."}, {"review_id": "MT51SRosPi", "reviewer": "Reviewer_2oRe", "summary": "The paper proposes GyroAtt, a framework for generalizing attention in (Euclidean) deep neural networks to a class of matrix manifold spaces (including the SPD, SPSD and Grassmannian manifolds). Methods using the framework are empirically validated on 4 real EEG datasets.", "questions": "*Key points*:\n1. Are there other obvious applications of the GyroAtt framework outside of EEG-related data? Elaborating a bit on the \"various applications\" on Line 16 would help make the motivation and potential impact more clear.\n\n2. Line 1051 states plans to release code for accessing the data. Are there no plans to release implementations of GyroAtt?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Quality*:\n- (+) mathematically rigorous\n- (+) extensive empirical evaluation\n- (+) coherent and insightful framework for learning across different geometries\n\n*Clarity*:\n- (+) quite well-written and thorough\n\n*Significance*:\n- (+) potential for good impact within the field of BCI, and I imagine (though am less familiar with) many other applications\n- (+) also interesting more abstractly for the mathematical deep learning community\n- (-) seems to be no plans for sharing implementations of GyroAtt, which I consider a substantial weakness\n\n*Originality*:\n- (+) novel generalization/unification of existing approaches", "was_revised": true, "final_justification": "Both of the issues I originally raised (potential applications outside of EEG data, and opensource code) are addressed in the rebuttal. I think it's a mathematically solid paper that will have impact across a number of applications, (but I can't certify that it's a \"technically flawless paper with groundbreaking impact) so I maintain my rating of 5.", "rebuttal": "We thank Reviewer $\\textcolor{red}{2oRe (R2)}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😄\n\n## **1. GyroAtt is applicable to a wide range of manifold learning tasks beyond EEG.**\n\nThank you for the insightful question. While our experiments focus on EEG decoding, the GyroAtt framework is broadly applicable to learning tasks involving manifold-valued data. Data that lie on matrix manifolds—such as SPD, SPSD, or Grassmannian manifolds—are ubiquitous in various application domains, such as medical imaging [a], shape analysis [b], drone classification [c], image recognition [d], human behavior analysis [e, f], EEG [g], graph classification [h], and knowledge graph completion (KGC) [i, j].\n\nTo further support its generalizability, we include **two additional experiments** beyond EEG:\n\n**Graph Node Classification.** We evaluate GyroAtt-SPD on two benchmark datasets—**Pubmed** and **Cora**—using SPD-GAT [h] as the base model, where the standard attention module is replaced with either GyroAtt-SPD or MAtt. Results are summarized below:\n\n|**Method**|**Pubmed**|**Cora**|\n|-|-|-|\n|H-GAT|77.5±1.6|78.1±1.1|\n|SPD-GAT|77.8±0.6|79.4±0.6|\n|SPD-GAT-MAtt|77.5±0.6|80.1±1.1|\n|**SPD-GAT-GyroAtt**|**78.5±0.4**|**81.6±1.0**|\n\n**Drone Recognition.** We further test GyroAtt on the **Radar** dataset for drone classification, following the GDLNet [g] architecture. The results are as follows:\n\n|**Method**|**Accuracy(%)**|\n|-|-|\n|MAtt|96.8±1.1|\n|GDLNet|94.7±0.9|\n|**GyroAtt-SPD**|**98.5±0.8**|\n|GyroAtt-Gr|96.2±1.2|\n|GyroAtt-SPSD|96.9±0.9|\n\n\nGyroAtt-SPD again achieves the highest accuracy, further validating its adaptability to non-EEG modalities such as radar signals.\n\n\n## **2. We will release the full GyroAtt implementation**\n\nApologies for the ambiguity. We fully intend to release the complete GyroAtt implementation, including training pipelines and model definitions, alongside the data access code. The sentence in Line 1051 will be updated in the final version to reflect this more clearly. \n\n\n**References**\n\n> [a] Fast and Simple Computations on Tensors with Log-Euclidean Metrics.\n> \n> [b] Covariance Descriptors for 3D Shape Matching and Retrieval.\n> \n> [c] Riemannian Batch Normalization for SPD Neural Networks.\n> \n> [d] DeepKSPD: Learning Kernel-Matrix-Based SPD Representation for Fine-Grained Image Recognition.\n> \n> [e] Deep Manifold Learning of Symmetric Positive Definite Matrices with Application to Face Recognition.\n> \n> [f] GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition.\n> \n> [g] MAtt: A Manifold Attention Network for EEG Decoding.\n> \n> [h] Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices.\n> \n> [i] The Gyro-Structure of Some Matrix Manifolds.\n> \n> [j] Matrix Manifold Neural Networks++."}, {"review_id": "HgoTpR2T06", "reviewer": "Reviewer_UbxM", "summary": "This paper proposes GyroAttr, an attention framework that unifies operations across diverse matrix manifolds through a common gyrovector space formulation. GyroAttr is applied to three types of manifolds: the SPD manifold, the Grassmannian manifold, and the SPSD manifold. Experimental results show that the proposed method gets improvements in related tasks.", "questions": "1.\tSome case studies or visualizations would help clarify the limitations of previous work. I suggest the authors either include some experiments to illustrate the issue or explain whether this is a commonly recognized challenge in the field with appropriate citations.\n2.\tIs it possible to align the experimental setup with that of any previously published work? Alternatively, could the authors clarify why the evaluation metrics used in this paper are more convincing?", "limitations": "NA", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros\n1.\tThe experimental results show improvement\n2.\tThe network architecture is described in sufficient detail, which is helpful for reproducibility.\n3.\tThe effectiveness of GyroAttr on multiple types of manifolds shows the generality of the proposed framework\nCons\n1.\tThe motivation of this paper (lines 100–108) is not well supported. For example, the authors claim that \"these approaches typically rely on geometry-specific operations, such as the BiMap layer [32] for SPD manifolds or the FrMap layer [33] for Grassmannian manifolds, which hinders their applicability to other manifolds and limits generalization.\" Although Section 3.1 revisits the network structures of previous methods, it is important to note that deep learning models are often treated as a kind of black-box system. Therefore, differences in network structure are insufficient to conclude that previous methods exist significant limitations and the proposed method effectively addresses them. \n2.\tAlthough the reported experimental results suggest that the proposed method outperforms existing approaches, the inconsistency in evaluation metrics raises concerns, especially since the authors claim in line 212: “Building on prior works [54, 40], we evaluate on four benchmarking datasets.” However, upon reviewing the cited works, it appears that there are no inter-session or inter-subject evaluation settings in Matt. In the SPD setting, the evaluation metrics used in this paper also appear to differ significantly.", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer $\\textcolor{orange}{UbxM (R3)}$ for the constructive suggestions and insightful comments! In the following, we respond to the concerns in detail. 😄\n\n## **1. The limitations of previous work**\n\nIn Euclidean spaces, basic layers like convolution and fully connected operations provide a flexible basis for diverse network designs. However, in the context of manifold-valued data, there are no counterparts to these foundational layers due to geometry-specific constraints. This limits the flexibility of layer design and hinders reuse across different manifolds. Although deep learning models are often treated as black-box systems, it is essential in such settings to consider whether these layers (i) preserve the input’s geometric structure and (ii) effectively leverage the underlying Riemannian geometry.\n\n**Preserving geometric structure.** The commonly used BiMap layer is inherently restricted to SPD matrices of shape $d \\times d$ and operates as:\n$$\n\\mathrm{BiMap}(X) = W^\\top X W,\n$$\nwhere $X \\in  S^d$ and $W \\in \\mathbb{R}^{d \\times k}$. A fundamental requirement of this operation is that the input must be square. However, for data lying on the Grassmannian manifold, such as subspace features $Y \\in \\mathbb{R}^{d \\times q}$, this assumption no longer holds. Since $Y$ is rectangular and not square, BiMap becomes ill-defined and cannot be directly applied.\n\nConversely, the FrMap layer, used in Grassmannian settings, applies a transformation of the form:\n$$\n\\mathrm{FrMap}(Y) = W Y,\n$$\nwhich is compatible with Grassmannian data. Although SPD matrices $X \\in S^d$ can technically be input to this operation, it destroys the key structural properties: $WX$ is generally not symmetric and not positive definite, thus violating the manifold constraint and pushing the data off the SPD manifold. In summary, these geometry-specific designs lack portability and generality and are infeasible outside their intended manifolds.\n\nAdditionally, geometry-specific layers lead to fragmented architectures with low reusability. This increases implementation overhead. In contrast, GyroAtt adopts a unified architecture that seamlessly supports SPD, SPSD, and Grassmann manifolds without redesign, enabling consistent and modular learning across diverse geometries.\n\n\n**Leveraging Riemannian geometry.** From a theoretical standpoint, Gyro Hom offers three key advantages:\n  - **Gyro Algebraic Structure Simplifies Manifold Layer Design**: Gyro spaces define operators such as gyro addition ⊕ and scalar multiplication ⊗, providing an algebraic framework that is both Euclidean-like and geometry-aware. This facilitates the construction of manifold layers while respecting intrinsic geometry.\n  - **Respecting Gyro Algebraic Structure via Gyro Homomorphisms**: GyroAtt employs gyro homomorphisms in its transformation layers, preserving the underlying algebraic structure and Riemannian geometry. In contrast, MAtt and GDLNet rely on BiMap/FrMap, which only partially adhere to Riemannian geometry. While these layers enforce manifold constraints, they lack isometric properties, potentially distorting the intrinsic geometric structure. For example, the distance $d(S_1, S_2)$ between two SPD matrices $S_1$ and $S_2$ may not be preserved after transformation, i.e., $d(\\mathrm{BiMap}(S_1), \\mathrm{BiMap}(S_2)) \\neq d(S_1, S_2)$. This limitation undermines their ability to fully respect and retain the original geometric properties of the data, which can negatively impact performance.\n  - **Nonlinear Activation**: Existing SPD networks often use ReEig for nonlinear activation, which enforces SPD-ness through eigenvalue regularization, but lacks geometric interpretability. In contrast, GyroAtt adopts a matrix power-based nonlinear activation function that essential activates the latent Riemannian geometry. As shown in [a, Fig. 1], the matrix power can deform any given Riemannian metric. For instance, as the parameter $\\theta \\to 0$, the matrix power deform AIM into LEM.\n\n## **2. Clarification on Experimental Setup and Evaluation Metrics**\n\nWe thank the reviewer for this important question. Our experimental setup closely follows the protocols established in prior works, and we clarify the details as follows:\n\n- For MAMEM-SSVEP-II and BCI-ERN datasets, we strictly adhere to the evaluation setup and metrics described in MAtt [b]. Specifically, we use sessions 1–3 for training, session 4 for validation, and session 5 for testing. We also adopt the same evaluation metrics as in [b]: accuracy for MAMEM-SSVEP-II and AUC for BCI-ERN to account for class imbalance.\n\n- For BNCI2014001 and BNCI2015001 datasets, our evaluation is based on the protocol in [c]. We adopt their inter-session and inter-subject evaluation splits and use the same performance metric: balanced accuracy, computed as the average recall across all classes.\n\n- Regarding preprocessing, we follow [b] for MAMEM-SSVEP-II and BCI-ERN, and [c] for BNCI2014001 and BNCI2015001 to ensure consistency and reproducibility. Details of the used datasets and preprocessing steps are provided in App.D.1.\n\nWe will revise the manuscript to explicitly clarify these details and remove any ambiguity around our evaluation protocol.\n\n**References**\n> [a] RMLR: Extending Multinomial Logistic Regression into General Geometries.\n> \n> [b] MAtt: A manifold attention network for EEG decoding.\n> \n> [c] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG."}, {"review_id": "pZw6F2cCX7", "reviewer": "Reviewer_3rxG", "summary": "This paper proposes a general attention mechanism tailored for matrix manifolds by leveraging operations within gyrovector spaces. It introduces a homomorphism as a fundamental component of the proposed attention framework, specifically designed for Symmetric Positive Definite (SPD), Grassmannian, and Symmetric Positive Semi-Definite (SPSD) manifolds. The effectiveness of the proposed method is empirically validated through experiments on EEG signal classification tasks.", "questions": "See the weaknesses section", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- The paper is well-written and easy to follow.\n- The newly proposed homomorphisms appear to be novel contributions.\n\n### Weaknesses\n\n- The proposed method exhibits poor performance, with overlapping confidence intervals between GyroAtt and baseline methods undermining the claimed superiority.\n- Although the primary contribution is the introduction of homomorphisms, the paper lacks both theoretical justification and empirical validation demonstrating that these homomorphisms offer advantages over alternative transformations, such as multi-layer perceptrons (MLPs) with matrix manifold outputs.\n- While the paper claims a unified perspective on attention mechanisms across matrix manifolds, it fails to derive meaningful insights from this unified framework. Moreover, due to the practical complexity involved in implementing homomorphisms for arbitrary matrix manifolds, the unified view has limited applicability. Consequently, the contribution effectively reduces to separately proposing attention mechanisms for three distinct manifolds rather than establishing a genuinely unified methodology.", "was_revised": true, "final_justification": "### Resolved concerns\n- Performance: I now agree that the performance improvement of the proposed method is not marginal compared to the baselines.\n\n### Remaining concerns\n- Meaningless framework: the proposed GyroAtt framework do not really provides insight or special results due to the difficulty on generalizing the isometry on different manifolds and the absence of theoretical and empirical evidence on homomorphism.\n\nDue to the remaining concerns, I remain my negative score.", "rebuttal": "We thank Reviewer $\\textcolor{purple}{\\rm{3rxG (R4)}}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😊\n\n## **1. GyroAtt yields consistent and statistically significant improvements over baselines.**\n\nTable A: Comparison between GyroAtt and the baseline MAtt, where * denotes $p$-vale < 0.05 and n.s. denotes \"not significant\".\n||MAMEM|BCI-CHA|BNCI2014001 inter-session|BNCI2014001 inter-subject|BNCI2015001 inter-session|BNCI2015001 inter-subject|\n|-|-|-|-|-|-|-|\ngains|3.2%|3.4%|7.8%|8.9%|5.4%|14.8%|\nstatistical significance|*|*|*|n.s.|n.s.|*|\n\nWe emphasize that the observed performance gains are both substantial and consistent across datasets. As shown in Table A, four out of six cases show statistical significance at $p < 0.05$, based on paired t-tests across cross-validation folds. Importantly, GyroAtt consistently outperforms MAtt across all datasets, with a notable $+14.8\\%$ gain in the inter-subject setting, which is particularly challenging in EEG decoding. While some confidence intervals do overlap, this is expected due to the intrinsic characteristics of EEG data, including low signal-to-noise ratios (SNR), significant inter-/intra-session domain shifts, and high variability in subject responses—all of which naturally widen confidence intervals. Despite this high-variance regime, the consistent trend and statistical significance in most cases support the robustness of GyroAtt.\n\n## **2. Advantages of Gyro Hom over manifold MLP‑like layers**\n\nTable B: Ablation on transformation layers (mean ± std, %). Here, / means that the method cannot converge.\n|Transformation|BNCI2014001 inter-session|#Time|BNCI2014001 inter-subject|#Time|BNCI2015001 inter-session|#Time|BNCI2015001 inter-subject|#Time|\n|-|-|-|-|-|-|-|-|-|\n|BiMap|74.0±6.5|6.31|52.3±15.0|115.85|85.2±7.2|3.56|77.2±13.2|73.81|\n|SPD-FC-LEM|70.54±6.8|14.71|40.6±13.2|257.77|84.33±6.2|6.13|74.81±14.8|171.29|\n|SPD-FC-LCM|49.27±11.2|14.48|/|/|74.03±10.4|5.84|/|/|\n|Homomorphisms|75.4±7.4|7.89|53.1±14.1|138.59|86.2±4.5|4.28|77.9±13.0|80.66|\n\n### **2.1 Comparison with SPD-FC Layers.** \n\nAs reported in Table 8, we benchmark BiMap [a] as the canonical MLP‑like transformation on SPD. For a broader comparison, we also include SPD‑FC‑LE/LC [b], which implements manifold fully connected (FC) layers under Log‑Euclidean/Log‑Cholesky metrics. Taking the Log-Euclidean FC layer (SPD-FC-LE) as an example, let $v _{(i,j)}(X) = \\langle \\ominus _{le} P _{(i,j)} \\oplus _{le} X, W _{(i,j)} \\rangle ^{le}, P _{(i,j)}, W _{(i,j)} \\in \\text{Sym} _n ^{+,le},i \\leq j,i,j = 1,\\ldots,m$. Then the output of an FC layer is computed as $Y = \\mathrm{expm}([y _{(i,j)}] _{i,j=1} ^m)$, where $y _{(i,j)}$ is given by\n$$\ny _{(i,j)} = \n\\begin{cases}\nv _{(i,j)}(X), & \\text{if } i = j \\\\\\\\\n\\frac{1}{\\sqrt{2}} v _{(i,j)}(X), & \\text{if } i < j \\\\\\\\\n\\frac{1}{\\sqrt{2}} v _{(j,i)}(X), & \\text{if } i > j \\\\\\\\\n\\end{cases}\n$$\n\nWe now show that SPD-FC-LE is essentially a Euclidean FC layer in the log domain and corresponds to a special case of Gyro Hom under LEM. Here, the LE addition reduces to Euclidean operations in the log domain. Specifically,\n$$\n\\ominus_{\\text{le}} P \\oplus_{\\text{le}} X = \\mathrm{expm}(\\mathrm{logm}X - \\mathrm{logm} P).\n$$\nso\n$$\nv_{(i,j)}(X)= \\langle\\mathrm{logm} X - \\mathrm{logm} P_{(i,j)}, \\mathrm{logm} W_{(i,j)}\\rangle = \\langle\\mathrm{logm} X, \\mathrm{logm} W_{(i,j)}\\rangle + b_{(i,j)},\n$$\nwhere $b_{(i,j)} = -\\langle \\mathrm{logm} P_{(i,j)}, \\mathrm{logm} W_{(i,j)} \\rangle$. This shows that SPD-FC-LE is exactly a Euclidean FC layer applied in the log domain, followed by a matrix exponential:\n$$\nF(X) = \\mathrm{expm}(f(\\mathrm{logm}\\, X)), \\quad \\text{with} f \\text{ is FC on } \\mathrm{Sym}(n).\n$$\n\nImportantly, when $b_{(i,j)} = 0$, i.e., $P_{(i,j)} = I$, the transformation becomes a linear map. In this case, $F$ satisfies the gyro homomorphism:\n$$\nF(X \\oplus_{le} Y) = F(X) \\oplus_{le} F(Y), \\quad F(t \\otimes_{le} X) = t \\otimes_{le} F(X),\n$$\nwhich confirms that SPD-FC-LE is a special case of Gyro Hom under LEM. Similarly, SPD-FC-LC is a special case under the LCM metric.\n\nOnly replacing Gyro Hom with BiMap or SPD‑FC‑LE/LC while keeping all other architectural components unchanged, we observe that Gyro‑Hom consistently outperforms SPD‑FC‑LE/LC across four datasets (Table B).  In addition, Gyro‑Hom is significantly more efficient in computational complexity. Recall that each $v_{(i,j)}$ in SPD-FC-LE requires a pair of symmetric matrices $(P_{(i,j)}, W_{(i,j)}) \\in \\mathrm{Sym}(n)$, leading to $n(n+1)$ parameters per component. For $S^d\\to S^d$, this amounts to $\\tfrac{1}{2} n^2(n+1)^2$ parameters in total. Moreover, the computational complexity is $\\mathcal{O}(n^4)$ per forward pass, in addition to the $\\mathcal{O}(n^3)$ cost of matrix logarithms and exponentials. In contrast, our Gyro Hom uses only an $n\\times n$ parameter with **$n^2$** parameters and $\\mathcal{O}(n^3)$ flops (matrix multiplications and matrix logarithms and exponentials), matching the observed wall‑clock savings.\n\n### **2.2 Comparison with BiMap.** \n\nTo further highlight the benefits of our method beyond efficiency, we analyze its theoretical advantages over BiMap.\n- **Gyro Algebraic Structure Simplifies Manifold Layer Design**: Gyro spaces define operators such as gyro addition $\\oplus$ and scalar multiplication $\\otimes$, providing an algebraic framework that is both Euclidean-like and geometry-aware. This facilitates the construction of manifold layers while respecting intrinsic geometry.\n- **Respecting Gyro Algebraic Structure via Gyro Homomorphisms**: GyroAtt employs gyro homomorphisms in its transformation layers, preserving the underlying algebraic structure and Riemannian geometry. In contrast, MAtt and GDLNet rely on BiMap/FrMap, which only partially adhere to Riemannian geometry. While these layers enforce manifold constraints, they lack isometric properties, potentially distorting the intrinsic geometric structure. For example, the distance $d(S_1, S_2)$ between two SPD matrices $S_1$ and $S_2$ may not be preserved after transformation, i.e., $d(\\mathrm{BiMap}(S_1), \\mathrm{BiMap}(S_2)) \\neq d(S_1, S_2)$. This limitation undermines their ability to fully respect and retain the original geometric properties of the data, which can negatively impact performance.\n\n\n\n## **3. Insights and theoretical advantages**\nTable B: Comparison between the GyroAtt and Euclidean Attention.\n|Component|GyroAtt|Euclidean Attention|\n|-|-|-|\n|Transformation|G1: Gyro homomorphism|E1: Linear map (homomorphism)|\n|Attention|G2: Distance-based similarity|E2: Inner-product-based similarity|\n|Aggregation|G3: Weighted Fréchet mean|E3: Weighted arithmetic mean|\n\n- **Theoretical generality:** Gyrovector spaces naturally generalize Euclidean vector spaces. As shown in Table B, each GyroAtt component extends its Euclidean counterpart in a mathematically grounded way.\n- **Natural Extension:** When the manifold is standard Euclidean geometry, G1 and G3 in GyroAtt exactly reduce to E1 and E3 in Euclidean Attention.\n- **Scalability across geometries:** The unified formulation allowed us to instantiate GyroAtt over seven variants (three SPD, three SPSD, one Grassmann) under the consistent network archetecture, highlighting the applicability of the framework across diverse settings.\n\n\n## **4. Principled identification of gyro homomorphisms**\nThanks for the constructive comments. There are indeed a principled way to identify the gyro homomorphism. As gyrovector space over the manifold is normally defined on the Riemannian homogeneous space [Sec. 3, c], let us focus on the homogeneous space.\n\n**Theorem A (Sufficient Condition)** Let $\\mathcal{M}$ be a Riemannian homogeneous space whose group of isometries is $G$. Besides, $(\\mathcal{M},\\oplus,\\otimes)$ forms a gyrovector space, where $\\oplus$ and $\\otimes$ are defined as [Eqs. (1-3), d]. Let $e$ be the gyro identity element. Every isometry $f \\in G$ that fixes the identity $f(e) = e$ is a gyro homomorphism.\n\nIntuitively, we can identify a homomorphism from the isometries, which are known for a given homogeneous space. This theorem applies to all known matrix gyrovector spaces (e.g., SPD, SPSD, and Grassmannian). For example, one the SPD manifold under the AIM, $GL(n)$ is the isometry group, each of which characterize an isometry, $f _A : \\mathcal{S} _{++} ^n \\in S \\mapsto A S A ^\\top \\in \\mathcal{S} _ {++} ^n$, with $A \\in GL(n)$. Applying $f _{A}(I)=I$ implies that $A$ should be an orthogonal matrix. This is exactly the result in Thm. 4.1. \n\nBesides, if a manifold has a flat structure, we can directly identify the gyro homomorphism from the flat structure. This is how we obtain the homomorphism for the SPD manifold under LEM and LCM (Thm. 4.1). Specifically, for the LEM, the SPD manifold becomes a flat space under the matrix logarithm. This allows us to construct a gyro homomorphism via linear operations in the log domain. In this case, the homomorphism reduces to the form\n$$\n\\mathrm{hom}_{le}(P)=\\mathrm{expm}(M\\mathrm{logm}(P)M^\\top),\n$$\nwhich matches the second case in Eq. (8) of Theorem 4.1.\n\n**Proof of Thm. A** This theorem can be induced from Lem. 2.1-2.2 in [d]. Let the Riemannian metric tensor be $g$. We have\n$$\nf: (M,f^*g) \\to (M,g), \\text{ with } f(e) = e,\n$$\nwhere $f^*g$ is the pullback metric under the diffeomorphism $f$. As $f$ is an isometry, we have $f^*g=g$. Applying Lem. 2.1-2.2 in [d], one can immediately prove\n$f(P \\oplus Q) = f(P) \\oplus f(Q)$ and $f(t \\otimes Q) = t \\otimes f(Q)$ for any $t \\in \\mathbb{R}$ and $P,Q \\in M$.\n\n\n**References**\n\n> [a] A Riemannian network for SPD matrix learning.\n> \n> [b] Matrix Manifold Neural Networks++.\n> \n> [c] The Gyro-Structure of Some Matrix Manifolds.\n> \n> [d] Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach."}]

Author rebuttal:

Meta-review: The reviewers acknowledged the well-motivated and rigorous method, the extensive evaluation, and the effectiveness of the method. Nevertheless, they expressed some concerns about the computational cost of the method, the limited detail on the baselines, the limited evidence of the impact of some aspects of the method, and the statistical significance of the results. The authors' feedback addressed some of these concerns, resulting in 3 reviewers recommending acceptance. Reviewer 3rxG nonetheless remained unconvinced by the rebuttal, recommending rejection. In a discussion with the AC, the authors pointed out the difference between the general framework that they propose and the unified framework suggested by 3rxG, able to dynamically adapt to data from matrix manifolds of different natures. The AC argues that the fact that the same framework can be applied to different matrix manifolds, even though it cannot handle different data types simultaneously, is sufficient, and thus agrees with the other reviewers that the paper meets the NeurIPS acceptance threshold. The authors are nonetheless encouraged to incorporate elements of their answers in the final version of the paper.
Decision: Accept (poster)