Gyrogroup Batch Normalization
Authors: Ziheng Chen
Yue Song
Xiaojun Wu
Nicu Sebe
Venue: ICLR 2025 Poster (2025)
Keywords: Gyrovector Spaces
Riemannian Manifolds
Riemannian Batch Normalization
Grassmannian Manifolds
Hyperbolic Manifolds
Primary area: learning on graphs and other geometries & topologies
TL;DR: This paper proposes a general framework for Riemannian Batch Normalization (RBN) over gyrogroups (GyroBN), which incorporate several existing RBN methods, and showcase our framework on the Grassmannian and hyperbolic geometries.

Abstract: Several Riemannian manifolds in machine learning, such as Symmetric Positive Definite (SPD), Grassmann, spherical, and hyperbolic manifolds, have been proven to admit gyro structures, thus enabling a principled and effective extension of Euclidean Deep Neural Networks (DNNs) to manifolds. Inspired by this, this study introduces a general Riemannian Batch Normalization (RBN) framework on gyrogroups, termed GyroBN. We identify the least requirements to guarantee GyroBN with theoretical control over sample statistics, referred to as \textit{pseudo-reduction} and \textit{gyroisometric gyrations}, which are satisfied by all the existing gyrogroups in machine learning. Besides, our GyroBN incorporates several existing normalization methods, including the one on general Lie groups and different types of RBN on the non-group SPD geometry. Lastly, we instantiate our GyroBN on the Grassmannian and hyperbolic spaces. Experiments on the Grassmannian and hyperbolic networks demonstrate the effectiveness of our GyroBN. The code is available at https://github.com/GitZH-Chen/GyroBN.git.

Published as a conference paper at ICLR 2025
GYROGROUP BATCH NORMALIZATION
Ziheng Chen', Yue Song', Xiao-Jun Wu & Nicu Sebe' University of Trento, Jiangnan University
ABSTRACT
Several Riemannian manifolds in machine learning, such as Symmetric Positive Definite (SPD), Grassmann, spherical, and hyperbolic manifolds, have been proven to admit gyro structures, thus enabling a principled and effective extension Euclidean Deep Neural Networks (DNNs) to manifolds. Inspired by this, this study introduces a general Riemannian Batch Normalization (RBN) framework gyrogroups, termed GyroBN. We identify the least requirements to guarantee GyroBN with theoretical control over sample statistics, referred to as pseudo- reduction and gyroisometric gyrations, which are satisfied by all the existing gyrogroups in machine learning. Besides, our GyroBN incorporates several existing normalization methods, including the one on general Lie groups and different types of RBN on the non-group SPD geometry. Lastly, we instantiate our GyroBN on the Grassmannian and hyperbolic spaces. Experiments on the Grassmannian and hyperbolic networks demonstrate the effectiveness of our GyroBN. The code is available https: : / /gttbb..
Bet.tttt--eee/Grr..... git.
INTRODUCTION
Deep Neural Networks (DNNs) Riemannian manifolds have gained increasing interest in various machine learning applications, such as computer vision (Huang et al., 2017; Huang & Van Gool, 2017; Huang al., 2018; Skopek et al., 2019; Wang et al., 2022b;a; Chen et al., 2023c; Gao et al., 2023; Wang et al., 2024b; Chen et al,, 2025), natural language processing (Ganea et al., 201 8; Shimizu al., 2020), drone classification (Brooks al., 2019; Chen et al., 2024a), human neuroimaging (Pan et al,, 2022; Kobler et al., 2022a; Ju et al., 2024; Wang et al., 2024a), medical imaging (Huang et al., 2019; Chakraborty et al., 2020), node and graph classification (Chami et al., 2019; Dai al., 2021; Zhao et al., 2023; Chen et al., 2023b; Nguyen et al., 2024; Chen et al., 2024c). As core techniques in DNNs, normalization techniques (Ioffe & Szegedy, 2015; Ba al., 2016; Ulyanov al., 2016; Wu He, 2018; Chen et al., 2023a) have also been extended into different geometries.
However, most existing Riemannian normalization methods are designed for selected few geometries or fail to normalize the sample statistics. Brooks et al. (2019); Kobler et al. (2022b;a) introduced Riemannian Batch Normalization (RBN) on SPD manifolds under the specific Affine- Invariann Metric (AIM). Chakraborty (2020) generalized this idea and proposed Riemannian normalization framework over homogeneous spaces. However, this approach cannot generally normalize the sample statistics. Similar formulation and issue can also be found in Lou et al. (2020, Alg. 2) and Bdeir al. (2024, Sec. 4.2). Besides, Chakraborty (2020) also developed Riemannian normalization for matrix Lic groups. Although this approach can control first- and second-order statistics, it is limited a specific type of distance (Chakraborty, 2020, Sec. 3.2). Chen et al. (2024b) took one step further and developed RBN over the general Lie group, referred to as LieBN. Although LieBN can effectively normalize sample statistics, many geometries do not admit group structure. In summary, the existing RBN methods fail to normalize manifold-vales samples in principled manner.
Recently, building Riemannian networks based on gyro structures has shown notable success across various geometries, including Symmetric Positive Definite (SPD) (Nguyen, 2022a;b), Grassmannian (Nguyen, 2022b), hyperbolic (Ganea et al., 2018), and spherical (Skopek et al., 2019) manifolds. Gyro structures, natural extensions of vector structures, offer powerful mathematical tools for building Riemannian neural networks. Moreover, gyrogroups naturally encompass Lie groups and extend to non-group geometries. For instance, AIM on the SPD manifold, as well as Grassmannian, hyperbolic, and spherical manifolds, do not form Lie groups but instead gyrogroups.
"Correspondence ziheng_che ch@@ 163.com

---

Published as a conference paper at ICLR 2025
GyroBN on the Grassmannian
GyroBN the Poincard Ball
Merulinnd hando
65
35
10
E3
6.1
120
0.
-35
-05
10
1- -10
Figure 1: Illustration of GyroBN on the Grassmannian and hyperbolic spaces. As (r(1,3) is homeomorphic to the real projective space RP?, we illustrate Gr(1, 3) as the unit hemisphere with antipodal points identified. We set the bias as I13 (1. 0. 0) and the scaling scalar as 0.2 for better illustration. For the hyperbolic space, we visualize the GyroBN on the Poincaré ball model IP,, which is the interior of the unit sphere in R We set bias and shift as zero vector and 0.7, respectively.
Table I: Comparison of previous RBN methods with our GyroBN, where M and V denote the sample mean and variance. Compared with the existing RBN methods, our GyroBN can normalize sample statisties in principled manner. Besides, several previous RBN methods with theoretical control over sample statistics are special cases of our GyroBN.
Method
Controllable Statisties
Applied Geometries
Incorporated by GyroBN
SPDBN
SPD manifolds under AIM
(Brooks et ul., 2019)
SPDBN
M+V
SPD manifolds under AIM
(Kobler et al., 2022b)
SPDDSMBN
M+V
SPD manifolds under AIM
(Kobler ct 20224)
ManifoldNorm (Chakraborty. 2020, Algs. 1-2)
N/A
Riemannian homogeneous space
x
ManifoldNorm
Matrix Lie groups under the distance d(x, Y) = [illg (x X
M+V
(Chakraborty, 2020, Algs. 3-4) RBN
(Lou et al. 2020, 2020, Alg. 2) LieBN
N/A
Geodesically complete manifolds
x
M+V
General Lie groups
(Chen et al., 2024b)
Pscudo-reductive gyrogroups with gyro isometric gyrations
GymoNN
M+V
N/A
Based on the above analysis, this paper introduces GyroBN, an RBN framework for general gy- rogroups. We use the gyro addition, subtraction, and scalar product to extend the centering (vector subtraction), biasing (vector addition), and scaling (vector scalar product) in the Euclidean BN into manifolds in a principled manner. For broader applicability and in-depth theoretical analysis, we adapt the existing gyrogroup into a more relaxed structure, termed pseudo-reductio gyrogroup. Our theoretical analysis shows that pseudo-reductive gyrogroups with gyroisometric gyrations can enable GyroBN with theoretical control over sample statistics. More importantly, these requirements are satisfied by all the existing gyrogroups machine learning. Therefore, compared with the existing RBN methods, our GyroBN can normalize sample statistics in principled manner. Besides, several existing RBN methods are incorporated by our GyroBN as special cases, including the LieBN on Lie groups, such as three SPD Lie groups and rotation matrices, and different types of RBN on the non-group SPD geometry. We provide a detailed comparison in Tab. 1. Empirically, we instanti- ate GyroBN the Grassmannian and hyperbolic spaces, as illustrated in Fig. 1. To the best of our knowledge, our Grassmannian GyroBN is the first implementation of Grassmannian RBN. Experiments on the Grassmannian and hyperbolic networks validate the effectiveness of our GyroBN.
Our main theoretical contributions are summarized as follows: (1) We propose the peeudv-reductive gyrogroup, a relaxed structure of the gyrogroup, and present relevant theoretical analyses; (2) We identify the requirements that guarantee our GyroBN with theoretical control over sample statistics, i.e., ree.pseudo-reducctioo and gyroisometric gyrations; (3) We propose GyroBN framework for RBN over general gyrogroups, which can be manifested in various geometries in plug-and and-player manner. (4) We implement our GyroBN on the Grassmannian and hyperbolic spaces. Extensive experiments on popular Grassmannian and hyperbolic networks validate the effectiveness of our framework.
2

---

Published as a conference paper at ICLR 2025
Main theoretical results: Def. 3.1 relaxes the existing gyrogroup into the pseudo-reductive gy- rogroup. Prop. 3.2 reveals that the non-reductive Grassmannian gyrogroup is, in fact, pscudo-reductive. Thms. 3.3 and 3.5 highlight that the invariance of gyronorm under gyrations is crucial for enabling several operators to act as gyroisometries. Prop. 3.6 confirms that all gyrogroups listed in Tab. pseudo-reductive and their gyrations are gyroisometries. These two properties are essential for enabling GyroBN in Alg. to normalize sample statistics, which are formalized in Thm. 4.1. Sec. discusses how several existing RBN methods are special cases of our GyroBN. Lastly, Sec. 6 manifest our GyroBN on the Grassmannian and hyperbolic spaces, where Prop. 6.1 discuss the efficient implementation on the Grassmannian. Due to page limits, all the proofs are presented in App. G.
PRELIMINARIES
This section recaps gyrogroups (Ungar, 2009) and several concrete gyrogroups in machine learning. Definition 2.1 (Gyrogroups (Ungar, 2009)). Given nonempty set G with binary operation @ C:GxC G + G, (G, @ forms a gyrogroup its binary operation satisfies the following axioms for any a, G:
(GI) There is at least one element e E G called a left identity (or neutral element) such that e 3 a = a.
(G2) There is an element ea G' called a left inverse of a such that Go @
(G3) There an automorphism gyr(a, b] + G for each a, E G such that
@ (b c) (a @ gyr(a, b|c (Left Gyroassociative Law).
(1)
The automorphism gyr[a, b] is called the gyroautomorphism, or the gyration of G' generated by a, b. (G4) Left reduction law: gyr[a, b] = gyrla la @ b b.
Definition 2.2 (Gyrocommutative Gyrogroups (Ungar, 2009)). A gyrogroup {G, {) is gyrocommu- tative if it satisfies
+ b = gyrla, (a, b| 3 a) (Gyrocommutativee Law).
Definition 2.3 (Nonreductive Gyrogroups (Nguyen, 2022a)). A groupoid {G, b} is a nonreductive gyrogroup if satisfies axioms (GI), (G2), and (G3).
Intuitively, gyrogroups are natural generalizations of groups. Unlike groups, gyrogroups are non- associative but have gyroassociativity characterized by gyrations. Since all gyrations in any (Lie) group are the identity map, every (Lie) group is automatically gyrogroup.
As shown by Nguyen Yang (2023), given P. Q and in a manifold M and t E R, the gyro structures can be defined as:
Gyro addition: P@ Q = Expp (PT E-P (Logg
Gyro scalar product: / O P = Expg (1 Logg (P)),
Gyro inverse: e P = - P = ExpE (- Loge (P))
(5)
Gyration: gyr[P, Q (e(P @ Q1) @ (P @ (Q @ R)),
(6)
Gyro inner product: (P, Q) gr = (Loge( Log (Q)
(7)
Gyro norm: || = (P,
(8)
Gyrodistance: dgey (P, Q) = IIeP a Q11
(9)
where E is the gyro identity element, and Log and (+. .) is the Riemannian logarithm and metric at E. A bijection w G > G is called gyroisometry, if it preserves the gyrodistance
(wr)
day (P.Q).
(10)
Note that the gyro scalar product @ is required for gyrogroup to form a gyrovector space (Nguyen, 2022b). Although this paper only involves gyrogroups, we also recap gyrovector spaces in App. C.
Several geometries in machine learning admit a gyro structure defined in Eq. (3)-Eq. (9) and form (nonreductive) gyrogroup, such as Affine-Invarial Metric (AIM) (Pennec et al., 2006), Aug-Fueli Metric (LEM) (Arsigny et al., 2005), and Log-Cholese Metric (LCM) (Lin, 2019) on the SPD manifold S (Nguyen, 2022a), Orthonormal Basis (ONB) perspective Gr(p, n) (Bendokat et al.,
2024) and projector perspective Gr(p, n) (Bendokat al., 2024) for the Grassmannian (Nguyen,
3

---

Published as a conference paper at ICLR 2025
2022a; Nguyen & Yang, 2023), Poincaré ball Pk for the hyperboloid (Ungar, 2009; Ganea et al., 2018), and projected hypersphere D for the hypersphere (Skopek et al., 2019), Besides, the gyrogroups proposed by Nguyen (2022b) on the SPD manifold under the LEM and LCM coincide with the Lie groups proposed by Arsigny et al. (2005); Lin (2019).
We denote Mи IPx (K < 0), D3 (K 0) and ( 0), respectively. Mx is known as the Constant Curvature Space (CCS) (Do Carmo & Flaherty Francis, 1992). We summarize all the necessary gyro properties in Tab. 2.
Table 2: Gyrogroup properties on several geometries. Related notations are defined in App. C.3.2.
Genmetry
Symhol
P Q Dv
E
oP Cr
Lie group
Gyrogrupp
References
57+
gA1
PTQ
p-1
(Ngayen, 20226)
LEM ST
6AR
mexpiming P) mlog(qi)
p-1 In
(Ansigey et 2005)
(Nguyen, 20226)
(Lin, 2019)
LCM
Peoletu(P) (1))
(Ngryen, 2022h)
(Chen nl.,
Grip. n)
@
me mexpp(-2)
mmapp (3) mexp((1)
(Nguyen, 2022a)
Grip.n) n)
musspp f)1,..
Noe-reductive
(Nguyen & Yang, 2023)
Mi
x
[L-2Ntxa) Kial 1-
1- TePloP
Xy for K=O)
(Skopek et ol, 20199
(Ganca et al., 2018)
[Ungar, 2009)
3 PSEUDO-RED REDUCTIVE GYROGROUPS
As shown in Tab. the Grassmannian does not satisfy left reduction (G4) in Def. 2.1. However, we find that a relaxed version of (G4) is necessary to guarantee the sample normalization over gyrogroups. Therefore, this section proposes an intermediate between the gyrogroup and the nonreductive one, called pseudo-reductive gyrogroups.
Unless specifically emphasized, the gyro structure in this paper defined as Eq. (3)-Eq. (9). Given gyrogroup {G, }), the left gyrotranslation by e is defined
Lp G + G, with (Q(Q) = P + Q. E
(11)
any left gyrotranslation a gyroisometry, we can use gyrotranslation to center samples. Nguyen & Yang (2023) show that any left gyrotranslation on the SPD and Grassmannian manifolds is gyroisometry. However, the proof relies on the left cancellation law of gyrogroups, which does not hold for nonreductive gyrogroups. Therefore, the proof is questionable for the Grassmannian. This subsection proposes an intermediate structure, referred to as pseudo-reductive gyrogroups, which can support left cancellation law in general and, therefore, gyroisometry of left gyrotranslation. We illustrate the derivation logic in Fig. 2.
Axiom (GI-3) Pseudo-reduction
Ours:
Left concellation law
Oz
Gyroisometries of gyration gyrotannsation
Previous:
Asiom (Gi-3) Left reduction (G4)
Lefl gyrotrashtion law
Invariance of gynorom under gyratonn
Gyroisometry gyroinvense
Gyrocommutativvty
The conceptual comparison of derivation logic of gyroisometries of our work against previous work (Nguyen & Yang, 2023), where the left gyrotranslation law is presented in Lem. G. The previous work proves the results on the SPD and Grassmannian manifolds in a case-by-case manner. In contrast, we relax the left reduction into pseudo-reduction and give a general theoretical framework. Our framework also corrects the proof for the Grassmannian cases.
Figure 2:
Definition 3.1 (Pseudo-reductive Gyrogroups). A groupoid {G, (G, is pseudo-reductive gyrogroup it satisfies axioms (G1), (G2), (G3) and the following pseudo-reductive law:
gyr X, P] 1, for any left inverse X of in G,
(12)
4

---

GYROGROUPS Published as a conference paper at ICLR 2025
where 1 is the identity map.
Eq. (12) can be intuitively viewed as the intermediate between reduction and non-rcduction. For gyrogroups, Eq. (12) can be directly obtained from the left gyroassociativity (G3) and reduction (G4) (Ungar, 2009, p. 12). However, there no theoretical guarantee that Eq. (12) holds for general non-reductive gyrogroups. Therefore, we name Eq. (12) as pseudo-reduction. Nevertheless, for the specific non-reductive Grassmannian, it is indeed pscudo-reductive.
Proposition 3.2. [1]] Gr(p, n) and Gr(p, n) form pseudo-reductive reductive and gyrocommulative gyrogroups.
shares most of the basic properties of gyrogroups (Ungar, 2009, Thms. 1.13 - 1. 14), which are detiled in Thm. D.I. The most related property in Thm. D.1 is the left cancellation law, one of the key prerequisites for gyro translation gyroisometry. Note that the left cancellation the gyrogroup comes from left gyro associativity and Eq. (12) (Ungar, 2009, P. 12). Therefore, left cancellation does not generally hold for non-reductive gyrogroups, but exists in pseudo-reductive gyrogroups. Next, point an iff statement about gyroisometry, which will be useful in the following.
Our pseudo-reductive gyrogroup naturally generalizes the vanilla gyrogroup.
Theorem 3.3. [4] Given a pseudo-reductive gyrogroup {G, )), gyr(P, Q) preserves gyronorm for any P,Q E G. iff gy Q] is gyroisometry for any P. P.Q E G.
We find that the isometry of any gyration is key prerequisite for other gyro operators as isometries. Definition 3,4 (Gyro Left- Left-invariance). The gyrodistance or gyrogroup is gyro left- left-invariant if any left gyrotranslation is gyroisometry. Theorem 3.5 (Gyroisometriee), [J] Given pseudo gyrogroup {G, + with any gyrl:, -] a gyroisometry, then we have the following:
1. The gyrodistance (Eq. (9)) is gyro left- offt-invariant;
2. Symmetry of the gyrodistance: VP,Q E G,d dgry(QQ
3. If {G, + is gyrocommutative, then the gyroinverse is gyroisometry;
Proposition 3.6. [4] For every (pseudo-reductive) gyrogroup in Tab. 2, the gyrodistance is iden- tical to the geodesic distance (therefore symmetricc.. The gyroinverse, any gyration and any left gyrotranslation are gyroisometries.
Credit of the proof. For the Grassmannian and SPD manifolds, the isometries of gyration, gyroin- verse, and left gyrotranslation have been proven by Nguyen & Yang (2023, Thms. 2. 12 14 and 2.16 2. 18). Nevertheless, in the proof of Thms. 2. 12 2. 14, the authors view the non-reductive Grassmannian as a gyrogroup, as they use the left cancellation property of the gyrogroup. Fortunately, Prop. 3.2 and Thm. D.1 shows that the Grassmannian still enjoys left cancellation. Therefore, all the results on isometries in their Thms. 12-2...14 are correct. Nevertheless, these results on the SPD and Grassmannian manifolds can be directly obtained by our Thms. 3.3 and 3.5, these gyrogroups are pseudo-reducttv and the gyrations preserve the gyronorm. Besides, we show the expressions for the gyrodistance of all gyrogroups Tab. 2, and the associated gyroisometries on MM, both of which is none-trivial. The detailed proof is presented in App. G.4.
4 GYROBN ON GENERAL
Prop. 3.6 shows that several geometries enjoy isometric gyrotranslation, offering theoretical foundation for normalizing samples over gyrogroups in principled manner. Inspired by this, this section develops Riemannian Batch Normalization (RBN) for general pseudo-reductive gyrogroups, referred to as GyroBN. In the following, (.M @ is assumed as a pseudo-reductive gyrogroup with the gyro structure defined Eq. (3)-Eq. (
4.1 EUCLIDEAN BATCH NORMALIZATION REVISITED
As the core operations of different Euclidean normalization variants (loffe & Szegedy, 2015; Ba et al., 2016; Ulyanov et al., 2016; Wu & He, 2018) are similar, this paper focuses on BN. Given a batch of activations {#;...n] ), the core operations of BN can be expressed as:
L; 1
Vi < N 24 <-
+
(13)
Ve"
In our GyroBN, @ is required comply with the axioms of the gyrovector space (Def. C.4).
5

---

Published as a conference paper at ICLR 2025
where μ., v², ,, and are the sample mean, sample variance, scaling parameter, and biasing parameter, respectively.
4.2 GYROBN
To generalize the Euclidean BN into gyrogroups, we first define sample mean, sample variance, centering, biasing, and scaling over gyrogroups. Then, we introduce our GyroBN framework with theoretical analysis of the ability to normalize sample statistics.
define the gyromean as the Fréchet mean under the gyrodistance:
FM((P:) argmin QEM N
Σ
da gry (P:
(14)
The gyrovariance is the Fréchet variance, the minimization of the rightest hand side Eq. (14). When the distance in Eq. (14) is the geodesic distance, the Fréchet mean and variance are known as the Riemannian mean and variance. Although the gyromean and gyrovariance are not necessarily the same as the Riemannian ones, Prop. 3.6 indicates the equivalence for the gyrogroups in Tab. 2.
Easy computation shows that the centering and biasing in the Euclidean BN (Eq. (13)) can be viewed as gyro addition (Eq. (3)) in I",, and the scaling can be viewed as gyro scalar product (Eq, (4)), scaling in the tangent space at the identity element. Inspired by this, we define the normalization over gyrogroups by gyro addition and gyro scalar product. Given a batch of activations {Pi...N E &M. we defined the core operations of GyroBN
Scaling
Blasing
Centering
Vi N B
O + P
(15)
where M M and 22 are gyromean and gyrovariance, B M is the biasing parameter, E IR. is the scaling parameter, and is small value for numerical stability.
Theorem 4.1 (Homogeneity). [L] Supposing {M, }} is a pseudo-reductive gyrogroup with any
gyration gyrl. as a gyroisometry, for N samples [P.... E M), we have the following properties:
Homogeneity of gyromean: FM((B + P.) B FM((P))),BBEA
(16)
Homogeneity of dispersion from E:
1 Σ En Byy (to P. N
1 E
d Dey (P,
(17)
The most important property of the Euclidean BN (Eq. (13)) lies in its ability normalize data distribution by the control over sample mean and variance. Similarly, our formulation in Eq. (15) can also normalize gyromean and gyrovariance. Specifically, given rreudt-reductive gyrogroup with isometric gyrations, Eq. (16) indicates that the centering and biasing can transfer the gyromean, while Eq. (17) can scale the sample variance, since after centering, the resulting gyromcan is the identity element E.
To finalize GyroBN, we define the rumning mean updates over gyrogroups the binary barycenter based on gyrodistance:
(Pr, argminqeM (md (Pi, Q) (1 -1- (P2, ())
with v E [0, (18)
Notably, when the gyrodistance is identical the geodesic distance, the binary barycenter can be calculated by geodesic.
With all the above ingredients, the general framework for our GyroBN is presented in Alg. 1. Thm. 4.1 indicates that given a pseudo-reductive gyrogroup with any gyration as gyroisometry, our GyroBN enjoys theoretical guarantee of control over the gyro statisties. Specifically, for the gyrogroups in Tab. 2, our GyroBN can control the gyromean and gyrovariance. Besides, as the gyromean and gyrovariance identical the Riemannian counterparts, the GyroBNs these gyrogroups also normalize the Riemannian statistics. Especially, simple computation shows that our GyroBN recovers the standard Euclidean BN (loffe & Szegedy, 2015) when M R

---

Published as a conference paper at ICLR 2025
Algorithm I: Gyrogroup Batch Normalization (GyroBN)
Require
batch of activations {PI...N E MM. small positive constant e, and momentum 7 E [0, 1J, running mean M,, running variance vf. biasing parameter B E M, scaling parameter E IR.
Return
nnormalized batch { (PI...N E cM}
training then
Compute batch mean M, and variance of of {Pi...N::
Update running statisties M, Bar, My), 02 = = 8 + (1 ?)?
end
(M, 2²) (M/, vgg) if training else (M, v22)
vi N. N. B Vo'+e C (eM + P.)
5 A GYRO PERSPECTIVE FOR THE EXISTING RIEMANNIAN NORMALIZATIONS
Several existing Riemannian normalization methods on different geometries enjoy theoretical control of sample mean and variance, including LieBN on general Lie groups (Chen et al., 2024b), and SPDBNs based on the specific AIM geometry (Brooks et al., 2019; Kobler et al., 2022b;a). This subsection further reveals that they are concrete implementations of our GyroBN.
5.1 LIEBN AS A SPECIAL CASE OF GYROBN
Chakraborty (2020, Algs. 3-4) first proposed Riemannian normalization on matrix Lie groups under a specific distance. Chen et al. (2024b) extended their framework into general Lie groups, referred to LieBN, with theoretical control over Riemannian
and variance. This subsection shows that
LieBN is indeed a special case of our GyroBN.
LieBN is established under left-invariant metric on the Lie group. The centering and biasing are defined by left group translation, and scaling is defined by the scaling on the tangent space at the identity element (Chen al., 2024b, Eq. 13-5)..) As every Lie group automatically gyrogroup. the gyrotranslation is the exact group translation. Therefore, the centering, biasing, and scaling are the same under GyroBN and LieBN. However, the mean, variance, and running mean updates on LieBN are defined based on the geodesic distance. In contrast, the counterparts on the GyroBN are based on gyrodistance. Nevertheless, the following proposition demonstrates the equivalence of these operators under the GyroBN with LieBN.
Proposition 5.1. [4] Given a Lie group with a left-invariant metric, the gyrodistance and geodesic distance are identical. The GyroBN is, therefore, identical to the LieBN (Chen et al., 2024b, Alg. 1).
Chen et al. (2024b) implemented LieBN on four left- intarinvariant geometries, including SPD manifold with AIM LEM and LCM, and rotation matrices. According Prop. 5.1, these implementations are immediately the special cases our GyroBN.
5.2 AIM-BASED SPDBNS AS SPECIAL CASES OF GYROBN
Several RBNs on the SPD manifold were developed based on AIM (Brooks et al., 2019; Kobler et al., 2022b;a). The core operations of these approaches can be expressed as the following:
Normalization: Vi < N, P. B (M- Vo B
(19)
where M and 2 are the Riemannian mean and variance, the Fréchet mean and variance under the geodesic distance. The running mean is updated by binary barycenter under the geodesic distance.
Prop. 3.6 indicates that, under the AIM geometry, the gyrodistance identical to the geodesic distance. Therefore, the gyromean and gyrovariance are identical to the Riemannian mean and variance. The running mean updates are also identical under the gyrodistance and geodesic distance. Besides, simple computations show that Eq. (19) is exactly the specific implementation of Eq. (15) under {S AAI, edd J. where A denotes AIM. Therefore, the SPDBNs developed by Brooks et al. (2019); Kobler et al. (2022b;a) also special cases of our GryoBN.
PAT is left-invariant w.r.t. the Lie group operation P Liell Q LQQ._ with I. as the Cholesky factor of P L.I." (Thanwerdas Pennee, 2022). This group structure differs from the one presented in Tab. 2.
7

---

Published as a conference paper at ICLR 2025
Remark 5.2. Brooks et al. (2019) only consider centering and biasing. Kobler et al. (2022b) use running mean for centering during the training. Kobler et al. (2022a) use different momentum to update running statistics for training and testing and multi-channel mechanisms for domain adaptation. Nevertheless, all of them are based on Eq. (19). Therefore, tricks such as multi-channel and separate momentum can also be applied incorporates their approaches.
GyroBN. This is what mean by claiming that GyroBN
6 GYROBNS ON GRASSMANNIAN AND HYPERBOLIC SPACES
As indicated by Prop. 3.6 and Thm. 4.1, our GyroBN can be implemented on the Grassmannian and hyperbolic spaces, with the ability to normalize sample statistics. Our Alg. allows us to implement GyroBN in a plug-and-play manner. This section clarifies additional technical details regarding its application in these two spaces. As Prop. 3.6 has demonstrated the equivalence of the gyrodistance and geodesic distance on these spaces, we use the terms "gyromean" and "gyrovariance" interchangeably with their Riemannian counterparts.
6.1
GRASSMANNIAN GYROBN
We focus on the ONB perspective. To the best of our knowledge, this is the first RBN on the Grassmannian. Given a batch of activations (U1...v over (Gr(p, n), @Gr), }, Eq. (15) can be expressed
as
Centering the identity Ip. U/ mexp
(-IM Ip.ll) Us.
(20)
Scaling the dispersion from Ip.:: U? mexp
[U((U))T. Tpunl) Ip
(21)
Vo2 + (B, U?
Biasing towards parameter E M: U mexp
(22)
where (.) Log7, (+) with Log as the Riemannian logarithm under the projector perspective, and M the Riemannian mean of (U1...) and Jp.n Ip JT, p.n is the identity element under the projector perspective. The Riemannian mean can be calculated by the Karcher flow (Karcher, 1977). We use Alg. 5.3 by Bendokat al. (2024) for a stable and efficient computation the Riemannian logarithm required in the Karcher flow. For [M MT Tp.m] and [U] (U!)T, Ip.m], inspired by Bendokat et al. (2024, Alg. 5.3) and Nguyen al. (2024, Prop. 3.12), we have the following for fast computation. Proposition 6.1. [1] Given U (UT Ug) we have the following
) E Gr(p, n) with U1 RRxP and U2 (n then
UT 0
LUT Ip.n]
(23)
U2
where U2 = U2Q (S) R. and UT SVD = QSRT.
Here S is in ascending order, Q and R are
siiuly-wisely flipped accordingly, and S= -1-1155
Remark 6.2. Although focus on the GyroBN under the ONB perspective, the GyroBN under the projector perspective can be calculated via the ONB perspective by the following process: (1) mapping data into the ONB perspective by -| Gr n) + Gr(p, n); (2) normalizing data by the GyroBN under Gr(p, r)); (3) mapping normalized data back to Gr(p, n) by n. Technical details are presented in App. E.
6.2 HYPERBOLIC GYROBN
We focus the Poincaré ball model over the hyperbolic space, i.e.Pr The specific manifestation can be conducted in a plug-in manner. We simply need to plug the required operators from Tab. 2 and Tab. 8 into Alg. 1. Besides, the Poincaré Fréchet mean can be calculated by Lou al. (2020, Alg. 1)]. Remark 6.3. As shown by Cannon et al. (1997), there are five isometric models that one can work for the hyperbolic spaces. Although we focus on the Poincaré ball, the GyroBN under other isometric metrics can also be easily constructed via the Poincaré GyroBN. The overall process is similar to Rmk. 6.2. For more detail, please refer to Lem. E. and Thm. E.2.
8

---

Published as a conference paper at ICLR 2025
7 EXPERIMENTS
Our GyroBN layers are model-agnostic and can be seamlessly integrated into any network operating over the gyrospaces listed in Tab. 2. This section evaluates the effectiveness of our GyroBN on Grassmannian and hyperbolic neural networks.
7.1 EXPERIMENTS ON THE GRASSMANNIAN
Implementation. We focus on newly developed Grassmannian network, GyroGr (Nguyen & Yang, 2023), which replaces the non-intrinsic transformation block (FRMap + ReOrth layers) in the classic GrNet (Huang et al., 2018) with Grassmannian left gyrotranslation. This modification has demonstrated improved numerical performance and stability (Nguyen & Yang, 2023). GyroGr constructed over the ONB Grassmannian and consists of three basic blocks: left gyrotranslation, pooling (Huang et al., 2018), and the projection map (ProjMap) (Huang et al., 2018). The projection map functions as an activation layer that maps data into symmetrie matrices for classification. Following Nguyen & Yang (2023), we evaluate our method on skeleton-based action recognition tasks, including the HDM05 (Müller et al., 2007), NTU60 (Shahroudy et al., 2016), and NTU120 (Liu et al., 2019) datasets, focusing mutual actions for NTU60 and NTU120. For fair comparison, we also extend ManifoldNorm (Chakraborty, 2020, Alg. 1-2) and RBN (Lou et al., 2020, Alg. 2) the Grassmannian. Although these two BN methods the Grassmannian, they can be adapted by leveraging Riemannian operators such as geodesics, exponential and logarithmic maps, and parallel transport. The key difference is that our GyroBN can normalize data distributions across different geometries, whereas the other two methods cannot, Further details on datasets, implementation. and training efficiency are provided in App. F2.
not originally implemented over
Table 3: Comparison of GyroBN against other Grassmannian BNs under GyroGr backbone.
BN
None
ManifoldNorm Gr
RBN-Gr
GyroBN-Gr Gr
Ace.
Mcanistd
Max
Mcan#std
Max
Mcantstd
Max
Mcantstd
Max
HDM05
48.97+0.24
49.23
49.67+0.76
50.41
48.64=0.77
49.49
51.8920.37
52,43
NTU60
13
70.32
68.56+0.43
69.14
67.77=0.52
68.35
72.60=0.04
72.65
NTU120
53.7640.18
53.96
51.41
51.92
50.56:0.22
50.82
55.47=0.10
55.59
Table 4: Ablation of Grassmannian GyroBN under various network architectures.
HDMOS
NTU60
NTU120
Architecture
1Block
2Block
3Blookk
4Block
1Block
2Block
3Block
4Block
1Block
2Block
3Block
4Block
GyroGr
49.23
49.09
47.02
27.36
70.32
70.14
70.23
65.03
53.96
54.1
$4.59
47.59
GyroGrBN
52.43
50.62
51.56
30.29
72.65
71.93
72.25
66.67
$5.59
$6.15
54.63
48.9
Main results. We compare our GyroBN with previous BN methods under the 1Block GyroGr backbone, which consists of one block of gyrotranslation and pooling layers followed by ProjMap layer. We apply the BN after the pooling layer. The 5-fold results are presented in Tab. 3. GyroBN consistently delivers improved performance, enhancing the average performance of the vanilla GyroGr by 2.92%, 2.47%, and 1.71%. %.. In contrast, Grassmannian ManifoldNorm and RBN could degrade the vanilla GyroGr network, particularly the NTU60 and NTU120 datasets. This is primarily due to GyroBN's theoretical guarantee of normalizing sample statistics, a capability lacking in previous methods such ManifoldNorm and RBN, as summarized in Tab. 1. Additionally, we observe that GyroBN mitigates the performance gap between training and testing, indicating its ability to enhance the model's generalization. Detailed discussions are provided in App. F.5. Overall, the above findings highlight the effectiveness of our GyroBN in facilitating network training.
Ablations on the architecture. We further validate our GyroBN across different architectures within the GyroGr baseline, which includes up to four blocks of gyrotranslation and pooling. GyroBN is applied after the first pooling layer, and we denote GyroGr with GyroBN as GyroGrBN. As implied by Tab. 3, both GyroGr and GyroGrBN exhibit relatively small variances, allowing us to conduct ablations using a single trial. Tab. 4 reports the results across all three datasets. We observe that GyroBN consistently improves the performance of the vanilla GyroGr bascline, highlighting the effectiveness of the GyroBN framework. Notably, as the network depth increases, the performance of the GyroGr backbone, with without GyroBN, degrades. This is because the dimensionality of the

---

Published as a conference paper at ICLR 2025
final feature becomes excessively low, leading underfitting. For instance, in the deepest architecture with four blocks on the HDM05 dataset, the final dimension is 12 x 10, which is insufficient to effectively capture discriminative information. Despite this, our GyroBN still generally enhances the performance of the GyroGr backbone.
Validation AUC on the Disease detaset
Validation AUC on the Airport detaset
Validation AUC on the Pubmed detaset
100
100
133
HNM
HNN
H-Ht
HRN RBNH
HMN-RENIH PRRA-GynoBM4et
HRR--RBhi
V4li44tion AulienrAAcc 190
ww Mag
WWM
Volddvionn.o AUC
Voldationn
70-
30
59
Spazh
Epach
Epoch
Figure 3: Validation AUC of HNN with or without GyroBNH or RBN-H H on the Disease, Airport and Pubmed datasets. The results indicate that GyroBN facilitates the training of HNNs, improving both convergence and performance, and surpasses the previous RBN-H.
7.2 EXPERIMENTS ON THE HYPERBOLIC SPACE
Implementation. Following Lou et al. (2020), we validate our hyperbolic GyroBN, referred to as GyroBN-H, on the Hyperbolic Neural Network (HNN) (Ganea et al., 2018) backbone for the link prediction task using the Cora (Sen et al., 2008), Disease (An-
Table 5: Comparison of HNN with or without GyroBN-H or RBN-H on the link prediction task.
Dataset
HNN
HNN-RBN-H
HNN-GyroBN-H
Cora
0.1
0.5
94.3 # 0.2
derson & May, 1991). Airport (Zhang & Chen, 2018), and Pubmed (Namata et al., 2012) datasets. We also com- pare GyroBN-H with the hyperbolic RBN proposed by Lou et al. (2020, Alg. 2), denoted as RBN-H. Notably, the core difference between
Disease
0.3
76.6
81.2 + 0.9
Airport
90.8 + 0.2
94.2 0.4
95.4 + 0.2
Pubmed
94.9 1..900.11 0.1
93.4 + 0.2
95.8 # 0.1
RBN and our GyroBN lies in the normalization our GyroBN normalize sample statistics across different geometries, while RBN lacks this guarantee. More details can be found in App. F.3.
Results. Tab. 5 reports 5-fold average results of testing AUC on the four datasets. Although RBN and our GyroBN can generally improve the vanilla HNN, our GyroBN shows consistently better improvement than RBN. We also observe that RBN compromises HNN on the Pubmed dataset, while our GyroBN can consistently facilitate network training. Empowered by the normalization of sample statistics, our GyroBN also shows smaller standard deviation than RBN. The above empirical results demonstrate the superiority of our GyroBN framework. Additionally, the validation AUC curves shown in Fig. further highlight the effecctiveness of our GyroBN in facilitating network training.
8 CONCLUSION
We propose a novel gyro structure on manifolds, termed the pseudo-reductive gyrogroup. This structure establishes the minimal prerequisites for developing valid normalization techniques over gyro spaces. Building on this gyro structure, we introduce a novel framework for batch normalization over gyrogroups, called GyroBN, which enables the normalization of -on-Euclidean statistics. Several previous RBN methods, such as LieBN on general Lie groups (including three groups on SPD manifolds and the group of rotation matrices) and various AIM- SPDBN methods, are revealed to be special cases of our GyroBN, Additionally, we manifest GyroBN on the Grassmannian and hyperbolic spaces. To the best of our knowledge, our Grassmannian GyroBN is the first RBN method on the Grassmannian manifold with a theoretical capability to normalize sample statistics. Experiments Grassmannian and hyperbolic networks confirm the effectiveness of GyroBN. As future avenue, we will implement our GyroBN over gyro spaces. We hope that our work will facilitate the development of deep networks for data with non-trivial geometries.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was partly supported by the MUR PNRR project FAIR (PE00000013) funded by the NextGenerationEUU and the EU Horizon project ELIAS (No. 101120237). The authors also gratefully acknowledge the financial support from the China Scholarship Council (CSC), as well as the CINECA award under the ISCRA initiative for the availability of partial HPC resources support.
REFERENCES
Roy M Anderson and Robert M May. Infectious diseases of humans: dynamics and control. Oxford University Press, 1991.
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and simple computations on tensors with log- Euclidean metrics. PhD thesis, INRIA, 2005.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey Hinton. Layer normalization. arXiv preprint 17X1::1607.06450,, 2016.
Ekkehard Batzies, Knut Hüper, Lais Machado, and F Silva Leite. Geometric mean and geodesic regression on Grassmannians. Linear Algebra and its Applications, 466:83-101, 2015.
Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully hyperbolic convolutional neural net- works for computer vision. The Twelfth International Conference on Learning Representations, 2024.
Thomas Bendokat, Ralf Zimmermann, and P-A Absil. A Grassmann manifold handbook: Basic geometry and computational aspects. Advances in Computational Mathematics,, 50(1):1-51,2022 2024.
Daniel Brooks, Olivier Schwander, Frédéric Barbaresco, lean-Yves Schneider, and Matthieu Cord. Riemannian batch normalization for SPD neural networks. In Advances in Neural Information Processing Systems, volume 32, 2019.
James W Cannon, William Floyd, Richard Kenyon, Walter R Parry, et al. Hyperbolic geometry. Flavors of geometry, 31(59-115):2, 1997.
Rudrasis Chakraborty, ManifoldNorm: Extending normalizations Riemannian manifolds. arXiv preprint arXiv:2003. /3869, 2020.
Rudrasis Chakraborty, Jose Bouza, Jonathan Manton, and Baba C Vemuri. Manifoldnet: A deep neural network for manifold-valus data with applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances in Neural Information Processing Systems, 32, 2019.
Kaixuan Chen, Shunyu Liu, Tongtian Zhu, Ji Qiao, Yun Su, Yingjie Tian, Tongya Zheng, Haofei Zhang, Zunlei Feng, Jingwen Ye, et Improving expressivity of GNNs with subgraph-specific fac- tor embedded normalization. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 237-249, 2023a.
Kaixuan Chen, Jie Song, Shunyu Liu, Na Yu, Zunlei Feng, Gengshi Han, and Mingli Song. Dis- tribution knowledge embedding for graph pooling. IEEE Transactions on Knowledge and Data Engineering, 35(8):8898-7900 2023b.
Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, and Josef Kittler. Riemannian local mechanism for SPD neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 7104-71111 12, 2023c.
Ziheng Chen, Yue Song. Gaowen Liu, Ramana Rao Kompella, Xiaojun Wu, and Nicu Sebe. Rie- mannian multinomial logistics regression for SPD neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024a.
Ziheng Chen, Yue Song. Yunmei Liu, and Nicu Sebe. A Lie group approach to Riemannian batch normalization. In The Twelfth International Conference on Learning Representations, 2024b.
11

---

Published as a conference paper at ICLR 2025
Ziheng Chen, Yue Song, Rui Wang, Xiaojun Wu, and Nicu Sebe. RMLR: Extending multinomial logistic regression into general geometries. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024c.
Ziheng Chen, Yue Song, Xiao-Jun Wu, and Nicu Sebe. Product geometries on Cholesky manifolds with applications to SPD manifolds. arXiv preprint urXiv:2407.02660) 2024d.
Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, and Nicu Sebe. Adaptive Bog-Euclidean metrics for SPD matrix learning. IEEE Transactions on Image Processing, 2024e.
Ziheng Chen, Yue Song, Xiaojun Wu, Gaowen Liu, and Nicu Sebe. Understanding matrix function normalizations in covariance pooling from the lens of Riemannian geometry. In The Thirteenth International Conference on Learning Representations. 2025.
Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A network. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 154-163, 2021.
graph convolutional
Manfredo Perdigao Do Carmo and J Flaherty Francis. Riemannian Geometry, volume 6. Springer, 1992.
Alan Edelman, Tomás A Arias, and Steven T Smith. The geometry of algorithms with orthogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2):303-353 1998.
JEAN QUAINTANCE Gallier and Jocelyn Quaintance. Differential geometry and Lie groups, volume 12. Springer, 2020.
Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. Advances in Neural Information Processing Systems, 31, 2018.
Zhi Gao, Chen Xu, Feng Li, Yunde Jia, Mehrtash Harandi, and Yuwei Wu. Exploring data geometry for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14325-24334 2023.
Uwe Helmke and John B Moore. Optimization and dynamical systems. Springer Science & Business Media, 2012.
Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Zhiwu Huang and Lue Van Gool. A Riemannian network for SPD matrix learning. In Thirty-first AAAI Conference on Artificial Intelligence, 2017.
Zhiwu Huang, Chengde Wan, Thomas Probst, and Lue Van Gool. Deep learning Lie groups for Akeleton-based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6099-6108 2017.
Zhiwu Huang, Jiqing Wu, and Lue Van Gool. Building deep networks on Grassmann manifolds, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Zhiwu Huang, Jiqing Wu, and Lue Van Gool. Manifold- valueed image generation with Wasserstein generative adversarial nets. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3886-3893, 2019.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. International Conference on Machine Learning, pp. 448-456 PMLR, 2015.
Ce Ju, Reinmar Kobler, Liyao Tang, Cuntai Guan, and Motoaki Kawanabe. Deep geodesic canonical correlation analysis for covariance-basee neuroimaging data. In The Twelfth International Conference on Learning Representations, 2024.
Hermann Karcher. Riemannian center mass and mollifier smoothing. Communications on Pure and Applied Mathematics, 30(5):509-541, 1977.
12

---

Published as a conference paper at ICLR 2025
Isay Katsman, Eric Chen, Sidhanth Holalkere, Anna Asch, Aaron Lou, Ser Nam Lim, and Christo- pher M De Sa. Riemannian residual neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
Reinmar Kobler, Jun-ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD domain-specific batch normalization crack interpretable unsupervised domain adaptation in EEG, Advances in Neural Information Processing Systems, 35:6219-6235, 2022a.
Reinmar J Kobler, Jun-ichiro Hirayama, and Motoaki Kawanabe. Controlling the Fréchet variance improves batch normalization on the symmetric positive definite manifold. In ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3863-3861 IEEE, 2022b.
John M Lee. Introduction to smooth manifolds. Springer, 2013.
John M Lee. Introduction to Riemannian manifolds, volume 2. Springer, 2018.
Mario Lezcano Casado. Trivializations for gradient- optimization on manifolds. Advances in Neural Information Processing Systems, 32, 2019.
Zhenhua Lin. Riemannian geometry of symmetric positive definite matrices via Cholesky decompo- sition. SIAM Journal on Matrix Analysis and Applications, 40(4):1353-) 2019.
Jun Liu, Amir Shahroudy, Mauricio Perez, Gang Wang, Ling-Yu Duan, and Alex C Kot. NTU RGB+D 120: A large-scale benchmark for 3D human activity understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10))2644-270. 2019.
Aaron Lou, Isay Katsman, Qingxuan Jiang, Serge Belongie, Ser-Nam Lim, and Christopher De Sa. Differentiating through the Fréchet mean. In International Conference on Machine Learning, pp- 6393-6403. PMLR, 2020.
Meinard Müller, Tido Röder, Michael Clausen, Bernhard Eberhardt, Björn Krüger, and Andreas Weber. Documentation mocap database HDM05. Technical report, Universität Bonn, 2007.
Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, volume 8, pp. 1, 2012.
Xuan Son Nguyen. The Gyro-structun of some matrix manifolds. In Advances in Neural Information Processing Systems, volume 35. pp. 26618 1--188--,,, 2022a.
Xuan Son Nguyen. A Gyrovector space approach for symmetric positive semi-definite matrix learning. In Proceedings of the European Conference on Computer Vision, pp. 52 28, 2022b.
Xuan Son Nguyen and Shuo Yang. Building neural networks on matrix manifolds: A gyrovector space approach. arXiv preprint arXiv:2305.04550 2023.
Xuan Son Nguyen, Shuo Yang, and Aymeric Histace. Matrix manifold neural networks++. In The Twelfth International Conference on Learning Representations, 2024.
Yue-Ting Pan, Jin Chou, and Chun-Shu Wei. Matt: manifold attention network for EEG decoding. Advances in Neural Information Processing Systems, 35:3 35:31116-3129) 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, hig-pperformanc deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1 11:41-666 66, 2006.
Peter Petersen. Riemannian geometry, volume 171. Springer, 2006.
13

---

Published as a conference paper at ICLR 2025
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Rriassi-Rad Collective classification in network data. Al magazine, 29(3):93-92) 93, 2008.
Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+D: A large scale dataset for 3D human activity analysis. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 1010-1019, 2016.
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. arXiv preprint arXiv:2006.08221 082/0, 2020.
Ondrej Skopek, coders. arXiv preprint 1rXiv:/911 1e11//111.011111 2019.
Octavian- Eugen Ganea, and Gary Bécigneul. Mixed-curvature variational autoen-
Yann Thanwerdas and Xavier Pennec. Theoretically and computationally convenient geometries on full-rank correlation matrices. SIAM Journal on Matrix Analysis and Applications, 43(4): 1851-1872, 2022.
Loring W. Tu. An introduction to manifolds. Springer, 2011.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv: 2016.
Abraham A Ungar. Analytic hyperbolic geometry: Mathematical foundations and applications. World Scientific, 2005.
Abraham A Ungar. Beyond the Einstein addition law and its gyroscopic Thomas precession: The theory of gyrogroups and gyrovector spaces. Springer Science & Business Media, 2012.
Abraham Albert Ungar. A Gyrovector Spuce Approuch to Hyperbolic Geomelry. Springer, 2009.
Abraham Albert Ungar. Analytic hyperbolic geometry in n dimensions: An introduction. CRC Press, 2014.
Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. DreamNet: A deep Rie- mannian manifold network for SPD matrix learning. In Proceedings of the Asian Conference on Computer Vision, pp. 3241-3257 -3257, 2022a.
Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. Learning a discriminative SPD manifold neural network for image set classification. Neural Networks, 151:94-110, 2022b.
Rui Wang, Chen Hu, Ziheng Chen, Xiao-Jun Wu, and Xiaoning Song. A Grassmannian manifold self-attention network for signal classification. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 2024a.
Rui Wang, Xiao-Jun Wu, Ziheng Chen. Cong Hu, and Josef Kittler. SPD manifold deep metric learning for image set classification. IEEE Transactions on Neural Networks and Learning Systems, 2024b.
Yung-Cue Choo- Wong. Differential geometry of Grassmann manifolds. Proceedings of the National Academy Sciences. 57(3):589-5555 1967.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European Conference Computer Vision (ECCV), pp. 3-19, 2018.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. In Advances in neural information processing systems, volume 31, 2018.
Wei Zhao, Federico Lopez, J Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, and Steve Trettel. Modeling graphs beyond hyperbolic: Graph neural networks in symmetric positive definite matrices. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 139- Springer, 2023.
Vladimir Antonovich Zorich and Octavio Paniagua. Mathematical analysis 11, volume 220. Springer, 2016.
14

---

Published as a conference paper at ICLR 2025
APPENDIX CONTENTS
A Limitation
17
B Notations
17
C Preliminaries
18
C.I Riemannian geometry
18
C.2 Gyrovector spaces
19
C.3 Matrix and vector manifolds
19
C.3.1 Definitions
19
C.3.2 Gyro and Riemannian structures
20
C,4 Intuitive explanations of gyrogroups
20
First pseudo-reductive gyrogroups properties
Grassmannian GyroBN under the projector perspective
22
Experimental details and additional ablations
24
F11
Summary of operators in Grassmannian and hyperbolic GyroBNs
24
F.2 Details on the Grassmannian experiments
25
F2.1
Datasets and preprocessing.
25
F.2.2
Implementation details
26
F3
Details on the hyperbolic experiments
26
F3.1
Datasets
26
E.3.2
Implementation details
26
E4
Hardware
26
F.5
GyroBN and model generalization ability
26
F66
Training efficiency
27
F66.1
Training efficiency on the Grassmannian networks
27
F.6.2
Training efficiency on the hyperbolic networks
27
F.7
Experiments on other hyperbolic baselines
28
E7.1
Experiments on HNN+
28
F.7.2
Experiments RResNet
28
................................
F88
Experiments on covariate shifts
29
F8.11
Covariate shifts on the Grassmannian network
29
F.8.2
Covariate shifts the hyperbolic network
29
F.9
Experiments on condition numbers
29
F9.1
Condition numbers of weight matrices of the transformation layer
29
R9.2
Condition number of network Jacobian
30
15

---

Published as a conference paper at ICLR 2025
G Proofs
30
G.1 Proof of Prop. 3.2
30
G.2 Proof of Thm. 3.3
31
G.3 Proof of Thm. 3.5
32
G.4 Proof of Prop. 3.6
32
G.5 Proof of Thm. 4.1
35
G.6 Proof of Prop. 5.1
36
G.7 Proof of Prop. 6.1
37
16

---

Published as a conference paper at ICLR 2025
A LIMITATION
As shown in Prop. 3.6, several geometries, including SPD, Grassmannian, hyperbolic, and hyperpher- ical manifolds, have gyro structures that can enable GyroBN with a theoretical control over sample statistics. However, some geometries do not have gyro structures, or their gyro structures are still untouched. Therefore, GyroBN cannot be established on these manifolds. We will explore other techniques to establish an RBN framework for these non-gyro geometries.
B NOTATIONS
Tab. 6 summarizes all the notations in the main paper.
Table 6: Summary of notations.
Notation
Explanation
{G, #) or abbreviated as (M, (.9) 9) or ubbreviated as
A gyrorroup
Riemannian manifold (M,g) g) with a gyrogroup structure induced by g
identity map
TpM
Tangent space
.M
( ) or )
Riemannian metric at P C M
The norm induced by ( on T
dgea (1)))
Geodesic distance
Loge Expp
Riemannian logarithm at P
Riemannian exponentiation
The differential map of the smooth map / at P .M
PT
Purallel transportation along the geodesic connecting I' and Q Gyro identity of [M, @)
ep
Group inverse of e: M
Gyro scalar product
gyr
Gyration Gyro inner product
Gyronorm
dey (1)
Gyrodistance
FM
Fréchet under gyrodistance
Bara(y)
Binary barycenter hased gyrodistance
The standard Frobenius inner product
Norm induced by the standard Frobenius inner product
mlog
Matrix logarithm
mexp
Matrix exponentiation
Cholesky decomposition
Dlog
The diagonal element-wise logarithm
Dlog o.
S
The SPD manifold
S"
The Euclidean space of symmetric matrices
PR
n-dimensional Poincaré with curvature K <
g"
n--dimensional Euclidean space
Dg.
n-dimensional projected hypersphere with curvature K 0
Mr
Constant Curvature Spaces (CCS)
Gr(ynn) n)
The Grassmannian under the ONB perspective
Gr(p.n)
The Grassmannian under the projector perspective
The Grassmannian identity under the ONB perspective
The Grassmannian identity under the projector perspective
The Riemannian isometry from Gr(p, n)) onto Gr(p.w)
Matrix commutator
An element in Gr (p. n),, which equivalent class n identity matrix
pe
Matrix power SPD matrix
DA Le and ote
Gyro additions on the manifold under AIM, LEM and LCM
T Gr
and Cr
Gyro additions on the Grassmannian under the ONB and projector perspectives Gyro additions on the CCS
@x
()
(-) Logr (-) with Log as Riemannian logarithm on Cr(p.n) n)
9
and g
AIM. LEM, LCM the SPD manifold
or and Grr tang
Riemannian metries the Grassmannian under the ONB and projector perspectives
tank = tan if k 0, elif K > 0, tang = tanh
17

---

Published as a conference paper at ICLR 2025
C PRELIMINARIES
C.I RIEMANNIAN GEOMETRY
Manifolds can be intuitively understood as locally Euclidean spaces. Differentials generalize the concept of classical derivatives. For a detailed introduction to smooth manifolds, we refer readers to Tu (2011); Lee (2013). Riemannian manifold is a manifold equipped with a Riemannian metric, which can be intuitively interpreted as a point-wise inner product. This metric allows for the adaptation of various Euclidean operators to the manifold setting. For an in-depth discussion on Riemannian manifolds, see Do Carmo & Flaherty Francis (1992); Lee (2018).
Definition C.1 (Riemannian Manifolds (Do Carmo & Flaherty Francis, 1992)). Riemannian metric on M is smooth symmetric covariant 2-tensor field M, which is positive definite at every point. A Riemannian manifold is a pair {M,g) g). where M is a smooth manifold and is a Riemannian metric.
The isometries generalize the bijection in the set theory into the Riemannian geometry. two manifolds are isometric, they can be viewed equivalent. The Riemannian operators in these two manifolds are also closely related (Chen et al., 2024d, App. A. 2). The following defines the Riemannian isometry.
Definition C.2 (Isometries (Lee, 2018)). (M,g) and (M,g) are both Riemannian manifolds, a smooth map = + .M is called a (Riemannian) isometry if it is diffcomorphism that satisfies
gp (V, W)
(24)
where (-) T.M + M is the differential map of f at p E M, and V, W Ty.. are two tangent vectors.
The exponential & logarithmic maps and parallel transportation are also crucial for Riemannian approaches machine learning. To bypass the notation burdens caused by their definitions, we review the geometric reinterpretations of these operators (Pennec et al., 2006; Do Carmo & Flaherty Francis, 1992). In detail, in a manifold M. geodesics correspond to straight lines in Euclidean space. A tangent vector zy E T&M can be locally identified to point y the manifold by geodesic starting at with an initial velocity of Ty.. i.e. y = Exp,, (zg)- - On the other hand, the logarithmic map is the inverse of the exponential map, generating the initial velocity of the geodesic connecting x and y, Le.rTy = (o). These two operators generalize the idea of addition and subtraction Euclidean space. For the parallel transportation PT it a generalization of parallelly moving a vector along a curve in Euclidean space, we summarize the reinterpretation in Tab. 7.
Table 7: The geometric reinterpretations of Riemannian operators.
Operations
Euclidean spaces
Riemannian manifolds
Straight line Subtraction Addition Parallelly moving
Straight line Ty y x
Geodesic
F = T Log
T7 - V
y = Exp, (x
PT PT
A Lie group is manifold with smooth group structure. It is combination of algebra and geometry. Definition C.3 (Lie Groups). A manifold is Lie group, if it forms a group with group operation @ such that m( y) x @ y and i(x:) z are both smooth, where x: is the group inverse of z.
The following are some naive examples of the Lie group:
1. The set of real numbers R, whose group operation is the addition.
2. The set of n x invertible matrices GL(n), whose group operation is the matrix product. This group is known as the general linear group.
3. The set of n. X orthogonal matrices O(n), whose group operation is the matrix product. This group is subgroup of GL(n), known the orthogonal group.
18

---

Published as a conference paper at ICLR 2025
C.2 GYROVECTOR SPACES
Gyrogroups in Def. 2.1 generalize groups to non-associative algebraic systems by gyrations. Similarly, the gyrovector space generalizes the vector space, which has shown impressive success in hyperbolic geometry (Ungar, 2005; 2009; 2012; 2014).
Definition C.4 (Gyrovector Spaces (Nguyen, 2022a)). A gyrocommutative gyrogroup {G, @} equipped with scalar multiplication : G G is called gyrovector space if it satisfies the following axioms for 8, I E and a, E G:
and (-1) @ Ga.
(2)
(s=)))))==)
(V4) gv(a,,(() gyrla, tjc.
gv) where is the identity map.
Gyrovector spaces generalize vector spaces to curved geometries, such as the SPD and Grassman- nian manifolds (Nguyen, 2022a). While retaining familiar properties like distributivity (V2) and associativity (V3), they incorporate the complexities of gyrations.
C.3 MATRIX AND VECTOR MANIFOLDS
C.3.1 DEFINITIONS
SPD:
The set S of n x n SPD matrices form a manifold, named the SPD manifold (Pennec et al., 2006). We focus three popular Riemannian metrics the SPD manifold: Invar--Ivvarant Metric
(AIM) (Pennec et al., 2006), Log-Euclidean Metric (LEM) (Arsigny et al., 2005), and Jog-Cholesky Metric (LCM) (Lin, 2019).
Grassmannian: The Grassmannian is the set p-dimensional subspace of n--dimensional vector space (Tu, 2011), which has two matrix representations (Bendokat et al., 2024):
Projector perspective: Gr(p, n) (P S" p2 =P Prrakn((P) p):
(25)
ONB perspective: Gr(p,n) {[V] [U] = (Ū St(p, n) Ü -UR
where S is the Euclidean space of symmetric matrices, St(p, n) is the Stiefel manifold, and O(p) is the orthogonal group. By abuse of notations, we use [V] and U interchangeably for the element of Gr(p, n). As shown by Helmke & Moore (2012), the ONB perspective Gr(p, n) is diffeomorphism Gr(p.n) by
T(U) = UUT, YU e Gr(p.n),
(26)
where the n x column-wise orthonormal matrix U should be more precisely understood as a representative of equivalence class (Bendokat et al., 2024).
The Poincaré ball PR for the hyperbolic space (Ungar, 2009; Ganea et al., 2018), projected hypersphere DR for the hypersphere (Skopek et al., 2019), and standard Euclidean space IR" (Zorich & Paniagua, 2016) are more generally called the Constant Curvature Space (CCS) (Do Carmo & Flaherty Francis, 1992), as they have constant sectional curvature K. The n-dimensional Poincaré ball and projected hypersphere are represented as Pe = {x E R (I. xr) < - ++) ) with < 0 and D& R with K > 0. When K the CCS becomes the standard Fuclidean space R..
CCS:
19

---

Published as a conference paper at ICLR 2025
C.3.2 GYRO AND RIEMANNIAN STRUCTURES
For matrix manifold M and CCS MM. we make the following notations:
{ } (The SPD manifold under AIM)
(S7+.ghF) (The SPD manifold under LEM)
(M,g)
{ (The SPD manifold under LEM)
(27)
{Gr(p. (r) (The Grassmannian under the ONB perspective)
(Gr(p, ] (The Grassmannian under the projector perspective)
Pn for K < 0
MK =
TR" for K = 0 DDk
(28)
for K > 0 if K > 0
tan
tank
(29)
tanh
if K <0
Notations in Tab. 2: We summarize all the necessary group operations in Tab. 2 with the following notations. Given any P.Q E M with M as sh Gr(p, n)), and x E Mx with Mx as P < 0). D (K > 0) or R.(K 0), we make the following notations. For the Grassmannian, U= (P) and V x (Q) are the ONB representations. We denote matrix exponential, matrix logarithm, and Cholesky decomposition as mexp, mlog, and L, respectively. We denote L/c Dlog o,Y, where $L is the diagonal logarithm. As shown by Chen et al. (2024e), LCM and the associated Lie group on S are pulled back by dLC for the Euclidean space of lower triangular matrices. 7 is the n X identity matrix, Ppnn (I).0) TRXx,, and Tp.n * pp...) For the Grassmannian Gr p). Ω [p.l, where P = Log (P) and -] is the matrix commutator. We denote (.-) and
the standard (matrix & vector) inner product and norm.
We further make the following notations for the related Riemannian operators in Tab. 8. Given P M (x E Mg), we denote the tangent vector V TpM (v Ty.M)). We denote the geodesic distance, Riemannian logarithm, and Riemannian exponential as dgeo, Log and Exp, respectively.
Table Riemannian geometries of several matrix and vector manifolds. For CCS MM. we present the operators for K # 0, as when K = 0, all the Riemannian operators are reduced to the familiar vector operators.
Manitols
F..) deall.))
Logo Q Loz.y
Expy V x Exg,
Reforences
(5
ImingiP) alosi(i)
(mkk,,)) (miog(() alopiP))
mopp (ulopiP) -(V)
Ousigny 2005)
(s"")
Inlos 1
Phake top--)r--e)) f)
uso (y-b -)
(87t..
l0he(P) Sucd
(Lis,
Grüve)
pig 900 OCRT
U. PPTIQU"" qj 9N oser
[PRee(gH"= (as5gn""--imnn -0mm)001 V 30 ose"
(ddelass al., 1998) Chorkkkkt 2016)
Grige)
Imlogg
10)), aPM
Vinlog(t,, 201(1 2PA.
(Beudoka 2024)
2011)
Do Canto & Flakeny Francis, 1992
My
Qi wing (v ([l--
Virryy tong! (vin) vI)
arew tame (viw Al0e)
(Prinnss, 2000)
(Slopek zt 2019)
Remark C.5 (The Grassmannian and cut locus). Due to the cut locus of the Grassmannian, the logarithm map does not exist globally (Bendokat et al., 2024). In this paper, when we use (og implicitly assume P and are not in each other's 's cut locus. Besides, more
on the Grassmannian, precisely, the gyro addition and scalar multiplication on Grassmannian are also not globally defined (Nguyen, 2022a), due to the cut locus. Following Nguyen (2022a) and Nguyen & Yang (2023), implicitly assume the gyro operations are well-defined on the Grassmannian.
C.4 INTUITIVE EXPLANATIONS OF GYROGROUPS
This subsection intuitively explains gyrogroups by contrast with the trivial IR. Gyrogroups are proposed to generalize the concept of addition in R or R to non-Euclidean spaces.
Gyrogroups in Def. 2.1 extend the concept of groups to non non-associative algebraic systems, i.e.,a + c) # (a b) The gyrogroups over the manifolds in Tab. 2 are defined by Eqs. (3), (5) and (6).
20

---

Published as a conference paper at ICLR 2025
More importantly, these definitions natural generalizations of Euclidean vector operations. Take the gyro addition Eq. (3) as an example. When the manifold M is the Euclidean space R", the gyro addition defined by Eq. (3) becomes the familiar vector addition.
G + G for each Gyration & gyroassociativity: The gyration is an automorphism gyr(a, 6] a, E G. is called an automorphism as it can preserve the gyro addition:
gyrla, (e d) gyr(a, (](c) gyrla, Ve
(30)
The gyration is used generalize the associativity:
+ + (a 6)+ ba,b,ceR.
(31)
Take the Grassmannian gyrogroup (Gr(p, @ Grr an example. For any U, V, R @
we have
Non-associativity:
(U + (V Gr U) R.
(32)
Left gyroassociativity: V Gr (U + Gr R) = (V e Gr U) @r gyr (R
(33)
Eq. (33) differs with Eq. (31) only gyration. The concrete expression of the gyration over the Grassmannian is presented by Nguyen (2022a, Def. 3.18).
Left reduction is defined as gyr(a, bj = gyrla @ b, b] for any and in the gyrogroup G. It called "left reduction" because b in b can be canceled out.
Gyrocommutativity in Def. 2.2 generalizes the commutative property (a+ b = b a) by the gyration operator gyr(a, b).
Nonreductive gyrogroups in Def. 2.3 are relaxed gyrogroups, allowing structures where the left reduction law (G4) does not hold. Its prototype comes from the gyrogroup of the Grassmannian (Nguyen, 2022a, Thm. 3.20), where (G4) does not hold.
Relation between left reduction, non-reduction, & pseudo-reduction: The left reduction can induce many basic properties of gyrogroups ( see Thm. D.I and Ungar (2009, Thms. 1.13 and 1.14)). However, non-reduction does not guarantee several basic properties. This drawback will undermine the rationality the non-reductive gyrogroups. Therefore, as an intermediate, we propose reductive gyrogroups maintain most of the basic properties
pseudo-reduction. The associated of gyrogroups. Please refer to Thm. D.I and its remark for more details.
D FIRST PSEUDO-REDUCTIVV GYROGROUPS PROPERTIES
Theorem D.1 (First Iseudd-reductto Gyrogroups Properties). Let {G, (D be pseudo-reductive gyrogroup. For any elements P, Q, R, X G. have:
1. P @ P R. then Q R (General Left Cancellation law; see (9) below).
2. gyr E, Pl = for any left identity E in G.
3. gyr[X, P] for any left inverse X of P in G.
4. There is P left identity which is P right identity.
5. There is only one left identity.
6. Every left inverse is P right inverse.
7. There is only one left inverse, OP, of and ((PP) = P.
8. The left cancellation law: e ( =
9. The gyrator identity: syr[P.Q]X (P
10. gy Q]E = E.
11. P. (ex)) g
12. E)
13. The gyrosum inversion law: (( =
21

---

Published as a conference paper at ICLR 2025
Credit of the proof. Most of the proof is borrowed from Ungar (2009, Thms. 1.13 and 1.14), which presents the corresponding properties for gyrogroups. After re-analyzing these two theorems, we find that most of the proof can be readily extended into pscudo-reductive gyrogroups.
Proof. This theorem follows Ungar (2009, Thms. 1.13 and 1.14), which presents some useful properties for gyrogroups. We argue that all the properties except gyrla, aj = 1 are independent of the left reduction law (G4), and are therefore satisfied on pseudo-reductive gyrogroups. All the properties can be proven in the same way as the ones for Thms. 1.13 and 1. 14 by Ungar (2009). We summarize the logic in the following:
left gyroassociativity
left gyroassociatiiity + 2
definition 3
left gyroassociativity + +1 1+33 4
definition 5
left gyroassociativity + 1111+++44 +1++++++++++
1+6
left gyroassociativity +3 + 8
left gyroassociativity left cancellation in 8
gyro identity + 10
10 11
left cancellation in 8 + gyro identity in 9 + 12
left cancellation in 8 gyro identity in 9 13
Remark D.2. For non-reductive gyrogroups, 2 and agnostic. Therefore, every property based on or 3 is not guaranteed, such as 4, and from 6 to 10. The missing of these basic properties will undermine the rationality of non-reduetive gyrogroups. In contrast, most basic properties of gyrogroups are preserved in our pseudo-reductive gyrogroups.
E GRASSMANNIAN GYROBN UNDER THE PROJECTOR PERSPECTIVE
Given two isometric manifolds {M1,g*) and {M2, g²), the induced gyro structures in Eq. (3)- Eq. (9) have the following relations. We first present useful lemma.
Lemma E.1. Given manifolds (M1,g') and (M2.g g²) and Riemannian isometry f MM M2. we have the following:
1. The groupoid (M1, (11) induced by g is reductvee (left-invariant), iff the groupoid {M2, Φ2) induced by g² is pseudo-reductive (left-invariant)
2. Any gyration in {Migg²} preserves gyronorm any gyration in (M2,g92) 92} preserves gyronorm;
3. J preserves the gyrodistance.
Proof. First, we review some facts about gyrogroups under the Riemannian isometry. As shown by Nguyen & Yang (2023, Thm. 2.5), (M1, $1 satisfies (G1-G3) in Def. 2.1 iff {M2, B2) satisfies (GI-G3). Besides, f-1 is an isomorphism satisfying
f I 02 Q) = (P) ( P) = t (P),,
(34)
(35)
(e, = 0, (P)
(36)
(37)
22

---

Published as a conference paper at ICLR 2025
where P. P,Q, R E M2 arbitrary points, E R is real scalar, G₁ gyT, the gyro inverses and gyrations on M, for = 1, f is also an isomorphism with similar properties.
We only need to prove direction for the iff condition for the pseudo-reduction norm invariance. We focus on = and follow the above notations the following.
Pseudo-reduction:
gyT. PP. f gyr ( (Eq. (37))
 (osr: (Eq. (36))
(38)
=
L.
Gyronorm invariance under gyrations: For simplicity, we denote the gyronorm, identity element, and Riemannian logarithm on M, as I|II, E,, and Log', respectively, First, the following demonstrates that the Riemannian isometry f preserves gyronorm:
PPll Loge, (P)ll
@
Log' (Ea) (f (F-(()))| (-(
(39)
2 Logg, (P))l))
[P)]
(1) As f Mi + M2 is Riemannian isometry, we have the following equations:
Logp Q = (-.)-( (pog)- (Q)) &
(40)
(V, W) = -f=_)p(((==))_)0)))=))))))
W.W.& e TpM2.
(41)
where (-), is the differential map:
(2) Eg = f(Ei).
Then have the following:
[
|f ( (=,01
(42)
(3)
(-))]
|
The above derivation comes from the following.
(1) Eq. (37);
(2) Eq. (39);
(3) Any gyration on MM can preserve the gyronorm;
(4) Eq. (39).
Invariance of gyrodistance under f: Denoting the gyrodistance on M, as d, for any U, e Mi, we have the following:
di (U, = | @ VIl
[f(e, 0 Q)l,
(43)
|0if(P) 02 f(Q)l1, d(()
The above derivation comes from the following.
(1) f preserves gyronorm:
23

---

Published as a conference paper at ICLR 2025
(2) f is an isomorphism.
Given batch of activations (Pi...v) } on a gyrogroup (M, @), we denote the GyroBN as Gyro B, 6,
(44)
where B E M and are biasing and scaling parameters, a small positive value, and 17 is the momentum.
Theorem E.2. Given manifolds (Mi.g") and (M2,g") g²) and Riemannian isometry f: MM M2. for batch of activation {PI...)) y in M1, GyroBN (P; B, c,7,7,) in M1 can be calculated as GyroBN (P;; 8,8,6, 6, 7) = f ((B)
(45)
where GyroBN, is the GyroBN in M2-
Proof. This theorem is inspired by Chen et al. (2024b, thm. 5.3), which characterizes the LieBNs under isometric manifolds. As gyrogroups are natural generalizations of Lie groups, our GyroBN is expected to have similar results. The following proof follows similar logic to the one by Chen et al. (2024b, thm. 5.3), except that all operations are gyro operations.
For i = 2, we denote Eq. (15) and binary gyro barycenter on M, as {'())M, v², B,s) s) and (. ()
Let = (P....) and S(B) [S(PI...v)) We only need to show the following: M2 f(Mi), 01 U2
(46)
('PP, 2 B.s) f- ( v² ,f()),)),
(47)
Bar, = s (Ba))
VP,Q Mi.
(48)
where M; and v; are the batch Fréchet mean and variance over M; for 1, Eqs. (46) and (48)
can be directly obtained by the invariance of gyrodistance under f (Lem. E.I), We only need to show Eq. (47).
We have the following:
(J(B) 02 (t02 (02 32 f(P)))
@ r 01 (1o 01 P.)))
(49)
B 6 (t (9:Me P.)
<' (P, M, v, B,s),
where Vo=Te" The above derivation comes from the following.
(1) f is an isomorphism preserving gyro operations.
As -- Gr(p, n) - Gr(p. n) is Riemannian isometry, Thm. E.2 indicates that the GyroBN under the projector perspective can be calculated by the ONB perspective by the following process:
1. mapping data into the ONB perspective by -- Gr(p, Gr(p,n):
2. normalizing data by the GyroBN under Gr(p,n);
3. mapping normalized data back to Gr n) by
Besides, both Lem. E. or Prop. 3.6 can guarantee theoretical control over the gyromean and gyrovariance under the projector perspective.
F EXPERIMENTAL DETAILS AND ADDITIONAL ABLATIONS
F.1 SUMMARY OF OPERATORS IN GRASSMANNIAN AND HYPERBOLIC GYROBNs
The specific implementation of Alg. 1 on gyrogroup can be carried out plug-in manner. This involves simply substituting the required operators from Tabs. 2 and into Alg. 1. To streamline this process, we summarize the discussion Sec. in Tab. 9, where we present all the required operators for the Grassmannian and hyperbolic GyroBNs.
24

---

Published as a conference paper at ICLR 2025
Table 9: Key operators in calculating GyroBN on the Grassmannian and hyperbolic manifolds. Here P. Q E Gr(p, n.) are two ONB Grassmannian points, while x; E Pk are two Poincaré vectors. Other notations follow from Tabs. 2 and 8.
Operator
Gr(p,n)
Pkk O R
Identity element P g Q or DK M g p or OKT
1- 11-2((zgp livil" J [1+K1+11=11
mV
1 2x (2.3)) K - Bill
mexp( pp.
t t Port Ox
(txp(ttt)pp..
tanh (ttanh (v K 1
'
Tall
Bar
(O(O(QQ (Q. P) Bar (v) Fréchet Mean
Expp (7 Logs" (Q))
xr tx (- dxx 8) Ox t
Karcher Flow (Karcher, 1977)
(Lou et al., 2020. Alg. 1)
NTU60 Training
NTU60 Testing
GyroGr Test GymGraN - Tast
GyroGr Train
GyroGrBN Train
50
100 Epoch
150
50
100 Epoch
150
NTU120 Training
NTU120 Testing
GyroGr-
GyroGrBN Test
90
55-
70
45
60
GyroGr Train
GyroGrBN- Train
50
40
Epoch
Epoch
Figure 4: Training and testing performance on the NTU datasets of Block GyroGr. Our BN improves the generalization abilities of GyroGr.
F.2 DETAILS ON THE GRASSMANNIAN EXPERIMENTS
F.2.1 DATASETS AND PREPROCESSING
HDM05 (Müller al., 2007). It consists of 2,273 skeleton-based motion capture sequences executed by different actors. Each frame consists of 3D coordinates of 31 joints. We remove the under-represented clips, trimming the dataset down 2086 instances scattered throughout 117 classes. Following Nguyen & Yang (2023), we model each sequence 93 x 10 Grassmannian matrix.
NTU60 (Shahroudy et al., 2016). It has 56,880 sequences of 3D skeleton data classified into 60 classes, where each frame contains the 3D coordinates of 25 or 50 body joints. We use mutual actions and follow the cross-view protocol (Shahroudy et al., 2016). Following Nguyen & Yang (2023), model each sequence as a 150 X 10 Grassmannian matrix.
2 (Liu et al., 2019). This dataset contains 1 114,480 sequences in 120 action classes. We use mutual actions and follow the cross-setup protocol (Liu et al., 2019). Following Nguyen & Yang (2023), model each sequence 150 x 10 Grassmannian matrix.
25

---

Published as a conference paper at ICLR 2025
F.2.2 IMPLEMENTATION DETAILS
Architectures. The IBlock GyroGr is structured as: gyrotranslation pooling - [GyroBN] + ProjMap - classification. The KBlock GyroGr consists of K blocks of gyrotranslation and pooling, followed by final MrojMap layer for classification. Since each pooling operation halves the dimensionality, we omit pooling in the last block when K >> 1. Following Huang et al. (2018), the number of channels is set to 8. The momentum for GyroBN the default value of 0.1. Additionally, in line with Nguyen & Yang (2023), we use the Cayley transform to approximate matrix exponentiation over skew-symmetric matrices.
Optimization. Following Nguyen & Yang (2023), adopt the trivialization tricks (Lezcano Casado, 2019) for the Grassmannian parameters the gyrotranslation and our GyroBN layers. Given (x))))
parameter E Gr(p, n), we parameterize by a matrix R (00-p)xx xP such that
0
11
UUT p.a
(50)
11
The biasing parameter U can be retrieved by
= mcxp [TUT,
Ip.n mexp (1
-11T 0
(51)
In this way, all parameters lie in Euclidean spaces. We, therefore, can directly use the optimizers in Pytorch (Paszke al., 2019). In all experiments, we use an SGD optimizer with a learning rate of 5e-2 and zero weight decay. The batch size is 30, and training epochs are 400, 200, and 200 for the HDM05, NTU60, and NTU120 datasets.
F.3 DETAILS ON THE HYPERBOLIC EXPERIMENTS
F.3.1 DATASETS
Cora (Sen et al., 2008). It a citation network where nodes represent scientific papers in the area of machine learning, edges are citations between them, and node labels academic (sub)areas.
Disease (Anderson & May, 1991). represents a disease propagation tree, simulating the SIR disease transmission model, with each node representing either an infection non-infection state.
Airport (Zhang Chen, 2018). It is a transductive dataset where nodes represent airports and edges represent the airline routes as from OpenFlights.orgg
Pubmed (Namata et al., 2012). This is a standard benchmark describing citation networks where nodes represent scientific papers in the area of medicine, edges are citations between them, and node labels are academic (sub)areas.
F.3.2 IMPLEMENTATION DETAILS
We use the official implementations HGCN' (Chami et al., 2019) and Lou et al. (2020)6 to conduct the experiments. We follow all the settings as Lou et al. (2020, Sec. H.1) to implement our experiments. Specifically, the baseline encoder is an HNN (Ganea et al., 2018) with two layers: the first maps the input feature dimension to 128, and the second maps 128 to 128. Hyperbolic BN layers, such as GyroBN or RBN, follow each layer. We use the Adam optimizer (Kingma, 2014), with a learning rate of -2 and weight decay of 1 except for the Cora dataset, where the weight decay is set to
F.4 HARDWARE
All experiments except the ones on the Pubmed dataset use an Intel Core 19-7960X CPU with 32GB RAM and an NVIDIA GeForce RTX 2080 Ti GPU, The experiments on the Pubmed dataset are conducted on a single NVIDIA Quadro RTX A6000 48GB GPU.
F,5 GYROBN AND MODEL GENERALIZATION ABILITY
We denoted GyroGr with GyroBN GyroGrBN. The training and testing curves the NTU datasets are presented in Fig. 4. We can observe that the gap between training and testing performance much lower when GyroGr is endowed with the Grassmannian GyroBN. This phenomenon indicates that our GyroBN can improve the generalization ability of the Grassmannian neural networks.
'tttps: : / thub. htt.tt/....o//tyteeeeeeecc/////e 'ttpps: / //gi thub. comb /D1 /ttpm//ccc////fffeeeettt
Wwwchet Mean
26

---

Published as a conference paper at ICLR 2025
F.6 TRAINING EFFICIENCY
F.6.1 TRAINING EFFICIENCY ON THE GRASSMANNIAN NETWORKS
Table 10: Average training time (seconds/epoch) for GyroGr with and without different Grassmannian
BNs. The dimensions of the input Grassmannian matrices for the BN layer are also reported.
HDM05 (47 x 10)
NTU60 (75 10)
NTU120 (75 10)
Methods
GyroGr Mynofr-ManifoldNom
2.19
50.92
80.72
4.98
242.12
409.48
GyroGr-RBN
5.16
242.63
410.08
GyroGr-GyroBN 3.10
59.55
108.92
As the Riemannian computations over the ONB Grassmannian Gr(p, n)) involve matrix decomposi- tions, the disparity in efficiency is more evident.
Empirical Results. Tab. 10 shows the training efficiency of GyroGr with and without different Grassmannian BNs. Compared to other Grassmannian BNs, GyroBN is significantly more efficient due to the simplicity of the gyro operation.
Analysis. The key difference between GyroBN (Alg. 1) and the ManifoldNorm (Chakraborty, 2020, Algs. 1-2) or RBN (Lou et al., 2020, Alg. 2) lies in their methods for centering, biasing, and scaling, GyroBN uses gyro operations, while ManifoldNorm and RBN rely on Riemannian operators, such as parallel transport and the Riemannian logarithmic and exponential maps. This distinction underpins the efficiency of GyroBN. The three primary contributing factors, ranked by importance, are as follows:
1. Riemann vs. Gyro. The Riemannian operators over the Grassmannian involve computa- tionally expensive processes like SVD decomposition or matrix inversion (see Tab. 8 for Riemannian exp and log, and (Edelman et al., 1998, Thm. 2.4) for parallel transport). Con- sequently, ManifoldNorm and RBN require multiple SVD or matrix inversion operations. In contrast, GyroBN is relatively simpler. As discussed in Sec. 6.1, the gyro operation can be further simplified, and the involved SVD is performed on reduced x p matrix instead of n p. Additionally, noted in App. F.22, computationally intensive matrix exponentiation is efficiently approximated using the Cayley map.
2. Reduced Matrix Products. As shown in Tab, 8, each Riemannian operator involves several matrix products over p matrices. GyroBN reduces these matrix products over (n - p) x or p x matrices, which are computationally more efficient, such as Prop. 6.1.
3. Optimization.
The biasing parameter optimization in GyroBN is simpler. ManifoldNorm n orthogonal matrix for biasing. and RBN employs an n x Grassmannian matrix, both requiring Riemannian optimization. In contrast, GyroBN applies trivialization tricks (parameterizing manifold data via the Riemannian exp (Lezcano Casado, 2019), as shown in Eq. (51)), making the biasing parameter a (n-p) xp Euclidean matrix. Furthermore, Eq. (51) and Eq. (22) share similar form, allowing them to be jointly simplified. Although RBN could adopt similar trivialization for biasing, this would introduce an additional Riemannian exp step, leaving little advantage over the Riemannian optimization. In summary, GyroBN benefits from joint simplification with trivialization, whereas the other two Grassmannian BN methods require additional Riemannian optimization.
F.6.2 TRAINING EFFICIENCY ON THE HYPERBOLIC NETWORKS
Table 11:
Average training time (seconds/epoch) for HNN with and without different hyperbolic BNs.
Methods
Cora
Disease
Airport
Pubmed
HNN
0.0323
0.0271
0.054
0.1253
HNN-RBN-H
0.0905
0.0883
0.1215
0.3416
HNN- -
0.0757
0.0842
0.119
0.3351
27

---

Published as a conference paper at ICLR 2025
Recalling Tab. 8, the Riemannian logarithmic and exponential maps over the hyperbolic space contain gyro operations. Therefore, it is expected that our GyroBN is more efficient than RBN. Tab. 11 reports the average training time per epoch, indicating that our GyroBN is more efficient than RBN (Lou et al., 2020) in the hyperbolic space.
F.7 EXPERIMENTS ON OTHER HYPERBOLIC BASELINES
The subsection validates our GryoBN on the HNN++ (Shimizu et al., 2020) and RResNet (Katsman al., 2024) backbones. We follow the HNN experiments and use these backbones the encoder for the link prediction task.
F.7.1 EXPERIMENTS ON HNN++
Table 12: Comparison of HNN++ with or without RBN-H and GyroBN-H on the Cora, Disease, and Airport datasets.
Dataset
Cora
Disease
Airport
HNN+
91.06 = 0.47
77.83 = 1.39
94.93 # 0.23
HNN++RBBN-H
88.22 0.67
59.50 = 1.67
94.91 + 0.23
GyroNNG
92.83 + 0.32
79.53 + 0.74
95.89 = 0.10
We evaluate HNN++ with and without RBN-H (Lou et al., 2020) and our GyroBN-H on the Cora, Disease, and Airport datasets. Like the HNN experiments, we used two-layer HNN++ backbone as the encoder, with hyperbolic hidden feature dimensions set 128. Tab. A reports the five-fold average results, leading to the following findings:
1. GyroBN can improve and stabilize HNN+ While RBN can degrade HNN++'s perfor- mance, particularly on the Cora dataset, GyroBN consistently enhances it. Additionally, we observed that HNN++ could be sensitive to weight decay. For instance, on the Cora dataset, setting the weight decay to le reduces the performance to 62.55, prompting us to use zero weight decay for HNN++ on this dataset. In contrast, HNN+ with GyroBN is robust to weight decay. Furthermore, HNN++-GrrBBN achieves the smallest performance variance across three datasets, indicating that GyroBN facilitates more robust learning.
2. GyroBN can accelerate convergence. On the Cora dataset, HNN++ requires over 400 epochs to converge. In contrast, HNN++-GyrOBN converges in approximately 150 epochs, significantly improving convergence.
F.7.2 EXPERIMENTS ON RRESNET
Table 13: Comparison of RResNet with without RBN-H and GyroBN-H under different hidden feature dimensions on the Cora dataset.
Dim RResNet
16
32
128
71.77 6.67
77.38 + 10.82
77.14 + 7.48
66.73+11.88
80.75+4.11
RResNci-RBN-H
61.94 + 2.28
63.54 # 3.01
62.26 # 3.48
60.38+2.97
87 87.9242.61
RResNet-GyroBNNH
84.58 5.44
87.67 # 2.53
88.08 + 2.90
86.50+1.59
89.52+3.42
We focus on the RResNet Horo (Katsman et al., 2024), which leverages a vector field induced by the horosphere projection feature map. Following the settings of HNN and HNN++ we use 2 residual blocks as the backbone. For a more comprehensive comparison, we evaluate hidden feature dimensions ranging from 128. Tab. 13 reports the five-fold average results, leading the following findings:
GyroBN consistently improves performance. While RBN performs well only under the -i8deinensionll setting, it fails under other dimensions, performing worse than the baseline RResNet. In contrast, GyroBN consistently improves RResNet's performance across all hidden feature dimensions.
2. GyroBN stabilizes performance and preserves representation power. Both RResNet and RResNer-RBN exhibit significant fluctuations in performance across different hidden
28

---

Published as a conference paper at ICLR 2025
dimensions. However, (ResNet-GyroBhi shows more stable performance. This indicates that our GyroBN could maintain the representation power the backbone network.
F.8 EXPERIMENTS ON COVARIATE SHIFTS
The covariate shift serves as the main motivation for the classical Euclidean BN (Toffe & Szegedy, 2015). Similar challenges arise in Riemannian networks, where the complexity of the manifold often disrupts data distributions, underscoring the utility of our proposed GyroBN. To demonstrate covariate shifts, we perform numerical experiments on Grassmannian and hyperbolic networks.
F.8.1 COVARIATE SHIFTS ON THE GRASSMANNIAN NETWORK
Table 14: Ten-fold results for geodesic distance dl Mot, Ip.a) and shift A d(Mous. x 100, where b vpz is the boundary of geodesic distance over Gr(p, n..
Pooling
Transformation
d(Mous outs p,)) A (%)
2.90 0.11 58.47% + 2.28%
4.18 + 0.04
84.22% + 0.72%
The transformation and pooling layers in the GyroGr baseline can be expressed
ftrans Gr(p, Gr(p, n),, fpooling: Gr(p, n) - Gr(p, n/2).
(52)
(53)
For simplicity, we assume is even for the pooling layer. Since the ONB Grassmannian Gr(p, n) a quotient manifold and pooling changes dimensions, we use the geodesic distance between the batch mean and the identity element Ip.n = (Ip. 0) T E Rnxp as consistent measure. Note that the geodesic distance on Gr(p, n.) is bounded by /p (Wong, 1967, Thm. 8).
We randomly generate 30 Grassmannian matrices of size 100 x 10 with an initial batch mean the identity element pp..- We denote the resulting batch mean after transformation or pooling as Mout- the distribution were preserved, the geodesic distance d( [out p..) (or d( d(Mouty pp.//2) for pooling) would be zero. Tab. 14 shows that Mout significantly deviates from Ip.. indicating the covariate shift.
F.8.2 COVARIATE SHIFTS ON HYPERBOLIC NETWORK
We examine the covariate shift in the transformation layer (Ganea et al., 2018, Lem. 6) HNN. We focus the canonical Poincaré ball (curvature K -1). We randomly generate 30 5-dimensional Poincaré vectors. The batch mean vectors of input and output in the transformation layer are
Min | 0..00.005, 0.0503, 0.0913, -0.0493, 0.0452) 0.0652
(54)
Moat [-0. 0.0099 0..4433, 0.0385, 0.0936, -0.0770
(55)
The geodesic distance between Mn, and Mout is 0.55, indicating the covariate shift in HNN.
F.9 EXPERIMENTS ON CONDITION NUMBERS
In the GyroGr network, the transformation layer outputs n X Grassmannian matrices, whose condition numbers are trivial since all singular values of Grassmannian matrix are 1. Similarly, for the HNN model, the transformation layer outputs a vector, leading to a trivial condition number well. Thus, focus on evaluating the condition numbers of the weight matrices in the hidden transformation layer and the overall network Jacobian matrix. We evaluate Grassmannian GyroGr networks without loss of generality under block architecture, both with and without different BN layers, using each trained model. The results demonstrate that GyroBN consistently reduces the condition numbers of the weight matrices and the network Jacobian.
F.9.1 CONDITION NUMBERS OF WEIGHT MATRICES OF THE TRANSFORMATION LAYER
As shown Tab. 15, GyroBN consistently reduces the condition numbers of the weight matrices across datasets. Interestingly, although ManifoldNorm achieves the lowest condition numbers, its performance could be even worse than the GyroGr baseline, suggesting that excessively reducing the condition number could limit the model's capacity capture sufficient expressive features.
29

---

Published as a conference paper at ICLR 2025
Table 15: Statistics of condition numbers for the weight matrices in the GyroGr transformation layer along with different normalization methods. As weight matrices have channels, we report the mean, standard deviation (std), minimum, and maximum values of their condition numbers. The dimensions of the 8-channel transformation matrices are 83 x 10 the HDM05 dataset and 140 x 10 on the other datasets.
Dataset
BN
Mean
Min
Max
None
3.81 + 0.23
3.46
4.17
ManifoldNorm-Gr
1.97 0.15
1.83
2.30
HDM05
RBN-Gr
3.49 = 0.46
2.96
4.43
GyroBN-Gr
2.17
2.12
2.67
None
0.12
2.61
2.94
ManifoldNorm-Gr Gr
1.76
1.63
1.92
NTU60
RBN-Gr
0.20
1.94
2.51
GyroBN-Gr
0.14
1.82
2.28
None
3.28
2.97
3.72
ManifoldNorm-Gr Gr
0.10
1.80
2.14
NTU120
RBN-Gr
2.12
2.00
2.36
GyroBN-Gr
0.16
2.00
2.33
F.9.2 CONDITION NUMBER OF NETWORK JACOBIAN
We further analyze the condition number of the network Jacobian on the HDMOS and NTU120 datasets. We randomly select 100 samples calculate the statistics of the associated network condi- tion numbers, as shown in Tab. 16. Our GyroBN consistently lowers condition numbers compared to other methods across both datasets, indicating that GyroBN stabilizes network training. Additionally, the reduced maximum condition numbers, especially on the NTU120 dataset, suggest that GyroBN avoids extreme outliers in conditioning, further highlighting its advantage over previous methods. Notably, on the HDM05 dataset, both ManifoldNome and RBN increase network conditioning.
Table 16: Condition numbers of network Jacobian (output w.r.t. input) across different normalization methods the GyroGr baseline. The statistics come from 100 random samples. The dimensions of Jacobian matrices are 117 x 930 on the HDM05 dataset and 11 x 9600 on the NTU120 dataset.
Dataset
BN
Mean
Min
Max
None
61.05 + 4.15
52.19
74.29
ManifoldNorm-Gr
67.38 + 6.82
51.15
88.81
HDM05
RBN-Gr
64.16 + 6.44
50.54
80.07
GyroBN-Gr
59.85 + 4.33
48,46
70.47
None
122.24 + 68.45
61.8
399.28
ManifoldNorm-Gr
116.75 + 65.17
75.46
443.01
NTU120
RBN-Gr
108.75 + 52.27
63.19
349.25
GyroBN-Gr
97.01 = 41.88
61.19
262.55
G PROOFS
G.1 PROOF OF PROP. 3.2
Proof of Prop. 3.2. By Nguyen Yang (2023, Lem. 2.3), casy computations show that Eq. (12) holds for Gr(p, n) iff it holds for Gr(p, n). Without loss of generality, we prove the case for the projector perspective.
Given any P.Q Gr(p,n), Def. 3.18 by Nguyen (2022a) gives the expression for gyration:
gyr(oP, PlQ F(eP, (
(56)
30

---

Published as a conference paper at ICLR 2025
with P) defined as
F(OP,P) mexp
3 (- @ P, pp.)))
mexp ((P ppm 1) mexp ((P:Tpa))
(57)
where (-) -ogg,, p.m
(-). This equation can be further simplified as
P( !1) me mexp (TTP. mexp ((P, (F..T..)))
PPP Ip. mexp ((P,
(58)
mexp
In.
The above derivation follows from
ep In 0 E Raxa.
(2) mexp(0) = In.
(p=P) mexp ([P,Ip..)
(3) eP = and mexp
Therefore, gyr(eP, Pl is the identity map.
G.2 PROOF OF THM. 3.3
Proof of Thm. 3.3. We first prove a useful lemma.
Lemma G.1 (Left Gyrotranslation Law). Every pseudo-reductive gyrogroups {G, Q} verifies the left
gyrotranslation law:
PP
R) = gyr(g,r[p. (PQ VP.
(59)
Proof. This lemma generalizes Lems. I.1 and L.1 by Nguyen & Yang (2023), which proves the left gyrotranslation law on the specific gyrogroups the SPD and Grassmannian manifolds. Their proof only relies the left cancellation and the basic axioms (G1-3). Note that the original proof of left gyrotranslation on the Grassmannian (Nguyen & Yang, 2023, Lems. I.1) questionable, as relies on the left cancellation of gyrogroups, and the Grassmannian is not gyrogroup but non-reductive gyrogroup. Fortunately, as we show Thm. D.1, the general pseudo-reductive gyrogroups, including the Grassmannian, enjoy left cancellation. Therefore. all the proof by Nguyen Yang (2023, Lems. I.I) can be readily generalized into the general pscudo-reductive gyrogroups.
For any R2 S G, the gyroautomorphism be expressed by the gyrator identity in Thm. D.I:
gyr[P, QR X + R,
(60)
gyr Q|5 X S,
(61)
where X = R
R), and S P (Q S). Then Eq. (30) in (Nguyen & Yang, 2023) for the specific Grassmannian can be directly extended into the pseudo-reductive gyrogroup, only relies on left gyrotranslation, invariance of the norm under gyroautomorphisms, and the axioms of (GI-G3).
(7 Thm. D.I indicates eE E) |gyr Q[R) ([eE e R|
(yrr @ (R))) (automorphism)
dgry (xx(
(62)
dgry RE,R),
= | + Rll
[Rllgr
31

---

Published as a conference paper at ICLR 2025
G.3 PROOF OF THM. 3.5
Proof of Thm. 3.5. Given any P G, we make the following proof.
Gyroisometry of the left gyrotranslation: This property generalizes Thms. 2.12 and 2.16 by Nguyen Yang (2023), which deal with the gyrotranslations in the SPD and Grassmannian manifolds. We have the following:
dgry Lp)
dg @ Q, P + R),
||e Q) @ (P P R)
llgyr[P, Q] (eQ R)
(left gyrotranslation),
(63)
|1QQ @ Rllgr (gyroisometry the automorphism),
dgr (Q,
Symmetry of the gyrodistance:
(p,(P l|eP @ Q gr
|i(eP + Q) gr (Eq. (5) and Eq. (7)),
llyr[e QeeQQ P)l gr (gyrosum inversion law),
(64)
||@Q () Pllgr (gyroisometry the automorphism), dQ,
Gyroisometry of the gyroinverse:
(EP, = @ll
|1QQ @ Pllge (gyrocommutativity and gyroisometry of the automorphism),
(65)
dg, P):
(gry Q) (Symmetry of the gyrodistance).
G.4 PROOF OF PROP. 3.6
Proof of Prop. 3.6. We first show the expressions of the gyrodistance M.. Then proceed to show the gyrodistance and gyroisometries on MM. We follow all the notations Tab. 2 and denote the geodesic distance under specific geometry dgeo.
Expressions of gyrodistance on M:
For {8 @AI), have the following:
day (P,Q)
p Q
gr
mlog( P -- b)
(Log, = mlog),
(66)
dgoo (P,
For (S @LE), we have the following:
dgy (P,Q)= |mexp(- mlog( P) + mlog mlog(Q)) gr>
|mlog (- (ml((m) + mloo(Q))]), (P) (mlog_
(67)
un)og(P)- mlog(())))) [Q) (the pullback of LEM (Chen et al., 2024c))
dgeo (P Q),
where mlog.,) the inverse of the differential map of mlog, and || is the norm induced by the LEM at T.
32

---

Published as a conference paper at ICLR 2025
As LCM is also pullback metric (Chen al., 2024c), (S++, @... ||+Ec(- (L)(
follow the same logic:
dgey (P.Q)
( P)) ( (Logy(P) (frc))
(68)
IVLc(P) cic Q) (the pullback of LCM (Chen al., 2024e))
ddcco(P Q)
For (Gr(p.n), Gr we have the following:
dRy (P.Q)
mexp( - -[,, Io.1l)Q mexp([p, Inm))
(PP=-P)
gr
P QP (P mexp((P. p.,1)) E (O(n)),
g1
(69)
2 Logg
1
(PTQP) (11-11 I1-11),
1
[2, I Tp.il|
2
where [2, Ip.0l
= LogT
[ QP)..
For 22 we have the following:
1
mlog
-2h QP)
2P) (I, -Tpmm))))) ((Bendokat et al., 2024, Prop. 5.6)).
1
mlog
P (In -2Q) PPT (I, - 1-2PII, P)
2
( (In - -2Q) (In - -.-PIpppp P).
=
mlog
2
(70)
(1)
PT
mlog
(In-20) 2Q) (n-2PT PT pT))) P-
PT
mlog ((Tn 2Q) (In - -211) P.
TTP..
Eq. (70) follows from the following facts:
(1) When mlog(B) (B) is well-defined and A non-singular, 1 (log B)A log (A) (Horn & Johnson, 2012).
(2) P mexp DI. mexp(- (P,,p))) (Nguyen, 2022a, Eq. (36)).
(3) Let 5 mlog ((I, 2Q) (In 2P))
Combining Eqs. (69) and (70), we have
1
dgy (P.Q)
Ω p.
2
PTP.. T
Tp.nI] |
2
@
p
Ω2 PT p. PT p
@
[2 PT
PT
(71)
(8)
|1 P||,
(4)
[Q)l
2
[[ogp(())lpp
(P,
33

---

Published as a conference paper at ICLR 2025
Eq. (71) follows from the following facts:
(1)
PP SiP, 1p,0]= P - Ip SiP,
= [PPIn PT - PT P.
(72)
= D. p. PTIP.
(2) Euclidean norm is invariant under the action A OAO VA p 0 E O(n).
(3) P mexp([P, (P.TPPPT) (p..)) mexp(_IP,
(4) (Q) [S, PI.
(5) 1ppppp } 111-11, VP E Gr(p,n).
(6) dgeo (P.Q) | for any P.Q. Gr(p, n) not each other's cut locus.
For (Gr(p, n), @ GrG, we first make the following notations: We denote the geodesic distance, gyrodistance, Riemannian logarithm at pp.. and Riemannian metric at Ip.n Gr(p, n) as dgeo dgyy. Logı,.. and I1-11pp.. The counterparts on Gr(p, n) are dgeo- dgyy. Logi,... and I1-11pp.. As shown
by Nguyen et al. (2024, App. N), т : Gr(p. n) - Gr(p, n) is Riemannian isometry. Then, for any U, V Gr(p. with (l)) P and (V) Q, have the following:
ONB
dg.((U
Log (e CIT +
ONB
Logipn
K 1-
P
(2)
LogT pm
3 PD
Gr
ONB
(73)
(3)
Gr p
Gr
PP
LogT
Ipm
day (P.Q)
dgeo (P,Q)
dge0 (U,V)
The above comes from the following.
(1) Gyro additions under the Riemannian isometry (Nguyen & Yang, 2023, Lem. 2.1). (2,3.4) Riemannian operators under the Riemannian isometry (Gallier & Quaintance, 2020).
Constant Curvature Spaces:
When Mи R" = 0), the gyro structures defined in Eq. (3)-Eq. (9) are reduced to the vector structures. The claim can be directly proved. In the following, we present the proof for K * O. We first show the expression for the gyrodistance under MR. then the isometry of gyration, and finally the results on the gyroisometry of gyrotranslation and gyroinverse. In the following, a, b, x. are arbitrary points in MR.
Gyrodistance and geodesic distance: The Riemannian metric, logarithm, and geodesic distance on the CCS (Skopek et al., 2019) are
K = =>* g Log (y)
(74)
tan (v 11 1 +K vll
0K y
(75)
v VKT
dgoo y)
K tan (VIK)I-=@vIl.
(76)
34

---

Published as a conference paper at ICLR 2025
|= 9 is the standard Euclidean inner product, and @x is the gyro addition in Tab. 2. Especially, when O, have
where X
9 22g
(77)
Logu K (v)
1
tank 11
(Viki
l1all)
(78)
VTK
M|vl
For the gyrodistance, we have the following:
(gry(E, -| y yllgr:
Logo (-x v)llo-
V tan (VIK]ll- vll) 1-x 2 yl
(79)
2
tan
(vikll-x
(As > 0, tan (s) > 0),
|
where || llo is the norm induced by 90'
Norm invariance under gyration: As MM forms a real inner product gyrovector spaces (Ungar,
2009, Def. 3.2), any gyration on CCS preserves the norm induced by standard inner product:
gyr[a, 6](x]]
||]|, Vz.
(80)
For the gyronorm, we have the following:
llgyr(a, b|ر||
2 T.goo (gvr(a, (a, b)o:
2
tan (viK)
K
lerrla, b]x|]]
K
(81)
lan
K
|
Isometry left gyrotranslation and gyroineers: Note that MM forms gyrocommutative gyrogroup.
According to Thm. 3.5, we can obtain the results.
G.5 PROOF OF THM. 4.1
Proof of Thm. 4.1. According to Thm. 3.3 and Thm. 3.5, any left gyrotranslation is gyroisometry. Therefore, for any Q E M, we have the following:
dRy (B P., Q) day (eB e (B@ P.). eB @ Q)
gry (@B ④ B) @ gyr(eB (J())) Q)
(82)
dgry (Ps, eB + Q)
The above comes from the following.
(1) Any left gyrotranslation a gyroisometry.
(2) Left gyroassociative law.
(3) e B + B = E and pseudo-reduction.
Denoting the gyromean of {P.} and {B a P.) as M and M, we have the following:
B M (1) B (eB M)
gyr (B
(83)
M.
The above comes from the following.
35

---

Published as a conference paper at ICLR 2025
(1) Eq. (82) indicates that . = eB @ M.
(2) Left gyroassociative law.
Pseudo-reduction.
Now, we proceed to deal with the second property. We have the following:
d ggy (1
Par F)
| @ (t P.) gr
||t Piller
LItTogE Pell
|4||TTog Pille
(84)
|:11
(3)
| @ P.ll
Mldgery E, P:)
(1) Hf/dgry (P,, E)
The above follows from the following.
(1) Symmetry of gyrodistance (Thm. 3.5).
(2) e = E.
(3) eE P..
(4) Symmetry of gyrodistance.
The last equation Eq. (84) indicates the homogeneity of dispersion from E.
G.6 PROOF OF PROP. 5.1
Proof of Prop. 5.1. We only need to prove the equivalence of gyrodistance and geodesic distance.
Note that every Lie group automatically gyrogroup with each gyration the identity map.
We denote [M, $,g} gf as a Lie group with left-invariant metric For any P and Q in M, we have the following:
dgy (P, Q)
||ep + Q11
[[oog<(PP@ Q)Il
dgeo
EE Q)
(85)
d geo
(P,P
(g
The above derivation comes from the following.
1. Definition of gyrodistance Eq. (9).
2. Definition of gyronorm Eq. (8).
3. Under left-invariant metric, any left Lie group translation a Riemannian isometry.
4.
P Q) (P P) @ Q( the associative of group addition)
= E 100 Q
(86)
Q.
Therefore, the gyromean and gyrovariance exactly the Fréchet mean and variance under the geodesic distance, while the running mean updates are also identical under gyrodistance and geodesic distance.
36

---

Published as a conference paper at ICLR 2025
G.7 PROOF OF PROP. 6.1
We first review fast and stable algorithm for the ONB Grassmannian logarithm (Bendokat et al., 2024, Alg. 5.3), and the calculation of Grassmannian logarithm under the projector perspective by the ONB Grassmannian logarithm (Nguyen et al., 2024, Prop. 3.12).
Algorithm 2: Grassmann logarithm under the ONB perspective (Bendokat al., 2024, Alg. 5.3)
Input: U.Y Y E Gr(p, n) are Stiefle representatives under ONB perspective.
QSR SVD = YTU with S in ascending order, and Q and R column-wisely flipped accordingly;
S Inn S²;
A (I, UUT) YYQ aresin( 8) RT:
5
Output: Loge (Y) A
Alg. 2 reviews a fast and stable algorithm for the Grassmannian Riemannian logarithm under the ONB perspective Gr(p,n) n). The vanilla Riemannian logarithm in Tab. 8 requires an p SVD and p x p matrix inverse, while Alg. 2 only requires an x p SVD. Therefore, Alg. is more efficient than the vanilla logarithm. Besides, Alg. can also return unique tangent vector when Y is in the cut locus of U. For more details, please refer to Bendokat et al. (2024, Sec. 5.2).
As the projector perspective is isometric to the ONB perspective, the Grassmannian logarithm under the projector perspective can be calculated by the ONB Grassmannian logarithm (Nguyen et al., 2024, Prop. 3.12).
Proposition G.2 ((Nguyen et al., 2024)), Given any P,Q E Gr(p, n) with U = n (P) and
V x (Q), the Riemannian logarithm (Qg)(Q) on Gr(p, n) is given
(ogr(Q) T.,U (Logu V),
(87)
where Log is the Riemannian logarithm under the ONB perspective, n.ZU Trrr(p, TpGr(p,n) n) the differential map of x at U, which defined as
7
(A) = AUT +UA
"YAT,VAE AA GrGr(,)
(88)
Now, we begin present the proof.
Proof of Prop. 6.1. We first show the expression for Log and Logi
First note the following:
(In - Innttpnn p.ppp.) =
0
(89)
In-p
UT UpI = (UT, U (
1p
0
(90)
UT.
By the above equations, the ONB Grassmannian logarithm 4p.0 is
)
U1
)o
Q arcsin(S)
LogI, (U)
RT (Alg.
0 In-p
Un
S
U
(resin(S) S) s
RT
(91)
0
U2
where QSR SVD
UT with S in ascending order, and Q and R column-wisely flipped accordingly,
and 5 = =v, 52.
37

---

Published as a conference paper at ICLR 2025
For Logi,.. we have
LogI,.. (UUT)
(Logr... (1)
0
U2
(92)
(3)
U2
U2
0
The above derivation comes from the following.
Prop. G.2
(2) Eq. (91)
For any A (A) AT) TTy,.. Gr(p, where A p x p, have the following
Ku.ppnn ((
A
A2
))
=
A
A
(4p:
0)
+
I
(AT
A
0
A
A
A
+
(93)
A
+
AJ
A
Combining all the above results together, we have the following:
TUT I Ip.nl=
Log; pin (UU ) I Ipmm
0
02 T
T
U2
0
1p
1p
:
))
U
(94)
U2
0
0
U2
0
0
Uf
Ü3
0
38


Reviews:
[{"review_id": "cDumNhv2F8", "reviewer": "Reviewer_oCvr", "summary": "The paper proposes a new batch normalization method on Riemannian manifolds named GyroBN. GryoBN is a new batch algorithm on a class of algebras geometries named “pseudo-reductive gyrogroups”, which relaxes the left reduction law of a gryogroup as defined by [Ungar 2009]. The work generalizes prior work on batch normalization on Lie groups, such as LieBN [Chen et al., 2024b], and the SPD batch norm proposed by Brooks et al., 2019.", "questions": "- Is there a reason why the notations for the identity and left inverse different in Eq. 12 than the ones used in Section 2? Is Eq 12 equivalent to saying that $gry[ \\ominus b, b] = e$?\n- How efficient is the mean and variance calculation? Prior work based on Karcher flow are known to be slow. How does GyroBN’s speed compare to prior work?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "To me, the main contribution of the work is how it theoretically generalizes several prior work on batch norm for manifolds. The work generalizes LieBN [Chen et al., 2024b] and the SPD Batch norm proposed in [Brooks et al., 2019] to a wider class of geometries. While there have been some work such as [Lou et al., 2020] which propose general batch norm algorithms, it does not give control over sample statistics. Although the originality of extending prior work on gyrononreductive gyrogroups (as done by [Nguyen, 2022] to the proposed pseudo-reductive gryogroups) seems limited, it still appears to be an important extension.", "weaknesses": "While the paper provides experiments on Grasmannian and hyperbolic manifolds, it is not clear from the paper why empirically GryroBN outperforms prior work. Is there a theoretical reason why using gryodistance may outperform prior work that use geodesic distance for batch norm? I wish the paper would do a deeper analysis into this difference, and how it affects what the networks are learning. Here are some possible experiments that could strengthen the paper's analysis:\n- The original reason behind batch norm for Euclidean neural networks is covariate shift. Can you show empirically that the baselines suffer from covariate shift, and that GryoBN reduces covariate shift? Even a result showing the opposite would be an interesting contribution to the geo DL community. \n- Another experiment is to analyze the condition numbers of network's layers. Is is true that GryoBN makes the outputs of the neural networks better conditioned?\n\n\n\nAnother weakness is that the paper does not analyze how efficient the proposed batch norm algorithm is compared to prior work. Prior work based on Karcher flow are known to be slow. It would strengthen the paper to include this analysis for the experiments on Grassmannian and hyperbolic spaces. I see that there is a table showing training efficiency in Appendix F.1.4, but it only a comparison to prior work without batch normalization, not other work that do use batch norm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QnXwcAOuKI", "reviewer": "Reviewer_MrZQ", "summary": "This paper addresses the problem of extending batch normalization, which has accelerated deep learning in many areas, to non-Euclidean geometries, where the data lies on a Riemannian manifold. While computing mean and variance are trivial in Euclidean spaces, they become non-trivial in general Riemannian manifolds, because there is no analogue of addition that is associative, and would allow for an easy computation of the mean. The main goal is to extend the algebra for Riemannian manifolds, where mean and variance in the BN operator should be replaced by  Fréchet mean and Fréchet variance over the manifold. The main theoretical contribution of the work is unifying the analysis of some previous works that cover special cases of Riemannian manifolds, and extending them to some new types of Riemannian manifolds, so that batch normalization can be defined and computed. They go on to show the empirical value of their proposed method by testing newly developed Grassmannian network, GyroGr, with different types of batch normalization, and their proposed normalization improves the test accuracies. \n\nThe paper builds on previous works that has addressed these problems for specific forms of Riemannian manifolds (Table 1). However, authors argue that a more general approach can combine these as special cases, thereby unifying the analysis. Furthermore, they introduce a more relaxed notion than earlier works, termed \"pseudo-reductive gyrogroup\" that encompasses previous works, but can admit new Riemannian structures due to its relaxed condition, such as Grassmannian manifolds. More specifically, they introduce the relaxed condition, pseudo-reductive, which replaces left reduction (G4) condition that is more stringent, which covers Grassmannian manifolds.\n\n**Update**\nIn light of the other discussions as well as the explanations and clarifications by the authors, I'm happy to increase my score from 5 (marginally below acceptance) to 6 (marginally above the acceptance)", "questions": "No questions.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Normalization layers and in particular batch normalization have been key to success in training many modern architectures. However, when physical or mathematical constraints imply non-Euclidean geometries where data is on a manifold, computing mean and variance for BN becomes a non-trivial task. While one could take an ad hoc or heuristic approach, a more principled approach is likely to further our understanding and improve the state of the art in this field. \n\nThis paper addresses this problem by studying a  relaxed set of properties, that admit a sufficiently rich algebra for computing mean and variance, and thereby a BN operator. This principled approach is certainly a welcome development.  To the extent that I could understand which was admittedly limited, the theoretical statements and assertions seem to be sound. Finally, in section 7 they present experiments mostly on GyroGr which is a Grassmannian network as a backbone, and compare it without BN, or with other types of BN, and show their proposed BN achieves the best results. While the empirical setup is sound, I cannot assess the significance of these results, given that they mostly focus on one network and one particular type of Riemannian manifold.", "weaknesses": "While this work seems to be on solid theoretical grounds and is mathematically sound, my main concern is the overall structure and writing, which make the theoretical contributions of the paper rather hard to grasp. I've added a few concrete suggestions that I believe would improve the readability of this paper. \n\n- An expanded preliminary on various concepts non-Euclidean geometry. While authors can assume that their readers is limited to those with deep and expert knowledge of this topic, such an assumption would narrow  the audience of this paper quite a lot. For reference, I spent >5 hours to try educate myself on some of the basics that I did not know before reading the paper, but besides \"parsing\" the theoretical statements, I could not gain a deeper insight. With an expanded preliminary that illuminates these concepts, I could have understood the results on a much deeper level. \n- Adding high level ideas : For theoretical contributions, it is always valuable to the reader to get a high level view of the ideas, techniques, or proof insights. In this particular case of Riemannian geometries, which presents itself as highly abstract, these types of explanations are even more crucial. \n- Adding concrete examples for abstract concepts: there are numerous technical terms, some classical and some introduced in the paper, that are left without any high level explanations, eg. gyrations, left gyroassociative law, left reduction law, pseudo-reductive gyrogroups, are all concepts that are formally introduced, but no high level explanations are given. So the reader is forced to just parse and compile the formal statements, which is not easy, unless they are already an expert on this topic. Would it be possible for authors to give concrete examples for each concept as they go along with their formal definitions and statements? For example, can they use a working example of a Grassmannian, and explain the rough geometric intuition behind each notion?  While that might not be easy in all cases, it would tremendously improve the readability and value of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4rd54eE8RE", "reviewer": "Reviewer_A3ej", "summary": "This paper proposes a geometric neural network batch normalization operation that is based off of a proposed gyrodistance frechet mean.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper's method seems principled, and the derivations seem right to me.\n2. The empirical results are reasonably convincing, showing good improvement over prior methods.", "weaknesses": "Overall, there are no major concerns from me. I believe that the paper is technically sound and complete, but the scope is rather limited and the overall contribution may ultimately be incremental (over prior Riemannian batch norm papers). Beyond that, I have two minor nitpicks:\n\n1. A natural issue is that not all manifolds are gyrogroups. This means that the proposed method is not general. This is probably okay since most manifold methods are not general (by dint of the fact that it is hard to work on manifolds), but might affect some use cases that were not covered.\n2. The empirical results are not very convincing. In particular, while they are controlled against other RBN methods, the base architecture is often weak/basic. I would like to see comparisons with other architectures (e.g. HNN++, Riemannian ResNet [1]). This may also be important since the architectures used are built solely from gyro-operations, and using different architectures could control for this confounding factor (gyro operations helping gyro operations).\n\n[1] https://arxiv.org/abs/2310.10013", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D4RfJNl2gF", "reviewer": "Reviewer_Yeux", "summary": "The paper proposes a new Riemannian Batch Normalization (RBN), named Gyrogroup Batch Normalization. This new RBN method provides controllable means and variances and includes several previous RBN methods. To include the case of Grassmannian, this work proposes the pseudo-reductive gyrogroups that lie between the gyrogroup and the nonreductive one. Theoretical analysis regarding this new approach is provided with numerical verifications across several experiments.", "questions": "1. What is the computational complexity of GyroBN, and how does it scale with the input dimension compared to previous methods?\n\n2. How do you predetermine the running mean, running variance, biasing parameter, and scaling parameter in Algorithm 1?\n\n3. Can you provide any discussion if the gyrations are not isometric?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well-written and well-structured and the comparisons with previous RBN methods are comprehensive. In particular, the comparison of assumptions is presented clearly. The theories seem solid and the performance is improved compared to the existing methods.", "weaknesses": "I suggest the authors provide a comprehensive version of Algorithm 1 to specify the calculation of each quantity or elaborate in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)