RMLR: Extending Multinomial Logistic Regression into General Geometries
Authors: Ziheng Chen
Yue Song
Rui Wang
Xiaojun Wu
Nicu Sebe
Venue: NeurIPS 2024 poster (2024)
Keywords: Riemannian neural networks
Matrix manifolds
SPD manifolds
Special orthogonal groups
Primary area: deep_learning_architectures
TL;DR: We propose a general framework of building intrinsic Riemannian classifiers for general geometries , and showcase our framework on the SPD manifold and special orthogonal group.

Abstract: Riemannian neural networks, which extend deep learning techniques to Riemannian spaces, have gained significant attention in machine learning. To better classify the manifold-valued features, researchers have started extending Euclidean multinomial logistic regression (MLR) into Riemannian manifolds. However, existing approaches suffer from limited applicability due to their strong reliance on specific geometric properties. This paper proposes a framework for designing Riemannian MLR over general geometries, referred to as RMLR. Our framework only requires minimal geometric properties, thus exhibiting broad applicability and enabling its use with a wide range of geometries. Specifically, we showcase our framework on the Symmetric Positive Definite (SPD) manifold and special orthogonal group, i.e., the set of rotation matrices. On the SPD manifold, we develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric. Extensive experiments on different Riemannian backbone networks validate the effectiveness of our framework.

RMLR: Extending Multinomial Logistic Regression into General Geometries
Ziheng Chen', Yue Song?; : Rui Wang', Xiao-Jun Wu Nicu Sebe'
University of Trento, Caltech, 3 Jiangnan University ziheng_ch@163. com, yuesongöcaltech.com edu
Abstract
Riemannian neural networks, which extend deep learning techniques to Rieman- nian spaces, have gained significant attention in machine learning. To hetter classify the manifold-valued features, researchers have started extending Euclidean multi- nomial logistic regression (MLR) into Riemannian manifolds. However, existing approaches suffer from limited applicability due to their strong reliance on specific geometric properties. This paper proposes framework for designing Riemannian MLR over general geometries, referred as RMLR. Our framework only requires minimal geometric properties, thus exhibiting broad applicability and enabling its use with a wide range of geometries. Specifically, we showcase our framework on the Symmetric Positive Definite (SPD) manifold and special orthogonal group (n))), i.e., the set of rotation matrices in R", On the SPD manifold, we develop five families of SPD MLRs under five types of power-deformed metrics. On SO(n), we propose Lie MLR based on the popular bi-invariant metric. Extensive experi- ments on different Riemannian backbone networks validate the effectiveness of our framework. The code is available at https: // titt......... com/ com/GitZh GitZH- Chen/RMLA
Introduction
In recent years, significant advancements have been achieved in Deep Neural Networks (DNNs), enabling them to effectively analyze complex patterns from various types of data, including images, videos, and speech [29, 38, 27, 66]. However, most existing models have primarily assumed the underlying data with a Euclidean structure. Recently, growing body of research has emerged, recognizing that the latent spaces of many applications exhibit oun-uuciddean geometries, such as Riemannian geometries [9]. Various rrequently-encounterr posed interesting challenges and opportunities, including special orthogonal groups SO(n) [67, 31], symmetric positive definite (SPD) [30, 10, 42, 73, 18, 19], Gaussian [14, 47], Grassmannian [32, 72] spherical [56], and hyperbolic manifolds [23]. These manifolds share an important Riemannian property their Riemannian operators, including geodesics, exponential & logarithmic maps, and parallel transportation, often possess closed-form expressions. Leveraging these Riemannian operators, researchers have successfully generalized different types of DNNs into manifolds, dubbed Riemannian neural networks.
manifolds in machine learning have
Although Riemannian networks demonstrated success in many applications, most approaches still rely on Euclidean spaces for classification, such tangent spaces [30, 31, 10, 47, 69, 71, 48, 49, 37, 70, 15], ambient Euclidean spaces [68, 57, 58], or coordinate systems [12]. However, these strategies distort the intrinsic geometry of the manifold, undermining the effectiveness of Riemannian networks. Researchers have recently started directly developing Riemannian Multinomial Logistic Regression (RMLR) on manifolds. Inspired by the idea of hyperplane margin [39], Ganea et al. [23] developed a hyperbolic MLR the Poincaré ball for Hyperbolic Neural Networks (HNNs). Motivated by HNNs, Nguyen and Yang [50] developed three kinds of gyro SPD MLRs based on three distinct gyro
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

structures of the SPD manifold. In parallel, Chen et al. [16] proposed framework for building SPD MLRs induced by the flat metries on the SPD manifold. Nguyen et al. [51] proposed gyro MLRs for the Symmetric Positive Semi-definite (SPSD) manifold based on the product of gyro spaces. However, these classifiers often rely on specific Riemannian properties, limiting their generalizability to other geometries. For instance, the hyperbolic MLR [23] relies on the generalized law of sine, while the gyro MLRs [50, 51] rely on the gyro structures.
This paper presents a framework of RMLR over general geometries. In contrast to previous works, our framework only requires the explicit expression of the Riemannian logarithm, which is the minimal requirement in extending the Euclidean MLR into manifolds. Since this property is satisfied by many commonly encountered manifolds in machine learning, our framework can be broadly applied to various types of manifolds. Empirically, we showcase our framework on the SPD manifold and rotation matrices. On the SPD manifold, we systematically propose SPD MLRs under five families of power-deformed metrics. We also present a complete theoretical discussion on the geometric properties of these metrics. In the Lie group of (O(n). we propose Lie MLR based on the widely used bi-invariant metric to build the Lie MLR. Our work is the first to extend the Euclidean MLR into Lic groups. Besides, our framework incorporates several previous Riemannian MLRs, including gyro SPD MLRs in [50], SPD MLRs in [16], and gyro SPSD MLRs in [51].
Our SPD MLRs are validated on four SPD backbone networks, including SPDNet [30] on the radar and human action recognition tasks and TSMNet [37] on the eleetroenccephalograpyy (EEG) classification tasks for the Riemannian feedforward network, RResNet [36] on the human action recognition task for the Riemannian residual network, and SPDGCN [76] the node classification for the Riemannian graph neural network. Our Lie MLR is validated on the classic LieNet [31] backbone for the human action recognition task. Compared with previous non-intrinss classifiers, our MLRs achieve consistent performance gains. Especially, our SPD MLRs outperform the previous classifiers by 14.23% % on SPDNet and 13.72% RResNet for human action recognition, and 4.46% on TSMNet for EEG inter-subject classification. Furthermore, our Lie MLR can improve both the training stability and performance. In summary, our main theoretical contributions the following: (a) We develop a general framework for designing Riemannian MLR over general geometries, incorporating several previous Riemannian MLRs on different geometries. (b) We systematically propose 5 families of SPD MLRs based on different geometries of the SPD manifold. (c) We propose novel Lie MLR for deep neural networks on SO(n).
Main theoretical results: We solve the Riemannian margin distance to the hyperplane in Thm. 3.2 and present our RMLR framework in Thm. 3.3. As shown in Tab. 1, our RMLR incorporates several existing MLRs on different geometries. Thm. 4.2 showcases our RMLR on the SPD manifold under five families of metrics summarized in Tab. 2. To remedy the numerical instability of BWM geometry on the SPD manifold, we also propose a BWM in App. F.2.2 Thm. 5.2 proposes the Lie MLR for the Lie group SO(n). Due to the page limits, we put all the proofs in App. H.
cetrooaagation-rienndl solver for the SPD MLR under
2 Preliminaries
This section provides a brief review of the basic geometries of SPD manifolds and special orthogonal groups Detailed review and notations are left in Apps. B and B.I.
SPD manifolds: The set of X n symmetric positive definite (SPD) matrices is an open submanifold
of the Fuclidean space S of symmetric matrices, referred to as the SPD manifold S There
are five kinds of popular Riemannian metrics on S A ffine-Invarian Metric (AIM) [52]. Log- Euclidean Metric (LEM) [3], Fower-Fuclidean Metries (PEM) [22], Loog-Chllssky Metric (L.C) Metric (BWM) [5]. Note that, when power equals 1, the PEM is reduced to the Euclidean Metric (EM). Thanwerdas and Pennec [63] generalized AIM, LEM, and EM into two-parameeers families of O metrics, (.e. A)-AIM, (a. 9LMMMM, and (a, -)M, with min (a, a + n.3) > O. We denote the metric tensor of (a, 3)-AIM, (a, LEMM. 3)MM,, LCM, and BWM as g (aa..f)-AIM, . (f...))LLMM. g(a EM .LCMM, and g BWM respectively.
[41], and
The special orthogonal group SO(n) is the set of n orthogonal matrices with unit determinant, the elements of which are also referred to as rotation matrices. As shown in [25], (O()) forms a Lie group. We adopt the widely used bi-invariant Riemannian metric [8].
Rotation matrices:
2

---

3 Riemannian multinomial logistic regression
Inspired by [39], Ganea et al. [23], Nguyen and Yang [50], Chen et al. [16], Nguyen al. [51] extended the Euclidean MLR into hyperbolic, SPD, and SPSD manifolds. However, these classifiers rely on specific Riemannian properties, such as the generalized law of sines, gyro structures, and flat metrics, which limits their generality. In this section, we first revisit several existing MLRs and then propose our Riemannian classifiers with minimal geometric requirements.
3.1 Revisiting existing multinomial logistic regressions
Given C. classes, the Euclidean MLR computes the multinomial probability of each class: Vk E (I,..., where by. E R. and ag E R0) As shown in [23]. the Euclidean MLR can be reformulated by the margin distance the hyperplane:
ply = k x:) exp (Lux, .z) -bx,,
(I)
ply k |12) exp (sign( (ai, (aa))laxll [ld(a, Hay.p.)) H Harmm = {x E R" (ax: x - pph) 0},
(2)
(3)
where (ax, pk) = by. and Happp is a hyperplane.
Eqs. (2) and (3) can be naturally extended into manifolds M by Riemannian operators:
ply = k S) exp (sign((As, Logp, (5)) Axlr, a(s, Hx.,)))
(4)
HA.. {S M: gP, (Logp, S, As) = =},
(5)
where P& M, Ag E Tp, M,10. sr, logarithm at Ps. The margin distance is defined as an infimum:
gr, is the Riemannian metric at Ps. and Logg, is the Riemannian
inf d(s, H A.)) Q
d(s,Q).
(6)
The MLRs proposed in [39, 23, 50, 16] can be viewed as different implementations of Eq. (4)- Eq. (6). To calculate the MLR in Eq. (4), one has to compute the associated Riemannian metrics, logarithmic maps, and margin distance. The associated Riemannian metrics and logarithmic maps often have closed-fome expressions on the frequently-encoume
manifolds in machine learning. However, the computation of the margin distance be challenging. On the Poincaré ball of hyperbolic manifolds, the generalized law sines simplifies the calculation of Eq. (6) [23]. However, the generalized law of sines is not universally guaranteed other manifolds. Additionally, Chen et al. [16] developed a closed-form solution of margin distance on the SPD manifold under any metric pulled back from Euclidean spaces. For curved manifolds, solving Eq. (6) would become non-convex optimization problem. To address this challenge, Nguyen and Yang [50] defined gyro structures on the SPD manifold and proposed pseudo-gyrodistane gyrodistance calculate the margin distance. Similarly. Nguyen et al. [51] proposed a pseudo-gyynn -eeuud-gyrodistance on the SPSD manifold based on the gyro product space. However, gyro structures do not necessarily exist in general geometries. In summary, the aforementioned methods often rely on specific properties of their associated Riemannian metrics, which usually do not generalize to general geometries.
3.2 Riemannian multinomial logistic regression
Recalling Eqs. (4) and (5), the least requirement of extending Euclidean MLR into manifolds is the well-definedness of Logp, (S) for each k. In this subsection, we will develop Riemannian MLR, which depends solely on the Riemannian logarithm, without additional requirements. such gyro structures and generalized law sines. In the following, we always assume the well-definedness of the Riemannian logarithm. We start by reformulating the Euclidean margin distance to the hyperplane from a trigonometry perspective and then present our Riemannian MI.R.
As we discussed before, obtaining the margin distance of Eq. (6) could be challenging. Inspired by [50], we resort to the perspective of trigonometry to reinterpret Euclidean margin distance. In Euclidean space, the margin distance is equivalent to
d(z, Happ) siin<rpy")de )d(z, p), with y" = arg max (cos Lapy).
ye (r)
We extend Eq. (7) to manifolds by the Riemannian trigonometry and geodesic distance, the counter- parts of Euclidean trigonometry and distance.
3

---

Definition 3.1 (Riemannian Margin Distance). Let H.r be Riemannian hyperplane defined in Eq. (5), and E M. The Riemannian margin distance from S to A.P is defined as
d(s, H x,P) = sin sin(<SPY))d(S,P)
(8)
where d(S, P) is the geodesic distance, and Y argmax LSPY) with YE E H,,. The initial velocities of geodesies define cos SSPY:
cos /SPY (Logr Y, Logp
(9)
LLgg Y p: Logp SiIp'
where (;-)) is the Riemannian metric at P, and || p is the associated norm. The Riemannian margin distance in Def. 3.1 has a closed-form expression.
Theorem 3.2. [4] The Riemannian margin distance defined in Def. 3.1 is given as
d(s, H A.P) (Logp S, P!
(10)
|||||
Putting the Eq. (10) into Eq. (4), we closed-form expression for Riemannian MLR.
Theorem 3.3 (RMLR). [L] Given a Riemannian manifold {M. g), the Riemannian MLR induced by g is
ply k S M) x exp (Logr, P.)
(11)
where P& E M, A E MA and Log is the Riemannian logarithm.
Ag in Eq. (11) can not be directly viewed as Euclidean parameter, Ag E MP depends on Pa
and P& varies during the training. However, the tangent vector As can be generated from a tangent space at fixed point. Several tricks can be used, such as Riemannian parallel transportation [21]] vector transportation [1], the differential of Lie group or gyrogroup translation [64, 65]. Following previous work [23, 16, 50], we focus parallel transportation and Lie group translation:
Åg I TQ- Ak:
(12)
Åg = L 1,00''0.044
(13)
where Q E M is a fixed point, A& E TQ.Mi(0).
the parallel transportation along geodesic
connecting Q and Ps. and L +.Q denotes the differential map of left translation L p2
with Ps Q denoting Lie group product and inverse. In this way, Ag lies in fixed tangent space and, therefore, can be optimized by Euclidean optimizer.
Remark 3.4. We make the following remarks w.r.t our Riemannian MLR.
(a). The reformulation of Eq. (7) in gyro MLR [50, 51] and ours are different. Gyro MLR adopts gyro trigonometry and gyro distance to reformulate Eq. (7), while our method directly uses Riemannian trigonometry and geodesic distance.
(b). Compared with the MLRs on hyperbolic, SPD, or SPSD manifolds in [23, 50, 16, 51], our framework enjoys broader applicability, as our framework only requires the Riemannian logarithm. This property is commonly satisfied by most manifolds encountered in machine learning, such as the five metrics SPD manifolds mentioned in Sec. 2, the invariant metric on SO(n) [8], and hyperbolic & spherical manifolds [11, 56]. Besides, several existing MLRs on different geometries are special cases of our Riemannian MLR, which are detailed in Tab. 1.
(c). The well-definedness of the Riemannian logarithm a much weaker requirement compared to the existence of the gyro structure. The gyro structure only requires the Riemannian logarithm but also implicitly requires geodesic completeness [50, Eqs. (1-2)). For instance, on SPD manifolds, EM and BWM [63] incomplete, undermining the well-definedness of gyro operations.
4 SPD multinomial logistic regressions
This section showcases our RMLR framework on the SPD manifold. We first systematically discuss the power-deformed geometries of SPD manifolds. Based on these metries, we will develop five families of deformed SPD MLRs.

---

Table 1: Several MLRs on different geometries are special cases of our MLR.
Incorporated by Our MLR
MLR
Geometries
Requirements
Euclidean MLR (Éq. (1))
Euclidean geometry
N/A
<(App. C)
Gyro SPD MLRs [50]
AIM, LEM & LCM on S
Gyro structures
(Rem. 4.3)
Gyro SPSD MLRs [51]
SPSD product gyro spaces
Gyro structures
(App. D)
Pullback metrics from the Euclidean space
Flat SPD MLRs [16]
(a, B-LMMMMMM& & (0)-LCM on S++
/Rem. 4.3)
Ours
General Geometries
Riemannian logarithm
N/A
0 m05
0-0
(0.00 of.0) (a.)) (1.0)
(0.a.)) (2.1.0)
BWM-
20-BMM
-LM
PAM
(s. (1.0)
(a.f) (1.0)
EM
(.
10 RAAA
AIM
(0. (-1.1.)
SREM (0.0.)) - (0.5.1.0)
1EM-
(0.E.
3-LEM
0-CM LCM
Viman Variin
FILM
Figure I: Illustration on the deformation (left) and Venn diagram (right) of metries on SPD manifolds, where IEM, SREM, and 1 PAM denotes Inverse Euclidean Metric, Square Root Euclidean Metrie, and Polar Affine Metric scaled by 1/4.
4.1 Deformed geometries of SPD manifolds
Table 2: Properties of deformed metrics on SPD manifolds (0 o 0 and min(a, a + +33) > 0).
Name
Properties
(0, or, -EM
Bi- Bi-Invariance, ((x)- Invarianne,, Geodesically Completeness
(0, ot, AIM
Lie Group Left-Invariance,
Geodesically Completeness
(0,a, -MM
O(n) IIvariance
0-LCM
Lie Group Bi- Bi-Invariance, Geodesically Completeness
26-BWM
-nvaa-nne
As discussed in Sec. 2, there are five popular Riemannian metrics on SPD manifolds. These metrics can be all extended into power-deformed metrics. For metric g on 8 the power-deformed metric is defined as
1 DP (V. W) = 02 pre (VV)
S V.W E TpST
(14)
where de = pd the matrix power, and (ds).,P is the differential map. The deformed metric ğ can interpolate between LEM-like metric (0 - 0) and (0 = 1) [61]. Previous work has extended (a. -)MMM,, (a. LLEMM LCM, and BWM into power-deformed metrics and (a. -LMMMM is proven to be invariant under the power deformation [17]. We denote these metrics as (8, a, 3)-AIM [59],
(a, 3))LEM [17], 28-B BWM [61], and 6-LCM [17], respectively. The deformation of these metrics is discussed in App. 1. We further define the power-deformed metric of (a, 9)-MM by Eq. (14),
denoted as (0, or BM- We have the following for the deformation of (0, o. FMM.
Proposition 4.1. [1] (0,a, a. 3)-EM interpolates between (a, ())-LEM (6 0) and (a, 3)- (0 1).
So far, all five popular Riemannian metrics on SPD manifolds have been generalized into power- deformed families of metries. We summarize their associated properties in Tab. 2 and present their theoretical relation in Fig. 1. We leave technical details in App. E.2.
4.2 Five families of SPD multinomial logistic regressions
This subsection presents five families of specific SPD MLRs by our general framework in Thm. 3.3 and metrics discussed in Sec. 4.1. We focus on generating Aş by parallel transportation from the identity matrix, except for 20-BWM. Since the parallel transportation under 20-BWM would undermine numerical stability (please refer to App. F.2.1 for more details), we resort to newly
5

---

LEM SPD Hyperplane
Hyperplane
PEM SPD Hyperplane
BWM SPD Hyperplane
LCM SPD Hyperplane
3
2
3
3
2
Figure 2: Conceptual illustration of SPD hyperplanes induced by five families of Riemannian metrics. The black dots denote the boundary of S
developed Lie group operation [62]:
S S2 = L, L1 vS,, S2 E ST....
(15)
where L1 = Cho is the Cholesky decomposition.
Theorem 4.2 (SPD MLRs), [1] By abuse of notation, omit the subscripts k of As. and Ps. Given SPD feature S, the SPD MLRs, pvy = k I E S4+), are proportional to
(),3)-LEM eep
(log(S) log( (P)g(P),444 A)),
(16)
(0, AAMM :ep
(log( P- 55 p- )), A)
(17)
(0, o. -3--MM exp
(5 po A)(0
(18)
[LR] - - (Dog((D(R)) DID)( LAL + 1 =
0-LCM : exp
(19)
20-BWM eex
((p20 520 + (520 p20 2p20 Lyz (T.AI.T)
(20)
40
where A E STST Mo} is symmetric matrix, log(-) is the matrix logarithm, Cp(V) ) is the solution to the matrix linear system Lp PV V, known as the Lyapunov operator, Diog(.) is the diagonal element- logarithm,, I-] is the strictly lower part of square matrix, and (D-) is diagonal matrix with diagonal elements of a square matrix. Besides, log.,p p is the differential maps P, K )h) L Chol(P"), and L Chol(P29).
The Lyapunov operator in Eq. (20) requires the eigendecomposi- tion. However, the backpropagation of eigendecomposition involves 1/(a,-az) [34], undermining the numerical stability. Therefore, we propose numerically stable backpropagation for the Lyapunov operator, detailed in App. F.2.2.
As 2 x 2 SPD matrices can be embedded into R as an open cone [74], we illustrate SPD hyperplanes induced by five families of metrics in Fig. 2.
0y
Remark 4.3. Our SPD MLRs extend the existing SPD MLRs [50, 16]. The eeeudo-gyrodistance to a SPD hyperplane in [50, Thms. 2.23- 2.25] is incorporated by our Thm. 3.2, while the flat SPD MIRs under (o,, 3)MMEM and 6-LCM in [16, Cor. 4.1] are special cases of our Thm. 4.2. Furthermore, our approach extends the scope of prior work as neither [16] nor [50] explored SPD MLRs based on (0, a. EMM and 20-BWM. The gyro operations [50, Eq. (1)] implicitly requires geodesic completeness, whereas (0, a, -EM
Figure 3: Conceptual illustra- tion of a Lie hyperplane. Each pair of antipodal black dots corresponds to a rotation ma- trix with an Euler angle of n, while the green dots denote Lie hyperplane.
and 20-BWM are incomplete. As neither (8, (,,a. EEM nor 20-BWM belong to pullback Euclidean metrics, the framework presented in [16] cannot be applied to these metrics. To the best of our knowledge, our work the first to apply PEM and BWM to establish Riemannian neural networks, opening up new possibilities for utilizing these metrics in machine learning applications. Besides, neither Nguyen and Yang [50] nor Chen et al. [16] explore the deformed metrics for building SPD MLRs.

---

Table 3: Comparison of SPDNet with LogEig against SPD MLRs on the Radar dataset.
8)-AM
19.
(....F-DM DM
()-M)
26- BW-BMM
U.LCM
Architectues
Logfig MLR
(1.1.0)
(1.10)
(1.1.//)
(1.1.))
(LL..))
(0.5)
(0.25)
(1)
(0.5)
2-Block S-Block
92.88e1.0 05
94.3340,95 94.32e1.94
96.2420.55
94.329.00
93.5961.21 94.0000
95644088
94.9920.47 93.69H0.66
92.2240.83
91.4921.25
94.5940.81
93.4780.45
95.11=9822
95.1+0.84
95.8751.58
94.84411.68
93.9860.95
95, 162l.67
Table 4: Comparison of SPDNet with LogEig against SPD MLRs on the HDMO5 dataset.
(0,4,8)-AIM
(0, a. 8)-M
(L,B):LEM
BWM
0.LCM
Architectures
LogEig MLR
(1.1.))
(1.1.0)
(0.5....)/3))
(1.1,0)
(0.5)
(I)
(0.5)
1-Block
$7.42s1.31
58.0740.64
66. 66.3220.63
71.6540.8 88
56.97a0.61
70.24=0.92
63.84a1.31
65.66:40.73
2-Block
60.6920.66
60.7240.62
66.40s50877
70.56+0.39
60.69.1.00
70.4620.71
62.6111.46
65.7920.63
3-Block
60.76:0.80
61.14a0.94
7.2
70.2240.81
60.2840.91
70.20=0.91
62. 03.3342.15
65.71.0.75
Table 5: Inter-session experiments of TSMNet with different MLRs on the Hinss2021 dataset.
(0.o, o. AIMAIM
(8, or 3)--EM
(o.
29-BWM
#-LCM
Classifiers
LogEig MLR
(1.1,0)
(0.5.10.05)
(1.1.0)
(1.1.0)
(0.5)
(1)
(1.5)
Balanced Acc.
53.83+9.77
$3.36+9.92
55.27=8.68
54.4849.21
53.51+10002
55.54=7.45
55.71=8.57
56.4348.79
Table 6: Inter-subject experiments of TSMNet with different MLRs on the Hinss2021 dataset.
().MM)
(...
[o.
20-8WM
61MM
Chasities
LegDig
(1.5.1.0)
(1.5.119)))
(1.1.0)
Balaneed
49.6887,88
50,65e8.13 51.1567,83
50,0255811
51,3665.7
51.4147.98
50.2647.23
51.678873
52.936.76
54.1468.36
5 Lie multinomial logistic regression
This section introduces our Lie MLR on SO(n) based on the general RMLR framework Thm. 3.3. The Riemannian metric on SO(n) is assumed to be the invariant metric in Tab. 13.
The two ways generate A in RMLR, i.e.E.qs. (12) and (13), are equivalent on SO(n).
Lemma 5.1. [4]
TQ-P- L VP, Q SO(n).
(21)
Similar with SPD MLRs, we set Q I. The Lie MLR on SO(n) is presented in the following.
Theorem 5.2. (4] The Lie MIR on (O()) is given as
p(y = k R E SO(n)) x (log( P. S), At),
(22)
where Pa (O(n) and As E sn)).
We refer to the Riemannian hyperplanes (Eq. (5)) SO(n) as Lie hyperplanes. As SO(3) is homeomorphic to 3-dimensional real projective space RP3S [26], Fig. 3 illustrates Lie hyperplanes in the closed ball in 3 of radius т.
Experiments
We first validate our SPD) MLRs on four SPD neural networks: SPDNet [30] and TSMNet [37] for Riemannian feedforward networks, RResNet [36] for Riemannian residual networks, and SPDGCN [76] for Riemannian graph neural networks. Then, we proceed with experiments of our Lie MLR under the classic LieNet architecture [31]. The classifier in all the above networks is the LogEig MLR (matrix logarithm + FC + softmax), a Euclidean MLR on the tangent space at the identity matrix. We substitute the original non-intrinsic LogEig MLR in each baseline model with our RMLRs. Notably, the gyro SPD MLRs [50] are special cases of our SPD MLRs under the standard AIM, LEM, and LCM ((,, a, 3) (1, 0)), while flat SPD MLRs [16] are incorporated by our SPD MLRs under (a., -L-MMMM and 0-LCM. More implementation details are presented in App. G.
6.1 Experiments on the proposed SPD MLRs
In the following, we abbreviate SPD MI-R-metric as metric. For instance, (0, or 3)-AIM denotes the baseline endowed with the SPD MLR induced by (0, a, AAMAI and (1,1,0) as the value of (0, a, B))
7

---

6.1.1 Experiments on the Riemannian feedforward network
We evaluate our SPD MLRs for Riemannian feedforward networks under the SPDNet and TSMNet
backbones. Following [30, 10], on SPDNet, we use the Radar dataset [10] for radar recognition and the HDM05 dataset [44] for human action recognition. TSMNet [37] is one of the state- of-the-art methods for the EEG classification task. Following [37]. we use the Hinss2021 [28] dataset. For each family of SPD MLRs, we report the SPD MLR induced from the standard metric (0 = 1, a 1, 8 = 0), and the one induced from the deformed metric with best (0, 3, Besides, if the standard SPD MLR is already saturated, we only report the results of the standard one. Under each metric. We highlight the results in bold of our SPD MLR under the best hyperparameters. We visualize the results in App. G.I.6.
Radar: In line with [10], we evaluate our classifiers under two network architectures: 2-Block and 5-Block configurations. The 10-fold results (meanistd) are presented in Tab. 3. Note that the SPD MLR induced by standard AIM is saturated. Generally speaking, our SPD MLRs achieve superior performance against the vanilla LogEig MLR. Moreover, for most families of metrics, the associated SPD MLRs with proper (0, a, 3) outperform the standard SPD MLR, demonstrating the effectiveness of our parameterization. Besides, among all SPD MLRs, the ones induced by (a, 3)-LEM achieve the best performance.
HDM05: Following [30], three architectures are adopted: 1-Block, 2-Block and 3-Block configura- tions. The 10-fold results (meantstd) are presented in Tab. 4. Note that the standard SPD MLRs under AIM, LEM, and BWM are already saturated on this dataset. As the Radar dataset, similar observations can be made on this dataset. Our SPD MLRs can bring consistent performance gain for SPDNet, and properly selected hyperparameters can bring further improvement. Particularly, among all the SPD MLRs, the ones based on the 29-BWM and (0, a, 3) achieve the best performance. Compared to the vanilla LogEig MLR, the highest performance improvement is 14.23% %, highlighting our approach's effectiveness. Notably, since 20-BBM and (0, o, 33-EM are geodesically incomplete and not pulled back from Euclidean space, the SPD MLR under these two metrics can not be derived by the framework of gyro or flat MLR. This contrast confirms the applicability of our theoretical framework broader range of geometries.
The results (mean=std) of leaving 5% out cross-validation are reported in Tabs. and 6. Once again, intrinsic classifiers demonstrate improved performance compared to the LogEig MLR in both inter-session and inter-subject scenarios. Besides, the SPD MLRs based on 6-LCM achieve the best performance, outperforming the vanilla classifier by 2.6 % for inter-session and by 4.46% % for inter-subject. This finding highlights the versatility of our framework.
Hinss2021:
Table 7: Comparison of LogEig against SPD MLRs under the RResNet architecture.
Datasets
LogEigMLR
(f, (f)AIM
(6, (8,o, 1M,.)--MM EM
(a, 3)-LMM
28-BWM
6-LCM
HDM05
58.17 + 2.07
60.23 # 1.26
71.89 + 0.60 (1 13.72)
59.44 + 0.87
69.85 0.23
65.76 + 0.96
NTU60
45.22 # 1.23
48.94 + 0.68
52.24 + 1,25
46.99 = 0.41
50.56 + 0.59
53.63 = 0.95 (1 8.41)
6.1.2 Experiments on the Riemannian residual network Following [36], we use the HDM05 and NTU60 [55] datasets on the RResNet backbone. For the hyperparameter (f,a, (3) in our SPD MLRs, we borrow the best ones from Tab. 4. Tab. 7 reports the 10-fold and 5-fold results on the HDM05 and NTU datasets. The SPD MLRs still consistently outperform the vanilla LogEig MLR. Besides, similar to the SPD MLRs under the SPDNet backbone for action recognition (Tab. 4), the SPD MLR based on O-LCM, 20-BWM, or (0, a. BEM outperforms the vanilla LogEig MLR by large margin. Especially, the highest performance improvement is 13.72% and 8.4% on these two datasets.
6.1.3 Experiments on the Riemannian graph network
We use SPDGCN [76] as the backbone network for the Riemannian graph network. Following [76].
we use the Disease [2], Cora [54], and Pubmed [46] datasets for node classification. The 10-fold average and maximum results of the vanilla LogBig MLR against our SPD MLR with best (0, a, .) are reported in Tab. 8. Similar to the previous results, our SPD MLRs outperform the LogEig MLR. Besides, the SPD MLR based on (a. - generally achieves the best performance for SPDGCN.
6.1.4 Ablations of SPD MLRs on direct classification
For a more straightforward comparison, we compare LogFig against our SPD ML.s for direct classification. We adopt the Radar, HDM05, and Hinss2021 datasets. We follow the preprocessing of
8

---

Table 8: Comparison of LogEig against SPD MLRs under the SPDGCN architecture.
Disease
Cora
Pubmed
Classifiers
Mc
Max
Mcan+STD
Max
Mcan=STD
Max
LogEig MLR
90.55 4.83
96.85
78.04 # 1.27
79.6
70.99 + 5.12
77.6
(8, oz AMM
94,84 2.27
98.43
79.79 1.44
81.6
77.83 + 1.08
80
(0, -3)-MM
90.87 5.14
98.03
79.05 1.23
81
79.5
(a, LEM
96.33 + 2.19
98.82
79.89 + 0.99
81.8
78.16 + 2.41
79.5
24-BWM
91.93 3.64
96.85
73.46
2.18
77.7
73.22 4.06
78.1
6-LCC
93.01 2.14
98.43
77.59
1.20
80.1
74.46
5.81
78.9
Table 9: Comparison of LogEig against SPD MLRs for direct classification.
Hinss2021
Classifiers
Radar
HDM05
Inster-session
Inster-subject
LogEig MLR
91.93 + 1,30
48.43 + 1.25
39.76 + 7.60
44.66 + 7.17
(0.a, A.M
95.21 + 0.81
49.17 1,08
41.14 # 7.26
45.89 6.52
00,,,))-EM 3J-EM
92.25 + 1.20
61,60 = 0.69
45.78 8.51 (1 6.02)
45.84 + 4.75
(a, 8)-LEM
95.09 0.57
49.05 0.91
40.88 = 7.46
46.02 + 5,96 (f 1.36)
20-BWM
94.89 + 0.41
66.77 + 1.34 (1 18.34)
44.84 = 8.00
45.21 + 7.44
0-LCM
95.67 0.61 (1 3.74)
58.66 + 0.51
43.17 = 6.21
45.10 + 6.20
SPDNet and TSMNet to model features into the SPD manifold and directly use LogEig or our SPD MLRs for classification. The average results are presented in Tab. 9. The hyperparameters (0, a, 3) are borrowed from Tabs. 3 to 6. Our SPD MLRs consistently outperform the vanilla LogEig MLR. Particularly on the HDM05 dataset, the highest performance improvement by our SPD MLRs 18.34% surpassing the non-intrinsic LogEig MLR by large margin. Ablations on model efficiency are also discussed App. G.1.5.
6.2 Experiments on the proposed Lie MLR
Table 10: Results of LogEig MLR against Lie MLR under the LieNet architecture
G3D
HDM05
Classifiers
STD
Max
Mca
Max
LogEig MLR
87.91
89.73
76.92+1.27
79.11
Lie MLR
89.13+1.7
92.12
78.24+1.03
80.25
the We apply our Lie MLR into the classic SO(n) network, LieNe [31], where features
Lie group of SO(3) x x SO(3). Following LieNet [31]. we use G3D [6] and HDM05 [44] datasets. We also extend the Riemannian optimization package geoopt [4] into SO(3). allowing for Riemannian optimization. We find that Riemannian SGD performs best for LieNet. Tab. 10 presents the 10-fold average results of LieNet with or without Lie MLR. Note that on the HDM05 datasets, the LieNet might fail to converge, fluctuating between the validation accuracy of 70% 75%. Therefore, we select 10-fold best performance out of 20-fold experiments. It can be observed that Lie MLR can improve the performance of LieNet. Besides, our Lie MLR can also improve the training stability. On the HDMOS dataset, LieNet fails to converge out of 20 folds. However, when endowed with our Lie MLR, LieNet+LieMLR only encounters convergence failures in 2 folds.
7 Conclusions
This paper presents novel and versatile framework for designing RMLR for general geometries, with specific focus on SPD manifolds and (n)).. On the SPD manifold, we systematically explore five families of Riemannian metrics and utilize them to construct five families of deformed SPD MLRs. On (O(n), we develop the Lie MLR for classifying rotation matrices. Extensive experiments demonstrate the superiority of our intrinsic classifiers. We expect that our work could present promising direction for designing intrinsic classifiers on diverse geometries.

---

Acknowledgments and Disclosure of Funding
This work was partly supported by the MUR PNRR project FAIR (PE00000013) funded by the NextGenerationEUU the EU Horizon project ELIAS (No. 101 120237,, donation from Cisco, the National Natural Science Foundation of China (62306127), the Natural Science Foundation of Jiangsu Province (BK20231040) and the Fundamental Research Funds for the Central Universities (JUSRP124015), The authors also gratefully acknowledge the financial support from the China Scholarship Council (CSC).
References
[1] P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008
[2] Roy M Anderson and Robert M May. Infectious diseases of humans: dynamics and control. Oxford University Press, 1991.
[3] Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and simple computa- tions on tensors with Eug-iicci metrics. PhD thesis, INRIA, 2005.
[4] Gary Bécigneul and Octavian-Euge -ugenn Ganea. Riemannian adaptive optimization methods. arXiv preprint 1810.00766 2018.
[5] Rajendra Bhatia, Tanvi Jain, and Yongdo Lim. On the Bures- Wasserstein distance between positive definite matrices. Expositiones Mathematicae, 37(2):165-19)) 2019.
[6] Victoria Bloom, Dimitrios Makris, and Vasilcios Argyriou. G3D: A gaming action dataset and real time action recognition evaluation framework. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 7-12. IEEE, 2012.
[7] Silvere Bonnabel, Anne Collard, and Rodolphe Sepulchre. Rank-preserving geometric means of positive semi-definite matrices. Linear Algebra and its Applications, 438(8):3202-3211 2013.
[8] Nicolas Boumal and P-A Absil. A discrete regression method on manifolds and its application data on SO(n). IFAC Proceedings Volumes, 44(1):2284-2289, 2011.
[9] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond Euclidean data. IEFE Signal Processing Magazine, 34 (4):18 42, 2017.
[10] Daniel Brooks, Olivier Schwander, Frédéric Barbaresco, Jean-Yves Schneider, and Matthieu Cord. Riemannian batch normalization for SPD neural networks. In Advances in Neural Information Processing Systems, volume 32, 2019.
[11] James Cannon, William Floyd, Richard Kenyon, Walter Parry, et al. Hyperbolic geometry. Flavors of geometry, (19-5-115)) 1997.
[12] Rudrasis Chakraborty, Chun-Hao Yang. Xingjian Zhen, Monami Banerjee, Derek Archer, David Vaillancourt, Vikas Singh, and Baba Vemuri. A statistical recurrent model on the manifold of symmetric positive definite matrices. Advances in Neural Information Processing Systems, 31, 2018.
[13] Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domain- specific batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7354-7362, 2019.
[14] Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, and Josef Kittler. Hybrid Riemannian graph-embedding metric learning for image set classification. IEEE Transactions on Big Data, 2021.
[15] Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, and Josef Kittler. Rieman- nian local mechanism for SPD neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, pages 7104-7112, 2023.
10

---

[16] Ziheng Chen, Yue Song. Gaowen Liu, Ramana Rao Kompella, Xiaojun Wu, and Nicu Sebe. Riemannian multinomial logistics regression for SPD neural networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024.
[17] Ziheng Chen, Yue Song, Yunmei Liu, and Nicu Sebe. A Lie group approach to Riemannian batch normalization. In The Twelfth International Conference on Learning Representations, 2024.
[18] Ziheng Chen, Yue Song, Xiao-Jun Wu, Gaowen Liu, and Nicu Sebe. Understanding matrix function normalizations covariance pooling through the lens of Riemannian geometry. arXiv preprint arXiv:2407.10484 2024.
[19] Ziheng Chen, Yue Song, Xiao-Jun Wu, and Nicu Sebe. Product geometries Cholesky manifolds with applications to SPD manifolds. arXiv preprint arXiv:2407.02607, 2024.
[20] Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, and Nicu Sebe. Adaptive Fuclidean metrics for SPD matrix learning. IEEE Transactions Image Processing, 2024.
[21] Manfredo Perdigao Do Carmo and Flaherty Francis. Springer, 1992.
Riemannian Geometry, volume 6.
[22] Ian L Dryden, Xavier Pennec, and Jean-Mare Peyrat. Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging, arXiv preprint 10Xiv:100, 2010.
[23] Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks, Advances in Neural Information Processing Systems, 31, 2018.
[24] Alexandre Gramfort. MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience, 2013.
[25] Brian Hall and Brian C Hall. Lie groups, Lie algebras, and representations. Springer, 2013.
[26] Richard Hartley, Jochen Trumpf, Yuchao Dai, and Hongdong Li. Rotation averaging. Interna- tional Journal of Computer Vision, 103:267-305, 2013.
[27] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 770-778, 2016.
[28] Marcel F. Hinss, Ludovic Darmet, Bertille Somon, Emilie Jahanpour, Fabien Lotte, Simon Ladouce, and Raphaëlle N. Roy. An EEG dataset for cross-session mental workload estimation: Passive BCI competition of the Neuroergonomics Conference 2021, 2021.
[29] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, (8):1735-1780, 1997.
[30] Zhiwu Huang and Luc Van Gool. A Riemannian network for SPD matrix learning. In Thirty-furst AAAI Conference on Artificial Intelligence, 2017.
[31] Zhiwu Huang, Chengde Wan, Thomas Probst, and Luc Van Gool. Deep learning Lie groups for skeleton-based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6099-6108, 2017.
[32] Zhiwu Huang, Jiqing Wu, and Lue Van Gool. Building deep networks on Grassmann manifolds. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[33] Catalin lonescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix backpropagation for deep networks with structured layers. In Proceedings of the IEEE International Conference on Computer Vision, pages 2965-2973. 2015.
[34] Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Training deep networks with structured layers by matrix backpropagation. arXiv preprint arXiv: (t0.:::500078...0 009.07888, 2015.
[35] Vinay Jayaram and Alexandre Barachant. MOABB: trustworthy algorithm benchmarking for BCIs. Journal of Neural Engineering, (56))))0015()))666666 1566)106601,, 2018.

---

[36] Isay Katsman, Eric Ming Chen, Sidhanth Holalkere, Anna Asch, Aaron Lou, Ser-Nam Lim, and Christopher De Sa. Riemannian residual neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[37] Reinmar Kobler, Jun-ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD domain- specific batch normalization to crack interpretable unsupervised domain adaptation in EEG. Advances in Neural Information Processing Systems, 35:6219-665 6235, 2022.
[38] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
[39] Guy Lehanon and John Lafferty. Hyperplane margin classifiers the multinomial manifold. In Proceedings of the Twenty-first International Conference on Machine Learning, page 66, 2004.
[40] John M Lee. Introduction smooth manifolds. Springer, 2013.
[41] Zhenhua Lin. Riemannian geometry of symmetric positive definite matrices via Cholesky decomposition. SIAM Journal on Matrix Analysis and Applications, 1353-31353-3370 2019.
[42] Federico I. López, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Vector- valued distance and Gyrocalculus on the space symmetric positive definite matrices. Advances in Neural Information Processing Systems, 01118800-83666 2021.
[43] Ha Quang Minh. Alpha Procrustes metrics between positive definite operators: a unifying formulation for the wasse-Wassrnntein and LoggEuciidean/Log HawpelLLoo--ii---SShieeee Hilbert-Schmid metrics. Linear Algebra and its Applications, 636:25-68, 2022.
[44] Meinard Müller, Tido Röder, Michael Clausen, Bernhard Eberhardt, Björn Krüger, and Andreas Weber. Documentation mocap database HDM05. Technical report, Universität Bonn, 2007.
[45] Richard M Murray, Zexiang Li, and S Shankar Sastry. A mathematical introduction to robotic manipulation. CRC press, 2017.
[46] Galileo Namata, Ben London, Lise Getoor, Bert Huang, and Edu. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, volume page 2012.
[47] Xuan Son Nguyen. Geomnet: A neural network based on Riemannian geometries of SPD matrix space and Cholesky space for 3D skeleton-basee interaction recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 13339, 2021.
[48] Xuan Son Nguyen. The Gyro-structure of some matrix manifolds. In Advances in Neural Information Processing Systems, volume 35, pages 26618-2666 2022.
[49] Xuan Son Nguyen. A Gyrovector space approach for symmetric positive semi definiefinite matrix learning. In Proceedings of the European Conference Computer Vision, pages 52-68, 2022.
[50] Xuan Son Nguyen and Shuo Yang. Building neural networks on matrix manifolds: A Gyrovector space approach. arXiv preprint arXiv:2305.0455 2023.
[51] Xuan Son Nguyen, Shuo Yang, and Aymeric Histace. Matrix manifold neural networks++. In The Twelfth International Conference on Learning Representations, 2024.
[52] Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computing. International Journal of Computer Vision, 66(1):41-66, 2006.
[53] Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan- Yen Lo, Justin John- son, and Georgia Gkioxari. Accelerating 3D deep learning with PyTorch3D. arXiv preprint arXiv:2007.0850/ 2020.
[54] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29(3):93-93, 2008.
12

---

[55] Amir Shahroudy, Jun Liu, Tian-Tsong Ng. and Gang Wang. NTU RGB+ D: A large scale dataset for 3D human activity analysis. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 1010-1019, 2016.
[56] Ondrej Skopek, Octavian-Eugen Ganea, and Gary Bécigneul. Mixed-curvature variational autoencoders. arXiv preprint /9Xiv:191.084//1 2011..
[57] Yue Song, Nicu Sebe, and Wei Wang. Why approximate matrix square root outperforms accurate SVD in global covariance pooling? In Proceedings of the IEFE International Conference on Computer Vision, pages 115-1123, 2021.
[58] Yue Song, Nicu Sehe, and Wei Wang. On the eigenvalues of global covariance pooling for fine- grained visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3554-3566, 2022.
[59] Yann Thanwerdas and Xavier Pennec. Is affine-invarianee well defined on SPD matrices? principled continuum of metrics. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27-29, 2019, Proceedings 4, pages 502-510. Springer, 2019.
[60] Yann Thanwerdas and Xavier Pennec. Exploration of balanced metrics on symmetric positive definite matrices. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27-29, 2019, Proceedings 4, pages 484 493. Springer, 2019.
[61] Yann Thanwerdas and Xavier Pennec. The geometry of positive definite matrices. Differential Geometry and its Applications, 81:101867, 2022.
metrics symmetric
[62] Yann Thanwerdas and Xavier Pennec. Theoretically and computationally convenient geometries on full-rank correlation matrices. SIAM Journal on Matrix Analysis and Applicatioons, 43(4):
1851-1872, 2022.
[63] Yann Thanwerdas and Xavier Pennec. O (n)-invariant Riemannian metrics on SPD matrices.
Linear Algebra and its Applications, 661:163-201, 2023.
[64] Loring W. Tu. An introduction to manifolds. Springer, 2011.
[65]
Abraham A Ungar. Analytic hyperbolic geometry: Mathematical World Scientific, 2005.
forhematicalluunndioon foundations
and applications.
[66] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, volume 30, 2017.
[67] Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3D skeletons as points in Lie group. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 588-595, 2014.
[68] Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang, and Peihua Li. Deep CNNs meet global covariance pooling: Better representation and generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8):2582-251) 2020.
[69] Rui Wang, Xiao-Jun Wu, and Josef Kittler. SymNet: A simple symmetric positive definite manifold deep learning method for image set classification. IEEE Transactions on Neural Networks and Learning Systems, (3(5):2208-2222 2021.
[70] Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. DreamNet: A deep Riemannian manifold network for SPD matrix learning. In Proceedings of the Asian Conference on Computer Vision, pages 3241-3257, 2022.
[71] Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. Learning a discrimina- tive SPD manifold neural network for image set classification. Neural networks, 151:94-110, 2022.
13

---

[72] Rui Wang, Chen Hu, Ziheng Chen, Xiao-Jun Wu, and Xiaoning Song. A Grassmannian manifold self-attention network for signal classification. In Proceedings of the Thirty-Thire
International Joint Conference Artificial Intelligence, pages 5099-5107, 2024.
[73] Rui Wang, Xiao-Jun Wu, Ziheng Chen, Cong Hu, and Josef Kitter. SPD manifold deep metric learning for image set classification. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[74] Or Yair, Mirela Ben-Chen, and Ronen Talmon. Parallel transport the cone manifold of SPD matrices for domain adaptation. IEEE Transactions on Signal Processing, 6777)11977-1811, 2019.
[75] Hongwei Yong, Jianqiang Huang, Deyu Meng, Xiansheng Hua, and Lei Zhang. Momentum batch normalization for deep learning with small batch size. In Computer Vision-ECCe ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16, pages 124-240. Springer, 2020.
[76] Wei Zhao, Federico Lopez, J Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, and Steve Trettel. Modeling graphs beyond hyperbolie: Graph neural networks in symmetric positive definite matrices. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 122-139, Springer, 2023.
14

---

Appendix Contents
A Limitations and future avenues
17
Preliminaries
17
B.1 Notations
17
B.2 Brief review of Riemannian geometry
17
B.3 Basic geometries of SPD manifolds
18
B.4 Basic geometry of rotation matrices
18
RMLR natural extension of the Euclidean MLR
19
D Gyro SPSD MLR special cases of our RMLR
19
E Theories on the deformed metrics
21
E.1 Limiting cases of the deformed metrics
21
E,2 Proof of the properties of the deformed metrics (Tab. 2)
22
Computational details on the SPD MLR under power-deformed BWM
23
E1 Matrix square roots in the SPD MLR under power-deformed BWM
23
F22
Numerical stability of the SPD MLR under power-deformed BWM
23
F2211 Instability of parallel transportation under power-deformed BWM
23
F.2.2
Numerically stable methods for the SPD MLR under power-deformed BWM
23
G
Implementation details and additional experiments
24
G.I
Additional details and experiments on the SPD MLRs
24
G.1.1 Basic layers SPDNet and TSMNet
G.1.2 Datasets and preprocessing
25
G.1.3
Implementation details
25
G.1.4
Hyper-purameters
26
G.1.5 Model efficiency
26
G.1.6
Visualization
27
G.2
Additional details and experiments on the Lie MLR
27
G.2.1
Basic layers in LieNet
27
G.2.2
Datasets and preprocessing
28
G.2.3
Implementation details
28
G.3 Hardware
28
15

---

H Proofs
28
H.1 Proof of Thm. 3.2
28
H.2 Proof of Thm. 3.3
29
H.3 Proof of Prop. 4.1
29
H.4 Proof of Thm. 4.2
29
H.5 Proof of Lem. 5.1
32
H.6 Proof of Thm. 5.2
32
16

---

A Limitations and future avenues
Limitation: Recalling our RMLR in Eq. (11), our RMLR might over weer-paameterized. In our RMLR, each class would require a Riemannian parameter Ps and Euclidean parameter Aç. Consequently, as the number of classes grows, the classification layer would become burdened with excessive parameters. We will address this problem in future work.
Future work: We highlight the advantage of approach compared to existing methods that our framework only requires the Riemannian logarithm, which is commonly satisfied by various manifolds encountered in machine learning. Therefore, as a future avenue, framework offers various possibilities for designing intrinsic classifiers for neural networks on other manifolds.
B Preliminaries
B.1 Notations
We briefly summarize the notations in Tab. 11 for better clarity.
Table 11: Summary of notations.
Notation
Explanation
(MM abbreviated .M
Riemannian manifold
TpM
The tangent space at P C M
Se(
or (r
The Riemannian metric at P M
I -le
norm induced by (- 1) T.-
Logr
The Riemannian logarithm at P
T
The Riemannian parallel transportation ulong the geodesic connecting P and Q
Hap
The Euclidean hyperplane
Ha
The Riemannian hyperplanc
A Lic group operation
(M
A Lie group
Pe
The group inverse of P under @ The Lie group left translation by E E.M The differential map of the smooth map E
Le
Jap
I'g
The pullback metric by f from
S
The manifold
SO(n)
The special orthegonal group
S"
The Euclideun space of symmetric matrices
Cholesky manifold, the of
triangular matrices with positive diagonal elements
C ( o
The Euclidean space of
real lower triagular matrices
The standard Frobenius inner product
ST = {(o, 8) O(n)-invariant Euclidean inner product
min(a, + n38) 00
LES
The Riemannian metric (a, 9)-LEM
The Riemannian metric of {a, 3MMMM
gln gtae
The Riemannian
of 8)-EM
The Riemannian metric of BWM
goe
The Riemannian metric of LCM
log(-)
matrix logarithm Cholesky decomposition
Chal
Dlog(.)
The diagonal element wise logarithm
The strietly lower tringulaa part square matrix
DC-)
A diagonal matrix with diagonal elements from square matrix
CrHl
The Lyapunov operator
() or )o() 2())
The matrix power
Retum orthogonal matrix by QR decomposition
skew(.)
skew A) = A-A"
B.2 Brief review of Riemannian geometry
Intuitively, manifolds are locally Euclidean spaces. Differentials are the generalization of derivatives in classic calculus. For more details on smooth manifolds, please refer to [64, 40J. Riemannian manifolds are the manifolds endowed with Riemannian metrics, which can be intuitively viewed as point-wise inner products.
Definition B.1 (Riemannian Manifolds), A Riemannian metric M is smooth symmetric covariant 2-tensor field on M, which is positive definite at every point. A Riemannian manifold is pair (M,g). where M is smooth manifold and g a Riemannian metric.
W.l.o.g., we abbreviate (M,g) .9) as M. The Riemannian metric g induces various Riemannian opera- tors, including the geodesic, exponential, and logarithmic maps, and parallel transportation. These
17

---

operators correspond to straight lines, vector addition, vector subtraction, and parallel displacement in Euclidean spaces, respectively [52, Tabel 1J. plethora of discussions on Riemannian geometry can be found in [21].
When a manifold M is endowed with a smooth operation, it is referred to as a Lie group.
Definition B.2 (Lie Groups). A manifold is Lie group, if it forms group with group operation @ such that m1 y) + x @ y and i(x) > o are both smooth, where Ti is the group inverse of z.
Lastly, we review the definition of pullback metric, a common technique in Riemannian geometry. This idea is natural generalization of bijection from set theory.
Definition B.3 (Pullback Metrics). Suppose M.N are smooth manifolds, g is a Riemannian metric on N,, and f : M + .N is smooth. Then the pullback of g by I is defined goint-wisely,
(V2)), (V')),(V,, V22 (VVV)
(23)
where p E M. M. (.) is the differential map of f at p. and V, E T,M. If f'g is positive definite, a Riemannian metric on M, which is called the pullback metric defined by f.
Table 12: The associated Riemannian operators and properties of five basic metries on SPD manifolds.
Metrics
Lesp 2
Peoperizes
[ (kx))) kuiP))
Ofi' Ievorieneee
Complexaos
Lie Group Left Invuriancs Oln) Invuriaess Gontrcsally Comploaoee
()-AMM
palkigg 11-11g--111. pua
(n (w.0)RMM
Q-P
Ofwj-lecrtaace
ECM
Eo, R. Ex P. (Chol") [_K] [4] moo Regit (Chet" [P]+ ((X))()))) (n())
Goodesiallly Conpleneess
Lie Group Bislavaranees
8WM
(PQ) (ppyls 27
(rr or
Olwi-beoriizee
B.3
Basic geometries of SPD manifolds
Let S be the set of symmetric positive definite (SPD) matrices. As shown in [3], S is an open submanifold of the Euclidean space S of symmetric matrices. There are five kinds of popular Riemannian metrics S Affine Invariant Metric (AIM) [52], Log-Euclidean Metric (LEM) [3], Power-Euclidean Metrics (PEM) [22], Log-Cholesky Metric (LCM) [41], and Bures- Wasserstein Metric (BWM) [5]. Note that, when power equals 1, the PEM is reduced to the Euclidean Metric (EM). The standard LEM, AIM, and EM have been generalized into parametrized families of metrics. We define ST {(a, B) IR22 min(a, & + n3) 0). and denote the (inn-invrr Euclidean
metric on Sth [63] as
(V, WV(a)) a(,, W) + 3 tr(V) tr(W),
(24)
where (a. E ST. and (..) is the Frobenius inner product. By (mn)-invariant Euclidean metric on S", Thanwerdas and Pennec [63] generalized AIM. LEM, and EM into two-parameters families of O(m)-invariant metrics, i.e., (a.,8)) 3)-IM,, (o, B)-LEM, and (a,, 3)- with (a, B) E ST. We denote the metric tensor of (a, -MM.,, (a, LEM,, B)-MM, LCM, and BWM as g ((a.)))AIM, (o. 8.. gL.MM and g BWM respectively.
For any SPD points P, Q S and tangent vectors V.W E TTS we follow the notations in Tab. 11 and further denote V Chol,, W Chol_,p For parallel transportation under the BWM, we only present the case where P.Q are commuting matrices, i.e.P = U and Q = UAUT. We summarize the associated Riemannian operators and properties in Tab. 12. Although there also exist other metrics on SPD manifolds [60, 61, 63]. their lack of closed-form Riemannian operators makes them problematic be applied in machine learning.
Chol P. and K Chol Q.
B.4
Basic geometry of rotation matrices
Table 13: The associated Riemannian operators on Rotation matrices.
Operators
9u(A1, Az)
Logg (A))(A) Projection Map Il)(U)
Retraction of A T,)O(n) R
Expression
(A1, Az)
log(RTs) $)
skew R U)
Q = Q(R + RA)
18

---

We denote R SO(n), A1, A₂ (RSO(n), U E R skew(A) A-A and 2(-) as the function return an orthogonal matrix by QR decomposition. There are two equivalent representations for the tangent vector on (O(n). In this paper, use the Lie algebra representation, (e..TR = so(n) as the set of skew-symmetrie matrices. We summarize all the necessary Riemannian ingredients for (O(n) in Tab. 13.
For the specific case of R E SO(3), can be represented by the Euler angle and axis [26, Sec. 3.2]:
tr R -
(R) arccos
(25)
R(3,2) R(2,3)
w (R) 2 sin (0 (R))
R(1.3) R(3.) 1)
(26)
R(2,1) R(1,2)
Besides, the matrix logarithm on SO(3) can be calculated without decomposition [45, Ex. A.14]:
if (R)
log( [t)
8(R) RR)
otherwise
(27)
2 sin O (sn RR)
where 0 is the Euler angle. Obviously, the matrix logarithm is related to the Euler angle and axis when 0 of O.
natural extension of the Euclidean MLR Proposition C.1. When M = R is the standard Euclidean space, the RMLR defined in Thm. 3.3 becomes the Euclidean MLR Eq. (1).
C RMLR
Proof. On the standard Euclidean space R", Logg x = - y. Vz, y E R². Besides, the differential maps of left translation and parallel transportation are the identity maps. Therefore, given x, Pk RR" and E R2" /{0] 2 ToR" /(0}. we have
ply = x E RR) a exp((Log,) x, as)p)); a exp((r pk: ux)), o exp((2x, ax) bb),
(28)
(29)
(30)
where by = pt).
D Gyro SPSD MLR as special cases of our RMLR
Gyro SPSD MLR [51] is derived by the product of the Grassmannian and SPD gyro spaces. This section will show that the gyro SPSD MLR is the special case of our RMLR on the product geometry of the SPSD manifold. We first review some necessary results about gyro SPSD MLR and then show the equivalence.
Following the notations in [51], denote the Grassmannian with canonical metric under the projector and ONB perspective as Gr(p, n) and Gr(p, n). respectively. The space n x n SPSD matrices with fixed rank p. denoted as S forms SPSD manifold [7]. As shown in [7, 51], the SPSD manifold is a product space, St n. S Gr(p.n) n) x Sh In other words, every P e S.p can be decomposed as P U with Up E Gr(p, n) and Sp E S +. We further denote S as the SPD manifold with metric 9, where g could be AIM, LEM, and LCM. As shown in [51], the gyro space in S+ n.p can be defined by the product of gyro spaces of Gr(p, n) and S By this product structure, Nguyen al. [51] proposed the SPSD Pseudo-gy soaudd--yyodittncee to a hyperplane.
Definition D.I. (SPSD Hypergyroplanes [51]) Let P, W Gr(p, n) ST. Then hypergyroplanes in structure space Gr(p.n) x St. are defined as
Hped.g = {o e Gr(p.n) n) x ST. (Oped.g P @pod, Q, -WW o}
(31)
where Sped.y and (. (.)ped.g are gyro addition and gyro inner product, which are defined in [51].
19

---

Theorem D.2. (SPSD Eseudo-gyrodisig [51]) Let W = P = (p) = (Ux, Sx) E Gr(p.n) S+4 and H be a hypergyroplane in structure space Gr(p, n) x S Then the somiddo-gyodistancee from X to HPrp is given by
(f Up Ux) y "" + ( Bg Sx Sw) v (| 1 11 (ISw
d (x, (x,1
(32)
where 11-1p" and I1-119 are the gyro norms on the Grassmann and SPD [51], and (,)a and (,)9 are gyro inner products [51]. Ggr and Gg are gyro additions Gr(p, n) and 444
Denoting gd as the canonical metric on Gr(p, n) and g as AIM, LEM, or LCM, we can prove that Thm. D.2 is the special case of our Thm. 3.2.
Theorem D.3. Under the product metric grdd. Ag x g. the Riemannian hyperplane Eq. (5) on the SPSD manifold equals the SPSD hypergyroplane in Def. D.I. Similarly, the Riemannian margin distance in Thm. 3.2 on the SPSD manifold equals SPSD Rseudo-groudstance in Thm. D.2.
Proof. Following the notations Def. D.I and Thm. D.2, we further denote P = UP Q UQSQUq. W SW and X UxSXUX with Up, Up, Uw, Ux E Gr(p, n) and Sp, Sq. Sx Sx E S Ip is the p x identity matrix. Tp.n = (Tp, 0)T is the gyro identity on Gr(p.n). [3r, Log" -r- and 6)0,, are Riemannian parallel transport along a geodesic, logarithm and Riemannian metric at Up on Gr(p.n). rs, Log", and {,)), are Riemannian parallel transport along geodesic, logarithm and Riemannian metric at Sp on S pesd,g, Log"*d. and (, (.yeed are Riemannian parallel transport along geodesic, logarithm and Riemannian metric on Sttpp Gr(p, n) x S
First, we show that the SPSD) hypergyroplane equals our Riemannian hyperplane in Eq. (5). We have the following
Spsd. @psdii Q, W. pad.9
UP Ogr UQ Uw) + (0,Sp 0g Sq. Sw)
(33)
X(Loüü UQ. Ao + (Log3,, Sq. Asw 1s
(3)
(Log nod Q, A)
where AUw
oor
or +Up (Togi,, (Uw)). Asw = T4,-sP (Logi, (5w) and =
SP. with @ as the Cartesian product. The above (Aow, Asw) E Ttt = Tu, Gr(p, P) @ St4 derivation comes from the following.
The definition of gyro addition, gyro inverse, and gyro inner product on the SPSD manifold [51, Sec. 3.3].
(2) The proof of [51, Prop. 3.2] indicates that similar results also hold on the Grassmannian. Combining Prop. 3.2 and its counterparts in the Grassmannian, one can obtain the equation.
(3) The Riemannian product geometry.
By the product geometry of the SPSD manifold, can immediately get
Ā = (Aow Asw)
peed.g
pod.9 ly.n 1 (Log) (W)
(34)
where Lp, In is the gyro identity the SPSD manifold.
20

---

Next, we show the equivalence between SPSD soauddo-yrodistancee and our Riemannian margin distance:
! (ppd Opsd. X, Wypod.
d (x, 11..p
||W]pedd
[Log X, Appads
2
||
(oogpedg X, Aypad
3
psd,g
Loggoch (W)
Ip.
(35)
(Logr ood
X, A
ped.0
Tosd.g
(Loglpn (W)
p.
(Logp padd X, Ayp Aypads
5
ped.a
6
d(s, H.P)
(1) The definition of gyro addition, gyro inverse, gyro inner product, and gyro the SPSD manifold.
Eq. (33).
(3) The definition of SPSD gyro norm [51].
(4) Riemannian parallel transportation maintains the norm of the tangent vector [21, Def. 3.1]
(5) Eq. (34)
(6) Thm. 3.2
Remark D.4. We make the following remark w.r.t gyro and our MLR on the SPSD manifold.
Eq. (34) indicates that when generating À in our RMLR by parallel transporting a tangent the initial velocity of W Eq. (32).
St., Á vector A E St.
2. Putting grrudo-gyrodistacee and Riemannian margin distance into Eq. (4), one can get gyro MLR and our Riemannian MLR. Therefore, Thm. D.3 indicates the equivalence of the gyro MLR with our RMLR on the SPSD.
3. As g required to induce gyro structures, the metric g in gyro SPSD MLR is confined within AIM, LEM, and LCM. However, our SPSD MLR can the product space of the Grassmannian and SPD manifold under other metries, such as BWM and PEM, framework does not require gyro structures.
E
Theories on the deformed metrics
E.1 Limiting cases of the deformed metries
Thanwerdas and Pennec [59] generalized (o. 3))MMMMMM into three-parameed families of metrics by power deformation, i.e., a, A)-AI.. The family of (8, a, AAM comprises (a, -)-MM for 0 and approaches (a, LEM with 0 0 [59].
Chen al. [17] extended LCM and (a, into power-deformed metrics, denoted as (0, a, (B)-LEM and (-LCM. The authors show that (0, o, ())-LMM is equal to (a,8)-LEM, and 0-LCM interpolates between ğ-LEM (0 + and LCM 1), with ğ-LEM defined
1 (Vi 12) P } (V, V2) 1
- (DVV)) VV, E TpSt
(36)
21

---

where V. = log., p (V;) with log,,r P the differential map of matrix logarithm, and DVV)) is diagonal matrix consisting of the diagonal elements of V,
Thanwerdas and Pennec [61] identified the Proca-trssss metric [43] with power-deformed BWM, denote 2M. Similarly, 20-BWM becomes BWM with 0 0.5 [61]. We further show the limiting case of 20-BWM under + O.
Proposition E.1. 20-BWM tends to be (1, 0)-LEM with 0 + O.
Before starting the proof, we first recall well-known property of deformed metrics [61].
Lemma E.2. Let da ogg be the deformed metric on SPD manifolds pulled back from 9 by the matrix power Φg and scaled by Φz. Then when tends to for all P E S4+ and all V E TTS$ we have
[/2 (000)) - g1 (log...,(( (VV) (V)).
(37)
Now, we present our proof for the limiting cases of deformed metries.
Proof of Prop. E.I. First, we have
WWM (V,V) - (V.
(38)
By Lem. E.2, we have the following:
2W
(V.V) 0 gp wwm (log., (V),, log, (V))
9P
= (Vo),,,((),,ogg,
(39)
( .0--LEMM DP (V,V).
E.2 Proof of the properties of the deformed metrics (Tab. 2)
In this subsection, prove the properties presented in Tab. 2. We first present useful lemma and then present our detailed proof. This lemma will be useful in the proof our SPD MLRs as well.
Lemma E.3. Supposing a Riemannian manifold {M, g) and a positive real scalar >> O, the scaling metric ag over M shares the same Riemannian logarithmic & exponential maps and parallel transportation with g.
Proof. Since the Christoffel symbols of ag identical to those of g, the geodesics and parallel transportation under both og and remain unchanged. The equivalence of geodesics implies that the Riemannian exponential maps are the same for ag and g. As the inverse of the Riemannian exponential maps, the Riemannian logarithm maps under ag and are also identical.
According to Lem. E.3, the geodesic completeness is independent of the scaling factor > O. By the definition of (n))- left-, right-, and bi-invariance, these invariant properties are also independent of the scaling factor >> 0. Without loss of generality, we will omit the scaling factor in the following proof.
Proof. Firstly, we prove ((n)-invarianl of (0, a0,a, 3)-LEM, (0, -EM,, (0, 3)-MM,, and 20- BWM. Since the differential of da is -(n)-equivaaiint, and (a, LLMM, (a, 3)-EM,,,, (a, 3)-MM, and BWM are ((n) -invarint [63], O(n) iinvarinneeee are thus acquired.
Next, we focus geodesic completeness. It can be easily proven that Riemannian isometries preserve geodesic completeness. On the other hand, (a, 8L-MM, (a, 3)-AI,, and LCM are geodesically complete [63, 41]. As direct corollary, geodesic completeness can be obtained since do is Riemannian isometry.
Finally, we deal with Lie group invariance. Similarly, it can be readily proved that Lie group invariance is preserved under isometries. LCM, LEM, and (a, B)-MM are Lie group bi-invariant [41], bi-invariant [3], and left-invariant [62]. As an isometric pullback metric from the standard LEM [63], (a, 3) LMM is, therefore, Lie group bi-invariant. As pullback metrics, (0, a, LLMM,MM, (0, a, B) AIM. and 0-LCM are therefore bi-invariant, left-invanne iftf-inaariant,, and bi bi-invariant, respectively. D
22

---

Computational details on the SPD MLR under power-deformed BWM
F1 Matrix square roots in the SPD MLR under power-deformed BWM
In the case of MLRs induced by 20-BWM, computing square roots like (BA)² and (AB) with B, A E S poses challenge. Eigendecomposition cannot be directly applied since BA and AB are no longer symmetric, let alone positive definitity. Instead, we use the following formulas to compute these square roots [43]:
(BA)* 3
and (AB) =
(40)
where the involved square roots be computed using eigendecompositiinn singular value decomposition (SVD).
F.2
Numerical stability of the SPD MLR under power-deformed BWM
Let us first explain why we abandon parallel transportation the SPD MLR derived from 20-BWM. Then, we propose our numerically stable methods for computing the SPD MLR based on 20-BWM.
F2211 Instability of parallel transportation under power-deformed BWM
As discussed in Thm. 3.3, there are two ways to generate Á in SPD MLR: parallel transportation and Lie group translation. However, parallel transportation under 20-BWM could cause numerical problems. W.l.o.g,, focus on the standard BWM as 30-BWM is isometric to the BWM.
Although the general solution of parallel transportation under BWM is the solution of an ODE, for the case of parallel transportation starting from the identity matrix, we have closed-form expression [63]:
[v- U [V
a. oj IU [UTVU]
U
(41)
where P UEUT is the eigendecomposition of P E Sh There would be no problem in the forward computation of Eq. (41). However, during backpropagation (BP), Eq. (41) would require the BP of eigendecomposition. involving the calculation of 1/(a,-a)) [33, Prop. 2]. When o, is close to o. the BP of eigendecomposition could problematic.
F22. Numerically stable methods for the SPD MLR under power-deformed BWM
To bypass the instability of parallel transportation under BWM, we use Lie group left translation to generate À in MLRs induced from 20-BWM. However, there is another problem that could cause instability. The computation of the Riemannian metric of 20-BWM requires solving the Lyapunov operator, pe. PLr[IV] = V. Under the case of symmetric matrices, the Lyapunov operator can be obtained by eigendecomposition:
Pp(V= -
UT
(42)
Oj
where V E S", UVUT = V. and P = U is the eigendecomposstion of P ST Similar with Eq. (41), the BP of Eq. (42) requires 1/(o,-o)), undermining the numerical stability.
To remedy this problem, proposed the following formula to stably compute the BP of Eq. (42).
Proposition F.I. For all E & and all E S", we denote the Lyapunov equation as
XP + PX=V,
(43)
where = Lv(V). Given the gradient 8x loss I w.r.l. X, then the BP of the Lyapunov operator can be computed by:
0L = Lp OL
(44)
@V 0X 5
ØL 0P 0.
ÖL
0L Xx 0x
= XLrl
- Lrl
(45)
where Cp[-] can be computed by Eq. (42).
23

---

Proof. Differentiating both sides of Eq. (43), we obtain
dXP X d + dPX Pdx
(46)
= dxP 1 PdX = - dPX,
(47)
dx Cp(dV - -
(48)
Besides, easy computations show that
Cp[V] : W = V LP[W] MW, V E S",
(49)
where : denotes the standard Frobenius inner product.
Then we have the following:
aL
ØL ax
dX
LpldV - -xPp-
(50)
0x
Cpl dL ox
ÖL (-XCpl Ox
AL Lpl 5x 1x) :dP.
dx
dV
(51)
0x
Remark F.2. Eq. (42) needs to be computed in the Lyapunov operator's forward and backward process. Therefore, the forward process, we can save the intermediate matrices U and K with Kr, [oto; (. and then use them compute the backward process efficiently.
G Implementation details and additional experiments
This section offers additional details on the experiments of SPD and Lie MLRs.
G.1
Additional details and experiments on the SPD MLRs
G.1.1 Basic layers SPDNet and TSMNet
SPDNet [30] is the most classic SPD neural network. SPDNet mimics the conventional densely connected feedforward network, consisting of three basic building blocks:
BiMap layer: * W "W--WWTTT with W semi-orthogonal,
(52)
ReEig layer: s U* max(E*=1 .enn with S Uh 1y UR-IT
(53)
LogEig layer: 5 = log(s*-1)
(54)
where max() is element-wise maximization. BiMap and ReEig mimic transformation and non- linear activation, while LogEig maps SPD matrices into the tangent space at the identity matrix for classification.
SPDNetBN [10] further proposed Riemannian batch normalization based on AIM:
Centering from geometric mean Vi S N, = - 5,6
(55)
Biasing towards SPD parameter G : Vi < N, S.
(56)
SPD domain-specifie momentum batch normalization (SPDDSMBN) [37] is an improved version of SPDNetBN. Apart from controlling the mean, it can also control variance. The key operation in SPDDSMBN of controlling mean and variance is:
I' -1 o (S))
(57)
where and e are Riemannian mean and variance. Inspired by [75], during the training stage, SPDDSMBN generates running means and running variances for training and testing with distinct momentum parameters. Besides, it sets & and ū as the running mean and running variance w.r.. training for training and the ones w.r.t. testing for testing. SPDDSMBN also applies domain-specifi. techniques [13], keeping multiple parallel BN layers and distributing observations according to the associated domains. To crack cross-domain knowledge, is uniformly learned across all domains, and G is set to be the identity matrix. TSMNet [37] adopted SPDDSMBN to solve domain adaptation in EEG classification.
In the above models, the Euclidean MLR the co-domain matrix logarithm (matrix logarithm + FC + softmax) is used for classification. Following the terminology in [16], we call this classifier as LogEig MIR. The LogFig MI.R is the Fuclidean classifier the tangent space at the identity, which might distort the innate geometry of the SPD manifold.
24

---

G.1.2 Datasets and preprocessing
Radar": This dataset [10] consists of 3,000 synthetic radar signals. Following the protocol in [10], each signal is split into windows of length 20, resulting in 3,000 SPD covariance matrices of 20 x 20 equally distributed in classes.
HDM05 This dataset [44] contains 2,273 skeleton-based motion capture sequences executed
by various actors. Each frame consists of 3D coordinates of 31 joints of the subjects, and each sequence can be, therefore, modeled by 93 x 93 covariance matrix. Following the protocol in [10], we trim the dataset down to 2086 sequences scattered throughout 117 classes by removing some under-represented classes.
This dataset [28] is a recent competition dataset consisting of EEG signals for mental workload estimation. The dataset is used for two types of experiments: inter-session and inter-subject, which are modeled as domain adaptation problems. Recently, geometry-aware methods have shown promising performance in EEG classification [74, 37J. We choose the SOTA method, TSMNet [37], as our baseline model this dataset. We follow the Python implementation" [37] to carry out preprocessing. In detail, the python package MOABB [35] and MNE [24] are used to preprocess the datasets. The applied steps include resampling the EEG signals 250/256 Hz, applying temporal filters to extract oscillatory EEG activity in the 4 to 36 Hz range, extracting short segments ( < 3s) associated with a class label, and finally obtaining 40 x 40 SPD covariance matrices.
Hinss2021
Disease [2]: It represents disease propagation tree, simulating the SIR disease transmission model [2], with each node representing either an infection or a non-infection state.
Cora [54]] It is a citation network where nodes represent scientific papers in the area of machine learning, edges are citations between them, and node labels academic (sub)areas.
Pubmed [46]: This is a standard benchmark describing citation networks where nodes represent scientific papers in the area of medicine, edges are citations between them, and node labels are academic (sub)areas.
For the Disease, Cora and Pubmed datasets, we follow [76] to model features into S
G.1.3 Implementation details
SPDNet [30] and TSMNet [37]] We follow the official Pytorch code of SPDNetBN and TSMNer to implement experiments. To evaluate the performance of our intrinsic classifiers, substitute the LogEig MLR SPDNet and TSMNet with our SPD MLRs. We implement our SPD MLRs induced from five parameterized metrics. On the Radar and HDM05 datasets, the learning rate is 1e-2, and the batch size is 30. On the Hinss2021 dataset, following [37], the learning rate is le-3 with a 1 weight decay, and batch size is 50. The maximum training epoch is 200, 200, and 50, respectively. We use the standard-cross entropy loss as the training objective and optimize the parameters with the Riemannian AMSGrad optimizer [4].
RResNet [36]: We focus on the AlM-based RResNet, and use the official code and suggested network settings to implement the experiments on the RResNet. We conduct 10-fold and f-fold experiments the HDM05 and NTU datasets. Since RResNet is developed based on SPDNet, we use the same learning settings with the SPDNet for the action recognition task, and borrow the best (6, a, B) from Tab. 4 for SPD MLRs under the RResNet backbone.
SPDGCN [76]: We use the official code" and the suggested network settings in [76]. Note that the SPDGCN with SPD MLR remains the same network settings as the vanilla SPDGCN. Tab. 14 presents the hyperparameters (0, (3) on different datasets.
httpss//ithubb.com//ooob///////
Supplemental.zip
hanp:://gthhh.com/okooee////
25

---

Table 14: (0, 3) of SPD MLRs the SPDGCN backbone.
Datasets
(.-AIM
(8, a,, 8)-MM
(0.
28-BWM
8-LCM
Disease
(0.25,1.0)
(0.25,1.0)
(1,1)
0.25
0.5
Cora
(0.5,1.0)
(0.25,11/9)
(1,3/9
0.25
0.5
Pubmed
(0.5,1.0)
(0.5,1.0)
(1,-1/3)
0.25
0.5
Network Architectures: We denote the network architecture as [dg, ............ dr.). where the dimen- sion of the parameter in the i-th BiMap layer (App. G.I.I) is d, d;-1. For SPDNet, we also validate our SPD MLRs under different network architectures on the Radar and HDM05 datasets. The network architectures
the Radar dataset [20, 16, for the 2-block configuration and [20, 16, 14, 12, 10, 8] for the 5-block configuration. while on the HDM05 dataset, the network architectures are [93, 30] for 1-block, [93. 70, 30] for 2-block, and [93, 70, 50, 30] for 3-block. For TSMNet, the 1-block architecture is [40,20].
In line with the previous work [10, 37, 76, 36], average recall across classes, as the scoring metric for the Hinss2021 dataset, and accuracy for other datasets. On the Hinss2021 dataset, models fit and evaluated with randomized leave 5% of the sessions (inter-session) or subjects (inter-subject) out cross-validation (CV) scheme. On other datasets, K-fold experiments are carried out with randomized initialization and split,
Scoring Metrics:
balanced accuracy, the
G.1.4 Hyper-parameters
We implement the SPD MLRs induced by not only five standard metrics, i.e,, AIM, EM, LCM, and B BWM, but also five families of parameterized metrics. Therefore. in our SPD MLRs, we have a maximum of three typer-parametent i.e,0, 0, at, 3,, where (a, B) associated with (nn) -iviiinvr and controls deformation. For (a,, (3 in (0, or, (3)-EMM, (0, or 3)-AIM, and (0, o, 3)-EM, recalling Eq. (24), is less important [59], we set {1, 1/22, 0, -//n + e, 11/2), where the dimension of input SPD matrices SPD MLRs. The purpose of including É 1 is to ensure O(n)-invariance ((a, 3) ST). These chosen values components, depending the
scaling factors, while measures the relative significance of traces. As scaling 1. As for the value of 3, we select it from predetined set:
for 33 allow for amplifying, neutralizing, characteristics of the datasets. For the deformation factor 6, we roughly select its value around its deformation boundary, i.e., [0.25.1.5] for (8, a, B)MM [0.5,1.5] for f-LCM, [0.25,1.5] and (0, a. B)MM, [0.25,0.75] for 2W.BWM. The details values are listed in Tab. 15.
suppressing the
Table 15: Candidate values for hyper-parameters in SPD MLRs
Metric
(0, a. 9MMAIM
(0, 3)--)))MM
0-LCM
20-BWM
Candidate Values
(0255055.0755112255.555
(0.5,1,1.5
(0.5.1,1.5)
(0.25,0.5.0.75
G.1.5
Model efficiency
Table 16: Training efficiency (s/epoch).
Hinss2021
Methods
Radar
HDM05
Inter-session
Inter-subject
Baseline
1.36
1.95
0.18
8.31
AIM-MLR
1.75
31.64
0.38
13.3
EM-MLR
1.34
3.91
0.19
8.23
LEM-MLR
1.5
4.7
0.24
10.13
BWM-MLR
1.75
33.14
0.38
13.84
LCM-MLR
1.35
3.29
0.18
8.35
We adopt the deepest architectures, namely [20, 16, 14, 12, 10, 8] the Radar dataset, [93, 70, 50, 30] for the HDM05 dataset, and [40, 20] for the Hinss2021 dataset. For simplicity, we focus on the SPD MLRs induced by standard metrics, AIM, EM, LEM, BWM, and LCM. The average training time (in seconds) per epoch is reported in Tab. 16. Generally, when the number of classes is small
26

---

(e.g.,3 in the Radar and Hinss2021 datasets), our SPD MLRs only bring minor additional training time compared the bascline LogEig MLR. However, when dealing with a larger number of classes (e.g.111 117 classes in the HDM05 dataset), there could be some inefficiency caused by our SPD MIRs. This is because each class requires an SPD parameter, and each parameter might require matrix decomposition in the forward or optimization processes during training. Nonetheless, the SPD MLRs induced by EM or LCM generally achieve comparable efficiency with the vanilla LogEig MLR. This is due to the fast computation of their Riemannian operators, making them efficient choices for tasks with a larger number of classes. This result highlights the flexibility of our framework and its applicability to various scenarios.
Radar
98
73
70
96-
67
94
64
61
58
55
SPDNee
Mi R-P,, (M-M
MLR-120)-WW
MUR-0 20-AMM
MLB--9 B3
MLR-IDI-LCM
Figure 4: Visualization of 10-fold average accuracy of SPDNet with different SPD MLRs on the Radar and HDM05 datasets. The error bar denotes the standard deviation.
G.1.6 Visualization
We visualize the 10-fold average results of SPDNet with different classifiers the Radar and HDM05 datasets. We focus on the deepest architectures, Lee. 12,, 10,8) for the Radar dataset, and [93,70,50,30] for the HDM05 dataset. Note that we only report the SPD MLR with the best hyper-parameters (0, or, 3). The figures are presented in Fig. 4. All the results are sourced from Tabs. and 4.
G.2
Additional details and experiments on the Lie MLR
G.2.1 Basic layers in LieNet
LieNet [31] is the most classic neural network on rotation matrices. The latent space of LieNet is the Lie group SO (3) = SO(3) x SO(3) .... x SO(3), i.e., R = (Ri.... Rv) E SO (3). The group and manifold structures on SO (3) are defined by product spaces. For instance, Rtd o R2= 1R, R.,) There are three basic layers in LieNet:
RotMap layer: R W Rk-1 with W E (SON(3),
(58)
Rk-1 mmim
i (R > O (R
RotPooling layer: R2
(59)
otherwise,
LogMap layer: R² log log( (R*
(60)
where ((-) is the Euler angle, and (M,, m²) are two indexes. The RotMap and RotPooling layers mimic the convolution and pooling layers, while the LogMap layer map rotation matrices into tangent space for classification. In the official Matlab implementation, the LogMap layer is implemented as the Fuler anis-angle representation. The classification performed by Fuler axis-angle FC + Softmax. As the axis-angle is an equivalent representation of matrix logarithm, we call this classifier as LogEig MIR as well. This classifier is, therefore, also non-intrinsic.
In LieNet, each rotation feature has shape of [num, frame, 3, 3], where num and frame denote spatial and temporal dimensions. The RotPooling layer is performed either along spatial or temporal dimensions, while the RotMap layer is performed along spatial dimensions, iee., W with a size of [num, 3, 3].
27

---

G.2.2 Datasets and preprocessing
For fair comparison, we follow LieNet to use G3D [6] and HDM05 datasets to validate our Lie MLR.
G3D[6]:
This dataset consists of 663 sequences of 20 different gaming actions. Each sequence is recorded by 3D locations of 20 joints (i.e., 19 bones).
HDM05: We trim it down by removing some under-representer sequences, resulting in 2,326 sequences scattered throughout 122 classes. Following [30], we use the code of [67] to represent each skeleton sequence as a point on the Lie group SO SXN (3), where N and T denote spatial and temporal dimensions. As preprocessed in [30], we set 7 as 100 and 16 for each sequence on the G3D and HDM05 datasets, respectively.
G.2.3 Implementation details
LieNet: Note that the official code of " is developed by Matlab. We follow the open-sourced Pytorch code"! to implement our experiments. To reproduce LieNet more faithfully, we made the following modifications to this Pytorch code. We re-code the LogMap and RotPooling layers to make them consistent with the official Matlab implementation. In addition, we also extend the existing Riemannian optimization package geoopt [4] into SO(3) allow for Riemannian version of SGD, ADAM, and AMSGrad on SO(3), which missing in the current package. However, we find that SGD is the best optimizer for LieNet. Therefore, we use SGD our optimizer during the experiments.
Lie MIR: We use our Lie MLR to replace the axis-angle classifier in LieNet and call the re- sulting network LieNet+LieMLR. To alleviate the computational burden, we set each Ps as the dimension [num, 3, 3]. where num is the spacial dimension of the input of the Lie MLR layer. In other words, Ps is shared in the temporal dimension. We adopt Pytroch3D [53] to calculate the matrix logarithm. Due to the instabilities of pytorch3d. wtw.chhdd.rrasstee....... transforms so3 soo_log._____ Log_map, we use pytorch3d. transfor foms..rrm.... matrix_to _taxssi__is______ _nngle first calculate the quaternion axis and angle, and then convert this representation into matrix logarithm 12.
Training Details:
Following [31], we focus on the 3Blocks and 2Blocks architecture for the G3D and HDM05 datasets, which are the suggested architectures for these two datasets. The learning rate 1e-2 on both datasets, and we further set weight decay as 1e-5 5 on the G3D dataset. For LieNet and LieNet+LieMLR,
torch. .n. utils. beiip_grd_norre norm_ for gradient clipping with clipping factor of 5. The clipping is imposed to the dimensionality reduction weight in the final FC linear on LieNet, or, accordingly, A (Ai,... } in the Lie MLR layer on LieNet+LieMLR,
For the G3D dataset, following LieNet [31], we adopt 10-fold cross-subject test setting. where half the subjects are used for training and the other half are employed for testing, For the HDM05 dataset, following [31], we randomly select half of the sequences for training and the rest for testing. Due the instabilities of LieNet, we conduct 20-fold experiments and select the best 10 folds to evaluate the performance.
Scoring Metrics:
G.3 Hardware
All experiments use an Intel Core 19-7960X CPU with 32GB RAM and an NVIDIA GeForce RTX 2080 Ti GPU.
H Proofs
H.1 Proof of Thm. 3.2
Proof Thm. 3.2. Let us first solve 4 in Eq. (8), which is the solution to the following constrained optimization problem:
(Log, Y, Logp S)p
max Lox, p || Log, Sl(p)
s.t.(Logp S, A)
(61)
/hhutp://gittubbb http://gghhh..ooo//////huuuu//// httr::///itthhhommm///9999////t
28

---

Note that Eq. (61) is well-defined due to the existence of the Riemannian logarithm. Although, Eq. (61) is normally non-convex, Eq. (61) and Eq. (8) can be reduced Euclidean problem:
S)
max Y |||||||||P
s.t. A) 0,
(62)
(5, H. = <im
(63)
where y Logp Y and $ = Logp
Let us first discuss Eq. (62). Denote the solution of Eq. (62) as V Note that y is not necessarily unique. Note that Expp only well-defined locally, More precisely, Expp well-defined in an open ball (0) centered at 0 E T.M. Therefore, Y* might not be in (0). In this case, we can scale Y into (0), and the scaled y is still the maximizer of Eq. (62). Therefore, w.l.o.g. we assume Y E (0).
Putting Y into Eq. (63), Eq. (63) is reduced to the distance to the hyperplane (Y, 0 in the Euclidean space (TpM, (r), which has closed-fore solution:
d(s, H x.r)
(64)
||A||p
(Logp S, A)
(65)
||
H2 Proof of Thm. 3.3
Proof Thm. 3.3 Putting the margin distance (Eq. (10)) into Eq. (4), we have the following:
pạy k () x exp
(sign(( (Ag, Logr, )I d(S, HA..Pe)..
(Logp, (S), As)PPl)
exp
sign((A, Log,,,
(66)
|| A
(Logr, S.A))
exp
H.3 Proof of Prop. 4.1
Proof for Prop. 4.1
The Riemannian metric (o,3)-EM at I
(o.0)-EM 91 (V.V) = (V, (a.
(67)
By Lem. E.2, have the following
()...)..MM
(V,V)
0+30 g (0. (log., (VV, (og),
(log.. (VV,, log,, p (a.((
(a,8)
(68)
Ip (V,V).
()-LM
H.4 Proof ('Th..44 4.2
As the five families metries presented in Thm. 4.2 are pullback metrics, we first present general result regarding Riemannian MLRs under pullback metrics.
29

---

Lemma H.1 (Riemannian MLRs under Pullback Metries), Supposing (N.g) g. is Riemannian manifold and é .M - N is diffeomorphism between manifolds, the Riemannian MLR by parallel transportation (Eq. (11) Eq. (12)) on M under o' 9 can be obtained by g:
ply = k |SE M) x exp
[Lögp, P.
(69)
exp
(Lo8o(P)) ((S), Au) Au)ocra)
(70)
where Av I' -o())) ( with As E ToM, Log, I' are Riemannian logarithm and parallel transportation under 9. and Log, are the counterparts under g.
Furthermore, N has a Lie group operation G, .M could be endowed with a Lie group structure e by S. The Riemannian MLR by left translation (Eq. (11) + Eq. (13)) on M under g and @ can be calculated by g and @:
ply = k 1 x exp (Lögr, S, Au)
(71)
exxp
(Logorr, 0(5), Au)o(rw)
(72)
where Å = (R)) o Φ0.((AA)) Rs = PRO.% R = (P) o (Q)) and i 1.0g is the left translation under c.
Proof for Lem. H.I. Before starting, we should point out that since & is a diffeomorphism, and 9 are indeed well defined, and {M forms Riemannian manifold and (M forms Lic group. We denote o, as the differential of Φ We first focus on the Riemannian MLR by parallel transportation:
p(y= M)
x exp(ārı (Log S, S.
exp go(P)) Pd o ((P)) Log) ((S), P. o t (1t0)r)aoo10000000000..0 (73)
cxp 9o(P)) (Logo(P)) (SS), ( (R
In the case of the Riemannian MLR by left translation, we first note that:
L. = T.((P))00(0) d.
(74)
Therefore, the associated differential is:
R. o L P.P o..
(75)
Putting Eq. (75) in Eq. (71), we can obtain the results.
L
Now, apply Lem. H.1 to derive the expressions of our SPD MLRs presented in Thm. 4.2. For our cases of SPD MLRs, we set Q = I. For simplicity, we will omit the subscript & for Ps and Ag- We will first derive the expressions of SPD MLRs under (0,a,, a, 3)-LEM, O-LCM. (0, o, B)-M, and (0, a, A)MAIM, as they are sourced from Eq. (70). Then we will proceed to present the expression of MLR under 20-BWM, which is sourced from Eq. (72). According Lem. E.3, the scaled metric ag shares the same Riemannian operators as g. We will use this fact throughout the following proof.
Proof of Thm. 4.2. For simplicity, we abbreviate de as & during the proof. Note that for 20-BWM, should be understood as d2e. We first show d(I) and the differential map d..,, which will be frequently required in the following proof:
2(I) = I, (A) 0A, EAETSS.
(76)
(77)
Denoting Φ : (8 + +. then the SPD MLR under ğ by parallel transportation with
Q = I
ply S E M) exp [[op) ((S), T T
(78)
30

---

Next, we begin to prove the five SPD MLRs one by one.
(a, LBM-LMMM As shown by Chen et al. [20], the standard LEM is the pullback metric from the Euclidean space S Similarly, (a, 33MLEM is also pullback metric:
(S+ ( L.. lon (5tm gla.8)
(79)
By Eq. (70), we have
ply k k = exp (log(5) log(P)) log,, (A))
(80)
exp (log(S) log(P), A))
(81)
8-LCM: Simple computations show that 6-LCM the scaled pullback metric of standard Euclidean metric in the Euclidean space of lower triangular matrices L":
15 GLMM - {S (ST Chol (CT,ggM)) Dlog { gg],
(82)
where oE is the standard Frobenius inner product, and g is the Cholesky metric on the Cholesky space 44 [41]. Denoting Ç = Dlog Chol oØ, then we have
(A, = (LAI + 2 ((AA)
AA E TIS"_
(83)
Similar with the case of (0, a. B)-LEM, we have
ply S E M) x exp
(C(S) - ((P))....1)
(84)
[ - Lβ] + [loog(D(K)) Diog(D(E)) LAJ + -
= exp
(85)
where K = SS)) i. Chol (P),, DP). (R) is a diagonal matrix with diagonal elements from K, and [K] is strictly lower triangular matrix from K.
(6, a. -M: Let j0j p. Simple computation shows that (0, o, 3)-MM is the pullback metric of (a. EM::
Ss + g (... - {s (a.
(86)
Besides, we have the following for ry:
(87)
According to Eq. (70), we have
ply l 1-SEEM) exp ((((S) TT(P), (0)
(88)
exp 1 15 - -po AA)
(89)
()-AIM: Putting (0.8))-AMM into Eq. (78), have
ply = l k S x exp
g 9o(P)
(a.0)-IM
(p$ log(p-
(90)
[ _ (log(P Mlog(P-25')) 2 A) ()()
exp
(91)
28-BWM: We first simplify Eq. (72) under the cases of SPD manifolds and then proceed focus on the case of g g RWM Denote o (S ô] + {S+ +. @) where the Lie group operation O [62] is defined as
S L, VS S2 E S44, with L Chol(Si).
(92)
31

---

Note that I is the identity element of {S @) and for any S E + the differential map of the left translation Ls under +
(V) LVL vQ E 4 V TQS+.. with L Chol(S).
(93)
For the induced Lie group {S++ o). the left translation p under is
i por x s(P))O(1 oØ
(94)
= = L29 og. (o(P) ( p29)
(95)
The associated differential at I is
L Irōr'',, (A) = (p)((P) o (0 2r,1 (A),
(96)
=00 (
(97)
where 7. (ho)) Then the SPD MLRs under and @ by left translation is
ply k S M) exp (0g (Logoce) (SS), LALT]
(98)
Setting g = =9" RWM (We omit the scaling factor.), we obtain the SPD MLR under 20-BWM:
ply 1 e E M) exp
20
BWM
BWM (oog((P)
LAL),
(99)
402
(P)
exp
((p20 520
p20 2p20 P2o (LALT))
(100)
40
H.5 Proof of Lem. 5.1
Proof of Lem. 5.1. During this proof, we use the ambient representation of tangent vectors. We only need prove the following:
TQ-P= = LPQ-,.. VP, Q SO(n).
(101)
Given rotation matrices P, Q and a tangent vector H ToSO(n). the parallel transportation [8, Tab. 1] is
-H) = PQ H PQ H.
(102)
On the other hand, given curve c(t) over (On), satisfying c(0) and d' d(0) H. the differential of the left translation LPQ at Q is
dPQ dl
Q())
PQQ1111 11
(103)
==
which concludes the proof.
H.6 Proof of Thm. 5.2
Proof of Thm. 5.2. Lem. 5.1 indicates we can use either parallel transportation or group translation.
Putting the associated expressions from Tab. 13 into Eq. (11) Eq. (12), can directly obtain the results.
32

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction (Sec. accurately reflect the paper's theoretical and empirical contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations specifically in App. A.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
33

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Assumptions are clearly claimed in each theorem, and all the proofs are presented in App. H.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided not)?
Answer: [Yes]
Justification:
Implementation details are discussed in App. G.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
34

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: All datasets (Apps. G.1.2 and G.2.2) are publicly available. The code will be released after the review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips.cc///// wttpcccccccclccguceeeee
httgg//ddde//eeuuuuiiiiiinee sww.uubmiiiiioooooooyyoo for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
In App. G, we present the experimental details for reproducing the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
35

---

7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Mean, STD, and max of K-fold results are presented in Sec,
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula. call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative eтror rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Hardware is mentioned in App. G.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Fthics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttps::/neurrps.cccpbbbb htt.c//pppiiiiiihh..cccceeeeeee.
Answer: [Yes]
Justification: No ethic issue.
36

---

Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
No societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
37

---

We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Original papers and datasets have been cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi sepprswithcode thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Code will be released after the review.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/coddemoodll as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
38

---

Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
39


Reviews:
[{"review_id": "owYoHlHe6J", "reviewer": "Reviewer_dBzm", "summary": "This paper extends the multiclass logistic regression into general Riemannian spaces, contributing to the field of Riemannian deep learning. Starting from the concept of Riemannian hyperplanes, the present work constructs the distance from Riemannian points to Riemannian hyperplanes and derives Riemannian Multinomial Logistic Regression (RMLR) on Riemannian manifolds. The RMLR framework is then showcased under 5 geometries on the SPD manifold, and SO(n). Extensive experiments on different Riemannian backbone networks, including Riemannian feedforward, Riemannian residual, and Riemannian graph neural networks, validate the effectiveness of the proposed RMLR. Especially, the results in Tab. 9 on direct classification (LogEig v.s. SPD MLR) show a clear advantage of the proposed classifiers (up to 18.34 improvement).", "questions": "1. What are the complexities (memory and time) of different metrics, theoretically and experimentally?\n2. For the SPD metrics, how can the researcher select the involved hyper-parameters in practice?", "limitations": "N/A", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.  The proposed RMLR framework (Thm. 3.3) can be easily implemented in different geometries. For a specific geometry, one only needs to put the involved operators into Eq. 11.\n2.  The proposed RMLR not only generalizes the Euclidean MLR, but also incorporates several previous MLRs, such as gyro SPD MLR, gyro SPSD MLR, and flat SPD MLR (Tab. 1). Besides, it further can deal with the geometry which is non-flat or agnostic to gyro structure.\n3.  A complete study of 5 families of deformed SPD metrics is presented in Tab. 2 and Fig. 1.\n4.  5 SPD MLRs and one Lie MLR are specifically implemented. The experiments on different network backbones including Riemannian feedforward, Riemannian residual, and Riemannian graph neural networks, validate the effectiveness of the proposed RMLR.\n5.  The presentation is clear, such as SPD and Lie MLRs in Thms. 4.2 and 5.2.", "weaknesses": "1. More details on the optimization for learning the parameters of the MLR should be presented.\n1. For the SPD manifold, there are at most three hyperparameters: $\\theta, \\alpha, \\beta$. Although these indicate the generality of the proposed framework, how to select the parameter should also be discussed from a practical view.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer $\\textcolor{blue}{dBzm}$ for encouraging feedback and valuable comments. Below, we address the comments in detail. 😄\n***\n\n**1. Details on the parameters learning of the MLR.**\n\nDue to page limits, the optimization details are discussed in Apps. G.1.3 and G.2.3. Generally, our RMLR (Eq. (11)) requires Riemannian optimization for each manifold-valued parameter $P _k$. For the SPD MLR, we use geoopt [a] to optimize the SPD parameter $P _k$. For the Lie MLR, $P _k$ is a rotation matrix, whose Riemannian computation is not supported by geoopt. We, therefore, further extend the geometry in geoopt to include rotation matrices to update our Lie MLR. \n\n**2. Hyperparameters in SPD MLRs: $\\theta$ > $(\\alpha, \\beta)$ .**\n\nThe hyperparameter selection has been discussed in App. G.1.4. For a specific SPD MLR, there are at most three kinds of hyperparameters: deformation factor $\\theta$, $\\alpha$, and $\\beta$. The general order of importance should be $\\theta > (\\alpha, \\beta)$.\n\n- The most significant parameter is the deformation factor $\\theta$. As we discussed in Sec. 4.1, $\\theta$ interpolates between different types of metrics ($\\theta=1$ and $\\theta \\rightarrow 0$). Therefore, one can select its value around the deformation boundary, which has been systematically presented in Fig. 1. Generally speaking, the recommended candidate values for $\\theta$ in AIM, PEM, and LCM are $\\{0.5, 1, 1.5\\}$, while the ones for $2\\theta$ in BWM are $\\{0.25, 0.5, 0.75\\}$.\n\n- The less important parameters are $(\\alpha,\\beta)$. Recalling Tab. 12, $(\\alpha, \\beta)$ only affects the Riemannian metric tensor, *i.e.*, the inner products over tangent spaces. For our SPD MLRs in Thm 4.2, they only affect inner products, which should have fewer effects. Our experiments indicate that $(\\alpha,\\beta)=(1,0)$ is saturated for most cases.\n\n**3. Model complexities.**\n\nAs the logics of RMLR are similar under different geometries, we focus on the SPD MLR in the following for simplicity.\n\n- **Memory complexities:** Recalling Thm. 4.2, each class $k$ requires an SPD parameter $P _k$ and a Euclidean parameter $A _k$ in the SPD MLR. The memory costs depend on the number of class $C$ and the dimension of $P _k$ or $A _k$. Supposing there is $C$ class and the dimension of $P _k$ is $n \\times n$. SPD MLR needs to store 2C matrix parameters of $n \\times n$, *i.e*, $2Cn^2$. On the other hand, the classifier (LogEig MLR) on the tangent space at the identity requires $C \\times \\left(\\frac{n(n+1)}{2}+1 \\right)$, where $\\frac{n(n+1)}{2}$ is the dimension of tangent space. Although our SPD MLR requires more parameters than the vanilla LogEig MLR, we achieve much better performance across different network architectures.\n\n- **Computational complexities:**  The efficiency of our SPD MLRs against the vanilla nonintrinsic LogEig MLR has been discussed in App. G.1.5. The key factor of the computational complexity of SPD MLRs under different metrics lies in the number of matrix functions, such as matrix power, logarithm, Lyapunov operator, and Cholesky decomposition. These matrix functions are divided into two categories: one is based on eigendecomposition, and the other is the Cholesky decomposition. Note that Cholesky decomposition is more efficient than eigendecomposition. Tab. A summarizes the number of matrix functions in each SPD MLR. With deformation, the general efficiency of SPD MLR should be LCM>EM>LEM>AIM>BWM, while without deformation, the order should be EM>LCM>LEM>AIM>BWM. The following Tab. B comes from Tab. 16 in App. G.1.5, which reports the average training time (in seconds) per epoch of each classifier. Please refer to App. G.1.5 for more details. Tab. B shows that EM-, LEM-, and LCM-based SPD MLR are more efficient than AIM- and BWM-based MLR. Notably, we also observe that our LCM achieves even comparable efficiency with the vanilla LogEig MLR on the radar and Hinss2021 datasets.\n\nTable A: Number of matrix functions for each class $k$ in different SPD MLRs. a (b) means the number of matrix functions in the SPD MLR under the deformed (standard) metric.\n| Metric | Eig-based Matrix Functions | Cholesky Decomposition | In Total |\n|:------:|:--------------------------:|:----------------------:|:-----:|\n| LEM    |             1 (1)            |           0 (0)           |  1 (1) |\n| AIM    |             3 (2)            |           0 (0)           |  3 (2) |\n| EM     |             1 (0)            |           0 (0)           |  1 (0) |\n| LCM    |             1 (0)            |           1 (1)           |  2 (1) |\n| BWM    |             5 (4)            |           1 (1)           |  6 (5) |\n\nTable B: Training efficiency (s/epoch) of different classifiers.\n| Methods  | Radar | HDM05 | Hinss2021 Inter-session | Hinss2021 Inter-subject |\n|----------|-------|-------|-------------------------|-------------------------|\n| LogEig MLR | 1.36  | 1.95  | 0.18                    | 8.31                    |\n| AIM-MLR  | 1.75  | 31.64 | 0.38                    | 13.3                    |\n| EM-MLR   | 1.34  | 3.91  | 0.19                    | 8.23                    |\n| LEM-MLR  | 1.5   | 4.7   | 0.24                    | 10.13                   |\n| BWM-MLR  | 1.75  | 33.14 | 0.38                    | 13.84                   |\n| LCM-MLR  | 1.35  | 3.29  | 0.18                    | 8.35                    |\n\n**Reference**\n\n> [a] Riemannian adaptive optimization methods. ICLR, 2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VevmR7pqdu", "reviewer": "Reviewer_5QCn", "summary": "The authors extend multinomial logistic regression into spaces where they only require a logarithmic map. They do so in order to accomplish tasks such as classification", "questions": "Can the authors elaborate on how their work is a distinct contribution compared to the SOTA methods which are mentioned? I understand this is more general, as is shown in Table 1 but I fail to see the entire picture.", "limitations": "NA", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well organized and written. There is a good balance of theoretical results and practical applications. It is nice to see a thorough exposition of the SPD manifold with all the commonly used metrics.", "weaknesses": "I don't understand why lines 29-31 seem to look down on the use of things such as tangent spaces and coordinate systems because the proposed method relies on the log map which itself effectively requires tangent spaces and coordinate systems.\n\nThe experiment section could use more thorough explanations which are found in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer $\\textcolor{green}{5QCn}$ for the careful review and the suggestive comments. Below, we address the comments in detail. 😄\n***\n\n**1. Linearization by a single fixed tangent space or coordinate system fails to capture global geometry; Our RMLR adopts *distinct dynamic* tangent spaces for each class respecting Riemannian geometry.**\n\nLines 29-31 summarize several previous classifiers in Riemannian neural networks, which linearize the whole manifold as a **fixed Euclidean space**, such as a fixed tangent space (at the identity matrix) or fixed coordinate system. In contrast, our Riemannian Multinomial Logistic Regression (RMLR) does not identify the manifold with a flat Euclidean space. Instead, it directly builds the classifier based on Riemannian geometry.\n\n- **Theoretically, our RMLR extends the Euclidean MLR into manifolds by Riemannian geometry.** Def. 3.1 extends the Euclidean distance to the hyperplane in Eq. (7) into manifolds by Riemannian trigonometry and geodesic distance. Thm. 3.2 offers a general solution for this hyperplane distance. Putting this solution into Eq. (4), RMLR is introduced in Eq. (11). Although Eq. (11) uses Riemannian logarithm, it is derived from distances to Riemannian hyperplanes, respecting the underlying Riemannian geometry. \n\n- **Numerically, our RMLR in Eq. (11) adopts *distinct dynamic* tangent spaces for each class**:\n$$ p(y=k \\mid S \\in \\mathcal{M}) \\propto \\exp \\left(\\left\\langle\\operatorname{Log} _{P _k} S, \\tilde{A} _k\\right\\rangle _{P _k}\\right), \\forall k \\in \\{ 1 \\cdots C \\}\n$$\nwhere $P _k \\in \\mathcal{M}$ and $\\tilde{A} _k \\in T _{P _k} \\mathcal{M}$. Each class $k$ involves a distinct calculation of $\\left\\langle\\operatorname{Log} _{P _k} S, \\tilde{A} _k\\right\\rangle _{P _k}$ in the tangent space at $P _k$, *i.e.*, $T _{P _k} \\mathcal{M}$. More importantly, **each tangent space $T _{P _k} \\mathcal{M}$ is dynamically learned, as each $P _k$ is a learnable parameter**. Besides, $\\left\\langle\\operatorname{Log} _{P _k} S, \\tilde{A} _k\\right\\rangle _{P _k}$ respects the Riemannian hyperplane (Eq. (5)), which generalizes the Euclidean decision plane (Eq. (3)). As illustrated in Figs. 2-3, the Riemannian hyperplane for each class is a **curved surface**.\n\n**2. Core technical contribution: We derive the Riemannian marginal distance by the Riemannian trigonometry and geodesic distance, which only requires matrix logarithm.**\n\nAs discussed in lines 104-118, the core challenge in building Riemannian MLRs is reformulating the Eucliedan marginal distance in Eq. (7) into Riemannian counterparts. Previous Riemannian MLRs mainly resort to gyro structures and pullback metrics from the Euclidean space. These approaches fail to handle general geometries due to the strong requirements of the metrics.\n\n- As the core technical contribution, we re-interpret the Riemannian marginal distance using Riemannian trigonometry and geodesic distance, which only requires matrix logarithm. As discussed in lines 120-121, the matrix logarithm is the minimal requirement for building Riemannian MLRs, and it exists in the most popular geometries in the machine learning community. Based on this, our Thm. 3.3 provides a solution for RMLR under general geometries. We thus see our RMLR is easy to use and can handle general geometries. Further, as shown in Tab. 1, many previous MLRs are special instantiations of our RMLR under different metrics. We will emphasize this core contribution in the revised introduction.\n\n- **Further remark:** As we claimed, given a Riemannian metric, our RMLR can be implemented in a plug-in manner, by simply putting the required Riemannian logarithm into Eq. (11). Therefore, our RMLR can also be implemented on other manifolds, such as correlation matrices [a] and special Euclidean groups [b], as the required Riemannian operators have explicit expressions. We will explore these in the future.\n\n**3. Experimental explanations in the appendix will be briefly summarized in the main paper.**\n\nDue to page limits, we left implementation details and RMLR efficiency in the appendix. In the final version, we will briefly summarize our findings in the main paper and provide proper references to the appendix.\n\n\n**References**\n\n> [a] Thanwerdas Y. Permutation-invariant log-Euclidean geometries on full-rank correlation matrices. SIAM Journal on Matrix Analysis and Applications, 2024.\n>\n> [b] Murray R M, Li Z, Sastry S S. A mathematical introduction to robotic manipulation[M]. CRC press, 2017.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IUCFmNJJ4a", "reviewer": "Reviewer_jvDJ", "summary": "- Instead of adopting complex approaches for extending MLR to Riemannian manifolds via general geometry extensions such as gyro structures and generalized SINE rules, this study generalizes to Riemannian manifolds using a simple approach based on the logarithm map.\n- The authors show the several experimental results with various types of datasets.", "questions": "1. Does Eq. 8 satisfy the axioms of distance?\n2. The authors adopt parallel transport to determine A~_k in Eq. 11 but projecting a point in Euclidean space to the tangent space might be simpler. Parallel transport can be computationally intensive and needs to be defined according to the type of manifold. Why did the authors choose parallel transport? Also, regarding Weakness 1, if parallel transport needs to be determined individually, is there a contradiction with the authors' claim that only the logarithm map is needed?", "limitations": "- Discussed in Appendix A.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. This study generalizes to Riemannian manifolds using a simple approach based on the logarithm map, avoiding complex approaches like gyro structures and generalized row of sines.\n2. The geometric aspects of this paper are well-founded. Conducting geometrically valid computations using the logarithm map and parallel transport is a standard method for tangent space analysis.\n3. Evaluation is conducted on various types of datasets.", "weaknesses": "1. I find this paper somewhat confusing to read. What does the claim \"our framework only requires the explicit expression of the Riemannian logarithm\" in the introduction mean? For example, parallel transport requires explicit expressions for each of types of Riemannian manifold or metrics (Table 12). Is there a contradiction with the authors' claim?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer $\\textcolor{brown}{jvDJ}$ for the constructive suggestions and insightful comments! In the following, we respond to the concerns in detail. 😄\n***\n\n**1. Parallel transport is not the only option for determining $\\tilde{A} _k$ in the RMLR (Eq. (11)).**\n\n- **The ways to determine $\\tilde{A} _k$ are flexible.** As discussed in lines 142-150, as $P _k$ varies, we can not directly optimize $\\tilde{A} _k \\in T _{P _k} \\mathcal{M}$ by the Euclidean optimization. The key is to use a map $f: T _{Q}\\mathcal{M} \\rightarrow T _{P _k}\\mathcal{M}$ to generate $\\tilde{A} _k$ from a fixed tangent space. The choice of $f$ is pretty flexible. In lines 142-150, we discussed four instances of $f$, including parallel transport, vector transport [a] (the approximation of parallel transport), differential of Lie group translation, and differential of gyro group translation. Apart from these, we believe there will be alternative proper $f$. For a specific manifold, the suitable choice of $f$ depends on factors such as the geometry of the manifold, simplicity, and numerical stability.\n\n- **We mainly focus on parallel transport due to its advantageous properties and theoretic convenience.** Parallel transport has many nice properties, such as preserving inner product across tangent spaces [b, Ch. 3.3]. Furthermore, **in our SPD and Lie MLRs involving parallel transport, parallel transport can be canceled out, and the MLR expression can be further simplified**. For instance, Although the parallel transport under AIM could be complex (as shown in Tab. 12), it is canceled out and further simplifies the final expression of AIM-based SPD MLR. Please check the SPD MLRs in Thm. 4.2 (except the one under BWM) and Lie MLR in Thm. 5.2, as well as their proofs, for more details.\n\n- Furthermore, **we also use the differential of Lie group translation for better numerical stability**. As detailed in App. F.2.1, the parallel transport under BWM is backpropagation-unfriendly. Therefore, we use the differential of Lie group translation to determine $\\tilde{A} _k$ for the BWM-based SPD MLR as a more stable alternative.\n\n- Finally, **the projection map might omit crucial aspects of the latent geometry**. We assume the projection map you mentioned is the orthogonal projection for the Riemannian submanifold or quotient manifold [a, Ch. 3.6]. It is widely used in Riemannian optimization, as it is equivalent to transforming the Euclidean gradient to Riemannian counterparts. However, the projection map only captures the horizontal part and discards the vertical part, leading to information loss or redundancy. For instance, it fails to preserve the inner product. Besides, it is not bijective, potentially introducing many-to-one redundancy in $\\tilde{A} _{k}$. Therefore, although it can be numerically used to determine $\\tilde{A} _{k}$, we did not use this map. Nevertheless, we will explore other maps, which properly map the Euclidean vector into the tangent space for determining $\\tilde{A} _k$.\n\n**In summary, parallel transport is not a necessary ingredient for determining $\\tilde{A} _k$, and there are different alternative choices.** The specific choice depends on theoretical rationality and computational simplicity \\& stability.\n\n**2. Eq. (8) is not from point to point, but from point to hyperplane; therefore, the axioms of metric space cannot be applied to our distance.**\n\n- The Riemannian margin distance in Eq. (8) differs from the distance in a metric space. The distance in a metric space is defined as $\\mathcal{M} \\times \\mathcal{M} \\rightarrow \\mathbb{R}$. However, the distance in Eq. (8) is \\$\\\\mathcal{M} \\\\times \\\\{ \\\\tilde{H} _{\\\\tilde{A},P}\\\\} \\\\rightarrow \\\\mathbb{R}\\$, where \\$\\\\{ \\tilde{H} _{\\\\tilde{A},P}\\\\}\\$ denotes the set of all Riemannian hyperplanes. Therefore, the axioms of metric space can not be applied to our Eq. (8). Nevertheless, Eq. (10) indicates the positiveness of our Riemannian margin distance.\n\n- Besides, our Riemannian margin distance is a direct generalization of Euclidean margin distance. When the latent manifold $\\mathcal{M}=\\mathbb{R} ^n$, Eq. (8) is reduced to the familiar Euclidean margin distance (Eq. 7).\n\n**Reference**\n\n> [a] Absil P A, Mahony R, Sepulchre R. Optimization algorithms on matrix manifolds. Princeton University Press, 2008.\n> \n> [b] Do Carmo M P, Flaherty Francis J. Riemannian geometry[M]. Boston: Birkhäuser, 1992.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper generalizes multiclass logistic regression to Riemannian manifolds. All reviewers agree that the contribution to the community is valuable and that the approach is novel. The discussion phase largely clarified minor points and worked on an improved presentation.
Decision: Accept (poster)