Title: RMLR: Extending Multinomial Logistic Regression into General Geometries
Venue: NeurIPS 2024
Accepted: yes
Keywords: Riemannian neural networks

Matrix manifolds

SPD manifolds

Special orthogonal groups
Abstract: Riemannian neural networks, which extend deep learning techniques to Riemannian spaces, have gained significant attention in machine learning. To better classify the manifold-valued features, researchers have started extending Euclidean multinomial logistic regression (MLR) into Riemannian manifolds. However, existing approaches suffer from limited applicability due to their strong reliance on specific geometric properties. This paper proposes a framework for designing Riemannian MLR over general geometries, referred to as RMLR. Our framework only requires minimal geometric properties, thus exhibiting broad applicability and enabling its use with a wide range of geometries. Specifically, we showcase our framework on the Symmetric Positive Definite (SPD) manifold and special orthogonal group, i.e., the set of rotation matrices. On the SPD manifold, we develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric. Extensive experiments on different Riemannian backbone networks validate the effectiveness of our framework.

RMLR: Extending Multinomial Logistic Regression into General Geometries

Abstract

Riemannian neural networks, which extend deep learning techniques to Riemannian spaces, have gained significant attention in machine learning. To better classify the manifold-valued features, researchers have started extending Euclidean multinomial logistic regression (MLR) into Riemannian manifolds. However, existing approaches suffer from limited applicability due to their strong reliance on specific geometric properties. This paper proposes a framework for designing Riemannian MLR over general geometries, referred to as RMLR. Our framework only requires minimal geometric properties, thus exhibiting broad applicability and enabling its use with a wide range of geometries. Specifically, we showcase our framework on the Symmetric Positive Definite (SPD) manifold and special orthogonal group SO( n ) , i.e., the set of rotation matrices in R n . On the SPD manifold, we develop five families of SPD MLRs under five types of power-deformed metrics. On SO( n ) , we propose Lie MLR based on the popular bi-invariant metric. Extensive experiments on different Riemannian backbone networks validate the effectiveness of our framework. The code is available at https://github.com/GitZH-Chen/RMLR .

1 Introduction

In recent years, significant advancements have been achieved in Deep Neural Networks (DNNs), enabling them to effectively analyze complex patterns from various types of data, including images, videos, and speech [29, 38, 27, 66]. However, most existing models have primarily assumed the underlying data with a Euclidean structure. Recently, a growing body of research has emerged, recognizing that the latent spaces of many applications exhibit non-Euclidean geometries, such as Riemannian geometries [9]. Various frequently-encountered manifolds in machine learning have posed interesting challenges and opportunities, including special orthogonal groups SO( n ) [67, 31], symmetric positive definite (SPD) [30, 10, 42, 73, 18, 19], Gaussian [14, 47], Grassmannian [32, 72] spherical [56], and hyperbolic manifolds [23]. These manifolds share an important Riemannian property - their Riemannian operators, including geodesics, exponential & logarithmic maps, and parallel transportation, often possess closed-form expressions. Leveraging these Riemannian operators, researchers have successfully generalized different types of DNNs into manifolds, dubbed Riemannian neural networks .

Although Riemannian networks demonstrated success in many applications, most approaches still rely on Euclidean spaces for classification, such as tangent spaces [30, 31, 10, 47, 69, 71, 48, 49, 37, 70, 15], ambient Euclidean spaces [68, 57, 58], or coordinate systems [12]. However, these strategies distort the intrinsic geometry of the manifold, undermining the effectiveness of Riemannian networks. Researchers have recently started directly developing Riemannian Multinomial Logistic Regression (RMLR) on manifolds. Inspired by the idea of hyperplane margin [39], Ganea et al. [23] developed a hyperbolic MLR in the Poincaré ball for Hyperbolic Neural Networks (HNNs). Motivated by HNNs, Nguyen and Yang [50] developed three kinds of gyro SPD MLRs based on three distinct gyro

structures of the SPD manifold. In parallel, Chen et al. [16] proposed a framework for building SPD MLRs induced by the flat metrics on the SPD manifold. Nguyen et al. [51] proposed gyro MLRs for the Symmetric Positive Semi-definite (SPSD) manifold based on the product of gyro spaces. However, these classifiers often rely on specific Riemannian properties, limiting their generalizability to other geometries. For instance, the hyperbolic MLR [23] relies on the generalized law of sine, while the gyro MLRs [50, 51] rely on the gyro structures.

This paper presents a framework of RMLR over general geometries. In contrast to previous works, our framework only requires the explicit expression of the Riemannian logarithm, which is the minimal requirement in extending the Euclidean MLR into manifolds. Since this property is satisfied by many commonly encountered manifolds in machine learning, our framework can be broadly applied to various types of manifolds. Empirically, we showcase our framework on the SPD manifold and rotation matrices. On the SPD manifold, we systematically propose SPD MLRs under five families of power-deformed metrics. We also present a complete theoretical discussion on the geometric properties of these metrics. In the Lie group of SO( n ) , we propose Lie MLR based on the widely used bi-invariant metric to build the Lie MLR. Our work is the first to extend the Euclidean MLR into Lie groups. Besides, our framework incorporates several previous Riemannian MLRs, including gyro SPD MLRs in [50], SPD MLRs in [16], and gyro SPSD MLRs in [51].

Our SPD MLRs are validated on four SPD backbone networks, including SPDNet [30] on the radar and human action recognition tasks and TSMNet [37] on the electroencephalography (EEG) classification tasks for the Riemannian feedforward network, RResNet [36] on the human action recognition task for the Riemannian residual network, and SPDGCN [76] on the node classification for the Riemannian graph neural network. Our Lie MLR is validated on the classic LieNet [31] backbone for the human action recognition task. Compared with previous non-intrinsic classifiers, our MLRs achieve consistent performance gains. Especially, our SPD MLRs outperform the previous classifiers by 14.23% on SPDNet and 13.72% on RResNet for human action recognition, and 4.46% on TSMNet for EEG inter-subject classification. Furthermore, our Lie MLR can improve both the training stability and performance. In summary, our main theoretical contributions are the following: (a) We develop a general framework for designing Riemannian MLR over general geometries, incorporating several previous Riemannian MLRs on different geometries. (b) We systematically propose 5 families of SPD MLRs based on different geometries of the SPD manifold. (c) We propose a novel Lie MLR for deep neural networks on SO( n ) .

Main theoretical results: We solve the Riemannian margin distance to the hyperplane in Thm. 3.2 and present our RMLR framework in Thm. 3.3. As shown in Tab. 1, our RMLR incorporates several existing MLRs on different geometries. Thm. 4.2 showcases our RMLR on the SPD manifold under five families of metrics summarized in Tab. 2. To remedy the numerical instability of BWM geometry on the SPD manifold, we also propose a backpropagation-friendly solver for the SPD MLR under BWM in App. F.2.2. Thm. 5.2 proposes the Lie MLR for the Lie group SO( n ) . Due to the page limits, we put all the proofs in App. H.

2 Preliminaries

This section provides a brief review of the basic geometries of SPD manifolds and special orthogonal groups. Detailed review and notations are left in Apps. B and B.1.

SPD manifolds: The set of n × n symmetric positive definite (SPD) matrices is an open submanifold of the Euclidean space S n of symmetric matrices, referred to as the SPD manifold S n ++ [3]. There are five kinds of popular Riemannian metrics on S n ++ : Affine-Invariant Metric (AIM) [52], LogEuclidean Metric (LEM) [3], Power-Euclidean Metrics (PEM) [22], Log-Cholesky Metric (LCM) [41], and Bures-Wasserstein Metric (BWM) [5]. Note that, when power equals 1, the PEM is reduced to the Euclidean Metric (EM). Thanwerdas and Pennec [63] generalized AIM, LEM, and EM into two-parameters families of O( n ) -invariant metrics, i.e., ( α, β ) -AIM, ( α, β ) -LEM, and ( α, β ) -EM, with min( α, α + nβ ) > 0 . We denote the metric tensor of ( α, β ) -AIM, ( α, β ) -LEM, ( α, β ) -EM, LCM, and BWM as g ( α,β ) -AIM , g ( α,β ) -LEM , g ( α,β ) -EM , g LCM , and g BWM , respectively.

Rotation matrices: The special orthogonal group SO( n ) is the set of n × n orthogonal matrices with unit determinant, the elements of which are also referred to as rotation matrices. As shown in [25], SO( n ) forms a Lie group. We adopt the widely used bi-invariant Riemannian metric [8].

3 Riemannian multinomial logistic regression

Inspired by [39], Ganea et al. [23], Nguyen and Yang [50], Chen et al. [16], Nguyen et al. [51] extended the Euclidean MLR into hyperbolic, SPD, and SPSD manifolds. However, these classifiers rely on specific Riemannian properties, such as the generalized law of sines, gyro structures, and flat metrics, which limits their generality. In this section, we first revisit several existing MLRs and then propose our Riemannian classifiers with minimal geometric requirements.

3.1 Revisiting existing multinomial logistic regressions

Given C classes, the Euclidean MLR computes the multinomial probability of each class:

where b k ∈ R , and x, a k ∈ R n \{ 0 } . As shown in [23], the Euclidean MLR can be reformulated by the margin distance to the hyperplane:

where ⟨ a k , p k ⟩ = b k , and H a k ,p k is a hyperplane.

Eqs. (2) and (3) can be naturally extended into manifolds M by Riemannian operators:

where P k ∈ M , ˜ A k ∈ T P k M\{ 0 } , g P k is the Riemannian metric at P k , and Log P k is the Riemannian logarithm at P k . The margin distance is defined as an infimum:

The MLRs proposed in [39, 23, 50, 16] can be viewed as different implementations of Eq. (4)Eq. (6). To calculate the MLR in Eq. (4), one has to compute the associated Riemannian metrics, logarithmic maps, and margin distance. The associated Riemannian metrics and logarithmic maps often have closed-form expressions on the frequently-encounter manifolds in machine learning. However, the computation of the margin distance can be challenging. On the Poincaré ball of hyperbolic manifolds, the generalized law of sines simplifies the calculation of Eq. (6) [23]. However, the generalized law of sines is not universally guaranteed on other manifolds. Additionally, Chen et al. [16] developed a closed-form solution of margin distance on the SPD manifold under any metric pulled back from Euclidean spaces. For curved manifolds, solving Eq. (6) would become a non-convex optimization problem. To address this challenge, Nguyen and Yang [50] defined gyro structures on the SPD manifold and proposed a pseudo-gyrodistance to calculate the margin distance. Similarly, Nguyen et al. [51] proposed a pseudo-gyrodistance on the SPSD manifold based on the gyro product space. However, gyro structures do not necessarily exist in general geometries. In summary, the aforementioned methods often rely on specific properties of their associated Riemannian metrics, which usually do not generalize to general geometries.

3.2 Riemannian multinomial logistic regression

Recalling Eqs. (4) and (5), the least requirement of extending Euclidean MLR into manifolds is the well-definedness of Log P k ( S ) for each k . In this subsection, we will develop Riemannian MLR, which depends solely on the Riemannian logarithm, without additional requirements, such as gyro structures and generalized law of sines. In the following, we always assume the well-definedness of the Riemannian logarithm. We start by reformulating the Euclidean margin distance to the hyperplane from a trigonometry perspective and then present our Riemannian MLR.

As we discussed before, obtaining the margin distance of Eq. (6) could be challenging. Inspired by [50], we resort to the perspective of trigonometry to reinterpret Euclidean margin distance. In Euclidean space, the margin distance is equivalent to

We extend Eq. (7) to manifolds by the Riemannian trigonometry and geodesic distance, the counterparts of Euclidean trigonometry and distance.

Definition 3.1 (Riemannian Margin Distance) . Let ˜ H ˜ A,P be a Riemannian hyperplane defined in Eq. (5), and S ∈ M . The Riemannian margin distance from S to ˜ H ˜ A,P is defined as

where d ( S, P ) is the geodesic distance, and Y ∗ = argmax(cos ∠ SPY ) with Y ∈ ˜ H ˜ A,P \{ P } . The initial velocities of geodesics define cos ∠ SPY :

where ⟨· , ·⟩ P is the Riemannian metric at P , and ∥ · ∥ P is the associated norm.

The Riemannian margin distance in Def. 3.1 has a closed-form expression.

Theorem 3.2. [ ↓ ] The Riemannian margin distance defined in Def. 3.1 is given as

Putting the Eq. (10) into Eq. (4), we can a closed-form expression for Riemannian MLR.

Theorem 3.3 (RMLR) . [ ↓ ] Given a Riemannian manifold {M , g } , the Riemannian MLR induced by g is

where P k ∈ M , ˜ A k ∈ T P k M\{ 0 } , and Log is the Riemannian logarithm.

˜ A k in Eq. (11) can not be directly viewed as a Euclidean parameter, as ˜ A k ∈ T P k M depends on P k and P k varies during the training. However, the tangent vector ˜ A k can be generated from a tangent space at a fixed point. Several tricks can be used, such as Riemannian parallel transportation [21], vector transportation [1], the differential of Lie group or gyrogroup translation [64, 65]. Following previous work [23, 16, 50], we focus on parallel transportation and Lie group translation:

where Q ∈ M is a fixed point, A k ∈ T Q M\{ 0 } , Γ is the parallel transportation along geodesic connecting Q and P k , and L P k ⊙ Q -1 ⊙ ∗ ,Q denotes the differential map at Q of left translation L P k ⊙ Q -1 ⊙ with P k ⊙ Q -1 ⊙ denoting Lie group product and inverse. In this way, A k lies in a fixed tangent space and, therefore, can be optimized by a Euclidean optimizer.

Remark 3.4 . We make the following remarks w.r.t. our Riemannian MLR.

(a). The reformulation of Eq. (7) in gyro MLR [50, 51] and ours are different. Gyro MLR adopts gyro trigonometry and gyro distance to reformulate Eq. (7), while our method directly uses Riemannian trigonometry and geodesic distance.

(b). Compared with the MLRs on hyperbolic, SPD, or SPSD manifolds in [23, 50, 16, 51], our framework enjoys broader applicability, as our framework only requires the Riemannian logarithm. This property is commonly satisfied by most manifolds encountered in machine learning, such as the five metrics on SPD manifolds mentioned in Sec. 2, the invariant metric on SO( n ) [8], and hyperbolic &spherical manifolds [11, 56]. Besides, several existing MLRs on different geometries are special cases of our Riemannian MLR, which are detailed in Tab. 1.

(c). The well-definedness of the Riemannian logarithm is a much weaker requirement compared to the existence of the gyro structure. The gyro structure not only requires the Riemannian logarithm but also implicitly requires geodesic completeness [50, Eqs. (1-2)]. For instance, on SPD manifolds, EM and BWM [63] are incomplete, undermining the well-definedness of gyro operations.

4 SPD multinomial logistic regressions

This section showcases our RMLR framework on the SPD manifold. We first systematically discuss the power-deformed geometries of SPD manifolds. Based on these metrics, we will develop five families of deformed SPD MLRs.

Figure 1: Illustration on the deformation ( left ) and Venn diagram ( right ) of metrics on SPD manifolds, where IEM, SREM, and 1 4 PAM denotes Inverse Euclidean Metric, Square Root Euclidean Metric, and Polar Affine Metric scaled by 1 / 4 .

4.1 Deformed geometries of SPD manifolds

̸

As discussed in Sec. 2, there are five popular Riemannian metrics on SPD manifolds. These metrics can be all extended into power-deformed metrics. For a metric g on S n ++ , the power-deformed metric is defined as

where ϕ θ ( P ) = P θ is the matrix power, and ( ϕ θ ) ∗ ,P is the differential map. The deformed metric ˜ g can interpolate between a LEM-like metric ( θ → 0 ) and g ( θ = 1 ) [61]. Previous work has extended ( α, β ) -AIM, ( α, β ) -LEM, LCM, and BWM into power-deformed metrics and ( α, β ) -LEM is proven to be invariant under the power deformation [17]. We denote these metrics as ( θ, α, β ) -AIM [59], ( α, β ) -LEM [17], 2 θ -BWM [61], and θ -LCM [17], respectively. The deformation of these metrics is discussed in App. E.1. We further define the power-deformed metric of ( α, β ) -EM by Eq. (14), denoted as ( θ, α, β ) -EM. We have the following for the deformation of ( θ, α, β ) -EM.

So far, all five popular Riemannian metrics on SPD manifolds have been generalized into powerdeformed families of metrics. We summarize their associated properties in Tab. 2 and present their theoretical relation in Fig. 1. We leave technical details in App. E.2.

4.2 Five families of SPD multinomial logistic regressions

This subsection presents five families of specific SPD MLRs by our general framework in Thm. 3.3 and metrics discussed in Sec. 4.1. We focus on generating ˜ A k by parallel transportation from the identity matrix, except for 2 θ -BWM. Since the parallel transportation under 2 θ -BWM would undermine numerical stability (please refer to App. F.2.1 for more details), we resort to a newly

x

0

1

2

3

y

3

1

1

3

z

0

1

2

3

x

0

1

2

3

y

3

1

1

3

z

0

1

2

3

AIM SPD Hyperplane

x

0

1

2

3

y

3

1

1

3

z

0

1

2

3

PEM SPD Hyperplane

x

0

1

2

3

y

3

1

1

3

z

0

1

2

3

BWM SPD Hyperplane

x

0

1

2

3

y

3

1

1

3

z

0

1

2

3

LCM SPD Hyperplane

developed Lie group operation [62]:

where L 1 = Chol( S 1 ) is the Cholesky decomposition.

Theorem 4.2 (SPD MLRs) . [ ↓ ] By abuse of notation, we omit the subscripts k of A k and P k . Given SPD feature S , the SPD MLRs, p ( y = k | S ∈ S n ++ ) , are proportional to

where A ∈ T I S n ++ \{ 0 } is a symmetric matrix, log( · ) is the matrix logarithm, L P ( V ) is the solution to the matrix linear system L P [ V ] P + P L P [ V ] = V , known as the Lyapunov operator, Dlog( · ) is the diagonal element-wise logarithm, ⌊·⌋ is the strictly lower part of a square matrix, and D ( · ) is a diagonal matrix with diagonal elements of a square matrix. Besides, log ∗ ,P is the differential maps at P , ˜ K = Chol( S θ ) , ˜ L = Chol( P θ ) , and ¯ L = Chol( P 2 θ ) .

The Lyapunov operator in Eq. (20) requires the eigendecomposition. However, the backpropagation of eigendecomposition involves 1 / ( σ i -σ j ) [34], undermining the numerical stability. Therefore, we propose a numerically stable backpropagation for the Lyapunov operator, detailed in App. F.2.2.

As 2 × 2 SPD matrices can be embedded into R 3 as an open cone [74], we illustrate SPD hyperplanes induced by five families of metrics in Fig. 2.

Remark 4.3 . Our SPD MLRs extend the existing SPD MLRs [50, 16]. The pseudo-gyrodistance to a SPD hyperplane in [50, Thms. 2.232.25] is incorporated by our Thm. 3.2, while the flat SPD MLRs under ( α, β ) -LEM and θ -LCM in [16, Cor. 4.1] are special cases of our Thm. 4.2. Furthermore, our approach extends the scope of prior work as neither [16] nor [50] explored SPD MLRs based on ( θ, α, β ) -EM and 2 θ -BWM. The gyro operations in [50, Eq. (1)] implicitly requires geodesic completeness, whereas ( θ, α, β ) -EM

x

4

2

0

2

y

2

0

2

z

4

2

0

2

4

and 2 θ -BWM are incomplete. As neither ( θ, α, β ) -EM nor 2 θ -BWM belong to pullback Euclidean metrics, the framework presented in [16] cannot be applied to these metrics. To the best of our knowledge, our work is the first to apply PEM and BWM to establish Riemannian neural networks, opening up new possibilities for utilizing these metrics in machine learning applications. Besides, neither Nguyen and Yang [50] nor Chen et al. [16] explore the deformed metrics for building SPD MLRs.

5 Lie multinomial logistic regression

This section introduces our Lie MLR on SO( n ) based on the general RMLR framework in Thm. 3.3. The Riemannian metric on SO( n ) is assumed to be the invariant metric in Tab. 13.

The two ways to generate ˜ A k in RMLR, i.e., Eqs. (12) and (13), are equivalent on SO( n ) .

Lemma 5.1. [ ↓ ]

Similar with SPD MLRs, we set Q = I . The Lie MLR on SO( n ) is presented in the following.

where P k ∈ SO( n ) and A k ∈ so ( n ) .

We refer to the Riemannian hyperplanes (Eq. (5)) on SO( n ) as Lie hyperplanes. As SO(3) is homeomorphic to 3-dimensional real projective space RP 3 [26], Fig. 3 illustrates Lie hyperplanes in the closed ball in R 3 of radius π .

6 Experiments

We first validate our SPD MLRs on four SPD neural networks: SPDNet [30] and TSMNet [37] for Riemannian feedforward networks, RResNet [36] for Riemannian residual networks, and SPDGCN [76] for Riemannian graph neural networks. Then, we proceed with experiments of our Lie MLR under the classic LieNet architecture [31]. The classifier in all the above networks is the LogEig MLR (matrix logarithm + FC + softmax), a Euclidean MLR on the tangent space at the identity matrix. We substitute the original non-intrinsic LogEig MLR in each baseline model with our RMLRs. Notably, the gyro SPD MLRs [50] are special cases of our SPD MLRs under the standard AIM, LEM, and LCM ( ( θ, α, β ) = (1 , 1 , 0) ), while flat SPD MLRs [16] are incorporated by our SPD MLRs under ( α, β ) -LEM and θ -LCM. More implementation details are presented in App. G.

6.1 Experiments on the proposed SPD MLRs

In the following, we abbreviate SPD MLR-metric as metric . For instance, ( θ, α, β ) -AIM denotes the baseline endowed with the SPD MLR induced by ( θ, α, β ) -AIM and (1,1,0) as the value of ( θ, α, β ) .

6.1.1 Experiments on the Riemannian feedforward network

We evaluate our SPD MLRs for Riemannian feedforward networks under the SPDNet and TSMNet backbones. Following [30, 10], on SPDNet, we use the Radar dataset [10] for radar recognition and the HDM05 dataset [44] for human action recognition. TSMNet [37] is one of the stateof-the-art methods for the EEG classification task. Following [37], we use the Hinss2021 [28] dataset. For each family of SPD MLRs, we report the SPD MLR induced from the standard metric ( θ = 1 , α = 1 , β = 0 ), and the one induced from the deformed metric with best ( θ, α, β ) . Besides, if the standard SPD MLR is already saturated, we only report the results of the standard one. Under each metric, We highlight the results in bold of our SPD MLR under the best hyperparameters. We visualize the results in App. G.1.6.

Radar: In line with [10], we evaluate our classifiers under two network architectures: 2-Block and 5-Block configurations. The 10-fold results (mean±std) are presented in Tab. 3. Note that the SPD MLR induced by standard AIM is saturated. Generally speaking, our SPD MLRs achieve superior performance against the vanilla LogEig MLR. Moreover, for most families of metrics, the associated SPD MLRs with proper ( θ, α, β ) outperform the standard SPD MLR, demonstrating the effectiveness of our parameterization. Besides, among all SPD MLRs, the ones induced by ( α, β ) -LEM achieve the best performance.

HDM05: Following [30], three architectures are adopted: 1-Block, 2-Block and 3-Block configurations. The 10-fold results (mean±std) are presented in Tab. 4. Note that the standard SPD MLRs under AIM, LEM, and BWM are already saturated on this dataset. As the Radar dataset, similar observations can be made on this dataset. Our SPD MLRs can bring consistent performance gain for SPDNet, and properly selected hyperparameters can bring further improvement. Particularly, among all the SPD MLRs, the ones based on the 2 θ -BWM and ( θ, α, β ) -EM achieve the best performance. Compared to the vanilla LogEig MLR, the highest performance improvement is 14.23% , highlighting our approach's effectiveness. Notably, since 2 θ -BWM and ( θ, α, β ) -EM are geodesically incomplete and not pulled back from a Euclidean space, the SPD MLR under these two metrics can not be derived by the framework of gyro or flat MLR. This contrast confirms the applicability of our theoretical framework to a broader range of geometries.

Hinss2021: The results (mean±std) of leaving 5% out cross-validation are reported in Tabs. 5 and 6. Once again, our intrinsic classifiers demonstrate improved performance compared to the LogEig MLR in both inter-session and inter-subject scenarios. Besides, the SPD MLRs based on θ -LCM achieve the best performance, outperforming the vanilla classifier by 2.6% for inter-session and by 4.46% for inter-subject . This finding highlights the versatility of our framework.

6.1.2 Experiments on the Riemannian residual network

Following [36], we use the HDM05 and NTU60 [55] datasets on the RResNet backbone. For the hyperparameter ( θ, α, β ) in our SPD MLRs, we borrow the best ones from Tab. 4. Tab. 7 reports the 10-fold and 5-fold results on the HDM05 and NTU datasets. The SPD MLRs still consistently outperform the vanilla LogEig MLR. Besides, similar to the SPD MLRs under the SPDNet backbone for action recognition (Tab. 4), the SPD MLR based on θ -LCM, 2 θ -BWM, or ( θ, α, β ) -EM outperforms the vanilla LogEig MLR by a large margin. Especially, the highest performance improvement is 13.72% and 8.4% on these two datasets.

6.1.3 Experiments on the Riemannian graph network

We use SPDGCN [76] as the backbone network for the Riemannian graph network. Following [76], we use the Disease [2], Cora [54], and Pubmed [46] datasets for node classification. The 10-fold average and maximum results of the vanilla LogEig MLR against our SPD MLR with best ( θ, α, β ) are reported in Tab. 8. Similar to the previous results, our SPD MLRs outperform the LogEig MLR. Besides, the SPD MLR based on ( α, β ) -LEM generally achieves the best performance for SPDGCN.

6.1.4 Ablations of SPD MLRs on direct classification

For a more straightforward comparison, we compare LogEig against our SPD MLRs for direct classification. We adopt the Radar, HDM05, and Hinss2021 datasets. We follow the preprocessing of

SPDNet and TSMNet to model features into the SPD manifold and directly use LogEig or our SPD MLRs for classification. The average results are presented in Tab. 9. The hyperparameters ( θ, α, β ) are borrowed from Tabs. 3 to 6. Our SPD MLRs consistently outperform the vanilla LogEig MLR. Particularly on the HDM05 dataset, the highest performance improvement by our SPD MLRs is 18.34% , surpassing the non-intrinsic LogEig MLR by a large margin. Ablations on model efficiency are also discussed in App. G.1.5.

6.2 Experiments on the proposed Lie MLR

We apply our Lie MLR into the classic SO( n ) network, i.e., LieNet [31], where features are on the Lie group of SO(3) × · · · × SO(3) . Following LieNet [31], we use G3D [6] and HDM05 [44] datasets. We also extend the Riemannian optimization package geoopt [4] into SO(3) , allowing for Riemannian optimization. We find that Riemannian SGD performs best for LieNet. Tab. 10 presents the 10-fold average results of LieNet with or without Lie MLR. Note that on the HDM05 datasets, the LieNet might fail to converge, fluctuating between the validation accuracy of 70% - 75%. Therefore, we select 10-fold best performance out of 20-fold experiments. It can be observed that our Lie MLR can improve the performance of LieNet. Besides, our Lie MLR can also improve the training stability. On the HDM05 dataset, LieNet fails to converge in 8 out of 20 folds. However, when endowed with our Lie MLR, LieNet+LieMLR only encounters convergence failures in 2 folds.

7 Conclusions

This paper presents a novel and versatile framework for designing RMLR for general geometries, with a specific focus on SPD manifolds and SO( n ) . On the SPD manifold, we systematically explore five families of Riemannian metrics and utilize them to construct five families of deformed SPD MLRs. On SO( n ) , we develop the Lie MLR for classifying rotation matrices. Extensive experiments demonstrate the superiority of our intrinsic classifiers. We expect that our work could present a promising direction for designing intrinsic classifiers on diverse geometries.

Acknowledgments and Disclosure of Funding

This work was partly supported by the MUR PNRR project FAIR (PE00000013) funded by the NextGenerationEU, the EU Horizon project ELIAS (No. 101120237), a donation from Cisco, the National Natural Science Foundation of China (62306127), the Natural Science Foundation of Jiangsu Province (BK20231040), and the Fundamental Research Funds for the Central Universities (JUSRP124015). The authors also gratefully acknowledge the financial support from the China Scholarship Council (CSC).

References

P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds . Princeton University Press, 2008.

Roy M Anderson and Robert M May. Infectious diseases of humans: dynamics and control . Oxford University Press, 1991.

Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and simple computations on tensors with Log-Euclidean metrics. PhD thesis, INRIA, 2005.

Gary Bécigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods. arXiv preprint arXiv:1810.00760 , 2018.

Rajendra Bhatia, Tanvi Jain, and Yongdo Lim. On the Bures-Wasserstein distance between positive definite matrices. Expositiones Mathematicae , 37(2):165-191, 2019.

Victoria Bloom, Dimitrios Makris, and Vasileios Argyriou. G3D: A gaming action dataset and real time action recognition evaluation framework. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops , pages 7-12. IEEE, 2012.

Silvere Bonnabel, Anne Collard, and Rodolphe Sepulchre. Rank-preserving geometric means of positive semi-definite matrices. Linear Algebra and its Applications , 438(8):3202-3216, 2013.

Nicolas Boumal and P-A Absil. A discrete regression method on manifolds and its application to data on SO(n). IFAC Proceedings Volumes , 44(1):2284-2289, 2011.

Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine , 34 (4):18-42, 2017.

Daniel Brooks, Olivier Schwander, Frédéric Barbaresco, Jean-Yves Schneider, and Matthieu Cord. Riemannian batch normalization for SPD neural networks. In Advances in Neural Information Processing Systems , volume 32, 2019.

James W Cannon, William J Floyd, Richard Kenyon, Walter R Parry, et al. Hyperbolic geometry. Flavors of geometry , 31(59-115):2, 1997.

Rudrasis Chakraborty, Chun-Hao Yang, Xingjian Zhen, Monami Banerjee, Derek Archer, David Vaillancourt, Vikas Singh, and Baba Vemuri. A statistical recurrent model on the manifold of symmetric positive definite matrices. Advances in Neural Information Processing Systems , 31, 2018.

Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domainspecific batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7354-7362, 2019.

Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, and Josef Kittler. Hybrid Riemannian graph-embedding metric learning for image set classification. IEEE Transactions on Big Data , 2021.

Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, and Josef Kittler. Riemannian local mechanism for SPD neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 7104-7112, 2023.

Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, Xiaojun Wu, and Nicu Sebe. Riemannian multinomial logistics regression for SPD neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2024.

Ziheng Chen, Yue Song, Yunmei Liu, and Nicu Sebe. A Lie group approach to Riemannian batch normalization. In The Twelfth International Conference on Learning Representations , 2024.

Ziheng Chen, Yue Song, Xiao-Jun Wu, Gaowen Liu, and Nicu Sebe. Understanding matrix function normalizations in covariance pooling through the lens of Riemannian geometry. arXiv preprint arXiv:2407.10484 , 2024.

Ziheng Chen, Yue Song, Xiao-Jun Wu, and Nicu Sebe. Product geometries on Cholesky manifolds with applications to SPD manifolds. arXiv preprint arXiv:2407.02607 , 2024.

Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, and Nicu Sebe. Adaptive Log-Euclidean metrics for SPD matrix learning. IEEE Transactions on Image Processing , 2024.

Manfredo Perdigao Do Carmo and J Flaherty Francis. Riemannian Geometry , volume 6. Springer, 1992.

Ian L Dryden, Xavier Pennec, and Jean-Marc Peyrat. Power Euclidean metrics for covariance matrices with application to diffusion tensor imaging. arXiv preprint arXiv:1009.3045 , 2010.

Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. Advances in Neural Information Processing Systems , 31, 2018.

Alexandre Gramfort. MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience , 7, 2013.

Brian C Hall and Brian C Hall. Lie groups, Lie algebras, and representations . Springer, 2013.

Richard Hartley, Jochen Trumpf, Yuchao Dai, and Hongdong Li. Rotation averaging. International Journal of Computer Vision , 103:267-305, 2013.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 770-778, 2016.

Marcel F. Hinss, Ludovic Darmet, Bertille Somon, Emilie Jahanpour, Fabien Lotte, Simon Ladouce, and Raphaëlle N. Roy. An EEG dataset for cross-session mental workload estimation: Passive BCI competition of the Neuroergonomics Conference 2021, 2021.

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation , 9 (8):1735-1780, 1997.

Zhiwu Huang and Luc Van Gool. A Riemannian network for SPD matrix learning. In Thirty-first AAAI Conference on Artificial Intelligence , 2017.

Zhiwu Huang, Chengde Wan, Thomas Probst, and Luc Van Gool. Deep learning on Lie groups for skeleton-based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 6099-6108, 2017.

Zhiwu Huang, Jiqing Wu, and Luc Van Gool. Building deep networks on Grassmann manifolds. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018.

Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix backpropagation for deep networks with structured layers. In Proceedings of the IEEE International Conference on Computer Vision , pages 2965-2973, 2015.

Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Training deep networks with structured layers by matrix backpropagation. arXiv preprint arXiv:1509.07838 , 2015.

Vinay Jayaram and Alexandre Barachant. MOABB: trustworthy algorithm benchmarking for BCIs. Journal of Neural Engineering , 15(6):066011, 2018.

Isay Katsman, Eric Ming Chen, Sidhanth Holalkere, Anna Asch, Aaron Lou, Ser-Nam Lim, and Christopher De Sa. Riemannian residual neural networks. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Reinmar Kobler, Jun-ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD domainspecific batch normalization to crack interpretable unsupervised domain adaptation in EEG. Advances in Neural Information Processing Systems , 35:6219-6235, 2022.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems , 25, 2012.

Guy Lebanon and John Lafferty. Hyperplane margin classifiers on the multinomial manifold. In Proceedings of the Twenty-first International Conference on Machine Learning , page 66, 2004.

John M Lee. Introduction to smooth manifolds . Springer, 2013.

Zhenhua Lin. Riemannian geometry of symmetric positive definite matrices via Cholesky decomposition. SIAM Journal on Matrix Analysis and Applications , 40(4):1353-1370, 2019.

Federico López, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Vectorvalued distance and Gyrocalculus on the space of symmetric positive definite matrices. Advances in Neural Information Processing Systems , 34:18350-18366, 2021.

Hà Quang Minh. Alpha Procrustes metrics between positive definite operators: a unifying formulation for the Bures-Wasserstein and Log-Euclidean/Log-Hilbert-Schmidt metrics. Linear Algebra and its Applications , 636:25-68, 2022.

Meinard Müller, Tido Röder, Michael Clausen, Bernhard Eberhardt, Björn Krüger, and Andreas Weber. Documentation mocap database HDM05. Technical report, Universität Bonn, 2007.

Richard M Murray, Zexiang Li, and S Shankar Sastry. A mathematical introduction to robotic manipulation . CRC press, 2017.

Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs , volume 8, page 1, 2012.

Xuan Son Nguyen. Geomnet: A neural network based on Riemannian geometries of SPD matrix space and Cholesky space for 3D skeleton-based interaction recognition. In Proceedings of the IEEE International Conference on Computer Vision , pages 13379-13389, 2021.

Xuan Son Nguyen. The Gyro-structure of some matrix manifolds. In Advances in Neural Information Processing Systems , volume 35, pages 26618-26630, 2022.

Xuan Son Nguyen. A Gyrovector space approach for symmetric positive semi-definite matrix learning. In Proceedings of the European Conference on Computer Vision , pages 52-68, 2022.

Xuan Son Nguyen and Shuo Yang. Building neural networks on matrix manifolds: A Gyrovector space approach. arXiv preprint arXiv:2305.04560 , 2023.

Xuan Son Nguyen, Shuo Yang, and Aymeric Histace. Matrix manifold neural networks++. In The Twelfth International Conference on Learning Representations , 2024.

Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computing. International Journal of Computer Vision , 66(1):41-66, 2006.

Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3D deep learning with PyTorch3D. arXiv preprint arXiv:2007.08501 , 2020.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina EliassiRad. Collective classification in network data. AI magazine , 29(3):93-93, 2008.

Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+ D: A large scale dataset for 3D human activity analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1010-1019, 2016.

Ondrej Skopek, Octavian-Eugen Ganea, and Gary Bécigneul. Mixed-curvature variational autoencoders. arXiv preprint arXiv:1911.08411 , 2019.

Yue Song, Nicu Sebe, and Wei Wang. Why approximate matrix square root outperforms accurate SVD in global covariance pooling? In Proceedings of the IEEE International Conference on Computer Vision , pages 1115-1123, 2021.

Yue Song, Nicu Sebe, and Wei Wang. On the eigenvalues of global covariance pooling for finegrained visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(3):3554-3566, 2022.

Yann Thanwerdas and Xavier Pennec. Is affine-invariance well defined on SPD matrices? a principled continuum of metrics. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27-29, 2019, Proceedings 4 , pages 502-510. Springer, 2019.

Yann Thanwerdas and Xavier Pennec. Exploration of balanced metrics on symmetric positive definite matrices. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27-29, 2019, Proceedings 4 , pages 484-493. Springer, 2019.

Yann Thanwerdas and Xavier Pennec. The geometry of mixed-Euclidean metrics on symmetric positive definite matrices. Differential Geometry and its Applications , 81:101867, 2022.

Yann Thanwerdas and Xavier Pennec. Theoretically and computationally convenient geometries on full-rank correlation matrices. SIAM Journal on Matrix Analysis and Applications , 43(4): 1851-1872, 2022.

Yann Thanwerdas and Xavier Pennec. O (n)-invariant Riemannian metrics on SPD matrices. Linear Algebra and its Applications , 661:163-201, 2023.

Loring W.. Tu. An introduction to manifolds . Springer, 2011.

Abraham A Ungar. Analytic hyperbolic geometry: Mathematical foundations and applications . World Scientific, 2005.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems , volume 30, 2017.

Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3D skeletons as points in a Lie group. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 588-595, 2014.

Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang, and Peihua Li. Deep CNNs meet global covariance pooling: Better representation and generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(8):2582-2597, 2020.

Rui Wang, Xiao-Jun Wu, and Josef Kittler. SymNet: A simple symmetric positive definite manifold deep learning method for image set classification. IEEE Transactions on Neural Networks and Learning Systems , 33(5):2208-2222, 2021.

Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. DreamNet: A deep Riemannian manifold network for SPD matrix learning. In Proceedings of the Asian Conference on Computer Vision , pages 3241-3257, 2022.

Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. Learning a discriminative SPD manifold neural network for image set classification. Neural networks , 151:94-110, 2022.

Rui Wang, Chen Hu, Ziheng Chen, Xiao-Jun Wu, and Xiaoning Song. A Grassmannian manifold self-attention network for signal classification. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages 5099-5107, 2024.

Rui Wang, Xiao-Jun Wu, Ziheng Chen, Cong Hu, and Josef Kittler. SPD manifold deep metric learning for image set classification. IEEE Transactions on Neural Networks and Learning Systems , 2024.

Or Yair, Mirela Ben-Chen, and Ronen Talmon. Parallel transport on the cone manifold of SPD matrices for domain adaptation. IEEE Transactions on Signal Processing , 67(7):1797-1811, 2019.

Hongwei Yong, Jianqiang Huang, Deyu Meng, Xiansheng Hua, and Lei Zhang. Momentum batch normalization for deep learning with small batch size. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16 , pages 224-240. Springer, 2020.

Wei Zhao, Federico Lopez, J Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, and Steve Trettel. Modeling graphs beyond hyperbolic: Graph neural networks in symmetric positive definite matrices. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 122-139. Springer, 2023.

Appendix Contents

A Limitations and future avenues

Limitation: Recalling our RMLR in Eq. (11), our RMLR might be over-parameterized. In our RMLR, each class would require a Riemannian parameter P k and Euclidean parameter A k . Consequently, as the number of classes grows, the classification layer would become burdened with excessive parameters. We will address this problem in future work.

Future work: We highlight the advantage of our approach compared to existing methods that our framework only requires the Riemannian logarithm, which is commonly satisfied by various manifolds encountered in machine learning. Therefore, as a future avenue, our framework offers various possibilities for designing intrinsic classifiers for neural networks on other manifolds.

B Preliminaries

B.1 Notations

We briefly summarize the notations in Tab. 11 for better clarity.

B.2 Brief review of Riemannian geometry

Intuitively, manifolds are locally Euclidean spaces. Differentials are the generalization of derivatives in classic calculus. For more details on smooth manifolds, please refer to [64, 40]. Riemannian manifolds are the manifolds endowed with Riemannian metrics, which can be intuitively viewed as point-wise inner products.

Definition B.1 (Riemannian Manifolds) . ARiemannian metric on M is a smooth symmetric covariant 2-tensor field on M , which is positive definite at every point. A Riemannian manifold is a pair {M , g } , where M is a smooth manifold and g is a Riemannian metric.

W.l.o.g., we abbreviate {M , g } as M . The Riemannian metric g induces various Riemannian operators, including the geodesic, exponential, and logarithmic maps, and parallel transportation. These

operators correspond to straight lines, vector addition, vector subtraction, and parallel displacement in Euclidean spaces, respectively [52, Tabel 1]. A plethora of discussions on Riemannian geometry can be found in [21].

When a manifold M is endowed with a smooth operation, it is referred to as a Lie group.

Definition B.2 (Lie Groups) . A manifold is a Lie group, if it forms a group with a group operation ⊙ such that m ( x, y ) ↦→ x ⊙ y and i ( x ) ↦→ x -1 ⊙ are both smooth, where x -1 ⊙ is the group inverse of x .

Lastly, we review the definition of pullback metric, a common technique in Riemannian geometry. This idea is a natural generalization of bijection from set theory.

Definition B.3 (Pullback Metrics) . Suppose M , N are smooth manifolds, g is a Riemannian metric on N , and f : M→N is smooth. Then the pullback of g by f is defined point-wisely,

where p ∈ M , f ∗ ,p ( · ) is the differential map of f at p , and V i ∈ T p M . If f ∗ g is positive definite, it is a Riemannian metric on M , which is called the pullback metric defined by f .

B.3 Basic geometries of SPD manifolds

Let S n ++ be the set of n × n symmetric positive definite (SPD) matrices. As shown in [3], S n ++ is an open submanifold of the Euclidean space S n of symmetric matrices. There are five kinds of popular Riemannian metrics on S n ++ : Affine-Invariant Metric (AIM) [52], Log-Euclidean Metric (LEM) [3], Power-Euclidean Metrics (PEM) [22], Log-Cholesky Metric (LCM) [41], and Bures-Wasserstein Metric (BWM) [5]. Note that, when power equals 1, the PEM is reduced to the Euclidean Metric (EM). The standard LEM, AIM, and EM have been generalized into parametrized families of metrics. We define ST = { ( α, β ) ∈ R 2 | min( α, α + nβ ) > 0 } , and denote the O( n ) -invariant Euclidean metric on S n [63] as

where ( α, β ) ∈ ST , and ⟨· , ·⟩ is the Frobenius inner product. By O( n ) -invariant Euclidean metric on S n , Thanwerdas and Pennec [63] generalized AIM, LEM, and EM into two-parameters families of O( n ) -invariant metrics, i.e., ( α, β ) -AIM, ( α, β ) -LEM, and ( α, β ) -EM, with ( α, β ) ∈ ST . We denote the metric tensor of ( α, β ) -AIM, ( α, β ) -LEM, ( α, β ) -EM, LCM, and BWM as g ( α,β ) -AIM , g ( α,β ) -LEM , g ( α,β ) -EM , g LCM , and g BWM , respectively.

For any SPD points P, Q ∈ S n ++ and tangent vectors V, W ∈ T P S n ++ , we follow the notations in Tab. 11 and further denote ˜ V = Chol ∗ ,P ( V ) , ˜ W = Chol ∗ ,P ( W ) , L = Chol P , and K = Chol Q . For parallel transportation under the BWM, we only present the case where P, Q are commuting matrices, i.e., P = U Σ U ⊤ and Q = U ∆ U ⊤ . We summarize the associated Riemannian operators and properties in Tab. 12. Although there also exist other metrics on SPD manifolds [60, 61, 63], their lack of closed-form Riemannian operators makes them problematic to be applied in machine learning.

B.4 Basic geometry of rotation matrices

We denote R ∈ SO( n ) , A 1 , A 2 ∈ T R SO( n ) , U ∈ R n × n , skew( A ) = A -A ⊤ 2 , and Q ( · ) as the function return an orthogonal matrix by QR decomposition. There are two equivalent representations for the tangent vector on SO( n ) . In this paper, we use the Lie algebra representation, i.e., T R SO( n ) ∼ = so ( n ) as the set of skew-symmetric matrices. We summarize all the necessary Riemannian ingredients for SO( n ) in Tab. 13.

For the specific case of R ∈ SO(3) , R can be represented by the Euler angle and axis [26, Sec. 3.2]:

Besides, the matrix logarithm on SO(3) can be calculated without decomposition [45, Ex. A.14]:

where θ is the Euler angle. Obviously, the matrix logarithm is related to the Euler angle and axis when θ = 0 .

̸

C RMLRas a natural extension of the Euclidean MLR

Proposition C.1. When M = R n is the standard Euclidean space, the RMLR defined in Thm. 3.3 becomes the Euclidean MLR in Eq. (1) .

Proof. On the standard Euclidean space R n , Log y x = x -y, ∀ x, y ∈ R n . Besides, the differential maps of left translation and parallel transportation are the identity maps. Therefore, given x, p k ∈ R n and a k ∈ R n / { 0 } ∼ = T 0 R n / { 0 } , we have

where b k = ⟨ x, p k ⟩ .

D Gyro SPSD MLR as special cases of our RMLR

Gyro SPSD MLR [51] is derived by the product of the Grassmannian and SPD gyro spaces. This section will show that the gyro SPSD MLR is the special case of our RMLR on the product geometry of the SPSD manifold. We first review some necessary results about gyro SPSD MLR and then show the equivalence.

Following the notations in [51], we denote the Grassmannian with canonical metric under the projector and ONB perspective as Gr( p, n ) and ˜ Gr( p, n ) , respectively. The space of n × n SPSD matrices with a fixed rank p , denoted as S + n,p , forms an SPSD manifold [7]. As shown in [7, 51], the SPSD manifold is a product space, i.e., S + n,p ∼ = ˜ Gr( p, n ) ×S p ++ . In other words, every P ∈ S + n,p can be decomposed as P = U P S P U ⊤ P with U p ∈ ˜ Gr( p, n ) and S P ∈ S p ++ . We further denote S p,g ++ as the SPD manifold with metric g , where g could be AIM, LEM, and LCM. As shown in [51], the gyro space in S + n,p can be defined by the product of gyro spaces of ˜ Gr( p, n ) and S p,g ++ . By this product structure, Nguyen et al. [51] proposed the SPSD Pseudo-gyrodistance to a hyperplane.

Definition D.1. (SPSD Hypergyroplanes [51]) Let P, W ∈ ˜ Gr( p, n ) ×S n,g ++ . Then hypergyroplanes in structure space ˜ Gr( p, n ) ×S n,g ++ are defined as

where ⊕ psd,g and ⟨ , ⟩ psd,g are gyro addition and gyro inner product, which are defined in [51].

Theorem D.2. (SPSD Pseudo-gyrodistance [51]) Let W = ( U W , S W ) , P = ( U P , S P ) , X = ( U X , S X ) ∈ ˜ Gr( p, n ) ×S n,g ++ , and H psd,g A,P be a hypergyroplane in structure space ˜ Gr( p, n ) ×S n,g ++ . Then the pseudo-gyrodistance from X to H psd,g A,P is given by

where ∥ . ∥ gr and ∥ . ∥ g are the gyro norms on the Grassmann and SPD [51], and ⟨ , ⟩ gr and ⟨ , ⟩ g are gyro inner products [51]. ˜ ⊕ gr and ⊕ g are gyro additions on ˜ Gr( p, n ) and S p,g ++ .

Denoting g gr as the canonical metric on ˜ Gr( p, n ) and g as AIM, LEM, or LCM, we can prove that Thm. D.2 is the special case of our Thm. 3.2.

Theorem D.3. Under the product metric g psd,g = λg gr × g , the Riemannian hyperplane in Eq. (5) on the SPSD manifold equals the SPSD hypergyroplane in Def. D.1. Similarly, the Riemannian margin distance in Thm. 3.2 on the SPSD manifold equals SPSD Pseudo-gyrodistance in Thm. D.2.

Proof. Following the notations in Def. D.1 and Thm. D.2, we further denote P = U P S P U ⊤ P , Q = U Q S Q U ⊤ Q , W = U W S W U ⊤ W , and X = U X S X U ⊤ X with U P , U Q , U W , U X ∈ ˜ Gr( p, n ) and S P , S Q , S W , S X ∈ S p,g ++ . I p is the p × p identity matrix. ˜ I p,n = ( I p , 0) ⊤ is the gyro identity on ˜ Gr( p, n ) . ˜ Γ gr , ˜ Log gr , and ⟨ , ⟩ gr U p are Riemannian parallel transport along a geodesic, logarithm and Riemannian metric at U p on ˜ Gr( p, n ) . Γ g , Log g , and ⟨ , ⟩ g S p are Riemannian parallel transport along a geodesic, logarithm and Riemannian metric at S p on S p,g ++ . Γ psd,g , Log psd,g , and ⟨ , ⟩ psd,g X are Riemannian parallel transport along a geodesic, logarithm and Riemannian metric at X on S + n,p ∼ = ˜ Gr( p, n ) ×S p ++ .

First, we show that the SPSD hypergyroplane equals our Riemannian hyperplane in Eq. (5). We have the following

where ˜ A U W = ˜ Γ gr ˜ I p,n → U P ( ˜ Log gr ˜ I p,n ( U W ) ) , ˜ A S W = Γ g I p → S P ( Log g I p ( S W ) ) and ˜ A = ( ˜ A U W , ˜ A S W ) ∈ T P S + n,p ∼ = T U P ˜ Gr( p, P ) ⊗ T S P S p,g ++ with ⊗ as the Cartesian product. The above derivation comes from the following.

The definition of gyro addition, gyro inverse, and gyro inner product on the SPSD manifold [51, Sec. 3.3].

The proof of [51, Prop. 3.2] indicates that similar results also hold on the Grassmannian. Combining Prop. 3.2 and its counterparts in the Grassmannian, one can obtain the equation.

The Riemannian product geometry.

By the product geometry of the SPSD manifold, we can immediately get

where I p,n = ˜ I p,n ˜ I ⊤ p,n is the gyro identity on the SPSD manifold.

Next, we show the equivalence between SPSD pseudo-gyrodistance and our Riemannian margin distance:

The definition of gyro addition, gyro inverse, gyro inner product, and gyro norm on the SPSD manifold.

Eq. (33).

The definition of SPSD gyro norm [51].

Riemannian parallel transportation maintains the norm of the tangent vector [21, Def. 3.1]

Eq. (34)

Thm. 3.2

Remark D.4 . We make the following remark w.r.t. gyro and our MLR on the SPSD manifold.

Eq. (34) indicates that when generating ˜ A in our RMLR by parallel transporting a tangent vector A ∈ T I p,n S + n,p , ˜ A is the initial velocity of W in Eq. (32).

Putting pseudo-gyrodistance and Riemannian margin distance into Eq. (4), one can get gyro MLR and our Riemannian MLR. Therefore, Thm. D.3 indicates the equivalence of the gyro MLR with our RMLR on the SPSD.

As g are required to induce gyro structures, the metric g in gyro SPSD MLR is confined within AIM, LEM, and LCM. However, our SPSD MLR can be the product space of the Grassmannian and SPD manifold under other metrics, such as BWM and PEM, as our framework does not require gyro structures.

E Theories on the deformed metrics

E.1 Limiting cases of the deformed metrics

Thanwerdas and Pennec [59] generalized ( α, β ) -AIM into three-parameters families of metrics by power deformation, i.e., ( θ, α, β ) -AIM. The family of ( θ, α, β ) -AIM comprises ( α, β ) -AIM for θ = 1 and approaches ( α, β ) -LEM with θ → 0 [59].

Chen et al. [17] extended LCM and ( α, β ) -LEM into power-deformed metrics, denoted as ( θ, α, β ) -LEM and θ -LCM. The authors show that ( θ, α, β ) -LEM is equal to ( α, β ) -LEM, and θ -LCM interpolates between ˜ g -LEM ( θ → 0 ) and LCM ( θ = 1 ), with ˜ g -LEM defined as

where ˜ V i = log ∗ ,P ( V i ) with log ∗ ,P as the differential map of matrix logarithm, and D ( V i ) is a diagonal matrix consisting of the diagonal elements of V i .

Thanwerdas and Pennec [61] identified the Alpha-Procrustes metric [43] with power-deformed BWM, denote as 2 θ -BWM. Similarly, 2 θ -BWM becomes BWM with θ = 0 . 5 [61]. We further show the limiting case of 2 θ -BWM under θ → 0 .

Before starting the proof, we first recall a well-known property of deformed metrics [61].

Lemma E.2. Let 1 θ 2 ϕ ∗ θ g be the deformed metric on SPD manifolds pulled back from g by the matrix power ϕ θ and scaled by 1 θ 2 . Then when θ tends to 0, for all P ∈ S n ++ and all V ∈ T P S n ++ , we have

Now, we present our proof for the limiting cases of deformed metrics.

Proof of Prop. E.1. First, we have

By Lem. E.2, we have the following:

E.2 Proof of the properties of the deformed metrics (Tab. 2)

In this subsection, we prove the properties presented in Tab. 2. We first present a useful lemma and then present our detailed proof. This lemma will be useful in the proof of our SPD MLRs as well.

Lemma E.3. Supposing a Riemannian manifold {M , g } and a positive real scalar a > 0 , the scaling metric ag over M shares the same Riemannian logarithmic & exponential maps and parallel transportation with g .

Proof. Since the Christoffel symbols of ag are identical to those of g , the geodesics and parallel transportation under both ag and g remain unchanged. The equivalence of geodesics implies that the Riemannian exponential maps are the same for ag and g . As the inverse of the Riemannian exponential maps, the Riemannian logarithm maps under ag and g are also identical.

According to Lem. E.3, the geodesic completeness is independent of the scaling factor a > 0 . By the definition of O( n ) -, left-, right-, and bi-invariance, these invariant properties are also independent of the scaling factor a > 0 . Without loss of generality, we will omit the scaling factor in the following proof.

Proof. Firstly, we prove O( n ) -invariance of ( θ, α, β ) -LEM, ( θ, α, β ) -EM, ( θ, α, β ) -AIM, and 2 θ -BWM. Since the differential of ϕ θ is O( n ) -equivariant, and ( α, β ) -LEM, ( α, β ) -EM, ( α, β ) -AIM, and BWM are O( n ) -invariant [63], O( n ) -invariance are thus acquired.

Next, we focus on geodesic completeness. It can be easily proven that Riemannian isometries preserve geodesic completeness. On the other hand, ( α, β ) -LEM, ( α, β ) -AIM, and LCM are geodesically complete [63, 41]. As a direct corollary, geodesic completeness can be obtained since ϕ θ is a Riemannian isometry.

Finally, we deal with Lie group invariance. Similarly, it can be readily proved that Lie group invariance is preserved under isometries. LCM, LEM, and ( α, β ) -AIM are Lie group bi-invariant [41], bi-invariant [3], and left-invariant [62]. As an isometric pullback metric from the standard LEM [63], ( α, β ) -LEM is, therefore, Lie group bi-invariant. As pullback metrics, ( θ, α, β ) -LEM, ( θ, α, β ) -AIM, and θ -LCM are therefore bi-invariant, left-invariant, and bi-invariant, respectively.

F Computational details on the SPD MLR under power-deformed BWM

F.1 Matrix square roots in the SPD MLR under power-deformed BWM

In the case of MLRs induced by 2 θ -BWM, computing square roots like ( BA ) 1 2 and ( AB ) 1 2 with B,A ∈ S n ++ poses a challenge. Eigendecomposition cannot be directly applied since BA and AB are no longer symmetric, let alone positive definitity. Instead, we use the following formulas to compute these square roots [43]:

where the involved square roots can be computed using eigendecomposition or singular value decomposition (SVD).

F.2 Numerical stability of the SPD MLR under power-deformed BWM

Let us first explain why we abandon parallel transportation on the SPD MLR derived from 2 θ -BWM. Then, we propose our numerically stable methods for computing the SPD MLR based on 2 θ -BWM.

F.2.1 Instability of parallel transportation under power-deformed BWM

As discussed in Thm. 3.3, there are two ways to generate ˜ A in SPD MLR: parallel transportation and Lie group translation. However, parallel transportation under 2 θ -BWM could cause numerical problems. W.l.o.g., we focus on the standard BWM as 2 θ -BWM is isometric to the BWM.

Although the general solution of parallel transportation under BWM is the solution of an ODE, for the case of parallel transportation starting from the identity matrix, we have a closed-form expression [63]:

where P = U Σ U ⊤ is the eigendecomposition of P ∈ S n ++ . There would be no problem in the forward computation of Eq. (41). However, during backpropagation (BP), Eq. (41) would require the BP of eigendecomposition, involving the calculation of 1 / ( σ i -σ j ) [33, Prop. 2]. When σ i is close to σ j , the BP of eigendecomposition could be problematic.

F.2.2 Numerically stable methods for the SPD MLR under power-deformed BWM

To bypass the instability of parallel transportation under BWM, we use Lie group left translation to generate ˜ A in MLRs induced from 2 θ -BWM. However, there is another problem that could cause instability. The computation of the Riemannian metric of 2 θ -BWM requires solving the Lyapunov operator, i.e., L P [ V ] P + P L P [ V ] = V . Under the case of symmetric matrices, the Lyapunov operator can be obtained by eigendecomposition:

where V ∈ S n , UV ′ U ⊤ = V , and P = U Σ U ⊤ is the eigendecomposition of P ∈ S n ++ . Similar with Eq. (41), the BP of Eq. (42) requires 1 / ( σ i -σ j ) , undermining the numerical stability.

To remedy this problem, we proposed the following formula to stably compute the BP of Eq. (42).

Proposition F.1. For all P ∈ S n ++ and all V ∈ S n , we denote the Lyapunov equation as

where X = L P [ V ] . Given the gradient ∂L ∂X of loss L w.r.t. X , then the BP of the Lyapunov operator can be computed by:

where L P [ · ] can be computed by Eq. (42).

Proof. Differentiating both sides of Eq. (43), we obtain

Besides, easy computations show that

where · : · denotes the standard Frobenius inner product.

Then we have the following:

Remark F.2 . Eq. (42) needs to be computed in the Lyapunov operator's forward and backward process. Therefore, in the forward process, we can save the intermediate matrices U and K with K i,j = [ 1 σ i + σ j ] i,j , and then use them to compute the backward process efficiently.

G Implementation details and additional experiments

This section offers additional details on the experiments of SPD and Lie MLRs.

G.1 Additional details and experiments on the SPD MLRs

G.1.1 Basic layers in SPDNet and TSMNet

SPDNet [30] is the most classic SPD neural network. SPDNet mimics the conventional densely connected feedforward network, consisting of three basic building blocks:

where max() is element-wise maximization. BiMap and ReEig mimic transformation and nonlinear activation, while LogEig maps SPD matrices into the tangent space at the identity matrix for classification.

SPDNetBN [10] further proposed Riemannian batch normalization based on AIM:

SPD domain-specific momentum batch normalization (SPDDSMBN) [37] is an improved version of SPDNetBN. Apart from controlling the mean, it can also control variance. The key operation in SPDDSMBN of controlling mean and variance is:

where G and ¯ v are Riemannian mean and variance. Inspired by [75], during the training stage, SPDDSMBN generates running means and running variances for training and testing with distinct momentum parameters. Besides, it sets G and ¯ v as the running mean and running variance w.r.t. training for training and the ones w.r.t. testing for testing. SPDDSMBN also applies domain-specific techniques [13], keeping multiple parallel BN layers and distributing observations according to the associated domains. To crack cross-domain knowledge, v is uniformly learned across all domains, and G is set to be the identity matrix. TSMNet [37] adopted SPDDSMBN to solve domain adaptation in EEG classification.

In the above models, the Euclidean MLR in the co-domain of matrix logarithm (matrix logarithm + FC + softmax) is used for classification. Following the terminology in [16], we call this classifier as LogEig MLR . The LogEig MLR is the Euclidean classifier in the tangent space at the identity, which might distort the innate geometry of the SPD manifold.

G.1.2 Datasets and preprocessing

Radar 2 : This dataset [10] consists of 3,000 synthetic radar signals. Following the protocol in [10], each signal is split into windows of length 20, resulting in 3,000 SPD covariance matrices of 20 × 20 equally distributed in 3 classes.

HDM05 3 : This dataset [44] contains 2,273 skeleton-based motion capture sequences executed by various actors. Each frame consists of 3D coordinates of 31 joints of the subjects, and each sequence can be, therefore, modeled by a 93 × 93 covariance matrix. Following the protocol in [10], we trim the dataset down to 2086 sequences scattered throughout 117 classes by removing some under-represented classes.

Hinss2021 4 : This dataset [28] is a recent competition dataset consisting of EEG signals for mental workload estimation. The dataset is used for two types of experiments: inter-session and inter-subject, which are modeled as domain adaptation problems. Recently, geometry-aware methods have shown promising performance in EEG classification [74, 37]. We choose the SOTA method, TSMNet [37], as our baseline model on this dataset. We follow the Python implementation 5 [37] to carry out preprocessing. In detail, the python package MOABB [35] and MNE [24] are used to preprocess the datasets. The applied steps include resampling the EEG signals to 250/256 Hz, applying temporal filters to extract oscillatory EEG activity in the 4 to 36 Hz range, extracting short segments ( ≤ 3 s) associated with a class label, and finally obtaining 40 × 40 SPD covariance matrices.

Disease [2]: It represents a disease propagation tree, simulating the SIR disease transmission model [2], with each node representing either an infection or a non-infection state.

Cora [54]: It is a citation network where nodes represent scientific papers in the area of machine learning, edges are citations between them, and node labels are academic (sub)areas.

Pubmed [46]: This is a standard benchmark describing citation networks where nodes represent scientific papers in the area of medicine, edges are citations between them, and node labels are academic (sub)areas.

For the Disease, Cora and Pubmed datasets, we follow [76] to model features into S 3 ++ .

G.1.3 Implementation details

SPDNet [30] and TSMNet [37]: We follow the official Pytorch code of SPDNetBN 6 and TSMNet 7 to implement our experiments. To evaluate the performance of our intrinsic classifiers, we substitute the LogEig MLR in SPDNet and TSMNet with our SPD MLRs. We implement our SPD MLRs induced from five parameterized metrics. On the Radar and HDM05 datasets, the learning rate is 1 e -2 , and the batch size is 30. On the Hinss2021 dataset, following [37], the learning rate is 1 e -3 with a 1 e -4 weight decay, and batch size is 50. The maximum training epoch is 200, 200, and 50, respectively. We use the standard-cross entropy loss as the training objective and optimize the parameters with the Riemannian AMSGrad optimizer [4].

RResNet [36]: We focus on the AIM-based RResNet, and use the official code 8 and suggested network settings to implement the experiments on the RResNet. We conduct 10-fold and 5-fold experiments on the HDM05 and NTU datasets. Since RResNet is developed based on SPDNet, we use the same learning settings with the SPDNet for the action recognition task, and borrow the best ( θ, α, β ) from Tab. 4 for our SPD MLRs under the RResNet backbone.

SPDGCN [76]: We use the official code 9 and the suggested network settings in [76]. Note that the SPDGCN with SPD MLR remains the same network settings as the vanilla SPDGCN. Tab. 14 presents the hyperparameters ( θ, α, β ) on different datasets.

Network Architectures: We denote the network architecture as [ d 0 , d 1 , · · · , d L ] , where the dimension of the parameter in the i -th BiMap layer (App. G.1.1) is d i × d i -1 . For SPDNet, we also validate our SPD MLRs under different network architectures on the Radar and HDM05 datasets. The network architectures on the Radar dataset are [20, 16, 8] for the 2-block configuration and [20, 16, 14, 12, 10, 8] for the 5-block configuration, while on the HDM05 dataset, the network architectures are [93, 30] for 1-block, [93. 70, 30] for 2-block, and [93, 70, 50, 30] for 3-block. For TSMNet, the 1-block architecture is [40,20].

Scoring Metrics: In line with the previous work [10, 37, 76, 36], we use balanced accuracy, the average recall across classes, as the scoring metric for the Hinss2021 dataset, and accuracy for other datasets. On the Hinss2021 dataset, models are fit and evaluated with a randomized leave 5% of the sessions (inter-session) or subjects (inter-subject) out cross-validation (CV) scheme. On other datasets, K-fold experiments are carried out with randomized initialization and split,

G.1.4 Hyper-parameters

We implement the SPD MLRs induced by not only five standard metrics, i.e., LEM, AIM, EM, LCM, and BWM, but also five families of parameterized metrics. Therefore, in our SPD MLRs, we have a maximum of three hyper-parameters, i.e., θ, α, β , where ( α, β ) are associated with O( n ) -invariance and θ controls deformation. For ( α, β ) in ( θ, α, β ) -LEM, ( θ, α, β ) -AIM, and ( θ, α, β ) -EM, recalling Eq. (24), α is a scaling factors, while β measures the relative significance of traces. As scaling is less important [59], we set α = 1 . As for the value of β , we select it from a predefined set: { 1 , 1 / n , 1 / n 2 , 0 , -1 / n + ϵ, -1 / n 2 } , where n is the dimension of input SPD matrices in SPD MLRs. The purpose of including ϵ ∈ R + is to ensure O( n ) -invariance ( ( α, β ) ∈ ST ). These chosen values for β allow for amplifying, neutralizing, or suppressing the trace components, depending on the characteristics of the datasets. For the deformation factor θ , we roughly select its value around its deformation boundary, i.e., [0.25,1.5] for ( θ, α, β ) -AIM, [0.5,1.5] for θ -LCM, [0.25,1.5] and ( θ, α, β ) -EM, [0.25,0.75] for 2 θ -BWM. The details values are listed in Tab. 15.

G.1.5 Model efficiency

We adopt the deepest architectures, namely [20, 16, 14, 12, 10, 8] for the Radar dataset, [93, 70, 50, 30] for the HDM05 dataset, and [40, 20] for the Hinss2021 dataset. For simplicity, we focus on the SPD MLRs induced by standard metrics, i.e., AIM, EM, LEM, BWM, and LCM. The average training time (in seconds) per epoch is reported in Tab. 16. Generally, when the number of classes is small

( e.g., 3 in the Radar and Hinss2021 datasets), our SPD MLRs only bring minor additional training time compared to the baseline LogEig MLR. However, when dealing with a larger number of classes ( e.g., 117 classes in the HDM05 dataset), there could be some inefficiency caused by our SPD MLRs. This is because each class requires an SPD parameter, and each parameter might require matrix decomposition in the forward or optimization processes during training. Nonetheless, the SPD MLRs induced by EM or LCM generally achieve comparable efficiency with the vanilla LogEig MLR. This is due to the fast computation of their Riemannian operators, making them efficient choices for tasks with a larger number of classes. This result highlights the flexibility of our framework and its applicability to various scenarios.

90

92

94

96

98

Acc

Radar

55

58

61

64

67

70

73

HDM05

SPDNet

MLR-(

,

,

)-AIM

MLR-(

,

,

)-EM

MLR-(

,

,

)-LEM

MLR-(2 )-BWM

MLR-(

)-LCM

G.1.6 Visualization

We visualize the 10-fold average results of SPDNet with different classifiers on the Radar and HDM05 datasets. We focus on the deepest architectures, i.e., . [20,16,14,12,10,8] for the Radar dataset, and [93,70,50,30] for the HDM05 dataset. Note that we only report the SPD MLR with the best hyper-parameters ( θ, α, β ) . The figures are presented in Fig. 4. All the results are sourced from Tabs. 3 and 4.

G.2 Additional details and experiments on the Lie MLR

G.2.1 Basic layers in LieNet

LieNet [31] is the most classic neural network on rotation matrices. The latent space of LieNet is the Lie group SO N (3) = SO(3) × SO(3) · · ·× SO(3) , i.e., R = ( R 1 , · · · , R N ) ∈ SO N (3) . The group and manifold structures on SO N (3) are defined by product spaces. For instance, R 1 ⊙ R 2 = ( R 1 1 R 2 1 , · · · , R 1 N R 2 N ) . There are three basic layers in LieNet:

where Θ( · ) is the Euler angle, and ( n i , m i ) are two indexes. The RotMap and RotPooling layers mimic the convolution and pooling layers, while the LogMap layer map rotation matrices into tangent space for classification. In the official Matlab implementation, the LogMap layer is implemented as the Euler axis-angle representation. The classification is performed by Euler axis-angle + FC + Softmax. As the axis-angle is an equivalent representation of matrix logarithm, we call this classifier as LogEig MLR as well. This classifier is, therefore, also non-intrinsic.

In LieNet, each rotation feature has a shape of [num, frame, 3, 3], where num and frame denote spatial and temporal dimensions. The RotPooling layer is performed either along spatial or temporal dimensions, while the RotMap layer is performed along spatial dimensions, i.e., W k with a size of [num, 3, 3].

G.2.2 Datasets and preprocessing

For a fair comparison, we follow LieNet to use G3D [6] and HDM05 datasets to validate our Lie MLR.

G3D[6]: This dataset consists of 663 sequences of 20 different gaming actions. Each sequence is recorded by 3D locations of 20 joints (i.e., 19 bones).

HDM05: We trim it down by removing some under-represented sequences, resulting in 2,326 sequences scattered throughout 122 classes. Following [30], we use the code of [67] to represent each skeleton sequence as a point on the Lie group SO N × T (3) , where N and T denote spatial and temporal dimensions. As preprocessed in [30], we set T as 100 and 16 for each sequence on the G3D and HDM05 datasets, respectively.

G.2.3 Implementation details

LieNet: Note that the official code of LieNet 10 is developed by Matlab. We follow the open-sourced Pytorch code 11 to implement our experiments. To reproduce LieNet more faithfully, we made the following modifications to this Pytorch code. We re-code the LogMap and RotPooling layers to make them consistent with the official Matlab implementation. In addition, we also extend the existing Riemannian optimization package geoopt [4] into SO(3) to allow for Riemannian version of SGD, ADAM, and AMSGrad on SO(3) , which is missing in the current package. However, we find that SGD is the best optimizer for LieNet. Therefore, we use SGD as our optimizer during the experiments.

Lie MLR: We use our Lie MLR to replace the axis-angle classifier in LieNet and call the resulting network LieNet+LieMLR. To alleviate the computational burden, we set each P k as the dimension of [num, 3, 3], where num is the spacial dimension of the input of the Lie MLR layer. In other words, P k is shared in the temporal dimension. We adopt Pytroch3D [53] to calculate the matrix logarithm. Due to the instabilities of pytorch3d.transforms.so3_log_map , we use pytorch3d.transforms.matrix_to_axis_angle first to calculate the quaternion axis and angle, and then convert this representation into matrix logarithm 12 .

Training Details: Following [31], we focus on the 3Blocks and 2Blocks architecture for the G3D and HDM05 datasets, which are the suggested architectures for these two datasets. The learning rate is 1 e -2 on both datasets, and we further set weight decay as 1 e -5 on the G3D dataset. For LieNet and LieNet+LieMLR, we use torch.nn.utils.clip_grad_norm_ for gradient clipping with a clipping factor of 5. The clipping is imposed to the dimensionality reduction weight in the final FC linear on LieNet, or, accordingly, A = { A 1 , · · · , A k } in the Lie MLR layer on LieNet+LieMLR.

Scoring Metrics: For the G3D dataset, following LieNet [31], we adopt a 10-fold cross-subject test setting, where half the subjects are used for training and the other half are employed for testing. For the HDM05 dataset, following [31], we randomly select half of the sequences for training and the rest for testing. Due the instabilities of LieNet, we conduct 20-fold experiments and select the best 10 folds to evaluate the performance.

G.3 Hardware

All experiments use an Intel Core i9-7960X CPU with 32GB RAM and an NVIDIA GeForce RTX 2080 Ti GPU.

H Proofs

H.1 Proof of Thm. 3.2

Proof of Thm. 3.2. Let us first solve Y ∗ in Eq. (8), which is the solution to the following constrained optimization problem:

Note that Eq. (61) is well-defined due to the existence of the Riemannian logarithm. Although, Eq. (61) is normally non-convex, Eq. (61) and Eq. (8) can be reduced to a Euclidean problem:

where ˜ Y = Log P Y and ˜ S = Log P S .

Let us first discuss Eq. (62). Denote the solution of Eq. (62) as ˜ Y ∗ . Note that ˜ Y ∗ is not necessarily unique. Note that Exp P is only well-defined locally. More precisely, Exp P is well-defined in an open ball B ϵ (0) centered at 0 ∈ T P M . Therefore, ˜ Y ∗ might not be in B ϵ (0) . In this case, we can scale ˜ Y ∗ into B ϵ (0) , and the scaled ˜ Y ∗ is still the maximizer of Eq. (62). Therefore, w.l.o.g., we assume ˜ Y ∗ ∈ B ϵ (0) .

Putting ˜ Y ∗ into Eq. (63), Eq. (63) is reduced to the distance to the hyperplane ⟨ ˜ Y , ˜ A ⟩ P = 0 in the Euclidean space { T P M , ⟨· , ·⟩ P } , which has a closed-form solution:

H.2 Proof of Thm. 3.3

Proof for Thm. 3.3 . Putting the margin distance (Eq. (10)) into Eq. (4), we have the following:

H.3 Proof of Prop. 4.1

Proof for Prop. 4.1 . The Riemannian metric ( α, β ) -EM at I is

By Lem. E.2, we have the following

H.4 Proof of Thm. 4.2

As the five families of metrics presented in Thm. 4.2 are pullback metrics, we first present a general result regarding Riemannian MLRs under pullback metrics.

Lemma H.1 (Riemannian MLRs under Pullback Metrics) . Supposing {N , g } is a Riemannian manifold and ϕ : M→N is a diffeomorphism between manifolds, the Riemannian MLR by parallel transportation (Eq. (11) + Eq. (12) ) on M under ˜ g = ϕ ∗ g can be obtained by g :

where ˜ A k = Γ ϕ ( Q ) → ϕ ( P k ) ϕ ∗ ,Q ( A k ) with A k ∈ T Q M , ˜ Log , ˜ Γ are Riemannian logarithm and parallel transportation under ˜ g , and Log , Γ are the counterparts under g .

Furthermore, if N has a Lie group operation ⊙ , M could be endowed with a Lie group structure ˜ ⊙ by f . The Riemannian MLR by left translation (Eq. (11) + Eq. (13) ) on M under ˜ g and ˜ ⊙ can be calculated by g and ⊙ :

where ˜ A k = L R k ∗ ,ϕ ( Q ) ◦ ϕ ∗ ,Q ( A k ) , ˜ R k = P k ˜ ⊙ Q -1 ˜ ⊙ , R k = ϕ ( P ) ⊙ ϕ ( Q ) -1 ⊙ , and ˜ L P k ˜ ⊙ Q -1 ˜ ⊙ is the left translation under ˜ ⊙ .

Proof for Lem. H.1. Before starting, we should point out that since ϕ is a diffeomorphism, ˜ ⊙ and ˜ g are indeed well defined, and {M , ˜ g } forms a Riemannian manifold and {M , ˜ ⊙} forms a Lie group. We denote ϕ -1 ∗ as the differential of ϕ -1 . We first focus on the Riemannian MLR by parallel transportation:

In the case of the Riemannian MLR by left translation, we first note that:

Therefore, the associated differential is:

Putting Eq. (75) in Eq. (71), we can obtain the results.

Now, we apply Lem. H.1 to derive the expressions of our SPD MLRs presented in Thm. 4.2. For our cases of SPD MLRs, we set Q = I . For simplicity, we will omit the subscript k for P k and A k . We will first derive the expressions of SPD MLRs under ( θ, α, β ) -LEM, θ -LCM, ( θ, α, β ) -EM, and ( θ, α, β ) -AIM, as they are sourced from Eq. (70). Then we will proceed to present the expression of MLR under 2 θ -BWM, which is sourced from Eq. (72). According to Lem. E.3, the scaled metric ag shares the same Riemannian operators as g . We will use this fact throughout the following proof.

Proof of Thm. 4.2. For simplicity, we abbreviate ϕ θ as ϕ during the proof. Note that for 2 θ -BWM, ϕ should be understood as ϕ 2 θ . We first show ϕ ( I ) and the differential map ϕ ∗ ,I , which will be frequently required in the following proof:

Denoting ϕ : {S n ++ , ˜ g } → {S n ++ , g } , then the SPD MLR under ˜ g by parallel transportation with Q = I is

Next, we begin to prove the five SPD MLRs one by one.

( α, β ) -LEM: As shown by Chen et al. [20], the standard LEM is the pullback metric from the Euclidean space S n . Similarly, ( α, β ) -LEM is also a pullback metric:

By Eq. (70), we have

θ -LCM : Simple computations show that θ -LCM is the scaled pullback metric of standard Euclidean metric in the Euclidean space of lower triangular matrices L n :

where g E is the standard Frobenius inner product, and g CM is the Cholesky metric on the Cholesky space L n + [41]. Denoting ζ = Dlog ◦ Chol ◦ ϕ , then we have

Similar with the case of ( θ, α, β ) -LEM, we have

where ˜ K = Chol( S θ ) , ˜ L = Chol( P θ ) , D ( ˜ K ) is a diagonal matrix with diagonal elements from ˜ K , and ⌊ ˜ K ⌋ is a strictly lower triangular matrix from ˜ K .

( θ, α, β ) -EM: Let η = 1 | θ | ϕ . Simple computation shows that ( θ, α, β ) -EM is the pullback metric of ( α, β ) -EM:

Besides, we have the following for η :

According to Eq. (70), we have

( θ, α, β ) -AIM: Putting g ( α,β ) -AIM into Eq. (78), we have

2 θ -BWM : We first simplify Eq. (72) under the cases of SPD manifolds and then proceed to focus on the case of g = g BWM . Denote ϕ : {S n ++ , ˜ g, ˜ ⊙} → {S n ++ , g, ⊙} , where the Lie group operation ⊙ [62] is defined as

Note that I is the identity element of {S n ++ , ⊙} , and for any S ∈ S n ++ , the differential map of the left translation L S under ⊙ is

For the induced Lie group {S n ++ , ˜ ⊙} , the left translation ˜ L P ˜ ⊙ I -1 ˜ ⊙ under ˜ ⊙ is

The associated differential at I is

where ¯ L = Chol( P 2 θ ) . Then the SPD MLRs under ˜ g and ˜ ⊙ by left translation is

Setting g = g BWM (We omit the scaling factor.), we obtain the SPD MLR under 2 θ -BWM:

H.5 Proof of Lem. 5.1

Proof of Lem. 5.1. During this proof, we use the ambient representation of tangent vectors. We only need to prove the following:

Given rotation matrices P, Q and a tangent vector H ∈ T Q SO( n ) , the parallel transportation [8, Tab. 1] is

On the other hand, given a curve c ( t ) over SO( n ) , satisfying c (0) = Q and c ′ (0) = H , the differential of the left translation L PQ -1 at Q is

which concludes the proof.

H.6 Proof of Thm. 5.2

Proof of Thm. 5.2. Lem. 5.1 indicates we can use either parallel transportation or group translation. Putting the associated expressions from Tab. 13 into Eq. (11) + Eq. (12), one can directly obtain the results.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction (Sec. 1) accurately reflect the paper's theoretical and empirical contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations specifically in App. A.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Assumptions are clearly claimed in each theorem, and all the proofs are presented in App. H.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Implementation details are discussed in App. G.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All datasets (Apps. G.1.2 and G.2.2) are publicly available. The code will be released after the review.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: In App. G, we present the experimental details for reproducing the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Mean, STD, and max of K-fold results are presented in Sec. 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Hardware is mentioned in App. G.3.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: No ethic issue.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: No societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Original papers and datasets have been cited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: Code will be released after the review.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: No human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
- Instead of adopting complex approaches for extending MLR to Riemannian manifolds via general geometry extensions such as gyro structures and generalized SINE rules, this study generalizes to Riemannian manifolds using a simple approach based on the logarithm map.
- The authors show the several experimental results with various types of datasets.

STRENGTHS:
1. This study generalizes to Riemannian manifolds using a simple approach based on the logarithm map, avoiding complex approaches like gyro structures and generalized row of sines.
2. The geometric aspects of this paper are well-founded. Conducting geometrically valid computations using the logarithm map and parallel transport is a standard method for tangent space analysis.
3. Evaluation is conducted on various types of datasets.

WEAKNESSES:
1. I find this paper somewhat confusing to read. What does the claim "our framework only requires the explicit expression of the Riemannian logarithm" in the introduction mean? For example, parallel transport requires explicit expressions for each of types of Riemannian manifold or metrics (Table 12). Is there a contradiction with the authors' claim?

QUESTIONS:
1. Does Eq. 8 satisfy the axioms of distance?
2. The authors adopt parallel transport to determine A~_k in Eq. 11 but projecting a point in Euclidean space to the tangent space might be simpler. Parallel transport can be computationally intensive and needs to be defined according to the type of manifold. Why did the authors choose parallel transport? Also, regarding Weakness 1, if parallel transport needs to be determined individually, is there a contradiction with the authors' claim that only the logarithm map is needed?

LIMITATIONS:
- Discussed in Appendix A.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-11

comments: SUMMARY:
The authors extend multinomial logistic regression into spaces where they only require a logarithmic map. They do so in order to accomplish tasks such as classification

STRENGTHS:
The paper is well organized and written. There is a good balance of theoretical results and practical applications. It is nice to see a thorough exposition of the SPD manifold with all the commonly used metrics.

WEAKNESSES:
I don't understand why lines 29-31 seem to look down on the use of things such as tangent spaces and coordinate systems because the proposed method relies on the log map which itself effectively requires tangent spaces and coordinate systems.

The experiment section could use more thorough explanations which are found in the appendix.

QUESTIONS:
Can the authors elaborate on how their work is a distinct contribution compared to the SOTA methods which are mentioned? I understand this is more general, as is shown in Table 1 but I fail to see the entire picture.

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends the multiclass logistic regression into general Riemannian spaces, contributing to the field of Riemannian deep learning. Starting from the concept of Riemannian hyperplanes, the present work constructs the distance from Riemannian points to Riemannian hyperplanes and derives Riemannian Multinomial Logistic Regression (RMLR) on Riemannian manifolds. The RMLR framework is then showcased under 5 geometries on the SPD manifold, and SO(n). Extensive experiments on different Riemannian backbone networks, including Riemannian feedforward, Riemannian residual, and Riemannian graph neural networks, validate the effectiveness of the proposed RMLR. Especially, the results in Tab. 9 on direct classification (LogEig v.s. SPD MLR) show a clear advantage of the proposed classifiers (up to 18.34 improvement).\n\nSTRENGTHS:\n1.  The proposed RMLR framework (Thm. 3.3) can be easily implemented in different geometries. For a specific geometry, one only needs to put the involved operators into Eq. 11.\n2.  The proposed RMLR not only generalizes the Euclidean MLR, but also incorporates several previous MLRs, such as gyro SPD MLR, gyro SPSD MLR, and flat SPD MLR (Tab. 1). Besides, it further can deal with the geometry which is non-flat or agnostic to gyro structure.\n3.  A complete study of 5 families of deformed SPD metrics is presented in Tab. 2 and Fig. 1.\n4.  5 SPD MLRs and one Lie MLR are specifically implemented. The experiments on different network backbones including Riemannian feedforward, Riemannian residual, and Riemannian graph neural networks, validate the effectiveness of the proposed RMLR.\n5.  The presentation is clear, such as SPD and Lie MLRs in Thms. 4.2 and 5.2.\n\nWEAKNESSES:\n1. More details on the optimization for learning the parameters of the MLR should be presented.\n1. For the SPD manifold, there are at most three hyperparameters: $\\theta, \\alpha, \\beta$. Although these indicate the generality of the proposed framework, how to select the parameter should also be discussed from a practical view.\n\nQUESTIONS:\n1. What are the complexities (memory and time) of different metrics, theoretically and experimentally?\n2. For the SPD metrics, how can the researcher select the involved hyper-parameters in practice?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}