Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry
Authors: Ziheng Chen
Yue Song
Xiaojun Wu
Gaowen Liu
Nicu Sebe
Venue: ICLR 2025 Poster (2025)
Keywords: Global covariance pooling
SPD manifolds
Representation Learning
Riemannian Manifolds
Primary area: learning on graphs and other geometries & topologies
TL;DR: We theoretically and empirically explain how matrix function normalization (e.g., matrix square root) enables Euclidean classifiers to effectively work with Riemannian covariance features in Global Covariance Pooling by Riemannian geometry.

Abstract: Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations. GCP typically performs classification of the covariance matrices by applying matrix function normalization, such as matrix logarithm or power, followed by a Euclidean classifier. However, covariance matrices inherently lie in a Riemannian manifold, known as the Symmetric Positive Definite (SPD) manifold. The current literature does not provide a satisfactory explanation of why Euclidean classifiers can be applied directly to Riemannian features after the normalization of the matrix power. To mitigate this gap, this paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective. The underlying mechanism of matrix functions in GCP is interpreted from two perspectives: one based on tangent classifiers (Euclidean classifiers on the tangent space) and the other based on Riemannian classifiers. Via theoretical analysis and empirical validation through extensive experiments on fine-grained and large-scale visual classification datasets, we conclude that the working mechanism of the matrix functions should be attributed to the Riemannian classifiers they implicitly respect. The code is available at https://github.com/GitZH-Chen/RiemGCP.git.

Published as a conference paper at ICLR 2025
UNDERSTANDING MATRIX FUNCTION NORMALIZA- TIONS IN COVARIANCE POOLING THROUGH THE LENS OF RIEMANNIAN GEOMETRY
Ziheng Chen', Yue Song"; Xiao-Jun Wu', Gaowen Liu & Nicu Sebe University of Trento, Jiangnan University, Cisco Systems z heng chê1 comm
com, yue.scog@unitn..t
ABSTRACT
Global Covariance Pooling (GCP) has been demonstrated to improve the perfor- mance of Deep Neural Networks (DNNs) by exploiting second-order statisties of high-level representations. GCP typically performs classification of the covari- ance matrices by applying matrix function normalization, such as matrix logarithm power, followed by a Euclidean classifier. However, covariance matrices inher- ently lie in a Riemannian manifold, known as the Symmetric Positive Definite (SPD) manifold. The current literature does not provide satisfactory explanation of why Euclidean classifiers can be applied directly to Riemannian features after the normalization of the matrix power. To mitigate this gap, this paper provides a comprehensive and unified understanding of the matrix logarithm and power from a Riemannian geometry perspective. The underlying mechanism matrix func- tions in GCP is interpreted from two perspectives: one based on tangent classifiers (Fuclidean classifiers on the tangent space) and the other based on Riemannian classifiers. Via theoretical analysis and empirical validation through extensive experiments on fine-grained and large-scale visual classification datasets, we con- clude that the working mechanism of the matrix functions should be attributed to the Riemannian classifiers they implicitly respect. The code available at com Git (eatttH-Chee/Reemeeee www.ec.........ec.eee.cc..
INTRODUCTION
Global Covariance Pooling (GCP), method used replacement for Global Average Pooling (GAP) in aggregating the final activations of Deep Neural Networks (DNNs), has demonstrated exceptional performance improvements across various applications (Lin al., 2015; Ionescu et al., 2015; Li et al., 2017; Wang et al., 2017; Koniusz et al., 2017; Li et al., 2018; Wang et al., 2020a; Rahman et al., 2020; Zhu et al., 2024). The research line of existing GCP methods mainly focuses improving performance by adopting different normalization methods (lonescu et al., 2015; Li et al., 2017; Wang et al., 2020a), exploiting richer statistics (Cui et al., 2017; Wang al., 2017; Koniusz et al., 2021; Rahman et al., 2023; Chen et al., 2023a), improving covariance conditioning (Song al.. 2022d:a), and obtaining compact representations (Gao al., 2016; Yu & Salzmann, 2018; Lin et al., 2018; Wang et al., 2022a). Generally, a GCP meta-layer computes the covariance matrix of the activations as the global representation, and then performs normalization either by matrix logarithm (lonescu et al., 2015) or matrix power (Li et al., 2017; 2018; Wang et al., 2020a). Finally, the normalized matrices are fed into a Euclidean classifier. The square root has emerged as the most effective normalization scheme, outperforming the logarithm counterpart by large margin (Li et al., 2017: Wang et al., 2020a; Song et al., 2021). Although the research community has provided some theoretical support for the matrix logarithm, there no intrinsic explanations for the matrix power.
The covariance matrices naturally lie in a Riemannian manifold, known as Symmetric Positive Def- inite (SPD) manifolds (Pennec al., 2006). For matrix logarithm, it maps SPD matrices into the Euclidean space of the tangent space at the identity matrix. Euclidean classifiers can, therefore, be applied after the matrix logarithm. However, the co-domain of the matrix power is still the SPD
"Corresponding author

---

Published as a conference paper at ICLR 2025
manifold, rendering the application of Euclidean classifiers following matrix power less mathemat- ically supported. Several works have attempted to explain the matrix power. The initial motivation of matrix power in GCP (Li et al., 2017) is that the distance induced by the matrix square root ap- proximates the geodesic distance under Log-Fuclidean Metric (LEM) (Dryden et al., 2010). Never- theless, Fig. shows that the gap between these two distances is still noticeable. Furthermore, Song et al. (2021) empirically explored the benefits of approximate matrix square root over its accurate counterpart, while Wang et al. (2020b) studied the merits of GCP from an optimization perspective. However, none of them touch upon the fundamental reason why Euclidean classifiers can be directly employed in the non- -on-Euciddaan co-domain of the matrix power. There appears to be discrepancy between theoretical principles and practical applications of matrix power and logarithm.
This study aims to offer a comprehensive theoreti- cal understanding of the matrix logarithm/power in GCP and reconcile the discrepancy between theory and practice. Without loss of generality, we refer to matrix logarithm and power collectively as matrix functions. Given that the matrix logarithm is Rie- mannian logarithmic map, mapping SPD data into the tangent space, first systematically study Rie- mannian logarithmic maps on SPD manifolds under seven families of metries, resulting in three types of Riemannian logarithmic maps, the ones based on the matrix logarithm, matrix power, and Sog-Cholssky Metric (LCM), respectively. Consequently, the ma- trix logarithm in GCP establishes a tangent classi- fier (Euclidean classifiers on the tangent space) for covariance classification. Also, by applying a sim- ple affine transformation, the matrix power in GCP constructs a tangent classifier. This indicates that might unify both matrix logarithm and power as tangent classifiers. However, our experiments sug- gest that this tangent classifier explanation fails account for the efficacy of matrix power. As the tangent space distorts the intrinsic geometry of manifolds, we conjecture that tangent classifiers might not be the underlying mechanisms.
Distances under LEM and PEM
400
Distance
300
LEM PEM
200
100
0
200 400 600 800 1000
Matrix Pair Index
Figure 1: The metric induced by the ma- trix power the Power Euclidean Metric (PEM). Although PEM approaches LEM as the power approaches O, the distances under PEM (0 = 0.5) and LEM still differ largely. We visualize these two distances for 1000 random pairs of 256 256 SPD matrices. The average difference is 335.84 + 1.61. This in- dicates that PEM is not proximate to LEM under the widely used 0 = 0.5.
To delve further, we move on to a more intrinsic explanation based on the recently developed SPD Multinomial Logistics Regression (MLR) (Nguyen Yang, 2023; Chen et al., 2024a;d), which extends the Euclidean MLR (FC + softmax) into manifolds. Based on previous work (Chen et al., 2024a), we find that matrix logarithm GCP implicitly constructs the SPD MLR under LEM. Furthermore, we theoretically demonstrate that the matrix power in GCP implicitly respects the SPD MLR under PEM. These findings suggest that matrix functions in GCP can be uniformly interpreted as Riemann classifiers. Therefore, the observed performance gap between the matrix power and logarithm can be attributed to the characteristics of the underlying Riemannian metrics. To validate this postulation, conduct experiments the ImageNet- 1k (Deng et al., 2009) and three Fine- Grained Visual Categorization (FGVC) datasets, namely Caltech University Birds (Birds) ( (Welinder al., 2010), Stanford Cars (Cars) (Krause al., 2013), and FGVC Airerafts (Aircrafts) (Maji et al., 2013). The results confirm that the Riemannian classifier rather than the tangent classifier contributes to the efficacy of matrix functions in GCP. We expect our work pave the way for a deeper theoretical understanding of GCP from a Riemannian perspective and inspire more research to explore the rich SPD geometries for more effective GCP applications. We present a teaser table in Tab. 1. Due to page limits, we put the related work in App. B and all the proofs in the appendix. Besides, tables of notations and abbreviations are presented in App. for better readability.
In summary, our main contributions are two-fold. (a). First intrinsic explanation for matrix nor- malization. We explain the working mechanism of matrix functions in GCP from the perspectives of tangent and Riemannian classifiers, and finally claim that the rationality of matrix functions should be attributed to the Riemannian classifiers they implicitly respect. To the best of our knowledge, this is the first Riemannian interpretation of the matrix functions in GCP. (b). Empirical validation by extensive experiments. We validate our theoretical argument on large-scale and FGVC datasets.
2

---

Published as a conference paper at ICLR 2025
Table 1: Main results:
The working mechanisms of matrix functions in GCP are attributed to Riemannian classifiers they implicitly respect.
Matrix function
Intrinsic explanation
Used in GCP
Reference
Logarithm
EEM-ndduced Riemannian Classifier
Log-EMLL EMLR (Eq. (4))
(Chen et al., 2024a, Prop. 5.1)
Power
PEM-induced Riemannian Classifier
Pow-EMLR (Eq. (5)
Thm. 2
(A4)
Similar distances under matrix power and logarithm
Provioudly
(Ba) Rolored expertments 0.5)
(AI)
Power approcimates claseifier Lag respects classifier
Matrix
(B1) Refated by experimerts
powerllggarthm
Tangent Classifiers
(A2)
They implicilyy consiructs Riemannian classifiers (rexpecting diffrrent metrics)
Validated empirically and thoorticaall
Riemannian Classifiers
Figure 2: Illustration on the main postulations (AO to A2) and empirical validations (BO to B2) of investigation, where 0 is the power in the matrix power. Postulation AO is adopted by Li et al. (2017) and is refuted by our experiments in Fig. for the specific = 0.5. Postulation A indicated by Tab. and is refuted by our experiments Sec. 6. Postulation A2 supported by Thm. 2 and is validated by our experiments in Sec. 6.
Main theoretical results: Tab. 2 presents complete list of Riemannian logarithmic maps on SPD manifolds under different metrics. It indicates that the matrix power, with a simple affine transfor- mation, serve a Riemannian logarithmic map. Since the matrix logarithm has been widely recognized as a building component of tangent classifiers, we also expect that the matrix power function can be explained by tangent classifiers. However, the preliminary experiments presented in Tab. refute this conjecture, suggesting the existence of more fundamental mechanisms. Therefore, we delve into this mystery Sec. 5 by leveraging the recently developed Riemannian classifiers. Thm. indicates that matrix power in GCP implicitly establishes Riemannian classifier for covari- ance matrix classification. Similar results also hold for the matrix logarithm (Chen al., 2024a). This implies that the matrix logarithm and power can be unifiedly interpreted as essential com- ponents of Riemannian classifiers. Tab. summarizes our theoretical findings. Sec. further validate our theoretical explanations by extensive experiments. The reasoning behind our analysis illustrated Fig.
THE GEOMETRY OF SPD MANIFOLDS
Let S be the set of SPD matrices. As shown by Arsigny et al. (2005), ST is open submanifold of the Euclidean space S" of symmetric matrices. There are five popular Riemannian metrics SPD manifolds: Affine-Invarian Metric (AIM) (Pennec et al., 2006), Log-Euclidean Metric (LEM) (Arsigny et al., 2005), Power-Fauclidean Metric (PEM) (Dryden et al., 2010), Log- Metric (BWM) (Bhatia et al., 2019).
Cholesky Metric (LCM) (Lin, 2019), and Note that when power equals 1, PEM reduces to the Euclidean Metric (EM). All of the above five standard metrics have been generalized into parameterized families of metrics.
Thanwerdas & Pennec (2023) generalized AIM, LEM, and EM into two wwo-maramtter metrics by the ((n)-invariam Euclidean inner product on S":
(1) where a > and 3 - o/1, V, W S", and (+-)) is the standard matrix inner product. These generalized metrics are denoted as (ax, AAMM (a, BLLM,,,,, and (a, BEM, respectively. Besides, Thanwerdas & Pennec (2022) defined the power-deformed metric ğ of metric g on S as
(V, W) (a.3) (a. = (V, W + 3 (t(((t(((())))) tr(),
1
gr (V, W) where Pow)(P) pd denotes matrix power, (Pows).,p is the differential map of Pow, at P. and TpST is the tangent space at P. Following Eq. (2), (o, A)MMI.M (a, 8)-MEM, (a, 8)M LCM,
Ø2 Dr ((Powo),, (Powo),, VP S V,W W. TyST...
3

---

Published as a conference paper at ICLR 2025
and BWM are generalized into power-deformed families of metrics, denoted (0, a, AAM,, (8, a, 8)-LM,, (0, a, -M) 9-LCM, and 20-BWM, respectively (Thanwerdas & Pennec, 2022; Chen et al., 2024d). Chen et al. (2024d) further shows that (0, a, 8)-MM equals (a, -3M Besides, as shown by Thanwerdas & Pennec (2022); Chen et al. (2024d), 0 serves as a deforma- tion from LEM-like metric. For instance, (0,a, az (3-AIMM becomes (a, ()-MIM when 0 1 and approaching (a, -)MLEM with 0 + O.
On the other hand, PEM was generalized into Mixed Power Euclidean Metric (MPEM) (Thanwerdas & Pennec, 2019) by two power factors 0₁ and 02, denoted as (0), 02 --E. When 0,-02. MPEM reduced to PEM. Han al. (2023) extended BWM into Generalized Bures- Wasserstein Metric (GBWM) by an SPD parameter M, denoted as M-BWM. When M-I, GBWM is reduced to BWM. We further generalize GBWM into (20, -BW by power deformation Eq. (2).
In total, three parameters (0, (3) are involved in the metrics on the SPD manifold. The power deformation 0 characterizes deformation (Chen et al., 2024d; Thanwerdas & Pennec, 2022), while powerful property in modeling covariance (Thanwerdas & Pennec, 2023). The above metrics have shown success in various applications, due to their closed- form expressions for Riemannian operators. such as the Riemannian logarithmic and exponential maps. We summarize the involved Riemannian operators in App. D.2.
(a, 3) characterizes the O
3 GLOBAL COVARIANCE POOLING REVISITED
GCP captures the second-order statistics of the features in the last layer of the deep network. The standard GCP procedure comprises calculating the covariance matrix, normalization with matrix function, vectorization, dimensionality reduction by an FC layer, and ultimately applying a Eu- clidean classifier. The sequence of these operations can be represented as follows:
X
Cov
fM
fver,
JPC
fEC
J.
(3)
where fM, fvee. fFC and fEC denote the matrix function, vectorization, FC layer, and Euclidean classifier, respectively. Typical candidates of matrix functions are matrix power and logarithm. As softmax is the most widely used classifier, fEC always denotes the softmax this paper. However, our discussions can also apply to other classifiers used in GCP, such as SVM (Li et al., 2017; Wang et al., 2020a), as other classifiers receive the FC features as their inputs.
FC + softmax is known as Euclidean Multinomial Logistics Regression (EMLR). When the matrix function is the matrix power, we call the process fEC o fFC o fvee a fM as the Pow-EMLR, while the counterpart of matrix logarithm is referred to Log-EMLR. Especially, setting power 1/2
in GCP normally reaches the optimal performance (Li et al., 2017, Fig. 3). The Pow-EMLR and Log-EMLR can be formally expressed as
Log-EN Dog-EMLR: softmax (F (fvee (mlog( (aalog(S):: AA
(4)
Pow-EMLR: softmax (J (fvee (5t) A))
(5)
where F(::A,b) denotes the FC layer with the transformation matrix A and biasing vector b.
4 MATRIX FUNCTIONS AND TANGENT CLASSIFIERS
The matrix logarithm is the Riemannian logarithmic map at the identity matrix J, mapping SPD matrices into the tangent space T,S ~ S". As tangent spaces are Euclidean spaces, it is natural to exploit FC and Euclidean classifiers on S directly. We refer to the Euclidean classifiers over the tangent space at the identity matrix, Ty tangent classifiers. This section systematically studies all Riemannian logarithmic maps between ST and Ty under seven families of metrics.
4.1 RIEMANNIAN LOGARITIMMS UNDER SEVEN DEFORMED METRICS
The matrix logarithm is generally characterized as the Riemannian logarithm Log, at under the standard LEM and AIM. Inspired by this, systematically investigate Riemannian logarithms on SPD manifolds. Let P denote SPD matrix and f. represent the Cholesky factor of pe Tab.
4

---

Published as a conference paper at ICLR 2025
Table 2: Log, under seven families of metrics. Op 0. for (01, On for (0, ax, 3)- and 20-BWM, and (20, @20 (P))-BWM
Metric
Log, P
Metric
Logz P
8)..MMMMMM (0. a. AMM
(0, a, 3)-EM
mlog(P) P)
(0,, 82 E-M
(
0-LCM
[ + 1 + (log((((
20-BWM
8o
(20, p --BWM
presents the Riemannian logarithms at under all seven metrics, where [.] the strictly lower triangular part of a square matrix, (D-) is diagonal matrix, and Dlog(-) is the diagonal logarithm. We leave technical details in App. E.
Remark 1. Let us elaborate further on the parameter of GBWM in Tab. 2. Given an SPD point P E S4++ P-BWM coincides with the standard AIM in the neighborhood of P (Han et al., 2021). This local property could be beneficial (Han et al., 2023). Similarly, (20, PBWM locally (20, 0)-AIM, the deformed metric of the standard AIM. Please refer to App. For technical details.
Tab. implies that there are three types of Logy:
Matrix-logaritm--basett mlog(P),
Matrix-power-baset. / (P" I):
1
(7)
LCM-based: - [ + L - Dlog((D(L)).
(8)
We denote the tangent MLR induced by Eq. (6), i.e. (6) + vectorization + FC + softmax, as Log- TMLR, while the counterparts of Eq. (7) and Eq. (8) is referred as Pow-TMLR and Cho-TMLR, respectively. Obviously, Log-TMLR is the exact Log-EMLR (Eq. (4)) used in GCP.
Table 3: Results of GCP on the ImageNet-Ie 1k and Cars datasets with NOw-TMLR or Pow-EMLR under the architecture of R8sNet-18.
ImageNet-Ik
Cars
Method
Top-1 Acc (%)
Top-5 Acc (%)
Top-1 Acc (%)
Top-5 Acc (%)
Pow-TMLR
71.62 73
89.73
51.14 80.43
74,29
Pow-EMLR
90.91
94.15
4.2 Pow-TMLR VERSUS Pow-EMLE
Pow-EMLR applies Euclidean MLR directly on the non-Euclidean SPD manifold, while Pow- TMLR applies Euclidean MLR on the Euclidean space of T +. In this sense, Pow-TMLR should be more theoretically advantageous than Pow-EMLR. Moreover, the difference between Pow-EMLR and Pow-TMLR seems to be minor. Pow-EMLR differs from Pow-TMLR only in simple affine transformation fx(X) = 1 (X T). Note that the composition of affine transformations remains affine, and the FC layer also an affine transformation. Therefore, Pow-EMLR might be viewed as the approximation of Pow-TMLR. Based on this discussion, we hypothesize that the tangent classifier serves the underlying mechanism of matrix functions in GCP. If this hypothesis holds, Now-EMLR should perform worse or at least similarly to Pow-TMLR.
To validate this postulation, we conduct experiments on the ImageNet- Ik (Deng et al., 2009) and Stanford Cars (Cars) (Krause et al., 2013) datasets. We use the architecture of ResNet-18 18 and ResNet-50 (He et al., 2016) on the ImageNet and Cars datasets, respectively. Following the clas- sic iSQRT-COV (Li et al., 2018), we set power=1/2 and use Newton-Schulz iteration to calculate the matrix square root. Note that Pow-EMLR under Newton-Schulz iteration is exactly the origi- nal implementation of iSQRT-COV. As shown in Tab. 3, opposite to our hypothesis, Pow-TMLR inferior to Pow-EMLR for classifying covariance matrices in GCP. Similar trends are also ob- served in additional experiments conducted FGVC datasets, as will be presented in Sec. 6. These findings suggest that instead of tangent classifiers, there should exist other more fundamental mechanisms for underpinning matrix functions in GCP.

---

Published as a conference paper at ICLR 2025
5 MATRIX FUNCTIONS AND RIEMANNIAN CLASSIFIERS
Recently, Riemannian classifiers on the SPD manifold, which can more faithfully respect the innate geometry, have exhibited more promising performance than tangent classifiers (Nguyen Yang, 2023; Chen et al., 2024a;d). This section will demonstrate that matrix functions in GCP implicitly respect Riemannian classifiers, which offers a unified theoretical explanation of the working mech- anism of matrix functions. We start with reviewing the Riemannian SPD classifiers and then present our theoretical analysis detail.
5.1 SPD MULTINOMIAL LOGISTICS REGRESSION REVISITED
Inspired by Lebanon Lafferty (2004); Ganea et al. (2018), some recent works (Nguyen & Yang, 2023; Chen et al., 2024a;d) extended the Fuclidean MLR into SPD manifolds. We first revisit the reformulation of the Euclidean MLR, and then move on to the SPD MLRs introduced by Chen et al. (2024d), especially the ones induced by (0, a, 3M-EM and (a, 3)-LEM.
The Euclidean MLR calculates the probability of each class by
Vk (I,...,C), ply x) x exp ((xx, 2) bk);
where I IR" an input vector, C the number of classes, bk R. and as E (0). Eq. (9) can be further rewritten as
pvu = k a)) exp ((ax, pk))
(10)
where P& satisfies (ak.Pk) = by- As shown in the previous literature (Lebanon & Lafferty, 2004; Ganea al., 2018), Eq. (10) can be further reformulated by the margin distance to the hyperplane:
ply = k x) x exp(sign( (ak, px))] [d(x, Han.pp)); where Ps I satisfying (ag, byg and the hyperplane Hazip is defined as:
(11)
H Hag.n {2 R (ax, pk) 0}.
(12)
Chen al. (2024d) generalized Eqs. (11) and (12) into general manifolds and proposed the SPD MLRs under five families of metrics. The SPD MLRs under (a, 3)-LEM and (0, a, 3)- are
(a, -abbbadaas p(y = k|S) exp
(log(S) -log( (B)) A Ac)(o.m)
(13)
[ 0 (50 - Ph Aa) (a.))
(0, a. -MMbased p(y = k|S) exp
(14)
where > 0, > -n/n, and S is an input SPD feature. Here, Ps E S and A Ty S are parameters for each class k. In Eqs. (13) and (14), the formula within exp(.) can be viewed as the counterpart of the Euclidean FC layer in SPD manifolds, extracting features to calculate softmax probabilities.
5.2 MATRIX FUNCTIONS AS SPD MULTINOMIAL LOGISTICS REGRESSION
Under the standard LEM ((1, ()-LEM) and PEM ((,, -MM) Eqs. (13) and (14) become
LEM-based: exp [log(S) - log((o))))))))))). Ab).
(15)
basedd exp 15 Pf, An)
(16)
Eqs. (15) and (16) appear to be far away from the Log-/Pow-EMLL in GCP, as the SPD parameters {Pi...)} requires Riemannian optimization. However, Chen et al. (2024a, Prop. 5.1) show that under the LEM-based Riemannian Stochastic Gradient Descent (RSGD) for each Ps and Euclidean SGD for each Ag. Eq. (15) is equivalent to Euclidean MLR optimized by the Euclidean SGD in the co-domiin of the matrix logarithm. Similar to LEM, we have the following proposition w.r.t
PEM.

---

Published as a conference paper at ICLR 2025
Table 4: Intrinsic explanations of some classifiers for GCP. For Cho-TMLR, = Chol(S").
For Pow-TMLA tn = ei+02 for (01. -2M. 8 = 0 for (0, a, 3)- 80 = 20 for 20-BWM and (20, (22 (S))-BWM.. Here, fa(+) denotes the softmax, F()) denotes the FC layer, and V
2
[L] + LL]T + DIog(D(L)) with S = LL the Cholesky decomposition.
Log-EMER &
Pow-EMLR
SestePow-FML.
Pow-TMLR R
Cho-TMER
Expression
(ee (Fee (11))(())))
1 (Joe (5*)))
(9 > D)
5. (155)) 10 0)
I 1 (Io (+t - o))) x (x (sm (v))
Explanation
SPD MLR
SPD MLR
SPD MLR
Tangent Classifeer
Tangent Classiffer
LEM
(0,
(0.
(0 op,o. SJ-EM. (0, (.)-EM. 29.BWM, (28, (.)()
e.LCM
Used in GCP
V(Bq. (1)
(0 0.5 Eq. (5))
V
x
Reference
(Chen et 2024a, Prop. 5.1)
Thm.
Thm.
Tab.
Tab
Theorem 2. (1] Under PEM with > O, optimizing each SPD parameter Ps in Eq. (16) by PEM- based RSGD) and Euclidean parameter Ag by Euclidean SGD, the PEM-based SPD) MLR is equiv- alent to a Euclidean MLR illustrated in Eq. (10) in the co-domain of Φol.) S4+ + S4 defined as
1 59 00,00, E (0) 0
(17)
We define ScalePow-EMI.R as softmax (F (Jvee (7,59): : A, Then, ScalePow-EMI.R respects the SPD MLR under the standard PEM. The only difference between ScalePow-EMMIR and Pow- EMLR (Eg. (5)) is the scalar product before vectorization, which is expected to have minor effeets on DNNs. Obviously, we have
F ((Fr0 (H55): :A.b) F FFvec (5"): A.
(18)
where A= A.A. Therefore, from a forward perspective, ScalePow-EMI.I is equivalent to the orig- inal Pow-EMLR. Besides, by scaled initialization and learning rate of A, ScalePow-EMLL. could be completely equivalent to Pow-EMLR during network training. Note that this analysis cannot be transferred into Pow-TMLE Please refer to App. G for more details.
Therefore, the PoW-EMLR in GCP is implicitly an SPD MLR induced by (0, 1, 0) -EM. For the widely used matrix square root normalization, it respects the SPD MLR induced by (122, 1, E.M. We summarize all the findings in Tab. 4. Besides, Thm. 2 can be easily extended into the case of 0 0. In this case, our work can also offer theoretical insights for the inverse of covariance (0- = 1) proposed by Rahman et al. (2023). More details are presented in App. J.
5.3 THEORETICAL INSIGHTS ON THE MATRIX POWER AND LOGARITHM
Previous studies on GCP (Li al., 2017; Wang al., 2020a; Song et al., 2021) have empirically demonstrated a clear advantage of the matrix power (particularly matrix square root) over matrix logarithm. This subsection offers novel theoretical insights to disentangle the different performance between the matrix logarithm and power in GCP.
As shown by Tab. 4, both matrix logarithm and matrix power implicitly build SPD MLRs. However, the Riemannian metrics they respect are different. Matrix power respects (0, 1, O)EM while matrix logarithm respects LEM. Both (0. 1. -M- and LEM share -ivarannee (Chen al., 2024d), a powerful property in characterizing covariance matrices. Besides, (0, 1, -M is a deformed metric of LEM, interpolating between the standard EM (0 1) and LEM (0 - 0) (Thanwerdas & Pennec, 2022). The standard EM might suffer from swelling effect for characterizing SPD matrices (Arsigny et al., 2005), while LEM might over-stretch the eigenvalues of SPD matrices due to the computation of matrix logarithm (Song et al., 2021). Consequently, (0, 1, 0)-EM represents balanced alternatives between the standard LEM and EM. In addition, as shown by Chen et al. (2024d, Tab. 4), (0, 0),0 could perform better than LEM regarding SPD MLR. Therefore, the empirical advantages of matrix power over matrix logarithm in the GCP could be attributed to the characteristics of the underlying Riemannian metrics.
7

---

Published as a conference paper at ICLR 2025
Table 5: Results of iSQRT-COV on four datasets with different covariance matrix classifiers. The backbone network on ImageNet is ResNet-18 18, while the one on the other three FGVC datasets is ResNet-50. Power is set be 1/2 for Pow-TMLR, ScalePow-EMee --ooo--E-RR--- and Pow-EMLR.
Classifier
Top-1 Acc (8)
Nes-
Top-5
Top- (8)
Aircratts
91.81 Top-5
Top-I
Birds
Top-5 (f))
Top-I (%)
Top-5
Che-TMLR
PoW-IMLK
71.62
77.80
Scal:Pow-FMM..
72.45
Pow-EMLLR
73
90.91
72.07
89.83
63.29
84.66
80.43
94.15
Tradd Airraai delaned
Traed azz Corrs tteiaee
Top-S the deetuned
90
Accunaty
Pon-CULL Pou-TTal
CMIL1
Pow-Chinn
00
Figure 3: The validation top-5 accuracy on the three FGVC datasets for iSQRT-COV with different classifiers using the ResNet-50 backbone.
6 EXPERIMENTS
In this section, we validate the following hypothesis based on our previous theoretical analysis.
(A1) As Pow-EMLR approximates the tangent classifier Pow-TMLR, the working mechanism of Pow-EMIR is attributed to the tangent classifier;
(A2) As both Pow-EMLR and Log-EMLR in GCP are equivalent to Riemannian classifiers, the mechanism of matrix normalization should be attributed to Riemannian classifiers.
We implement different classifiers for covariance matrix classification, including the original Pow- EMLR, the tangent classifiers Pow-TMLR and Cho-TMLR,, and the intrinsic ScalePow-EMMI.. We use the Caltech University Birds (Birds) (Welinder et al., 2010), FGVC Aircrafts (Aircrafts) (Maji al., 2013), Stanford Cars (Cars) (Krause et al., 2013), and ImageNet-Ik Ik (Deng et al., 2009) datasets. As the matrix square root is the most effective matrix function in GCP, we set power 1/2. In all experiments, we train the network from scratch. More implementation details are App. H.
6.1 MAIN RESULTS
Notably, although ScalePow-EMLR is equivalent to Pow-EMLR under scaled settings, we imple- ment them under the same network settings for complete comparison. The results four datasets are shown in Tab. 5. Our main empirical observations are as follows:
(1). POW-EMLR: EEW-EEERR---OWLLLL.. POW-TMLR. Pow-EMLR generally outperforms Pow-TMLR, especially on Cars and Birds datasets. Recalling in Tab. 4, the expression of Pow-EMLR differs from -Ww-TMLR only in an affine transformation. However, across all four datasets, Pow-EMLR consistently sur- passes Pow-TMLR. On the Birds and Cars datasets, Pow-EMLR outperforms Pow-TMLR by large margin. For example, on the Birds dataset, the top-5 accuracy of Pow-EMLR and Pow-TMLR is 84.66% and 77.80%, respectively, whereas, on the Cars dataset, it is 94.15% and 74.29%.
Pow-EMIR shows comparable performance to ScalePow- EMLR. Recalling in Tab. 4, the only difference between Pow-EMLR and ScalcPow-EMLRR is scalar product. Moreover, as discussed in Sec. 5.2 this minor difference can be further solved by scaled initialization of the FC layer. Although we the same initialization for fair comparison, Pow-EMLR and ScalePow- -E-LR show similar performance.
(2). Pow-EMM.R=-SSaleooxEENN
(3). Pow-EMLR Cho-TMLR. While Rho-TMLR demonstrates the best performance on the Air- crafts datasets, it exhibits the worst performance on the other two FGVC datasets. On the Cars and Birds datasets, Pow-EMIL surpasses Cho-TMLR by a large margin. The unstable performance of
8

---

Published as a conference paper at ICLR 2025
Cho-TMLE might be attributed the diagonal logarithm, which might overly stretch the diagonal elements of the Cholesky factor.
Based on the above empirical findings, we can reach the following conclusion. (A1) is refuted by
(1). The inferior performance of Pow-TMLR against Pow-EMLR in (1) indicates that Pow-EMLR
not be simply viewed equivalent the tangent classifier Pow-TMLR. (A2) is validated by (2).
(2) validates our theoretical postulation that the effectiveness of matrix power should be attributed to the Riemannian classifier it implicitly constructs.
Other findings. In the first and last observations, tangent classifiers are less effective than the Rie- mannian classifier. Tangent classifiers can distort the innate geometry of the manifold, as the tangent space is only local linear approximation of the manifold. In contrast, the Riemannian classifier can faithfully respect the geometry of the manifold. Besides, although Log-EMLR coincides with both tangent and Riemannian classifiers, the real underlying mechanism of matrix logarithm should also be attributed the Riemannian classifier instead of the tangent classifier.
Table 6: Ablations of Pow-EMLR, ScalePow-EMLR, and Pow-TMLR under different settings. (a) Results of different powers under the ResNet-50.
Aircratts
Curs
(b) Results under the AlexNet.
Classifier
Top-I Acc (96)
Top-5 Ace (1)
Top-1 Acc (9)
Top-5 Ace (98)
ScalePan- EMLR--0,2
Pou-TML 2--022
65.41
72.76
90.31
$6.71
61.78
41.47
$4.04
66.66
Dataset
Result
Pow-TMLR
Pow-EMLR
Pow BML& 0:25
71.47
90,01
62,88
$4.14
Top-1 Ace (%)
38.01
68,02
Pow- SOw-TMLK-41
67.9
$8.75
55,01
77.95
Aiterafts
Top-5 (8)
87.79
SealePow-FMM FM R--0.5
74.39
91.12
62.42
84.82
Top-1 (oc
28.57
59.13
EMLR 0.5
74.17
91.21
62.83
$4.85
Cans
Top Acc (%)
59.51
82,04
Pow- TMLR 0.7
65.92
30,68
74.12
SealeRo-HMMI R--077
74.26
64,22
EMLRR0077
24.17
61.41
6.2 TRAINING DYNAMICS AND ABLATIONS
Training dynamics. Fig. presents the top-5 validation accuracy curves on three FGVC datasets. Pow-EMLR exhibits comparable performance to ScalePow-EMMI throughout the training. More- over, Pow-EMIR consistently outperforms Pow-TML.R, particularly on the Cars and Birds datasets. This again suggests that the effectiveness of Pow-EMLR should be attributed to the Riemannian MLR rather than the tangent classifier. Furthermore, we note that the decreasing learning rate plays erucial role in Cho-TMLR. On the Airerafts dataset, before the 50th epoch, Cho-TMLR exhibits the worst performance among all classifiers. However, after the 50th epoch, when the learning rate reduces, Cho-TMLR surpasses all the other classifiers. Nonetheless, on the remaining two datasets, Cho-TMLR remains inferior throughout the training. This discrepancy may be attributed to the log- arithm operation in Cho-TMLR. Recalling Eq. (8), there is diagonal logarithm for the Cholesky factor. Similar to the matrix logarithm, Eq. (8) will also over-stretch the diagonal elements of the Cholesky factor, compromising the overall performance of Cho-TMLR.
Ablations. To further validate our postulation, we compare PoW-EMLR, SaclePow-EMM EMMR,, and Pow-TMLR with different powers under the ResNet-50 architecture, i.e., 0 0.25, 0.5, 0.7. We also compare Pow-EMLR against Pow-TMLR under the AlexNet architecture. The ablations are conducted on the Aircrafts and Car datasets. The results discussed below confirm again our findings that the mechanism of matrix functions in GCP should be attributed to Riemannian classifiers.
Impact of matrix power: Following Song et al. (2021), we use accurate SVD to calculate the matrix power and Padé approximant for backpropagation. The results are reported in Tab. 6a. Since use SVD for the matrix power here the results in Tab. 6a under = 0.5 are slightly different from Tab. 5. Nevertheless, Pow-EMLE consistently shows similar performance to ScalePow-EMLL. and outperforms Pow- Tow-TMLR under different powers.
Impact of architectures. We also the vanilla AlexNet (Krizhevsky et al., 2012) alternative backbone. Tab. 6b presents the comparison results under the AlexNet architecture. Consistent with our previous observation, Pow-EML. R still outperforms Pow-TMLR.

---

Published as a conference paper at ICLR 2025
7 CONCLUSIONS
This paper presents a unified understanding of the role of matrix functions GCP, including matrix power and logarithm. Our study reveals that matrix functions implicitly construct Riemannian clas- sifiers for classifying covariance matrices, thus justifying the application of the Euclidean classifier after matrix power. We validate our findings through experiments conducted on three FGVC and the large-scale ImageNet datasets. To the best of our knowledge, our work is the first to explain the theoretical mechanism behind matrix functions from the perspective of Riemannian geometry. Therefore, our work opens a novel possibility for designing GCP classifiers from Riemannian per- spective. As future avenue, we will design effective GCP classifiers based on other Riemannian MLRs (Nguyen & Yang, 2023; Chen et al., 2024d).
ACKNOWLEDGMENTS
This work was partly supported by the MUR PNRR project FAIR (PE00000013) funded by the NexIGenerationEU, the EU Horizon project ELIAS (No. 101 120237), and a donation from Cisco. The authors also gratefully acknowledge the financial support from the China Scholarship Council (CSC), as well as the CINECA award under the ISCRA initiative for the availability of partial HPC resources support.
REFERENCES
Dinesh Acharya, Zhiwu Huang, Danda Pani Paudel, and Lue Van Gool. Covariance pooling for facial expression recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 367-374, 2018. URL https: http://////ai..r/11.11.11.
CVPRW. 2018 00077.
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and simple computations on tensors with log- ang-Eulldeann metrics. PhD thesis, INRTA, 2005. URL https: /oo .org/ 10. 10.1007 111100771116465_111
Rajendra Bhatia. Positive Definite Matrices. Princeton University Press, 2009. URL https: //doi. www.org/11.. 5315/97811 00155//179002222 400827787.
Rajendra Bhatia, Tanvi Jain, and Yongdo Lim. On the Wassersennn distance between positive definite matrices. Expositiones Mathematicae, 37(2):165-191, 2019. URL https: /dooi
org/10. 1 exwath.....h.....mmmmmm......... 2018 002.
Silvere Bonnabel. Stochastic gradient descent on Riemannian manifolds. JEEE Transacttios Au- tomatic Control, 58(9):2217-2229, 2013. URL /tttp:://od..oog/10.11///// 2013.
2254619.
Kaixuan Chen, Jie Song, Shunyu Liu, Na Yu, Zunlei Feng, Gengshi Han, and Mingli Song. Dis- tribution knowledge embedding for graph pooling, JEEE Transactions on Knowledge and Data Engineering, 35(8))7898-7999) 7908, 2023a. URL
hwtp://ggg/ab///////.o /aabs/2199... 14333.
Ziheng Chen, Tianyang Xu, Xiao-Jun Wu, Rui Wang, Zhiwu Huang, and Josef Kittler. Riemannian local mechanism for SPD neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 7104-7112, 2023b. URL https: httdddiioooogo 10. 1609 awai.v3iiv.. v3716. 25867.
Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, Xiao-Jun Wu, and Nicu Sebe. Riemannian multinomial logistics regression for SPD neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17086-17096, 2024a. URL
thecvf.ee... com/ Chen_Ri www.nmiiaannnnnn.mmaa ittiiiannmaliiimmimiaiaaamm alg.amiaallllllllallaaloo wtw.ieeeeeeesssooorrrrrrre Neural Networks_cVPR_2022
_wwee2022_____________________er paper. html.
Ziheng Chen, Yue Song, Yunmei Liu, and Nicu Sebe. A Lie group approach to Riemannian normal- ization for SPD neural networks. In The Twelfth International Conference on Learning Represen- fations, 2024b. URL https: http::s://///eevieeee/ net / forum? d=okYdj8Yssu.
10

---

Published as a conference paper at ICLR 2025
Ziheng Chen, Yue Song, Xiao-Jun Wu, and Nicu Sebe. Product geometries on Cholesky manifolds with applications to SPD manifolds. arXiv preprint arXiv:2407.02607, 2024c. URL https:
02607.
Ziheng Chen, Yue Song, Xiaojun, and Nicu Sebe. RMLR: Extending multinomial logistic regression into general geometries. In Advances in Neural Information Processing Systems, 2024d. URL
https: / /ttpvvaaxvvvvvvv////v//.......v. org/ abs 22094499.. 19433.
Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, and Nicu Sebe. Adaptive log-Euclidean metrics for SPD matrix learning. IEEE Transactions Image Processing, 2024e.
URL httph::p///// //tp:://dooii http//oi.ooggoo/////o 1109/ TIP 2024 3451930.
Ziheng Chen, Yue Song, Xiaojun Wu, and Nicu Sebe. Gyrogroup batch normalization. In The Thirteenth International Conference on Learning Reprexentations, 2025. URL https:
// //oppneeeieww...mmooooeoeeoooeee fortt/oormmidddeee
JJW.
Yin Cui, Feng Zhou, Jiang Wang. Xiao Liu, Yuanqing Lin, and Serge Belongie. Kernel pooling for convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921-2930, 2017. https:/ /ttp//oo..ooo././////o////./.. htt./////..../////..11 1109/CVPR. 2017. 325.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255. IEEE, 2009, URL https: doi. /0g/ 1109/ CVPR. 2009. 520 6848.
Ian L Dryden, Xavier Pennec, and Jean-Mare Peyrat. Power Euclidean metrics for covariance ma- trices with application diffusion tensor imaging. arXiv preprint arXiv:1009.3045, 2010. URL
https: arxiv. org/ abaaaaaso000 10t.9.10..0000
Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. Advances in Neural Information Processing Systems, 31, 2018. URL https: / www.aam.....mmmmmmmmmm..mm...... ort/oor//o///...
0355/33373455 3327440.
Yang Gao, Oscar Beijhom, Ning Zhang, and Trevor Darrell. Compact bilinear pooling. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 317-326, 2016. URL https:/ /oii org/
1111//1PR..... 2016 6. 41.
Andi Han, Bamdev Mishra, Pratik Kumar Jawanpuria, and Junbin Gao. On Riemannian optimiza- tion over positive definite matrices with the Bures- Wasserstein geometry. Advances in Neural Information Processing Systems, 34:8940-8953 2021. URL attps://openreviews
Andi Han, Bamdev Mishra, Pratik Jawanpuria, and Junbin Gao. Learning with symmetric positive definite matrices via generalized Bures-Wasserse geometry. In International Conference on Geometric Science of Information, pp. 405-415. Springer, 2023. URL https: htt///iooorttoooooooo//
10. -31-38271000400 40.
031 38271
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings the 770-778, 2016. URL https: htt:/diioo// org/loggggo 10 1109 CVPR. 2016.900..00
the IEEE Conference on Computer Vision and Pattern Recognition, pp-
Zhiwu Huang, Chengde Wan, Thomas Probst, and Lue Van Gool. Deep learning Lie groups for skeleton-bard based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6099-6108, 2017. URL httph:t/p///cc http:///axvvvvv.o abs/1 612..
05877.
Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix backpropagation for deep net- works with structured layers. Proceedings of the IEFE International Conference on Computer Vision, pp. 2965-2973, 2015. URL https: / /ooi..... org/ / 0. 109 TCCV 2015 3.39.
Shu Kong and Charless Fowlkes. Low-rank bilinear pooling for fine-grained classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 365-374, 2017. URL httphttts///cc //oii wtweiiorg/11001 httpg//....19///P/..... 2017..2........ 743..

---

Published as a conference paper at ICLR 2025
Piotr Koniusz, Fei Yan, Philippe-Hoor Henri Gosselin, and Krystian Mikolajezyk. Higher-order occur- rence pooling for bags-of words::: Visual concept detection. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 39(2)313-326, 2017. URL httpp:::/c....o///////o//
TPAMT 2016 6. 2545667.
Piotr Koniusz, Lei Wang, and Anoop Cherian. Tensor representations for action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(2):648-6633 2021. URL https:
//doi. orgb:.oorg/00 110.1119994 1109/TPAMT 2021 3107166 6C.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3D object representations for fine- grained categorization. In Proceedings of the IEEE International Conference on Computer Ciston 2013..77......
Workshops, pp. 554-561, 2013. URL https:
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances in Neural Information Processing Systems, 25, 2012. URL https: / /oii
Guy Lebanon and John Lafferty. Hyperplane margin classifiers on the multinomial manifold. In Proceedings of the twenty-first international conference on Machine learning, pp. 66, 2004. URL.
https: /oii. owg/ooggooooooo..oooooooo....... http:/...11111111111 101.533333.113333.
Peihua Li, Jiangtao Xie, Qilong Wang, and Wangmeng Zuo. Is second-order information helpful for the large-scale visual recognition? In Proceedings the IEEE International Conference on Computer Vision, pp. 2070-2078, 2017. URL https: http://tiooo/o/////////oo/////// .1009 ICCV 2017.228.
Peihua Li, Jiangtao Xie, Qilong Wang, and Zilin Gao. Towards faster training of global covariance pooling networks by iterative matrix square root normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 947-955, 2018. URL https:
iiddi.orr/// 10. 111.11.1/.......... 2018 00105.
Tsung-Y Yu Lin and Subhransu Maji. Improved bilinear pooling with CNNs. arXiv preprint arXiv: 1707.07777 2017. URL https: wtw.iiivvvv..//.io wwwp:vii.orrroooooooooooooooo abs/ 17777700000 06772.
Tsung- Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. Bilinear CNN models for fine-grained visual recognition. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1449 1457, 2015. URL https:
1109 ICCV 2015. 170.
Tsung- Yu Lin, Subhransu Maji, and Piotr Koniusz. Second-order democratic aggregation. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pp. 620-636, 2018. URL https:
3 030- 01219 9_38.
Zhenhua Lin. Riemannian geometry of symmetric positive definite matrices via Cholesky decom- position. SIAM Journal on Matrix Analysis and Applications, 40(4):1353-1370, 2019, URL
https:/ //t////.i// orw.lorrgll.. 1137 //////MM1222 22104.....
Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In International Conference Learning Representations, 2019. URL httttt/s////eeee httppppp::///eeeeeieee..
net fortm? sid:iddeesessFee
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012-10022, 2021. URL
https:/ ////o wwt.o//////..///////.....
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151,, 2013. URL https://aara //axiv..
org/abs/ 1106633066000 5151.
Xuan Son Nguyen. Geomnet: neural network based on Riemannian geometries of SPD matrix space and Cholesky space for 3D skeleton-based interaction recognition. In Proceedings of the IEEE International Conference on Computer Vision, pp. 93379-13388 2021. URL https:
//doi. aadoi.org//00. //debl.org/111.111///////939
www.1199TTTC9992....m 202118222222222222 01333.
12

---

Published as a conference paper at ICLR 2025
Xuan Son Nguyen.
The Gyro-structure of some matrix manifolds. In Advances in Neural Information Processing Systems, volume 35, pp. 86618-26630 2022a. URL https: pttps::/prrceeee http::ss/pppcceeingg..... neurips cw/paperppppppcpcppppccccece...e https//appppeeeeeeeeeeeeeaapppee a 8
Xuan Son Nguyen. A Gyrovector space approach for symmetrie positive semi-definite matrix learn- ing. In Proceedings of the European Conference on Computer Vision, pp. 52-68, 2022b. URL
https://dof
orggorgg/oo 1 10. 1007 978 378-3-031-19812-1
Xuan Son Nguyen and Shuo Yang. Building neural networks on matrix manifolds: Gyrovector space approach. arXiv preprint arXiv::305.0455500 2023, http::/::///l org/doi/
10.5555/3618445 3619491.111
Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian framework for tensor computiin International Journal of Computer Vision, 66(1):41-66 2006. URL https:///dolt org/10.
1007/ 005-0005322200 3222-2.
Saimunur Rahman, Lei Wang, Changming Sun, and Luping Zhou. Redro: Efficiently learning large-sized SPD visual representation. In European Conference on Computer Vision, pp. 1-17. Springer, 2020. URL https::///// :ttp::://doii //oii /tt///......11100///9991111/
1007/9707777 -00-977---000000000000001100001-
6_1.
Saimunur Rahman, Piotr Koniusz, Lei Wang, Luping Zhou, Peyman Moghadam, and Changming Sun. Learning partial correlation based deep visual representation for image classification. Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6231 6240, 2023. URL https: /twii//oooii///// org/10 1109/ /Ct////VP2222/// 2023. 00603.
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. arXiv preprint arXiv:2006. 082006008///4 10, 2020. URL httpt: htttpps///..i...tttttttttt wtt./org//b/////e/ /206 08210.
Yue Song, Nicu Sebe, and Wei Wang. Why approximate matrix square root outperforms accurate SVD global covariance pooling? In Proceedings of the IEEE International Conference on Computer Vision, pp. 1115-1123, 2021. http::/::/// http:///....t.g..1111 1tt.11100//11122222 ICCV48922. 2021.00113
Yue Song, Nicu Sebe, and Wei Wang. On the eigenvalues of global covariance pooling for fine- grained visual recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (3):3554-3566 2022a. URL http::://oi..oorrttt ott.or../.................1..111 1109 TPAMII 2022. 31 3178802.
Yue Song, Nicu Sebe, and Wei Wang. Fast differentiable matrix square root and inverse square root. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7367-7380, 2022b. URL
/ittps://dole org. 1109 TPAMI 2022. 3216339.
Yue Song, Nicu Sebe, and Wei Wang. Fast differentiable matrix square root. In International Con- ference on Learning Representations, 2022c. URL https: hwtp///pppeeeeeeeeeeeeeeee / net forum?
11s=AA0E--VTTUS
Yue Song, Nieu Sebe, and Wei Wang. Improving covariance conditioning of the svd meta-layer by orthogonality. In European Conference on Computer Vision, pp. 356-372. Springer, 2022d. URL
hittps://do http://doooooooooo corooooorg/000 1007 978 031 22053--33333 3_21.
Yann Thanwerdas and Xavier Pennec. Exploration of balanced metrics on symmetric positive def- inite matrices. In Geometric Science of Information: 4th International Conference, GSI 2019, Toulouse, France, August 27-29, 2019, Proceedings 4, pp. 484-493. Springer, 2019. URL
https://dde //tp:///oiio///ooo .rrg/
00700070 978 000-003-25550 10:30226980-22 7_50.
Yann Thanwerdas and Xavier Pennec. The geometry of mixed- -uulideen metrics on symmetric positive definite matrices. Differential Geometry and its Applications, 81:101867, 2022. URL
https: doii worg/10....... 10. 1016 6/5 j. .di di 20gee.2202..2. 101867.
Yann Thanwerdas and Xavier Pennec. O (n) invariant Riemannian metrics on SPD matrices. Linear Algebra and its Applications, 661:163-20 201,,, 2023. URL https: //::///ooo /ooi. otg//1..111..../ lea 2022. 12. 009.
13

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference Machine Learning, pp. 10347-10357. PMLR, 2021. URL https:
//proceed i ngs. ml mal.... press / w1t1////t////./..
html.
Qilong Wang, Peihua Li, and Lei Zhang. G2DeNet: Global Gaussian distribution embedding net- work and its application to visual recognition. In Proceedings of the IEEE Conference Com- puter Vision and Pattern Recognition, pp. 2730-2739, 2017. URL pestps://ooi.oom 1109/ /.///CCVRR222 2017. 689.
Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang, and Peihua Li. Deep CNNs meet global co- variance pooling: Better representation and generalization. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 43(8):2582-25977 2020a. BRLhttps: pesttp://don org/ /targ/1.... 1...1....9
TPAMI 2020 29748337
Qilong Wang. Li Zhang, Banggu Wu, Dongwei Ren, Peihua Li. Wangmeng Zuo, and Qinghua Hu. What deep CNNs benefit from global covariance pooling: An optimization perspective. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 10771-10780, 2020b. URL httph::t/////. htttp:://..i..ggggggepo wtt.://////..11//////22000000 01000
Qilong Wang, Mingze Gao, Zhaolin Zhang, Jiangtao Xie, Peihua Li, and Qinghua Hu, Dropcov: a simple yet effective method for improving deep architectures. Advances in Neural Information Processing Systems, 35:33576-33588, 2022a. URL https: /ttp//pppeeeeeeeeeeeee/eeeeeeeee net/ forum?
id id-QLGuUwDx45.
Qilong Wang. Zhaolin Zhang, Mingze Gao, Jiangtao Xie, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu. Towards a deeper understanding of global covariance pooling in deep learning: An optimization perspective. IEEE Transactions Pattern Analysis and Machine Intelligence, 2023. URL ponttps:/1100 11 1109/TTAMN
Rui Wang, Xiao-Jun Wu, and Josef Kittler. SymNet: A simple symmetric positive definite manifold deep learning method for image set classification. IEEE Transactions Neural Networks and Learning Systems, 33(5):2208-25) -2222, 2021. URL ht https: /ttp::///oooooo/ org/ /10.//1... 1109/ 9 ttnns..
2020. 3044176.
Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. DreamNet: A deep Riemannian manifold network for SPD matrix learning. Proceedings of the Asian Confer- ence Computer Vision, pp. 3241-3257, 2022b. URL https:
978 3- 1831-26351-20 39.
Rui Wang. Xiao-Jun Wu. Ziheng Chen, Tianyang Xu, and Josef Kittler. Learning discriminative SPD manifold neural network for image set classification. Neural Networks, 151:94-110, 2022c. URL https:tp////.. doi.doooooooooooooooooo org./rg/g/// /016/j neunet 2022 03.012.
Rui Wang, Chen Hu, Ziheng Chen, Xiao-Jun Wu, and Xiaoning Song. A Grassmannian man- ifold self-attention network for signal classification. In Proceedings of the Thirty-Third In- ternational Joint Conference on Artificial Intelligence, pp. 5099-5107, 2024a. URL https: pwwceedings///p/eooopppppss..sse /024/ / 11564.p....
/ // h/www. ijcai. org/ http:/aaa......ccee.eiiieeeeoeee
Rui Wang, Xiao-Jun Wu, Ziheng Chen, Cong Hu, and Josef Kittler. SPD manifold deep metrie learn- ing for image set classification. JEEE Transactions Neural Networks and Learning Systems, 2024b. URL https::///// http:/oooooooo/oooo 10.
2022. 3216811.
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and wolorinnsschroro
Pietro Perona. Caltech-UCSD birds 200. 2010. URL https:
seeppbbiccations C0t://CUB-20000.00 .df.
Jiangtao Xie, Ruiren Zeng, Qilong Wang. Ziqi Zhou, and Peihua Li. Sot: Delving deeper into classification head for transformer. arXiv preprint arXiv:2104.10933 2021. URL https://
arxiv. org/ abs/ /2102/04422 10935.
Kaicheng Yu and Mathieu Salzmann. aatistically-motivate second-order pooling. In Proceedings of the European Conference on Computer Vision, pp. 600-616, 2018. URL https: : :/tos:///do. doi. org/
14

---

Published as a conference paper at ICLR 2025
Tan Yu, Yunfeng Cai, and Ping Li. Toward faster and simpler matrix normalization via rank-1 update. In Computer Vision-Eee ECC-EECVVV 2020: 16th European Conference, Glasgow, UK, August 23- 28, 2020, Proceedings, Part XIX 16, pp. 203-219 Springer, 2020. URL https://doi..rrr 10. 10.1007/9788 030 58529-88899999 7_13.
Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Zi-Hang Jiang, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens-to-token ViT: Training vision transformers from scratch on ImageNet. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 558-567, 2021. URL hww.t:tttt.//ttttmmtt /doi wro.or/g/////// /10. 1109/ CTt////CC////// 922 2021. 00060.
Heliang Zheng, Jianlong Fu, Zue Zha, and Jiebo Luo. Learning deep bilinear transformation for fine-graine image representation. Advances in Neural Information Processing Systems, 32, 2019. URL https: : / //ll. acm. org /t////olo01111 5555/35542233 34542874 3454672.
Pengfei Zhu, Jialu Li, Zhe Dong, Qinghua Hu, Xiao Wang, and Qilong Wang. CCP-GNN: Compet- itive covariance pooling for improving graph neural networks. IEEE Transactions on Neural Nei- works and Learning Systems, 2024. URL https: / /ww.ooiooooooo..oo org/10. 1109/TNNL.. 2024. 3390249.
15

---

Published as a conference paper at ICLR 2025
APPENDIX CONTENTS
Future work
17
Related work
17
C Notations and abbreviations
18
D Additional preliminaries
18
D.I Pullback metrics
18
D.2 Riemannian operators on the SPD manifold
19
Technical details on Riemannian Logarithm
19
F Power-deformed GBWM as local power-AIM
21
G Additional discussions on Pow-TMLR, Pow-EMLR, and ScalePow-EMLR
22
G.1 The equivalence of Pow-EMLR and ScalcPow-EMLI
22
G.2 The in-equivalence of Pow-EMLR and Pow-TMLR
23
G.3 A Riemannian perspective of Pow-TMLR vs. Pow-EMLR
23
H
Additional experimental details
23
H.1
Datasets
23
H.2 Implementation details
23
H.3 Experiments on the second-order Transformer
24
H.4 Ablations on the bias reformulation
25
I Proof of Thm.
25
Additional discussions Thm. 2
27
16

---

Published as a conference paper at ICLR 2025
A FUTURE WORK
While Chen et al. (2024d) also explored Riemannian MLRs induced by other metrics, these MLRs involve computationally expensive Riemannian computations, rendering them unsuitable for large- scale datasets. As a future avenue, we aim to simplify the Riemannian computations in these alter- native Riemannian classifiers and apply them GCP for improved covariance matrix classification.
B RELATED WORK
Global covariance pooling. GCP aims to leverage the second-ordes statistics of deep features to enhance the learning competence of DNNs. DeepOdP (Ionescu al., 2015), acknowledged as the first end-to-end global covariance pooling network, employs matrix logarithm for the classification of covariance matrices. This method also offers matrix backpropagation to differentiate the gradient w.r.t the decomposition based matrix functions. Following this pioneering work, B-CNN (Lin et al., 2015) employs the outer product of global features and carries out -liment-wise power normaliza- tion. However, there exist three limitations of the above two methods. Firstly, the high dimensional covariance feature considerably escalates the parameters of the FC layer, thereby introducing the risk of overfitting. Secondly, the matrix logarithm could over-stretch the small eigenvalues, under- mining the effectiveness of GCP, Thirdly, the matrix logarithm is based on matrix decomposition, which is computationally expensive. The subsequent research primarily focuses on four aspects: (a) adopting richer statistical representation (Wang et al., 2017; Zheng et al., 2019; Nguyen, 2021); (b) reducing the dimensionality of the covariance feature (Gao et al., 2016; Kong & Fowlkes, 2017; Cui et al., 2017; Acharya et al., 2018; Rahman et al., 2020; Wang et al., 2022a); (c) investigating effec- tive and efficient matrix normalization (Li et al., 2018; Zheng et al., 2019; Lin & Maji, 2017; Yu et al., 2020; Song et al., 2022c;b); (d) improving covariance conditioning for better generalization ability (Song et al., 2022d;a). In this work, we do not aim to achieve state-of-the-ant art performance over the existing GCP-based methods but rather to unravel the underlying theoretical mechanism of GCP matrix functions.
Interpretations of global covariance pooling. Along with the progress of GCP, several works began to study its mechanism. Wang et al. (2020b) investigated the effect of GCP on deep Convo- lutional Neural Networks (CNNs) from an optimization perspective, including accelerated conver- gence, stronger robustness, and good generalization ability. Wang et al. (2023) further broadened these investigations, substantiating the merits of GCP in other networks, such as vision transform- ers (Touvron et al., 2021; Yuan et al., 2021; Liu et al., 202 and differentiable Neural Architecture Search (NAS) (Liu al., 2019). Song et al. (2021) empirically studied the advantage of approximate matrix square root over the accurate one. Wang et al. (2022a) considered the matrix power as decor- relating representations and developed a adaptiee dropout to produce lover-dimensional covariance matrices. Nevertheless, existing literature does not fully address the fundamental ques- tion of why Euclidean classifiers operate effectively in the non-Buclidean space generated by the matrix power. Our research fills in this theoretical gap, offering intrinsic explanations regarding the role of the matrix functions in GCP.
Riemannian classifiers SPD manifolds. Since the matrix logarithm is diffcomorphism be- tween the SPD manifold and its tangent space at the identity (Arsigny et al., 2005), the most widely used classifier SPD manifolds is composed of the matrix logarithm and a Euclidean classifier (Wang et al., 2021; Chen et al., 2023b; Wang et al., 2022b; Nguyen, 2022a;b; Wang et al., 2022c; Chen al., 2024b; Wang et al., 2024b). However, this tangent classifier might distort the intrinsic geometry of SPD manifolds. Similar issues also arise in other manifolds (Huang et al., 2017; Wang al., 2024a; Chen et al., 2025) Inspired by HNNs (Ganea et al., 2018), recent studies have devel- oped intrinsic classifiers directly on SPD manifolds. Nguyen & Yang (2023) introduced three gyro structures on SPD manifolds induced by AIM, LEM, and L.M,, respectively. Based on these gyro structures, the authors generalize the Euclidean Multinomial Logistics Regression (MLR). Concur- rently, Chen et al. (2024a) proposed a formula for SPD MLR under Riemannian metrics pulled back from the Euclidean space. However, both works require specific Riemannian properties and focus on certain metrics on SPD manifolds. Chen et al. (2024d) presented a general framework for de- signing Riemannian MLRs on general geometries and showeased their framework under various metrics on SPD manifolds, covering the SPD MLRs introduced by Chen et al. (2024a); Nguyen & Yang (2023). Based this framework, Chen et al. (2024c) showcased the SPD MLR under their
17

---

Published as a conference paper at ICLR 2025
Table 7: Summary of notations.
Notation
Explanation
ST
The SPD manifold
S C TpST.
The Euclidean space of symmetric matrices
The Euclidean space of X n. lower triangular matrices
The tangent space at P E 84
9p or
(.)P
The Riemannian metric at P E S
or.::
The standard Frobenius inner product
Logp
The Riemannian logarithm P
Ha.
The Euclidean hyperplane The differential map of / at P E S
S..P
f"g
The pullback metric by f from g The adjoint operator of linear maps ST {ax E I2 min(a, a + n33) > 0}
ad(-)
ST
(..))0.))
The Oin)-invarin Euclidean inner product
(
The Riemannian metric of (a, 3)-LEM
(a.8))-1
The Riemannian metric of (ax A)MAIMM
10,0,0)-111
The Riemannian metric of (0, o; (3)AIMM
(
The Riemannian metric of (a:, 3)
(8,0,8)
The Riemannian metric of (0, ot, BE-M
g ((11.2))
The Riemannian metric of (0), 02)
g Bw
The Riemannian metric of BWM The Riemannian metric of M-BWM
g M-BW
g (WW
The Riemannian metric of (29, -WWM
glC
The Riemannian metric of LCM
S-LCC
The Riemannian metric of 6-LCM
FFC or F(; b)
The FC layer
Powe or (.)
The matrix power
fve
The vectorization Euclidean classifier
fEc
mlog
The matrix logarithm
Crl
The Lyapunov operator
Chol
The Cholesky decomposition
C
The generalized Lyapunov operator
Diog( fM I.
The diagonal element-wise logarithm The matrix function of matrix power or logarithm
The strictly lower triangular part of square matrix A diagonal matrix with diagonal elements from a square matrix
(O-)
proposed product metrics. In this paper, based on the Riemannian classifiers developed by Chen et al. (2024d), we present an intrinsic explanation for matrix functions in GCP.
C NOTATIONS AND ABBREVIATIONS
For better clarity, we summarize all the notations in Tab. and all the abbreviations in Tab. 8.
D ADDITIONAL PRELIMINARIES
D.1 PULLBACK METRICS
The power-deformed metrics the SPD manifold are special cases of pullback metrics. Pullback metrics are common techniques in Riemannian geometry, connecting different Riemannian metrics.
18

---

Published as a conference paper at ICLR 2025
Table 8: Summary of Abbreviations.
Abbreviation
Explanation
SPD
Symmetric Positive Definite
GCP
Global covariance pooling
GAP
Global Average Pooling
LEM
Log-
Log-Euclidean Metric
AIM
Affine-Invariant Metric
EM
Euclidean Metric
PEM
Power Euclidean Metric
MPEM
Mixed Power Euclidean Metric
BWM
Bures-Wasserstein Metric
GBWM
Generalized Mtrees-asserstechn Metric
FGVC
Fine- GaninGr Visual Categorization
MLR
Multinomial Logisties Regression
EMLR
Euclidean Multinomial Logistics Regression
RMLR
Riemannian Multinomial Logisties Regression
SPD MLR
RMLR SPD manifolds
Log-EMIR
Eq. (4)
Pow-EMLR
Eq. (5)
Pow-TMLR
EMLR the tangent space generated by Eq. (7)
ScalePow- EMLR
MaaeePow-EMLL. in Tab. 4
Cho-TMLR
EMLR the tangent space generated by Eq. (8)
Definition 3 (Pullback Metrics). Suppose M, are smooth manifolds, is a Riemannian metric on N, and ʃ M + is smooth. Then the pullback of g by ʃ is defined point-wisely,
(V'g),(V, V2) (o.
(19)
where p E M. (.)(()) is the differential map of f at p, and V; E T,M. If f"y is positive definite, it a Riemannian metric on M, which is called the pullback metric defined by f.
D.2 RIEMANNIAN OPERATORS ON THE SPD MANIFOLD
The O(n)-invarianl Euclidean inner product on S (Thanwerdas & Pennec, 2023) is defined
(V, Wy aVV W) + (r(())
(20)
where 8) E ST with ST () E 2 min(a, n3) > 0}, V.W E S", , (..-) is the standard matrix inner product.
We summarize deformed SPD metrics and associated Riemannian operators in Tab. with the fol- lowing notations. Specifically, P. Q, M E ST+ are SPD matrices, and V, W are tangent vectors in the tangent space P. i.e., Se. We denote .r(:,)) as the Riemannian metrie at P, and Logr(-) as the Riemannian logarithm at P, respecttivell Also, Chol and mlog represent the Cholesky decom- position and matrix logarithm, with their differential maps P denoted as Chol,, p and mlog.,
respectively. We denote -= -hhol,,p W- Cho),,,(W), T- Chol P),, and K= Chol(Q) [.] the strictly lower part of a square matrix, DD(-) is a diagonal matrix with diagonal elements of square matrix, and Diog(-) is a diagonal matrix consisting of the logarithm of the diagonal entries of a square matrix. We denote Lp.M [V] the generalized Lyapunov operator, ihe solution to the matrix linear system M [V PPP.PMMMM Lyapunov operator, denoted as [V].
When M=I, Lp [V] is reduced to the
E TECHNICAL DETAILS ON RIEMANNIAN LOGARITHM
We first review a well-known result for the pullback metric (Thanwerdas & Pennec, 2022, Tab. 2).
19

---

Published as a conference paper at ICLR 2025
Table 9: Riemannian operators and deformed metrics of seven basic metries on SPD manifolds. Note that for MPEM, P and Q must be commuting matrices when computing the Riemannian logarithm.
Deformation
Name
Riemannian Merric (r(V;
Riemannian I Lngarithm Log,
@
9Y-LEM
(mllgg., (V)), mig,,
(mllg...)) [nalog(Q)
mlog(P:)
(Thaweedis Pennec, 2023)
(a, AJ-MM
(
p1/ mlog (P-10pp-1))) pl
Powiy (hinps
(Thnnwerda & Pennee, 2025)
(Thrwwedas & Pennec, 2023)
9J-MM
Q- P
Pow gl
(Thawerdas & Penner, 2022)
18. 82)-EM
mu3 Powe. (VV), PWW) (Powe..)) 10" po), with 621/2
N/A
LCM (Liin, 2019)
May V. + 25 Voy (Chol ")., [[K] [1] (211)) Digg0X))
Powi o
(Bharia at, 2019)
(PQ) (QP))/2
Pow d
GBWM 2023)
((M (QM-)PM-1) M 2P
Powty g nw
Lemma 4. Given a Riemannian metric g on the SPD manifold S++ and a diffeomorphism f + + S4+, the Riemannian logarithm Logr under the pullback metric ğ f*g
LogpQ = (f=,p))² (Log f(0)):
(21)
where f=,P is the differential map at P, and Log is the Riemannian logarithm under g-
Next, we show a lemma about the scaling of a Riemannian metric.
Lemma 5. Supposing S is endowed with Riemannian metric g and > 0 is positive real scalar, the scaling metric og shares the same Riemannian logarithm map with g.
Proof. Since the Christoffel symbols of ag are identical to those of g. the geodesic functions under both ag and remain unchanged. This implies that the Riemannian exponential maps are the same for ag and g. As the inverse of the Riemannian exponential maps, the Riemannian logarithm maps under ag and g are also identical.
By the above lemmas, we can readily prove Tab. 2.
Proof. By Lem. 5, for the power-deformed metric of metric in S the Riemannian logarithm I is the same as the counterpart under Pow g. Therefore, in the following, without loss of generality, we compute Log, under Pow We further denote the Riemannian logarithm under as Log.
In the following, we denote P as SPD matrix, 0 as the X n zero matrix, and V as a tangent vector TyS Besides, we note that
PoWs., (V) = OV.
(22)
We first deal with (a, 3)MLEM and 6-LCM, as both of them are pullback metrics from the Euclidean space. Then, we proceed deal with other metrics
(a, -EM:: As shown by Thanwerdas & Pennec (2023), the Riemannian logarithm at / is
(og, = mlog, ) / (mlog(P) mlog(!))
(23)
mlog(P) P).
0-LCMM:
We define a map
f f v Chol Cholo
(24)
where (LL) = [L] + (Dog((()) for the lower triangular matrix L. Chen et al. (2024c) shows that LCM is the pullback metric by vi Chol from the Fuclidean space L of lower triangular matrices.
20

---

Published as a conference paper at ICLR 2025
Therefore, Pow' gf is the pullback metric from L" by f. Besides, we have the following:
S(P) = [ + DIo)(D(L)),
(25)
f(T) =0, 0,
(26)
(, - (LV) + (01))
(27)
where L Chol(P*). We have
Logy(P) (fx,p) ((f,,))-(((()) ((((P) - f(I))
- क L + -Lj + 2lolog
(28)
For (0, 3),MM.... (01, -MM (0, a, A)-AIM, 20-BWM, and (20, p28 JBWM. we denote Log, as their logarithm at I, while Log, as the logarithm under the metric before deformation. The results can be directly obtained by Eq. (22), Lem. 4, Lem. 5, and Tab. 9.
(0, a, -)EM:
1 Log,
Log, (P*)
(29)
1
pp@- 1).
0
(01, -EM: The Log, can be directly obtained by Tab. 9 and Eq. (22).
(0,a, a. AAMMM:
(P) Lög (Po)
(30)
mlog( Po))
ml)g(P).
28-BWM:
1
(P)
Log (p20)
20
(31)
pD - I).
(28, P28 -BWM: Under M-BMM, we have
Log, (M) 2(M - 1).
(32)
Therefore, for (0,.P2)-BBW' we have
Log, = 1
Log, (P20)
20
(33)
= (p) - I).
F POWER-DEFORMED GBWM AS LOCAL POWER-AIM
Let us first formalize this property. Proposition For any P S and V. W TpSt
we have the following:
p20 BW
(20,1,0)-al W.W. 9p
(V, W)
(34)
21

---

Published as a conference paper at ICLR 2025
Proof. As shown by Bhatia (2009), the Riemannian metric of the standard AIM ((1, 1, ()-MM is
g (V. W) vec(V)) (P P) ()c())
(35)
where vec is the column vectorization of V, @ is the Kronecker product.
For the (20, p we have the following:
(20 BW (V. W)
1
26 (P) BBBW (V, W,,W
9p
462
9
1
vec V) (P & Pe-)ve(
4
402
(36)
1
(V,
4 402
4 9
(20,1.0)-A1,
(20,1.0)Al((V. W),
(V,
where V Pow2s.,P(V), W = Pow20., (W), p29 and Eq. (36) can be obtain by Han et al.
(2023, Eq. 3)
G ADDITIONAL DISCUSSIONS ON POw-TMLR, POw-EMLR, AND
SCALEPOW-EMLR
G.1 THE EQUIVALENCE OF POW-EMLR AND SCALEPOW-EMLR
It can be proven that Pow-EMLR is equivalent to ScalePow PaMeeLLR--LLRR under scaled initial weight and learning rate in the FC layer. We denote the network as
3o E Red ((:0)
I E IRd fEG - LE
(37)
where xp. g(: 0), fFC, and are the input feature, feature extraction with parameter O, FC layer, and loss, respectively. The FC layers in Pow-EMI. R and ScalePow-FMM are denoted as y = + and y y= } + b. We set the initial values and learning rates of A and satisfying Ao = 1 A and y 82y, and maintain all the other settings the same. Then, we have the following for the gradient at A = Ag (or A = Au):
aL
ØL
1 10L 0ÿ T
aL
0 0y x
01
0 DA'
(38)
0L
OL
A ÖL
OL
A
dz
0g
Dy
dr
Under SGD, the updated values of Å satisfying the following:
1
+ 7 OL
0
A1
0
(Ao
0A
1
1 OL
(Ao + 7 '00A
1 Ao
ØL
(39)
+ DA
0
ØL.
Ao ? DA
A
Therefore, the updated values of A and still satisfy A A addition, the gradients of Pow-EMLR w.r.t x and are identical to ScalePow-EMMi w.r.t. and b. Therefore, Pow-EMLR equivalent to ScalePow-EMMI under scaled settings.
22

---

Published as a conference paper at ICLR 2025
G.2 THE IN-EQUIVALENCE OF Pow-EMLR AND POW-TMLE
We denote X = S. Then for Pow-TMLR, we have the following
y= F fvee (2(x+1) :A,b
F (rre (X + 1): :Abb)
(X+
(40)
F (vv (x);
where A= / and b b (T).
As A appears in b. the gradient of A is composed of two parts, one w.r.t. y and the other one w.r.t b. In contrast, in the standard FC layer = FXx; A,b, the gradient of A is independent of Therefore, Pow-TMLR cannot be simply viewed as equivalent to Pow-EMLR with transformed initialization.
G.3 A RIEMANNIAN PERSPECTIVE OF Pow-TMLR vs. POW-EMLR
Although the numerical expressions of Pow-TMLR and Pow-EMLR differ by a constant transfor- mation, they differ fundamentally in theory: Pow-TMLR is a tangent classifier, whereas Pow-EMLR a Riemannian classifier.
1. Tangent Classifier: The tangent classifier treats the entire manifold as single tangent space at the identity matrix. When mapping data into this tangent space, critical structural information, such as distances and angles, cannot be preserved. This distortion undermines classification performance. In contrast, Riemannian MLR is constructed based on Rieman- nian geometry, fully respecting the manifold's geometric structure.
2. Tangent as a Special Case of Riemannian Classifier. The tangent classifier can be seen reduced of the Riemannian classifier. For example, let take Eq. (16) as an example. When all SPD parameters Ps are set to the fixed identity matrix, Eq. (16) exactly corresponds to Pow-TMLR.
In summary, the Riemannian classifier enjoys significant theoretical advantages over the tangent classifier while incorporating the tangent classifier special case.
H ADDITIONAL EXPERIMENTAL DETAILS
H.1 DATASETS
The Caltech University Birds (Birds) (Welinder et al., 2010) dataset is composed of 11, 788 images distributed over 200 different bird species. The FGVC Airerafts (Aircrafts) (Maji et al., 2013) dataset comprises 10, 000 images of 100 classes of airplanes, while the Stanford Cars (Cars) (Krause et al., 2013) dataset consists of 16, 185 images representing 196 classes of cars. In addition these widely used FGVC datasets, we also evaluate our proposed theory on the large-scale ImageNet-I 1 (Deng al., 2009) dataset, which contains 1.28M training images, 50K validation images and 100K testing images distributed across 1K classes.
H.2 IMPLEMENTATION DETAILS
We follow the official Pytorch code iSQRT-COV' (Li ct al., 2018) to reimplement GCP. Follow- ing Wang et al. (2020a); Song et al. (2022a), we use ResNet-18 as our backbone network on the ImageNet dataset, and ResNet-50 on other three FGVC datasets. On Both the ImageNet- and FGVC datasets, the ResNet- 18 and Re are trained from scratch with the GCP layer.
23

---

Published as a conference paper at ICLR 2025
As the matrix square root is the most effective matrix function in GCP, we power 1/2 for matrix power normalization. Following Song et al. (2022a), we reduce the channels of the final convo- lutional features from 2048 to 256 for compact representation of covariance matrices, producing 256 x 256 spatial covariance matrices. We train the network from scratch with an SGD optimizer on all datasets. For fair comparison, the learning settings are identical for Pow-EMIR and ScalePow- EMLR, while we fine-tune Pow-TMLR w.r.t. learning rate, classifier factor, and batch size on three FGVC datasets. The learning rate of the FC layer is set be times larger than the convolutional layers, where l is called the classifier factor. We use a step-wise learning scheduler, dividing the learning rate by at n-th epoch. Tab. 10 summarizes the hyperparameters in our main experiments in Tab. 5.
For Cho-TML.R, the learning rate set as le-2, 5e 5e-3, and 3e-3 for the convolutional layers on the Aircrafts, Birds, and Cars dataset. The batch size on the Cars dataset is 4. On the Aircrafts dataset, the training lasts 60 epochs with the learning rate divided by epoch 50. On the Cars and Birds datasets, the training lasts 120 epochs with learning rate reduction by divisor of 10 at epoch 100.
The experiments on ImageNet use a workstation with 32-core AMD EPYC 7302 CPU and an NVIDIA RTX A6000, while other experiments use a workstation with 16-core AMD EPYC 7302 CPU and an NVIDIA GeForce RTX 2080 Ti GPU. Due to the heavy computational burden of Cholesky decomposition, we do not implement Cho-TMLB on the ImageNet.
Table 10: Summary of hyperparameters.
Backbone
ResNet18
ResNet50
Dataset
ImageNet
Aircrafts
Birds
Pow-EMLR
Pow-EMLR
Pow-EMLR ScalePow-EMLR
Pow-EMLR MonlePow-EMRR
Classifiers
ScalePow-EMM
ScalePow-FAM EM-R
Pow-TMIR
Pow-TMLR
Pow-TMLR
Pow-TMLR
Learning Rate
1e 1.1
5e-3
52-
5e-3
Weight Decay
le-4
1e-
1e-4
le-4
1e-4
Classifier Factor
10
I.R Scheduler
(30,45,0)
(21,50)
[50, 100)
100)
[50, 100)
[50, 100]
Batch Size
256
10
10
10
Epoch
60
50
100
100
100
100
H.3 EXPERIMENTS ON THE SECOND-ORDER TRANSFORMER
Table 11: Comparison of Pow-EMLR, ScalePow-EMMLL and Pow-TMLR under the SoT-7 backbone on the ImageNet- Ik dataset.
Classifier
Top-1 Acc (%)
Top-5 Acc (%)
Pow-TMLR
75.79
92.91
Pow--MLRRR
76.14
93.18
Pow-EMLR
76.11
93.05
To further validate our findings, we follow Song et al. (2022b) to conduct experiments using the Second-order Transformer (SoT) (Xie et al., 2021) the ImageNet- 1k dataset. Specifically, use the 7-layer SoT (SoT-7) architecture as the backbone network and train the model up to 250 epochs with batch size of 512, keeping the other settings the same as Song et al. (2022b).
As shown in Tab. 11, Pow-EMLR still achieves similar performance to ScalePow -MML,, but outper- forms Pow-TMLR. These results further support claim that tangent classifiers cannot adequately explain the matrix functions used in GCP, while the underlying mechanism can be better explained by Riemannian perspective.
24

---

Published as a conference paper at ICLR 2025
H.4 ABLATIONS ON THE BIAS REFORMULATION
Recalling the vanilla Pow-EMLR (Ax, S - bx. it can be rewritten as SA - Ph) according to Eq. (10), where (Pk. = by- This reformulation is the key step to extending Euclidean MLR. Although this reformulation has shown success in different Riemannian MLRs (Ganea et al., 2018; Shimizu et al., 2020; Chen et al., 2024a;d; Nguyen & Yang, 2023), we conduct ablations on this reformulation. For simplicity, set Ps identical for all k; otherwise, it will bring [k, n, n] inter- mediate tensor for each [n, n| covariance. We compare the following to MLR:
Pow-EMLR: (A), bk,
(41)
Pow-EMLRR: (A P)
(42)
where Ag, P E S". Tab. 12 shows the results on all three fine-grained datasets. We observe that Pow-EMLR performs similarly to Pow-EMLR'.
Comparison of Pow-TMLR, Pow-EMIR, and Pow-EMIR' on all three fine-grained datasets.
Table 12:
Air
Birds
Classifier
Top-1 Acc
Top-5 Ace (%)
Top-1 Acc (%)
Top-5 Acc
Top-1 Acc (9)
Top-5 Acc (%)
Pow-TMLR
69.58
88.68
52.97
77.8
51.14
74.29
DOw-MMRR
73.03
90.4
63.96
85.02
80.06
94.02
Pow-EMT.R
72.07
89.83
63.29
84.66
80.43
94.15
I PROOF OF THM.
This proposition is mainly inspired by Chen al. (2024a, Thm. 5). However, all the results by Chen et al. (2024a) require the metric to be pullback metric from a standard Euclidean space, while the metric in our Thm. a pullback metric from the SPD manifold. Nevertheless, we still can reach similar theoretical results. We first recap RSGD and then begin to present our proof.
RSGD (Bonnabel, 2013) is formulated as
W (xxI(w>
(43)
where EXPw is the Riemannian exponential map at W, and Tw maps the Euclidean gradient Vw the Riemannian gradient, and y denotes learning rate.
We denote (1, -)-MM as EM, and the metric tensor of it as gF. Instead of providing an ad hoc proof exclusively for PEM, we present the following two more general lemmas.
Lemma 7. Given diffeomorphism
+ + S4 +. & induces a pullback metrics on S from ())+ denoted as g The o --E --nduudde SPD MLR is
p(y k) exp (((()) 6(Pk), (v,,))l
(44)
where S E S is an input feature, Ps S and A E S" are parameters for each class k.
Proof. According to Chen al. (2024d, Thm. 3.3), the Riemannian MLR based on gh is given as p(y = k|S) exp
A-F gr. (Logp, S,PT An)]
exp (Ó(() @(Pa), o.,, (Ax))),
(45)
where Eq. (45) can be obtained by the properties of deformed metrics (Thanwerdas & Pennec, 2022, Tab. 2) and EM (Thanwerdas & Pennec, 2023, Tab. 3).
Following the notations in Lem. 7, we have the following lemma.
Lemma 8. Supposing $.,1 is the identity map and each SPD parameter Ps (Euclidean parameter A&) in Eq. (44) is optimized by F based RSGD (Euclidean SGD), the -.-sed SPD MLR is equivalent to Eaclidean MLR illustrated in Eq. (10) in the co-domain of é.
25

---

Published as a conference paper at ICLR 2025
Proof. We first show the projection operator IIp at P S under g and then the equivalence.
For any smooth function f S I S endowed with g denote its Euclidean and Riemannian gradient P E S++ Pff and Pff, respectively. Then, for any V TpS++ we have
(VPt.V)== V(f) P = [p((vr)) = -7 (ad(o., (Vrf),
(Vef.V)
(46)
where ad(-) is the adjoint operator of the linear map. According to Eq. (10), we define a Euclidean MLR the codomain of á as
pyy k S) exp((o(S) Ps. As))), with Pk. Ag S".
(47)
We call this classifier Ø-EMLR.
Following Lem. 7, the SPD MLR under g is
ply S) exp(()(S) ((Px), As)), with Ps E ES4,, Ax E
(48)
Supposing the SPD MLR and 6-EMLR satisfying Ps. ((P)). Other settings of the network are all the same, indicating the Euclidean gradients satisfying
ØL. ØT.
(49)
0PP Pp(P))
The updates of Ps in the ş-EMLR is
ØL Pk = Ps. -
(50)
The updates of Ps in the SPD MIR is
Pk = ExPp,
(1, (vr.f)
0L OPr
(51)
Φ- ((PP)) ())
Therefore ((P() satisfies
0L
p(P)) = 6(P)) (ad(or.)))
0Pr
aL (bb(P))
((P)) (ad(o.,
ad((),p))
(52)
OL
= (PP))
PP()P)
Pk-
The second equation comes from the Euclidean chain rule of differential. Let Y x(X), then have
aL. dY
ØL
aY ay : Φx,x (d
(53)
ØL
= ad(o..x)
ddX,
0Y
where : means Frobenius inner product.
The equivalence of Åς and Aş obvious. Since both forward and backward processes of Eq. (47) and Eq. (48) are identical, by natural induction, the lemma can be proven.
L
When > 0, simple computation shows that (0, 1MM is the pullback metric EM by Oo with Ø0,,r an identity map. According to Lems. and 8, readily prove Thm. 2.
26

---

Published as a conference paper at ICLR 2025
J ADDITIONAL DISCUSSIONS ON THM. 2
In Lem. 8, &..s is required to be identity map. However, Lem, 8 can be extended into the case where Φ-, I is not the identity map, which will further extend Thm. 2 into the case of 0 < O.
Following the notations in Lem. 7, let á - S be a diffeomorphism, whose differential at
I, i.. is not identity map. As is a diffeomorphism, the differential map Φ., TyS" (o is a linear isomorphism, i.e.a bijection preserving linear operations. Therefore, we can identify Ag- (As) with A& in Eq. (44), and the SPD MLR under godt is simplified as
ply k|S) x exp (1(()) ((Px), As)
(54)
where Av E S". As direct corollary, all the proof in Lem. 8 can be transferred to the case where Φx,1 is not an identity.
Corollary 9. Supposing
ST S++ a diffeomorphism and each SPD parameter Pk (Eu- clidean parameter A&) in Eq. (54) is optimized by based RSGD (Euclidean SGD), the R
based SPD MLR is equivalent to a Euclidean MLR in the co-domain of Φ.
As direct application of Cor. 9, Thm. can be generalized into the case of 0 < 0. We generalize the definition of dy as
( = 10 5 VS E ST+, with 0 + 0.
(55)
Obviously, Φo S following results.
S is still diffeomorphism. By Cor. 9, we can readily obtain the
Corollary 10. Under PEM with 0 + O, optimizing each SPD parameter Ps in Eq. (54) by PEM- based RSGD and Euclidean parameter Aş by Euclidean SGD, the PEM-based SPD MLR equiv- alent to a Euclidean MLR illustrated in Eq. (10) in the co co-domain of (.(.).
Rahman et al. (2023) adopted the inverse of the covariance matrix for GCP. Cor. 10 indicates that framework also explain the underlying mechanism of the inverse Rahman al. (2023).
27


Reviews:
[{"review_id": "djf6GEbOD0", "reviewer": "Reviewer_SGk2", "summary": "This paper provides a theoretical understanding of matrix function normalization in Global Covariance Pooling (GCP) from the perspective of Riemannian geometry, specifically addressing the use of matrix logarithm and power for classifying covariance matrices. By interpreting these matrix functions through Riemannian and tangent classifiers, the authors reveal that the efficacy of GCP is rooted in its respect for the Riemannian structure of Symmetric Positive Definite (SPD) matrices. Extensive experiments demonstrate that matrix power, often more effective in practical applications, implicitly respects a Riemannian classifier, justifying its empirical performance in GCP.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper offers a unified theoretical explanation of the working mechnism of matrix functions which demonstrates that matrix functions in GCP are implicitly Riemannian classifiers. Based on the unified theoretical analysis, it also shows that the empirical advantages of matrix power over matrix logarithm in the GCP could be attributed to the characteristics of the underlying Riemannian metrics.\n- The paper experimentally validates that the mechanism of matrix normalization should be attributed to Riemannian classifiers instead of tangent classifiers across different datasets and networks.", "weaknesses": "- In sec. 5.3, the analysis shows that the matrix power function inherently offers better performance over the matrix logarithm based on its associated Riemannian metric ((θ, 1, 0)-EM). The description of (θ, 1, 0)-EM as a “balanced” alternative to LEM might be overly simplified. The analysis could benefit from a more nuanced discussion of the limitations of each metric, such as specific conditions where (θ, 1, 0)-EM may not perform as well as LEM. \n- In sec. 6.1, although ScalePow-EMLR is claimed to be equivalent to Pow-EMLR under scaled settings, the analysis lacks a comprehensive justification for this equivalence. Exploring how different scaling factors impact performance would help.\n- The paper primarily evaluates ResNet-based architectures. How well would the findings generalize to other architectures, such as ViT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gTLxYjA27Q", "reviewer": "Reviewer_JvSJ", "summary": "This paper analyzes classifiers for normalized covariance-pooled features. Specifically, the authors explore a rationale behind the effective linear classifier (Euclidean MLR) directly applied to square-rooted covariance features. By analyzing empirical performance results and theoretical formulations of Euclidean MLR and SPD MLR, this paper finds out that the Euclidean MLR of square-rooted covariance features is well explained by the SPD MLR on power-Euclidean metric. The experimental results on image classification using global covariance pooling (GCP) validates the claim on various settings.", "questions": "See the above-mentioned weak points.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ The theoretical stuffs regarding SPD Riemannian geometry are insightful and the empirical performance comparison in Tab.3 is interesting.\n+ Practical performance analysis is conducted on various types of datasets and networks.", "weaknesses": "- The discussion to induce SPD MLR in Sec.5 sounds a bit strange. In Sec.5, a Euclidean-MLR is reformulated as $\\langle a,x\\rangle - b \\Rightarrow \\langle a,x-p\\rangle$ by re-parameterizing the bias as $b=\\langle a,p\\rangle$. If such re-parameterization is acceptable, one can more directly conclude that a Tangent-MLR is equivalent to the Euclidean-MLR by using the re-parameterization of TMLR: $\\langle a,x-1\\rangle -b \\Rightarrow \\langle a,x-p'\\rangle$: EMLR, where $b=\\langle a,p'-1\\rangle$. It skips SPD-MLR and thus is a different conclusion from this paper. Therefore, the authors' claim seems to be less convincing.\n\n- On the other hand, from a viewpoint of classifier formulation, Pow-EMLR is equivalent to Pow-TMLR since $\\langle a,x\\rangle -b=\\langle a,x-1\\rangle -b'$ with one-to-one correspondence between the parameters $(a,b)$ and $(a,b')$ via $b=b'+\\langle  a,1\\rangle$. Thus, the performance difference shown in Tab.3 comes from training dynamics only, especially in terms of the bias. This fundamental issue is not mentioned in this paper and is less connected to the classifier model (e.g., SDP-MLR) that this paper focuses on. As one could *guess* that disentanglement between classifier weights and a bias is preferable for training, it would be more interesting to delve into the training processes with much attention to the bias in the framework of covariance feature.\n\n- This paper insists that an SDP-MLR is a fundamental classifier for covariance features. To validate the claim, it is necessary to compare the SDP-MLR with Tangent-MLR and practical EMLR for respective metrics of LEM and PEM (and optionally LCM). Related to the above-mentioned bias issue, the SDP-MLR would exhibit different training dynamics from the EMLR $\\langle a,x\\rangle -b$, resulting in different performance. Thus, Tab.5 should contain results of SDP-MLR directly applied to covariance features.\n\n- The presentation of this paper is not so clear since it introduces less-relevant metrics in Sec.2, leading to messy notations (parameters) and rather complicated discussions. It would be enough to present three metrics of Log-Euclidean and Power-Euclidean; if this paper finds efficiency of Log-Cholesky metric for covariance features, it might be nice to contain LCM as an additional contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n2oecb0sdk", "reviewer": "Reviewer_24iT", "summary": "As global covariance pooling (GCP) becomes an increasingly important component in modern deep neural networks, a discrepancy remains between theoretical principles and practical applications of the normalization techniques used in GCP, especially regarding the matrix logarithm and matrix power. This work aims to bridge this gap by offering a theoretical understanding from a Riemannian geometry perspective. Through a combination of theoretical analysis and empirical evaluations, the study shows that the working mechanism of these matrix functions can be attributed to the Riemannian classifiers they implicitly respect.", "questions": "1. I am concerned that the theoretical contribution of this work appears to have a somewhat narrow application scope in its current presentation, as it focuses specifically on matrix normalization methods in GCP for deep neural networks, particularly on the matrix logarithm and matrix power. I suggest that the authors further clarify the practical significance of matrix normalization and the presented theories.\n2. There are some other matrix normalization methods in GCP beyond matrix logarithm and matrix power. Popular choices [1] include matrix logarithm, element-wise power, matrix square-root and matrix power normalization. Is the element-wise power normalization applicable to the presented theories? Does it require additional analysis?\n3. In Figure 1, in addition to showing the gap between LEM and PEM, it would be beneficial to highlight the consequences of this gap.\n\n\n[1] Wang, Qilong, et al. \"What deep CNNs benefit from global covariance pooling: An optimization perspective.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work identifies the discrepancy between the theory and practice of matrix normalization in GCP and addresses it by providing comprehensive theoretical explanations and underlying rationale for using Euclidean classifiers.\n2. The study is technically sound, combining theoretical proof with empirical evidence.\n3. Experiments on several real-world datasets are conducted to support the theoretical claims.\n4. Overall, this paper is well-written and thoughtfully structured.", "weaknesses": "1. I am concerned that the theoretical contribution of this work appears to have a somewhat narrow application scope in its current presentation, as it focuses specifically on matrix normalization methods in GCP for deep neural networks, particularly on the matrix logarithm and matrix power. I suggest that the authors further clarify the practical significance of matrix normalization and the presented theories.\n2. There are some other matrix normalization methods in GCP beyond matrix logarithm and matrix power. Popular choices [1] include matrix logarithm, element-wise power, matrix square-root and matrix power normalization. Is the element-wise power normalization applicable to the presented theories? Does it require additional analysis?\n3. In Figure 1, in addition to showing the gap between LEM and PEM, it would be beneficial to highlight the consequences of this gap.\n\n\n[1] Wang, Qilong, et al. \"What deep CNNs benefit from global covariance pooling: An optimization perspective.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "51F4Vynv3Z", "reviewer": "Reviewer_2skh", "summary": "This paper begins by exploring the motivation behind MPN-COV, focusing on how matrix power normalization (MPN) can be regarded as an approximation of geodesic distance (Log-E metric) for SPD matrices. Experimental results show there exist a significant discrepancy between MPN and Log-E metric.\n\nThen, the authors try to study whether the theoretical framework used for matrix logarithmic normalization can explain matrix power normalization. They find that matrix power normalization maps SPD matrices to the tangent space (Euclidean space) at the identity matrix, and construct Pow-TMLR. But experiments reveals that Pow-TMLR shows spoor performance.\n\nFinally, following by the works [Nguyen & Yang, 2023; Chen et al., 2024a;c] those expanded Euclidean MLR into SPD manifolds, this paper extends matrix logarithmic and matrix power normalization to SPD multinomial logistics regression. They construct ScalePow-EMLR based on this theory and demonstrate through experiments that this theoretical framework provides a coherent and unified explanation for the direct addition of Euclidean space classifiers after matrix power and logarithmic normalization.", "questions": "Please see paper weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper provides a different explanation on why matrix power normalization and matrix logarithmic normalization can be followed by a Euclidean space classifier.\n\nThis paper is well written.", "weaknesses": "1.\tThis paper provides an explanation based on SPD multinomial logistics regression for why matrix logarithmic normalization and matrix power normalization can be followed by Euclidean space classifiers. However, it seems not very clear why matrix power normalization significantly outperforms matrix logarithmic normalization under this framework. In other words, what are the significant performance differences between the two methods under the explanation framework of SPD MLR?\n\n2.\tAs shown in Eqn. (15) and Eqn. (16), the authors extend matrix power normalization and matrix logarithmic normalization to SPD MLR. However, it seems not very clear why matrix power normalization and matrix logarithmic normalization can be regarded as Riemannian classifiers based on Eqn. (15) and Eqn. (16).\n\n3.\tAs shown in Table 4, there only exists a difference of constant term between the formula of ScalePow-EMLR and one of Pow-TMLR. But, as compared in Tables 3, 5, and 6, there is a significant performance gap between ScalePow-EMLR and Pow-TMLR, where ScalePow-EMLR and Pow-EMLR are equivalent. For example, on the Cars dataset in Table 5, the Top-1 accuracy of ScalePow-EMLR is 80.31%, while that of Pow-TMLR is only 51.14%. I feel a bit confused why a constant term can bring significant performance gap? Could the authors give more explanation on this phenomenon?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)