Matrix Manifold Neural Networks++
Authors: Xuan Son Nguyen
Shuo Yang
Aymeric Histace
Venue: ICLR 2024 poster (2024)
Keywords: manifold learning
representation learning
gyrovector spaces
deep learning
Primary area: representation learning for computer vision, audio, language, and other modalities

Abstract: Deep neural networks (DNNs) on Riemannian manifolds have garnered increasing interest in various applied areas. For instance, DNNs on spherical and hyperbolic manifolds have been designed to solve a wide range of computer vision and nature language processing tasks. One of the key factors that contribute to the success of these networks is that spherical and hyperbolic manifolds have the rich algebraic structures of gyrogroups and gyrovector spaces. This enables principled and effective generalizations of the most successful DNNs to these manifolds. Recently, some works have shown that many concepts in the theory of gyrogroups and gyrovector spaces can also be generalized to matrix manifolds such as Symmetric Positive Definite (SPD) and Grassmann manifolds. As a result, some building blocks for SPD and Grassmann neural networks, e.g., isometric models and multinomial logistic regression (MLR) can be derived in a way that is fully analogous to their spherical and hyperbolic counterparts. Building upon these works, in this paper, we design fully-connected (FC) and convolutional layers for SPD neural networks. We also develop MLR on Symmetric Positive Semi-definite (SPSD) manifolds, and propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective. We demonstrate the effectiveness of the proposed approach in the human action recognition and node classification tasks.

Published as a conference paper at ICLR 2024
MATRIX MANIFOLD NEURAL NETWORKS++
Xuan Son Nguyen, Shuo Yang, Aymeric Histace ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (xuan-son. nguyen, shuo. yang, aymeric htteericcccsstaceeeeeeeeeee fr
ABSTRACT
Deep neural networks (DNNss) on Riemannian manifolds have garnered increasing interest various applied areas. For instance, DNNs on spherical and hyperbolic manifolds have been designed to solve wide range of computer vision and nature language processing tasks. One of the key factors that contribute to the success of these networks is that spherical and hyperbolic manifolds have the rich algebraic structures of gyrogroups and gyrovector spaces. This enables principled and ef- fective generalizations of the most successful DNNs to these manifolds. Recently, some works have shown that many concepts in the theory of gyrogroups and gy- rovector spaces can also generalized to matrix manifolds such as Symmetric Positive Definite (SPD) and Grassmann manifolds. As a result, some building blocks for SPD and Grassmann neural networks, e.g., isometric models and multi- nomial logistic regression (MLR) can be derived in way that is fully analogous to their spherical and hyperbolic counterparts. Building upon these works, we design fully-connected (FC) and convolutional layers for SPD neural networks. We also develop MLR on Symmetric Positive Semi-definite (SPSD) manifolds, and propose method for performing backpropagation with the Grassmann loga- rithmic map in the projector perspective. We demonstrate the effectiveness of the proposed approach in the human action recognition and node classification tasks.
INTRODUCTION
In recent years, deep neural networks on Riemannian manifolds have achieved impressive perfor- mance in many applications (Ganea et al., 2018 [Skopek et al. 2020) Cruceru et al. [022]] [Shiiizu al., 2021). The most popular neural networks in this family operate hyperbolic spaces. Such spaces of constant sectional curvature, like spherical spaces, have the rich algebraic structure of gyrovector spaces. The theory of gyrovector spaces (Ungar 2002 2005 2014) offers elegant and powerful framework based on which natural generalizations (Ganea et al. 2018 (Simizuu et al.) 2 of essential building blocks in DNNs are constructed for hyperbolic neural networks (HNNs).
Matrix manifolds such as SPD and Grassmann manifolds offer a convenient trade-off between struc- tural richness and computational tractability (Cruceru et al.) 2021 López et al. (021). Therefore, in many applications, neural networks on matrix manifolds are attractive alternatives to their hyperbolic
counterparts. However, unlike the approaches in Ganea al. (2018) Shimizu al. (2021). ) most
existing approaches for building SPD and Grassmann neural networks (Dong et al.J 2017 Huang
& Gool, 2017; [Huang et al., 2018, Nguyen al. 2019 Brooks al. 2019: Nguyen 2021 Wang
et al. (021)) do not provide necessary techniques and mathematical tools to generalize broad class of DNNs to the considered manifolds.
Recently, the authors of Kim (2020); Nguyen 2022b) have shown that SPD and Grassmann man- ifolds have the structure gyrovector spaces or that of nonreductive gyrovector spaces (N Nguyen 2022b) that share remarkable analogies with gyrovector spaces. The work in/ggn & Yang (023) takes one step forward in that direction by generalizing several notions in gyrovector spaces, e.g., the inner product and gyrodistance (Ungar 2014) to SPD and Grassmann manifolds. This allows one to characterize certain gyroisometries of these manifolds and to construct MLR on SPD manifolds.
Although some useful notions in gyrovector spaces have been generalized to SPD and Grassmann manifolds (Nguyen [022a[b] Nguyen & Yang 2023) that set the stage for an effective way of building neural networks on these manifolds, many questions remain open. In this paper, we aim

---

Published as a conference paper at ICLR 2024
addressing some limitations of existing works using gyrovector space approach. Our contributions can be summarized as follows:
1. We generalize FC and convolutional layers to the SPD manifold setting.
2. We propose a method for performing backpropagation with the Grassmann logarithmic map in the projector perspective (Bendokat et al., 2020) without resorting to any approxi- mation schemes. We then show how to construct graph convolutional networks (GCNs) on Grassmann manifolds.
3. We develop MLR on SPSD manifolds.
4. We showcase our approach in the human action recognition and node classification tasks.
PRELIMINARIES
2.1
SPD MANIFOLDS
The space of n X SPD matrices, when provided with some geometric structures like Riemannian metric, forms SPD manifold Sym; [Arsigny et al. 2005). Data lying on SPD manifolds are com- monly encountered in various domains (Huang & Gool 2017: Brooks et al., 2019 Nguyen 2021 Sukthanker et al., 2021: I Nguyen 2022b Nguyen & Yang 2023). In many applications, the use of Euclidean calculus on SPD manifolds often leads to unsatisfactory results (Arsigny et al. 2005). To tackle this issue, many Riemannian structures for SPD manifolds have been introduced. In this work, we focus on two widely used Riemannian metries, i.c., Affine-Invarian (AI) (Pennec et al., 2004) and log-Euclidean (LE) (Arsigny et al. 2005) metrics, and recently introduced Riemannian met- rics, i.c. Sho--Chlesky (LC) metries (Lin) 2019) that offer some advantages over and Log- Log-Euclidean metrics.
2.2 GRASSMANN MANIFOLDS
Grassmann manifolds Gra.p are the collection of linear subspaces of fixed dimension p of the Fu- clidean space R" (Edelman et al. 1998). Data lying Grassmann manifolds arise naturally in many applications (Absil et al. 2007 Bendokat et al. 2020). Points on Grassmann manifolds can be rep- resented from different perspectives (Bendokat et al. (020). Two typical approaches use projection matrices or those with orthonormal columns. Each of them can be effective some problems but might be inappropriate in some other contexts (Nguyen 2022b). Although geometrical descriptions of Grassmann manifolds have been given in numerous works (Edelman et al., 1998), some com- putational issues remain to be addressed. For instance, the question of how to effectively perform backpropagation with the Grassmann logarithmic map in the projector perspective remains open.
2.3
NEURAL NETWORKS ON SPD AND GRASSMANN MANIFOLDS
2.3.1
NEURAL NETWORKS ON SPD MANIFOLDS
The work in Huang & Gool (017 introduces SPDNet with three novel layers, i.e., Bimap, Log and ReEig layers that has become one of the most successful architectures in the field. In Brooks jet al. (2019), the authors further improve SPDNet by developing Riemannian versions of batch nor-
malization layers. Following these works, some works (Nguyen al. 2019 Nguyen 2021 Wang et al. 2021 Kobler et al., [0022] Ju & Guan 2023) design variants of Bimap and batch normalization layers in SPD neural networks. The work in Chakraborty et al. (2020) presents different approach based on intrinsic operations on SPD manifolds. Their proposed layers have nice theoretical prop- erties. common limitation of the above works is that they do not provide necessary mathematical tools for constructing many essential building blocks of DNNs on SPD manifolds. Recently, some works (Nguyen) [022a|b) Nguyen & Yangl 2023) take gyrovector space approach that enables natural generalizations of some building blocks of DNNs, e.g., MLR for SPD neural networks.
2.3.2 NEURAL NETWORKS ON GRASSMANN MANIFOLDS
In Huang et al. (2018), the authors propose GrNet that explores the same rule of matrix backprop- agation (lonescu et al. 2015) as SPDNet. Some existing works (Wang & Wu, 2020: Souza al.
2

---

Published as a conference paper at ICLR 2024
2020) are also inspired by GrNet. Like their SPD counterparts, most existing Grassmann neural networks are not built upon a mathematical framework that allows one to generalize a broad class of DNNs to Grassmann manifolds. Using gyrovector space approach, Nguyen & Yang (2023) has shown that some concepts in Euclidean spaces can be naturally extended to Grassmann manifolds.
3 PROPOSED APPROACH
3.1 NOTATION
Let M be homogeneous Riemannian manifold, TpM be the tangent space of . at P E M. Denote by exp(P) and (Pg(P) the usual matrix exponential and logarithm of P, (X))(W) the exponential map at P that associates to tangent vector W TpM point of M, Logp (Q) the logarithmic map of Q E M P, (P) the parallel transport of W from P to Q along geodesics connecting P and Q. For simplicity of exposition, Denote by Mp.m the space n x m matrices, Sym, the space of n SPD matrices, Sym, the space of n X n symmetric matrices, St.p the space of n x n SPSD matrices of rank S n, Gra.p the
will concentrate
real matrices.
p-dimensional subspaces of R in the projector perspective. For clarity of presentation, let Grn,p be the p-dimensional subspaces of I in the ONB (orthonormal basis) perspective (Bendokat al.) 2020). For notations related to SPD manifolds, we use the letter g E {aú, le, lej as a subscript (superscript) to indicate the considered Riemannian metric, unless otherwise stated. Other notations will be introduced in appropriate paragraphs. Our notations are summarized in Appendix
3.2 NEURAL NETWORKS ON SPD MANIFOLDS
In Nguyen 2022a b). the author has shown that SPD manifolds with Affine-Invariant Log- AI, LE, and L.C gy-
Fuclidean, and Log- Log-Cholesky metrics form gyrovector spaces referred rovector spaces, respectively. We adopt the notations in these works and consider the case where r (see Nguyen (20226), Definition 3.1). Let Bois Ste. and Ste be the binary operations in AI, LE, and LC gyrovector spaces, respectively. Let Oai: Gte, and ete be the inverse operations in AI, LE, and LC gyrovector spaces, respectively. These operations are given Appendix G
3.2.1 FC LAYERS SPD NEURAL NETWORKS
Our method for generalizing FC layers to the SPD manifold setting relies on a reformulation SPD hypergyroplanes (Nguyen & Yang 2023). We first recap the definition of SPD hypergyroplanes.
Definition 3.1 (SPD Hypergyroplanes (Nguyen & Yang 2023)). For P -Sm W
Tp Sym SPD hypergyroplanes are defined as
Hawd {Q E Sym (Logg (Q), w = 0],
where (.. denotes the inner product at P given by the considered Riemannian metric.
Proposition 3.2 gives an equivalent definition for SPD hypergyroplanes.
Proposition 3.2. Let P Sym ti W Tp Sym +.9 and H..P be the SPD hypergyroplanes
defined in Definition 3.7 Then
H... {Q E Sym (e,P P Bg Q Expi, (T (W)))* =),
where I, denotes the n x n identity matrix, and ., is the SPD inner product in Sym Nguuen & Yang 2023) (see Appendix for the definition of the SPD inner product).
Proof See Appendix 0
In DNNs, an FC layer linearly transforms the input in such a way that the k-th dimension of the output corresponds to the signed distance from the output to the hyperplane that contains the origin and is orthonormal to the k-th axis the output space. This interpretation has proven useful in generalizing FC layers the hyperbolic setting (Shimizu et al. 2021
3

---

Published as a conference paper at ICLR 2024
Notice that the equation of SPD hypergyroplanes in Proposition
3.2 has the form (e,P Bg Q, W O, where W E Sym, This equation can be seen
generalization of the hy- perplane equation (w, x) + bb= + 2, w) = O, where w, x. E IR, : b E R, and (p, w) = -b. Therefore, Proposition 3.2 suggests that any linear function of an SPD matrix X E Sym, t- can be written as (e,p Gg X, W)', where P, W E Sym The above interpretation of FC layers now can be applied our for constructing FC layers in SPD neural networks. For convenience of presentation, in Definition 3.3 we will index the dimensions (axes) of the output space using two subscripts corresponding the row and column indices in a matrix.
Definition 3.3. Let E ((a)) S j, = I,..., m be the (i.j)-th -tth axis of the output space. An SPD hypergyroplane that contains the origin and is orthonormal to the E((,)) axis can be defined as
Hapda Logi im (t.g)) ).1m = (Q Sym + (Q, ((,))) =0}
remains to specify an orthonormal basis for each family of the considered Riemannian metrics of SPD manifolds. Proposition 3.4 gives such an orthonormal basis for Al gyrovector spaces along metrics.
with the expression for the output of FC layers with
Proposition 3.4 (FC layers with Affine-Invariant Metries). Let (ci,..., cm), ||c = 1,
1..., m be an orthonormal basis of IR". Let (, voi be the Affine-Invarinn metric computed at t.at P E t
VV, WT Tr(VP- WP (T((VP Tr (T(PP-
i S j,ú,j = I...., of Symn t.as can be given as where B > 1 An orthonormal basis E (1,))
exp
eeee
1 (1
(1 ifi=j
Egi (35)
e,e; te,e
exp
j
V2
Denote by v(. (X) ((.)) Bas X, W.W.is (1.))) P(.)) W (i.) E Sym i S j,ú,j 1
1,...,m. m. Let a m (V1 m3 - 1). Then the output of an FC layer is computed as Y exp (Iy(.))))))) where ((j.)Kyj= is the matrix having V(i.)) as the element at the i-th row and j-th column, and V(s.2) given by
(X) + Em= (X),)(
ii = ifi <. j ifi
(i,)) (X),
(X))
Proof See Appendix
+.le can be obtained from any As shown in Arsigny et al. 2005). Log-Euclideds metric on Sym inner product on Sym,, this work, we consider a metric that invariant under all similarity transformations, i.e., the metric (W, V) Tr(WV). We have the following result.
Proposition 3.5 (FC layers with -uc-Eunli Metrics). An orthonormal basis Ele ((g) S
j,i, I..... of Symmt tele can be given by
exp
e c.e teje? v2
ifi=j
((,))
exp
<j
Let U(a,)) ((.) Bie X, ((.))) P(..)) W(s)) Sym t. S j. 1,...,
Then the output of an FC layer is computed as =exp (1)6,))))))))) where M6.1 is given by
(X).
ifi=
Ve (X))
i
((.)) =
(X), ifi
4

---

Published as a conference paper at ICLR 2024
Proof See Appendix K
Finally, we give the characterization of an orthonormal basis for LC gyrovector spaces and the expression for the output of FC layers with Log-Cholesky metrics.
Proposition 3.6 (FC layers with Log-Cholesky Metrics), An orthonormal basis Eoyy S j,ú,j -1,,,,,. ; m of Sym tle can be given by
(e-1))e;e e) +Im B le Im (eze Im), ifi <
i
Let ((.)) ( (i.)) Dic X, W (6n) P(s) W (i.)) E Sym t.c ; S j,i,j = 1,...,
Then the output of an FC layer is computed as Y YY where Y = ((r.))/= and V(i.)) is given by
(xp()(
ifi =
9(3))
((1))())
ifi <
0,
ifi>
Proof
See Appendix
3.2.2 CONVOLUTIIONAL LAYERS SPD NEURAL NETWORKS
Consider applying a 2D convolutional layer to a multi-channel image. Let Non and Nout be the numbers of imput and output channels, respectively. Denote by f(i.)) ,,...,,................ NNow
1,..., Neat, I,..., Nout the value of the k-th output channel at pixel (i,j). Then
Nin (L.).
(1.)) + 6*,
(I)
where xx.)) is receptive field of the 1-th input channel, w(..) is the filter associated with the I-th input channel and the k-th output channel, and 6 is the bias for the k-th output channel. W = concat(w(L.), ..... (Nnn.))), where operation concat(.) concatenates all of its arguments. Then Eq. {1 can be rewritten (Shimizu et al. 2021
Let ((.)) = concal((x) (1.)) -.,,) (6.)))
y (W & (1.))) +
Note that Eq. 2 has the form (w, x) + b and thus the computations discussed in Section 3.2.1 can be applied to implement convolutional layers in SPD neural networks. Specifically, given set of SPD matrices P. E Sym +-9 = 1, I,...,N, operation concat apd (P,,..., produces a block diagonal matrix having P, as diagonal elements.
In Chakraborty et al. 2020), the authors design a convolution operation for SPD neural networks. However, their method is based on the concept of weighted Fréchet Mean, while is built upon the concepts of SPD hypergyroplane and SPD neeudo-gyrodistane from an SPD matrix to an SPD hypergyroplane (Nguyen & Yang 2023). Also, convolution operation be used for dimen- sionality reduction, while theirs always produces an output of the same dimension as the inputs.
3.3 MLR IN STRUCTURE SPACES
Motivated by the works in Nguyen 2022a): Nguyen Yang (0023), in this section, we aim to build MIR on SPSD manifolds. For any P n,p= we consider the decomposition P = UP.
where Up E Gra.p and Sp E Sym Each element of S can be seen as a flat p-dimensional ellipsoid in _ (Bonnabel al. 2013). The flat ellipsoid belongs to p-dimensional subspace spanned by the columns of Ur, while the p X SPD matrix Sp defines the shape of the ellipsoid in Sym A canonical representation of P in structure space Gra. x Sym is computed by iden- tifying a common subspace and then rotating Up to this subspace. The SPD matrix Sp is rotated accordingly reflect the changes of Up. Details of these computations given in Appendix H
5

---

Published as a conference paper at ICLR 2024
Assuming that canonical representation in structure space GIn.p p x Sym is obtained for each point in p we now discuss how to build MLR in this space. As one of the first steps for developing network building blocks in gyrovector space approach is to construct some basic operations in the considered manifold, we give the definitions of the binary and inverse operations the following.
Definition 3.7 (The Binary Operation in Structure Spaces). Let (Up,Sp), (U), E Gra.p X
Sym +-0 Then the binary operation @psd.g in structure space Grn.p x Sym t9 is defined as (Up,Sp) @psd.g (Uq, Sq) @Up@gr Uq. Sp By SQ).
where Bgr is the binary operation in Grn.p (see Appendix G.6 for the definition of Bgr). Definition 3.8 (The Inverse Operation in Structure Spaces). Let (Ur,Sr) Grn.p Sym
Then the inverse operation Opsd.g in structure space Grn.p Sym is defined as
Opsd.g (U Sr) Ur
where Sgr is the inverse operation in Gra.p (see Appendix for the definition of Egr).
Our construction of the binary and inverse operations GIa.p Sym t-0 is clearly advantageous compared to the method in Nguyen (2022a) since this method does not preserve the information about the subspaces of the terms involved in these operations. In addition to the binary and inverse operations, also need to define the inner product in structure spaces.
Definition 3.9 (The Inner Product in Structure Spaces). Let (Up.Sp), (UQ.))) E Gra.p X Sym;. Then the inner product in structure space GIn.p Sym is defined as
p, Sp), Sq)) AU U UQU + SS,,
where À > O, .)r gr is the Grassmann inner product Nguyen & Yang 2023) (see Appendix G.7
The key idea to generalize MLR Riemannian manifold is to change the margin to reflect the geometry of the considered manifold (a formulation of MLR from the perspective of distances to hyperplanes is given in Appendix C). This requires the notions of hyperplanes and margin in the considered manifold that are referred to as hypergyroplanes and geeudo-gyrodistanee (Nguyen & Yang, 2023). respectively. In our case, the definition of hypergyroplanes in structure spaces, sug- gested by Proposition 3.2) can be given below.
Definition 3.10 (Hypergyroplanes in Structure Spaces). Let P, W E Grap x Sym, 9. Then hypergyroplanes in structure space Gra,p X Sym are defined
Hpsd.g = {Q E Gra.p x Sym (Opsd. Pppd. Q Wyrsd.g = 03.
Pseudo-gyrodistances in structure spaces can be defined in the same way as SPD pseudo- gyrodistances. We refer the reader to Appendix G.8 for all related notions. Theorem 3.1] gives an expression for the gruudo-yyrodittane from point to hypergyroplane in a structure space.
Theorem 3.11 sreuuo--yrodisttnces in Structure Spaces). Let W = -Uw Sw), P (Up.Sp), X SX) e GIn. Symf. and Hoad. be hypergyroplane in structure space GIn,p x Sym Then the pseudo- pseudo-gyrodistance from X. to Horde is given by
d(X, Hewd. I u Ux U Ux)T, Uw U + (e,Sp o Sx Sw)
VX(IIUW U ( S|SSw ||)
where ||r||9r and | are the norms induced by the Grassmann and SPD inner products, respectively.
Proof See Appendix M
The algorithm for computing the pseuuo-gymadistances is given in Appendix B

---

Published as a conference paper at ICLR 2024
3.4 NEURAL NETWORKS ON GRASSMANN MANIFOLDS
In this section, we present method for computing the Grassmann logarithmic map in the projector perspective. We then propose GCNs Grassmann manifolds.
3.4.1 GRASSMANN LOGARITHMIC MAP IN THE PROJECTOR PERSPECTIVE
The Grassmann logarithmic map is given (Batzies et al.
2015
Bendokat et al. 2020) by
Logp gr (Q) [),
where P. Q E GIn.p. and is computed as
log (II, - -Q)= -2))).
Notice that the matrix (I, (I) 2P) is generally not an SPD matrix. This raises an issue when one needs implement operation that requires the Grassmann logarithmic map in the projector perspective using popular deep learning frameworks like Py PyTorch and Tensorflow, since the matrix logarithm function is not differentiable in these frameworks. To deal with this issue, we rely on the following result that allows us to compute the Grassmann logarithmic map in the projector perspective from the Grassmann logarithmic map the ONB perspective.
Proposition 3.12. Let T be the mapping such that
T Grn.p > Gra.p: U UUT.
gr Let Logu (V), U. Gra.p be the logarithmic map of V U the ONB perspective. Then Loge (Q) = - (P)
(Log, (Log (p) (p Logr or
(p) (p) (T (Q))
Proof See Appendix N
gr Note that the Grassmann logarithmic map Logu (V) can be computed via singular value decom-
position (SVD) that is a differentiable operation in Py Torch and Tensorflow (see Appendix [E2.2]. Therefore, Proposition 3.12 provides effective implementation of the Grassmann logarithmic map in the projector perspective for gradicnt- brased learning.
3.4.2 GRAPH CONVOLUTIONAL NETWORKS ON GRASSMANN MANIFOLDS
We propose extend GCNs Grassmann geometry using approach similar Chami et al. 2019 Zhao et al. 2023). Let G = (V,E) ) be graph with vertex set V and edge set غ, x i V be the embedding of node at layer (I 0 indicates input node features), Niii) {j (i,7) E 8) be the set of neighbors of E V. W and b' be the weight and bias for layer L, and a (.) be non-linear activation function. basic GCN messagg-passing update (Zhao et al. 2023) can be expressed as
p W
(feature transformation)
q
Σ
W&P
(aggregation)
JEN(1)
x =[q +b²)
(bias and nonlinearity)
For the aggregation operation, the weights wiy can be computed using different methods (Kipf & Welling 2017 Hamilton et al. 2017). Let X E Grn,,, E V be the Grassmann embedding of node e at layer For feature transtormation on Grassmann manifolds, we use isometry maps based on left Grassmann gyrotranslations (Nguyen & Yang 2023). i.e.,
Φm x exp(LLog]" (M), In,p)) exp(- exp(- Logii" Inpp))
where In.p exponential map in Gra.p- Then the aggregation process is performed as
Mn... and M GIn.p is model parameter. Let Exp"" be the
Q Expt"
or
Σ
ka Logar f (P')):
JEN(()

---

Published as a conference paper at ICLR 2024
Thes Ging
MLR Bayer
Egp map
Yd Mww.p.lli Ji
CAA 100 DU 201
exp
Convolational layorr
msc
(M St
Iiput sequarce
Intilitctioon Feature transtormation
Aggregation
Figure
The pipelines of GyroSpd++ (left) and Gr-GCN+ (right).
where p and Q are the input and output node features of the aggregation operation, and kijj IW(O)) ( - represents the relative importance of node j node i. For any GIapp
let exp([Logf",, (X), In VU be QR decomposition, where In.p I
E Mn
0
V E Ma.p is matrix with orthonormal columns, and U Mp.y is upper- trger-triangular matrix.
Then the non-linear activation function (Nair Hinton 2010 Huang al. 2018) is given by
(X) = VVT.
Let B' E Gra,p be the bias for layer Then the message-passing update of our network can be summarized as
P dm (X! 1
(feature transformation)
Q =
Expr or
Σ
ki,j Logi or (P'))
(aggregation)
JEN(()
X (B' Bgr
Q!)
(bias and nonlinearity)
The Grassmann logarithmic maps in the aggregation operation are obtained using Proposition 3.12
Another approach for embedding graphs Grassmann manifolds has also been proposed in Zhou jct al. (2022). However, unlike our method, this method creates Grassmann representation for graph via SVD of the matrix formed from node embeddings previously learned by Euclidean neural network. Therefore, it not designed to learn node embeddings on Grassmann manifolds.
EXPERIMENTS
4.1 HUMAN ACTION RECOGNITION
We three datasets, i.e., HDMO5 (Müller et 2007). FPHA (Gercia-
2018 and
NTU RBG+D 60 (NTU60) (Shahroudy et al. (016)) We compare our networks against the following
state-of-the the- models: SPDNet (Huang Gool 2017 SPDNetBN (Brooks al. 2019 SPSD-
AI N Nguyen 2022a GyroAl-HAUNet (Nguyen 2022b ), and MLR-AI (Nguyen & Yang 2023
4.1.1 ABLATION STUDY
Convolutional layers in SPD neural networks Our network GyroSpd++ has a MLR layer
stacked top convolutional layer (see Fig. D. The motivation for using a convolutional layer
https://gi wwt.:/w.........uu........oo htwp://////..iiaaaaaaa//////////
httpss:/paape/. ni ps. ea/
wttppss..pp......p...ppppppppppp pc/pappr/201 9/hash wwwth//eeeebbaaeeeeeeeee
http://ee/.o..aaaaooaoaoaaoaoaaa 63
Abstract
html
8

---

Published as a conference paper at ICLR 2024
Table 1: Results (mean accuracy + standard deviation) and model sizes (MB) of various SPD neural networks on the three datasets (computed over 5 runs).
Method
HDM05
#HDMOS
FPHA
#FPHA
NTU60
#NTU60
SPDNet
71.36 1.49
88.79 + 0.36
0.99
76.14 = 1.43
1.80
SPDNetBN
75.05 1.38
6.68
91.02 + 0.25
1.03
78.35 = 1.34
2.06
GyroAI-HAUNee
77.05
95.65 0.23
93.27 1: 1.29
0.02
SPSD-AI
79.64 1.54
0.31
95.72 1 0.44
0.11
93,92 1 1.58
0.03
MLR-AI
78.26 1.37
0.60
95.70 + 0.26
0.21
94.27 = 1.32
0.05
GyroSpd++ (Ours)
79.78 = 1.42
0.76
96.84 = 0.27
0.27
95.28 # 1.37
0.07
GyroSpsd++ (Ours)
78.52 1.34
0.75
97.90 + 0.24
0.27
96.64 = 1.35
0.07
Results and computation times (seconds) per epoch of Gr- Gr-GCN++ and its variant based on the ONB perspective. Node embeddings learned Gr14,7 and G114,7 for Gr-GCN-ONB and Gr-GCN+,, respectively. Results are computed over 5 runs.
Table 2:
Method
Gr-GCN-ONB
Gr+GCN++
Accuracy 1 standard deviation
81.9 1.2
82.8 0.7
Airport
Training
0.49
0.97
Testing
0.40
0.69
Accuracy + standard deviation
76.2 1,5
80.3 0.5
Pubmed
Training
3,40
6.48
Testing
2.76
4.47
Accuracy + standard deviation
68.1 1.0
81.6
Cora
Training
0.57
0.77
Testing
0,46
0.52
that it can extract global features from local ones (covariance matrices computed from joint coor- dinates within sub-sequences of action sequence). We use Affine-Invarial metrics for the convo- lutional layer and Log-Euclidean metrics for the MLR layer. Results Tab. Tab.[ show that GyroSpd++ consistently outperforms the SPD baselines in terms of mean accuracy. Results of GyroSpd++ with different designs of Riemannian metrics for its layers are given Appendix D.4.I
MLR in structure spaces We build GyroSpsd++ by replacing the MLR layer of GyroSpd++ with a MLR layer proposed in Section 3.3 Results of GyroSpsd++ are given in Tab. Except SPSD- AI, GyroSpsd++ outperforms the other baselines HDM05 dataset in terms of mean accuracy. Furthermore, GyroSpsd++ outperforms GyroSpd++ and all the baselines on FPHA and NTU60 datasets terms of mean accuracy. These results show that MLR is effective when being designed in structure spaces from a gyrovector space perspective.
4.2 NODE CLASSIFICATION
We use three datasets, i.e., Airport (Zhang Chen 2018), Pubmed (Namata et al. 2012a), and Cora (Sen al. 2008). each of them contains single graph with thousands of labeled nodes. We compare our network Gr-GCN+ (see Fig. against its variant Gr-GCN-ON (see Appendix E.2.44 based on the ONB perspective. Results shown in Tab. 2 Both networks give the best performance for n = 14 and p = 7. It can be seen that Gr-GCN++ outperforms Gr-GCN-ONB all cases. The performance gaps are significant Pubmed and Cora datasets.
CONCLUSION
In this paper, we develop FC and convolutional layers for SPD neural networks, and MLR SPSD manifolds. We show how to perform backpropagation with the Grassmann logarithmic map in the projector perspective. Based on this method, we extend GCNs to Grassmann geometry. Finally, we present our experimental results demonstrating the efficacy of our approach in the human action recognition and node classification tasks.

---

Published as a conference paper at ICLR 2024
REFERENCES
Pierre-Antoine Absil, Robert E. Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2007.
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and Simple Computa- tions Tensors with Log-Fuclldean Metrics. Technical Report RR-5584, INRIA, 2005.
E. Batzies, K. Hper, Machado, and F. Silva Leite. Geometric Mean and Geodesic Regression on Grassmannians. Linear Algebra and its Applications, 466:83-101, 2015.
Thomas Bendokat, Ralf Zimmermann, and A. Absil. A Grassmann Manifold Handbook: Basic Geometry and Computational Aspects. CoRR, abs/2011 1. 1.12291..139,,,,, 2020. URL htttps://arxitt
Etg/abs/2011. 13699
Silvère Bonnabel, Anne Collard, and Rodolphe Sepulchre. Rank- Geometric Means of Positive Semi-definite Matrices. Linear Algebra and its Applications, 438:3202-3216, 2013,,
Daniel A. Brooks, Olivier Schwander, Frédéric Barbaresco, Jean-Yves Schneider, and Matthieu Cord. Riemannian Batch Normalization for SPD Neural Networks. In NeurlPS, pp. 15463- 15474, 2019.
Rudrasis Chakraborty, Jose Bouza, Jonathan H. Manton, and Baba C. Vemuri. ManifoldNet: A Deep Neural Network for Manifold-valued Data with Applications. TPAMI, 4400 2020.
Ines Chami, Rex Ying, Christopher R, and Jure Leskovec. Hyperbolic Graph Convolutional Neural Networks. CoRR, abs/1910. 12933, 2019. Utthtts:://arxie
Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Fully Hyperbolic Neural Networks. In ACL, pp. 5672-5686, 2022.
Calin Cruceru, Gary Bécigneul, and Octavian-Eugen Ganea. Computationally Tractable Riemannian Manifolds for Graph Embeddings. AAAI, pp. 7133-7141, 2021.
Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A Hyperholiccto---yeerolli Graph Convolutional Network. In CVPR, pp. 154-163, 2021.
Zhen Dong, Su Jia, Chi Zhang, Mingtao Pei, and Yuwei Wu. Deep Manifold Learning of Symmetric Positive Definite Matrices with Application to Face Recognition. In AAAI, pp. 4009-4015, 2017.
Alan Edelman, Tomás A. Arias, and Steven T. Smith. The Geometry of Algorithms with Orthogo- nality Constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303-335 1998.
Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. In NeurlPS, pp. 5350-5360, 2018.
Guillermo
Shanxin Yuan, Seungryul Back, and Tae Tae-Kyun Kim. First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations. In CVPR, pp- 409-419, 2018
William L. Hamilton, Rex Ying, and Jure Leskovee. Inductive Representation Learning on Large Graphs. In NIPS, pp. 1025-1035, 2017.
Mchrtash Harandi, Mathieu Salzmann, and Richard Hartley. Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Awar Methods. TPAMI, 40:48-62, 2018
Zhiwu Huang and Luc Van Gool. A Riemannian Network for SPD Matrix Learning. In AAAI, pp. 2036-2042, 2017.
Zhiwu Huang, Jiqing Wu, and Luc Van Gool. Building Deep Networks on Grassmann Manifolds. In AAAI, pp. 3279-3286, 2018.
Catalin lonescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix Backpropagation for Deep Networks with Structured Layers. In ICCV, pp. 2965-2973, 2015.
10

---

Published as a conference paper at ICLR 2024
Ce Ju and Cuntai Guan. Graph Neural Networks SPD Manifolds for Motor Imagery Classifica- tion: A Perspective From the Time--Freqeenn Analysis. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2023.
Qiyu Kang, Kai Zhao, Yang Song, Sijie Wang, and Wee Peng Tay. Node Embedding from Neural 2023.
Hamiltonian Orbits in Graph Neural Networks. In ICML, pp.
Sejong Kim. Ordered Gyrovector Spaces. Symmetry, 12(6), 2020.
Thomas N. Kipf and Max Welling, Semi-Supervised Classification with Graph Convolutional Net- 609. 02.007
works. CoRR, abs//600.000 609.02907, 2017.
Reinmar J. Kobler, Jun ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD Domain- specific Batch Normalization to Crack Interpretable Unsupervised Domain Adaptation in EEG. In NeurlPS, pp. 6219-6235, 2022.
Guy Lebanon and John Lafferty. Hyperplane Margin Classifiers on the Multinomial Manifold. In ICML, pp. 66, 2004.
Zhenhua Lin. Riemannian Geometry of Symmetric Positive Definite Matrices via Cholesky Decom- position. SIAM Journal Matrix Analysis and Applications, 40(4):1353-1370 2019.
Qi Liu, Maximilian Nickel, and Douwe Kiela, Hyperbolic Graph Neural Networks. NeurlPS, pp- 8228 000--8239,22 2019.
Ziyu Liu, Hongwen Zhang, Zhenghao Chen, Zhiyong Wang, and Wanli Ouyang. Disentangling and Unifying Graph Convolutions for Baaleton-Basee Action Recognition. In CVPR, pp. 143-152, 2020.
Federico López, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Vector- valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices. In NeurlPS, pp. 18350-18366, 2021.
Meinard Müller, Tido Röder, Michael Clausen, Bernhard Eberhardt, Bjöm Krüger,, and Andreas Weber. Documentation Mocap Database HDM05. Technical Report CG-2007-2, Universität Bonn, June 2007.
Vinod Nair and Geoffrey E. Hinton. Rectified Linear Units Improve Restricted Boltzmann Ma- chines. In ICMI., pp. 807-814, 2010.
Galileo Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven Active Surveying for Collective Classification. In 10th International Workshop Mining and Learning with Graphs, volume 8, pp. 1, 2012a.
Galileo Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven Active Surveying for Collective Classification. In Workshop Mining and Learning with Graphs. 2012b.
Xuan Son Nguyen. GeomNet: Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Ban Interaction Recognition. In ICCV, pp. 13379- 13389, 2021.
Xuan Son Nguyen. A Gyrovector Space Approach for Symmetric Positive Semi-definite Matrix Learning. In ECCV, pp. 52-68, 2022a.
Xuan Son Nguyen. The Syrro-Structure of Some Matrix Manifolds. In NeurlPS, pp. 26618-2666 2022b.
Xuan Son Nguyen and Shuo Yang. Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach. CoRR, 04422305.04660 2023. URL /tttpes//arriiioorr//a/////333
04560
Xuan Son Nguyen, Luc Brun, Olivier Lézoray, and Sébastien Bougleux. A Neural Network Based on SPD Manifold Learning for Skeleton- based Hand Gesture Recognition. In CVPR, pp. 12036- 12045, 2019.
11

---

Published as a conference paper at ICLR 2024
Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian Framework for Tensor Comput- ing. Technical Report R2-5, INRIA, 2004.
Xavier Pennee, Stefan Horst Sommer, and Tom Fletcher. Riemannian Geometric Statistics in Medi- cal Image Analysis. Academic Press, 2020.
Chiara Plizzari, Marco Cannici, and Matteo Matteucci. dkeleton-based Action Recognition via
Spatial and Temporal Transformer Networks. Computer Vision and Image Understanding, 208: 103219, 2021.
Prithviraj Sen, Galileo Mark Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Rliassi-Rad. Collective Classification in Network Data. Al Magazine, 29(3):93-106, 2008.
Amir Shahroudy, Jun Liu, Tian-Tsong Ng. and Gang Wang. NTU RGB+D: Large Scale Dataset for 3D Human Activity Analysis. In CVPR, pp. 1010-1019, 2016.
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Nerworks++. CoRR, abs/2006.08210, 2021. [tthhtts::://// wwwpiiiiorrr/aaseeeeeeee wwt.oorggaba..ooo.. 08210
Ondrej Skopek. Octavian- Ganea, and Gary Bécigneul, Mixed-curvature Variational Autoen- coders. CoRR, abs/1911.08411 2020. URL /tt.tttts//////...../////////// 1atp///991..01111111
Lincon S. Souza, Naoya Sogi, Bernardo B. Gatto, Takumi Kobayashi, and Kazuhiro Fukui. An Interface between Grassmann Manifolds and Vector Spaces. CVPRW, pp. 3695-3704, 2020.
Rhea Sanjay Sukthanker, Zhiwu Huang, Suryansh Kumar, Erik Goron Endsjo. Yan Wu, and Luc Van Gool. Neural Architecture Search of SPD Manifold Networks. In IJCAI, pp. 3002-3009, 2021.
Abraham Albert Ungar. Beyond the Einstein Addition Law and Its Gyroscopic Thomas Precession: The Theory of Gyrogroups and Gyrovector Spaces. Fundamental Theories of Physics, vol. 117, Springer, Netherlands, 2002.
Abraham Albert Ungar. Analytic Hyperbolic Geometry: Mathematical Foundations and Applica-
fions. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2005.
Abraham Albert Ungar. Analytic Hyperbolic Geometry N Dimensions: An Introduction. CRC
Press, 2014.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li, and Yoshua Bengio. Graph Attention Networks. CoRR, Ni111100.1000,, 2018. URL https://arxivv
org/abs/1710. 10903
Rui Wang and Xiao-Jun Wu. GrasNet: A Simple Grassmannian Network for Image Set Classifica- tion. Neural Processing Letters, 52(1):693771, 2020.
Rui Wang, Xiao-Jun Wu, and Josef Kittler. SymNet: A Simple Symmetric Positive Definite Mani- fold Deep Learning Method for Image Set Classification. IEEE Transactions on Neural Nenworks and Learning Systems, pp. 1-15, 2021.
Muhan Zhang and Yixin Chen. Link Prediction Based on Graph Neural Networks. abs/1 hbss/802..0691. 2018. URL wttttt:ss//tetttteeeeee
CoRR,
Yiding Zhang, Xiao Wang, Chuan Shi, Nian Liu, and Guojie Song. Lorentzian Graph Convolutional Networks. In Proceedings of the Web Conference 2021, pp. 1249-1261, 2021.
Yiding Zhang, Xiao Wang, Chuan Shi, Xunqiang Jiang, and Yanfang Ye. Hyperbolic Graph Atten- tion Network. IEEE Transactions Big Data, 1690-1191 2022.
Wei Zhao, Federico Lopez, J. Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, and Steve Trettel. Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices. CoRR, abs/2306. 02./2306.10644, 2023. URL
14064
Bingxin Zhou, Xuebin Zheng, Yu Guang Wang, Ming Li, and Junbin Gao. Embedding Graphs on Grassmann Manifold. Neural Newworks, 152:322-331, 2022.
Huanyu Zhou, Qingjie Liu, and Yunhong Wang. Learning Discriminative Representations for Skele- ton Based Action Recognition. In CVPR, pp. 10608-10617, 2023.
12

---

Published as a conference paper at ICLR 2024
Symbol
Name
Mn.
Space x m matrices
Sym,
Space of n x SPD matrices
Sym
Space of X SPD matrices with Al geometry
Sym,
Space n X n symmetric matrices
Grog
Grassmannian in the projector perspective
Gra.p
Grassmannian in the ONB perspective
Ship
Space of n x SPSD matrices of rank S n
M
Matrix manifold
TpM
Tangent space of JM at P
exp(P)
Matrix exponential of
log(P)
Matrix logarithm of P
Expp (W)
Exponential map of W P in Sym F.ad
Logp QQ)
Logarithmic map of Q at in Sym tiai
P W
Parallel transport of W from P to Q in Sym
Expp W
Exponential map of W P in GIa.
Logp gr
[Q)
Logarithmic map of Q at in Grap
Logp gr
(Q)
Logarithmic map of Q P in Grn.
Oai Oas
Binary and inverse operations in Sym, t. Binary and inverse operations in GIn.p Binary and inverse operations in Grypp Binary operation in Gra,, X Sym .a Inverse operation in Gru,p x &ym t.i Hypergyroplane Sym
Byr Sgr
Dgr Sgr
ppddat
ppddaa
updlas
Hw..P
Hasd
Hypergyroplane GIn.p x Sym
Eai (i.))
Orthonormal basis of Sym
(n yai
Inner product Sym
Inner product in Grap Inner product in Grap Sym Affine-Ivarriad metric at P n x identity matrix
ped,ai
( .
I,
Ip
In.
E Mn.n
0
I
In.p
Mn,
0
Table 3: The main notations used in the paper. For the notations related to SPD manifolds, only those associated with Affine-Invariat geometry are shown.
A NOTATIONS
Tab. presents the main notations used in our paper.
B MLR IN STRUCTURE SPACES
Algorithm summarizes all steps for the computation of pseudo-gyroddisaacees in Theorem 3.11
Details of some steps given below:
13

---

Published as a conference paper at ICLR 2024
Algorithm 1: Computation of Pseudo-gy sruudd-gyrodistances
Input: A batch of SPSD matrices X, E S 1,...,.. N
The number of classes C
The parameters for each class Up, Uw Gra.p Sp, Siv E Sym e I.....CC
A constant E [0,
Output: An array E MN.c
U < In.pi
(U, Vi).= +- (X)((X)))EL....)) (U.)E=I..... f (U.(;, pl).=.....N
/x X, = = */
if training then
U t GrMcan (U, )) U GrGoodesic( UTh U,?);
end
(U X x) GrCamonicalize((XX Ui), U"); for t to C do
([()))=l.....d Ux Ux UY (
10
(U (1 18)2
end
sVD((X)))=).....)) performs singular value decompositions for batch of matrices. GrMeen computes the Fréchet mean of its arguments. GrGeodesic(U" U,1) computes point on geodesic from U to U at step (7 = 0.1 in our experiments).
GrCanoniealize((XX (X,, U.) V) computes the canonical representations of X,, 1,..., N using V as a common subspace (see Appendix H
Line 10: Ux (U'x)==,..... Sx (S I......... and the computation of pseudo- gyrodistances is performed in batches.
C FORMULATION OF MLR FROM THE PERSPECTIVE OF DISTANCES TO HYPERPLANES
Given classes, MLR computes the probability of each the output classes as
explwe x + by) ply k|x) E exp(w] x b:)
exp(wi x bk).
(3)
where a is an input sample, by E RR, x. W/ RR, 1.. K. As shown in Lebanon & Lafferty (004). Eq. can be rewritten
p(y = k (r) x exp(sign( [w x + by.) bus/l( Hon. where d(x, b. is the distance from point
hyperplane Har by defined
Horb = (a E IR (w, x) + b= 0),
where R" \ (0), and E R.
D HUMAN ACTION RECOGNITION
D.1 DATASETS
HDM05 Müller et al. 2007) It has 2337 sequences of 3D skeleton data classified into 130 classes. Each frame contains the 3D coordinates of 31 body joints. We use all the action classes and follow the experimental protocol in Harandi et al. (2018) in which subjects are used for training and the remaining 3 subjects are used for testing.
14

---

Published as a conference paper at ICLR 2024
FPHA (Garcia-Hernando et al., 2018) It has 1175 sequences of 3D skeleton data classified into 45 classes. Each frame contains the 3D coordinates of hand joints. We follow the experimental protocol in Garcia- sequences are used for testing.
al. (2018) in which 600 sequences are used for training and 575
NTU60 (Shahroudy et al., 2016) It has 56880 sequences of 3D skeleton data classified into 60 classes. Each frame contains the 3D coordinates of 25 or 50 body joints. We use the mutual actions and follow the cross-subject experimental protocol in Shahroudy et al. (206) which data from 20 subjects are used for training, and those from the other 20 subjects are used for testing.
D.2 IMPLEMENTATION DETAILS
D.2.1 SETUP
We use the PyTorch framework to implement our networks and those from previous works. These networks are trained using cross-entropy loss and Adadelta optimizer for 2000 epochs. The learning rate is set to 10-3, The factors 3 (see Proposition 3.4) and À (see Definition 3.9 are set to 0 and 1, respectively. For GyroSpd++, the sizes of output matrices the convolutional layer are set to 34 34, 21 x 21, and 11 11 for the experiments respectively. For GyroSpsd++, the sizes of SPD matrices in structure spaces are set to 20 x 20, 14 14, and 8 for the experiments HDM05, FPHA, and NTU60 datasets, respectively. We use a batch size of 32 for HDM05 and FPHA datasets, and a batch size of 256 for NTU60 dataset.
HDM05, FPHA, and NTU60 datasets,
D.2.2 INPUT DATA
We use a similar method as in Nguyen (2022b) to compute the input data for our network Gy- roSpd++. We first identify closest left (right) neighbor of every joint based their distance to the hip (wrist) joint, and then combine the 3D coordinates of each joint and those of its left (right) neighbor to create feature vector for the joint. For given frame variance matrix E, are computed from the set of feature vectors of the frame and then combined to create an SPD matrix as
vector μ4, and
Σ, + Me (1e)))
(1))
The lower part of matrix log(Y Y.) ) is flattened obtain vector v, All vectors v, within a time window [t, c- 1], where is determined from temporal pyramid representation of the sequence (the number of temporal pyramids set to 2 in our experiments), are used compute a covariance matrix as
1 +
Z,
Σ (v. v.) v.)
(4)
where V, Eltc-1 v.. For GymoAI-HAUee,, SPSD-AL, MIR-AI, GyroSpd++, and Gy- roSpsd++, the input data are the set of matrices obtained in Eq. (
For SPDNet and SPDNetBN, each sequence is represented by covariance matrix (Huang & Gool 2017 Brooks et al. 2019). The sizes of the covariance matrices are 93 x 93, 60 60, and 150 x 150 for HDMO5, FPHA, and NTU60 datasets, respectively. For SPDNet, the same architecture as the one in Huang & Gool (201) is used with three Bimap layers. For SPDNetBN, the same architecture the one in Brooks et al. (2019) is used with three Bimap layers. The sizes of the transformation matrices for the experiments on HDM05, FPHA, and NTU60 datasets are set to 93 x 93, 60 x 60, and 150 150, respectively.
D.2.3 CONVOLUTIONAL LAYERS
In order to reduce the number of parameters and the computational cost for the convolutional layer in GyroSpd++, we assume a diagonal structure for the parameter P (i.j) (see Propositions 3.4 3.5 and 3.6 i.e.,
P ((.f) = concalspd (P ((.)) P(.)):
where I. is the number of input SPD matrices of operation concat,ypd (n). (.).
15

---

Published as a conference paper at ICLR 2024
D.2.4 OPTIMIZATION
For parameters that are SPD matrices, we model them on the space of symmetric matrices, and then apply the exponential map at the identity.
For any parameter P E Grn.ps parameterize it by matrix B E Mp,n-p such that
0
B
[Log (PP"), Inppp-
-B"
0
Then parameter P can be computed by
P (Logf, (PP"), In exp ([-B-
B 0 )TT
Thus, we can optimize all parameters on Euclidean spaces without having to resort to techniques developed on Riemannian manifolds.
D.3 TIME COMPLEXITY ANALYSIS
Let Tiin Rin be the size of input SPD matrices, Nout X Poutt be the size of output matrices of the convolutional layer in GyroSpd++, Arank X Trank be the size of SPD matrices in structure spaces, ne be the number of action classes, n, be the number of SPD matrices encoding a sequence.
GyroSpd++: The convolutional layer has time complexity mn mn)) The MLR layer has time complexity Onnamour)
GyroSpsd++: The convolutional layer has time complexity non The MLR layer has time complexity (n nen
D.4 MORE EXPERIMENTAL RESULTS
D.4.1 ABLATION STUDY
Impact of the factor in Affine-Invarint metrics To study the impact of the factor 3 in Affine- Invariant metrics on the performance of GyroSpd++, we follow the approach in Nguyen (2021). Denote by (4, Σ) the Gaussian distribution where μ E IR" and E Mn., are its mean and covari- ance. We can identify the Gaussian distribution (14, Σ) with the following matrix:
Σ + kuy (())
uak) I
(det Σ)
where l: 2 1, (k() is matrix with i identical column vectors μ. The natural symmetric Riemannian metric resulting from the above embedding is given (Nguyen 2021 by
1 n Tr(VP TWPP-)))
(V, w)p Tr(VP- wP-1)
where is an SPD matrix, V and W are two tangent vectors at point P of the manifold, This Riemannian metric belongs to the family of Affine-Invarian metrics where B = n
For this study, we replace matrix Z, in Eq. 4 with the following matrix:
Zz + VV.
ve(k:)]
Z, = (det Ze)
(A))TT
I
where ni x ni is the size of matrix Z. The input data for GyroSpd++ are then computed from Z, before.
Tab. reports the mean accuracies and standard deviations of GyroSpd++ with respect to different settings of 3 the three datasets. GyroSpd++ with the setting 3 0 generally works well all the datasets. Setting = 3 improves the accuracy of GyroSpd++ on NTU60 dataset. We also observe that setting l to high value, e.g., k 10 lowers the accuracies of GyroSpd++ the datasets.
16

---

Published as a conference paper at ICLR 2024
Table 4: Results (mean accuracy + standard deviation) of GyroSpd++ with respect to different settings of 3 on the three datasets (computed over 5 runs).
Datasct
HDM05
FPHA
NTU60
8 0
79.78 1 1.42
96.84 1 0.27
95.28 1 1.37
k 3
79.12 + 1.37
96. +: 0.25
96.32 + 1.33
t 10
78.25 + 1.39
95.91 1: 0.29
94.44 + 1.34
Table 5: Results and computation times (seconds) of GyroSpd++ with respect to different set- tings of the output dimension of the convolutional layer FPHA dataset (computed over 5 runs). Experiments are conducted on a machine with Intel Core 17-8565U CPU 1.80 GHz 24GB RAM.
Computation timc/epoch
Accuracy + standard deviation
Training
Testing
10
94.53 + 0.31
30.08
12.07
96.84 + 0.27
129.30
50.84
30
96.80 1 0.26
182.52
71.49
Output dimension of convolutional layers Tab. presents results and computation times of Gy- roSpd++ with respect to different settings of the output dimension of the convolutional layer on FPHA dataset. Results show that the setting terms of mean accuracy and standard deviation. However, compared to the setting m 21, the setting m 30 only increases the training and testing times without improving the mean accuracy of GyroSpd++.
21 clearly outperforms the setting m 10 in
Design of Riemannian metrics for network blocks The use of different Riemannian metrics for the convolutional and MLR layers GyroSpd++ results in different variants the same archi- tecture. Results of some of these variants FPHA dataset our architecture gives the best performance in terms mean accuracy, while the architecture with Log-Chol Chollsky geometry for the MLR layer performs the worst in terms of mean accuracy.
shown in Tab. 6 It is noted that
D.4.2 COMPARISON OF GYROSPD++ AGAINST STATE-OF-THR-ARR METHODS
Here we present more comparisons of our networks against state-of-the-arr networks. These net-
works belong to one of the following families of neural networks: (1) Hyperbolic neural net-
works: HypGRU (Ganea al. 2018 Graph neural networks: MS-G3D (Liu et al. 2020 !
TGN Zhou et al. 2023 (3) Transformers: ST-TR (Plizarri et al. 2021 Note that MS-G31 G3D,
TGN, and ST-TR are specifically designed for skeleton-based action recognition. We use default parameter settings for these networks. Results of our networks and their competitors on HDM05,
FPHA, and NTU60 datasets are shown in Tabs. 7 g and 9 respectively. On HDM05 dataset, Gy- roSpd++ outperforms HypGRU, MS-G3D, ST-TR, and TGN by 25.6%. 10.8%, 11.9%, and 9.5% points terms mean accuracy, respectively. On FPHA dataset, GyroSpd++ outperforms Hyp- GRU, MS- M3-G3D, ST-TR, and TGN by 38.6%, 8.5%. 10.9%, and 6.0% points in terms of mean accuracy, respectively. On NTU60 dataset, GyroSpd++ outperforms HypGRU, MS-G3D, ST-TR, and TGN by 7.0%, 3.1%, 4.2%, and 2.2% points in terms of mean accuracy, respectively. Overall, our networks are superior to their competitors in all cases.
Finally, we present comparison of computation times of SPD neural networks in Tah. 10
17

---

Published as a conference paper at ICLR 2024
Table 6: Results (mean accuracy + standard deviation) of GyroSpd++ with different designs of Riemannian metrics for its layers on FPHA dataset (computed over runs).
AI-LE (GyroSpd++)
LE-LE
AI-AI
LE-AI
AI-LC 89.16 0.26
96.84 = 0.27
94.72 + 0.25
94.35 + 0.29
95.21 + 0.26
Table 7: Results of our networks and some state-of-the-an methods on HDM05 dataset (computed over runs).
Method
Accuracy + Standard deviation
HypGRU
Ganea et al.
2018
54.18 + 1,51
MS-G3D
Liu et al. 2020
68.92 1 1.72
ST-TR
Plizzari et al.
2021
67.84 + 1.66
TGN
Zhou et al.
2023
70.26 1,48
GyroSpd++ (Ours)
79.78 1 1.42
GyroSpsd++ (Ours)
78.52 + 1.34
E NODE CLASSIFICATION
E.1 DATASETS
Airport (Chami et al. 2019) It is flight network dataset from OpenFlights.oom where nodes represent airports, edges represent the airline Routes, and node labels are the populations of the country where the airport belongs.
Pubmed (Namata et al. 2012b) It is standard benchmark describing citation networks where nodes represent scientific papers in the area of medicine, edges are citations between them, and node labels academic (sub)areas
Cora (Snn et al. 2008 It is citation network where nodes represent scientific papers in the of machine learning, edges are citations between them, and node labels are academic (sub)areas.
The statistics of the three datasets summarized in Tab.
E.2
IMPLEMENTATION DETAILS
E.2.
SETUP
Our network is implemented using the PyTorch framework. We set hyperparameters as in Zhao et al. 2023) that found via grid search for each graph architecture the development set of given
dataset. The best settings of n and p are found from (n,p) E {(2,, k)). k 2.3. 2. 10. The batch size is set the total number of graph nodes in dataset (Chami al. 2019 Zhao al. 2023). The
networks are trained using cross-entropy loss and Adam optimizer for maximum of 500 epochs. The learning rate is set 10-2 2 Early stopping is used when the loss on the development
has not decreased for 200 epochs. Each network has two layers that perform message passing twice at one iteration (Zhao et al., 2023). We use the 70/15/15 percent splits (Chami al. (019) for Airport dataset, and standard splits in Gipf & Welling (2017) for Pubmed and Cora datasets.
httterr//gtaaa.ccoo////a////ssss nn
httea///attbbb.oomm///ss///////
18

---

Published as a conference paper at ICLR 2024
Table 8: Results of our networks and some state-of-the-art methods on FPHA dataset (computed over 5 runs).
Method
Accuracy + Standard deviation
HypGRU Ganea et al.
2018
58.24 = 0.29
MS-G3D
Liu et al.
2020
88.26 1 0.67
ST-TR Plizzari et al. 2021
85.94 0.46
TGN Zhou ct al.
2023
90.81 + 0.53
GyroSpd++ (Ours)
96.84 1 0.27
GyroSpsd++ (Ours)
97.90 0.24
Table Results of our networks and some state-of-the-at methods on NTU60 dataset (computed over 5 runs).
Method
Accuracy i Standard deviation
HypGRU Ganea al.
2018
88.26 = 1.40
MS-G3D Liu et 2020
92.15 1.60
ST-TR Plizzari et al.
2021
91.04 1 1.52
TGN 7hou et al.
2023
93.02 + 1 1.56
GyroSpd++ (Ours)
95.28 1.37
GyroSpsd++ (Ours)
96.64 1.35
E.2.2 GRASSMANN LOGARITHMIC MAP IN THE ONB PERSPECTIVE
The Grassmann logarithmic map in the ONB perspective is given Edelman et al. 1998) by
Logp (Q) U arctan(E)
where P, Grapp U, E, and V are obtained from the SVD (I, PPP))PP''' UEVT.
E.2.3 GR-GCN++
To create Grassmann embeddings input node features, first transform d-dimensional input features into p(n - p)ddimmnsional vectors via a linear map. We then reshape each resulting vector matrix B M p.a-p- The input Grassmann embedding X E V is computed as
([-
D
1) exp (- [ B] D
In
X = exp
=xx
-B"
E.2.4 GR-GCN-ONB
To create Grassmann embeddings input node features, first transform d-dimensional input features into p(n - -)ddimensional vectors via a linear map. We then reshape each resulting vector matrix B M Mp The input Grassmann embedding X E V is computed as
B exp -B" BT 0 D1
x
Feature transformation is performed by first mapping the input to projection matrix (using the mapping 7 in Section 3.4. I), then applying an isometry map based on left Grassmann gyrotrans- lations (Nguyen & Yang. 2023), and finally mapping the result back to matrix with orthonormal columns. This is equivalent to performing the following mapping:
ΦM M exp([Log]" (MM"), In,n))))
where X E Granp and M Grap is model parameter.
19

---

Published as a conference paper at ICLR 2024
Table 10: Computation times (seconds) per epoch of networks and some state-of-the-art SPD neural networks on FPHA dataset. Experiments are conducted on machine with Intel Core i7- 8565U CPU 1.80 GHz 24GB RAM.
Method
SPDNet
SPDNetBN
GymAL-HAUNet
SPSD-AI
MI.-AI
GyroSpd++
GyraSpsd++
Training
17.52
40.08
62.21
73.73
102.58
129.30
126.08
Testing
3.48
6,22
30.83
35.54
46.28
50.84
48.06
Table 11:
Description of the datasets for node classification.
Dataset
#Nodes
#Edges
#Classes
#Features
Airport
3188
18631
Pubmed 19717
44338
500
Cora
2708
5429
7
1433
For any X E Gra.p. let X VU be a QR decomposition of X, where V E Ma.p is a matrix with orthonormal columns, and U Mp.p is upper-triangular matrix. Then the non-linear activation function is given by
o o V.
Bias addition is performed using operation 3gr instead of operation @gr. The output of Gr-GCN- ONB is mapped to a projection matrix for node classification.
E.2.5 OPTIMIZATION
For any parameter P E Gra.ps we parameterize it by a matrix B E Mp such that
0
B
Logg (P),Inp)) p.p
BT
0
=
Then parameter P can be computed by
P = exp ([-
0
B
(-
(-
0
B D
I exp
0
-B"
E.3 MORE EXPERIMENTAL RESULTS
E.3.1 ABLATION STUDY
Projector ONB perspective More results of Gr-GCN++ and Gr-GCN-ONB presented As can observed, Gr-GCN+ outperforms Gr-GCN-ONB in cases. In
Tabs. [2] and 13, particular, the former outperforms the latter by large margins Airport and Cora datasets. Results show that while both the networks learn node embeddings on Grassmann manifolds, the choice of perspective for representing these embeddings and the associated parameters can have significant impact on the network performance.
E.3.2 COMPARISON OF GR- GCN++ AGAINST STATE- SEATE-OF-THE-ALE METHODS
Tab. 14 shows results of Gr-GCN++ and some state-of-the-art methods on the three datasets. The hyperbolic networks outperform their SPD and Grassmann counterparts on Airport dataset with high hyperbolicity (Chami et al.) 2019). This agrees with previous works (Chami al. 2019 Zhang et al., 2022) that report good performances of hyperbolic embeddings on tree-like datasets. However, network and its SPD counterpart SPD-GCN outperform their competitors Pubmed and Cora datasets with low hyperbolicities. Compared to SPD-GCN, Gr- EC-GGN++ always gives more consistent results.
20

---

Published as a conference paper at ICLR 2024
Results and computation times (seconds) per epoch of Gr-GCN++ and its variant Gr- GCN-ONB based on the ONB perspective. Node embeddings are learned on Gr4,2 and Gr4,2 Gr-GCN-ONB and Gr-GCN++, respectively. Results are computed over 5 runs. Experiments are conducted on a machine with Intel Core 17-9700 CPU 3.00 GHz 15GB RAM.
Table 12:
Method
Gr-GCN-ONB
Gr-GCN++
Accuracy = standard deviation
53.2 + 1.9
60.1 + 1.3
Airport
Training
0.07
0.21
Testing
0.05
0.12
Accuracy 1 standard deviation
75.7 1 2.1
77.5 + 1.1
Pubmed
Training
0.50
0.90
Testing
0.38
0.54
Accuracy + standard deviation
33.9 + 2.3
64.4 + 1.4
Cora
Training
0.10
0.12
Testing
0.07
0.08
Results and computation times (seconds) per epoch of Gr-GCN++ and its variant Gr- GCN-ONB based on the ONB perspective. Node embeddings are learned on GIG,3 and Grg.3 for Gr-GCN-ONB and Gr-GCN++, respectively. Results are computed over runs. Experiments are conducted on machine with Intel Core i7-9700 CPU 3.00 GHz 15GB RAM.
Table 13:
Method
Gr-GCN-ONB
Gr-GCN++
Accuracy 1 standard deviation
65.8 1: 1.5
74.1 1 0.9
Airport
Training
0.19
0.34
Testing
0.14
0.21
Accuracy + standard deviation
75.8 + 2.0
78.5 + 0.9
Pubmed
Training
0.90
1.76
Testing
0.75
1.05
Accuracy + standard deviation
41.4 2.2
70.5 = 1.1
Cora
Training
0.16
0.22
Testing
0.12
0.16
F LIMITATIONS OF OUR WORK
Our SPD network GyroSpd++ relies on different Riemannian metrics across the layers, i.e., the convolutional layer is based on Affine-Invarian metrics while the MLR layer is based on Log- Euclidean metrics. Although we have provided the experimental results demonstrating that Gy- roSpd++ achieves good performance on all the datasets compared to state-of-the-art methods, it is clear if our design is optimal for the human action recognition task. When comes building deep SPD architecture, it is useful to provide insights into Riemannian metrics one should use for each network block order to obtain good performance on target task.
In Grassmann network Gr-GCN++, the feature transformation and bias and nonlinearity op- erations are performed on Grassmann manifolds, while the aggregation operation is performed in tangent spaces. Previous works (Dai et al. 2021 Chen al. (022) HNNs have shown that this hybrid method limits the modeling ability of networks. Therefore, is desirable to develop GCNs where all the operations are formalized on Grassmann manifolds.
21

---

Published as a conference paper at ICLR 2024
Table 14: Results (mean accuracy + standard deviation) of Gr-GCN++ and some state-of-the-art methods on the three datasets. The best and second best results in terms of mean accuracy are highlighted in red and blue, respectively.
Method
Airport
Pubmed
Cora
GCN
Kipf & Welling
2017
82.2 + 0.6
77.8 + 0.8
80.2 + 2.3
GAT
Veličković al.
2018
92.9 = 0.8
77.6 + 0.8
80.3 + 0.6
HGNN
(Liu et al.
2019
84.5 = 0.7
76.6 + 1.4
79.5 + 0.9
HGCN
Chami al.
2019
85.3 = 0.6
76.4 0.8
78.7 0.9
LGCN Zhang ct al.
2021
88.2 = 0.2
77.3 + 1.4
80.6 = 0.9
HGAT Zhang al.
2022)
87.5 + 0.9
78.0 0.5
80.9 + 0.7
SPD-GCN (Zhao et al.
2023
82.6 = 1.5
78.7 + 0.5
82.3 + 0.5
HamGNN Kang et
2023)
95.9 + 0.1
78.3 + 0.6
80.1 1.6
Gr-GCN+ (Ours)
82.8 = 0.7
80.3 + 0.5
81.6 + 0.4
G SOME RELATED DEFINITIONS
G.1 GYROGROUPS AND GYROVECTOR SPACES
Gyrovector spaces form the setting for hyperbolic geometry in the same way that vector spaces form the setting for Euclidean geometry (Ungar 2002, 2005 2014). We recap the definitions of gyrogroups and gyrocommutative gyrogroups proposed in Ungar (2002; 2005 2014). For greater mathematical detail and in-depth discussion, refer the interested reader to these papers.
Definition G.1 (Gyrogroups (Unarr, 2014)). ). pair (G, @) is a groupoid in the sense that it is A groupoid (G, ()) is gyrogroup if its binary
nonempty set, G. with binary operation, operation satisfies the following axioms for a, b, G:
(GI) There is at least one element G called left identity such that @
(G2) There is an element Ga E G called a left inverse of a such that Sa + a = e.
(G3) There is an automorphism gyrla, G G for each a, b, E G such that + (b (a b) @ gyrla, blc (Left Gyroassociative Law).
The automorphism gyr(a, b] is called the gyroautomorphismm, or the gyration of generated by a, b. (G4) gyr(a, bj gyrja @ b, b) (Left Reduction Property).
Definition G.2 (Gyrocommutative Gyrogroups (Ungar 2014)). gyrogroup (G, + gyrocom- mutative if it satisfies
gyr(a, b) @ a) (Gyrocommutative Law).
The following definition of gyrovector spaces is slightly different from Definition 3.2 in Ungar (004))
Definition G.3 (Gyrovector Spaces). A gyrcommmutative gyrogroup (G, () equipped with a scalar multiplication
xt,z) -t E
is called a gyrovector space it satisfies the following axioms for s, E R and a, b, E G: and (-1) a Oa.
(V4) gy)(a, = 1 gyr(a, blc.
(V5) gyr(s Id. where Id is the identity map.
22

---

Published as a conference paper at ICLR 2024
G.2 AI GYROVECTOR SPACES
For P, Q E Sym,, the binary operation Nguyen 2022a is given as
PAQP!.
P Pa Q =
The inverse operation (Nguyen 2022a is given by
O P
G.3 LE GYROVECTOR SPACES
For P, Q S the binary operation Nguyen 2022a is given
P tie Q
exp(log(P) (P) + log log(
The inverse operation (Nguyen 2022a is given
O P
G.4 LC GYROVECTOR SPACES
For P, Q Sym the binary operation
Nguyenn 2022a given as
Po Q =LL0==1H220000X12200100000000000
where Y is matrix of the same size as matrix Y Mn.. whose (i.j) element is Y (()) if
and is zero otherwise, ID is diagonal matrix of the same size as matrix Y whose (i, i) element (E.i). and (P) denotes the Cholesky factor P, i... (P)) is lower triangular matrix with
positive diagonal entries such that P = (P)))
The inverse operation (Nguyen
2022a) is given by
O P= (-) (-
G.5 GRASSMANN MANIFOLDS IN THE PROJECTOR PERSPECTIVE
For P, Q Gra.pp the binary operation Nguyen 2022b is given
P Bgr exp((L (P), I. In, ) eeepxx[opglpee In,)),
where [..-] denotes the matrix commutator.
The inverse operation (Nguyen 2022b) is defined as
gr P
Exprr (- Log (P)).
G.6 GRASSMANN MANIFOLDS IN THE ONB PERSPECTIVE
For U, V E Gra.p- the binary operation
Nguyen & Yang 2023) defined as
U gr V exp([Log (UU"), L.Vl
The inverse operation can be defined using the approach Nguyen & Yang 2003 (see Section 2.3.1), i.e.,
6 U U= @gr (UU*)),
where the mapping defined in Proposition 3.12 i.e.,
Grap
Grap: U U
23

---

Published as a conference paper at ICLR 2024
G.7 THE SPD AND GRASSMANN INNER PRODUCTS
Definition G.4 (The SPD Inner Product). Ler P. Q E Sym,. Then the SPD inner product of P and Q defined as
P (Logf, Logi, (Q))..
Definition G.5 (The Grassmann Inner Product). Let P, Q & Gra.g Then the Grassmann inner product of and Q is defined
9) (Login., (P), Login, (Q)
where (m., denotes the inner product at In.p given by the canonical metric of GIn.p
G.8 THE GYROCOSINE FUNCTION AND GYROANGLES IN STRUCTURE SPACES
Definition G.6 (The Gyrocosine Function and Gyroangles). Let P, Q and R be three distinct gyropoints in structure space Grup x Sym The gyrocosine of the meusure of the gyroangle a, 0 S a тτ, between @psd.g P Ppdd. Q and ppdd.g @ped.y
R is given by the equation
(Spsd.g P @psd.g Opedd P ppsd.g R R.ped.
cos
ppsd. P @psd.y Q pad.9 .l @psd.g P Ppsd,
R pad.g"
where ||- is the norm induced by the inner product in structure spaces. The gyroangle a is denoted by ZQPR.
G.9 THE GYRODISTANCE FUNCTION IN STRUCTURE SPACES
Definition G.7 (The Gyrodistance Function in Structure Spaces). Let P.Q Grn.p x Sym Then the gyrodistance function in structure spaces GIn,p Sym is defined as
d(P. Q) Opad. P @psd.g Ql1s....
G.10 THE PSEUDO-GYRODISTANCE FUNCTION IN STRUCTURE SPACES
Definition G.8 (The Pseudo-gyrodistance Function in Structure Spaces). Let HW be a hyper- gyroplane in structure space GFn.p Sym and X E Grn,p x Sym Then the pseudo-gyann gyrodistance from X to Hpaddg P is defined as
d(x, HW..)) sin( (iinnXP))
where Q is given by
Q = arg max
cos(LQPX)
QERenda
By convention, sin(2XPQ) for any X, Q Hesd.
H COMPUTATION OF CANONICAL REPRESENTATIONS
Let Vnp be the space of x matrices with orthonormal columns. For any P E St let Up e Gra.p- Sp E Sym such that UpSPUT Denote by W the common subspace used for computing canonical representation P. first compute two bases of span(Up and span(W) denoted respectively by U and W, such that
dy U W) de.., (ppan(
24

---

Published as a conference paper at ICLR 2024
where dy (...) and dr.,p (..-) are the distances between two points in Vn and GIa.p. respec- tively. These two bases can be computed as U Up W = WV, where Y and V are obtained from a SVD of (Up)"W, ii,
(Up) U W = Y(cos Σ)v²
The SPD matrix Sp in the canonical representation of P is then computed as
Sp vU=
I PROOF OF PROPOSITION 3.2
Proof. We first recall the definition of the binary operation 3 in Nguyen (2022b).
Definition LI (The Binary Operation (Nguyen 2022b) Let P.Q. Q E Sym Then the binary
operation 2gg is defined as
P Q = Expo ( (Q))).
We have
(Togy (Q), W W)
(T (Tog. T (W)),,
(5)
(Expi, (T (Log ()) ExpÝ, (TEI,
(W))
where (1) follows from the invariance of the inner product under parallel transport, and (2) follows from Definition G.4
Let R = Expi, (T (Logg (Q)) Then
Logi, (R) T. (Log](Q))
which results in
-P Logg, (R)) Logg (Q).
Hence
Εxp1 (Th, (hh Logi, (R)) Q.
By the Left Cancellation Law,
Q= (1)g P 4g Q).
Therefore
Εxp$ (T.PP( Logi, (R))
Bg (e,P Q)
Expa Th. Logg, (eg 0 Q))
where the last equality follows from Definition M
We thus have
e P + Q = R
(6)
= -Expi (T (Logz
Combining Eqs. (5) and 16. we get
(Log (Q), = (e, @ Q, Expx (To., (W))},
which concludes the proof of Proposition 3.2
25

---

Published as a conference paper at ICLR 2024
J PROOF OF PROPOSITION 3.4
Proof. The first part of Proposition 3.4 can be easily verified using the definition of the SPD inner product (see Definition G.4 and that Affne-Invariant metrics (Pennec et al. 2020) (see Chapter 3).
To prove the second part of Proposition 3.4 we will use the notion of SPD pseudo- gyrodistance (Nguyen & Yang] (023) in our interpretation of FC layers on SPD manifolds, i.e., the signed distance is replaced with the signed SPD aeudo-gyrodistann
the interpretation given in Section 3.2.1 First, we need the following result from Nguyen & Yang (0223).
Theorem J.1 (The SPD Pseudo-gyrodistance from SPD Matrix to SPD Hypergyroplane
in an AI Gyrovector Space (Nguyen & Yang, ).23). Let Hw p be an SPD hypergyroplane in gyrovector space (Sym Dai, Rai), and X E Sym Then the SPD aseudo-gyrodistare from X to Hw, p is given by
Klog(P P XP 3), P WP- b
P : WP
|
By Theorem J.I the signed SPD secudo-gyrodistance from Y to an SPD hypergyroplane that con- tains the origin and is orthogonal to the E axis is given by
(log( (Y. Logf (E (E())) Logi ai (Ear (Bg))))
d(Y, Hondy (B J.m.)
According to our interpretation of FC layers,
((1))()) (log( Y), Logi (E
Logīn. (E (E(ii))
We consider two cases:
Case 1: i < j.
(X) (log( Y), 1 (ere +e,e?))
v2
1 (e + e,oo))l
YY, ff0ff + +c,of))
(log(
Y)()) (log(Y)
log(()) (Y)).)
We thus deduce that
log( (og(Y),.)) = v2 ((,))
Case 2: i i= j.
(X) (log( e,e," 1 vitm Imnd
leeee 1 V.mm ImllF
1
(log( (log((), e,o"" m v + m3 Im)r.
(1
26

---

Published as a conference paper at ICLR 2024
This leads to
1
U(e.s) (X) log( (x,)) (1
log( (15)))
v1 + m3
for i = 1,..., m. By summing up (((.5) (X) 1,... m, we get
Σ ((a
log Y) ().)))
v1 m,
or equivalently,
m
log( log(Y) (1.)) /1 m3
Σ (1.0) (X)).
(8)
Replacing the term Σラ= log( (og(Y)or))) (o)) in Eq. 7 with the expression on the right-hand side of Eq. results in
m
log(Y)((n)) = (X)
(VI+ m3
1) Σ
(X).
m
Note that Y = exp((log( (Y (B)bog(())))))))))))) (( This concludes the proof of Proposition 3,4
K PROOF OF PROPOSITION 3.5
Proof. This proposition is a direct consequence of Proposition 3.4 for 3 =
L PROOF OF PROPOSITION 3.6
Proof. The first part of Proposition 3.6 can be easily verified using the definition of the SPD inner product (see Definition G.4 and that of Log- Log-Cholesky metries (Lin) 2019).
To prove the second part of Proposition 3.6 we first recall the following result from Nguyen & Yang 2023
Theorem L.1 (The SPD Gyrodistance from an SPD Matrix to an SPD Hypergyroplane in LC Gyrovector Space (gguyen & Yang 2023)). Let Hw, p be an SPD hypergyroplane a gyrovector space (Sym Dte, Stc), and X E Sym Then the SPD pseudo-gyeen gyrodistance from X. to Hw .p is equal the SPD gyrodistance from X to Hw.p and is given by
d(X, Hw.P) (A,, r)
|B]F
where
A = (P) + log +
B = [W] + D(p -
W + (P)
where YY] and D(Y), Y Mn, are defined in Section G.4 and (PP) (P)).
By Theorem L.I
the signed SPD pseudo- eoroodo-gyyddittncee from Y an SPD hypergyroplane that con- tains the origin and is orthogonal to the (i,)) axis is given by
(l((Y)) + log(D((((Y))) (Logf.. (E(r,)))) |I( Logg (F(o glll
d Hoogle (Bl )mn)))))))===
27

---

Published as a conference paper at ICLR 2024
According to interpretation of FC layers,
(() (Lp(Y)) + log( || (Logf... (B )) ||I
(Logt.. (B(r))))
We consider two Case 1: i < j.
(X) (Le(Y)) + (D
|le,e |
(Lx(Y)) (og(())))))))
+((()))))
We thus have
(()) = (X)
Case 2: i j.
(X) = (Lx(Y)) + (og()((c(Y))).eee)
le,e?
(lx(Y)) (og(D(((Y))))
= -log(((()
Hence
(Y) = exp(x)a((
Setting (p(Y) [b(s)))ī)==> then ((.)) are given by
exp (())))
ifi= j if < if
Vi..0)
(X),
Since (YY is the Cholesky factor of Y, have
Y = (Y)x(Y))
which concludes the proof of Proposition 3.6
M PROOF OF THEOREM 3.11
Proof. Let HwLp be hypergyroplane in structure space Gra.p x Sym and E Grup x Sym By the definition of the neeuo-gyroddsstnnee function,
d(x, Hwwde) sin( (XPQ)
where Q is given by
Q
arg max
cos(ZQPX)
W.F
(Spdd.g P @psddg Q Opsdy P @padd X xed.g
QEnand [PP)
arg max
Opsd. P @psd. Q psd.g || Opsd.g P Oped. X X|psdd.
28

---

Published as a conference paper at ICLR 2024
By the definitions of the binary and inverse operations in structure spaces, (6 U G e Sx),
ppd.. P @pdd_ X
ppsd.g P Epsd,g Q q Ug SQ).
Hence
(Gpdd.g @psd.y X, Spdd.g @psd. Q)ody ux0 Ux)"
Up U2) UpG Up)
+ (ep 0g Sx S 6g SQ)
Let
A1
Logi U (x) or Ux)").
B
Login. (6 Ur Gp U2)))
A2
Logt, Sp
Bg Sx),
and
B2 = Logi, (0 By SQ). Then we have
A(A1, BJ) F + (A2, B2)
Q =
arg max
Q o (PP
A F + A₂ B, | + B
(9)
([VAA, A2). /
arg max O (PP)
|||] F. |
where || is the concatenation operation similar to operation concal, apd (-)
From the equation of hypergyroplanes in structure space Gra.p Sym
(pped.g @psd.g Q, W) 0.
Let W (Uw, Sw). Then have
X((5gr U Up (2g Up UQ) "U (Uw)7o) (0,Sp 3g Sq. Sw>* = 0.
(10)
Let W = Log (Uw(Uw)))) W2 Logg (Sw). Then Eq. 10) can be rewritten
AB1, WT) + (2,, W2)s = 0,
which is equivalent to
O.
(1I)
Now, the problem in 99) to find the minimum angle between the vector [VAA,]|A] and the Euclidean hyperplane described by Eq. (11). The secudo-gyrodistance from X Hped.g be obtained as
W thus can
([VAA, /VWW W [[VÁW [22]lF A W + (A2, W2)
d(x, HWWd.)
W. B + W2ll
Some simple manipulations lead to
( U or (x) or Ur Ux) Uw U )or + (0,Sr Bg Sx. Sw)Sw))
d(x, HW
| |1)2
which concludes the proof of Theorem 3.11
29

---

Published as a conference paper at ICLR 2024
N PROOF OF PROPOSITION 3.12
Proof. We need the following result from Nguyen & Yang 2023
Proposition N.1. Let M and N be two Riemannian manifolds. Let Φ M N be an isometry.
Then
Logp Q) (Do
((P) ( (Logs (o(Q))
where P,Q M, DTR(W) denotes the directional derivative of mapping at point R E N along direction W E TR Log(.) and Log(.) are the logarithmic maps in manifolds M and N, respectively.
We adopt the notations in Bendokat et al. 2020
The Riemannian metric (Q(...) on O, is the
standard inner product given (Edelman al.
1998
Bendokat al. 2020) as
9Q (2), (9 92)
Tr(2)
where E On. S2,, O2 TTQO On-
Gr Let U E Gra.p. D2.D2 E TUGIn.p. The canonical metric JU (D1. D2) on Gra.p is the restric- tion of the Riemannian metric (....) . to the horizontal space of TQ O, (multiplied by 1/2) and is
given (Edelman al. 1998 Bendokat et al. 2020) by
90 (D. D2) Tr (D (I, (UU")D2)
(12)
Let P Grn.p. A, Tp Gr..p- The canonical metric of (A1, A2) on Grn.p is the restric- tion of the Riemannian metric 36 (...) (...) the horizontal space of TQ O, (multiplied by 1/2) and is given (Edelman et al. 1998 Bendokat et al. 2020) by
(A,, A2) 2 Tr (AA
hor are the horizontal lifts of A and A₂ to Q, respectively. Here, Q is related to by Q = (U U and P UUT, where U E Gra,p and is the orthogonal completion of U.
where A hor and Ahor Q
Denote by Horu Grapp the horizontal space of TuGrapp Then this subspace is characterized by
Hory Grnpp
[U B E Mn-ppl-
From Eq. (3.2) in Bendokat et al. (2020).
(Ai, (2) = TI ((AA.)) A
(13)
where A Alg and Aeer are the horizontal lifts of A and Ag to U, respectively. Therefore, by Eq.
Grn.P 9u (A1.0 Aor) Tr ((A).0) (I, UUT (UT))A
Tr ((A T hor
Tr ((A
har UUT
hor
2.U
2.U
Tr ((A).o)) T AA.0))
Tr (TU(BB)
(OUBBB)UUTT TUUTU
(14)
2
Tr ((A).0)) T 2,0)
Tr (B
= Tr ((At.))
hor
2,U,,
where the last equality follows from the fact that U 0. Combining Eqs. 13 and
we get
OP
Gr
(A A2)
Su
Grn.p
(A
L. 220)
hor A hor
30

---

Published as a conference paper at ICLR 2024
By Proposition N.1
Logp of (F) = (DT-110))1(888-11((
gr (F))) (F))),
where P. F. E GTn.p.
From Eq. (3.15) in Bendokat al. 2020).
DrR(W)= RW WR".
Therefore
Logp (F) = (P (T
gr
T + Log. (1 (TF)) (B))T-((P)) (P)"
gr
which concludes the proof of Proposition 3.12
31


Reviews:
[{"review_id": "PNUwVZUHPM", "reviewer": "Reviewer_2o5X", "summary": "The paper proposed a gyro space and gyro-vector based DNN on Grassmannian and SPD. The authors proposed formulation of building blocks like Convolution, Fully connected based on formulation derived in Nguyen, 2022a;b. The authors showed improved performance on human action recognition and node classification tasks formulated as on SPD and Grassmannian.", "questions": "The authors should elaborate what are the limitations in Chakraborty et al (2020) as they stated “Their proposed layers have nice theoretical prop- erties. A common limitation of the above works is that they do not provide necessary mathematical tools for constructing many essential building blocks of DNNs on SPD manifolds” without mentioning what mathematical building blocks are missing. A quick search reveals that same authors proposed other building blocks such as normalization using similar tools, please see “https://arxiv.org/pdf/2003.13869.pdf\"\nIn 3.1, it is quite confusing why authors choose different notation to denote Grassmannian using different representations.\nThe authors should explain the abbreviation of the metrics on SPD like ai, le, lc used in section 3.1. For a general reader it will be easier to understand.\nAs FC layer can be treated as special case of convolution (with full kernel size), the authors should restructure the paper by defining convolution first, then the authors can just address FC trivially. \nIt is not clear why authors suddenly defining MLR in section 3.3 on positive semi-definite matrices. The MLR for SPD (and trivially can be extended to SPSD ) by using formulation in  Nguyen (2022a)\nWhat is the motivation behind using different metrics: “We use Affine-Invariant metrics for the convolutional layer and Log-Euclidean metrics for the MLR layer”?\nWhat is rationale behind better performance using structure space representation as mentioned in “These results show that MLR is effective when being designed in structure spaces from a gyrovector space perspective.”? They should essentially represent same space, why the difference in achieving different optima?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is nicely motivated in the context of gyro vector representation.\n2. The formulation of convolution and fully connected layers are nicely derived and formulated from Nguyen, 2022a;b.", "weaknesses": "1. The paper mostly based on formulation by Nguyen, 2022a;b. I don't want to treat this as a weakness, but a lack of strength.\n2. The experiment results are rather naive, in recent years there is increasing literature in manifold DNNs so one wants to see thorough experimentations both in terms of different setting. Also some comparisons are missing including that with Chakraborty et al. (2020).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G24MQCV6bU", "reviewer": "Reviewer_pWPq", "summary": "This paper proposes novel formulations of fully-connected and convolutional layers, as well as MLR, on the SPD manifold based on gyrovector calculus.", "questions": "What non linearities were used? Is it a classical ReLU applied on the manifold, in the tangent space, or a specifically adapted rectifier such as those typically used with SPD deep nets, e.g., to inflate the small eigenvalues?\n\nCan the authors restructure the text to improve the flow of the exposition and the introduction of the networks for the experimental evaluation?\n\nAs it stands, a lot of the content is in the supplementary.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "The paper is well written, easy to follow, and the mathematics appear sound. The theoretical contributions are extensive and significant, with the potential of impacting research on matrix manifold neural networks in different areas.\n\nShowing how to compute the Grassman logarithmic map in a differentiable way is another contribution.\n\nThe experimental evaluation for action recognition includes existing SPD deep learning methods and shows improvements.", "weaknesses": "The formulation of convolutional layers is more like a sketch of the proof than an actual definition.\n\nThe experimental evaluation for node classification as shown in the main text is fairly weak: the authors did not include any baseline, at least a few Euclidean-featured GNNs should have been included, as well as hyperbolic graph neural networks (Chami et al., and newer architectures).\n\nThere is actually no definition of what GyroSpd++ in the paper beyond a description of the matrix dimensions. Likewise, Gr-GCN is not properly defined in the experimental evaluation. Overall, this makes it difficult to understand the full network design and what is being evaluated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R4E9EcMGRO", "reviewer": "Reviewer_iDyb", "summary": "- This paper discusses the design of deep neural networks (DNNs) on matrix manifolds, specifically Symmetric Positive Definite (SPD) and Grassmann manifolds.\n- This paper mathematically proposes a way to compute the Fully Connected layers and Convolutional layers for SPD matrices. \n- The paper also presents a method for performing back-propagation with the Grassmann logarithmic map in the projector perspective for Grassmann manifolds.\n- Experiments are designed in human action recognition and node classification tasks, while both are very small datasets.\n- The paper tries to address limitations of existing works and provides necessary mathematical tools for building DNNs on SPD and Grassmann manifolds.\n\n----\nI've read the rebuttal and would like to slightly increase the rating.\nThanks.", "questions": "- What is the main difference between this paper and other existing manifold network?\n- The author mentioned \"our convolution operation can be used for dimensionality reduction\", can you explain this more? From my understanding, if the input is NxN SPD matrix, the output has to lie within this NxN space. So I'm a little confusing about the reduction.\n- Some tiny comments, the author defined exp(P) and log(P) but seems not use this annotation at all.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper builds upon the theory of gyrovector spaces and extends it to matrix manifolds such as SPD and Grassmann manifolds.\n- This paper defines a new way to build the basic blocks of neural networks - fully-connected and convolutional layers, for SPD matrices, and most specially for Grassmann, which is rarely discovered in the field.\n- The authors demonstrate the mathematical rigor and effectiveness of their approach.\n- The authors provide an ablation study and comparison against state-of-the-art methods, further validating the effectiveness of their approach.", "weaknesses": "- The experimental evaluation is based on some small and not generally used dataset, (not as big as ImageNet or equivalent). This limits the overall generalizability of the proposed approach.\n- The network structures are limited with FC and CNN in most cases, while other network structures are missing, such as attention/ activation/ etc.\n- It would be helpful if the author can clearly highlight the novel contributions and how they differ from or improve upon the existing theories discussed in other papers, for example [1,2,3,4]. Most of these existing papers discussed the building blocks of SPD network/ Grassmann network/ etc. Some other paper discussed other manifold such as Hyperbolic [5,6]. \n\n\n\n[1] Building Deep Networks on Grassmann Manifolds. Zhiwu Huang, Jiqing Wu, and Luc Van Gool.\n\n[2] ManifoldNet: A Deep Neural Network for Manifold-valued Data with Applications. Rudrasis Chakraborty, Jose Bouza, Jonathan H. Manton, and Baba C. Vemuri.\n\n[3] Dilated Convolutional Neural Networks for Sequential Manifold-valued Data. Xingjian Zhen, Rudrasis Chakraborty, Nicholas Vogt, Barbara B Bendlin, and Vikas Singh\n\n[4] Neural Architecture Search of SPD Manifold Networks. Rhea Sanjay Sukthanker, Zhiwu Huang, Suryansh Kumar, Erik Goron Endsjo, Yan Wu, and Luc Van Gool.\n\n[5] Hyperbolic Neural Networks. Octavian Ganea, Gary Becigneul, and Thomas Hofmann\n\n[6] Hyperbolic Attention Network. Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)