Geometry-Aware visualization of high dimensional Symmetric Positive Definite matrices
Authors: Thibault de Surrel
Sylvain Chevallier
Fabien Lotte
Florian Yger
Venue: Accepted by TMLR ()

Abstract: Symmetric Positive Definite (SPD) matrices are pervasive in machine learning, from data features (such as covariance matrices) to optimization process.These matrices induce a Riemannian structure, where the curvature plays a critical role in the success of approaches based on those geometries. Yet, for ML practitioners wanting to visualize SPD matrices, the existing (flat) Euclidean approaches will hide the curvature of the manifold.
To overcome this lack of expressivity in the existing algorithms, we introduce Riemannian versions of two state-of-the-art techniques, namely t-SNE and Multidimensional Scaling. Therefore, we are able to reduce a set of $c \times c$ SPD matrices into a set of $2 \times 2$ SPD matrices in order to capture the curvature information and avoid any distortion induced by flattening the representation in an Euclidean setup. Moreover, our approaches pave the way for targeting more general dimensionality reduction applications while preserving the geometry of the data. We performed experiments on controlled synthetic dataset to ensure that the low-dimensional representation preserves the geometric properties of both SPD Gaussians and geodesics. We also conduct experiments on various real datasets, such as video, anomaly detection, brain signal and others.

Published in Transactions on Machine Learning Research (02/2025)
Geometry-Aware visualization of high dimensional Symmet- ric Positive Definite matrices
Thibault de Surrel LAMSADE, CNRS,
thibault de-tddddd-ssueeee www.dd-wwwreeelloodddddddaiiiiae
PSL Univ. Paris- Daubse,ee,,,e, Poris, France
Sylvain Chevallier LISN,
sylvain, sylooin. wwwp:///wwweteeeuouuuoorrrrreeee
University Paris-Soclay, Gif-sur- Yvelle, France
Fabien Lotte
Sbbien.lotte@inniinn
Inria center at the University of Bordeawar, LaBRI, Talence, France
Florian Yger
LITIS
INSA Nouen-Normandy,
Rouen, France
Reviewed on OpenReview:
https: // openreview. net/ net/ forum? id- DYCSRf3uby
Abstract
Symmetric Positive Definite (SPD) matrices are pervasive in machine learning, from data features (such as covariance matrices) to optimization process. These matrices induce a Riemannian structure, where the curvature plays a critical role the success of approaches based on those geometrics. Yet, for ML practitioners wanting to visualize SPD matrices, the existing (flat) Euclidean approaches will hide the curvature of the manifold. To overcome this lack of expressivity in the existing algorithms, two state-of-tho-art techniques, namely t-SNE and Multidimensional Scaling, Therefore, we are able to reduce set of x e SPD matrices into set of x 2 SPD matrices in order to capture the curvature information and avoid any distortion induced by flattening the representation in a Euclidean setup. Moreover, our approaches pave the way for targeting more general dimensionality reduction applications while preserving the geometry of the data. We performed experiments controlled synthetie dataset to ensure that the low- dimensional reprosentation preserves the goometric properties of both SPD Gaussian and geodesics. We also conduct experiments on various real dalasets, such as video, anomaly detection, brain signal and others.
introduce Riemannian versions of
1 Introduction
Covariance matricos are being used in a lot of different scientific areas to better understand how multiple random variables are related to each other. Such matrices can be used in process control (Willjuice Irutha- yarajan & Baskar 2010), fault detection (Russell et al.) 2000), biomedical image analysis (Penned 2020 Kobayashi al. 2023) or Brain Computer Interfaces (BCI) (Lotte et al.) 2018). Unfortunatoly, as soon one manipulates a point cloud of covariance matrices each summarizing more than two random variables,
it becomes impossible to visualize the whole point cloud in or 3 dimensional space. A common way

---

Published in Transactions on Machine Learning Research (02/2025)
Geodesic
Baclidenn path
(a) Different examples of projections of the curved two- (b) The set of 2 x 2 SPD matrices. The green path dimensional surface of globe on a plane. All three of these the geodesic between two SPD matrices and the projections distort in way or another the surface.
blue path is the usual Euclidean path.
Figure 1: Illustration of the distortion induced by projection and the set of 2 2 SPD matrices.
to overcome this issue is to use dimensionality reduction algorithm. The goal of such algorithms is to transform data from a high-dimensional space into a low dimensional space so that the low dimensional representation contains some meaningful properties of the original data. The ideal case is when we reduce sufficiently the dimension so that we are able visualize the low-dimensional representation (for example, the low dimensional space is R² or IR*).
When dealing with covariance matrices, the natural geometry being used is the affine invariant Riemannian geometry (Bhatial 2007; Pennee et al. 2006). Indeed, covariance matrices have special structure, they are symmetric, definite and posilive (SPD), therefore they live on the Riemannian manifold of SPD matrices. This manifold not flat, contrary to usual Euclidean spaces (see Figure 1b to visualize the set of x 2 SPD matrices and the geodesic between two points). This curvature implies that one can not reproduce accurately point cloud sampled in a Riemannian space into a Euclidean one. For example, it is well known (Mulcahy & Clarke 2001) that one can not project the Earth (the curved two-dinensional Riemannian surface of a sphere) plan without distorting the surface in examples of projection). The same phenomenon appears when dealing with SPD matrices. A noticeable disadvantage of considering the Euclidean geometry when dealing with SPD matrices is the swelling effect (Arsigny et al. 2006). The swelling effect tells us that the determinant the Euclidean mcan can be strictly larger than the original determinants. As the determinant is au important measure of the dispersion of the associated multivariate random variable, we would like this dispersion be correctly summarized in the mean. This negative effect disappears when considering the correct Riemannian geometry and its associated Riemannian mean.
way and to some extent (sce Figure la for some
Certain domains rely profoundly on covariance matrices such as BCT, where the goal is to translate brain signals into computer commands. In this context, the covariance matrices of ElectoEnnephhaloGrapy (EEG) brain signals are used as EEG representations to classify BCI users' mental commands. In BCI, it has been shown that using the Riemannian geometry is the correct way deal with SPD matrices, and geometry aware classificrs have grcatly improved the state of the art (Yeer et al.) 2017 Lotte et al., 2018 Barachant of SPD
et al. 2013; Chowdhury & Andreu-Perez) 2021 Roy et. al. 2022). However, for visualizing may be tempted to use Euclidean tools such as the Euclidean t-SNE (Nishimoto et al. 2020 Zanini et al.] 2018). Unfortunately, by doing so, the obtained visualizations do not take into account the Riemannian geometry that works so well for classification.
matrices coming from BCI experiment,
In this paper, we adapt two Euclidean algorithms: the Multidimensional Scaling (MDS) and the t-SNE to Riemannian setting. We want to reduce a set of e x SPD matrices while preserving the curvature of the original space into a lower dimension embedding, with 2 SPD matrices as an ideal candidate for visualization purpose. After adapting MDS and t-SNE to reduce matrices of the SPD manifold, we show
2

---

Published in Transactions on Machine Learning Research (02/2025)
that the properties of the high-dimensional point cloud stay true in the small dimensional space of 2 x 2 SPD matrices. Specifically, we show that the important affine-nnariancc propriety is preserved and that geodesics and SPD) Gaussians are correctly reproduced.
The paper is organized as follows: we start by reviewing the related works on dimensionality reduction in Section 2 Then, we introduce the Riemannian geometry on the set of SPD matrices in Section B The core of the paper is Section 4 where we describe the two Riemannian versions of the MDS and the t-SNE. We give some theoretical results on these algorithms in Section and O In Section 7 we test
algorithms synthetic datasets and real-life datasets from 4 different applications. We compare the proposed Riemannian versions of the algorithms against their Euclidean counterparts as well as against other geometry aware dimensionality reduction methods. Finally, we argue on our choice of the affine-invariant metric on the manifold of SPD matrices in section
2
Related works
Dimension reduction (DR) is not new topic in data analysis and machine learning. DR methods can be classified into two main categories: linear and non-linear methods.
Linear methods: The most famous linear method is probably the Principal Component Analysis (PCA) (Hastie et al. 2009) which seeks directions that best explain the original data. Another linear method is the Independent Component Analysis (TCA) (Hastie et al. 2009), that aims to separate a multivariate signal into additive, independent components. A supervised DR technique Linear Discriminant Analysis (LDA) (Izenman 2008 that seeks find linear combination of features that maximizes the separation between different classes in the data.
For non-linear methods, we have the Multidimensional Scaling (MDS) (Borg & Groenen 1997) and t-SNE (van der Maaten & Hinton 2008) that will be detailed in more depth in Section Other non-linear methods include Kernel Principal Component Analysis (KPCA) (Schölkopf et al. 1997 which is an extension of traditional PCA that uses the kernel method to better work in high-dinensional spaces. Locally Linear Embedding (LLE) (Roweis & Saul 2000) seeks to preserve local relationships within the high-dimensional data. It reconstructs each data point as a lincar combination of its nearest neighbors. Another mothod whose goal is to find the intrinsic structures of data from a non-linear manifold is ISOMAP (Tenenbaum et al. 2000). It a generalization of the MDS algorithm where the initial distances are computed using& neighborhood graph. A well known geometry-aware method is Uniform Manifold Approximation and Projection (UMAP) (MeInnes et al. 2020) based on Riemannian geometry and tools from algebraic topology. The core idea is similar to t-SNE, that is to compute similarity matrices in the high and in the low-dimensional space and then optimize the oin-diiensional one. The differences with t-SNE lie in the way the similarities are computed and the way they are optimized (UMAP uses the cross-entropy whereas (-SNE uses the Leibler divergence). Finally, another method that focuses on discovering the underlying manifold of the data by modeling diffusion processes is Diffusion Maps (Coifman & Lafon (006). For most of these methods, a common assumption is that the data lies on an unknown manifold, and the goal is to learn this unknown manifold. In our case, instead, the data lies on known manifold, the manifold of SPD matrices, and we leverage this information in our approaches.
Non-linear methods:
Riemannian methods: Of course, some of these algorithms have been adapted to be used with data living on known manifold by taking into account the geometry inherited by it. The PCA algorithm has been generalized to a manifold setting by Fletcher et al. (004)) to build the Principal Geodesic Analysis (PGA). Instead of seeking for the best directions to explain the data, PGA identifies the principal geodesics, which are the curves representing the most significant directions of variation on the manifold. Other algorithms have been extended to a Riemannian sotting such as Locally Lincar Embodding (LLE) (Saul & Rowcis 2003), and Hessian LLE (HLLE) (Goh & Vidal 2008).
Hyperbolic methods: Some works have been done to reduce data living on hyperbolic spaces, as such spaces can be used in the representation learning of word embeddings (Nickel & Kiela, 2017) visual inputs
3

---

Published in Transactions on Machine Learning Research (02/2025)
[Khrulkov et al. 2020). For example, PCA fitted for data lying on hyperbolic space has been proposed by Chami et al. (2021) called HoroPCA. The difference with the PGA is that HoroPCA looks for directions rather than subspaces that explain the data. A similar work to ours is the adaptation of the t-SNE algorithm to hyperbolic spaces called CO-SNE by Guo et al. (022).
Methods for SPD matrices: Some algorithms have been especially designed for the manifold of SPD matrices. In Harandi et al. (2014), they learn mapping from high-dimensional manifold to lower dimen- sional one without relying tangent space approximations of the manifold. This algorithm demonstrated good results on image and video databases. Another extension of the PCA to the SPD manifold was pro- posed in Horev et al. (2017). This truly Riemannian PCA takes into account the curvature of the space and preserves the global properties of the manifold. The authors tested their algorithm on BCI data, as we will do in part [74 BCI is an interesting motivation to build DR. algorithms for SPD matrices as the dimension of the covariance matrices that is dealt with grows quadratically with the number of sensors used to record the EEG. A lot of algorithms have been built with this application in mind (Coelho Rodrigues
et al. 2017; Davoudi et al.) 2017 Tanaka et al. 2016 Peng et al. 2023 Xie et al. 2017). Some work focused on the classification of SPD matricos (Huang & Van Gool 2016 Brooks et al. 2019). It is possible to use SPDNet to perform dimensionality reduction, but it requires class information to learn the adequate low dimension representation, hence limiting scope of applications. Moreover, the learned embedding would be Euclidean therefore losing precious information about the geometry. The main difference between all the above algorithms and our approach is that their final goal is to classify the SPD matrices whereas our goal is to do visualization. Their goal is to preserve as much structure as possible, while increasing the separation between the classes to classify the data in the low-dimensional manifold better than in the original one. Our goal is different: we want to visualize the original dataset, with 2D or 3D visualization reflecting the structure of the original space. In López et al. (202I), the authors develop a vector-valued distance and gyrocalculus for SPD matrices. The visualizations that they obtain are a by-product of the theory they develop. Morcover, and unlike our work, the visualizations they produce represent distances and angles of the high-dimensional SPD matrices. In our work, we reduce directly the SPD matrices in low-dimension and in Riemannian manifold corresponding to the type of data we are working with (SPD matrices).
3 A primer on SPD matrices
The Riemannian geometry of SPD matrices (Bhatia 2007 Pennec et al., 2006 A c X covariance matrix belongs to the set Pe of symmetric, positive definite (SPD) matricos:
P. = X E R = XT, Vu E RC (0), u. Xu Yu>0 0}
As Pe is an open and convex set of the set of symmetric e matrices Se, it is submanifold of dimension a(c+ + 1)/2. Moreover, its tangent spaces TxPo are identified with S.. Let us now define the following scalar product on each tangent space of Pe:
Definition 3.1. Let X E Po. We define the following scalor product on TxPo:
YU,V E TxPo. (U,V)x = tr(x -U--VV
(1)
This metric is called the Affine Inveriant Riemannian Metric (AIRM) as, if A is an invertible matrix, (AU AT AV AT)xAXA = (U,V).. The application X (x))x being continuous, when Pe endowed with this metric, it becomes a Riemannian manifold. Moreover, the manifold Pa is complete and therefore, each pair of points (X,Y) can be connected by a unique minimizing geodesie:
Proposition 3.2. Let X,Y E Po. The expression of the unique geodesic I' linking them is:
[0.1) T(1) = =02/2 10/2(x-10)) (x-1/2XX-172) x1/2.
This geodesic is the shortest path from X to Y on the manifold. The Riemannian distance between X and Y, denoted 5, is the length of the geodesic I connecting X to Y (with log being the matrix logarithm):
5(X,Y) = || logg(X -1/2 --1///111/2211
(2)
4

---

Published in Transactions on Machine Learning Research (02/2025)
An important property of the Riemannian distance defined above is its affine invariance property that inherits from the affine invariance of the metric:
Proposition 3.3 (Affine invariance of the AIRM distance Bhatia 2007
Let X,Y Pe and R be an
invertible matriz of size x c, then one has 8(RXR", RYYR R ) 8(X, Y).
One can find more information on the Riemannian geometry of SPD matrices in appendix A It should be noted that the ATRM geometry on the manifold of SPD matrices coincides (up to factor 1/2) with the Fisher-Rao metric Multivariate Normal (Skovgaard 1984.. Another famous geometry used the set Pa is the Log- Eag-Euclidean geomelry (Arsigny el al. 2006). We argue on our choice of the AIRM metric in section
The special case of 2 x 2 SPD matrices
Now that we have developed the general theory of x symmetric, positive definite matrices, let us focus on the special case of P2, the space of 2 x 2 SPD matrices. Indeed, it has a specific structure:
P2 - {(:
b) c)
s.t. a> 0, c c> 0, ac b²
Therefore, one can represent in R3 a 2 x 2 SPD matrix using (a, b, c) and, will lie strictly inside the cone of equation + ae where a, 0. We give a representation of this cone in Figure 1b.
4 Riemannian dimensionality reduction
In this section, describe the two algorithms studied and adapted to Riemannian setting: the Mullidimenssonal Scaling (MDS) and the I-SNE. The main difference between the two algorithms is that the MDS is a distance-based method whereas the t-SNE is a similarity-based method.
4.1 Riemannian Multidimensional Scaling
The goal of the Multidimensional Scaling (MDS) algorithm (Borg Groenen 1997 to reproduce the pairwise distances between the high-dimensional point cloud and the low-dimensional one.
Given N SPD matrices XI,..,, of size x C, we can compute their distance matrix D where Dij Xy). We recall that 8 is the Riemannian distance defined in Eq. 2 We are then looking for N SPD
matrices Yi,..,, YN of size 2 x 2 that minimize the stress function
Yn) = > (6(Y, -).)). K
(3)
The goal is to find points the low dimensional space P2 such that the pairwise distances are preserved well as possible. It is a natural DR problem formulation, as we want the point cloud in low dimension (Yş), to "look' like the point cloud in high dimension (X;); much as possible.
The stress function SR defined in Eq. goes from P. into R so, can apply a Riemannian gradient descent algorithm (see Appendix B for more details). To be precise, we apply this Riemannian gradient descent algorithm on the product manifold P2 x ... x P2 which is also a manifold. For this algorithm, we need to compute the gradient our cost function, here SR. So, having mind the gradient of the Riemannian distance as given in Proposition C.1 we have the following gradient:
Dig
-1) Logy,
Sy = 2
8(Y,,
jzti
where we recall that Logy, is the Riemannian logarithm as given in Proposition A.1 Details on the compu- fation of the gradient can be found in Appendix C.2

---

Published in Transactions on Machine Learning Research (02/2025)
4.2 Riemannian t-SNE
We are now going to look into another DR algorithm this time based on probabilistic framework. It was introduced in van der Maaten & Hinton (2008) and is called t-SNE, SNE stands for Stochastic Neighbor Embedding. The main difference with the MDS algorithm is that the MDS focuses on having a good low- dimensional representation of points that are far away. Indeed, in the cost function SR defined in Eq. B if two points X, and Xj in the high-dimensional space are far apart, the quantity 8(X;, X)) will have a lot of importance in the cost function, way more than the distance of two points close to each other. On the contrary, we will see that the t-SNE algorithm focuses on retaining the local structure of the data.
Given set of N high-dimensional data points X1,..., NN in Pe, we want to compute conditional probabilities that represent similarities. In the Euclidean setting, the similarity of X2 to X; is the conditional probability Pjji that X; would pick X, as its neighbor if they were picked in proportion to their probability density under a Gaussian centered at X,, To generalize this definition to SPD matrices, we use the generalization of the Gaussian to the manifold of SPD matrices introducod Said al. (2016) (more details in Appendix A.2 this distribution). Thus, Pije given by:
Xx)
(4)
PJl Eny; Xx.
where of is the variance of the Gaussian centered at T₁. To choose o,, perform binary search that produces P; = (D))))=1....NN with fixed perplexity chosen by the user The perplexity is parameter defined the same way as in the original t-SNE (van der Maaten & Hinton 2008). It is defined as Perp(P.) = (P:)) where (P.) is the Shannon entropy of P; (see) Shhannn (1948) for its definition). We then symmetrize
the conditional probabilities by setting Pij PuatPul when + and pa = 0.
2NV
Then, the t-SNE algorithm aims at learning Y1...., Yn E P2. that reflect the similarities Psj as well as possible. The Euclidean setting uses Student t-distribution with one degree of freedom as the heavy-tailed distribution the low-dimensional map. This way, dissimilar data points in high dimension will be modeled far away in low dimension. We will extend this distribution to the manifold of SPD matrices and define the joint probabilities qí as:
(1+5(K, Pij ExS((1 8(Y,, Y)2)-I
We then want to minimize the mismatch between the two conditional probabilities Pij and 4ij- To measure this, the Riemannian t-SNE (as the Euclidean t-SNE) uses the Kullback-Leibler (KL) divergence and tries to minimize it. The cost function CR is then:
N
N = ΣV
CR = Σ KL(P.I1.))
Piy
log
Pij
(6)
i=1
i=1
jpt
As for the MDS, a Riemannian gradient descent method is performed in order to minimize the KT. divergence with respect to the points Y. The gradient of CR with respect to Y, is given by:
N
Vr.Cn =
Pig - 4i Logy, (Y,).
I Y,Y,,,
More details on this derivation are given in Appendix C.3
5 Affine invariance of the algorithms
In this section, we give a theoretical result on the two algorithms we proposed Section 4 This result regards the affine invariance of our problem and of our algorithms. Indeed, we recall the important property of the affine invariance of the AIRM distance 3.3 Therefore, as both the MDS and the t-SNE depend only

---

Published in Transactions on Machine Learning Research (02/2025)
the distances between the point in high dimension, the two algorithms are also affine-invarian. We detail only the Riemannian MDS case, the results and proof being the same for the Riemannian t-SNE.
Proposition 5.1 (Affine invariance of the algorithms). Let XI,...N be points in Pa and R be an invertible matria.
1. Let (X1...,XXN (RX,,""..,,RXA."") Then, the set of solutions of the Riemannian MDS when the initial digh-dimensional points are (X)...,Nx) is the same as the set of solutions of the Riemannian MDS when the initial high-dimensiondl points are (K.....NN).
Let (Yi,.,, YN) a solution of the Riemannian MDS when the initial high sigh-dimensional points are (X (x), Then (RY, y.... y.. is also solution of the problem.
The proof can be found in Appendix This affine invariance property is particularly useful for EEG analysis. Indeed, full rank spatial filters are often used to process EEG and are alfine transformations (Congedo et al. 2017). Thus, comparing covariance matrices in the sensor space (i.e., obtained without applying spatial filter) amounts to directly comparing covariance matrices in the source space (i.e, that would be obtained with a spatial filter, e.g., with inverse solutions). Moreover, when comparing the results of a same subject over two sessions, it likely that the EEG electrodes are not exactly the exact same position on the head. Therefore, there exists an affine transformation between the covariance matrices recorded over two sessions. As our algorithms are affine invariant, the reduced point clouds will still be comparable between the two sessions and will be representative of the actual brain activity, matter the filter applied. Other Euclidean DR algorithms do not have this property of affine invariance, therefore, in the situations described above, some artifacts could appear when comparing two point clouds different up to an affine transformation.
6 A convergence result
We want to comment on the convergence of the Riemannian MDS. For this, let us consider Y E Pos d 0 and let us define the following sub function fY,d of the Riemannian MDS:
SY,a(X) = (8(X,Y) - d)²
As one can see using Eq. 3 the Riemannian MDS loss can be rewritten the sum of such functions, Unfortunatcly, this function is not g-convex (sce Appendix B for the definition of g-convex function and a counter-example for fy,a), so do not have the usual convergence results. However, the function fy,d be rewritten the following way:
= + dp -2)((X
g(X)
A(X)
Moreover, we know from Sra & Hosseini (2015 (Example 20) that X 8(X,Y) is g-convex. Therefore, d 2 0, both g and h are convex and the function fy.a be written difference of g-convex functions. Moreover, g and h are also lower semi-continuous and proper functions. One can then use the Difference of Convex Algorithm (DCA) proposed in [ergmman et al. (2023) to solve the sub-problem. In this case, the sequence of points generated by the DCA will converge to critical point of the cost function fy
7 Synthetic experiments
Now that we have described the two algorithms that we introduced, we can test them. We now show some experiments, first on synthetic datasets and then on real data. We used pyRiemann [Baacchant et al. 2023 deal with SPD matrices Python
'Qur code http://////gaaaaaaaaaaaaaaeeeerr
https://gSthub.co

---

Published in Transactions on Machine Learning Research (02/2025)
7.1 The contestants
In these experiments, assess how the algorithms proposed in the previous section perform against other DR algorithms. We first want to confirm that using Riemannian geometry is better when dealing with SPD matrices. For this, we compare 3 versions of the MDS and the t-SNE:
A Riemannian version: the input and output distances are computed using the AIRM.
A Riemannian-Eucliden version: the imput distances are computed using the AIRM, but the output are points from IR3 equipped with the Fuclidean structure.
A Euclidean version: the imput distances are computed using the Euclidean structure of symmetric matricos (the Frobenius norm) and the output are also Euclidean points in RR..
We also compare our algorithms with other DR algorithms. To do this, we compare the proposed algorithus with the Principal Geodesic Analysis (PGA) (Fletcher et al. 2004), UMAP (Meinnes al.) 2020) and the Riemannian PCA (Horev et al.) 2017). They are all described in Sec. 2 The output spaces of PGA and UMAP is the Euclidean space I while for the Riemannian PCA, it is P2 equipped with the AIRM. The input of the UMAP algorithm is the Riemannian distance malrix.
7.2 The evaluation metrics
As our goal is to build a visualization tool for datasets of SPD matrices, a first criterion is a qualitative one: how it looks. We will also use a quantitative motric: the trustworthiness cocfficient (Venna & Kaski, 2001). It ranges from 0 to 1 (with 1 being the highest score) and is computed by ranking the order of each point's neighbors and comparing these ranks between the original high-dimensional space and the reduced low-dimensional space. Any nearest neighbors in the reduced space that differ from the original space are penalized according to their ranking in the input space. The number of nearest neighbors k is a hyperparameter that will help us understand to what extent the local and global structures of the point cloud are preserved. One needs & < N/2 where N the total number of points.
7.3 Synthetic experiments
Reducing SPD Gaussians For this first experiment, reduce mixture of three Gaussians the set of SPD matrices. We sample N = 222 points from mixture ())), + G(X2, + G(X,, 03) where the means (X,) are the means of the 3 classes of the dataset AirQuality that is described in Section 7.4 and where 0 = 1/2, 02 = 1/3, 03 = 1/4. The points are balancod between the 3 Gaussians. We then use the different algorithms detailed in Sec. 7.11 to reduce this high-dimenssonal point cloud. The results' are given Figure 2 We can see that the fully Riemannian algorithms place the different Gaussians along a geodesic (see Figure [b] to see what geodesic looks like in the space of SPD matrices). Therefore, the blue and green Gaussians are further apart from each other than they in a fully Euclidean setting. Between the two Riemannian versions, the Riemannian t-SNE produces a nicer visualization, the Gaussians being scparated and less clustered at the origin. Finally, while the Cammannian-Eullidee separating the Gaussians, the resulting point cloud is in the Euclidean space R3, and not along geodesics as they should be. Thus, it does not bring any insight on how the Riemannian geometry affects the data.
algorithms are correctly
Reducing geodesics Tn this second experiment, we investigate how a high--dimensional geodesic in Pa is reduced using our algorithms. We want to see how close the reduced trajectory is to the corresponding geodesic in P2. More details on the sctup are given in appendix E In this experiment, we can only compare the Riemannian I-SNE with the Riemannian MDS we need the reduced points to lie in Pg- We conducted some experiments varying some parameters: the dimension of the high-dimensional geodesic, the noise its sampling and the number of points N sampled.
Those figures should be taken with caution, as they only 2D representations 3D point clouds. The interested reader can find some interactive plots the supplementary materials.
8

---

Published in Transactions on Machine Learning Research (02/2023)
caniaaniian-iemmanian MDS Gaussian
Ceteraanniin--uuccdeee MDS
Euclidean- Thclidean-Euclidenn MDS
Gaussian n
Ganssian n'3
Riemannian Riemannian t-SNE Riemannian Euclidean t-SNE
Bunnannia--Buldidann
Euclidean-Euckkean t-SNE
Figure 2: Results of the first synthetic experiment a comparison of the reduction of a mixture of Gaussians.
Distance vs dimension
Distance vs noise
Distance vs number of points
Algocithaee
300
Algorishmss
Algocithuus
Bicm
700
Roam
Rom Biem
Risu-Riem L-NE
Färu-Raam 1-NB
Rism-Rim LSNE
Distatre
500
38-
300
110
90
60
60
100
0.0
0.5
0.1
0.5
1.0
50
100
145
100
177
Dimenaion
Scale of Che noise
Number of points N
Figure 3: Results of the second synthetic experiment: comparison of the distances of the reduced geodesic by the t-SNE or the MDS algorithm to the real geodesic.
From qualitative point of view, and based on the visual example given in Figure 11 both algorithms preserve the structure of the high-dimensional geodesic. However, the MDS seams to do a better job than the t-SNE. Indeed, at each extremity of the geodesic reduced using I-SNE, the points are less precisely on the geodesic whereas the points in the middle are exactly on the geodesic. This can be explained by the fact that the t-SNE focuses more on reproducing correctly points that are close one another while the MDS gives an equal importance each point. We want to compare the reduced trajectory to true geodesic in Pa. For this, compute the geodesic going from one extremity to the other of the reduced trajectory. Then, we compare the distance between this true geodesic and the reduced trajectory. The results over 5 simulations are given in Figure 3 We can see that the Riemannian t-SNE is more robust than the Riemannian MDS. In fact, when varying the dimensson of the high-dimensional space or the noise, the distance to the true geodesic remains virtually unchanged. However, as the number of points rises, the distance grows. This is coherent as the optimization problem becomes harder the more points there are. The MDS, on the other hand, is very noisy and the quality of the reduction worsens the dimension or the noise grows. We must clarify that the t-SNE is not entirely robust to noise and changes in dimensionality. However, it demonstrates significantly
9

---

Published in Transactions on Machine Learning Research (02/2025)
Name
Domain
Number of matrices
Dimension
Ref.
1NC12014001
Brain Computer Interlaces
288 subjects
22
Tangermann
2012
BNC12014002
Brain Computer Interfaces
160 12 subjects
15
SSleyrl
AlexMI
Brain Computer Interfaces
subjects
16
Barachanl
2012
AirQuality
Atmospheric pollutants
102
6
[[uu [202] Smith et
2022a
FPHA
Video sequences hand actions
108
63
Harcia-Hernaadoo ct 2018 Wang 2023
TEP
Anomaly detection
420
x 52
[Dwns & Vogel 1993 Smith et al. 20226
Table 1: Summary of the datasets used for the experiments.
greater robustness compared MDS, which is highly sensitive to these two factors. The relative robustness of (-SNE to noise comes from the fundamental design of the algorithm explained above.
7.4 Experiments on real datasets
Setup After testing our algorithms on synthetic datasets, we now study their behavior on real datasets from different applications that are summarized in Table [ More details on the datasets as well as on the setup are given in Appendix G We used the different algorithms presented in Sec. 77. reduce the datasets. For the Riemannian t-SNE (AIRM and Log- -og-Euclidean),
chose perplexity of A where N is the total number of points. For the Euclidean t-SNE, we chose a fixed perplexity of 30 as it is the default parameter for the t-SNE of scikit-learn
[Pedregosa et al. (see Appendix for more insights on these choices).
2011
Riemannian t-SNE
left hand right_han hand
Riemannian MDS
Session 1
UMAP
Session 2
GOD GEOR
Euclidean t-SNE
Euchdean MDS
PCA
Figure 4: Result of the different algorithms on subject 8 of BNCT2014001
Tangermann et al. 2012
Results We give in Figure 4 the results of the different algorithms studied when reducing the subject 8 of the dataset BNC12014001 (Tangermann et al. 2012) from BCI experiment. The colors correspond to the classes and the shape of the markers corresponds to the two sessions during which this dataset was recorded. We see that the geometry aware algorithms (Riemannian t-SNE, Riemannian MDS, PGA and UMAP) are able to separate the two classes. We also clearly see that the two sessions are well separated. This is not surprising as the signals recorded by an EEG during BCI experiment are sensitive to the user's mental state (Saha & Baumert] 2020 Krumpe al. 2017). so is likely that the data has shifted from one session to another. We also give in Figure 13 the results of the different algorithms on the dataset AirQuality.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Comparaison of the differeut t-SNE
Comparsison of the different MDS
Comparaison of the different slgorithums
109-
Trucwearhhm hinessss
04-
04-
Alguritlatte
Algorii
Algoritlans
Rirn-ltiem sSNK
Rarm iiem MIDS
Rimm-ltton t-NK
PGA
Ran-Ru
Ea--Ra: MTS
Ramn-Rie MDS
Rion PCA
Roen-koo SSNIS
Riem iis MDS
0.70
UKAP
Percent of neighbors
Percent of neighbors
Percent of neigaboos
Figure 5: The results of the experiments on the real datasets: we plot the average of the trustworthiness coefficients over 6 datasets. The x-axis represents the size of neighborhood used to compute the trustwor- thiness cocfficient as % of the total number of points.
Dataset
Riem-Riem t-SNE
Euc-Ent t-SNE
Riem-Riem MDS
Euc- Euc MDS
BNCT201 4001
198.34
(+101.58)
274.92 (+6.99)
733.20
(+558.440
19.08 (+35.211
BNC12014002
62.11
494
113.38 (#31.79)
330.17 (+246.47)
3.49 (+1883
AlexMI
7.45
(#3.3
0.71 (+0.4
50.33 (=1.11
0.42 (0.24
AirQuality FPHA
2.53
0.75
379.26
0.71
10.58
2.31
383.45
0.81
Table 2: Computation time of the different algorithms on the different datasets. The time is given in seconds. For the BCI datasets, we displayed the mean time over all the subjects.
Our goal is to quantitatively see how the local and global structures of the high-dimenniooal point cloud are preserved. For this, we look at how the trustworthiness coefficient evolves when the neighborhood expands (sec. 7.2). We give in Figure 5 the results of this experiment and in Appendix H the detailed results. A first important observation is that considering the Riemannian geometry of SPD matrices is important as both the t-SNE and the MDS work better when the distances are computed using the ATRM distance of Eq. 2 When focusing on the different t-SNE, we see that the local structure (less than 10% of all the neighbors) is well-preserved using the Crmmaannian-Euclidee t-SNE but, as soon as the neighborhood gets bigger, the fully Riemannian t-SNE conserves better the structure. This is explainable as, locally, Riemannian manifold can be approximated by a Euclidean space (the tangent space) but globally, the curvature induces some deformations of the space. This is exactly what we observe and therefore, the Riemannian t-SNE should be used when trying to reduce a dataset of SPD matrices. For the MDS, we were not able to see any quantitative one.
differences between the fully Riemannian MDS and the Riemamnian
When comparing our algorithms with PGA, UMAP and the Riemannian PCA, we clearly see that the Riemannian t-SNE is the best algorithm on these datasets. As UMAP does not have any a priori information the geometry of the point cloud, it only manages to reproduce correctly the local structure of the point cloud and collapses as the neighborhood grows. PGA seems to be able to preserve the global structure of the point cloud, but is not very good when dealing with local structures. The Riemannian PCA greatly under- performs compared to the proposed algorithms. The Riemannian t-SNE works better than the Riemannian MDS, as we had alrcady witnessod in the previous experiments. All in all, the Riemannian t-SNE is the best trade-off between preserving the local and the global structures of a set of tigh-dimensional SPD matrices.
7.5 Details on computation times
The computations were made on a Macbook Pro M3 with 36 Go of memory. The time taken by the different algorithms to roduce the various datasets are detailed in Table 2 The Euclidean algorithms are, of course, faster than the Riemannian ones as each iteration is longer to compute. Indeed, at each iteration of the gradient descent, the distance matrix between all the points needs to be computed. As the AIRM distance
11

---

Published in Transactions on Machine Learning Research (02/2025)
AIRM vs Log-Euclidean
0.95
0.91
Trushworthhess
0.32
0.30
0.88
0.85
Algorithms
AIRM 6-NNE
Log 5-NNE
0.82
AIRM MDS
LogEue MDS
0.80
0.0
0.1
0.2
0.3
0.4
0.5
Percent of neighbors
Figure 6: Comparing the two distances on the manifold of SPD matrices. The trustworthiness coefficient over the 6 datasets averaged.
requires computing matrix inverse square roots and a matrix logarithm, it takes more time to compute than its Euclidean counterpart. However, one should remark that the dimension of the high-dimensional points has very little influence on the time each iteration takes to compute: at each iteration, only 2 x 2 SPD matrices are at stake, so the distance matrix is always computed on 2 x 2 matrices. We also remarked that the Riemannian t-SNE needed much less time to converge than the Riemannian MDS.
8 Affine invariant metric or Log- -ug-Educi metric ?
In section 3 we presented the Affine Inveriant Riemannian metric (AIRM) for the manifold of SPD matrices.
Tt is not the only metric that exists on the manifold Pe.- Another widely used (Huang et al. 2015 Chevallier et al. 2021) distance is the Log- Eucliieinnn distance (Pennec et al. 2006) that preserves theoretical properties while requiring fewer computations. The main idea of the Log-Euclidean is to apply the standard Euclidean metric to the matrices after transforming them with the matrix logarithm. This transformation maps them to the tangent space at the identity matrix, which is Euclidean space. We define this metric below.
Definition 8.1 (Log-Euclidess distance). The Log- Euclidann distance êLæ between X E Pa and Y E P. is:
SLE(X, = [X)( - log(Y))l
In this section, we want to compare the Riemannian t-SNE and Riemannian MDS based on the AIRM metric (as done so far) and based on the Log-Eucline Euclidenn distance described above. For this, we implemented versions of the 1-SNE and the MDS that uses the Log-Euclides distance instead of the AIRM distance. We refer to the two versions of the t-SNE as AIRM t-SNE and LogEue t-SNE and likewisc for the MDS. As the EuglEucli does not have the affine invariance property contrary to the AIRM distance, the built algorithms using the Log- Euglidenn are not affine invariant.
We conducted the same experiments as in section 7.4 on the real datascts, and we show the results in Figure G. For the L-SNE, it is clear that using the AIRM distance leads to better results than using the Log-Enclidean distance. For the MDS, changing the distance used does not lead to significant changes in the results. As we loose some important propertics (such as the affine invariance) when computing distances with the Log- Euclidenn distance without any result improvement, we conclude that one should use the AIRM distance when reducing SPD from high-dimensional space to low-dimensional one.
12

---

Published in Transactions on Machine Learning Research (02/2025)
9 Conclusion
In this paper, we introduced variant for t-SNE and MDS to handle data the Riemannian manifold of SPD matrices. We showed that the Riemannian algorithms are able to retain important properties of the geometry such as the affine invariance. Moreover, we showed that we could cast our problem into a class of optimization problems that can be written as a difference of g-convex functions. In the experiments, we did not implement this DCA, but rather a Riemannian Gradient Descent (RGD) algorithm. We do not have any theoretical guarantees on the convergence of this problem using a RGD, however, in practice, we stopped the RGD when the gradient was small enough (< 10-6), There are also no guarantees that the critical point reached is indeed minimizer of our cost function. Finally, and real-world data and demonstrated their usefulness. The Riemannian t-SNE is the most. interesting in real-world applications. Hence, if the goal is to visualize set of SPD matrices, and therefore we care about the faithfulness of the representation, our Riemannian t-SNE should be used. Furthermore, even if using a less computationally heavy distance on the manifold of SPD matrices (such as the log- Euclidean one), we showed that the best suited metric for our work the AIRM one.
tested algorithms synthetic
In future works, we consider several promising extensions and applications of our methods. First, the optimization algorithms computationally expensive and may limit the use of our approaches for very large datasets. A stochastic optimization setting on manifolds (Bonnabel 2013) is promising solution. Our Riemannian dimensionality reduction could be used as visual feedback tool to help users train controlling BCIs. This could help to tackle the issue of BCT illiteracy /deficiecy (Allison & Neuper 2010), in the spirit Duan et al. (2021) but conforming with the AIRM geometry. Finally, our approaches focused on the case of SPD matrices, but could be extended to other manifolds. For this, one would need a manifold that can be viewed and plotted in two three dimensions such Stfel manifolds that can be used to visualize sets of subspaces Yamamoto et al. 2021
Acknowledgments
This work was funded by the French National Research Agency for project PROTEUS (grant ANR-22-CE33- 0015-01). Part of this work was carried out while Florian Yger was a member of PSL-Dauphine University, and he acknowledges the support of the ANR as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 31A Institute). Sylvain Chevallier is supported by DATAIA (ANR-17-CONV- 0003)
References
Brendan Z Allison and Christa Neuper. Could anyone use a BCI? Brain-computer interfaces: Applying our minds to human-compater interaction, pp. 35-54, 2010.
Bruno Aristimunha, Igor Carrara, Pierre Guetschel, Sara Sedlar, Pedro Rodrigues, Jan Sosulski, Divyesh Narayanan, Erik Bjareholt, Barthelemy Quentin, Robin Tibor Schirrmeister, Emmanuel Kalunga, Ludovic Darmet, Cattan Gregoire, Ali Abdul Hussain, Ramiro Gatti, Vladislav Goncharenko, Jordy Thielen, Thomas Moreau, Yannick Roy, Vinay Jayaram, Alexandre Barachant, and Sylvain Chevallier. Mother of all BCT Benchmarks, 2023. URL https: thbttbb.
thub. com comm/KeerrTecch/maaaaa
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Geometric Means in a Novel Vector Space Structure on Sysmetric Cositive-Definite Matrices. SIAM J. Matria Analysis Applications, 29:328 347, January 2006. doi: 001370050033996
A. Barachant. Commande robuste d' I'un effecteur par une interface cerveau machine EEG asynchrone. Theses, Université de Grenoble, March 2012. URL https://theses: hal. www.hhhl...cc....... htt..h..cccennneeeee-1111077777.
Alexandre Barachant, Stéphane Bonnet, Marco Congedo, and Christian Jutten. Classification of covariance matrices using a Riemannian baaeed kernel for BCI applications. Neurocomputing. 112:172, July 2013. doi: 10.1016/j.neucom.200 12.00.....9.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Alexandre Barachant, Quentin Barthélemy, Jean-Rémi King, Alexandre Gramfort, Sylvain Chevallier, Pedro L. C. Rodrigues, Emanuele Olivetti, Vladislav Goncharenko, Gabriel Wagner vom Berg, Ghiles Reguig, Arthur Lebeurrier, Erik Bjäreholt, Maria Sayu Yamamoto, Pierre Clisson, and Carst-Const Corsi.
aobeemann/pyriemmane v0.5, 2023. URL betps://dofi www..o.....////oood/ddooooooo 0w8.52221d22aa .tt.//22oooo..089383
Ronny Bergmann, Orizon P. Ferreira, Elianderson M. Santos, and João Carlos O. Souza. The difference of convex algorithm on Hadamard manifolds, May 2023.
Rajendra Bhatía. Posilive Definile Matrices. Princeton Series in Applied Mathematics. Princeton University Press, Princeton, N.J, 2007. ISBN 978-0-691-1291181.
Silvère Bonnabel. Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control, (8()):2217-2229, 2013. doi: 10.100//TAC.20032255611.
Ingwer Borg and Patrick Groenen. Modern Multidimennsional Sealing. Springer Series in Statistics. Springer, New York, NY, 1997. ISBN 978-1-1757-2713-5 977--77777-77 doi: 10.1007/976-1--777-2711111
Nicolas Boumal. An Introduction. to Oplimization on Smooth Manifolds. Cambridge University Press, 1 edition, March 2023. ISBN 978-1-00-916616- 978-1-00-916617-1 978-1-00-916615-7. 9781009166164.
doi: 10.1017/
Danicl Brooks, Olivier Schwander, Frederic Barbaresco, Jean- Yves Schneider, and Matthieu Cord. Ric- mannian batch normalization for spd neural networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- une 32. Curran Associates, Inc., 2019. URL htttps://rroceeding, pewpeddiinnnnuuuiss... wwwp.ppp..cppppprr.aee. siteer_fllleee (illes/paper/ 0a0/ff//o6680b0baa77777777744430
pdf
Ines Chami, Albert Gu, Dat Nguyen, and Christopher Ré. HoroPCA: Hyperbolic Dimensionality Reduction via Horospherical Projections, June 2021.
S. Chevallier, E. K. Kalunga, Q. Barthélemy, and E. Monacelli. Review of Riemannian Distances and Divergences, Applied to SSVEP-based BCL. Neuroinformatics, 19(1):93 106, January 2021. ISSN 1559- 0089. doi: 10.1007 111007//12021-000-00000 09473-9.
Anirban Chowdhury and Javier Aeree. Clinical brain computr interface challenge 2020 (CBCIC WCCT2020): Overview, methods and results. IEFF Transactions Medical Robotics and Bionics, 3(3): 661-670, 2021.
Pedro Luiz Coelho Rodrigues, Florent Bouchard, Marco Congedo, and Christian Jutten. Dimensionality Reduction for BCI classification using Riemannian geomelry. In BCI 2017 - 7th International Brain- Computer Interface Conference, Graz, Austria, September 2017. Gernot R. Müller-Pute doi: 10.3217/
978-3-85122--53--111
Ronald R. Coifman and Stéphane Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21(1):5 30, July 2006. ISSN 1063-5203. doi: http///.acha.2066.0....
Marco Congedo, Alexandre Barachant, and Rajendra Bhatia. Riemannian geometry for eeg-based brain- computer interfaces; a primer and review. Brain- muain-Computer Interfaces, 4:1-20, 03 2017. doi: 10.1080/ 2326263X.2017.12977
Alireza Davoudi, Saced Shiry Ghidary, and Khadijeh Sadatnejad. Dimensionality reduction based on distance preservation to local mean for symmetric positive definite matrices and its application in brain computer interfaces. Journal of Neural Engineering, 14(3):036019, March 2017. ISSN 1741-2552. doi: 10.1088/
1741-2552/ 7741-2552/aa6lbbb abb.bb.
James J. Downs and Ernest F. Vogel. A plant-wide industrial process control problem. Computers g chemical engineering, 13332:245-255 1993. URL http://www..ciemeddrrer..co htt.://ciee.....ooo/oeocc/eeea.a pis/009911359938001111
14

---

Published in Transactions on Machine Learning Research (02/2025)
Xu Duan, Songyun Xie, Xinzhou Xie, Klaus Obermayer, Yujie Cui, and Zhenzhen Wang. An Online Data Vi- sualization Feedback Protocol for Motor Imagery-Based BCT Training. Prontiers in Human Newroscience, 15, 2021. ISSN 1662-5161.
P.T. Fletcher, Conglin Lu, S.M. Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8):995-1005, August 2004. ISSN 1558-254X. doi: 10.1 10.1109/TMM. 2004..00.4.31
Guillermo Mercia--ernando, Shanxin Yuan, Seungryul Back, and Tae-Kyun Kim. First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations, April 2018.
Alvina Goh and Rene Vidal. Clustering and dimensionality reduction on Riemannian manifolds. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-7, Anchorage, AK, USA, June 2008. IEEE. ISBN 978-1-4244-2242-5 doi: 10.1 10.1109/M 1011009///P2200 200PR.2008.45874222
Yunhui Guo, Haoran Guo, and Stella X. Yu. CO-SNE: Dimensionality Reduction and Visualization for Hyperbolic Data. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11 20, New Orleans, LA, USA, June 2022. IEEE. ISBN 978-1-66546-946-3. doi: DV.1109/CVPR52688. 2022.00011.
Mehrtash T. Harandi, Mathieu Salzmann, and Richard Hartley. From Manifold to Manifold: Geometry- Aware Dimensionality Reduction for SPD Matrices, November 2014.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer
Series in Statistics. Springer, New York, NY, 2009. ISBN 978-0-387-84857-- 978-0-387-84858-7. doi:
00.1007/978-0--87-888888
Inbal Horev, Florian Yger, and Masashi Sugiyama. Geometry-awarr principal component analysis for sym- metric positive definite matrices. Machine Learning, 106(4):493 522, April 2017. ISSN 0885-6125, 1573- 0565. doi: 1.11007/100991-06-666666
Jinxi Hua, Yuanxun Zhang, Benjamin de Foy, Xiaodong Mei, Jing Shang, and Chuan Feng. Competing PM2.5 and NO2 holiday effects in the Beijing area vary locally due to differences in residential coal burning and traffic patterns. Science of The Total Environment, 750:141575, Jamuary 2021. ISSN 0048-9697. doi: 10.1016/).scitotenn... 202...110220..1...
Zhiwu Huang and Lue Van Gool. A Riemannian Network for SPD Matrix Learning, December 2016.
Zhiwu Huang, Ruiping Wang, Shiguang Shan, Xianqiu Li, and Xilin Chen. Log-euclidean metric learning on symmetric positive definite manifold with application to image set classification. In International conference on machine learning, pp. 720-729. 729. PMLR, 2015.
Alan Julian Izenman. Linear Discriminant Analysis. In Alan J. Izemman (ed.), Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning, Springer Texts in Statistics, pp. 237-280. Springer, New York, NY, 2008. ISBN 978-0-387-78189-1 doi: 10.10077997-00-377-8888-- 8.
Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky. Hy- perbolic Image Embeddings. 2020 IEEE/CVP Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 6417-6427, Seattle, WA, USA, June 2020. IEEE. ISBN 978-1-72817-168-5. doi:
10.1109/CVPR42600 20..000000000000
Goro Kobayashi, Talsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. Deep Geodesic Canonical Correlation Analysis for Covariance- Based Neuroimaging Data. In The Twelfth International Conference on Learning Representations, October 2023.
Tanja Krumpe, Katrin Baumgaertner, Wolfgang Rosenstiel, and Martin Spüler. Non-stationarity and inter- subject variability of ceg characteristies in the context of bei development. In Graz Brain- Computer Interface Conference, 2017. URL
httpstt:::/// https://api. semanticscholar org/ ott.a//ccccc.ooooooooooooooooooa
15

---

Published in Transactions on Machine Learning Research (02/2025)
Olivier Ledoit and Michael Wolf. A well-conditioned estimator for large-dimensional covariance matri- ces. Journal of Multivuriate Analysis, 88(2):365-41 411, February 2004. ISSN 0047259X. doi: 10.1016/ 50044-2559X
Federico López, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Vector-valued Dis- tance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices. In Advances in Neural Information. Processing Systems, volume 34, pp. 18350-18360 18300. Curran Associates, Inc., 2021.
F. Lotte, L. Bougrain, A. Cichocki, M. Clere, M. Congedo, A. Rakotomamonjs, and F. Yger. A review of classification algorithms for EEG-based brain computer interfaces: A 10 year update. Journal of Neural Engineering, 15(3):031005, April 2018. ISSN 1741-2552. doi: 10.
Leland McInnes, John Healy, and James Melville. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction, September 2020.
Francesco Mezzadri. How to Generate Random Matrices from the Classical Compact Groups. Notices of the AMS, 54(5), 2007.
Karen A. Mulcahy and Keith C. Clarke. Symbolization of map projection distortion: review. Cartogra- phy and Geographic Information Science, 28 2001. doi: 141158/1123340017821133334 URL https://doi.co org/ 10. 10011556/115224441778115444444
Maximillian Nickel and Douwe Kiela. Poincaré Embeddings for Learning Hierarchical Representations. In Advances in Newral Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Takashi Nishimoto, Hiroshi Higashi, Hiroshi Morioka, and Shin Ishii. EEG-based personal identification method using unsupervised feature extraction and its robustness against intra-subject variability. Journal of Neural Engineering, 1722)00,6007, March 2020. ISSN 1741-2552, doi: 10. 10.1088/ 174117552 /bb6/8...
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
Zhen Peng, Hongyi Li, Di Zhao, and Chengwei Pan. Reducing the Dimensionality SPD Matri- ces with Neural Networks in BCI. Mothematics, 11(7):1570, January 2023. ISSN 2227-7390. doi: 10.3390/math110771
Xavier Pennec. 3 manifold-valued image processing with spd matrices. In Xavier Pennec, Stefan Sommer, and Tom Fletcher (eds.), Riemannian Geometric Statistics in Medical Image Analysis, pp. 75 134. Aca- demic Press, 2020. ISBN 978-0-12-814425522 doi: https://dbi..rrg/// 110.1.000011111161//11 1971 URL www.s/cccccceet.ooocccaaaaaaaaaa
www. www.seeiiieeeeeetcccm
Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian Framework for Tensor Computing. International Journal of Computer Vision, 66(1):41-66, January 2006. ISSN 1573-1405. doi: 10.1007/
sl 11126-000-3322--1
Sam T. Roweis and Lawrence K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 2900050)): 2322-2326,,, December 2000. doi: 20.1126/science.o 290.e500..20502.
Raphaëlle N Roy, Marcel F Hinss, Ludovic Darmet, Simon Ladouce, Emilie S Jahanpour, Bertille Somon, Xiaoqi Xu, Nicolas Drougard, Frédéric Dehais, and Fabien Lotte. Retrospective on the first passive brain- computer interface competition cross-session workload estimation. Frontiers in Neurvergonomiiss 3: 838342, 2022.
Evan L. Russell, Leo H. Chiang, and Richard D. Braatz. Fault detection in industrial processes using canon- ical variate analysis and dynamic principal component analysis. Chemometrics and Intelligent Laboratory Systems, (1(1))81-99 93, 2000. ISSN 7439977439. doi: http:///doooor////o.0/////061110 http:://ww.ccccccacc.c.coaaaaaap
URL
16

---

Published in Transactions on Machine Learning Research (02/2025)
Simanto Saha and Mathias Baumert. Intra- and Inter-subject Variability in EEG-Based Sensorimotor Brain Computer Interface: A Review. Frontiers in Compulational Neuroscience, 13, 2020. TSSN 1662-5188.
Salem Said, Lionel Bombrun, Yannick Berthoumieu, and Jonathan Manton. Riemannian Gaussian Distri-
butions on the Space of Symmetric Positive Definite Matrices, December 2016.
Lawrence K. Saul and Sam T. Roweis. Think Globally, Fit Locally: Unsupervised Learning of Low Dimen- sional Manifolds. Journal of Machine Learning Research, 4(Jun):119 155, 2003. ISSN ISSN 1533-7928.
Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Kernel principal component analysis. In Wulfram Gerstner, Alain Germond, Martin Hasler, and Jean-Danicl Nicoud (ods.), Artificial Neural Net- works- - NN'7,' Lecture Notes in Computer Science, pp. 583 588, Berlin, Heidelberg, 1997. Springer. ISBN 978-3-540-69620-9, doi: BF0//BFb0020211
C. E. Shannon. A mathematical theory of communication. The Bell System Technicol Journal, 27(3): 379 423, 1948. doi: 10.1002/jj1533-7795.. 190...994...011303
Lene Theil Skovgaard. A riemannian geometry of the multivariate normal model. Scandinavian Journal of Statistics, 11(4):211-223 1984. ISSN 03036898, 14679469. URL htttp://wee http:://www.ssto.orrrg.... ottpsttt...og.aallee//eeeo
4615960
Alexander Smith, Jinxi Hua, Benjamin de Foy, James Schauer, and Victor Zavala. Multi-Sile, Multi- Pollutant Atmospheric Data Analysis Using Riemannian Geometry, December 2022a.
Alexander Smith, Benjamin Laubach, Ivan Castillo, and Victor M. Zavala, Data analysis using Riemannian geometry and applications to chemical engineering. Computers & Chemical Engineering, 168:108023, www.omceemgg..2222....ooo
December 2022b. ISSN 0098-1354. doi:
Suvrit Sra and Reshad Hosscini. Conic geometric optimisation on the manifold of positive definite matrices. SIAM Journal on Optimization, 25(1):713-775 January 2015. ISSN 1052-6234, 1095-7189. doi: 10.1137/ 140978168.
D. Steyrl, R. Scherer, J. Faller, and G. Müller-Putz. Random forests in non-invasive sensorimotor rhythm brain-computer interfaces: A practical and convenient non-linear classifier. Biomedizinische Tech- nik/Biomedical Engineering, 61:77-86, 04 2015. doi: 10.1515/bmt-20144-0 2014-0117.
Toshihisa Tanaka, Takashi Uehara, and Yuichi Tanaka. Dimensionality reduction of sample covariance matrices by graph fourier transform for motor imagery wrain-machine interface. In 2016 IEEE Statistical Signal Processing Workshop (SSP), pp. 1-5, June 2016. doi: 10.1109/SSI /St.//SS......551111
M Tangermann, K-R. Müller, A. Acrtsen, N. Birbaumer, C. Braun, C. Brunner, R. Leeb, C. Mchring, K. Miller, G. Mueller-Putss G. Nolte, G. Pfurtscheller, H. Preissl, G. Schalk, A. Schlögl, C. Vidaurre, S. Waldert, and B. Blankertz. Review of the bei competition iv. Fronliers in Newroscience, 6, 2012. ISSN htt.//nninn.111..2
1662-453X. doi:
URL https:/ http://www.fff www.tiereiiinnnnoo orgg/articless/1.3888
fnins. 2012.00055
Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. Global Gcometric Framework for Nonlincar Dimensionality Reduction. Science, 290(5500):2319 2323, December 2000. ISSN 0036-8075, 1095-9203. doi: 10.1 http://ccccccc..5550000
James Townsend, Niklas Koep, and Sebastian Weichwald. Pymanopt: A python toolbox for optimization on manifolds using automatic differentiation. Journal of Machine Learning Research, 17(137):1 5, 2016. URL http://jmlr www.aiirrrggrpppeoeeeeeeo
org/ paapers/v17116-177..ao
Hugues Van Assel, Titouan Vayer, Rémi Flamary, and Nicolas Courty. SNEkhorm: Dimension Reduction with Symmetric Entropic Affinities, May 2023.
Laurens van der Maaten and Geoffrey Hinton. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86) 2205, 2008. ISSN 1533-7928.
17

---

Published in Transactions on Machine Learning Research (02/2025)
Jarkko Venna and Samuel Kaski. Neighborhood Preservation in Nonlinear Projection Methods: An Exper- imental Study. In Georg Dorffner, Horst Bischof, and Kurt Hornik (eds.), Artificial Newral Networks ICANN 2001, Lecture Notes in Computer Science, pp. 485 491, Berlin, Heidelberg, 2001. Springer. ISBN 978-3-540-44668-2. doi: 10.1007/3-500-44666-- 00--540-44666---111 68.
Rui Wang, Xiao-Jun Wu, Ziheng Chen, Tianyang Xu, and Josef Kittler. DreamNet: A Deep Riemannian Manifold Network for SPD Matrix Learning. In Lei Wang, Juergen Gall, Tat-Jun Chin, Imari Sato, and Rama Chellappa (eds.), Computer Vision ACCV 2022, volume 13846, pp. 646 663. Springer Nature Switzerland, Cham, 2023. ISBN 978-3-031-26350- 978-3-031--6351-4 doi: 10.1007/978-3-031-222 39.
Martin Wattenberg, Fernanda Viégas, and lan Johnson. How to Use t-SNE Effectively. Distill, 1(10)ee2, October 2016. ISSN 2476-0757. doi: 10. 22915/disttill0000
M. Willjuice Iruthayarajan and S. Baskar. Covariance matrix adaptation evolution strategy based design of centralized pid controller. Expert Systems with Applications, 37(5))5775--78L1 2010. ISSN 0957-4174. doi: http:////0..000//.............0 pi1/50967417440000009
URL htttps///ww..ccecceeeret.ccc httpd//rrrr.roeesseee////rriieeo
Xiaofeng Xie, Zhu Liang Yu, Haiping Lu, Zhenghui Gu, and Yuanqing Li. Motor Imagery Classification Based on Bilinear Sub-Manifold Learning of Symmetric Positive-Defmite Matrices. IEEE Transaclions on Newral Systems and Rehabilitation Engineering, 25(6):504-55 June 2017. ISSN 1558-0210. doi:
10.1109/TNSRE::006225533
Maria Sayu Yamamoto, Florian Yger, and Sylvain Chevallier Subspace oddity optimization on product of Stiefel manifolds for EEG data. ICASSP. Toronto, Canada, June 2021. URL https://hal http:://ccaasssence///eee
hal-03202357
Florian Yger, Maxime Berar, and Fabien Lotte. Riemannian Approaches in Brain- CompuCe Interfaces: A Review. IEEE Transactions Neural Systems and Rehabiitaaion Engineering, 25(10):1753-177 17622 October 2017. ISSN 1558-0210. doi: 201109/TNSRE.201.222222
Paolo Zanini, Marco Congedo, Christian Jutten, Salem Said, and Yannick Berthoumien. Transfer learning: A Riemannian geometry framework with applications to Brain-Compute Interfaces. IEEE Transactions on Biomedical Engineering, 65(5):1107-11 1116, 2018. doi: 10. 2T.109/TTBMB..01.2775555.
A More on the Riemannian geometry of SPD matrices
In this section, we introduce some useful tools study the Riemannian geometry of Symmetric Positive Definite (SPD) matrices. This section completes what was said in section 3
A.1 The exponential and logarithm mapping
The mapping Exp allows going from the tangent space Tx to the manifold Pe and its inverse map Log are useful tools in Riemannian geometry. the case the SPD manifold Pa, equipped with the affine invariant metric given Eq. [ the expression of the Exp and Log mappings are:
Proposition A.1. Given X,Y E Pe and U E TxPe, one has:
Expx (U) = 1/2 exp (x (x-1/2UX-11 112) X/2
(8)
Logx (Y) = X1/2 (x11/2Xx-111 1/2) X1/2
18

---

Published in Transactions on Machine Learning Research (02/2025)
A.2
Distributions on the manifold of SPD matrices
In the sequel and in order to define the conditional probabilities for the Riemannian L-SNE, we need a Gaussian distribution on the space of symmetric positive definite matrices Pc. A generalization of the usual Euclidean Gaussian distribution on Pa was introduced in Said et al. (016). Let us consider Y E Pe and
a > O.
Definition A.2. The Gaussian distribution G(Y,o) ,)) on Pe is defined using its probability density function Py,o and {(a) its normalizing factor as:
8 Y)2
Py o (Y)
exp
((
20
The interpretation of the two parameters Y and are the same as in the Euclidean case: Y is the center of mass of the distribution and o is parameter that controls the dispersion of the distribution. An important property is that the normalizing factor {(a) does not depend on the parameter Y, only on a. An expression of ((o) can be found in proposition of Said et al. (2016).
B Optimizing on a Riemannian manifold
We introduce here basic tools for optimizing Riemannian manifold M. More details on the following tools can be found in Boumal (2023)
B.1 Riemannian Gradient Descent (RGD)
We study the following problem:
min f(x)
REM
where M:M- + R is a smooth function called the cost function. We are going to take inspiration from the Euclidean gradient descent #k+1 = 23 - ag grad f(x)) == 0, However, we need to adapt this algorithm to the Riemannian manifold M. As M is not always a vector space, it is not guaranteed that, for given Fk, 2y - (zg) still lies on the manifold. We need a way to project back this point onto the manifold. This is done using relraction, an operator R2: T.M M which projects a point from given tangent space back onto the manifold. We can then use this retraction to adapt the Euclidean gradient descent:
Algorithm B.1 (Riemannian gradient descent). Given Riemanniaa manifold M, an initial point xo E M and a retraction R on M, we iterate
1s+1 Rz. V((x))
KEN
where, each iteration k, we pick step-size ak.
To choose the step-size og, the most common method is to use line-search where the goal is to minimize the function g(t) = f (Roa ftvf((x))) approximately.
B.2 Geodesic convexity
In a usual Fuclidean setting, when one wants to have some results on an optimization problem, the first reflex chock if the loss function is convex. As we are Riemannian setting, we need adapt the notion of convexily: we introduce the notion of geodesically convex (or g-convex) function:
Definition B.2. A function f: A + is geodesically convex (or g-convex) if A geodesically convex subset of a manifold M and f o: [0, - is convex for each geodesic segment П: [o, 1] + - whose image is in A (with ((0) * T(1)). This means that.
V 1), f(T()) s -1 - ()S((()) +T((())))
(9)
In our case, as the set Pe of symmetric positive definite matrices is convex and complete, it is geodesically convex and therefore, we study the g-convexity of the loss functions encounter.
19

---

Published in Transactions on Machine Learning Research (02/2025)
B.3 Counter example the geodesically convexity of the uub-problem of the Riemannian MDS
(raTT) (1 (iAa() + +taaaa
030
030
Figure 7: Counter example of the g-convexity of the sub-function fy,d of the Riemannian MDS.
In this section, we prove the result stated in Soction D the sub-function fr,a the Riemannian MDS is not g-convex. We recall that this sub-function is defined
fY,d (8(X,Y) 2)
with Y Pe and > For fy, to be g-convex, nceds to satisfy the incquality g for all geodesic T.
However, if consider geodesic that passes through Y, this inequality not satisfied as shown Figure For this plot, chose two random SPD matrices X1 and X₂ and chose Y in the middle of the geodesic going from X1 to X2. We also fixed d 0.1. We can clearly see that for this geodesic T, the inequality g is not satisfied. Therefore, fv,d is not g-convex.
20

---

Published in Transactions on Machine Learning Research (02/2025)
C Derivation of the gradients
C.1 The gradient of the Riemannian distance
We start by expressing the gradient of the squared Riemannian distance given at Eq. 2 for the definition of a Riemannian gradient).
ssee
(oumal
2023
Proposition C.1. Let Y P. be fixed and let consider the following mapping: H: X E P. + 8(X
Then, the Riemannian gradient of H is
VH(X) -2 Logx (Y).
C.2 The gradient of Riemannian MDS
Here, compute the gradient of the Riemannian MDS stress function. We recall that the stress function S' is defined as follows (sec Eq. 3
S(Y,.... Yn) = Σ Y, -
(10)
So, using the gradient of the Riemannian distance as given in Proposition C.1
we have the following gradient:
= 2 3
Di 8(Y,
-1) Logy, (Y₁)
where we recall that Logy, is the Riemannian logarithm as given in Proposition A.1
Proof. We will compute the gradient with respect to Ys. First, we have:
S(Y, ... Yx) Σ (0(Y,, Yk) D terms independent of Ys.
sc(V))
Moreover, we have that the Riemannian gradient of h²(Y)) 6(Y,, Y) is, using C.1
Logy(Y)) (Yi) Vh-(Y) 5(V, Y)
(11)
Therefore,
Dik
-1) Logya (V))
VY, =2 (8(Y, Yk)
Thus, we have the full gradient:
Dut
-1) Logr, (11)
2 Σ Sy, =
6(Y,
1z
We have checked numorically this gradient using the tools in PyManopt (Townscnd al. 2016). The
function check_
gradient check_gradient compares the gradient the first order Taylor expansion of the function. More
details in Boumal (2023
21

---

Published in Transactions on Machine Learning Research (02/2025)
C.3 Derivation of the gradient of Riemannian t-SNE
Let us now compute the gradient of the Riemannian t-SNE. We recall the following notations:
Pily + Pals
(1+ K,P-)-
Púj
T6 Ek + 5(Yr, 31)
2
We want the gradient with respect to Y, of the following cost function:
N
N = Σ
C Σ (L(P)IIQ))=
Pij log
Pij (ij
1=1
jØi
This gradient is given by:
N
Vr = -4 - j=l
Pij 4h Logn; (Y)).
+ 8(Y,, Y,)²
Let us define the following variables to facilitate the computations:
dj 8(Y,, Y2)
Z Σ (1 5(Y,, 21) Kyd
Therefore, we have diş (1+df, )- of d, dys:
To compute the gradient Vv; C, we use the chain rule and the symmeerr
vy Σ OC Vy,dg.
(12)
ddij
Using the computation the gradient of the Riemannian MDS and especially the Eq. 11 we have
Vy Logy, (Y₁)
dij
Therefore, we will focus on the computation of ad.
Using the notations defined above, we have
(1+ de, -1
C=
Pty log Pij
Σ
Ps log
j=
The first part this sum does not depend on dig, thus, we have
ac
N
dai))
Phll Adij
log
Odij
k=1 1z
i Pkol Adis
(- log 2 -log(1 + dhi))
(13)
17
a
a ad.j
Pkat
adis
log Z +
Σ
Phi
log(1 + dla)
R= I
Let compute those two ferms separately:
22

---

Published in Transactions on Machine Learning Research (02/2025)
For the first term, let us start by computing a
log Z: äd:
0 Odiy
1 0 z log Z
7. aclij
1 0
E1+ (1+
= Z Odij
definition of Z
leff
2dg
the only nou-zero the derivative the sum when k=í and I=j.
(1 de )
= 2 (1 + d
l6j
dis
using the fact that 014 1-1
Therefore,
a Odij
log Z -2 + d
pkt 24g + 7
Pkl
For the second term, we start by computing od.,
8 log(1 + +Zz.):
1
2di (1 (l)d))) [d
if and =
log(1 + dhi)= = (1 + dl. dd.)
(1 + d
0d.3
0
otherwise
Thus,
log(1 + di.) = =2d,, + +.t)) d'y)-"p 'Paj-
PRI
Od.z
1=1 1
Stitching all together in Eq. 13
WC have
OC
29:s( (1 + df 1 dd + 2d + d2 = 2d.g(Dis - 91 -
Adg
We can now go back to Eq 12 and we have
(V,) (-1o0011____ dij
[Looy, -4 -4 E (P (1))(1 + +ly)) Logy, (8₁)-
v = 2 Σ (p - 91)) dby)-1
As with the gradient of the MDS, numerically checked this gradient using the check grrccingrrieetttt function of PyManopt.
Proof of the affine invariance property
Proof. Let (X))..Xxx) be points in Pe and an invertible matrix.
1. Let (X,, XN) (RX,RT .... RT. We introduce the two following functions:
S(ZI, ..... Zx) = Σ (8(21, 2,) - Dii)²
i<j
5R(ZI, ..., Zn) Σ
(8(2), 22) Diy)²
<<j
23

---

Published in Transactions on Machine Learning Research (02/2025)
where Di X,) and Di, X(), Then, 8 (resp. Sg) is the function one needs to minimize when wanting solve the Riemannian MDS problem when the initial high-dimensional points are (X1....X,N) (resp. (X1,...,Nv) Since the Riemannian distance is affine invariant (prop.
3.3 we have
Di 8(x x)) = 8(RX,R" RX,R") = X,), Dis. Therefore, S(Z,,.., Zx) SR(Z1, ..., IN) for all N-tuple of SPD matrices (Z1, ..., ZN). So the set of solutions are the same.
Let (Y,,.., Yx) be a solution of the Ricmannian MDS when the initial high-dimensional points are (X1, Xx). Let us denole (Y,,.., Yx) = (RY, R :.... RYvR"). Then, we have for all < J. 8(,, 8(,,, So, with the same notations as in point 1. We have that
S(Y Yn) S(V Yn).
As (Yi,..,, YN) is a solution of our problem, (Y,,.., Yx) too.
24

---

Published in Transactions on Machine Learning Research (02/2025)
E
Reducing a geodesic
13
0.5
Figure 8: An example of two matrices sampled on opposite sides of the cone of SPD matrices. The pink line represents the geodesic linking those two points. They all lie in the Pa cone represented in blue.
12
10
12
Figure 9: Results from t-SNE.
Figure 10: Results from MDS.
Figure 11: Examples of reduced geodesic using both algorithms and their comparison with the geodesic I going from the first to the last reduced point. The dimension of the high-dimensional space from which this example geodesic was sampled is c = 5. This is why, unlike in Figure in this example the MDS seams to lead to better results than the 1-SNE.
In experiment, investigate how a high-dimensional geodesic is transformed using our algorithms. To be more precise, sample random geodesic I in P, going from two randomly sampled points X, and Xx. the start and end point of the geodesic. We want X1 and XN to be "opposite sides' of the cone of SPD matrices. To do that, we will use their spectral decomposition. We start by sampling U uniformly in O(d) [Mezadrii (207].. We then sample the eigen values (A!...AA)) (resp. (AA,,.. ...AA))) )) of X, (resp. Xx). To do that, we sample the first half of the eigen values of X, in 2(3, 4)) and the second half in U((,, 1]). For Xx, we do the opposite, the first half of the cigen values are drawn from 24(10, 1]) and the second half from 4(3 4)). We can then compute X₁ (resp. Xv) X1 Udiag(A),... UU (resp. X Udiag(A (.., We give example of matrices sampled "opposite sides' of the cone in Figure
25

---

Published in Transactions on Machine Learning Research (02/2025)
Once the initial and final point X1 and XN of the geodesic are sampled, we consider (X1. XN), ) N points uniformly spaced on the geodesic r. We reduce these points using our algorithms into (Yi,...YN)) Yn): N points in P2. We compare the trajectory (YI)...Yx) to the geodesic I' in P2 going from Y1 to Yy. To do this, we consider (Yi,..,, Yy) uniformly spaced on the geodesic I' in P2 (therefore Y, = [( (-N-)) [). Then, we compute the distances between the points (Y). and (Yi, .x) that we denote dpoint (Y)), T):
N
Apoint ((Y)),, Σ 80Y, P.)2
(14)
Given this setup, we conducted some experiments varying some parameters: the dimension e of the true geodesic, the noise e on the sampling of the initial points on the true geodesic and the number of points N sampled on the geodesic. The base parameters are 50, 0 (no noise) and N = 100. The perplexity of
the Riemannian t-SNE was set to be N for all the experiments (see Appendix F
F Influence of the perplexity
In this section discuss the perplexity parameter that plays important role in the t-SNE algorithm. We recall that the perplexity is a hyperparameter chosen by the user and that will be used to compute the variance of the Gaussians in the conditional probabilities defined in Eq. 4 The role of the perplexity is to fix the entropy of the conditional probabilities. As explained in (Van Assel et al. 2023). the perplexity can be interpreted as the effective number of neighbors for each data point. It is a way of balancing attention between some local and some global aspects of the data one wants to reduce. is known that varying this parameter can change lot the results one gets from the t-SNE, and there is no silver bullet method to choose the correct perplexity (Wattenberg et al. 2016).
In the Euclidean case, van der Maaten & Hinton (0088) say that "The performance of SNE is fairly robust to changes in the perplexity, and typical values are between 5 and 50". We want to see if this statement is still true in the Riemannian case. For this, we sample two Riemannian Gaussian in Pa where c = 5 is the high-dimensional space, The first Gaussian has for mean X1 = Lo (the identity matrix) and a variance of 01 0.5 and the second one has a mcan of X1 = 10I, and a variance o1 = 0.25. We sample N = 60 points in each Gaussian and concatenate them logether to create the high-dimensional dataset. We then use the Riemannian version of the t-SNE algorithm with different values of the perplexity perp E [5, 10, 15, .... 50, 55]. The initial point of the optimization process is fixed to only see the influence of the perplexity. We show the results in Figure 12 We see that the perplexity has an important impact on the result. When the perplexity is too small, the points the further from the origin have huge coefficients and are all aligned. As the perplexity grows, the coefficients of the points get smaller and smaller, and we are able to see some structure in the dataset appear. Contrary to the Euclidean t-SNE, we need a larger perplexity to have a convineing result. However, when the perplexity is too big, here where it is equal to 120, all the point cloud gets flatten into hyperplane, all the points then aligned in a. 2D plan. Throughout our experiments, found empirically that taking the perplexity to be equal to 3/4 of the total number of points works well. In this example, it corresponds to taking a perplexity of 90, which seems to correctly reproduce the desired structure.
26

---

Published in Transactions on Machine Learning Research (02/202))
Perplexity: 10
Perplixiiyy :
Perplenciyy 30
Perplexity: 40
Perplexity 50
Perplexity: 60
Purplexit: 70
Purplesiiyy
Perplencty 90
Perplexity 100
Perplexiity 110
Perplexity 120
Figure 12: We show in this plot the same high-dimensional point cloud X that is reduced using the Rieman- nian t-SNE algorithm with different choices of the perplexity. The colors represent the two Gaussians from which we sampled X. On the first three plots, the red points are not visible, it is because the blue points have very big coefficients, and the red ones small coefficients, so we are not able to see both of them at the same time.
G Details on the experiments on real datasets
In this section, we give details on the different datasets we used in the experiments of section 7.4 Then, we give some information the computation times of the different algorithms.
G.1 Details the datasets
We give here brief introduction to each of the 7 datasets used in experiments at section 7.4
BCI datasets: 3 of the datasets are coming from Brain Computer Interfaces (BCI) experiments: BNC12014001 (Tangermann et al. 2012), BNCT2014002 (Steyrl et al. 2015) and AlcxMI (Barachant, 2012). These datasets come from MOABB (Aristimunha et al. 2023). The selected 3 motor imagery datasets con- sist of several subjects for each dataset and several sessions for each subject. The two classes each dataset are balanced and the size of the matrices are respectively 22 22, 15 15 and 16 16. We start by applying standard band-pass filter with range [7: 35) Hz for each dataset. To compute the covariance matrices from EEG, we used the Redoit-Wolll shrunk covariance matrix (Ledoit & Woll 2004 to avoid ill-conditioned matrices.
a dataset of air quality monitored from 34 different sites Beijing, China (Hua al., 2021). For each site, six atmospheric pollutants where recorded every hour: CO, NO2, O3, PM PM2.5 and SO2. We used the same preproccssing in (Smith et al.) 2022aa) to get point cloud of 102 covariance matrices of size 6 6. Each covariance matrix has label depending which period it represents: weekdays, weekends or holidays.
AirQuality dataset: This dataset from the Beijing Municipal Monitoring Center.
FPHA datasct: This is a dataset of vidco sequences in first person view used as benchmark for estimating hand actions. The original datasel (Garcia-Hernandee Hernando et al. 2018) consist of 1,175 video sequences distributed across 45 distinct categories. We chose 8 of those categories for a total of 108 videos. The categories chosen are drink_ give_card, give_cciiil coin, high_five five, open_juice___ppeeoeeeppppppoeees _ootlee, open_ ahen_peanut. butter, put_sugar, write_pen. As in (Wang et al. 2023), we converted the video sequences into 63 63 SPD matrices repre- senting how the 3D coordinates of 21 hand joints are linked together.
27

---

Published in Transactions on Machine Learning Research (02/2025)
The last dataset is obtained from a simulation of an industrial process known the Tennessee Fastman Process (TEP) (Downs & Vogel 1993). This dataset is used as a benchmark dataset testing and comparing various anomaly detection methods. We used the preprocessing described in Smith et al. (202bb) to get a dataset containing 420 SPD matrices of size 52 x 52.
TEP dataset:
The setup Once the datasets are reduced using the different algorithms, we want to compute the trustworthiness coefficient. For this, we need choose hyperparameter k for the number of near- est neighbors. As the different dataset. have different numbers of points N, in order to compare them choose & to be certain percentage of the total mumber of points N. We choose for our study k E {0.05,, 0.1,,, 0.2N, 0.3N, 0.4N, 0.5N) one needs k S N/2. All the results are given at Appendix H
28

---

Published in Transactions on Machine Learning Research (02/2025)
H
Detailed table of the results of the different algorithms on the different datasets.
Trustworthines with the neighborhood the total number points
Dataset
Algorithm
0.05
0.1
0.2
0.3
0.4
0.5
Riem-Riem t-SNE
0.9751
0.9822
0.9890
0.9924
0.9894
0.9899
Euc-Eue I-SNE
0.8785
0.8776
0.8819
0.8902
0,8771
0,8519
Riem-Eue t-SNE
0.9814
0.9834
0.9895
0.9895
0.9800
0.9739
Fiem-Ricen MDS
0.9783
0.9836
0,9919
0.9950
0.9948
0. 0.9935
AirQuality
Euc-Eue MDS
0.8747
0.8737
0.8772
0.8835
0.8657
0.8332
Riem Fuc MDS
0.9811
0.9834
0.9914
0.9933
0.9945
O. 0.9932
UMAP
0.9549
0.9580
0.9627
0.9567
0.9395
0.9-139
PGA
0.9665
0.9735
0.9885
0.9949
0.9946
0.9945
Riem PCA
0.9381
0.9503
0.9683
0.9750
0.9745
0.9706
Riem-RRies t-SNE
0.9479
0.9439
0.9461
0.9435
0.9414
0.9311
Euc-Eue E-SNE
0.7622
0.7551
0.7592
0.7620
0.7549
0.7351
Riem-E t-SNE
0.9653
0.9530
0.9366
0.9289
0.9210
0.9052
Riem-Riem MDS
0.9127
0.9072
0.9087
0.9092
0.9064
0.8916
AlexMI
Euc- Euc MDS
0.7585
0.7577
0.7668
0.7747
0.7754
0.7557
Riem-Fae MDS
0.9166
0.9143
0.9133
0.9127
0.9100
0.8965
UMAP
0.9252
0.9123
0.8930
0.8751
0.8577
0.8286
PGA
0.8851
0.9018
0.9204
0.9299
0.9341
0.9332
Riem PCA
0.7804
0.7924
0.8189
0.8316
0.838 1
0.8268
Riem- Riem t-SNE
0.9488
0.9543
0.9605
0.9623
0.9599
0.9419
Euc-Eur t-SNE
0.7075
0.6937
0.6938
0.6972
0.6961
0.6684
Riem- Euc t-SNE
0.9564
0.9454
0.9351
0.9219
0.9036
0.8376
Riem-Rieen MDS
0.9303
0.9333
0.9367
0.9366
0.9327
0.9205
BNC120140M
Euc- Euc MDS
0.7048
0.7071
0.7160
0.7203
0.7175
0.6891
Riem-Euc MDS
0.9304
0.9334
0.9360
0.9355
0.9310
0.9182
UMAP
0.9408
0.9316
0.9194
0.9035
0.8782
0.8062
PGA
0.9231
0.9319
0.9442
0.9511
0.9528
0.94448
Riem PCA
0.7762
0.7825
0.7945
0.8011
0,8008
0.7815
Riem--Riee t-SNE
0.9583
0.9629
0.9665
0.9676
0.9648
0.9549
Eue- Euc t-SNE
0.7370
0.7239
0.7191
0.7155
0.7067
0.6663
Riem--Eue t-SNE
0.9672
0.9553
0.9451
0.9335
0.9177
0.8808
Riem-Rient MDS
0.9376
0.9398
0.9429
0.9426
0.9386
0.9288
BNC12014002
Euc-Eue MDS
0.7429
0.7428
0.7437
0.7434
0.7346
0.6992
Riem- Fue MDS
0.9375
0.9406
0.9438
0.9434
0.9391
0.9279
UMAP
0.9476
0.9358
0.9166
0.8964
0.8731
0.8260
PGA
0.9317
0.9403
0.9527
0.9590
0.9610
0.9575
Riem PCA
0.8338
0.8455
0.8623
0.8729
0.8735
0.8590
Riem-
Riem-RRie Riem t-SNE
0.9652
0.9685
0.9704
0.9763
0.9771
0.9707
Fuc- Fmr t-SNE
0.9114
0.8771
0.8205
0.7711
0.7143
U.6034
Riem- Euc t-SNE
0.9845
0.9722
0.9645
0.9616
0.9587
0.9348
Riem-Riem MDS
0.8955
0.8889
0.9033
0.9269
0.9464
0.9610
FPHA
Euc- MDS
0.8665
0.8477
0. 0.7935
0.7345
0.6906
O. 0.6023
Riem- Euc MDS
0.9525
0,9500
0.9462
0.9439
0.9441
0.9395
UMAP
0.9695
0.9581
0.9486
0.9369
0.9268
0.8989
PGA
0.9512
0.9551
0. 0.9626
0.9686
0.9729
0.9766
Riem PCA
0.8490
0.8247
0.8174
0.8151
0.8080
U.7843
Riem-Riem t-SNE
0.9842
0.980
0.9780
0.9804
0.9761
0.9750
Euc-Eur t-SNE
0.8892
0.8849
0.9043
0.9264
0.9379
0.9465
Riem-Fit t-SNE
0.9938
0.9833
0.9797
0.9686
0.9328
0.9242
Riem-Riem MDS
0.9732
0,9621
0.9755
0.9844
0.9901
0,9949
TEP
Euc-Eue MDS
0.8902
0.8931
0.9061
0.9146
0.9186
0.9075
Riem Euc MDS
0.9945
0.9916
0.9921
0.9930
0.9927
0.9890
UMAP
0.9788
0.8448
0.7128
0.6287
0.5878
0.4989
PGA
0.9743
0.9736
0.9767
0.9781
0.9849
0.9909
Riem PCA
0.91413
0.8998
0.91-13
0.9355
0.9419
0.9575
29

---

Published in Transactions on Machine Learning Research (02/2025)
Plot of the results of the dataset AirQuality
Riemannian-Riemanniaa t-SNE
Euclidcan- t-SNE
coaiaannian-Euulidden t-SNE
300
Ricmannian-RRemaniinn MDS
Euclidean- MDS
Ricnamnian-Buclidean MDS
0.5
Holidays
-0.
Weekdays
Weekends
PGA
UMAP
Figure 13: Reduced point cloud of the dataset AirQuality using the different algorithms.
30


Reviews:
[{"review_id": "3dBrAqhCHX", "reviewer": "Reviewer_Zj4X", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: a new method for visualizing SPD while preserving their curvature as compared with Euclidean approaches. Extensive synthetic and real-data evaluations of the proposed method.\n\nWeakness: The writing requires substantial improvement. And this work lacks sufficient detail and guarantee for the convergence results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Riemannian versions of t-SNE and Multidimensional Scaling for visualizing Symmetric Positive Definite (SPD) matrices while preserving their manifold curvature. The experimental results demonstrate that these methods successfully maintain the geometric properties of SPD Gaussians and geodesics on both synthetic and real datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The paper would benefit from more professional use of parenthetical in-text citations to improve readability; for instance, in the first paragraph, it should be “in process control (Willjuice Iruthayarajan & Baskar, 2010)”. Additionally, Figure 1 lacks a main caption and figure number, with only subcaptions labeled (a) and (b) currently provided.\n\nAnother concern is the performance impact of Riemannian gradient descent on high-dimensional product manifolds when N is large. The paper should address potential issues related to convergence speed and guarantees, as well as any implications for practical scalability. The current Section 6 on convergence is brief and does not sufficiently address the performance or stability of the gradient descent algorithm. The DCA algorithm is also mentioned, but it does not appear to be implemented in the experiments; clarification on this would be helpful. Additionally, while the authors state that the method will converge to a critical point, it is unclear whether this point is guaranteed to be a minimizer, which is important for both the MDS and t-SNE approaches. The discussion on computational time could also be expanded in the numerical results section, including practical details such as the number of iterations typically required.\n\nIn Figure 3, the behavior of Riemannian t-SNE seems unaffected by changes in dimension or noise level. Could the authors provide a more intuitive explanation for this stability? It seems unlikely to be solely due to the robustness of the Riemannian t-SNE method, as it appears remarkably invariant across varied conditions.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aYqhOWb5qF", "reviewer": "Reviewer_bzCj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n* Nice intro, with complete set of references.\n* Clean formulations, propositions, and discussion with a variety of visualizations. \n* Fair comparisons with a number of well-known competitors.\n\n# Weaknesses\n* No significant weaknesses. A couple of comments in the below Requested Changes section regarding clarity on metrics.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors formulate two dimensionality reduction algorithms, Riemannian Multidimensional Scaling, and Riemannian t-SNE. They provide a proof that both algorithms are invariant to affine transformations, which is a major advantage when comparing data across sensor array instances that vary spatially. Both algorithms are solved via Riemanian gradient descent, and for Riemannian MDS a discussion on convergence is provided.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "# Requested Changes\n* Sec 7.3 \"Reducing geodesics\": the observation that MDS \"seems to do a better job than the t-SNE\" seems at odds with the figures in that section, which shows t-SNE being more robust to noise, dimension, and N-points. I see in Figure 11 what appears to be MDS \"fitting\" better to the manifold but I'm having trouble reconciling this with Fig 3. A more unified conclusion (not split across main/appendix) might help clarify this result.\n* You describe trustworthiness in one sentence as \"It ranges within [0, 1], and measures how well the ranks of the k-nearest neighbors of each point are preserved.\" I think it's worth adding a little more detail since this is the key performance comparator. Is this the rank of the distance /adjacency matrix?\n\n# Comments/Observations (not critical to recommendation)\n* The paper is very clean so I understand not wanting to expand too much discussion. But comparison / contextualization with information geometry terminology may be interesting (Fisher Matrix, Natural Gradient Descent)\n* Figure 4 is a very fair visualization; is there a way to somehow capture the falloff of PGA and UMAP that is shown in Fig 5? \n* Any reason why you didn't try UMAP with a Riemannian geodesic-based distance?\n* Does Sec 8 deserve its own section as opposed to simply adding it to the competitors above? (I appreciate the comparison, just asking)", "broader_impact_concerns": "no concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9F4268CNys", "reviewer": "Reviewer_asDR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors provide a comprehensive description of the currently commonly used visualization methods and showcase how they can result in distortion of the underlying data structure for the case of SPD matrices. \n- The presentation of the proposed algorithms is clear. Additionally, the investigation of their convergence and invariance properties adds to the completeness of this work.\n- The experimental results provide significant evidence regarding the benefits of the proposed extension of t-SNE and MDS for SPD matrices.\n\nWeaknesses\n- There is limited discussion and comparison of the convergence speed and computational resources of the proposed algorithms compared to their Euclidean counterparts. Are there any computational tradeoffs for the proposed methods or they are the same as the original visualization algorithms?\n- In the qualitative results of Figure 2, only the full Riemannian version is shown and compared with the Euclidean version. Since in the full Riemannian version the distance between the projected points is given by the geodesic presented in Figure 1b, it might be still difficult for a user to inspect the relative distances between these points since the geodesic distance cannot be intuitively inferred by visual inspection. Is it possible that the Riemannian-Euclidean version is a better tradeoff for visualization purposes although it might be worse in the trustworthiness metric?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors address a significant limitation of the commonly used visualization techniques, t-SNE and Multidimensional Scaling (MDS), when applied to Symmetric Positive Definite matrices (SPD). Specifically, they observe that due to the curvature of the Riemannian manifold of SPD matrices, t-SNE and MDS distort the relative distances of the visualized points since they project them into a flat Euclidean space. To solve this problem they propose a novel extension to the t-SNE and Multidimensional Scaling (MDS) algorithms that respects the Riemannian structure of the manifold where the SPD matrices live. In addition to a detailed description of the proposed extension, the authors provide theoretical results regarding the invariance properties of the proposed algorithms and their convergence. Finally, the authors offer qualitative and quantitative experimental results showcasing the benefits of using the proposed algorithms compared to their Euclidean counterparts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Some recommended changes of medium importance:\n- A more detailed discussion of the time and memory resources required by the proposed algorithm compared to the Euclidean counterparts will add to the completeness of the presentation of the proposed algorithms.\n- Regarding the second weakness: it would be valuable to provide a qualitative comparison of the visualization using the full Riemannian versions of the algorithms and the Riemannian-Euclidean versions.", "broader_impact_concerns": "There are no concerns on the ethical implications of the work that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces Riemannian variants of the MDS and t-SNE algorithms, enabling visualization of SPD matrices from higher dimensions to 2x2 matrices. Additionally, it discusses which metric is more suitable. Overall, the reviewers appreciated the paper and its contributions.

However, reviewers Zj4X and asDR have raised concerns regarding certain promised revisions that have not been addressed in the paper, even though they were acknowledged in the comments.

1. The authors committed to providing visualizations that illustrate the qualitative differences between the Euclidean, full Riemannian, and Riemannian-Euclidean approaches.
2. The paper lacks a detailed discussion on the convergence speed and theoretical insights of the implemented algorithms. Additionally, while the response was useful, it is somewhat brief and lacks concrete details such as a quantitative comparison of convergence rates and computational times.

It would be beneficial if these points could be included in the revised draft version.
Decision: Accept (poster)