Cartan Networks: Group theoretical Hyperbolic Deep Learning

## Abstract

Abstract Hyperbolic deep learning leverages the metric properties of hyperbolic spaces to develop efficient and informative embeddings of hierarchical data. Here, we focus on the solvable group structure of hyperbolic spaces, which follows naturally from their construction as symmetric spaces. This dual nature of Lie group and Riemannian manifold allows us to propose a new class of hyperbolic deep learning algorithms where group homomorphisms are interleaved with metric-preserving diffeomorphisms. The resulting algorithms, which we call Cartan networks , show promising results on various benchmark data sets and open the way to a novel class of hyperbolic deep learning architectures.

## 1 Introduction

The concept of distance is the core of machine learning and pattern recognition. While much classical machine learning can be recast as learning distances directly from data (e.g. [1]), recent developments have pointed out that common data structures, such as trees and graphs, cannot easily be accommodated within Euclidean spaces, requiring a more radical rethink of the geometry of data spaces. In this context, the $n$-dimensional hyperbolic space $\mathbb{H}^{n}$ has received significant attention as a suitable space in which to embed hierarchically structured data [2], spurring a productive line of research combining hyperbolic geometry with various deep learning architectures [3, 4, 5, 6, 7, 8, 9]. These so-called hyperbolic neural networks have found applications in fields as diverse as neuroscience [10], single-cell transcriptomics [11], and recommender systems [12].

Geometrically $\mathbb{H}^{n}$ is a $n$-dimensional hyperboloid, namely the quadric locus $\sum_{i=1}^{n}X_{i}^{2}-X_{n+1}^{2}=-1$ in $\mathbb{R}^{n+1}$. It is also a coset manifold, namely the quotient of a Lie Group modulus a maximal Lie subgroup, $\mathbb{H}^{n}\simeq\mathrm{SO}(1,n)/\mathrm{SO}(n)$, and more specifically a symmetric space. The study and classification of symmetric spaces is one of the monumental achievements of the French mathematician Èlie Cartan [13, 14, 15, 16]. The non-compact symmetric spaces are all metrically equivalent to a corresponding solvable Lie group $S$ of the same dimension, a mathematical result that was discovered and developed in the context of Supergravity Theory
[17, 18, 19, 20, 21, 22], and amply reviewed
and systematically reorganized for machine learning applications in [23].

This result, to our knowledge not known so far in the machine learning literature, has significant algorithmic consequences, which we explore in this work and will develop in a more general theoretical paper in preparation [24]. The dual nature of group and Riemannian manifold of the hyperbolic space $\mathbb{H}^{n}$ enables us to construct a deep learning framework based entirely on intrinsic geometric operations, where group homomorphisms are interleaved with metric-preserving diffeomorphisms in creating a powerful function approximation machine. Importantly, the nonlinearities naturally arising from group-theoretic exponential and logarithmic maps give flexibility to the framework, which achieves promising results on benchmark data sets when compared with similar-sized standard deep learning architectures.

The main contributions of this work are as follows:

- •
We highlight the metric equivalence of the hyperbolic space with a solvable Lie group to exploit the group structure as a tool in architecture design.
- •
We propose a new deep learning architecture where each layer is a solvable Lie group $S_{i}$ and where the map from layer $i$ to layer $i+1$ can be represented as a combination of homomorphisms from the solvable Lie group $S_{i}$ to the next one $S_{i+1}$ and the isometries of $S_{i+1}$. The construction is general for any symmetric space, and we implement it for the hyperbolic space $\mathbb{H}^{n}$.
- •
We extensively benchmark these architectures on real and synthetic datasets, showing competitive or better performance w.r.t. Euclidean and standard hyperbolic neural networks.

### 1.1 Previous literature

Early works in hyperbolic deep learning focused on hyperbolic embeddings for hierarchical data. [2] introduced Poincaré embeddings, showing superior hierarchical representation compared to Euclidean embeddings. Ganea et al. ([3]) and subsequent works [4, 5, 6, 9], extended hyperbolic geometry to deep learning by developing hyperbolic neural networks, using Möbius operations [25]. Various generalizations of hyperbolic networks have been explored. Convolutional networks [26, 27], graph neural networks [7], and attention mechanisms [8] hyperbolic variants were introduced to handle different datasets.

Lie groups and Lie algebras are often studied in deep learning for their equivariance properties [28, 29, 30]. Architectures based on semisimple Lie algebras have been introduced under the name Lie Neurons [31], focusing on making these networks adjoint-equivariant.

The notion that $\mathbb{H}^{n}$ is isometric to a Lie group was explored in the context of probability distributions and Frechét means [32]. However, the isometry between symmetric spaces and solvable groups was not highlighted in full generality, and the knowledge was never applied to the study of deep learning architectures.

## 2 Theoretical preliminaries

We will assume basic knowledge of Lie groups (see App. [A](https://arxiv.org/html/2505.24353v1#A1) for a brief introduction).

#### Solvable groups and Cartan subalgebras.

###### Definition 2.1 (Subalgebra commutator) .

Let $\mathfrak{h_{1},h_{2}}$ be two subalgebras of $\mathfrak{g}$. Their commutator subalgebra is

$$ $[\mathfrak{h}_{1},\mathfrak{h_{2}}]:=\{[h_{1},h_{2}]\in\mathfrak{g}\>|\>h_{1} \in\mathfrak{h_{1}},\>h_{2}\in\mathfrak{h}_{2}\}$ (1) $$

where $[\>\cdot\>,\>\cdot\>]$ denotes the Lie bracket of the algebra.

###### Definition 2.2 (Derived series) .

Let $\mathfrak{g}$ be a Lie algebra. Its *derived series* is the series

$$ $\mathfrak{g}^{(0)}=\mathfrak{g},\quad\quad\mathfrak{g}^{(n+1)}=[\mathfrak{g}^{ (n)},\mathfrak{g}^{(n)}]\quad\forall n\in\mathbb{N}_{+}$ (2) $$

The derived series is a decreasing sequence of ideals in the algebra.

###### Definition 2.3 (Solvable algebras) .

A Lie algebra $\mathfrak{g}$ is *solvable* if its derived series is eventually 0, that is to say, if

$$ $\exists n\in\mathbb{N}\quad\mathrm{s.t.}\quad\mathfrak{g}^{(n)}=0$ $$

A Lie group is solvable if its Lie Algebra is solvable.

In practice, solvable groups are best understood in terms of their matrix representation. In fact,

###### Theorem 2.1 (Lie’s theorem [ 33 ] ) .

Let $\mathfrak{g}$ be a solvable subalgebra of the general linear group $\mathfrak{gl}_{V}$. Then there exists a basis of $V$ with respect to which $\mathfrak{g}$ is made of upper triangular matrices.

This theorem shows we can think of solvable groups as upper-triangular matrix Lie groups.

###### Definition 2.4 (Cartan subalgebras) .

Let $\mathfrak{g}\subseteq\mathfrak{gl}_{n}(\mathbb{R})$ be a matrix Lie algebra consisting of upper triangular matrices. Its *Cartan subalgebra* is the subspace of diagonal matrices.

###### Definition 2.1 (Subalgebra commutator) .

###### Definition 2.2 (Derived series) .

###### Definition 2.3 (Solvable algebras) .

###### Theorem 2.1 (Lie’s theorem [ 33 ] ) .

###### Definition 2.4 (Cartan subalgebras) .

#### Symmetric spaces.

Let $G$ be a Lie group and $H$ a normal subgroup, $\mathfrak{g}$ and $\mathfrak{h}$ the corresponding Lie algebras. A coset manifold $\mathrm{G/H}$ is a symmetric space if and only if there is an
orthogonal decomposition of $\mathfrak{g}$, as a vector space, as follows:

$$ $\mathfrak{g}\,=\,\mathfrak{h}\oplus\mathfrak{m}\quad;\quad\left\{\begin{array} []{lcr}\left[\mathfrak{h}\,,\,\mathfrak{h}\right]&\subset&\mathfrak{h}\\ \left[\mathfrak{h}\,,\,\mathfrak{m}\right]&\subset&\mathfrak{m}\\ \left[\mathfrak{m}\,,\,\mathfrak{m}\right]&\subset&\mathfrak{h}\\ \end{array}\right.$ (3) $$

One interesting class of non-compact symmetric spaces is given by

$$ $\mathcal{M}^{[r,r+p]}=\dfrac{\text{SO}(r,r+p)}{\text{SO}(r)\times\text{SO}(r+p )},\>\>r>0,\,p\geq 0$ (4) $$

This family of manifolds is easily tractable thanks to the metric equivalence between these and an appropriate solvable Lie
group, studied in the context of theoretical physics in [23],

$$ $\mathcal{M}^{[r,p]}\simeq\mathrm{Exp}\left[{\mathrm{Solv}_{[r,p]}}\right]$ $$

where we denote $\mathrm{Solv}_{[r,p]}$ the solvable Lie algebra of the solvable Lie subgroup $S_{[r,p]}\subset\mathrm{SO}(r,r+p)$ with $r$ Cartan generators. For $r=1$ we realize the hyperbolic space $\mathbb{H}^{p-1}\simeq\mathcal{M}^{[1,\,1+p]}$ (where $\simeq$ denotes a metric equivalence).

#### Solvable coordinates of hyperbolic space.

The hyperbolic space $\mathbb{H}^{n}$ (and all the other non-compact symmetric spaces) is metrically equivalent to an appropriate solvable Lie group, whose structure was never used in statistical learning.

$$ $\mathcal{M}^{[1,\,1+q]}=\dfrac{\text{SO}(1,\,1+q)}{\text{SO}(1+q)}\simeq \mathbb{H}^{q+1}$ (5) $$

As this manifold is a Lie group, we will parametrize the manifold with a set of coordinates

$$ $\Upsilon=\left[\Upsilon_{1},\boldsymbol{\Upsilon_{2}}\right]^{\intercal}=\left [\Upsilon_{1},\Upsilon_{2,1},\,\dots,\,\Upsilon_{2,q}\right]^{\intercal},$ (6) $$

called the solvable coordinates of the manifold [23], and we will use them for our formulation of hyperbolic learning. We separate the first component $\Upsilon_{1}$ (which we will call the Cartan coordinate since it corresponds to the unique generator of the Cartan subalgebra) from the others (which we call the paint coordinates following [23]).
This choice of coordinate system for the hyperbolic space is convenient for many reasons discussed throughout this work. A convenient property of all non-compact symmetric spaces is that they can be easily parametrized by a single chart with domain $\mathbb{R}^{n}$, thus bypassing the numerical problems of the Lorentz and Poincaré models exposed by [34].

#### Group operation.

The group operation is the matrix multiplication between the solvable group elements. Given two points $\Upsilon,\Psi\in\mathcal{M}^{[1,\,q]}$, the group operation is

$$ $\Psi*\Upsilon=\begin{bmatrix}\Upsilon_{1}+\Psi_{1}\\ \boldsymbol{\Upsilon_{2}}+\mathrm{e}^{-\Upsilon_{1}}\boldsymbol{\Psi_{2}}\end{bmatrix}$ (7) $$

Similarly, the inverse element is given by $\Upsilon^{-1}=\left[-\Upsilon_{1},-\mathrm{e}^{\Upsilon_{1}}\boldsymbol{
\Upsilon_{2}}\right]^{\intercal}$. The matrix representative is expressed in Eq. [25](https://arxiv.org/html/2505.24353v1#A2.E25) in App. [B](https://arxiv.org/html/2505.24353v1#A2), alongside a deeper discussion of the solvable coordinates parametrization, and the identity element is the point $\Upsilon=\boldsymbol{0}$.

The group operations can be expressed in terms of the non-solvable Poincaré ball coordinates (see Eq. [32](https://arxiv.org/html/2505.24353v1#A2.E32) in App. [B](https://arxiv.org/html/2505.24353v1#A2.SS0.SSS0.Px1) for the transition function) or other coordinate systems.

## 3 Learning in symmetric spaces

### 3.1 General principles of Cartan networks

We propose creating a network whose layers are a sequence of solvable groups $\{S_{i}\}_{i=1}^{N}$.

The map from layer $i-1$ to layer $i$ is the composition of a group homomorphism with an isometry of the target space. Specifically, each transformation consists of a homomorphism (a map between groups that preserves the group operation):

$$ $h_{i}(W_{i})\,:\,S_{i-1}\longrightarrow S_{i}$ (8) $$

from one solvable Lie group to the next, defined intrinsically by parameters $W_{i}$, composed with an isometry (a metric-preserving, and thus distance-preserving, map) acting on $S_{i}$:

$$ $\varphi_{i}(\theta_{i})\,:\,S_{i}\longrightarrow S_{i}$ (9) $$

parametrized by $\theta_{i}$. In the following, we develop the architecture in the case of the hyperbolic space, so $S_{i}\simeq\mathcal{M}^{[1,1+q_{i}]}$.

Figure: Figure 1: Structure of Cartan network (binary classification). This figure illustrates the composition of the proposed Cartan networks between symmetric spaces. By alternating homomorphisms and isometries, our networks parametrize a larger class of maps while only using geometrically motivated functions.
Refer to caption: extracted/6493579/figures/layers_s2.png

### 3.2 Maps between hyperbolic spaces

#### Isometries.

The set of isometries of $\mathcal{M}^{[1,\,1+q]}$ into itself is given by $\text{SO}(1,\,1+q)$ (these are parameterized in terms of the Poincarè ball coordinates by [32, 35]). These isometries are a composition of three distinct isometries, namely the paint rotation (an orthogonal transformation of the paint coordinates $\boldsymbol{\Upsilon}_{2}$), the group operation, and the fiber rotation, which mixes Cartan and paint coordinates. Of these, only the paint rotation is also a homomorphism of the group into itself. Refer to App. [D](https://arxiv.org/html/2505.24353v1#A4) for a detailed derivation.

A general isometry $\varphi\in\text{Iso}(\mathcal{M}^{[1,\,1+q]})$ can be parametrized as

$$ $\varphi(\Upsilon;Q,\beta,u)=R_{\boldsymbol{u}}\left(\begin{bmatrix}\beta_{1}\\ \boldsymbol{\beta}_{2}\end{bmatrix}*\begin{bmatrix}1&0\\ 0&Q\end{bmatrix}\begin{bmatrix}\Upsilon_{1}\\ \boldsymbol{\Upsilon}_{2}\end{bmatrix}\right)$ (10) $$

where $Q\in\text{SO}(q)$, $\beta\in\mathcal{M}^{[1,\,1+q]}$, $\boldsymbol{u}\in\mathbb{S}^{q+1}$ is a parameter on the n-sphere, and
the fiber rotation $R_{\boldsymbol{u}}$ is given by

$$ $R_{u}(\Upsilon)=\begin{bmatrix}-\log\left(-\dfrac{1}{2}(\mathrm{e}^{\Upsilon_{ 1}}(1+|\boldsymbol{\Upsilon}_{2}|^{2})+\mathrm{e}^{-\Upsilon_{1}})\,(1+u_{0})+ \mathrm{e}^{-\Upsilon_{1}}u_{0}-\boldsymbol{\Upsilon}_{2}\cdot\boldsymbol{u}^{ \prime}\right)\\ \boldsymbol{\Upsilon}_{2}-\left(\dfrac{\boldsymbol{\Upsilon}_{2}\cdot \boldsymbol{u}^{\prime}}{1+u_{0}}+\dfrac{1}{2}(\mathrm{e}^{\Upsilon_{1}}(1+| \boldsymbol{\Upsilon}_{2}|^{2})-\mathrm{e}^{-\Upsilon_{1}})\right)\boldsymbol{ u}^{\prime}\\ \end{bmatrix}$ (11) $$

having defined $\boldsymbol{u}=[u_{0},u_{1},\dots,u_{q}]^{\intercal}\in\mathbb{S}^{q+1}$, and $\boldsymbol{u}^{\prime}=[u_{1},\dots,u_{q}]^{\intercal}$.

#### Solvable group homomorphisms.

The set of group homomorphisms is given by the linear maps between the corresponding solvable algebras that preserve the group structure. These are not linear in the coordinates in general, but the equations simplify in the $r=1$ case.

###### Theorem 3.1 .

Let $h\in\text{Hom}(\mathcal{M}^{[1,\,1+q]},\,\mathcal{M}^{[1,\,1+p]}),\>p,q\geq 1$, $\mathrm{dim}(h(\mathcal{M}^{[1,1+q]}))>1$. Then
there exist a unique $W\in\mathbb{R}^{p\times q}$ and $\boldsymbol{b}\in\mathbb{R}^{p}$ such that

$$ $h(\Upsilon)=\begin{bmatrix}\Upsilon_{1}\\ W\boldsymbol{\Upsilon_{2}}+(1-e^{-\Upsilon_{1}})\,\boldsymbol{b}\end{bmatrix}$ (12) $$

Conversely, for every pair $(W,\boldsymbol{b})\in\mathbb{R}^{p\times q}\times\mathbb{R}^{p}$, the map $h$ defined by ([12](https://arxiv.org/html/2505.24353v1#S3.E12)) is a homomorphism.

The proof of the theorem is in App. [E](https://arxiv.org/html/2505.24353v1#A5), and relies on defining the homomorphisms on the algebra generators. Notice that we can also use a non-square $W$ to change the manifold dimension.

###### Theorem 3.1 .

#### General linear layer.

We want to define the linear layer as a composition of homomorphisms from a solvable
group to the next one and isometries from the group to itself, as discussed in Sec. [2](https://arxiv.org/html/2505.24353v1#S2). By combining Eq. [10](https://arxiv.org/html/2505.24353v1#S3.E10)-[12](https://arxiv.org/html/2505.24353v1#S3.E12), we find the hyperbolic linear layer as the transformation $f_{\text{lin}}:\mathcal{M}^{[1,\,1+q]}\to\mathcal{M}^{[1,\,1+r]}$ given by

$$ $f_{\text{lin}}(\Upsilon)=R_{\boldsymbol{u}}\left(\begin{bmatrix}\beta_{1}\\ \boldsymbol{\beta}_{2}\end{bmatrix}*\begin{bmatrix}\Upsilon_{1}\\ W\boldsymbol{\Upsilon}_{2}+\boldsymbol{b}\end{bmatrix}\right)$ (13) $$

where $W\in\mathbb{R}^{r\times q}$, $\boldsymbol{b}\in\mathbb{R}^{r}$, $\beta\in\mathcal{M}^{[1,\,1+r]}$ and $\boldsymbol{u}\in\mathbb{S}^{r+1}$, which are the parameters that are learned during training. Notice that the orthogonal matrix $Q$ of Eq. [10](https://arxiv.org/html/2505.24353v1#S3.E10) has been absorbed in the matrix $W$.

Our formulation of hyperbolic layers is different from previous iterations [3, 4], which rely on Riemannian logarithmic and exponential maps. The hyperbolic linear layers are usually defined as

$$ $y=\exp_{b}\left(\text{P}_{\textbf{0}\to b}W\log_{\textbf{0}}(x)\right)$ (14) $$

where $\exp_{b}:T_{b}\mathcal{M}\to\mathcal{M}$ is the Riemannian exponential map in the point $b\in\mathcal{M}^{[1,1+q]}$, $\log_{\textbf{0}}:\mathcal{M}\to T_{\textbf{0}}\mathcal{M}$ is the Riemannian logarithmic map in the origin, $\text{P}_{\textbf{0}\to b}$ is the parallel transport from 0 to $b$, and $W\in\mathbb{R}^{(q+1)\times(q+1)}$.

As any $\varphi\in\text{Iso}(\mathcal{M}^{[1,1+q]})$ can be written (from the Cartan–Ambrose–Hicks theorem [36]) through the Riemannian exponential map substituting $W$ with $Q\in\mathrm{SO}(1+q)$ in Eq. [14](https://arxiv.org/html/2505.24353v1#S3.E14), we find that existing architectures parametrize all the isometries of the space. However, since $W$ is a generic linear operation on the coordinates, it is a generic nonlinear operation on the algebra, and hence breaks the symmetries between layers.

Each application of a hyperbolic linear layer (Eq. [13](https://arxiv.org/html/2505.24353v1#S3.E13)) mixes the Cartan coordinate and the fiber coordinates through the fiber rotation. The first coordinate of Eq. [11](https://arxiv.org/html/2505.24353v1#S3.E11) is then exponentiated in the following layer, adding nonlinearities to the expression, so stacking hyperbolic layers increases expressivity even without the addition of an activation function.

### 3.3 Hyperbolic softmax

#### Hyperbolic hyperplanes.

In analogy to Euclidean space, we consider the set of geodesically complete submanifolds that separate $\mathcal{M}^{[1,\,1+q]}$ into two halves. These manifolds are the same subspaces as the Poincaré hyperplanes of [3, 4] and are introduced as geodesically convex hulls in [37]. They are given by all possible isometric immersions of $\mathcal{M}^{[1,\,q]}$ into $\mathcal{M}^{[1,\,1+q]}$.

The general equation for these hyperplanes in solvable coordinates is as follows:

$$ $\begin{gathered}\text{H}_{\alpha,\beta,\boldsymbol{w}}=\{\Upsilon\in\mathcal{M }^{[1,\,1+q]}\;s.t.\\ h_{\alpha,\beta,\boldsymbol{w}}(\Upsilon)=\alpha\,e^{-\Upsilon_{1}}+\langle \boldsymbol{w},\,\boldsymbol{\Upsilon}_{2}\rangle+\beta\,e^{\Upsilon_{1}}\left (1+|\boldsymbol{\Upsilon}_{2}|^{2}\right)=0\}\\ \text{with: }|\boldsymbol{w}|^{2}-4\,\alpha\,\beta>0,\;\;\alpha,\,\beta\in \mathbb{R},\,\boldsymbol{w}\in\mathbb{R}^{q}\end{gathered}$ (15) $$

where $\langle\;,\,\rangle$ is the Euclidean scalar product. For details on the derivation, refer to App. [F](https://arxiv.org/html/2505.24353v1#A6).

Figure: Figure 2: Hyperplanes in $\mathcal{M}^{[1,2]}\simeq\mathbb{H}^{n}$. This figure illustrates an example of the hyperplanes that divide the hyperbolic space. For $q=1$, they correspond to the set of all the geodesics. (a) In the Poincarè disk model, the geodesics consist of all arcs of Euclidean circles orthogonal to the disk boundary, plus all the disk diameters. (b) Geodesics obtained by applying the isometry given by left multiplication (Eq. [7](https://arxiv.org/html/2505.24353v1#S2.E7)) to the whole space. (c) Geodesics obtained by applying a fiber rotation (Eq. [11](https://arxiv.org/html/2505.24353v1#S3.E11)) (d) The same geodesics as b) in solvable coordinates.
Refer to caption: extracted/6493579/figures/fig0.jpeg

#### Logistic regression layer.

The general formula for logistic regression in hyperbolic space is

$$ $p(y=1\,|\Upsilon)=\hat{y}(\Upsilon)=\sigma\left(h_{\alpha,\beta,\boldsymbol{w} }(\Upsilon)\right)$ (16) $$

The distance of a point $\Upsilon$ from a generic separator is

$$ $d(\Upsilon,H_{\alpha,\beta,\boldsymbol{w}}(\Upsilon))=\dfrac{1}{2}\mathrm{ arccosh}\left(1+2\,\dfrac{h^{2}_{\alpha,\beta,\boldsymbol{w}}(\Upsilon)}{| \boldsymbol{w}|^{2}-4\alpha\beta}\right)$ (17) $$

where the distance from a subspace $\mathcal{S}$ is defined as $d(\Upsilon,\mathcal{S})=\underset{\Psi\in{\mathcal{S}}}{\text{min}}\,d(
\Upsilon,\Psi)$. The argument of the sigmoid in eq. [16](https://arxiv.org/html/2505.24353v1#S3.E16) is then a nonlinear monotonic function of the distance between each point and the hyperplane (notice the subtle difference from the Euclidean case).
Similarly, when classifying between $K$ classes in hyperbolic space, we can define the analogous hyperbolic softmax regression as

$$ $p(y=k\,|\,\Upsilon)=\dfrac{\exp\left(h_{k}(\Upsilon)\right)}{\sum_{j=1}^{K} \exp\left(h_{j}(\Upsilon)\right)}$ (18) $$

where $h_{j}(\Upsilon)=h_{\alpha_{j},\beta_{j},\boldsymbol{w}_{j}}(\Upsilon)$.

### 3.4 Hyperbolic Cartan networks

We construct the simplest hyperbolic neural network by stacking $L$ hyperbolic linear layers such that

$$ $h^{\ell}=R_{\boldsymbol{u^{\ell}}}\left(\beta^{\ell}*\begin{bmatrix}h^{\ell-1} _{1}\\ W^{\ell}\boldsymbol{h}^{\ell-1}_{2}+\boldsymbol{b}^{\ell}\end{bmatrix}\right)$ (19) $$

and predicting on the $L$-th layer representations through the logistic layer (binary classification) or the logistic softmax (multiclass classification).

We also propose that the initial embedding of the starting data points $\boldsymbol{x_{i}}\in\mathbb{R}^{d}$ into the first hyperbolic layer $\mathcal{M}^{[1,\,1+d]}$ is as follows:

$$ $h^{1}=\begin{bmatrix}0\\ \boldsymbol{x}\end{bmatrix}$ (20) $$

Notice that by setting $\boldsymbol{u^{\prime}}^{\ell}=\boldsymbol{0},\,\beta^{\ell}_{1}=0\,\forall\ell$ this architecture becomes a stack of Euclidean linear layers.

These architectures can then be optimized on traditional loss functions (such as MSE and categorical cross entropy) using Riemannian SGD or Adam [38, 39]. We will discuss optimization in depth in App. [G.3](https://arxiv.org/html/2505.24353v1#A7.SS3).

This is conceptually different from previous iterations of hyperbolic networks that applied nonlinearities to the tangent spaces [3, 4, 5, 9]. Hyperbolic geometry as a tool for data science is generally employed using Riemannian manifold models, where the geometry of the space and the operations therein defined are the geodesic
exponential map and the geodesic logarithmic map plus the Möbius addition, defined in terms of the gyrovector space formulation [25].

Do we still need pointwise activations?

From Eq. [19](https://arxiv.org/html/2505.24353v1#S3.E19), by combining group homomorphisms and isometries, we obtain maps that are in neither class while being more expressive with the addition of each layer. However, a network composed of a single layer is not a universal approximator for $q\to\infty$, but, in contrast to Euclidean linear layers, stacking layers creates increasingly more expressive functions. If one wants to recover an expressivity scaling with network width, it is necessary to add nonlinearities between layers. To preserve the differentiable nature of the manifolds, we extend known nonlinearities so that they are continuously differentiable maps from $\mathbb{R}$ to $\mathbb{R}$.

We define a diffeomorphic pointwise activation function as

$$ $\text{DiLU}(x)=\dfrac{\text{ELU}(x)+\alpha x}{1+\alpha}$ (21) $$

with $\alpha\in[0,\infty)$, where the ELU [40] is defined as

$$ $\text{ELU}(x)=\begin{cases}x&x>0,\\ e^{x}-1&x\leq 0\end{cases}$ (22) $$

Given a pointwise nonlinearity $\sigma:\mathbb{R}\to\mathbb{R}$, we apply it to our coordinates by

$$ $\sigma(\Upsilon)=\begin{bmatrix}\Upsilon_{1}\\ \sigma(\boldsymbol{\Upsilon}_{2})\end{bmatrix}$ (23) $$

Since the DiLU is continuously differentiable everywhere and is a bijection from $\mathbb{R}\text{ to }\mathbb{R}$, Eq. [23](https://arxiv.org/html/2505.24353v1#S3.E23) defines a diffeomorphism between manifolds.

## 4 Results

We compare the performance of hyperbolic Cartan networks trained on real datasets with that of traditional neural networks and hyperbolic neural networks with Poincaré coordinates (described in detail in App. [G.4](https://arxiv.org/html/2505.24353v1#A7.SS4)). Notice that a Euclidean fully-connected layer ($Wx+b$) from $n$ to $m$ neurons has $m(n+1)$ parameters, while a Lie hyperbolic linear layer $\mathcal{M}^{[1,\,n]}\to\mathcal{M}^{[1,\,m]}$ has $m(n+1)-1$ parameters. We will then compare networks with the same number of layers of the same size. For Cartan networks, we will train them with and without pointwise linearities, since even though they are not universal approximators for arbitrary width, their expressivity increases with the network depth.

We use several benchmark tasks, and some toy regression tasks (for a discussion of these datasets, refer to App. [G.2](https://arxiv.org/html/2505.24353v1#A7.SS2)). For the toy datasets, we use a training set of 200 samples drawn from a uniform distribution $\mathcal{U}([-1,1]^{d})$, namely a $d$-dimensional hypercube, and tested on 1000 samples from the same distribution. We set $d=10$. For the real datasets, we use the standard train/test split provided by the torchvision library in Pytorch [41]. All models were trained with the same architecture: two hidden layers of 20 neurons each, and optimized with mean squared error (MSE) loss for the regression tasks and cross-entropy loss for the classification tasks with Riemannian Adam with $\text{lr}=0.01$ for the regression tasks and $\text{lr}=10^{-4}$ for the classification tasks (additional details on optimization are in App. [G.3](https://arxiv.org/html/2505.24353v1#A7.SS3)). Results are shown in Tab. [1](https://arxiv.org/html/2505.24353v1#S4.T1)-[2](https://arxiv.org/html/2505.24353v1#S4.T2). We highlight the model(s) with the best performance for each dataset. If multiple models are not statistically distinguishable based on Welch’s t-test ($p_{value}>5\%$), we highlight all of them.

**Table 1: Test $R^{2}$ on toy regression datasets (mean $\pm$ std, $n_{\mathrm{runs}}$ = 10)**
| Problem | Euclidean + DiLU | Cartan + DiLU | Cartan | Poincaré + DiLU |
| --- | --- | --- | --- | --- |
| $\frac{1}{n}\left(\sum_{i}^{n-1}x_{i}^{2}-x_{n}^{2}\right)$ | $0.85\pm 0.02$ | $0.973\pm 0.005$ | $0.93\pm 0.08$ | $0.965\pm 0.006$ |
| $x_{0}+x_{0}x_{1}$ | $0.9991\pm 0.0003$ | $0.999\,48\pm 0.000\,09$ | $0.9996\pm 0.0001$ | $0.9974\pm 0.0005$ |
| $x_{0}+x_{0}x_{1}+x_{0}x_{1}x_{2}$ | $0.991\pm 0.002$ | $0.991\pm 0.004$ | $0.985\pm 0.003$ | $0.989\pm 0.002$ |
| $\mathrm{Sinc}(\|x\|_{2})$ | $0.86\pm 0.02$ | $0.979\pm 0.004$ | $0.988\pm 0.002$ | $0.7\pm 0.3$ |
| $\mathrm{Sinc}(\|x\|_{3})$ | $0.78\pm 0.06$ | $0.91\pm 0.01$ | $0.933\pm 0.008$ | $0.6\pm 0.3$ |

**Table 2: Test accuracy on real-world datasets (mean $\pm$ std, $n_{\text{runs}}$ = 10)**
| Problem | Euclidean + DiLU | Cartan + DiLU | Cartan | Logistic | Poincaré + DiLU |
| --- | --- | --- | --- | --- | --- |
| Cifar10 | $0.477\pm 0.003$ | $0.476\pm 0.003$ | $0.444\pm 0.007$ | $0.410\pm 0.002$ | $0.475\pm 0.003$ |
| FMNIST | $0.868\pm 0.002$ | $0.869\pm 0.002$ | $0.856\pm 0.003$ | $0.8490\pm 0.0005$ | $0.874\pm 0.002$ |
| KMNIST | $0.814\pm 0.006$ | $0.807\pm 0.006$ | $0.77\pm 0.01$ | $0.7038\pm 0.0006$ | $0.821\pm 0.005$ |
| MNIST | $0.965\pm 0.001$ | $0.962\pm 0.003$ | $0.954\pm 0.003$ | $0.9284\pm 0.0003$ | $0.964\pm 0.002$ |

Our experiments show that hyperbolic neural networks can match their traditional Euclidean counterparts across several benchmark tasks. In the regression table, the Cartan networks without activations perform competitively, achieving the best result on the $\mathrm{Sinc}(\|x\|_{2})$ and $\mathrm{Sinc}(\|x\|_{3})$ problems. Cartan networks with activations and Poincaré networks sometimes surpass Cartan alone, especially on lower-order polynomial tasks. However, Cartan’s consistent performance and strong results without DiLU activations suggest its intrinsic representational strength. On the classification tasks, however, Cartan slightly underperforms its activations-augmented variants and the Poincaré networks, especially on CIFAR10 and KMNIST, suggesting that additional nonlinearity enhances performance in higher-dimensional and more complex datasets.

Figure: Figure 3: Network performance for depth and hidden layer size on classification datasets. Boxplot showing reached accuracy during training as detailed in Sec. [4](https://arxiv.org/html/2505.24353v1#S4) for Euclidean, Cartan, and Poincaré neural networks. The rightmost column of each subplot depicts the accuracy of Cartan networks without nonlinearities. Different colors represent hidden layer sizes, while the network depth increases from left to right within a column. Boxes show data falling between the 1st and 3rd quartile, while points further than 1.5 IQR of the quartiles are considered outliers. 10 runs per configuration.
Refer to caption: extracted/6493579/figures/comparison_all.jpeg

To further characterize the performance of the proposed architecture, we train Cartan networks on the real-world classification datasets, varying depth (1-7 layers for KMNIST and 1-5 for the others) and hidden layer size (20, 40, and 100 neurons), with and without nonlinearities, comparing their test accuracy to Euclidean and Poincarè ball networks (Fig. [3](https://arxiv.org/html/2505.24353v1#S4.F3)). Our proposed architecture achieves performance comparable to the alternatives in a wide hyperparameter range, surpassing the Euclidean variant when deeper, wider architectures are tested on fMNIST and kMNIST. For all tasks except MNIST Cartan networks show increasing accuracy with depth at all hidden layer sizes, contrasting the trend of Euclidean and Poincaré architectures. Best accuracies across configurations for these tasks are shown in Tab. [3](https://arxiv.org/html/2505.24353v1#S4.T3).

**Table 3: Best accuracy on real-world datasets (mean $\pm$ std, $n_{\mathrm{runs}}=10$)**
| Problem | Cartan | Cartan + DiLU | Euclidean + DiLU | Poincaré + DiLU |
| --- | --- | --- | --- | --- |
| KMNIST | $0.845\pm 0.005$ | $0.893\pm 0.002$ | $0.886\pm 0.003$ | $0.878\pm 0.004$ |
| Cifar10 | $0.502\pm 0.006$ | $0.535\pm 0.003$ | $0.539\pm 0.002$ | $0.526\pm 0.003$ |
| MNIST | $0.964\pm 0.003$ | $0.9777\pm 0.0006$ | $0.9786\pm 0.0007$ | $0.9752\pm 0.0009$ |
| fMNIST | $0.873\pm 0.002$ | $0.889\pm 0.002$ | $0.883\pm 0.001$ | $0.884\pm 0.002$ |

Although Cartan networks without nonlinearities do not achieve competitive performance at a given parameter size, these models show a remarkable increase in accuracy with depth, suggesting that the nonlinear nature of the Cartan coordinate boosts the overall network expressivity.

## 5 Discussion

This work introduced Cartan networks, a novel hyperbolic deep learning architecture based entirely on intrinsic group theoretic and metric operations. The long-term goal of this new theoretical construction is to develop a framework for machine learning algorithms that is both expressive and mathematically consistent, resulting in models that can be more easily analyzed and interpreted.

Cartan networks complement the view of Hyperbolic Neural Networks as a sequence of exponential and logarithmic maps [3]. Our architecture exploits the dual nature of hyperbolic spaces as solvable groups and Riemannian manifolds, alternating isometries and homomorphisms in its layers, both intrinsically defined and geometrically motivated operations. Importantly, we do not need to incorporate component-wise non-linearities on the tangent space of the manifold, which are incompatible with its geometric structure and introduce a dependency on the particular choice of coordinates, which is arbitrary.

The experiments performed, while not exhaustive, show that the proposed architecture is competitive with comparable Euclidean and hyperbolic architectures on a variety of real and synthetic tasks.
These results are particularly encouraging because the hyperbolic space is the simplest representative of the family of non-compact symmetric spaces; exploring more complex manifolds, and investigating how to specialize our architecture to convolutional or sequential data, are major directions for future research.

While our results demonstrate the potential of the proposed framework, several limitations must be acknowledged. First, due to resource constraints, our experiments were conducted on a limited number of datasets and with a relatively narrow range of hyperparameter configurations.
Secondly, the architectural modifications introduced to incorporate group-theoretical structure lead to increased computational overhead. While this is somewhat inevitable given the high optimization of standard neural network software, improving the computational performance of our approach is an important step to ensure its adoption.

## 6 Supplementary materials

The entire code to reproduce all the results shown in this article is available at

[https://github.com/FedericoMilanesio/CartanNetworks](https://github.com/FedericoMilanesio/CartanNetworks)

## 7 Contributions

PGF conceived the idea presented. All authors contributed to the theoretical discussion. FM and MS wrote the code and conducted the numerical experiments. All authors wrote and reviewed the final manuscript.

## 8 Acknowledgements

P. G. Fré acknowledges support by the company Additati&Partners Consulting s.r.l during the development of the present research. Furthermore, the Ph.D. fellowships of F. Milanesio and M. Santoro are cofinanced by the company Additati&Partners
Consulting s.r.l at UniTO and SISSA, respectively.

## References

- [1]
Christopher Bishop.
Pattern Recognition and Machine Learning.
Springer, 2006.
- [2]
Maximillian Nickel and Douwe Kiela.
Poincaré embeddings for learning hierarchical representations.
In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
- [3]
Octavian-Eugen Ganea, Gary Becigneul, and Thomas Hofmann.
Hyperbolic Neural Networks.
In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
- [4]
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada.
Hyperbolic neural networks++.
In International Conference on Learning Representations, 2021.
- [5]
Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou.
Fully hyperbolic neural networks.
In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5672–5686, Dublin, Ireland, 2022. Association for Computational Linguistics.
- [6]
Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr.
Fully hyperbolic convolutional neural networks for computer vision.
In The Twelfth International Conference on Learning Representations, 2024.
- [7]
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec.
Hyperbolic Graph Convolutional Neural Networks.
In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- [8]
Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas.
Hyperbolic attention networks.
In International Conference on Learning Representations, 2019.
- [9]
Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, and Guoying Zhao.
Hyperbolic Deep Neural Networks: A Survey.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):10023–10044, 2022.
- [10]
Siyuan Gao, Gal Mishne, and Dustin Scheinost.
Poincaré embedding reveals edge-based functional networks of the brain.
In Medical Image Computing and Computer Assisted Intervention, pages 448–457. Springer International Publishing, 2020.
- [11]
Anna Klimovskaia, David Lopez-Paz, Léon Bottou, and Maximilian Nickel.
Poincaré maps for analyzing complex hierarchies in single-cell data.
Nature Communications, 11(1):2966, 2020.
- [12]
Benjamin Paul Chamberlain, Stephen R. Hardwick, David R. Wardrope, Fabon Dzogang, Fabio Daolio, and Saúl Vargas.
Scalable hyperbolic recommender systems.
arXiv preprint arXiv:1902.08648 [cs.IR], 2019.
- [13]
Élie Cartan.
Sur une classe remarquable d’espaces de Riemann.
Bulletin de la Société Mathématique de France, 54:214–264, 1926.
- [14]
Sigurdur Helgason.
Differential geometry and symmetric spaces.
Academic press, 1962.
- [15]
Pietro G. Fré.
Discrete, Finite and Lie Groups.
De Gruyter, 2023.
- [16]
Ulrika Magnea.
An introduction to symmetric spaces.
arXiv preprint arXiv:cond-mat/0205288, 2002.
- [17]
Laura Andrianopoli, Riccardo D’Auria, Sergio Ferrara, Pietro G. Fré, and Mario Trigiante.
R-R scalars, U-duality and solvable Lie algebras.
Nuclear Physics B, 496(3):617–629, 1997.
- [18]
Laura Andrianopoli, Riccardo D’Auria, Sergio Ferrara, Pietro G. Fré, Ruben Minasian, and Mario Trigiante.
Solvable Lie algebras in type IIA, type IIB and M-theories.
Nuclear physics B, 493(1-2):249–277, 1997.
- [19]
Pietro G. Fré, Floriana Gargiulo, Jan Rosseel, Ksenya Rulik, Mario Trigiante, and Antoine Van Proeyen.
Tits-Satake projections of homogeneous special geometries.
Classical and Quantum Gravity, 24:27–78, 2007.
- [20]
Dmitri V. Alekseevsky.
Classification of quaternionic spaces with a transitive solvable group of motions.
Math. USSR Izvestija, 9:297–339, 1975.
- [21]
Vicente Cortés.
Alekseevskian spaces.
Differential Geometry and its Applications, 6(2):129–168, 1996.
- [22]
Dmitri V. Alekseevsky, Vicente Cortés, Chandrashekar Devchand, and Antoine Van Proeyen.
Polyvector superPoincare algebras.
Communications in Mathematical Physics, 253:385–422, 2004.
- [23]
Ugo Bruzzo, Pietro G. Fré, and Mario Trigiante.
The Paint Group Tits Satake Theory of Hyperbolic Symmetric Spaces: the distance function, paint invariants and discrete subgroups.
arXiv preprint arXiv:2503.07626 [math.DG], 2025.
- [24]
Pietro Fré, Federico Milanesio, Matteo Santoro, and Guido Sanguinetti.
Navigation through Non Compact Symmetric Spaces, that is Cartan Neural Networks.
ArXiv in preparation, 2025.
- [25]
Abraham Albert Ungar.
A Gyrovector Space Approach to Hyperbolic Geometry.
Synthesis Lectures on Mathematics & Statistics. Springer International Publishing, 2009.
- [26]
Andrii Skliar and Maurice Weiler.
Hyperbolic Convolutional Neural Networks.
arXiv preprint arXiv:2308.15639 [cs.LG], 2023.
- [27]
Sagar Ghosh, Kushal Bose, and Swagatam Das.
On the universal statistical consistency of expansive hyperbolic deep convolutional neural networks.
arXiv preprint arXiv:2411.10128 [stat.ML], 2024.
- [28]
Taco S. Cohen, Mario Geiger, and Maurice Weiler.
A general theory of equivariant cnns on homogeneous spaces.
In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- [29]
Shuxiao Chen, Edgar Dobriban, and Jane H. Lee.
A group-theoretic framework for data augmentation.
Journal of Machine Learning Research, 21(245):1–71, 2020.
- [30]
Samuel E. Otto, Nicholas Zolman, J. Nathan Kutz, and Steven L. Brunton.
A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning.
arXiv preprint arXiv:2311.00212 [cs.LG], 2024.
- [31]
Tzu-Yuan Lin, Minghan Zhu, and Maani Ghaffari.
Lie neurons: Adjoint-equivariant neural networks for semisimple lie algebras.
In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 30529–30545, 2024.
- [32]
Vladimir Jaćimović.
A group-theoretic framework for machine learning in hyperbolic spaces.
arXiv preprint arXiv:2501.06934 [cs.LG], 2025.
- [33]
James E. Humphreys.
Semisimple Lie Algebras.
In Introduction to Lie Algebras and Representation Theory, pages 15–41. Springer, New York, NY, 1972.
- [34]
Gal Mishne, Zhengchao Wan, Yusu Wang, and Sheng Yang.
The Numerical Stability of Hyperbolic Representation Learning.
In Proceedings of the 40th International Conference on Machine Learning, volume 202 of PMLR, pages 24925–24949, 2023.
- [35]
Vladimir Jaćimović and Aladin Crnkić.
Clustering in hyperbolic balls.
arXiv preprint arXiv:2501.19247 [cs.LG], 2025.
- [36]
Jeff Cheeger.
Compact manifolds of nonpositive curvature.
In Comparison Theorems in Riemannian Geometry, volume 9 of North-Holland Mathematical Library, pages 154–167. Elsevier, 1975.
- [37]
Ines Chami, Albert Gu, Dat P. Nguyen, and Christopher Re.
Horopca: Hyperbolic dimensionality reduction via horospherical projections.
In Proceedings of the 38th International Conference on Machine Learning, volume 139 of PMLR, pages 1419–1429, 2021.
- [38]
Silvère Bonnabel.
Stochastic gradient descent on riemannian manifolds.
IEEE Transactions on Automatic Control, 58(9):2217–2229, 2013.
- [39]
Gary Becigneul and Octavian-Eugen Ganea.
Riemannian adaptive optimization methods.
In International Conference on Learning Representations, 2019.
- [40]
Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter.
Fast and accurate deep network learning by exponential linear units (elus).
In International Conference on Learning Representations, 2016.
- [41]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.
PyTorch: An Imperative Style, High-Performance Deep Learning Library.
In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- [42]
Robert Gilmore.
Lie groups, Lie algebras, and some of their applications.
Dover Publications, 2016.
- [43]
Pietro G. Fré.
Gravity, a Geometrical Course, volume 1, 2.
Springer Science & Business Media, 2012.
- [44]
Manfredo P. do Carmo.
Riemannian Geometry.
Mathematics: Theory & Applications. Birkhäuser, 1992.
- [45]
Pietro G. Fré, Floriana Gargiulo, and Ksenya Rulik.
Cosmic billiards with painted walls in non-maximal supergravities: a worked out example.
Nuclear Physics B, 737(1):1–48, 2006.
- [46]
Brian C. Hall.
Lie Groups, Lie Algebras, and Representations: An Elementary Introduction, volume 222 of Graduate Texts in Mathematics.
Springer International Publishing, Cham, 2015.
- [47]
Shoshichi Kobayashi and Katsumi Nomizu.
Foundations of Differential Geometry, volume 2.
Wiley, 1963.
- [48]
Max Kochurov, Rasul Karimov, and Serge Kozlukov.
Geoopt: Riemannian optimization in pytorch.
arXiv preprint arXiv:2005.02819 [cs.CG], 2020.
- [49]
Emmanuel Abbe, Enric Boix-Adserà, and Theodor Misiakiewicz.
The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks.
In Annual Conference Computational Learning Theory, volume 178 of PMLR, pages 4782–4887, 2022.
- [50]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.
Gradient-based learning applied to document recognition.
Proceedings of the IEEE, 86(11):2278–2324, 1998.
- [51]
Han Xiao, Kashif Rasul, and Roland Vollgraf.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.
arXiv preprint arXiv:1708.07747 [cs.LG], 2017.
- [52]
Taichi Clanuwat, Marc Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Ku Yamamoto, and David Ha.
Deep learning for classical japanese literature.
arXiv preprint arXiv:1812.01718 [cs.CV], 2018.
- [53]
Alex Krizhevsky and Geoffrey Hinton.
Learning multiple layers of features from tiny images.
Technical report, University of Toronto, 2009.

## Appendix A Lie Groups

A Lie group [14, 15, 16, 33] is an analytic differentiable manifold $G$ endowed with a group structure such that the group operations of multiplication and inversion are infinitely differentiable. The group operation is a binary product operation:

$$ $*:G\times G\to G\quad(x*y)=xy\in G$ (24) $$

that must satisfy the group axioms. An abstract Lie group always admits an infinite series of matrix representations, which are determined by the abstract group structure, where the group operations are the matrix multiplication and inversion. In practice, Lie groups are both groups (having multiplication and inverses) and smooth manifolds (having a differentiable structure). Every Lie group has a corresponding Lie Algebra isomorphic as a vector space to its tangent space at the identity. It is formed by its (left/right) invariant vector fields that close under commutation. On Lie groups $G$ and on coset manifolds $G/H$, one can construct $G$ invariant Riemannian metrics that are unique or multiple depending on the structure of the coset.

## Appendix B Solvable coordinates parametrization of hyperbolic space

As introduced in [23], the matrix element parameterizing the hyperbolic space is as follows(^1^11Compared to the original formulation of Fré et al., we performed the following change of coordinates $\boldsymbol{\Upsilon}_{2}=\frac{1}{2}\boldsymbol{\Upsilon}_{2}^{\text{old}}$ for ease of formulation.):

$$ $\mathbb{L}(\Upsilon)=\begin{bmatrix}e^{\Upsilon_{1}}&\sqrt{2}e^{\Upsilon_{1}} \boldsymbol{\Upsilon_{2}}^{\intercal}&-e^{\Upsilon_{1}}|\boldsymbol{\Upsilon_{ 2}}|^{2}\\ 0&\mathbb{I}_{q}&-\sqrt{2}\boldsymbol{\Upsilon_{2}}\\ 0&0&e^{-\Upsilon_{1}}\\ \end{bmatrix}$ (25) $$

where

$$ $\Upsilon=\left[\Upsilon_{1},\boldsymbol{\Upsilon_{2}}\right]^{\intercal}=\left [\Upsilon_{1},\Upsilon_{2,1},\,\dots,\,\Upsilon_{2,q}\right]^{\intercal}$ (26) $$

are the solvable coordinates, $\mathbb{I}_{q}$ is the identity matrix of size $q$, and $|.|$ is the Euclidean norm. Group operation is given by matrix multiplication:

$$ $\mathbb{L}(\Psi*\Upsilon)=\mathbb{L}(\Psi)\cdot\mathbb{L}(\Upsilon)$ (27) $$

In the upper-triangular representation, the matrices preserve the Lorentz metric

$$ $\eta^{i}_{j}=\delta_{i,n-j},\quad i,j=1,\dots n$ (28) $$

In these coordinates, by left-transport of the metric induced on the solvable Lie algebra at the origin by the Einstein metric of the symmetric space, we find

$$ $g_{1,q}(\Upsilon)=\begin{bmatrix}1+|\boldsymbol{\Upsilon_{2}}|^{2}&\boldsymbol {\Upsilon_{2}}^{\intercal}\\ \boldsymbol{\Upsilon_{2}}&\mathbb{I}_{q}\\ \end{bmatrix}$ (29) $$

Notice that in these coordinates, the volume element is constant:

$$ $\sqrt{\det{g_{1,q}(\Upsilon)}}=1$ (30) $$

#### Transition to the Poincaré Ball coordinates.

In this section, we provide the transition function
from the solvable coordinates to the Poincaré Ball coordinates, namely the projective off-diagonal coordinates of
[42] and Equation (5.2.43) of [43]. Given a point in $\mathbb{H}^{q+1}$ labeled Poincaré ball coordinates $\boldsymbol{x}$, the same point is identified by a set of solvable coordinates $\boldsymbol{\Upsilon}$ related to $\boldsymbol{x}$ in the following way. First, we split the coordinates
as follows:

$$ $\boldsymbol{x}=[x_{1},\boldsymbol{x_{2}}]^{\intercal}=[x_{1},x_{2,1},\,\dots, \,x_{2,q}]^{\intercal}$ (31) $$

Then, the map from the solvable parametrization to the point in the Poincaré ball is given by

$$ $\begin{cases}x_{1}=1-\dfrac{1+e^{-\Upsilon_{1}}}{1+\cosh{N(\Upsilon)}}\\ \boldsymbol{x_{2}}=\dfrac{\boldsymbol{\Upsilon_{2}}}{1+\cosh{N(\Upsilon)}}\end {cases}$ (32) $$

## Appendix C Riemannian operations in solvable coordinates

#### Parallel transport.

The group operation on $\mathcal{M}^{[1,\,1+q]}$ naturally induces a notion of parallel transport. Specifically, the left group action $L_{\Psi*\Upsilon^{-1}}$ defines a diffeomorphism $dL_{\Psi\Upsilon^{-1}}:T_{\Upsilon}\mathcal{M}\to T_{\Psi}\mathcal{M}$ that acts on a tangent vector $\boldsymbol{v}\in T_{\Upsilon}\mathcal{M}$ as

$$ $dL_{\Psi*\Upsilon^{-1}}(\boldsymbol{v})=\begin{bmatrix}v_{1}+\boldsymbol{v}_{2 }\cdot(\boldsymbol{\Psi}_{2}-\boldsymbol{\Upsilon}_{2})\\ \boldsymbol{v}_{2}\end{bmatrix}$ (33) $$

#### Riemannian logarithmic map.

In general, the Riemannian logarithmic map can be retrieved from the geodesic distance equation from the following formula [44]:

$$ $\nabla_{\boldsymbol{\Psi}}(d(\boldsymbol{\Upsilon},\boldsymbol{\Psi}))^{2}=-2 \log_{\boldsymbol{\Psi}}{\boldsymbol{\Upsilon}}$ (34) $$

In solvable coordinates, then,

$$ $\log_{0}(\Upsilon)=\frac{N({\Upsilon})}{\sinh N({\Upsilon})}\begin{bmatrix} \cosh N(\Upsilon)-\mathrm{e}^{-\Upsilon_{1}}\\ \boldsymbol{\Upsilon}_{2}\end{bmatrix}$ (35) $$

At a general point $\Psi\in\mathcal{M}$, we can also use the left-invariance to compute the logarithmic map. That is, we first translate to $\Psi$ the origin, apply $\log_{0}$, and then use the inverse parallel transport to bring the result back to the tangent space at $\Psi$:

$$ $\log_{\Psi}(\Upsilon)=dL_{\Psi*\Upsilon^{-1}}^{-1}\left(\log_{0}(\Psi^{-1}* \Upsilon)\right)$ (36) $$

#### Geodesics.

The geodesic can be computed with the general method described in [23].
Given a tangent vector $\boldsymbol{v}=\{v_{1},\boldsymbol{v}_{2}\}\in T_{0}\mathcal{M}$ and its norm $|\boldsymbol{v}|:=\sqrt{\sum_{i}v_{i}^{2}}$, the formula for the geodesics from the origin is

$$ $\boldsymbol{\gamma}_{0}(\boldsymbol{v},t)=\begin{bmatrix}-\log{\left(\cosh(| \boldsymbol{v}|\,t)-\dfrac{v_{1}}{|\boldsymbol{v}|}\sinh(|\boldsymbol{v}|\,t) \right)}\\ \dfrac{\boldsymbol{v}_{2}}{|\boldsymbol{v}|}\>\sinh(|\boldsymbol{v}|\,t)\end{bmatrix}$ (37) $$

with $t\in[0,1]$.

Then the geodesic between points $\Upsilon,\Psi\in\mathcal{M}^{[1,\,1+q]}$ is obtained by applying the logarithmic map $\log_{0}$ to $\Upsilon*\Psi^{-1}$, tracing the geodesic $\boldsymbol{\gamma}_{0}$,
and translating back using the group action:

$$ $\boldsymbol{\gamma}_{\Psi\to\Upsilon}(t)=\Psi*\boldsymbol{\gamma}_{0}\left(t, \,\log_{0}(\Psi^{-1}*\Upsilon)\right)$ (38) $$

#### Exponential Riemannian map.

From Eq. [37](https://arxiv.org/html/2505.24353v1#A3.E37), we find that the exponential map from the origin is

$$ $\exp_{0}(\boldsymbol{v})=\boldsymbol{\gamma}_{0}(\boldsymbol{v},t=1)=\begin{ bmatrix}-\log{\left(\cosh(|\boldsymbol{v}|)-\dfrac{v_{1}}{|\boldsymbol{v}|} \sinh(|\boldsymbol{v}|)\right)}\\ \dfrac{\boldsymbol{v}_{2}}{|\boldsymbol{v}|}\>\sinh(|\boldsymbol{v}|)\end{bmatrix}$ (39) $$

for $\boldsymbol{v}\in T_{0}\mathcal{M}$. The generic exponential map is then

$$ $\exp_{\Upsilon}(\boldsymbol{v})=\Upsilon*\exp_{0}(dL_{0\Upsilon^{-1}}( \boldsymbol{v}))$ (40) $$

#### Distance between points.

Given $\Upsilon\in\mathcal{M}^{[1,\,1+q]}$, its norm is

$$ $N(\Upsilon)=\text{arccosh}\left(\frac{1}{2}(e^{-\Upsilon_{1}}+e^{\Upsilon_{1}} (1+|\boldsymbol{\Upsilon}_{2}|^{2}))\right)$ (41) $$

Then, for $\Upsilon,\Psi\in\mathcal{M}^{[1,\,q]}$, their distance is given by $d(\Upsilon,\Psi)=N(\Psi^{-1}*\Upsilon)$. We will describe the Riemannian exponential, the logarithmic map, and the equation for the geodesics, as well as the transition to the Poincarè ball coordinates in App. [B](https://arxiv.org/html/2505.24353v1#A2.SS0.SSS0.Px1).

## Appendix D Isometries

### D.1 Relevant isometries from group theory

The coset manifold $\mathcal{M}^{[1,q+1]}$ is defined as a quotient $\dfrac{\mathrm{SO}(1,q+1)}{\mathrm{SO}(q+1)}$ and metrically equivalent to a group $\mathrm{Exp}(\mathrm{Solv}_{1,q+1})$. Its isometries are all the transformations of the group $\mathrm{SO}(1,q+1)$, which we will classify into two groups.

- 1.
The multiplication by a solvable group element.
- 2.
The adjoint action of the full group on the solvable group.

As per [23], we can split the algebra $\mathbb{H}^{[1,q+1]}$ in two different components:

$$ $\mathbb{H}^{[1,q+1]}=\mathbb{G}_{\mathrm{paint}}^{[1,q+1]}\oplus\mathbb{H}_{F} ^{[1,q+1]}$ (42) $$

We call the exponential of the first component the paint group, while the second component corresponds to the fiber rotation. Together with the solvable element multiplication, these form the three categories in which we split the full algebra.

### D.2 Explicit derivation of isometries in the PGTS coordinates

The set of isometries (distance-preserving maps) of $\mathcal{M}^{[1\,1+q]}$ into itself is given by $\text{SO}(1,\,1+q)$ (these have been parameterized in terms of the Poincarè ball coordinates by [32]). These isometries are a composition of three distinct isometries (for a detailed derivation, refer to [23]).

#### Paint rotation.

The group of outer automorphisms (within the full isometry group $\mathrm{SO(1,1+q)}$) of the solvable
Lie group $\mathcal{S}$ metrically equivalent to our symmetric space corresponds to the notion of Paint Group originally introduced in
[45] and fully discussed in [23]. It is named $\mathcal{G}_{\text{paint}}$. For $r=1$, $\mathcal{G}_{\text{paint}}\sim\text{SO}(q)$, and each $Q\in\text{SO}(q)$ maps a point with solvable coordinates $\Upsilon$ by rotating $\boldsymbol{\Upsilon}_{2}$:

$$ $\begin{cases}\Upsilon^{\text{paint}}_{1}=\Upsilon_{1}\\ \boldsymbol{\Upsilon^{\text{paint}}_{2}}=Q\boldsymbol{\Upsilon_{2}}\end{cases}$ (43) $$

#### Group translation.

Each element $b\in\mathcal{M}^{[1,\,1+q]}$ defines an isometry of the symmetric space into itself through the group action. From the geometric point of view, this represents a rigid translation of the origin $\boldsymbol{0}$ into point $b$. This operation will take the role of the bias of classical logistic regression.

#### Fiber rotation.

The full group of outer automorphisms of $\mathrm{G/H}$ is given by the exponential of $\mathbb{H_{c}}=\mathbb{G}_{\mathrm{paint}}\oplus\mathbb{H_{\mathrm{F}}}$.
(see [23] for the theory of the non-compact symmetric space Grassmannian foliation to which the Lie subalgebra $\mathbb{G_{\mathrm{F}}}\subset\mathfrak{so}(1,1+q)$ is tightly connected).
By means of the exponential map the subalgebra $\mathbb{G_{\mathrm{F}}}\subset\mathfrak{so}(1,1+q)$ generates a $q$-dimensional group of isometries. Each of these isometries modifies the Cartan coordinate $\Upsilon_{1}$ and coordinate $\Upsilon_{2,j}$.

To derive an analytic expression, however, we use the fact that isometries of Riemannian manifolds can be parametrized in terms of the exponential map. In particular, as paint rotations are given by matrices $Q\in\mathrm{SO}(q)$, the remaining isometries are parametrized by the generators of the full group $\mathrm{SO}(q+1)$ without the paint generators $\mathrm{SO}(q)$, and can be computed accordingly.

Given a vector $\boldsymbol{u}=[u_{0},u_{1},\dots,u_{q}]^{\intercal}\in\mathbb{S}^{q+1}$ ($|\boldsymbol{u}|=1$), and defining $\boldsymbol{u}^{\prime}=[u_{1},\dots,u_{q}]^{\intercal}$, the total fiber rotation by $\boldsymbol{u}$ is given by

$$ $R_{u}(\Upsilon)=\begin{bmatrix}-\log\left(-\dfrac{1}{2}(\mathrm{e}^{\Upsilon_{ 1}}(1+|\boldsymbol{\Upsilon}_{2}|^{2})+\mathrm{e}^{-\Upsilon_{1}})\,(1+u_{0})+ \mathrm{e}^{-\Upsilon_{1}}u_{0}-\boldsymbol{\Upsilon}_{2}\cdot\boldsymbol{u}^{ \prime}\right)\\ \boldsymbol{\Upsilon}_{2}-x\left(\dfrac{\boldsymbol{\Upsilon}_{2}\cdot \boldsymbol{u}^{\prime}}{1+u_{0}}+\dfrac{1}{2}(\mathrm{e}^{\Upsilon_{1}}(1+| \boldsymbol{\Upsilon}_{2}|^{2})-\mathrm{e}^{-\Upsilon_{1}})\right)\boldsymbol{ u}^{\prime}\\ \end{bmatrix}$ (44) $$

A general isometry $f:\mathcal{M}^{[1,\,1+q]}\to\mathcal{M}^{[1,\,1+q]}$ can be parametrized as

$$ $f(\Upsilon)=R_{\boldsymbol{u}}\left(\begin{bmatrix}b_{1}\\ \boldsymbol{b}_{2}\end{bmatrix}*\begin{bmatrix}1&0\\ 0&Q\end{bmatrix}\begin{bmatrix}\Upsilon_{1}\\ \boldsymbol{\Upsilon}_{2}\end{bmatrix}\right)$ (45) $$

where $Q\in\text{SO}(q)$, $b\in\mathcal{M}^{[1,\,1+q]}$ and $\boldsymbol{u}\in\mathbb{S}^{q+1}$.

## Appendix E Homomorphisms

In this section, we prove Th. [3.1](https://arxiv.org/html/2505.24353v1#S3.Thmtheorem1).

###### Proof.

Let $h$ be an homomorphism between $\mathcal{M}^{[1,q+1]}$ and $\mathcal{M}^{[1,p+1]}$. Since they are both simply connected, Th. 5.6 from [46] applies, hence there exists a unique Lie Algebra morphism $\mathfrak{h}:\mathrm{Lie}(\mathcal{M}^{[1,q+1]})\rightarrow\mathrm{Lie}(
\mathcal{M}^{[1,p]+1})$ such that $\mathfrak{h}=\mathrm{d}h$.
To find all such morphisms, it is enough to parametrize all algebra homomorphisms $\mathfrak{h}$.

Since these homomorphisms are vector space morphisms, it is enough to define them on algebra generators. The generators are given in [23] and satisfy the following relationships:

$$ $[H,T_{i}]=T_{i}\quad[T_{i},T_{j}]=0$ $$

Let $H^{q},T_{i}^{q}$ be the generators of $\mathcal{M}^{[1,q]}$ and $H^{p},T_{i}^{p}$ be the generators of $\mathcal{M}^{[1,p+1]}$. It is enough to find linear maps that satisfy the commutator relations, that is

$$ $[\phi(H^{q}),\phi(T_{i}^{q})]=\phi(T_{i}^{q})$ $$

as all other relations will not give additional constraints. By setting

$$ $\phi(H^{q}):=\alpha H^{p}+\beta^{i}T_{i}^{p},\quad\phi(T_{j}^{q}):=\alpha_{j}H ^{p}+W_{j}^{i}T_{i}^{p},$ $$

one can check the commutators for all generators, thus obtaining

$$ $[\phi(H^{q}),\phi(T_{j}^{q})]=[\alpha H^{p}+\beta^{i}T_{i}^{p},\alpha_{j}H^{p} +W_{j}^{l}T_{l}]^{p}=\alpha W_{j}^{l}T_{l}^{p}-\beta^{i}\alpha_{j}T_{i}^{p}= \phi(T_{j}^{q})=\alpha_{j}H^{p}+W_{j}^{m}T_{m}^{p}$ $$

from which $\alpha_{j}=0$. As the dimension of the image is greater than 1 by assumption, at least one $T^{q}_{j}$ must have a nontrivial image, hence $\alpha=1$. Hence, the homomorphism matrix in the basis of these generators is given by

$$ $\tilde{W}=\begin{bmatrix}1&0\\ \boldsymbol{\beta}&W\\ \end{bmatrix}.$ $$

All that remains is to express these morphisms in terms of solvable coordinates. The relationship between solvable coordinates and algebra coordinates is given by the map $\chi$

$$ $\chi\left(\begin{bmatrix}t^{1}\\ \boldsymbol{t^{2}}\end{bmatrix}\right)=\begin{bmatrix}\Upsilon_{1}\\ \boldsymbol{\Upsilon_{2}}\>\dfrac{\Upsilon_{1}}{1-e^{-\Upsilon_{1}}}\end{bmatrix}$ $$

Then, our group element with coordinates $\Upsilon=\chi(t)$ is written as

$$ $\mathbb{L}(\chi(t))=\mathrm{Exp}(t^{1}H\>+\>t^{i}T_{i})$ $$

The homomorphism in coordinates is then the map

$$ $\mathfrak{h}=\chi\>\circ\>\tilde{W}\>\circ\chi^{-1}$ $$

which after trivial manipulation gives Eq. [12](https://arxiv.org/html/2505.24353v1#S3.E12).
∎

###### Remark .

Although the abstract exponential map from a Lie algebra to the component connected to the Identity of a corresponding Lie group is unique, its explicit realization in terms of group parameters
namely, coordinates on the group manifold depend on the definition of the atlas of open charts and can then take many different forms.
Since the solvable group $\mathrm{S}$ and hence its metric equivalent non-compact symmetric space $\mathrm{U/H}$ are diffeomorphic to $\mathbb{R}^{n}$, we have just one open chart that covers the entire non-compact manifold. However, this open chart, namely the utilized solvable coordinates, can be
chosen in several different ways, depending on the way the exponential map $\Sigma\,:\,Solv\to\mathcal{S}$ is done matrix-wise. As explained in
[23], for the normed solvable Lie Algebras uniquely associated to each n.c.
$\mathrm{G/H}$ the generators that are in one-to-one relations with the TS projection of the $\mathrm{G}$ root system have a natural grading in terms
of root heights and this introduces a canonical definition of the $\Sigma$ exponential map that is the one adopted in the present paper. The relation
between the canonical solvable coordinates $\boldsymbol{\Upsilon}_{i}$ of the $i$-th solvable group $\mathcal{S}_{i}$ and those
$\boldsymbol{\Upsilon}_{i+1}$ of its homomorphic image $\mathcal{S}_{i+1}$ generated by the linear homomorphism of the corresponding solvable
Lie algebras can be obtained by solving the first-order differential system provided by the linear relation between Maurer Cartan $1$-forms. Such
a system is always iteratively solvable by quadratures precisely because the Lie algebras are solvable.

###### Proof.

###### Remark .

## Appendix F Derivation of hyperbolic hyperplanes

In the hyperbolic space $\mathcal{M}^{[1,\,1+q]}$, the set of submanifolds of codimension 1 is given by all possible immersions of $\mathcal{M}^{[1,\,q]}$ [47].

These hyperplanes can be found by defining one such immersion, for example

$$ ${\text{H}_{\mathbf{0}}}^{1+q}=\{\Upsilon\in\mathcal{M}^{[1,\,1+q]}\,|\, \Upsilon_{2,q}=0\}\simeq\mathcal{M}^{[1,\,q]}$ (46) $$

and finding the set of isometries that do not leave $\text{H}_{\mathbf{0}}$ invariant. Given the complete isometry group $G_{q+1}$ of the manifold $\mathcal{M}^{[1,q+1]}$, embedding $\mathcal{M}^{[1,q]}\hookrightarrow\mathcal{M}^{[1,q+1]}$ also gives an injective homomorphism $G_{q}\hookrightarrow G_{q+1}$, so the set of isometries we look for is the quotient $G_{q+1}/G_{q}$. The isometry categories, given in App. [D.2](https://arxiv.org/html/2505.24353v1#A4.SS2), all have easily recognizable realizations in the quotient. For the paint rotation, we consider the rotations of the $q$-th paint coordinate onto the others, that is, the $q$-sphere $\mathrm{SO}(q)/\mathrm{SO}(q-1)$. For the fiber rotation, we consider the one-parameter subgroup generated by rotating the $q$-th coordinate. Since the points $\Psi\notin\ {\text{H}_{\mathbf{0}}}^{1+q}$ map the fundamental separator into a different separator, the remaining isometries can be thought without loss of generality as the group action of the points $\Psi\in\mathcal{M}^{[1,\,1+q]}$ with solvable coordinates

$$ $\Psi=[0,\,0,\,\dots,0,\,\Psi_{2,q}]^{\intercal}$ (47) $$

We obtain Eq. [15](https://arxiv.org/html/2505.24353v1#S3.E15) by combining these three isometries. In $\mathcal{M}^{[1,\,1+q]}$, totally geodesic hyperplanes can also be characterized as sets of points $\{\Upsilon\in\mathcal{M}^{[1,\,1+q]}\;s.t.\;\langle w,\log_{\Psi}(\Upsilon)
\rangle=0\}$, where $\log_{\Psi}$ is the logarithmic map at a fixed base point $\Psi\in\mathcal{M}^{[1,\,1+q]}$, and $w\in T_{\Psi}\mathcal{M}^{[1,\,1+q]}$ is a fixed vector. Indeed, we can also obtain Eq. [15](https://arxiv.org/html/2505.24353v1#S3.E15) from Eq. [36](https://arxiv.org/html/2505.24353v1#A3.E36) and this definition of hyperplanes.

The distance between a point and the submanifold $\text{H}_{\mathbf{0}}^{1+q}$ only depends on its $q$-th coordinate and is easily obtained by minimization and given by

$$ $d(\Upsilon,\pi_{0})=\dfrac{1}{2}\mathrm{arccosh}\left(1+2\,\Upsilon_{q+1}^{2}\right)$ (48) $$

Since every regression separator is the image of the subspace $\text{H}_{\mathbf{0}}^{1+q}$ through an isometry $\Phi$, $h_{\alpha,\beta,\boldsymbol{w}}(\Upsilon)$ in Eq. [15](https://arxiv.org/html/2505.24353v1#S3.E15) is proportional to the $q$-th coordinate of $\Phi(\Upsilon)$. The proportionality factor is $(|\boldsymbol{w}|^{2}-4\alpha\beta)^{-1}$, and from this we obtain Eq. [16](https://arxiv.org/html/2505.24353v1#S3.E16).

## Appendix G Numerical experiments

### G.1 Code

The code used for the experiments in this work is based on PyTorch, an open-source deep learning Python library [41]. Optimization routines, particularly those involving geometry-aware methods, utilize the Geoopt library [48].

The entire code to reproduce all the results shown in this article is available at

[https://github.com/CartanNetworks/Cartan_Networks](https://github.com/CartanNetworks/Cartan_Networks)

### G.2 Datasets

#### Synthetic datasets.

We construct some synthetic datasets using mathematical transformations of $d$-dimensional input vectors $x=(x_{1},\ldots,x_{d})$. Samples were drawn from a uniform distribution $\mathcal{U}([-1,1]^{d})$, namely a $d$-dimensional hypercube. We set $d=10$ for all our experiments.

The first dataset, $\frac{1}{d}\left(\sum_{i=1}^{d-1}x_{i}^{2}-x_{d}^{2}\right)$, involves a normalized difference between the sum of squared features (excluding the last feature, $x_{n}$) and the square of the last feature, introducing a global quadratic interaction across all dimensions. In all cases, 10 features are used as input, ensuring a consistent input dimensionality while varying the complexity and structure of the target function. Then the following two datasets, $\mathrm{Sinc}(\|x\|_{2})$ and $\mathrm{Sinc}(\|x\|_{3})$, apply the sinc function, defined as $\mathrm{sinc}(z)=\frac{\sin(z)}{z}$, to the $\ell_{2}$-norm and $\ell_{3}$-norm of the input vector, respectively. These introduce smooth radially symmetric variations based on the magnitude of the input vector under different norm constraints and have been proposed to test hyperbolic neural networks in [27]. The last datasets, $x_{1}+x_{1}x_{2}$ and $x_{1}+x_{1}x_{2}+x_{1}x_{2}x_{3}$, are a simple nonlinear combination of the first few features, representing a low-dimensional but nonlinearly interacting subset of the input (similar tasks have been discussed in [49]).

#### Real-world datasets.

We utilize four real-world benchmark datasets in our experiments; for these datasets, we use the standard train/test split provided by the torchvision library [41].

- •
MNIST [50], consisting of 70,000 grayscale images of handwritten digits (0–9) at 28×28 resolution.
- •
Fashion MNIST [51], which contains 70,000 grayscale images (28×28 pixels) of Zalando clothing items such as shirts, trousers, and shoes.
- •
K-MNIST [52], a dataset of 70,000 grayscale images (28×28 pixels) of Japanese characters from the Kuzushiji script.
- •
CIFAR-10 [53], composed of 60,000 color images (32×32 pixels) across ten categories, including animals (e.g., dogs, cats) and vehicles (e.g., cars, trucks).

### G.3 Numerical optimization

In contrast to Euclidean optimization, where gradients are computed in a flat vector space, Riemannian optimization takes into account the geometry of the manifold. Riemannian gradient methods compute gradients in this tangent space and use the retraction of the exponential maps to update parameters back onto the manifold. In Riemannian Stochastic Gradient Descent (RSGD) [38, 39], at each iteration $t$, the update is

$$ $\theta_{t+1}=\mathcal{R}_{\theta_{t}}(-\eta_{t}\nabla_{R}L(\theta_{t}))$ (49) $$

where $\nabla_{R}L(\theta_{t})=g^{-1}(\theta_{t})\,dL(\theta_{t})$ is the Riemannian gradient, $\eta_{t}$ is the learning rate, and $\mathcal{R}$ is a retraction that maps the tangent space back to the manifold. Since the exact exponential map is computationally expensive, we use a first-order approximation:

$$ $\mathcal{R}_{\theta}(v)=\theta+v$ (50) $$

where $v\in T_{\theta}\mathcal{M}$. In our implementation, the Riemannian versions of SGD and Adam were provided by Geoopt [48].

Due to the computationally intensive nature of the problem, classification datasets were optimized using early stopping with a buffer of 15 on the test loss for up to 1000 epochs, while the regression tasks were optimized for 5000 epochs on CPU only. Maximum reached accuracy and $R^{2}$ were reported for each run.

### G.4 Poincaré ball neural networks

To compare our results, we implemented hyperbolic neural networks [3] using the manifold parametrization provided by Geoopt [48]. A hyperbolic layer is given by

$$ $\mathrm{Poi}_{W,b}:=W\circ\mathrm{log}_{0}\circ\mathrm{exp}_{b}\circ P_{0 \rightarrow b}$ (51) $$

where $b$ is a point in the manifold and thus optimized via Riemannian Adam.
A neural network is obtained by alternating these layers and nonlinearities, stacked with a fully linear head, i.e.

$$ $\mathrm{PoiNN}:=W\circ\sigma\circ\mathrm{Poi}_{W_{n},b_{n}}\circ\sigma\>\circ \dots\circ\mathrm{Poi}_{W_{1},b_{1}}$ (52) $$

### G.5 Compute resources

For classification, models were trained on cluster nodes equipped with 2 NVIDIA Tesla P100 PCIe 16 GB each and Intel(R) Xeon(R) E5-2683 v4 @ 2.10GHz. Two models were trained concurrently per node, for a total of 140 compute hours for Tab. [2](https://arxiv.org/html/2505.24353v1#S4.T2) and 550 compute hours for KMNIST, 400 compute hours for CIFAR10, 425 compute hours for fMNIST and 400 compute hours for MNIST in Fig. [3](https://arxiv.org/html/2505.24353v1#S4.F3).
For regression tasks (Tab. [1](https://arxiv.org/html/2505.24353v1#S4.T1)), models were trained on a single machine equipped with two Intel(R) Xeon(R) Gold 6238R CPUs @ 2.20GHz processors for 6 hours.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. How do Cartan networks perform on very large-scale datasets or in high-dimensional hyperbolic spaces? Are there practical limits due to the complexity of solvable group operations and fiber rotations?

2. Can the representations learned by Cartan networks be transferred across tasks or datasets, similar to pre-trained Euclidean or graph neural networks? How well do they generalize beyond the training domain?
soundness: 3
strengths: 1. The proposed Cartan networks posses intrinsic Hyperbolic architecture which avoids the need for tangent-space approximations (as in Poincaré networks), making computations geometrically consistent and potentially more expressive.

2. Cartan networks allow stacking of homomorphisms and isometries, giving architectural flexibility and they are compatible with standard deep learning techniques (convolutions, batch normalization, dropout).

3. Cartan networks outperform Euclidean and standard hyperbolic networks on several benchmarks, including regression tasks, MNIST variants, CIFAR-100, and convolutional model.
summary: This paper proposes Cartan networks which is a new class of hyperbolic deep learning algorithms where group homomorphisms are interleaved with metric-preserving diffeomorphisms. In particular, the authors highlight the metric equivalence of the hyperbolic space with a solvable Lie group to exploit the group structure as a tool in architectural design. Extensive experiments show the proposed architectures achieve competitive or better performance w.r.t. Euclidean and standard hyperbolic neural networks on real and synthetic datasets.
weaknesses: 1. The proposed architectures seem complex and require careful handling of solvable coordinates, fiber rotations, and Lie group operations, which may be non-trivial to implement. Also, it is not clear if the the proposed architecture can be applied to large datasets and extended to larger architectures. 

2. Although parameter-efficient compared to some Euclidean layers, the matrix multiplications and exponential/fiber rotations could introduce extra computational cost.

3. The use of solvable group coordinates, fiber rotations, and Cartan/pain coordinates makes the learned representations even less interpretable compared to standard Euclidean embeddings.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: 1. Since the proposed method does not show convincing improvements over the baselines, especially against the Euclidean model on image tasks, why strong are the motivation for this method? Can the authors find an application or dataset where preserving the group structure of hyperbolic manifolds is actually beneficial?
2. Also see weakness
soundness: 3
strengths: 1. The proposed hyperbolic linear and regression layers based on group isomorphisms/homorphisms and series of solvable groups is interesting
2. The proposed layers are defined by operations that are rather intrinsic, which is a potential useful way to improve hyperbolic neural network operations
3. Experiments show incorporating activation in the model leads to improvement performance, which is consistent with the theory presented in the paper
summary: This paper proposes “Cartan networks,” a hyperbolic deep learning architecture that treats each layer as a solvable Lie group and maps between layers as a composition of a group homomorphism and an isometry. They proposed linear layers as a composition of homomorphisms between solvable groups with associated softmax and regression layers. They showed that adding pointwise activations recovers universal approximation and contains Euclidean networks as a special case. Their benchmark on synthetic and image classification tasks show in general comparable performance to Euclidean and tangent-space-based Poincare models.
weaknesses: 1. The experiments show more or less comparable performance against the Euclidean and Poincare model, with the few cases of noticeable improvements being synthetic datasets. This shows relatively weak motivation for the need for the model and whether the proposed method is actually effective
2. The baseline Poincare model is rather naive and simple. Since Ganea et al., 2018, many works have proposed hyperbolic linear and activation layers that lead to significant improvements (e.g. Chen et al. 2022, Shimizu et al. 2021, Yang et al. 2024). Missing comparison with these methods weakens the claims about the effectiveness of the proposed method as well. Coincidently, Chen et al. 2022 is also falsely cited to use nonlinearity on the tangent space on lines 358-359, when in reality they incorporate the nonlinearity into a fully hyperbolic layer. Yang et al. 2024 also does not use tangent space for nonlinearity.

Yang et al. 2024: https://arxiv.org/abs/2407.01290

3. The proposed method incurs computational overhead and rigidity into the model, even against some of the newer hyperbolic methods, but no analysis or discussion is given for this aspect.

---

confidence: 1
contribution: 2
initial_rating: 4
presentation: 2
questions: 1. You mention in the discussion that this new architecture could be more interpretable than existing architectures. Can you explain this claim a bit further?
2. What does metric equivalence mean within the context of your paper? The only common definition that I know is when two metrics on the same set induce some equivalent structure such as the same topology. However, I'm not sure I understand how this would work when there are two different underlying spaces.
soundness: 2
strengths: - The text of the paper seems well written.
- If the new formulation is correct, then it could be an interesting new perspective on hyperbolic learning.
- Again, if the formulation is correct, then automatically being able to generalize this kind of learning to any non-compact symmetric space could be very interesting as well.
summary: This paper introduces a new type of deep learning architecture named Cartan networks, where hidden features lie on solvable Lie groups and where each layer consists of a homomorphism from the source to the target group followed by an isometry within the target group. This type of network is motivated by a connection between non-compact symmetric spaces (such as hyperbolic space) and solvable Lie groups. The new type of layer is tested in small networks on small datasets.
weaknesses: I want to start by saying that I do not have a sufficiently strong background in Lie group theory for understanding the theoretical part of this paper, which spans most of it. As a result, I am incapable of providing any meaningful review of the theoretical part of the text. 

I do have significant experience with hyperbolic representation learning and computer vision, so I believe that I can judge the experimental part of this paper. There are, in my opinion, a few weaknesses to point out in the experiments:
1. The clearest weakness is the choice of datasets. Currently, the experiments involve a toy dataset, MNIST (and variants), CIFAR-10/100 and CelebA. All of these datasets are considered far too small and simple for drawing any conclusion regarding large scale networks. Therefore, I think some larger experiments (on ImageNet for example) should be included to support the authors' findings.
2. The experiments use small fully connected networks and an AlexNet-like network architecture. In order to see whether this type of layer actually scales to deep neural networks, the experiments should at least include something like a ResNet architecture in my opinion.
3. The layer is compared to the HNN layer in the Poincaré ball model by (Ganea et al., 2018) and to the standard Euclidean layer. However, as mentioned by the authors in the previous literature section, (Shimizu et al., 2021) have proposed an updated variant of the Poincaré layer and (Chen et al., 2022) have also proposed a (geometrically different) hyperboloid layer. A proper comparison should include at least both of these networks. ResNet-like architectures involving these layers already exist (Bdeir et al., 2024, [1]) and could easily be compared against as well.
4. The authors mention an increase in computational complexity, but the experiments section does not seem to contain a clear comparison in terms of actual computation time. Such an experiment would be very insightful. 
5. The results in the currently included experiments appear mixed to me, making it unclear to me what the benefit of this new formulation is.

Based on these weaknesses I am leaning towards rejection. However, given my lack of understanding of the theoretical parts, my rating should be taken with a grain of salt. 

If the authors can add the experiments that I mentioned above and find strong results, then I would be willing to increase my rating.

[1] Max van Spengler, Erwin Berkhout, and Pascal Mettes. *Poincaré resnet.* Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 1
questions: 1. What is truly new? Prior work already realizes isometries via exp/log pipelines, but the paper should prove that this yields strictly larger (or different) function classes than existing hyperbolic “linears,” beyond coordinate convenience.
soundness: 2
strengths: 1. Clear geometric formulation of isometries and homomorphisms with explicit coordinate formulas
2. The architecture is easy to implement with standard Riemannian optimizers (GeoOpt) and the code is promised.
summary: The paper proposes Cartan Networks, a hyperbolic deep learning architecture that treats hyperbolic space as a solvable Lie group (via its symmetric-space structure) and builds layers by alternating group homomorphisms and isometries. This yields closed-form “linear” layers (mixing Cartan/fiber coordinates) and a hyperbolic logistic/softmax based on geodesic hyperplanes.
weaknesses: 1. Most of what is framed as new (stacking “linear” layers built from isometries + homomorphisms; softmax via hyperbolic hyperplanes) is already implicit or explicit in prior gyrovector/Poincaré constructions and Lie-group views of $H^n$. The paper itself notes that prior hyperbolic layers parametrize all isometries (via exp/log + parallel transport) and that its layer differs mainly by coordinate choice and homomorphism tying. This reads as a repackaging rather than a fundamentally new capability.
2.  Benchmarks are small, with parity on CelebA/CIFAR-10 and a single strong gain on CIFAR-100; there is no task where Euclidean/Poincaré fails but Cartan succeeds, nor graphs/hierarchies where hyperbolic geometry is a priori motivated. This makes the practical utility unclear.
3. Hyperbolic hyperplanes and distance-monotone logits are standard (e.g., Poincaré SVM/softmax). The paper acknowledges they are the same subspaces as prior work; the novelty is just the solvable-coord formula.


Decision: Reject