Mixed-curvature decision trees and random forests

## Abstract

Abstract We extend decision tree and random forest algorithms to mixed-curvature product spaces.
Such spaces, defined as Cartesian products of Euclidean, hyperspherical, and hyperbolic manifolds, can often embed points from pairwise distances with much lower distortion than in single manifolds.
To date, all classifiers for product spaces fit a single linear decision boundary, and no regressor has been described.
Our method overcomes these limitations by enabling simple, expressive classification and regression in product manifolds.
We demonstrate the superior accuracy of our tool compared to Euclidean methods operating in the ambient space for component manifolds covering a wide range of curvatures, as well as on a selection of product manifolds.

## 1 Introduction

Product space manifolds, first described by Gu et al. (), can capture complex patterns in pairwise distances with much lower distortion than single manifolds can.
As product space manifolds are simply Cartesian products of several constant-curvature component manifolds, and since most operations in product space manifolds neatly factorize across these component manifolds, they are quite elegant.
Such manifolds are used in biology (McNeela et al., ) and knowledge graph representation (Wang et al., ).

However, the uptake of product manifold embeddings has thus far been limited.
This is partially due to a lack of tools for even the simplest downstream inference tasks, such as classification and regression on the basis of product manifold coordinates.
In fact, to our knowledge only Skopek et al. () have ever proposed an approach to this problem at all.
Their approach, while very cogent and influential to this paper, suffers from the major drawback of relying on a single linear decision boundary, which lacks the expressiveness of tools like decision trees and random forests.

Our inductive bias is that a good decision boundary should partition the space into convex decision regions and be equidistant to the closest two points it separates.
For efficiency, we would also like to learn the entire tree in $O(ndt)$ time, where $n$ is the number of points, $d$ is the number of dimensions, and $t$ is the maximum depth of the tree.
Since Euclidean decision trees violate convexity for some manifolds, we must modify them to suit these manifolds better.
We propose such a method that works for all component manifolds, modify it further to work on product space manifolds, and benchmark both methods’ effectiveness.

Our contributions.
Concretely, we contribute:

- 1.
A generalized algorithm for fitting decision trees to hyperspherical, Euclidean, and hyperbolic manifolds,
- 2.
A novel algorithm for fitting decision trees on product space manifolds,
- 3.
A generalized technique for sampling Gaussian mixtures in product space manifolds, and
- 4.
A preliminary benchmark demonstrating the effectiveness of our component- and product-space trees over classical decision trees for synthetic datasets.

## 2 Preliminaries

### 2.1 Riemannian manifolds

We will begin by reviewing key details of hyperspheres, hyperboloids, and Euclidean spaces.
For more details, readers can consult Do Carmo ().

Each of the spaces described is a Riemannian manifold, meaning that it is locally isomorphic to Euclidean space and equipped with a distance metric.
The shortest path between two points $\mathbf{u}$ and $\mathbf{v}$ on a manifold are referred to as geodesics.
As all three spaces we consider have constant Gaussian curvature, we define simple closed-forms for geodesic distances in each of the following subsections, in lieu of a more general discussion of geodesic distances in arbitrary Riemannian manifolds.

Each of our manifolds is described by a dimensionality $D$ and a curvature $K$.
They can also all be thought of as being embedded in an ambient space $\mathbb{R}^{D+1}$.

#### 2.1.1 Euclidean space

First, we define the inner product (the dot product) used in Euclidean space:

$$ $\langle\mathbf{u},\mathbf{v}\rangle=u_{0}v_{0}+u_{1}v_{1}+\ldots+u_{2}v_{2}$ (1) $$

Additionally, we use $\|\mathbf{u}\|=\sqrt{\langle\mathbf{u},\mathbf{u}\rangle}$.
Then, we can define the Euclidean distance function (the $\ell_{2}$ norm):

$$ $\delta_{\mathbb{E}}(\mathbf{u},\mathbf{v})=\|\mathbf{u}-\mathbf{v}\|$ (2) $$

#### 2.1.2 Hyperspherical space

We will describe hyperspherical space in terms of its coordinates in the ambient space.
Hyperspherical space uses the same inner products as Euclidean space.
The hypersphere is the set of points of constant Euclidean norm:

$$ $\mathbb{S}^{D,K}=\{\mathbf{x}\in\mathbb{R}^{D}\ :\ \|x\|=1/K\}$ (3) $$

We use the hyperspherical distance function:

$$ $\delta_{\mathbb{S}}(\mathbf{u},\mathbf{v})=\cos^{-1}(K^{2}\langle\mathbf{u},\mathbf{v}\rangle)/K.$ (4) $$

#### 2.1.3 Hyperbolic space

We will describe the hyperbolic space from the perspective of the hyperboloid model.
First, we need to define the ambient Minkowski space.
This is a vector space equipped with the Minkowski inner product:

$$ $\langle\mathbf{u},\mathbf{v}\rangle_{\mathcal{L}}=-u_{0}v_{0}+u_{1}v_{1}+\ldots+u_{n}v_{n}$ (5) $$

Analogous to the Euclidean case, we let $\|\mathbf{u}\|_{\mathcal{L}}=\sqrt{\langle\mathbf{u},\mathbf{u}\rangle_{\mathcal{L}}}$.
The hyperboloid of dimension $D$ and curvature $-K$, written $\mathbb{H}^{D,K}$, is a set of points with constant Minkowski norm:

$$ $\mathbb{H}^{D,K}=\{\mathbf{x}\in\mathbb{R}^{D}\ :\ \|\mathbf{x}\|_{\mathcal{L}}=-1/K\},$ (6) $$

with the further restriction that $x_{0}>0$, i.e. we only consider the upper sheet of the hyperboloid.

Finally, this space uses a hyperbolic distance function:

$$ $\delta_{\mathbb{H}}(\mathbf{u},\mathbf{v})=\cosh^{-1}(-K^{2}\langle u,v\rangle_{\mathcal{L}})/K$ (7) $$

#### 2.1.4 Mixed-curvature product spaces

Following the convention of using $\prod_{i}\mathbf{X_{i}}$ to refer to the iterated Cartesian product over sets, we define a product space manifold as

$$ $\mathcal{P}=\prod_{i=1}^{n}\mathbb{S}^{s_{i},K_{i}}\times\prod_{j=1}^{m}\mathbb{H}^{h_{j},K^{\prime}_{j}}\times\mathbb{E}^{d}$ (8) $$

The total number of dimensions is $\sum_{i}s_{i}+\sum_{j}h_{j}+d$.
Each individual manifold is called a component manifold.
The decomposition of the product space into component manifolds is called the signature.
Informally, the signature can be thought of as a list of dimensionalities and curvatures for each component manifold.

Distances in product manifolds decompose as the $\ell_{2}$ norm of the distances in each of the component manifolds:

$$ $\delta_{\mathcal{P}}(\mathbf{u},\mathbf{v})=\sqrt{\sum_{\mathcal{M}\in\mathcal{P}}\delta_{\mathcal{M}}(\mathbf{u},\mathbf{v})^{2}},$ (9) $$

where $\mathbf{u}$ and $\mathbf{v}$ are appropriately restricted to their coordinates in each component manifold and $\delta_{\mathcal{M}}$ refers the distance function in $\mathcal{M}$.

Note that this is the same formulation as in Gu et al. ().

### 2.2 Decision trees

The Classification and Regression Trees (CART) (Breiman, ) algorithm fits decision trees to a set of labels $\mathbf{y}$.
Specifically, it greedily selects a split at each set to partition the dataset in such a way as to maximize the information gain:

$$ $\operatorname{IG}(\mathbf{y})=C(\mathbf{y})-\frac{|\mathbf{y}^{+}|}{|\mathbf{y}|}C(\mathbf{y}^{+})-\frac{|\mathbf{y}^{-}|}{|\mathbf{y}|}C(\mathbf{y}^{-})$ (10) $$

In this case, $C(\cdot)$ is some sort of impurity function (in this paper, Gini impurity for classification, and variance for regression).
Some splitting function $S(\cdot)$ is used to partition the labels $\mathbf{y}$ into two classes, $\mathbf{y}^{+}$ and $\mathbf{y}^{-}$; however, $S(\cdot)$ also partitions the input space (corresponding to some $\mathbf{X}$ that does not appear in Eq. [10](#S2.E10)) into decision regions.
Classically, $S(\cdot)$ is a thresholding function and thus breaks the input space into box-like regions.

This algorithm is applied recursively to each of the decision regions until some stopping condition is met (for instance, information gain fails to hit some minimum threshold).
The result is a fitted decision tree, $\mathcal{T}$, which can be used for inference.
During inference, an unseen point $\mathbf{x}$ is passed through the decision tree until it reaches a leaf node corresponding to some decision region.
For classification, the point is then assigned the majority label inside that region; for regression, it is assigned the mean value inside that region.

Finally, a random forest is an ensemble of decision trees that is typically trained on a subsample of the points and features in $\mathbf{X}$ (Breiman, ).

## 3 Mixed-curvature decision trees

For each component manifold, we adapt the method, first described in Chlenski et al. (), to fit decision trees using homogenous hyperplanes, i.e. hyperplanes that contain the origin of the ambient space.
Such hyperplanes intersect any component manifold at geodesic submanifolds, meaning that if we let $\mathbb{P}$ be a plane, then $\mathcal{M}\cap\mathbb{P}$ contains all shortest paths between its own elements in the metric defined on $\mathcal{M}$.

For any decision tree, we must transform the input $\mathbf{X}$ into a set of candidate hyperplanes.
We consider a restricted set of homogenous hyperplanes whose normal vectors are nonzero in at most two dimensions: one dimension $d$, which varies, and one dimension (always the first dimension in our component manifolds, which we denote using index 0) called the special dimension.
The special dimension is used in every decision boundary fitting.

Given a dimension $d$ and a manifold $\mathcal{M}$, we first characterize each point as an angle:

$$ $\theta(\mathbf{x},d)=\tan^{-1}(x_{0}/x_{d})$ (11) $$

Note that, in our implementation, we use the NumPy arctan2 function to ensure that we can recover the full range of angles in $[0,2\pi)$.
This is essential for properly specifying decision boundaries in hyperspherical cases.

Once all angles are computed, we must compute angular midpoints such that we can find a hyperplane that intersects $\mathcal{M}$ in a location that is geodesically equidistant to both of the points considered.
We provide angular midpoint formulas for each of the component manifolds in the following sections.

Given a set of angular midpoints, we fit a decision tree using the usual dot-product reformulation of decision boundaries by greedily maximizing the information gain (as defined in Eq. [10](#S2.E10)) at each decision split until the maximum depth is reached, or other stopping criteria are met.
The exact annotation for a single point $\mathbf{x}$, a dimension index $d$, and an angle $\theta$, is given as

$$ $S(\mathbf{x},d,\theta)=\operatorname{sign}(\sin(\theta)x_{d}-\cos(\theta)x_{0}).$ (12) $$

The rest of the algorithm exactly follows the description in Section [2.2](#S2.SS2).

### 3.1 Euclidean decision trees

While homogenous hyperplanes in $\mathbb{R}^{D}$ trivially intersect $\mathbb{E}^{D}$ at geodesic submanifolds, these lack the expressiveness of an ambient-space formulation.
Thus, we instead think of $\mathbb{E}^{D}$ as embedded inside of $\mathbb{R}^{D+1}$ by applying a trivial lifting embedding:

$$ $\phi:\mathbb{R}^{D}\to\mathbb{R}^{D+1},\ \phi(\mathbf{u})=(1,\mathbf{u}).$ (13) $$

For two points $\mathbf{u},\mathbf{v}\in\mathbb{E}^{D}$, the midpoint angles in $\mathbb{E}^{D}$ are

$$ $m_{\mathbb{E}}(\mathbf{u},\mathbf{v})=\tan^{-1}(2/(u_{d}+v_{d}).$ (14) $$

While this presentation of Euclidean decision trees is unconventional, it is completely equivalent to thresholding in the basis dimensions.
See Appendix [C](#A3) for the proof.

### 3.2 Hyperbolic decision trees

For two points $\mathbf{u},\mathbf{v}\in\mathbb{H}^{D,K}$, we compute $\theta_{u}$ and $\theta_{v}$ according to Eq [11](#S3.E11) and follow Chlenski et al. () in computing the hyperbolic midpoint angle in $\mathbb{H}^{D,K}$ as:

$$ $\displaystyle V$ $\displaystyle:=\frac{\sin(2\theta_{u}-2\theta_{v})}{2\sin(\theta_{u}+\theta_{v})\sin(\theta_{v}-\theta_{u})}$ (15) $\displaystyle m_{\mathbb{H}}(\mathbf{u},\mathbf{v})$ $\displaystyle=\begin{cases}\cot^{-1}(V-\sqrt{V^{2}-1})&\text{ if }\theta_{u}+\theta_{v}<\pi\\ \cot^{-1}(V+\sqrt{V^{2}-1})&\text{ otherwise}\end{cases}$ (16) $$

### 3.3 Hyperspherical decision trees

The hyperspherical case is quite simple, except that unlike hyperbolic space and the “lifted” Euclidean space after applying Eq [13](#S3.E13), we lack a natural choice of special dimension.
We follow the convention of using the first dimension of the embedding space as the special dimension, which intuitively corresponds to fixing a “north pole” in the first dimension.

Given $\mathbf{u},\mathbf{v}\in\mathbb{S}^{D,K}$, the hyperspherical midpoint angle is the arithmetic arithmetic mean of $\theta_{u}$ and $\theta_{v}$ (computed using Eq [11](#S3.E11)):

$$ $m_{\mathbb{S}}(\mathbf{u},\mathbf{v})=(\theta_{u}+\theta_{v})/2$ (17) $$

### 3.4 Product space algorithm

Intuitively, the only modification of a decision tree in a single component manifold is that we now iterate over all (non-special) dimensions of $\mathcal{P}$ and keep track of which special dimension must be used when computing angles for candidate hyperplanes.
The full algorithm is given in Algorithm [1](#alg1).

Allowing for a single decision tree to span all components—rather than, for instance, an ensemble of decision trees each operating in a single component—allows the model to discover which component manifolds are most relevant to a given task, and to allocate more capacity to those.

Complete pseudocode for this algorithm is in Appendix [1](#alg1).

Note also that the regression and random forest formulations described in Section [2.2](#S2.SS2) can be applied to product space decision trees directly, without further modification.

## 4 Benchmarks

### 4.1 Sampling Gaussian mixtures in 𝒫 𝒫 \mathcal{P}

We extend the method developed by Nagano et al. () for Gaussians in $\mathbb{H}$ to sample Gaussian mixtures in $\mathcal{P}$.
First, we follow Chlenski et al. () in defining Gaussian mixtures in $\mathbb{H}$ by sampling a set of means in the tangent plane at the origin of $\mathcal{M}$ and projecting them directly onto $\mathcal{M}$.
We add three further modifications:

- 1.
We use the Wishart distribution (Wishart, ) to generate covariance matrices for Gaussian mixtures (Chatfield & Collins, );
- 2.
We rescale the covariance matrix according to the curvature to ensure the distribution of distances to the mean stays consistent across all submanifolds; and
- 3.
To generate mixtures of gaussians in $\mathcal{P}$, we concatenate together embeddings from each component $\mathcal{M}$.

Full details of our sampling method are in Appendix [A](#A1).

### 4.2 Component manifold benchmarks

Figure: Figure 1: Classification accuracies for Gaussian mixtures on manifolds of varying curvatures using our method (blue) and a CART decision tree (red). The difference in accuracy between the methods becomes more pronounced with higher curvatures, and the methods are identical (not just comparable) at curvature 0. Statistical significance ($p<0.05$) is marked with an asterisk.
Refer to caption: /html/2406.05227/assets/figures/fig1.png

In Figure [1](#S4.F1), we compare our method to CART in classification accuracy on Gaussian mixtures on a single component manifold $\mathcal{M}$ for both positive and negative curvatures.
For each curvature, we tested 100 different samples of 1,000 points in two dimensions with four classes using an 80:20 train-test split.
The negative-curvature results extend earlier findings by Chlenski et al. () to the full range of negative curvatures; the positive-curvature results are novel.
Both methods perform identically for curvature 0, corroborating the equivalence between our method and CART in the Euclidean case as shown in Appendix [C](#A3).

### 4.3 Product manifold benchmarks

In Table [1](#S4.T1), we show that our method outperforms Euclidean decision trees in classification accuracies across all tested product manifolds.
We follow Gu et al. () in choosing the six signatures tested, but opt to use a Gaussian mixture rather than a graph embedding for our benchmark.
For each signature, we tested 20 different samples of 1,000 points with four classes using an 80:20 train-test split.
Notably, our method outperformed Euclidean decision trees on all manifolds tested.
Comparable random forest results are found in Table [2](#A4.T2) in the Appendix.

**Table 1: Benchmark results. We report 95% confidence intervals for classification accuracies on mixtures of four Gaussians over a representative sample of product manifolds. Each manifold has 10 total dimensions. Higher scores are bolded; asterisks mark statistical significance ($p<.05$).**
| Signature | Euc. DT | Prod. DT |
| --- | --- | --- |
| $(\mathbb{H}^{5,1})$ | $93.6\pm 1.1$ | $\mathbf{97.3\pm 0.5^{*}}$ |
| $(\mathbb{S}^{5,1})$ | $62.2\pm 1.6$ | $\mathbf{64.1\pm 1.5^{*}}$ |
| $\mathbb{H}^{5,1}\times\mathbb{S}^{5,1}$ | $93.7\pm 1.1$ | $\mathbf{97.3\pm 0.5^{*}}$ |
| $(\mathbb{H}^{2,1})^{5}$ | $76.9\pm 1.9$ | $\mathbf{78.6\pm 1.9^{*}}$ |
| $(\mathbb{S}^{2,1})^{5}$ | $60.4\pm 1.9$ | $\mathbf{60.6\pm 1.9}$ |
| $(\mathbb{H}^{2,1})^{2}\times(\mathbb{S}^{2,1})^{2}\times\mathbb{E}^{2}$ | $78.2\pm 1.9$ | $\mathbf{79.5\pm 1.9^{*}}$ |

## 5 Conclusion

##### Contributions.

We present strong preliminary evidence in favor of mixed-curvature decision tree algorithms.
In particular, we motivate and describe our entire algorithm, and demonstrate the effectiveness of our method on effectiveness on component manifolds of all curvatures.
We also offer some evidence of our method’s effectiveness for Gaussian mixtures in product spaces.

##### Future work.

Future work will prioritize more thorough benchmarking of our methods.
In particular, we will compare our method to the product space classifier forms described in Skopek et al. () and benchmark node classification on mixed-curvature embeddings for graphs embedded according to the method of Gu et al. ().
We are especially interested in seeking out high-quality datasets for benchmarking the regression capabilities of our method, as well as in applications to biological data such as spatial transcriptomic.

Other work can include implementing algorithmic improvements such as boosting, exploring the implications of the lack of privileged basis dimensions in non-Euclidean spaces on the performance of decision tree algorithms, implementing other mixed-curvature classifiers such as multinomial and linear regression, testing ensembles of trees individually trained on single component manifolds, and exploring kernel-based interpretations of these methods.

##### Limitations.

Although this work is still in the early stages of implementation and benchmarking, we believe our method is mature enough to share.
However, one limitation of our method is its discontinuity.
Although we claim to vary curvature smoothly, the distance functions must change during the transition from negative to zero to positive curvature.
Moreover, though the limit of the radius as $K$ approaches $\infty$ is also $\infty$, we lift our Euclidean embeddings to $x_{0}=1$ instead.
A continuous unification and thorough benchmarks would substantially improve this work.

## Appendix A Gaussian mixture details

### A.1 Overall structure

The structure of our sampling algorithm is as follows.
Note that, rather than letting $\mathcal{M}$ be a manifold of arbitrary curvature, we force its curvature to be one of $\{-1,0,1\}$ for implementation reasons.
This necessitates rescaling steps, which take place in Equations [24](#A1.E24), [28](#A1.E28), and [34](#A1.E34).
The result is equivalent to performing the equivalent steps, without rescaling, on a manifold of the proper curvature.

- 1.
Generate $\mathbf{c}$, a vector that divides $m$ samples into $n$ clusters:
$\displaystyle\mathbf{p_{raw}}$
$\displaystyle=\langle p_{0},p_{1},\ldots,p_{n-1}\rangle$
(18)
$\displaystyle p_{i}$
$\displaystyle\sim\text{Uniform}(0,1)$
(19)
$\displaystyle\mathbf{p_{norm}}$
$\displaystyle=\frac{\mathbf{p_{raw}}}{\sum_{i=0}^{n-1}p_{i}}$
(20)
$\displaystyle\mathbf{c}$
$\displaystyle=\langle c_{0},c_{1},\ldots c_{m-1}\rangle$
(21)
$\displaystyle c_{i}$
$\displaystyle\sim\text{Categorical}(n,\mathbf{p_{norm}})$
(22)
- 2.
Sample $\mathbf{M_{euc}}$, an $n\times D$ matrix of $n$ class means:
$\displaystyle\mathbf{M_{euc}}$
$\displaystyle=\langle\mathbf{m_{0},m_{1},\ldots,m_{n-1}}\rangle^{T}$
(23)
$\displaystyle\mathbf{m_{i}}$
$\displaystyle\sim\mathcal{N}(0,\sqrt{K}\mathbf{I}).$
(24)
- 3.
Move $\mathbf{M_{euc}}$ into $T_{0}\mathcal{M}$, the tangent plane at the origin of $\mathcal{M}$, by applying $\psi:\mathbf{x}\to(0,\mathbf{x})$ per-row to $\mathbf{M_{euc}}$:
$\displaystyle\mathbf{M_{tan}}$
$\displaystyle=\langle\psi(\mathbf{m_{0}}),\psi(\mathbf{m_{1}}),\ldots\psi(\mathbf{m_{n-1}})\rangle^{T},$
(25)
$\displaystyle\psi:\mathbb{R}^{D}$
$\displaystyle\to\mathbb{R}^{D+1},\ \mathbf{x}\to\langle 0,\mathbf{x}\rangle.$
(26)
- 4.
Project $\mathbf{M_{tan}}$ onto $\mathcal{M}$ using the exponential map from $T_{0}\mathcal{M}$ to $\mathbf{M_{tan}}$:
$\mathbf{M}=\exp_{0}(\mathbf{M_{tan}}).$
(27)
- 5.
For $0\leq i<n$, sample a corresponding covariance matrix. Here, $\sigma$ is a variance scale parameter than can be set:
$\mathbf{\Sigma_{i}}\sim\text{Wishart}(\sigma\sqrt{K}\mathbf{I},D)$
(28)
- 6.
For $0\leq j<m$, sample $\mathbf{X_{euc}}$, a matrix of $m$ points according to their clusters’ covariance matrices:
$\displaystyle\mathbf{X_{euc}}$
$\displaystyle=\langle\mathbf{x_{0},x_{1},\ldots x_{m-1}}\rangle^{T}$
(29)
$\displaystyle x_{j}$
$\displaystyle\sim\mathcal{N}(0,\mathbf{\Sigma_{c_{j}}}).$
(30)
- 7.
Apply $\psi(\cdot)$ from Eq [26](#A1.E26) to each $\mathbf{x_{j}}$ to move it into $T_{0}\mathcal{M}$:
$\mathbf{X_{tan}}=\langle\psi(\mathbf{x_{0}}),\psi(\mathbf{x_{1}}),\ldots\psi(\mathbf{x_{m-1}})\rangle^{T}.$
(31)
- 8.
For each row in $\mathbf{X_{tan}}$, apply parallel transport from $T_{0}\mathcal{M}$ to its class mean:
$\displaystyle\mathbf{X_{PT}}$
$\displaystyle=\langle\mathbf{x_{0,\mu},x_{1,\mu},\ldots,x_{m-1,\mu}}\rangle$
(32)
$\displaystyle\mathbf{\mathbf{x}_{j,\mu}}$
$\displaystyle=PT_{0\to\mathbf{m_{c_{j}}}}(\mathbf{x_{j}})$
(33)
- 9.
Use the exponential map at $T_{\mu}\mathcal{M}$ to move the points onto the manifold:
$\displaystyle\mathbf{X}_{\mathcal{M}}$
$\displaystyle=\langle\mathbf{x_{0,\mathcal{M}},x_{1,\mathcal{M}},\ldots,x_{m-1,\mathcal{M}}}\rangle$
(34)
$\displaystyle\mathbf{x_{j,\mathcal{M}}}$
$\displaystyle=\frac{\exp_{\mathbf{m_{c_{j}}}}(\mathbf{x_{j,\mu}})}{\sqrt{K}}$
(35)
- 10.
Repeat steps 2–9 for as many manifolds as desired; produce a final embedding by concatenating all component embeddings column-wise:
$\mathbf{X}=\langle\mathbf{X}_{\mathcal{M}_{0}},\mathbf{X}_{\mathcal{M}_{1}},\ldots\mathbf{X}_{\mathcal{M}_{p}}\rangle$
(36)

### A.2 Equations for manifold operations

First, we provide the forms of the parallel transport operation in hyperbolic, hyperspherical, and Euclidean spaces:

$$ $\displaystyle PT_{\nu\to\mu}^{\mathbb{H}}(\mathbf{v})$ $\displaystyle=\mathbf{v}+\frac{\langle\mu-\alpha\nu,\nu\rangle_{\mathcal{L}}}{\alpha+1}(\nu+\mu)$ (37) $\displaystyle\alpha$ $\displaystyle=-\langle\nu,\mu\rangle_{\mathcal{L}}$ (38) $\displaystyle PT_{\nu\to\mu}^{\mathbb{S}}(\mathbf{v})$ $\displaystyle=\mathbf{v}\cos(d)+\frac{\sin(d)}{d}(\mu-\cos(d)\nu)$ (39) $\displaystyle d$ $\displaystyle=\cos^{-1}(\nu\cdot\mu)$ (40) $\displaystyle PT_{\nu\to\mu}^{\mathbb{E}}(\mathbf{v})$ $\displaystyle=\mathbf{v}+\mathbf{\mu-\nu}.$ (41) $$

The exponential map is defined as follows in each of the three spaces:

$$ $\displaystyle\exp_{\mu}(\mathbf{u})$ $\displaystyle=\cosh(\|\mathbf{u}\|_{\mathcal{L}})\mu+\sinh(\|\mathbf{u}\|_{\mathcal{L}})\frac{\mathbf{u}}{\|\mathbf{u}\|_{\mathcal{L}}}$ (42) $\displaystyle\exp_{\mu}(\mathbf{u})$ $\displaystyle=\cos(\|\mathbf{u}\|)\mu+\sin(\|\mathbf{u}\|)\frac{\mathbf{u}}{\|\mathbf{u}\|}$ (43) $\displaystyle\exp_{\mu}(\mathbf{u})$ $\displaystyle=\mathbf{u}.$ (44) $$

### A.3 Relationship to other work

Nagano et al. () developed the overall technique used for a single cluster and a single manifold, i.e. steps 6–9.
Chlenski et al. () modified this method to work for mixtures of Gaussians in $\mathbb{H}^{d,1}$, and deployed it for $d\in{2,4,8,16}$.
This corresponds to steps 1–5 of our procedure (although note that our covariance matrices are sampled differently in step 5).
Thus, our contribution is simply to add step 10, modify step 5 to use the Wishart distribution, and to add curvature-related scaling factors in Equations [24](#A1.E24), [28](#A1.E28), and [34](#A1.E34).

Additionally, to the best of our knowledge we are the first to apply this to hyperspherical manifolds, for which the von Mises-Fisher (VMF) distribution is typically preferred.
We do not argue for the superiority of our approach over the VMF distribution in general; however, we prefer to use ours for these benchmarks, as it allows us to draw simpler parallels between manifolds of different curvatures.

## Appendix B Product space decision tree pseudocode

Figure: Algorithm 1 Product Space Decision Tree

## Appendix C Proof of equivalence for Euclidean case

A classical CART tree splits data points according to whether their value in a given dimension is greater than or less than some threshold value $t$.
Midpoints are simple arithmetic means.
This can be written as:

$$ $\displaystyle S^{\prime}(\mathbf{x},d,t)$ $\displaystyle=\begin{cases}1\text{ if }x_{d}>t,\\ 0\text{ otherwise}.\end{cases}$ (45) $\displaystyle m_{DT}(\mathbf{u},\mathbf{v})$ $\displaystyle=\frac{u_{d}+v_{d}}{2}.$ (46) $$

In our transformed decision tree, we lift the data points by applying $\phi:\mathbf{x}\rightarrow(1,\mathbf{x})$ and then check which side of an axis-inclined hyperplane they fall on.
The splitting function is based on the angle $\theta$ of inclination with respect to the $(0,d)$ plane, i.e., $\langle 1,x_{d}\rangle$.
Our midpoints are computed to ensure equidistance in the original manifold:

$$ $\displaystyle S(\mathbf{x},d,\theta)$ $\displaystyle=\operatorname{sign}(\sin(\theta)x_{d}-\cos(\theta)x_{0})$ (47) $\displaystyle m_{\mathbb{E}}(\mathbf{u},\mathbf{v})$ $\displaystyle=\tan^{-1}\left(\frac{2}{u_{d}+v_{d}}\right)$ (48) $$

To demonstrate the equivalence of the classical decision tree formulation to our transformed algorithm in $\mathbb{E}$, we will show that Equation [45](#A3.E45) is equivalent to Equation [47](#A3.E47) and Equation [46](#A3.E46) is equivalent to Equation [48](#A3.E48) under

$$ $\theta=\cot^{-1}(t).$ (49) $$

### C.1 Equivalence of Splits

First, we show that Equations [45](#A3.E45) and [47](#A3.E47) are equivalent, assuming $t\neq 0$:

$$ $\displaystyle S(\mathbf{x},d,\theta)$ $\displaystyle=\operatorname{sign}(\sin(\theta)x_{d}-\cos(\theta)x_{0})=1$ (50) $\displaystyle\iff\sin(\theta)x_{d}-\cos(\theta)>0$ (51) $\displaystyle\iff\frac{\sin(\theta)}{\cos(\theta)}x_{d}=\tan(\theta)x_{d}>1$ (52) $\displaystyle\iff x_{d}/t>1$ (53) $\displaystyle\iff x_{d}>t$ (54) $\displaystyle\iff S^{\prime}(\mathbf{x},d,t)=1$ (55) $$

### C.2 Equivalence of midpoints

Now, we show that Equations [46](#A3.E46) and [48](#A3.E48) are equivalent:

$$ $\displaystyle\cot^{-1}(m_{DT}(\mathbf{u},\mathbf{v}))$ $\displaystyle=\cot^{-1}\left(\frac{u_{d}+v_{d}}{2}\right)$ (56) $\displaystyle=\tan^{-1}\left(\frac{2}{u_{d}-v_{d}}\right)$ (57) $\displaystyle=m_{\mathbb{E}}(\mathbf{u},\mathbf{v})$ (58) $$

## Appendix D Full benchmark table

We report the full version of Table [1](#S4.T1) here, in Table [2](#A4.T2).

**Table 2: Benchmark results with decision trees and random forests. We report 95% confidence intervals for classification accuracies on mixtures of four Gaussians over a representative sample of product manifolds. Each manifold has 10 total dimensions. Higher scores are bolded; asterisks mark statistical significance ($p<.05$).**
|  | Euclidean | Product space | Euclidean | Product space |
| --- | --- | --- | --- | --- |
| Signature | decision tree | decision tree | random forest | random forest |
| $(\mathbb{H}^{5,1})$ | $93.6\pm 1.1$ | $\mathbf{97.3\pm 0.5^{*}}$ | $90.2\pm 3.2$ | $\mathbf{97.0\pm 1.4^{*}}$ |
| $(\mathbb{S}^{5,1})$ | $62.2\pm 1.6$ | $\mathbf{64.1\pm 1.5^{*}}$ | $60.3\pm 4.4$ | $\mathbf{61.1\pm 4.0}$ |
| $\mathbb{H}^{5,1}\times\mathbb{S}^{5,1}$ | $93.7\pm 1.1$ | $\mathbf{97.3\pm 0.5^{*}}$ | $90.3\pm 2.5$ | $\mathbf{97.1\pm 1.4^{*}}$ |
| $(\mathbb{H}^{2,1})^{5}$ | $76.9\pm 1.9$ | $\mathbf{78.6\pm 1.9^{*}}$ | $78.0\pm 4.7$ | $\mathbf{80.8\pm 4.2^{*}}$ |
| $(\mathbb{S}^{2,1})^{5}$ | $60.4\pm 1.9$ | $\mathbf{60.6\pm 1.9}$ | $58.2\pm 4.4$ | $\mathbf{58.3\pm 4.7}$ |
| $(\mathbb{H}^{2,1})^{2}\times(\mathbb{S}^{2,1})^{2}\times\mathbb{E}^{2}$ | $78.2\pm 1.9$ | $\mathbf{79.5\pm 1.9^{*}}$ | $78.1\pm 4.8$ | $\mathbf{81.6\pm 3.9^{*}}$ |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: 1- Could the authors comment on this statement: "To this end, Chlenski et al. (2024) impose homogeneity and sparsity constraints on the hyperplanes they consider for hyperbolic DTs". If I understand correctly, the sparsity constraint imposes that the hyperplanes are defined by only 2 quantities (the timelike coordinate and the x_d, which makes the number of candidate subspaces tractable). However, could the authors explain why homogeneity is an important attribute of these subspaces? If  $\mathbb{P} \cap \mathbb{H}$ was not closed under shortest paths, what would happen?  Or is it just another attribute that is necessary for the thresholding?

2- Interpretability: in the euclidean case, RF and DT are great because they're non linear and more powerful than linear regression, but they're reasonably interpretable too (DT are very interpretable, RF get away with the feature importance sorting). Is there any notion that this could be the case in the curved manifold setting?

3- Parameters: I am a little unclear about how to use the algorithm though:
- how do we choose the product manifold? In the experiments (e.g. MNIST), there is a combination of S, E  and H. How do we go about determining what is the right combination? Is there some selection/tuning involved?
- In figure 7: "feature subsampling approaches in RFs." Is the number of features referring to the standard procedure in the RF algorithm where only a fraction of the samples are seen, to de-correlate trees?
- in figure 7: I don't understand what the caption a is referring to? Which part of the algorithm?
soundness: 4
strengths: 1. Curved manifolds (and product manifolds) have received already som attention from the ML community. Most of these approaches (at least, to my shallow knowledge of the literature) however, have focused on extending deep neural network methods. The authors contend here that an extension of the DT and RF would add a missing brick to the set of available tools for data analysts.

2. **Contribution of the authors:** From what I understand, the contributions of the authors are as follows:
- *Generalization of DT and RF to constant-curvature manifolds* : Chlenski et al seem to have already considered the problem of extending DT to work in hyperbolic spaces. The authors here generalise this approach to constant curvature manifold (not just negative curvature), which allows them to also work on hyperspheres. It also allows to generalize to Euclidean space:this is perhaps less interesting, but it goes to show that they devised a generalized method. The authors show rigorously in the appendix that their approach is equivalent to DT in basis space. The authors also provide explicit characterizations of the splits in different curved manifolds.
- *Generalization of DT and RF to product manifolds* :The authors borrow the idea of product manifolds from Gu et al, but this idea had not been explored in the existing RF and DT literature

3. **Strong experiments**: the authors seemed to have gone to great lengths to benchmark their approach, showing its performance on a variety of regression, classification, and even, link prediction tasks. They also perform ablation studies and test the effect of the different hyperparameters.
summary: Summary
This paper proposes to extend decision trees (DT) and random forests (RF) to mixed curvature spaces -- meaning that the data (here assumed to be in ambient space $\mathbb{R}^d$) is embedded on a curved manifold, and the random forest or DT works directly on this space. The authors argue indeed that curved manifolds are better suited to represent certain data types (e.g. hyperbolic spaces for taxonomies or phylogeny).  Product manifolds are even better at capturing the complexity of real-world data, and do not confine the analysis to choosing a single curvature, which could increase the potential of these methods in a number of real-life scenarios.

To this end, the paper is organized as follows:
- the authors start by generalizing DT and RF on manifolds with constant curvature. The authors start with a compact review of different Riemannian spaces (including hyperspherical spaces and hyperbolic spaces, which are the ones that have received the most attention), as well as the product manifolds. They then show that the  approach proposed by Chlenski et al for hyperbolic DT can actually be extended to any constant curvature manifold.
- the authors then show how to extend the approach to handle product manifold (i.e. they provide an algorithm).
- they then benchmark their approach on many different simulated and real-world datasets, showing the performance of the method.





*Disclaimer: I am not an expert in non-Euclidean geometry. I have followed the developments in hyperbolic Neural networks from a reasonable distance --- enough to know what they are and their advantages, but not enough to make me an expert on the topic or on related literature. I am happy to discuss with other reviewers and the authors any misconception that might transpire from this review.*
weaknesses: Overall, I find this to be a strong and very well-written paper.
A couple of criticisms perhaps:
- *Computational Time is not reported*: the authors do not report the compute time that their proposed method requires compared to, say, kNN or the ambient space methods. My guess would be that their method might be a bit more computationally expensive than these. That would be fine though, I think the method is interesting and, even it were considerably more expensive than the other methods, it would still be a worthy contribution that could spark follow up works. I would however encourage the authors to report the time, so that anyone interested in the idea has a sense of the current computational complexity of the method.
- *Motivation: why RF and DT?*: To come back to the motivation of the paper, why should we consider RF and DT and not just go with neural networks, for which there have been extensions and can perform better than RF and DT? I could see RF and DT being better alternatives in the low sample regime, or easier/ faster to train. I don't think the authors compare with any NN based method, if Im not mistaken?

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Please refer to Weaknesses.
soundness: 2
strengths: The proposed algorithms apply to constant-curvature manifolds, effectively capturing the mixed and complex structure of real-world datasets.
summary: This paper extends DT and RF algorithms to product manifolds, yielding splits that are geodesically convex, maximum-margin, and composable. Extensive experiments demonstrate the effectiveness of the proposed algorithms.
weaknesses: -	Novelty. The proposed algorithms appear to be a simple and direct extension of previous work [1,2], which integrates the notion of product space proposed by [1] with the HyperDT framework [2]. The method is somewhat incremental and lacks enough novelty.
-	Applicability. The framework's current restriction to constant-curvature manifolds presents a significant limitation. The applicability to more general scenarios, particularly stochastic-curvature manifolds, remains an open question that warrants investigation.
-	Comparison. The empirical validation would benefit from a comprehensive comparative analysis against contemporary approaches in the field. Notably absent are performance comparisons with state-of-the-art methods such as HORORF [3] and HyperDT [2].


[1] Tabaghi P, Pan C, Chien E, et al. Linear classifiers in product space forms.

[2] Chlenski P, Turok E, Moretti A, et al. Fast hyperboloid decision tree algorithms.

[3] Doorenbos L, Márquez-Neila P, Sznitman R, et al. Hyperbolic Random Forests.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Q1. The method’s ability to fit complex geometric patterns could lead to overfitting, for example on datasets with noise or less pronounced geometric structures. What potential regularization techniques could you use?

Q2. How does the method handle data distributions that do not perfectly conform to the constant-curvature assumption? Are there scenarios where mixed or varying curvature significantly degrades model performance?

Q3. Given the reliance on tangent plane approximations, how does the method account for distortions introduced in regions of high curvature? Can you quantify the impact of these distortions?

Q4. Are there edge cases where the proofs of geodesic convexity or margin maximization might not hold? If so, how does the method handle such situations?

Q5. The method involves complex geometric operations like exponential maps and parallel transport. How efficient are these operations in practice, in particular for large datasets? 

Q6. How sensitive is the method to hyperparameter choices, such as the variance scale in Gaussian sampling or the number of 2D projections considered for splitting?
soundness: 3
strengths: The paper presents a novel extension of traditional decision trees (DTs) and random forests (RFs) to mixed-curvature product manifolds. This is a substantial departure from prior work that primarily focuses on Euclidean spaces or, at best, single constant-curvature manifolds (hyperbolic or spherical). By integrating these different spaces, the method captures complex geometric patterns more effectively than existing techniques. The use of angular reformulations and geodesic convexity for decision boundaries is an original approach that leverages the intrinsic properties of non-Euclidean spaces. 

It presents a comprehensive framework that not only generalizes existing Euclidean and hyperbolic methods but also introduces new algorithms for spherical manifolds. This integration is both theoretically and practically novel, offering a unified approach to decision tree learning across mixed-curvature spaces.

The paper is grounded in mathematical principles and the theoretical derivations are detailed along with intuitive explanations.  The authors provide extensive empirical validation of their method on 57 datasets, including synthetic data, graph embeddings, and mixed-curvature variational autoencoder outputs. The benchmarks highlight the method's robustness and versatility across different tasks, such as classification, regression, and link prediction. The performance metrics and statistical significance tests give credibility to the claims made. It also compares the proposed approach against several baselines, including ambient space models, tangent plane methods, and k-nearest neighbors.

The method’s ability to handle data with hierarchical, cyclical, and mixed geometric structures makes it relevant for a wide range of domains.
summary: The paper introduces an extension of traditional decision tree (DT) and random forest (RF) algorithms, designed to work within product manifolds. These manifolds are constructed as Cartesian products of hyperbolic, spherical, and Euclidean spaces, making them suitable for efficiently capturing and analyzing datasets that exhibit diverse geometric properties.

The authors propose angular reformulations of DTs tailored to the specific geometry of product manifolds, ensuring that splits are geodesically convex, maximize margin separation, and can be effectively combined across different components of the manifold. This approach not only generalizes existing techniques for Euclidean and hyperbolic spaces but also introduces new algorithms for handling data in hyperspherical spaces. Through a series of benchmark evaluations, including classification, regression, and link prediction tasks, the proposed methods demonstrate strong performance, often surpassing existing approaches and achieving top rankings on various synthetic, graph-based, and real-world datasets.

Overall, this work provides a robust framework for utilizing decision trees and random forests in mixed-curvature spaces, enabling more effective analysis and interpretation of data with intricate geometric structures.
weaknesses: 1. One of the main limitations of the proposed method is the computational complexity involved in extending decision trees (DTs) and random forests (RFs) to mixed-curvature product manifolds. The algorithm requires handling several costly geometric operations, such as exponential maps, parallel transport, and angle computations. The paper could be improved by including a detailed complexity analysis and discussing possible optimizations or approximations that might reduce computational overhead.
In addition, given the need for complex manifold operations, the approach may struggle with datasets having millions of points or high-dimensional spaces. Including discussions about scalability and memory usage would strengthen the work.

2. The paper assumes that the underlying product manifolds have constant curvature components. This may not hold for many real-world datasets where curvature varies continuously or where the data distribution does not fit neatly into hyperbolic, spherical, or Euclidean categories. It could be helpful to discuss how deviations from this assumption impact performance and whether there are ways to relax or adapt this assumption.

3. The reliance on tangent plane approximations to map points from the manifold to Euclidean space could introduce inaccuracies, in particular in regions of high curvature. The paper does not quantify or analyze these potential inaccuracies. 

4. The mathematical derivations, notably those involving angle computations and midpoint selection, assume well-behaved data distributions. The method may not perform optimally in edge cases, such as when points are near the boundaries of hyperbolic space or when spherical data are tightly clustered around a pole. Discussing these limitations and potential solutions could help overall.

5. The paper compares its method primarily to other non-Euclidean tree-based models and baseline classifiers. However, deep learning models that can handle non-Euclidean data, such as graph neural networks (GNNs) with Riemannian embeddings, are not considered. A comparison to these models would provide a more comprehensive assessment of the proposed method’s strengths and weaknesses.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 4
questions: - I believe it would be helpful to clearly identify the technical differences between your proposed approach and previous research. As I reviewed your method, I struggled to find any significant technical novelty, as indicated in the Weaknesses section. If I have misunderstood something, please kindly let me know. 

- Is it possible to consider oblique decision trees? Specifically, can the method be extended to split not only within the same type of space but also across different spaces with oblique boundaries?

- Regarding the third contribution listed—“We extend techniques for sampling distributions in non-Euclidean manifolds to describe mixtures of Gaussians in product manifolds”—am I correct in understanding that this pertains to the creation of datasets, as described in Section 4.1 and Appendix A, rather than being directly related to learning decision trees on product manifolds? 

- In the Introduction, the intention to use decision trees and random forests is mentioned in the fourth paragraph. What is the motivation behind developing these tree-based models? If the motivation lies in accuracy, I believe a comparison with other non-tree-based models would be necessary. On the other hand, if the choice is driven by factors like interpretability, then it would be beneficial to provide experimental results or analysis supporting this aspect.
soundness: 3
strengths: The paper provides a thorough background knowledge, making it accessible even to readers unfamiliar with geometry and decision trees. Visualization quality is also good. Additionally, the experimental settings described in the Appendix are detailed, and the paper overall does not have unclear sections.
summary: This paper proposes a method for training decision trees that takes into account a product manifold structure, formed by combining manifolds with constant curvature. A notable technical contribution is the angular reformulation of tree splits. The effectiveness of the proposed method is demonstrated through experiments on both synthetic and real-world datasets.
weaknesses: While the paper makes a solid contribution, in my understanding, it falls short in terms of significance and novelty. For instance, Section 3, which serves as the core of the proposed method, raises some concerns. Section 3.1 discusses Euclidean decision trees, which is straightforward. Section 3.2 builds on existing research. Although Section 3.3 may introduce a new perspective, the authors themselves describe it as “quite simple”. The choice of does not seem to involve a particularly non-trivial approach. Furthermore, Section 3.4 primarily combines elements from Sections 3.1 through 3.3. While the paper mentions the advantages of redefining decision tree learning through angular comparisons, it is difficult to claim that these benefits are especially significant.

Additionally, although the experiments are well-executed, the impact of the proposed method could be better highlighted by including a more diverse set of baselines for comparison. For example, it would be beneficial to compare the method against non-tree models or conventional decision trees applied to real-world data, such as traffic data, without considering manifold structures. Even if the technical novelty is limited, demonstrating significant experimental results would enhance the paper’s relevance to the machine learning community. However, the current experimental results do not sufficiently showcase the advantages of the proposed method.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: Suggestions:

- It should be made clear in the text that product manifolds based on constant curvature manifolds cannot generate any arbitrary manifold, as part of limitations.

- Relevant works are missing: 

Historical context on manifold learning and machine learning: Algorithms for manifold learning by Cayton, Representation Learning: A Review and New Perspectives by Bengio

Hyperbolic spaces and machine learning: Before product manifolds, there were many works on using hyperbolic spaces which served as predecessor which are worth mentioning, for instance, Poincare embeddings for learning hierarchical representations, Neural embeddings of graphs in hyperbolic space, Hyperbolic entailment cones for learning hierarchical embeddings

Product manifolds and product manifold derived features: Heterogeneous manifolds for curvature-aware graph
 embedding, Neural Spacetimes for DAG Representation Learning

Computationally tractable manifolds beyong constant curvature manifolds: One of the motivations for using product manifolds of Euclidean, hyperbolic, and hyperspherical spaces, is that they allow machine learning algorithms to model data on more general manifolds than Euclidean space, while still providing a closed-form solution for geodesic distances, exponential maps, etc. However, product manifolds of this kind are also limited: it is not possible to model any arbitrary manifold by just using this construction. Other computationally tractable alternatives also exist: Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries, which is based on fractals, and Computationally tractable riemannian manifolds for graph embeddings, which is based on matrix manifolds.

- Curvature K is not defined in Section 2.1.2 for hyperspherical space.

- In equation 10 the wrong symbol is used for Cartesian product
- In equation 12 the wrong symbol is used for Concatenation
soundness: 2
strengths: - The paper generalizes decision trees and random forests to mixed-curvature representations which can lead to more expressive/flexible classifiers.
- They leverage ideas from previous papers in deep learning which use mixed-curvature, to improve random forests and decision trees.
- The proposed model may have applications in biology: which I believe is true and the authors mention in the introduction, but they do not really test it in the paper.
summary: The authors propose an extension of decision trees and random forests, which typically assume Euclidean space, to mixed-curvature representations using product manifolds based on constant curvature spaces: Euclidean space, hyperspherical space, and hyperbolic space (focusing on the hyperboloid model).
weaknesses: - It seems that the method suffers from similar issues to those discussed in: Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization. How does one select the product manifold signature? This should be discussed as part of the limitations. The paper mentioned above suggests using Bayesian optimization to select the signature, for instance.
- In general section 4 is poorly written, there are a lot of baselines, but it is unclear how DTs and RFs are used in different tasks. It is not clearly explained. Maybe additional explanations about the different tasks in the appendix would be helpful: that is, describing the problem setup rather than mainly focusing on reporting hyperaparameters.
- Table 2 has too many symbols. It is not clearly explained what is meant by Max Ambient and Max Product. Could you please define them in the main text or the caption.
- In table 2, for Cora and Citeseer if you are doing node classification, the performance is surprisingly low. What would be the results in terms of accuracy? In the case of Cora and Citeseer is more typical to see the results in accuracy than F1, so it would be helpful to know what they are to compare for instance to the performance of a GCN.
- In Section 4.4. the authors mention they use variational autoencoders for classification. Do you mean autoencoders? Variational autoencoders are generally used for generative modeling. Or do you mean you use the KL loss to regularize the latent space but then use it for classification. It is unclear how classification is performed: in the regularized latent space of an autoencoder? The performance seems very low too for MNIST, etc. KL regularization may be hurting classification performance, if you remove it the autoencoder has more freedom to encode the samples in the latent space without having to stay Gaussian-like and they are probably easier to classify. Could you provide a step-by-step description of the experimental procedure used please.
- The conclusion section lacks a discussion on limitations. As previously mentioned: it is not obvious how to choose the signature. Also, why use RF when there are other better performing methods for most of the tasks discussed in the experiment section? One could argue that RFs may be easier to implement for less experienced practitioners, but introducing the mixed-curvature representations makes the method more unaccessible to a general audience. Potential points to discuss: 1) Challenges in selecting appropriate signatures, 2) Trade-offs between RFs with mixed-curvature representations versus other high-performing methods, 3) Potential barriers to adoption for practitioners less familiar with non-Euclidean geometries

Meta-review: The authors propose to extend decision trees (DTs) and random forest (RF) into the product manifolds of hyperbolic, hyperspherical, or Euclidean components with a general angular split. The authors raised concerns about its novelty given results on product manifolds, and DTs/RF in the hyperbolic space. Although the authors evaluate the proposed methods on a large set of datasets, the Authors raised concerns on the sample size, baselines, time consumption. It is better to evaluate the proposed method on a systematic way to better convey the messages on advantages of the proposed method, and describe the experimental setting clearly. It is also good to specify how to determine the product manifold for a given dataset to apply the proposed approach. The Reviewers have mixed evaluations on the submission. Given aforementioned concerns, most of the Reviewers have not had enthusiastic supports on the submission yet. The authors may consider the Reviewers' comments to improve the submission.

Decision: Reject