Mixed-curvature decision trees and random forests
Authors: Philippe Chlenski
Quentin Chu
Raiyan R. Khan
Kaizhu Du
Antonio Khalil Moretti
Itsik Pe'er
Venue: ICML 2025 poster (2025)
Keywords: representation learning
non-euclidean geometry
decision trees
random forests
hyperbolic geometry
hyperspherical geometry
Primary area: general_machine_learning->representation_learning
TL;DR: Decision trees with geodesically convex splits extend to product space manifolds

Abstract: Decision trees (DTs) and their random forest (RF) extensions are workhorses of classification and regression in Euclidean spaces. However, algorithms for learning in non-Euclidean spaces are still limited. We extend DT and RF algorithms to product manifolds: Cartesian products of several hyperbolic, hyperspherical, or Euclidean components. Such manifolds handle heterogeneous curvature while still factorizing neatly into simpler components, making them compelling embedding spaces for complex datasets. Our novel angular reformulation respects manifold geometry while preserving the algorithmic properties that make decision trees effective. In the special cases of single-component manifolds, our method simplifies to its Euclidean or hyperbolic counterparts, or introduces hyperspherical DT algorithms, depending on the curvature. In benchmarks on a diverse suite of 57 classification, regression, and link prediction tasks, our product RFs ranked first on 29 tasks and came in the top 2 for 41. This highlights the value of product RFs as straightforward yet powerful new tools for data analysis in product manifolds. Code for our method is available at https://github.com/pchlenski/manify.

Mixed- Decision Trees and Random Forests
Philippe Chlenski! Quentin Chu Raiyan R. Khan' 1 Kaizhu Du
Antonio Khalil Moretti 23 Itsik Pe'er!
A Absrract
X, y sampled from P
Signature: P P= x S
Decision trees (DTs) and their random forest (RF) extensions are workhorses of classification and regression in Euclidean spaces. However, algo- rithms for learning in non-Euclidean spaces are still limited. We extend DT and RF algorithms product manifolds: Cartesian products of several hyperbolic, hyperspherical, or Euclidean compo- nents. Such manifolds handle heterogeneous cur- vature while still factorizing neatly into simpler components, making them compelling embedding spaces for complex datasets. Our novel angular reformulation respects manifold geometry while preserving the algorithmic properties that make decision trees effective. In the special cases of single-component manifolds, our method simpli- fies to its Euclidean or hyperbolic counterparts, or introduces hyperspherical DT algorithms, depend- ing on the curvature. In benchmarks on a diverse suite of 57 classification, regression, and link pre- diction tasks, our product RFs ranked first on 29 tasks and came in the top 2 for 41. This highlights the value of product RFs as straightforward yet powerful new tools for data analysis in product manifolds. Code for our method is available at wttp:ghhww...oomocccle///////eee
Split X
r FOO 7
01 0//4/4
Split X +
Split X
0₂ = 3T/ /4
0 = π
Decision regions on P
1. Introduction
Most machine learning algorithms assume Euclidean ge- ometry, but real datasets often have non- non-Euclidean struc- ture: tree-like hierarchies fit naturally in hyperbolic space (Sonthalia & Gilbert, 2020), while cyclical patterns suit spherical representations (Ding & Regev, 2021). Moreover, many real-world datasets don't conform to a single geo- metric structure. Any single constant-curvature geometry- hyperbolic, spherical, or struudea to capture all of their structural nuances simultaneously.
Figure 1: Given sample of labeled points (X, y) from torus P = S x S', we can factorize X into coordinates on each component manifold. Our DT splits these factorized coordinates, partitioning P into disjoint regions (colored positive or negative to reflect the classes).
Product manifolds (Gu et al., 2018), which combine mul- tiple constant-curvature components into a single product space, more expressive than single manifolds, facili- tating faithful representations of more kinds of underlying structure in the data. Although product manifolds have made inroads in biology (McNeela et al., 2024) and knowledge graph applications (Wang et al., 2021), general-puroose machine learning on product manifolds remains underex-
'Columbia University Barmard College 'Spelman College. Correspondence to: Philippe Chlenski <pac @ columbia.cdu>>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Eixed-Curvature Decision Trees and Random Forests
Continuity
Convexity
Equidistance
Our contributions:
1. We generalize DTs and RFs to all constant-curvature manifolds: By representing data and splits as angles in two-dimensional subspaces, we guarantee the five desiderata above. For single manifolds, this extends existing Euclidean and hyperbolic models or intro- duces hyperspherical DTs and RFs.
Figure 2: Decision tree splits are based on Euclidean dis- tances in the ambient space, and their assumptions some-
2. We introduce novel DT and RF algorithms for product manifolds.
times fail to hold for geodesic distances on non-Euclidean manifolds. For example, splits can create disconnected re- gions (Continuity), geodesics may cross boundaries within regions (Convexity), and distances to splits become unequal (Equidistance).
3. We extend techniques for sampling mixtures of Gaus- sians to product manifolds.
4. We show how problems like link prediction in graphs and signal analysis can be recast as inference problems on product manifolds.
5. We demonstrate the effectiveness of our algorithms on diverse suite of 57 benchmarks.
plored. In particular, decision trees (DTs) and random forests (RFs) some of the most successful and widely- used algorithms in machine learning-aa still lack product space variant.
1.1. Related work
Eunlddell representation learning. Important back- ground on manifolds in machine learning is given in Cayton (2005) and Bengio et al. (2014). Much of the work on product manifolds is indebted to early works on hyperbolic spaces, including Nickel & Kiela (2017); Chamberlain et al. (2017), and Ganea et al. (2018).
A natural approach would be to simply apply standard DTs/RFs to the coordinate representation of points embed- ded in the product manifold. However, this naive strategy ig- nores the underlying geometry: splits that appear reasonable in coordinate space can violate fundamental geometric prop- erties when interpreted on the manifold itself. We conjecture that the success of DTss stems from five key desiderata that standard Euclidean splits naturally preserve:
Machine learning in product manifolds. Tabaghi al. (2021) describe linear classifiers, including perceptron and support vector machines; Tabaghi et al. (2024) adapt prin- cipal component analysis; and Cho et al. (2023) generalize transformer architectures to product manifolds.
1. Continuity: Leaves partition space into connected regions;
Product manifold-derived features. Sun et al. (2021) and Borde et al. (2023b) use product manifolds to compute rich similarity measures as features for classification. Giovannni et al. (2022) introduce heterogeneous variant of product manifolds; Borde et al. (2024) combine quasi-metrics and partial orders for graph representation.
2. Convexity:
Leaves correspond geodesically convex subsets of the input space;
3. Equidistance: Splits maintain equal distance to near- est points on either side;
4. Efficiency: Consider (ndd) candidates per split; and
5. Speed: Evaluate splits in (1) time.
Manifold random forests. Our method is inspired by re- cent work by Doorenbos et al. (2023) and Chlenski et al. (2024) extending RFs to hyperbolic space. Other work has explored generalizations of random forests man- ifolds, including random forest regression for manifold- valued targets (Tsagkrasoulis & Montana, 2017), manifold oblique random forests (Li et al., 2022), and Fréchet random forests (Capitaine et al., 2024). Chlenski & Pe' er (2025) explores faster ways to train hyperbolic random forests.
Figure 2 illustrates how these desiderata fail on curved mani- folds: standard linear splits through the ambient space leave room for topologically discontinuous leaves, geodesics that cross decision boundaries, and unequal distances to the deci- sion boundary. Such issues could lead to misgeneralization, undermining the geometric intuition that makes decision trees interpretable and effective.
We present a unified framework that preserves all five desiderata across arbitrary product manifolds. By repre- senting splits as angles in two-dimensional subspaces, our approach naturally handles hierarchical data (hyperbolic components), cyclical patterns (spherical components), and traditional features (Euclidean components) within a single, principled framework.
Applications of product manifolds. Product manifolds are used to embed knowledge graphs (Wang et al., 2021; Van
Nguyen- Van al., 2023; Li al., 2024). In biology, they have been used to represent pathway graphs (McNeela et al., 2024), cryo-EM images (Zhang et al., 2021). and single-cell transcriptomic profiles (Tabaghi et al., 2021). Skopek et al. (2020) also embed image datasets into product manifolds.
2

---

Mixed-Cure Mixed-Curvature Decision Trees and Random Forests
Hyperspherical
Euclidean 37g
Hyperbolic
20
0
3d
3d
Figure 3: Decision boundaries in any constant-curvature manifold can be found by projection into 2-dimensional subspaces. Depending on the manifold, points end up inside circle, a line, or hyperbola; from there, data and splits can both be represented as single angle 6, Each split divides the manifold into positive and negative regions.
2. Preliminaries
2.1.2. HYPERSPHERICAL SPACE
We review relevant details of different Riemannian mani- folds (Euclidean spaces, hyperspheres, hyperboloids, and product manifolds), along with key properties of the Eu- clidean and hyperbolic variants of DTs and RFs.
Hyperspherical space is characterized by constant positive curvature. In such spaces, angles in any triangle sum to more than π and there are no parallel lines. The familiar examples of circles and 2-spheres are low-dimensional examples of hyperspherical spaces.
2.1. Riemannian manifolds
Hyperspheres can viewed as surfaces embedded a higher-dimensional, Euclidean ambient space. Hyperspheri- cal space uses the same inner products as Euclidean space. The hypersphere is the set of points in the ambient space having Euclidean norm equal to some radius inversely proportional to the curvature K > 0:
We will begin by reviewing key details of hyperspheres, hy- perboloids, and Euclidean spaces. For more details, readers can consult Do Carmo (1992).
Each space described a Riemannian manifold, meaning that it is locally isomorphic to Euclidean space and equipped with a distance metric. The shortest paths between two points and v on manifold are called geodesics. As all three spaces consider have constant Gaussian curvature, we define simple closed forms for geodesic distances in each of the following subsections in lieu of more general discussion of geodesics in arbitrary Riemannian manifolds.
SD. = E |>>|| = 1/KK.
(5)
Because shortest paths between two points u and v in SD,K through the ambient space leave the surface of the manifold, we must define the hyperspherical distance function for the shortest path entirely in S,, between and
Any constant-curvaturee manifold M is parameterized by dimensionality D and curvature K, and resides in an ambient space RD+1, Finally, for each point M, the tangent plane TxM, is the space of all tangent vectors at x:
8g (u.v) v) cos [K (K, v))/K.
(6)
1.3.3. HYPERBOLIC SPACE
Hyperbolic space is characterized by constant negative met- ric curvature. This has several consequences: for instance, the angles in any triangle sum to less than æ, many lines through a point can be parallel to any given line, and neigh- borhoods grow exponentially with radius.
TxM= (v RRRR D+1 (v,x)M= = 0}.
(1)
2. 1. EUCLIDEAN SPACE
Euclidean spaces are naturally understood as R,,,, but we will use the notation D RD when treating Euclidean spaces manifolds. In contrast, we will continue use D refer to ambient spaces. Fuclidean spaces use the familiar inner product (dot product), norm (lg norm), and distance function (Euclidean distance):
There are several equivalent models of hyperbolic space. For our purposes, will describe the hyperbolic space from the perspective of the hyperboloid model. First, we must define the ambient Minkowski space. This is a vector space equipped with the Minkowski inner product:
uu v) = ug ui + W2
(2)
(u, v) = - to + U1 +... + Un Un-
(7)
||u|| = / (u, u)
(3) Similar to the Euclidean case, we let ||u||c = (u, u) (we
(u, v) = -lu-v vll.
(4) not wish take the square root of negative number).
3

---

Mixed- Mixed-Curvature Decision Trees and Random Forests
The hyperboloid of dimension D and curvature K 0,
Here, C(-) is some impurity function (we Gini impurity for classification and variance for regression). A splitring function S(.) partitions the labels into yt and y and the input space into decision regions. Classically, S(.) is thresholding function which partitions the input space into high-dimensional boxes given dimension d and threshold 0:
written HD.KK, is a set of points with constant Minkowski norm:
0}. (8) H.K fXE |x||c = -1/K2, zu
Finally, the hyperbolic distance function for geodesic dis- tances between u, v E H. K is given by
S(x) 1(x > 0}.
(14)
og v) = cosh foohh-t(K - (K² ()2)KK
(9)
This algorithm is applied recursively to each decision region until stopping condition is met (e.g., maximum number of splits is reached). The result is a fitted DT, T, which can be used for inference. During inference, an unseen point is passed through T until it reaches a leaf node corresponding to some decision region. For classification, the point is then assigned the majority label inside that region; for regression, it assigned the mean value inside that region.
2. 1.4. MIXED-CURVATURE PRODUCT MANIFOLDS
We reiterate the definition of product manifolds from Gu et al. (2018). Following the convention of using II; X; to refer to the iterated Cartesian product over sets, the product manifold is defined as
11 II ==1
P
I t=1
H x Ed.
(10)
Finally, RF is an ensemble of DTs,, typically trained on bootstrapped subsample of the points and features in X (Breiman, 2001).
The total number of dimensions is Σr 8; E, hy +d. Each individual manifold is called a component manifold, and the decomposition of the product manifold into component manifolds is called the signature. Informally, the signature can be considered a list of dimensionalities and curvatures for each component manifold.
2.2.1. HYPERBOLIC DECISION TREE ALGORITTMM
The hyperplane perspective on DTs is helpful background for understanding our method: mathematically. threshold- ing x on a dimension equivalent to taking its dot prod- uet with the normal vector of separating hyperplane I,, even in hyperbolic space. Although this is easy to compute for classical thresholding boundaries, which are zero in all dimensions but d, this perspective principally admits any hyperplane P as a valid decision boundary.
Distances in P decompose the l norm of the distances in each of the component manifolds:
(p(,, v) = Σ
(u(um.vM))
(11)
MEP
Naturally, considerations around choosing an appropriate (and computationally efficient) IP abound. To this end, Chlenski et al. (2024) impose homogeneity and sparsity constraints on the hyperplanes they consider for hyperbolic DTs.. In hyperbolic space, homogenous hyperplanes hy- perplanes that contain the origin of the ambient space- intersect D. at geodesic submanifolds: that is, P. is closed under shortest paths according to ōg. The sparsity constraint enforces that the normal vectors of P must be nonzero only in two positions: the timelike coordinate on and some other za, which ensures that only O(nd) candi- date hyperplanes are considered per split, and each decision can be computed in O(1) time using sparse dot products.
where iM and V.M denotes the restriction of u and to their components in M and δM refers the distance function appropriate to M.
For x E P, the tangent plane at x, TxP, is the direct sum (concatenation) of all component tangent planes:
TxP
+ Txm MEP
(12)
We additionally define the origin P, Mo. as the concatena- tion of the origins of each respective manifold. The origin is (1/1K),0,...) for H.P.K and SD.K and (0,0, O.... for E.D.
3. Mixed-curvature decision trees
2.2. Decision trees and random forests
For any DT, we must transform the input X into set of candidate hyperplanes. To this end, we reframe and general- ize the hyperplane approach of hyperbolic DTs. First, we observe that homogenous hyperplanes are geodesically con- vex in any extend the hyperbolic DT approach to E and S. Second, we observe that fitting sparse, homogenous DTss is equivalent
The Classification and Regression Trees (CART) (Breiman, 2017) algorithm fits a DT T to set of labeled data (X,y). y). At each step, greedily selects the split which partitions the dataset with maximum information gain,
manifold; therefore, we can
y
y 1y
IG(y) = C(y)-
C(y+)
C(y
(13)
yyl

---

Mixed-Curvature Decision Trees and Random Forests
Algorithm Product Space Decision Tree
cally equidistant from the points either side of it (Euclidean DT's do this by sample-averaging the threshold values). Angular midpoints for each component manifold are described in the following sections and summarized in Table in the Appendix.
1: Procedure FIT(P, X. y):
2: for each component M of P do
3: for each dimension d > 0 of MM do
4: Oz,d f tan (0.,0//144 {Eq. 15]
5: end for
With the angular features and manifold-informed midpoint modifications in place, the rest of the algorithm follows Section 2.2 unmodified.
6: end for
7: return FI
8:
9: Procedure FITNODE depth):
3.1. Euclidean decision frees
10: if depth max depth or other stopping criteria then
11: return Leaf(y)
To unify our treatment of all three geometries, we first re- formulate the standard Euclidean DT in geometrically- informed, albeit unconventional, way that is equivalent to the classical model.
12: end if
13: (d",0", (-.
for each dimension of e do
15: for each GEECCANDIDATESS d) do
While the intersections of homogenous hyperplanes in R with E are (trivially) convex, these lack the expressiveness of an ambient-space formulation. Thus, we embed EE in RD+1 by trivial lift:
16:
Partition yY (0# , via Eq. 16
17:
IG + Eq. 13 on yy)
18:
IG IG then
19: (d", IG") + (d, IG)
20: end if
Φ : IE - IRD+, p(u) = (1, u).
(17)
end for
For two points u, V E EP, the midpoint angles in E can be described in terms of the coordinates of u and v or their respective projection angles (fu, 0x) as
end for
then 24: return Leaf(y)
23: if IG
25: else
mg [u, v) tan (2/(ua va)) tan (2/(cot(On) (ot)))
(18)
26:
N' + - Node(d",
27: Mleeft + FITNODE .y 1dephh+1)
(19)
riigght <- FITNODE O + y 1)
See Appendix B for a proof that this formulation is equiva- lent to thresholding on basis dimensions.
return.N 30: end if
31:
32: Procedure GETCANDIDATES(O.
3.2. Hyperbolic decision trees
33: 0 < sorted unique O..!
For two points u, v E H.K we compute On and 6, accord- ing to Eq 15 and follow Chlenski et al. (2024) in computing the hyperbolic midpoint angle in HP as:
34: return [m,(0,, 0411) (Eqs. 18-22}
to thresholding on angles under 2-dimensional projections.
sin (2 28x)
V sin + 0v) sin 0n)
(20)
We consider the of all projections which are computable in (11) time per projection by coor- dinate selection. First, we compute the projection angle:!
the basis { ra),
co vV=-1)
if O, + 0v < otherwise.
my (u, v)
cot co VV2-1)
(21)
0(x, d) = tan (zo/
(15)
Next, modified splitting criterion to account for the geometry of angular splits:
3.3. Hyperspherical decision trees
The hyperspherical case is quite simple, except that unlike hyperbolic space and the "lifted" Euclidean space after ap- plying Eq 17, we lack natural choice of Top We adopt the convention of fixing the first dimension of the embedding space as Zo. which intuitively corresponds to fixing a "north pole" at the origin
S(x, d, d) = T(O(x, d) E (0,0+))).
(16)
Once the best angle is selected, we must compute angular midpoints to select IP that intersects M at point geodesi-
(1/1K].0....)
We use the Py Torch arstan2 function to ensure that can recover the full range of angles in 2T). This is essential for properly specifying decision boundaries S.
Angular midpoints are particularly spherical manifolds: given u, E SD.K the hyperspherical
hyper-
5

---

Mixed-Curvature Decision Trees and Random Forests
midpoint angle is computed by finding Ou and Oy using Eq 15 and taking their mean:
Tp'.a' crosses into M implying 7p'.q' I S, contradict- ing initial assumption that S is geodesically convex. Tabaghi et al. (2021) shows that a linear classifier (that is, classifier inducing geodesically convex decision boundary) with weights w and bias b takes the form
my (u, v) (9u -8.
(22)
3.4. Product decision tree algorithm
Intuitively, the transition from DTss in single component manifold to a product manifold is that we now iterate over all preprocessed angles together, using the angular midpoint formula appropriate to each component. The complete pseu- docode for this algorithm is given in Algorithm
[ sign((Wg, WE) + ag sin '(wg. xg))
+ Ohp sinh (( Xx) c) + b);
(24)
where VM means the restriction of some P to compo- nent manifold .M and ar is weight term.
Letting a single DT span all components- opposed to ensemble of DTs, each operating in single component- allows the model to independently allocate its splits across components according to their relevance to the task at hand. Recasting DT learning in terms of angular comparisons has three major advantages over finding planar decision boundaries directly:
Under our angular reformulation, this simplifies to
P w = sign(ro cos(O) - zt sin(
(25)
where is our splitting angle and d is the dimension along which the split happens; because our split is confined to single manifold, the weights a,M do not affect the split. By restricting our attention to two dimensions within a single manifold, our formulation bypasses almost all of the com- plexity of evaluating geodesic splits in product manifolds.
1. We can consider angles under arbitrary linear pro- jections (not just projections onto basis dimensions) while maintaining (1) decision complexity.
2. It becomes possible to subsample the features (pre- computed angles) as is typical RFs.
This simplified form directly corresponds to our angular splitting criterion from Equation 16. Since this is an instance of the geodesically convex linear classifiers, our method is guaranteed to produce convex splits.
3. Product manifolds can always represent additional features in a new Euclidean manifold.
3.5. Geodesic convexity
Having detailed our algorithm, we now demonstrate that its splits satisfy the geodesic convexity property. Establishing this is essential to stated goals: geodesic convexity implies topological continuity. The remaining desiderata (3- 5) follow straightforwardly from our angular preprocessing and midpoint splitting strategy.
4. Benchmarks
We carried out benchmarks to evaluate which model, given labeled set of mixed-curvature embeddings, achieves the lowest error on a test set. While we produced embeddings using range of datasets and embedding techniques, results focus only on performance on downstream tasks. We describe our data generation/embedding methods in more detail in Appendix Section D.
subset of a manifold S M is said to be geodesically convex if p, q E S implies that the geodesic Pp.q C S. That is, for any two points in S, all points in the shortest path between them stay in S. Lack of geodesic convexity is potential source of misgeneralization for models, and several classifiers in hyperbolic spaces explicitly seek to partition the feature space in a gcodesically-conee manner (Cho et al., 2018; Chlenski et al., 2024).
We summarize our benchmark results, with references to specific figures and tables, in Table 1.
Table 1:
Benchmarks summary. The "Task" column is C (Classification), R (Regression), LP (Link prediction). "#Top-k" columns count how often product DT's or RFs were among the top k predictors for given set of benchmarks.
Building on Udrişte (1994), Chapter 3. we present a proof sketch that if c M is geodesically convex and partitions M into disjoint regions M and M then these regions are geodesically convex.
Manifold
Task
Ref
#Top-1
#Top-2
Total
Single
C
Fig. 4
11
11
By way of contradiction, suppose there exist p. E M+ such that their geodesic "pp.q crosses into = Since S separates M+ and M .p.q must follow this path:
Single
R
Fig. 5
7
9
11
Product
C
Tab. 2
6
10
18
Product
R
Tab. 3
8
10
11
Product
LP
Tab. 4
1
6
M+ + S M S M
(23)
Total
29
41
57
However, this implies the existence of p',q' E such that
6

---

Hixed-Curvature Decision Trees and Random Forests
Accuracy (1%)
Model
40
Product RF
Ambient RF
20
Tangent RF
N-eiighoor
MLP
-4
-2
-1
-0.5
-0.25
0.25
0.5
2
4
K-GCN
Curvature
Figure 4: Classification accuracies mixtures of Gaussians in single manifolds of curvature K. We omit results for all models that never achieved competitive results.
0.25
RMSE
Model
Product
0.20
Ambient RF
Tangent RF
0.15
k-Neighbors
-4
-2
-1
-0.5
-0.25
0.25
0.5
1
2
4
Curvature
Figure 5: Regression benchmarks (RMSE) on mixtures of 8 Gaussians in single manifolds of curvature K. We follow the convention of Figure 4 in omitting non-competitive models.
4.1. Experiment details
ing embeddings in each signature and selecting the signature with the lowest metric distortion. For link prediction, we embed all datasets in P (S²E²H?), then create bi- nary classification dataset by associating each pair of nodes with a point in p²E where each pair of points included and the last Euclidean dimension the manifold distance 8p (x), xj); labels are simply whether there an edge be- tween nodes and j. Full details graph embeddings described in Appendix Section D.4.
Given dataset X, set of labels y, and product manifold P. evaluate a variety of classifiers their ability to pre- diet y from X. We apply an identical 80:20 train-test split to all of our data, train our models on the training set, and evaluate performance on the test set. We report 95% confi- dence intervals for accuracy scores for classification, root mean squared error (RMSE) for regression, and accuracy scores again for link prediction benchmarks. Confidence intervals are based on 10 runs with different random seeds.
Mixed- cervatuue VAE latent space. We follow Skopek et al. (2020) in training variational autoencoders (VAEs) whose latent space is P. Once the VAE is trained, we use its encoder to generate embeddings for our dataset and classify these embeddings. Full details on VAE training and down- stream inference are described in Appendix Section D.5.
4.2. Datasets
Synthetic data. We develop novel method sample mixtures of Gaussians in P to generate classification and regression datasets. For classification, we generate 8 classes using 32 clusters. For regression, we generate a single scalar response variable using 32 clusters with randomly generated intercepts. For more details, see Appendix Section A.
Empirical datasets. Some datasets can be represented in non-Euclidean geometry without generating embeddings: for instance, geospatial data lives in S?, while cyclic time series embed in $2, describe our approach to generating embeddings for these empirical datasets in Appendix D.6.
Graph embeddings. For classification and regres- sion graph datasets, generate embeddings that approximate shortest-path distances in the graph us- ing the method described in Gu et al. (2018). We select the optimal signature from the candidate set ( H²E? H23?, H² (S?)2, H4, Et, by generat-
4.3. Baselines
We train two variants of Seikit- Mikktt-Lean Learn (Pedregosa et al., 2011) DT's and RFs: The ambient variant operates directly on coor-

---

Mixed- Sixed-Curvature Decision Trees and Random Forests
Table 2: Accuracies for all product manifold classification benchmarks. The highest scores for each dataset are shown in bold, while second- best predictors are underlined. For brevity, we omit columns for three oww-perfrnnin methods: product space perceptrons, ambient-space GNNs, and product space ML..
Dataset
Signature
Product RF
Ambient RF
Tangent RF
k-Neighbors
Ambient MLL
x-GCN
Synthetic (multi-K K)
Gaussian
E*
34.4 3.0
34.2 2.9
34.2 2.9
34.1 2.8
3.1
26.5 1
II
69.3 3.3
53.1 =
63.8
67.3
4.3
2.3
H?g²
2.8
4.0
4.3
44.7 2.9
2.79
2.6
(H
3.0
2.4
2.8
5.2
33.7
2.4
H²
41.8 + 2.5
2.6
2.7
38.0
2..
16.4 4.44
84
2.8
2.4
3.2
2.6
1.94
21.1
S
3.0
33,4
31.7
2.56
23.7
1..0
(2
2.5
3..6
2.3
28.1
2..
1.20
CiteSeer
(2
1.2
26.7
1.6
1.7
1.3
2..9
Graph
Cora
IH
2.14
29.1
2.71
2.7
2.9
29.6
PolBlogs
(
0.9
9..0
93.3
9.2
9.92
69.9 10.1
Blood
S (HHF)*
14.5 1.4
14
1..7
1.79
1.9
1.94
VAE
CIFAR-100
(H²))
0.9
1.2
1.1
8.9
7.6
5.7
Lymphoma
(
8..4
8.14
8.22
7.5
7..0
66.9 13.9
MNIST
S"E"H
5.3
5.6
3.5
34.8
6.89
1.90
Landmasscs
1.1
1.4
8..2
91.5
2.3
73.2
Other
Neuron 33
(8 (
64,2
6..
6.05
50.7 + 1.9
47.
47.3 + 2.0
Neuron 46
(S)
24.1 3.7
3.5
22.0 5.7
30.4 4.7
3.1 1.0
79.1
Table Regression performance (RMSE) for all product manifold classification benchmarks. Following the conventions of Table 2, emphasize high-scoring predictors and omit columns for predictors that achieve the highest score. CS PhDs is graph embedding dataset, whereas Temperature and Traffic empirical.
Dataset
Signature
Product
Ambient
Tangent
k-Neighbors
Synthetic (multi- K)
Gaussian
E4
0.023 0.004
0.023 # 0.003
0.023 # 0.003
0.023 0.004
H
0.015 0.003
0.020 0.003
0.017 = 0.003
0.016 0.003
0.023 0.004
0.024 0.004
0.024 0.004
0.023 0.004
(
0.021 0.004
0.024 0.003
0.022 # 0.003
0,021 0.003
H"
0.025 0.004
0.026
0.026 0.004
0,027 0.004
g*
0.023 0.003
0.023 # 0.003
0.024 = 0.004
0,024 2 0.004
S?T²
0.027 0.004
0.027 0.004
0.027 # 0.004
0.028 0.005
(3
0.028 0.004
0.028 # 0.005
0.029 + 0.005
0.031 + 0.005
CS PhDs
II"
205. 842 + 34.454
208.723 36.638
207.872 35.043
174.759 + 25,074
Other
Temperature
giga
77.942 23.689
48.221 * 23.516
99.561 # 24.984
138 650 + 25.
Traffic
E(S')1
0.307 0.039
0.220 0.017
0.229 + 0.021
0.255 0.021
Table 4: Accuracies for all link prediction benchmarks, also following the conventions in Table 2. The &cGCN is trained using a conventional link prediction approach; the other classifiers are trained as binary classifiers on the product of input embeddings X²_ All graphs were embedded into 22
Dataset
Product RF
Ambient RF
Tangent RF
k-Neighbors
Ambient MLP
K-GCN
AdjNoun
94.3 + 0.0
94.5 0.0
94.3 + 0.0
94.5 + 0.0
9.0
94.3 0.0
Dolphins
94.1 0.0
93.5
94.1
9.0
9.01
92.9 0.0
Football
0.0
0.0
0.0
7.0
9.0
95.9
Karate Club
9.0
9..8
0.0
7.66
95.9
95.9 0.0
PolBooks 9.0
9..6
91.8
89.8 0.0
9..4
9..4
8

---

Decision Trees and Random Forests
Product RF (Ace: 90.0%)
Ambient RF (Acc: 85.0%)
Tangent RF (Acc: 81.0%)
k-Neighbors (Acc: 89.5%)
Ambient MLP (Ace: 67.5%)
x-GCN (Acc: 65.0%)
Figure 6: We color a world map with each model's predicted P(Land) for the "Landmasses" dataset, land vs. water classification benchmark in S², Each RF consists of 12 DTs with max depth of 5. Note the artifacts learned by Euclidean RFs, tangent RFs, and k-Neighhors, well as the diffuse probabilities learned by the ambient MLP and x-GCN.
dinates in the ambient space D-1, while the tangent plane variant preprocesses points by projecting from P to Too P via the logarithmic map at μoo We use k-nearest neighbor (k-NN) classifiers and regressors with precomputed pair- wise distance matrices according to 6p (Eq. 11). We also use the product space perceptron algorithm (Tabaghi al., 2021) and multilayer perceptron (MLP) and x-GCN model (Bachmann et al., 2020), implemented in the Manify library (Chlenski et al., 2025).
5. Conclusion
We present strong evidence supporting the use of mixed- curvature DT's and RFs. In particular, motivate and de- scribe our entire algorithm and demonstrate its effectiveness across highly diverse set of 57 benchmarks.
Product DTs and RFs offer a valuable balance of expres- siveness and simplicity, positioned between extremely leg- ible but underpowered linear classifiers and powerful but uninterpretable neural networks operating in product mani- folds. We believe that these qualities, combined with their demonstrated performance across our benchmark datasets, are compelling evidence of our method's 's usefulness in a non- aun-Eucidean data analysis toolkit.
For our own models, we set hyperparameters identically to Scikit-Learn DTs and RFs, except we consider all (2) projections- for a total of 3 features per 2-dimensional component manifold, just like ambient space methods use. Full details for each model can be found in Appendix D.2.
Limitations. While our work is downstream of signature selection and embedding generation, it relies heavily on the availability of good product manifold embeddings. Prod- uet manifolds face challenges when selecting appropriate signatures (Borde et al., 2023a), and representing certain complex patterns in data (Borde & Kratsios, 2023). Fur- thermore, generating embeddings can be computationally intensive. Finally, the lack of privileged basis (Elhage et al., 2023) in -uclluelidean embeddings makes the induc- tive bias of decision trees less well-motivated compared to the classical tabular setting.
4.4. Results
For single-curvature synthetic datasets, method the best classifier in 7 out of 11 signatures (Figure 4) and the best regressor (Figure 5) for 7 out of 11 signatures. In Tables 2, 3, and 4, we demonstrate consistently good performance across a diverse range of benchmarks: We consistently outcompete baseline models in classification and regression, and perform on par for link prediction tasks. All in all, our method was the best for 29 out of 57 bench- marks (51 %), and was in the top-2 best for 41 out of 57 benchmarks (72%).
Future work. It may be possible to exploit non non- non-privileged basis dimensions using approaches such as rotation forests (Bagnall et al., 2020), random 2-D subspace angles, or oblique decision trees. continuous unification of all three geometries, like the x-stereographic model described in (Skopek al., 2020), may be more robust and elegant. Ex- tensions simplex geometry (Mishra al., 2020) are also worth considering
Further experiments can be found in the Appendix: we provide ablations in Appendix E, detailed tables and latent space visualizations in Appendix F, runtime and computa- tional complexity analysis in Appendix G, interpretability experiments in Appendix H, and benchmarks on more man- ifolds and baselines in Appendix I.

---

Mixed-Curvatee Decision Trees and Random Forests
Acknowledgements
Geometries, October 2023. URL http:////aarxi g/ als 2310.1500. arXiv:2310.1500.
This work was funded by NSF GREP grant DGE-2036197.
Borde, H. S. d. O., Arroo, A., Morales, I., Posner, I., and Dong, X. Neural Latent Geometry Search: Product Mani-
Impact statement
fold Inference via
Bayesian Optimization, October 2023a. URL http:/// /arxivv. org/abs /2399, 94810. 2ati::33000048100
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Borde, H. S. O., Kazi, A., Barbero, F., and Liø, P. Latent Graph Inference using Product Manifolds, June 2023b. 99.
URL http: / rxiv org/ /bbg//b///// 2211 arXiv:2211.166 [cs].
References
10x Genomics. Hodgkin's Lymphoma, Dissociated Tumor: Targeted, Immunology Panel, 2020a. URL https:// www. www.eoxics. http://cccccccaaaaattst//ooo.... s
Borde, H. S. d. O., Kratsios, A., Law, M. T., Dong, X., and Bronstein, M. Neural Spacetimes for DAG Representa- tion Learning, August 2024. URL http: arxiv.oo. g/ 3/abs/B 2408 13885. arXiv:2408. 13885.
d- ie-immmooogyy-paaee 3-1-standarr----- -0.
Breiman, Random forests. Machine Learning, 45(1): 5-32, October 2001. ISSN 1573-0565. doi: 10.1023/A: 1010933404324, URL https: /tttpps:/ddoo //tp://do.i// http:///dooo.ooo///..e. 3/A:101093340433
10x Genomics. PBMCs from a Healthy Donor: Targeted-
Compare, Immunology Panel, 2020b. URL https//
com http:///cccocceaaaatttse///e/eea www www.::xxgemmiiccccomoooco rom a hww.--hhheattt---ooo-ttteeeeeeee i immunology-pamell pttppppaane------aaaaaar-r------
Breiman, L.
Classification and Regression Trees. Routledge, New York, October 2017. ISBN 978-1-315-13947-0 doi: 10. 10.1201/97813151334770
Adamic, L. A. and Glance, N.
The political blogosphere and the 2004 U.S. election: divided they blog. In Proceed- ings of the 3rd international workshop Link discovery, LinkKDD '05, pp. 36-43, 20-43,Nec New York, NY, USA, August 2005. Association for Computing Machinery. ISBN 978- 1-59593-215-0, doi: 10.1145/11 https: https:://oi.crgg: org. http://...1115511111 277.
Capitaine, L., Bigot, J., Thicbaut, R., and Genuer, R. Fréchet random forests for metric space valued regression with euclidean predictors, February 2024. URL http: arxiv wwgg/abs............... /1..//996.000 01741. arXiv:1906.0177 [stat].
URL
Cayton, L. Algorithms for manifold learning. 2005.
Chamberlain, B. P., Clough, J., and Deisenroth, M. P. Neural
Embeddings of Graphs in Hyperbolic Space, May 2017.
Bachmann, G., Bécigneul, G., and Ganea, O.-E. Constant Curvature Graph Convolutional Networks, May 2020. URL http:////// /arxivivvv.. org/ b 1911. 05076. 1XXiv1111.00507 [cs].
URL http: warxiv.oxxp abs /1705. arXiv:1705.10355
Chami, I,, Ying. R., Ré, C., and Leskovec, J. Hyperbolic Graph Convolutional Neural Networks, October 2019. URL ht : ////axxivv.//////////////////avv org/ aabs/////// 19100. 12933. arXiv:1910.12933 [cs, stat].
Bagnall, A., Flynn, M., Large, J., Line, J., Bostrom, A., and Cawley, G. Is rotation forest the best classifier for problems with continuous features?, April 2020. URL http::/t////c /awp/vaxvvvv/vv////////////aaavv org/abs 1809.880000 06705. arXiv:1809.0670s [cs].
Chlenski, P. and Pe'er, I. Even Faster Hyperbolic Random
Forests: A Beltrami-Kled Wrapper Approach, June 2025. URL http: arxiv. org/ abs/ 25006.506... 0 04360. 04X4:2506004360 [es].
Becigneul, G. and Ganea, O.-E. Riemannian Adaptive Op- timization Methods. September 2018. URL https:
//openreview hww.e/wwweeeeeeeeee www.:/nrtffooaaa..e d=rle eiggii9..
Chlenski, P., Turok, E., Moretti, A., and Pe' er, I. Fast hy-
perboloid decision tree algorithms, March 2024. URL ht / /as//arxxx arxiv.i...... g a s 3 138 41. arXiv:2310.1382 [cs].
Bengio, Y., Courville, A., and Vincent, P. Representation Learning: Review and New Perspectives, April 2014. URL http:// /t///viivv..//////////////////av www.ii.orrg//oo wwg. abs/ 6206..266.. 5538. arXiv: 206.5538.
Chlenski, P., Du, K., Satow, D., and Pe'er 1 Manify: A Python Library for Learning Non-Euclidean Representa- tions, March 2025. URL hitp: arxii. org abs/ 2503 09576 6. arXiv:2503.09576 [cs].
Borde, H. S. d. O. and Kratsios, A. Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent
10

---

Mixed-Curvature Decision Trees and Random Forests
Cho, H., DeMeo, B., Peng, J., and Berger, B. Large- Margin Classification in Hyperbolic Space, June 2018. URL http: //tp//aaivvvv//vv///v//eaeeaaavl org porg// abs abs/1806. 00437. arXiv:1806.00433 [es, stat].
America, 99(12):7211-726, June 2002. ISSN 0027- 8424. doi: 10. 10.1073/pmas.122653337 URL https:
/ nww.. www..nmmm.i.. nim. htt../iih.w..p//////e/e..ee gwwppmc//rticcles.sssseee PMC122977/.
Cho. S., Cho, S., Park, S., Lee, H., Lee, H., and Lee, Curve Your Attention: CurvdtCuvvtture Transform- for Graph Representation Learning, September 2023.
Gu, A., Sala, F., Gunel, B., and Ré, C. Learning Mixed- Curvature Representations in Product Spaces. September 2018. URL httpt::/t///e.c http::/:///////vvee/e/e///m wet.eiewwweeeeeeeeeeeeeeeeeeeeee net/forurrrnr forum Pid-HUxewnCeF7,
URL http::hpp//..... rivii org/ abs 2309. 04082. arXiv:2309.04082 [es].
Hagberg, A. A., Schult, D. A., and Swart, P. J. Ex- ploring Network Structure, Dynamics, and Function us- ing NetworkX, pp. 1 11-15, Pasadena, California, June 2008. doi: 10.25080/TCWV98331 URL https: / //doi. wwweweuueeeecc..oco.. com/ /00 25080/ 25500//CC/9985511 TCWV9851.
Ding, J. and Regev, A. Deep generative model embedding of single-cell RNA-Seq profiles on hyperspheres and hy- perbolic spaces. Nature Communications, 12(1):2554, May 2021. ISSN 2041-1723. doi: (0.1008 -22851-4. URL https: / /www.. nature com/ a art cc16s/ 100002222800 -4, 4. Publisher: Nature
Hunter, J. D. Matplodii:: A 2D Graphics Environment.
Computing in Science & Engineering, 9()):0--55, May 2007. ISSN 1558-366X. doi: 10.1109/MCE.2200..555 URL https: / leeexplore eeee wrt..orggg.o.ooo.......oo ment 416 Conference Name: Computing in Science & Engineering.
Publishing Group.
Do Carmo, M. Riemannian Geometry. Springer US, 1992. URL 780817634902.
springer com/book/9
Johnson, D. The genealogy theoretical computer sci- ence: preliminary report. SIGACT News, 16(2):36-49, July 1984. ISSN 0163-5700. doi: 10. 10.1145/100895..1 08960. URL https: //di: acm. act.tpp//d..aa........mm acm.
Doorenbos, L., Márquez-Neila, P., Sznitman, R., and Mettes, P. Hyperbolic Random Forests, August 2023. URL htwp::htt//t /attp://arxiv... arxiv. htgg/bbs/2300 2tt.//22....177...... 13279. MxXiv:2308.1227 [cs].
45/1008959. 1 1008960.
Elhage, N., Lasenby, R., and Olah, C. Privileged Bases in the Transformer Residual Stream, 2023. URL https:
Jones, A. R., Overly, C. C., and Sunkin, S. M. The
Allen Brain Atlas: 5 years and beyond. Nature Reviews Neuroscience, 10(1 11:211-818888 November 2009. ISSN 1471-0048. doi: 10. 10.1038/nmm272)
serransformer-circuu cttpouc...tttttto1333333 pub/2023/privv/ deged-basiss/ndeex. html.
URL https: / //www. www..nature nwwuratuueeee. com art werricclee/n22222 Pub- lisher: Nature Publishing Group.
Fedesoriano. Traffic Prediction Dataset, 2020. URL http
s: s: st///ww.kaggle..oo http:///w....ccmmaaaaeeaaaaeaaae settiiasssstseeeesrriioo auop/traffic--reedittto----ae
Kingma, D. P. and Ba, J. Adam: A Method for Stochastic Optimization, January 2017. URL http: arxii.. org/ abs /11//1111112 6980. arXiv:1412.6980 [cs].
Ganea, O.-E., Bécigneul, G., and Hofmann, T. Hyperbolic Neural Networks, June 2018. URL http: / /aaxiv.ivvvvv org/ argg/bs/Die aaasaalss 09112. 11t:::1000001111 [cs].
Knuth, D. E. The Stanford GraphBase platform for combinatorial computing. New York, N.Y. ACM Press Reading, Mass. Addison- 1993. ISBN 978-0- 201-54275-2. URL httpp//// ils/ Llssstaanforddrapbbas0000 www.:rcccieeeeeeeeeeooooo org/deta
Giles, C. L., Bollacker, K. D., and Lawrence, S. CiteSeer: an automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, DL '98, pp. 89-98, New York, NY, USA, May 1998. Association for Computing Machinery. ISBN 978-0-89791-965-4
Kochurov, M., Karimov, R., and Kozlukov, S. Geoopt: Riemannian Optimization in PyTorch, July 2020. URL h tp:: rxxxxxxx ii. o a s 2 0.5 028 MeXiv:2005002819 [cs].
doi: 10. 10.1145/276655276665 URL https: /dl.a 276685.
cm. cm..rg//eeeee. wtrg//oi//1..oo
Giovanni, D., Luise, G., and Bronstein, M. Heteroge- manifolds for curvature-aware graph embedding. February 2022. URL http: //tp///avvvv///////////e///aawa/ org/ arg/abb. abs/22 02. 01185. arXiv:2202.0113 185.
Krebs, V. Books about US politics, 2004. URL http: / //www. www.or.gn....ooo com.
Krioukov, D., Papadopoulos, F., Kitsak, M., Vahdat, A., and
Girvan, M. and Newman, M. E. J. Community structure in social and biological networks. Proceedings of the National Academy of Sciences of the United States of
Boguñá, M. Hyperbolic geometry of complex networks. Physical Review E. 82(3):036166, September 2010. doi: 10.1 St..Phyyseee...2.36111111 URL https: /link

---

Mixed-Curvature Decision Trees and Random Forests
.aps .org orgg/do//i /ooi/1.... 10100.1103 11011113//100 1103/PhysRevE, 82. 036 106. Publisher: American Physical Society.
Nguyen-Van, T., Le, D. D., and Ta, T.-A. Improving Hetero- geneous Graph Learning with Weighted Mixed-Curvature Product Manifold, July 2023. URL http: / arxiv. org/ abs abs//a300 200. 04514. Sall:2307.045514 [cs].
Krizhevsky, A. Learning Multiple Layers of Features from Tiny Images. 2009.
Nickel, M. and Kiela, D. Poincaré Embeddings for Learning Hierarchical Representations, May 2017. URL http: a arxiv. wtwg/bs/////eooo/eo.ooo//eeeeee/ /175 08039. arXiv: 1ew::0005000999
Lecun, Y., Bottou, L.,, Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Proceed- November 1998. ISSN 1558-2256. doi: 10.1109/5.726799 URL https: htteeeerrr///uueet///e/ee @ow.mmnt/72611 Conference Name: Proceedings of the IEEE.
ings of the IEEE, 96(11)2227-233
Pedregosa, E,, Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12(85):2825-2830 2011. ISSN 1533-7928. URL StLbttp:// http://mmmrrrrorpeeeeeeeee s s/v12 htw.:peddeegssalaa.se...
/ //ieeexplore. ieee. org/
Li, A., Perry, R., Huynh, C., Tomita, T. M., Mehta, R., Arroyo, J., Patsolic, J., Falk, B., and Vogelstein, T. Manifold Oblique Random Forests: Towards Closing the Gap on Convolutional Deep Networks, September 2022. URL http:://///// arxiv. org/ abs 1909. 11799. [cs].
Sen, P., Namata, G., Bilgic, M.. Getoor, L., Gallagher, B., and Hliassi-Rad, T. Collective Classification in Network Data. Al Mag., 29(3):93-106, September 2008. ISSN 0738-4602. doi: 10. 10.1609/aima..v299322222 URL http s://doi. 1 609 hat0g///ama...//./////// 02911.2117
Li, K., Zhang. Y., Zhu, J., Li, X., and Wang. X. Multi-space interaction learning for disentangled knowledge-aware recommendation. Expert Systems with Applications, 254:
124458, November 2024. ISSN 0957-4174. doi: 10.101
Skopek, O., Ganea, O.-E., and Bécigneul, G. Mixed- curvature Variational Autoencoders, February 2020. URL htttp://azee arXiv:1911.00 1.t119111.811111 [cs, stat].
6fj.eswa.024.1144555 URL https: httttps:/www.scieeett
edi cwm/ htt//ciieeeee http:///eeeneeaaarce/pppppeeeeea 7417424013241
org/ http:/bb//11111110 1841.
Sonthalia, R. and Gilbert, A. C. Tree! am Tree! I am Low Dimensional Hyperbolic Embedding, October 2020. URL http: : arxiv....... org/ abs /tbs/2200 20000 03847. ar arXiv:2005.03847 [es, math, stat].
Lusseau, D., Schneider, K., Boisseau, O. J., Haase, P., Slooten, E., and Dawson, S. M. The bottlenose dolphin community of Doubtful Sound features large proportion of last-ng associations. Behavioral Ecology and So- ciobiology, 54(4):396 405, September 2003. ISSN 1432- 0762. doi: 10. //doi . org 10. 1007
Sun, L.,, Zhang. Z., Ye, J., Peng. H., Zhang. J., Su, S., and Yu, P. S. A Self-supervised Mixed-curvature Graph Neural Network, December 2021. URL http: :/ / aariiv... voor or 05393. arXiv:21 [es].
URL https: 00000000-0666630
g/
McNeela, D., Sala, F., and Gitter, A. Product Manifold Representations for Learning on Biological Pathways January 2024. URL http://arxi:: wwwp:iivvorr/aooommm abs/24 01.15478. arXiv:2401.15478 q-bio..
Tabaghi, P., Pan, C., Chien, E.,, Peng. J., and Milenkovic, O. Linear Classifiers in Product Space Forms, February 2021. URL http: / arxivv viv.v.vv org/ abs/ / 22/222222222 10204. arXiv:2102.10204 [cs, stat] version: 1.
Mishra, B., Kasai, H., and Jawanpuria, P. Riemannian optimization on the simplex of positive definite matrices, November 2020. URL http: //tpa/.iv.o//aaba..../. 1906. 6.
Tabaghi, P., Khanzadeh, M., Wang, Y., and Mirarab, S.
Principal Component Analysis in Space Forms, July 2024. URL ht http: : /arxivivvvvv.. org/ abs 223223111200 02750. Kkiv:2301.02755 [cs, eess, math, stat].
Nagano, Y., Yamaguchi, S., Fujita, Y., and Koyama, M. A Wrapped Normal Distribution on Hyperbolic Space for Gradient-Based Learning, May 2019. URL http:// arxiv. wwt.:ogg///bbsee 0291/ 02992. arXiv:1902.02992 [es, stat].
Tsagkrasoulis, D. and Montana, G. Random Forest re- gression for manifold-valued responses, February 2017. URL htt / a rx viv. org abs/ /00.. 08381.. arXiv: 1iti:1101111 701.0838. [stat].
Udrişte, C. Convex Functions and Optimization Methods on Riemannian Manifolds. Springer Netherlands, Dordrecht, 1994. ISBN 978- 90--990------0 -4440-2 778-994-015-8390- doi: 10. 10.1007/778-00 91-778999--------990 URL http: 1ink
Newman, M. E. Finding community structure networks using the eigenvectors of matrices, May 2006. URL ht tps://erxiv. www.rrwwwww....veooooe wtwg/bb/////eee..ee//gg///eeeerb wtt.:///s/psssscs////ee////////e
12

---

Mixed-Curvature Decision Trees and Random Forests
wpwinger.comm/1... ottpp/prr...ommoooooooo///e----- 0-9.
Virgolin, M. Time complexity for different machine learning
algorithms, February 2021. URL https::tpp..cccc https://mme rcov argolin.ggthhine httpeettas/detaalsssstttt
thms/.
Wang, S., Wei, X., Nogueira dos Santos, C. N., Wang. Z.,, Nallapati, R., Arool, A., Xiang, B., Yu, S., and Cruz, F. Mixed-Curvature Multi-Relationa Graph Neural Network for Knowledge Graph Completion. In Pro- ceedings of the Web Conference 2021, WWW '21, 1761-1771, New York, NY, USA, June 2021. Associa- tion for Computing Machinery. ISBN 978-1 118-1-4503-8322 URL https: 3450118.
7. doi: 10. 10.1145/34423811.4453 //doi.org/10/ 11e.:1..111111111
Wikipedia. List of cities by average temperature, August
2024. URL UBL.atress//err.uuiippiaa.......
rage_tempan ragg__ccmperstuuuecccldle::21111
te_note-1 Page Version ID: 1241784795.
Zachary, W. W. An Information Flow Model for Conflict and Fission in Small Groups. Journal of Anthropological Research, 33(4):452-473 1977. ISSN 0091-7710. URL https: /www. jstor. org/ httts/talleee//eee 3629752.. Publisher: [University of New Mexico, University of Chicago Press].
Zhang, S., Moscovich, A., and Singer, A. Product Mani- fold Learning. Proceedings of The 24th International Conference Artificial Intelligence and Statisties, pp- 3241-3249. PMLR, March 2021. https:/ eed ngs. mlr prress/vl33 wttp//////////111 zhang21 html. ISSN: 2640-3498.
Zheng, G. X. Y., Terry, J. M., Belgrader, P., Ryvkin, P., Bent, Z. W., Wilson, R., Ziraldo, S. B., Wheeler, T. D., McDermott, G. P., Zhu, J., Gregory, M. T., Shuga, J., Montesclaros, L., Underwood, J. G., Masquelier, D. A., Nishimura, S. Y., Schnall-Levin, M., Wyatt, P. W., Hind- son, C. M., Bharadwaj, R., Wong, A., Ness, K. D., Beppu, L. W., Deeg, H. J., McFarland, C., Loeb, K. R., Va- lente, W. J., Ericson, N. G., Stevens, E. A., Radich, P., Mikkelsen, T. S., Hindson, B. J., and Bielas. J. H. Mas- sively parallel digital transcriptional profiling of single cells. Nature Communications, 8(1):14049, January 2017. ISSN 2041-1723. doi: 10.1038/ncomms144449 URL attpss: : / www. nature. com/ htmm///ttcles/coom 949, Publisher: Nature Publishing Group.
13

---

Mixed-Curvature Decision Trees and Random Forests
A. Gaussian mixture details
A.1. Overall structure
The structure our sampling algorithm is as follows. Note that, rather than letting M be a manifold of arbitrary curvature. we force its curvature be one of f-1,0, 1, 0, 1} for implementation reasons. This necessitates rescaling steps, which take place
in Equations 32, 36, and 42. The result is equivalent to performing the equivalent steps, without rescaling, a manifold of the proper curvature.
1. Generate vector that divides samples into clusters:
Praw (po, :t..........1111
(26)
P: Uniform(0, 1)
(27)
Praw
Pnorm Σ Ps
(28)
(Co, C1,...C.m
(29)
C: ~ Categorical n. Pnorm)
(30)
2. Sample Meuc, D matrix of class means:
Meuc (mo. m1, m/
(31)
m; (o, KT).
(32)
3. Move Meuc into M, the tangent plane the origin of M, by applying v x (0, per-row M M
Mtan = (w).o)) e (m1), 3 (mm
(33)
R D+1 x + (0,x).
(34)
4. Project Mtan onto M using the exponential map from T&M Mtan:
M cMan)
(35)
5. For 0 i < sample corresponding covariance matrix. Here, a is variance scale parameter that can be set:
ΣS Wishart(oV (oVaart((VKK,,D) D)
(36)
6. For 0 < j < m, sample Xeue. matrix of m points according to their clusters' covariance matrices:
Xene (Xo, xI,... ))²
(37)
Ig ~ NO, Ecy).
(38)
7. Apply v-)) from Eq 34 each xj move it into TrM:
X (vx(x)); (xxx), ((mmm)))
(39)
8. For each row in Xtan apply parallel transport from ToM to its class mean:
XPT (Xo, Ni. Xm mm)
1.
(40)
Xj,x
PTo-mmm (x))
(41)
9. Use the exponential map at T to move the points onto the manifold:
X (X.MM:X1.M.-
X1,M
X 1.M)
(42)
exPm, (xj,a)
XJ.M
(43)
VK
10. Repeat steps 2-9 for many manifolds as desired; produce final embedding by concatenating all component embeddings column-wise:
X = (X XM XM₂)
(44)
14

---

Mixed-Curvature Decision Trees and Random Forests
A.2. Equations for manifold operations First, we provide the forms of the parallel transport operation in hyperbolic, hyperspherical, and Euclidean spaces:
PT PI (v) = v (u aua- -auuicc
(v ( + p)
(45)
a+1
a = H
(46)
+ sin(d) (1 cos(d)))
PT vー (v) = v cos(d)
(47)
d
d = cos foosst(vv (v.
(48)
PT PT = + μ-
(49)
The exponential map is defined as follows in each of the three spaces:
exp x (u) cosh
(|| =) + sinb([|ulle)
(50)
|u|c
exp 8 (u) cos( [|)ppp|pp- sin(||ul) u
(51)
||u||
(xp u.
(52)
A.3. Generating classification targets
To generate classification targets covering p S classes, all we need to do is map clusters classes. To ensure that each class has at least associated cluster, arbitrarily assign the first p clusters to the first p classes. In the p case, case, assign the remaining by
this is equal to the p-dimensional identity matrix, and we conclude. In the drawing assignments from a uniform categorical distribution over the classes.
A.4. Generating regression targets
To generate regression targets, we draw per-cluster slopes and intercepts:
fi,k Uniform(-1,11 a; ~ Uniform 10, 10
(53)
(54)
We then multiply each æj E Xauc (i.e. the pre-transport samples from the normal distribution) by 3 and add o:
y x t
(55)
To make the regression task more constrained and, therefore, make the RMSEs across samples comparable, further normalize the labels to the range [0, 1] by subtracting the minimum y value and dividing by the range.
A.5. Relationship to other work
Nagano al. (2019) developed the overall technique used for single cluster and a single manifold, i.e. steps 6-9. Chlenski et al. (2024) modified this method to work for mixtures of Gaussians in H. and deployed it for d 2, 4,8, 16. This corresponds to steps 1-5 of our procedure (although note that our covariance matrices are sampled differently in step 5), Thus, our contribution is simply to add step 10, modify step 5 to use the Wishart distribution, to add curvature-related scaling factors in Equations 32, 36, and 42, and to generate classification and regression targets as described in the preceding sections.
We apply this to hyperspherical manifolds, for which the von Mises-Fisher (VMF) distribution is typically preferred. This is unconventional choice, but has been employed previously by Skopek et al. (2020) in their mixed-curvature VAE formulation. We do not argue for the superiority of our approach over the VMF distribution in general; however, we prefer to use ours for these benchmarks, as it allows us to draw simpler parallels between manifolds of different curvatures.
15

---

Mixed- Mixed-Curvature Decision Trees and Random Forests
B. Proof of equivalence for Euclidean case
classical CART tree splits data points according to whether their value in a given dimension greater than or less than some threshold value Midpoints are simple arithmetic means. This can be written as:
af t,
S'x,
(56)
otherwise.
mbr(u, Ud + Ud 2
(57)
In our transformed DT, we lift the data points by applying o x + (1, x) and then check which side of an axis-inclined hyperplane they fall on. The splitting function is based on the angle 9 of inclination with respect to the (0, d) plane, i.e. (1, ra)- Our midpoints are computed to ensure equidistance the original manifold:
S(x, d, 0) = sign (sin(0 - cos(0)
(58)
m, v) = tan -1
(59)
ud + vd
To demonstrate the equivalence of the classical DT formulation to our transformed algorithm in E, we will show that Equation 56 is equivalent to Equation 58 and Equation 57 equivalent to Equation 59 under
cot (t).
(60)
B.1. Equivalence of Splits
First, we show that Equations 56 and are equivalent, assuming & * 0:
S(x, d) = sigu(sin(O)az, - cos ()))))))))))))))
(61)
sin(0) cos ()) > 0
(62)
sin(f)
cos(0) Id tan (f)x >
(63)
xalt=
(64)
Fd
(65)
S'(x, () =
(66)
B.2. Equivalence of midpoints Now, we show that Equations 57 and 59 are equivalent:
Ud + Ud
cot -o (mor(u, v)) =cot-1
(67)
2
2
=tan -1
(68)
Udd
UR
= =mg(u, v)
(69)
16

---

Mixed-Curvature Decision Trees and Random Forests
C. Summary of angular midpoint formulas
Table 5: Distance functions and midpoint angle formulas for each component manifold type.
Manifold M
Distance 8M (u, v)
Midpoint angle (u, v)
K (u, v)
u 8v
g
0Us
K
2
EP.0
(u, v)
lan
Un vu
cosh
-1 '(K2 (u. v)2)
H.
cot (V VV2 if
n.
K
cot (V + vV2 1) otherwise. sin - 20,)
sin(On + 0., sin Ou)
D. Full benchmark details
D.1. Product DT//RF hyperparameters
For our models, we set the n features = "n choose 2" parameter. This means that we consider all (?) linear projections. We do this because we restrict ourselves to 2-dimensional component manifolds, and therefore we only observe ()) 33 total angles, equal to the number of features used by ambient space Euclidean methods. All other hyperparameters set identically to the seikit-leam DT/RF settings below.
D.2. Scikit-learn hyperparameters
Random forests and decision trees. For fairness, we set all DT and RF hyperparameters identically. Specifically, we set the following hyperparameters for both DT's and RFs:
max-depth
min.samples.split 2
min.samples.leat =
min.impurisn decrease 0.0
For RFs, also set the following hyperparameters:
n_estimators 12 max_features = "sqrt"
bootstrap = True (subsamples the training data)
max-samples = None (draws samples from set of n points)
Because the scikit-learn implementation differs substantially from ours, subsamples vary even when the random seed is set. Nevertheless, we also employ the same random seed for all models.
k-nearest neighbor models. For k-nearest neighbors, we use default hyperparameters.
Product space perceptrons and SVMs. Product space perceptrons only have one hyperparameter, which is the relative weight assigned to each component manifold. We elect to give each component manifold equal weight.
Neither the SVM code provided by Tabaghi et al. (2021) nor our own reimplementation would run on our datasets. In particular, we had issues satisfying the convexity constraints described in their paper, causing the solver to crash. Correcting this mistake and augmenting our benchmarks with SVM evaluations direction for future research.
17

---

Mixed-Cure Mixed-Curvature Decision Trees and Random Forests
D.3. Neural networks
x-GCN overview. Neural networks, especially graph neural networks, are popular choice for representing and working with mixed-curvatur representations (Sun al., 2021; Cho al., 2023; Bachmann et al., 2020; McNeela et al., 2024). In particular, use the Manify (Chlenski et al., 2025) implementation of K- k-GCNs, described in Bachmann al. (2020), as the basis for our neural network models. This model is heavily inspired by prior work on generalizing graph convolutional networks (GCNs) in hyperbolic spaces (Chami et al., 2019; Ganea et al., 2018). Since the x-GCN model uses different models of non non-Eundle -on-Euclidean space (the Poincaré disk for hyperbolic space and the projected sphere for hyperspherical space), we transform our points to these spaces using the standard stereographic projection:
ID)) - 1 K
21
21
2D
(70)
1 v K| 250
V
K 200
At each layer, the k-GCN applies a weight matrix W and aggregates updated embeddings using an adjacency matrix A (analogous the traditional GCN update operation, H o o (AH'W), except using manifold-appropriate variants of left- and right-matrix multiplication, and applying the nonlinearity through logarithmic and exponential maps. At the final layer, the и- GCN computes stereographic logits:
Ply- x) Softmax (logits .n (x,
(71)
|ak Pk sin
2/ [ (Zk, ak)
logits (M k))
(72)
VK K
(1 + K 2 [2)[|aa))
where ak is column vector of the final weight matrix corresponding class k. Pk M is bias term, and Zk -P x. with @ denoting the Möbius addition operation. It is not totally apparent in Bachmann al. (2020) how the logits aggregate across different product manifolds; we follow Cho et al. (2023) in aggregating logits the fnrmrrm of component manifold logits, multiplied by the sign of the sum of the component inner products:
logitsp kx
logits ((giss
Σ (xM: ak
(73)
MEP
MEP
Intuitively, this generalizes the notion that output logits correspond to signed distances from some hyperplane specified by the column vectors and biases of the final layer; all modifications to the standard logit formula simply reflect the behavior of distances in these manifolds.
Implementation and variants. We use our own implementation of the R-GCN architecture, loosely based on the implementation of stereographic logits given Cho et al. (2023). The R-GCN class can be manipulated in several ways: in the K 0 case, it behaves exactly like Fuclidean graph convolutional network; when A I, i.e. the adjacency matrix provided is trivial, it behaves like manifold-appropiin version of an MLP. We use this to derive the neural models we benchmark as follows:
Table 6: A summary of the neural models benchmarked in our work. Here, Materen denotes the stereographic projection of M, means the ambient dimension of M, and d(-) is the stereographic projection.
Model
Preprocessing
Manifold
Hidden dimensions
A
Ambient MLP
ED
[E g22
I
Tangent MLP
Log (X)
ED
(E 322
I
Ambient GCN
RDD
(m32 12
A
Tangent GCN
log (X) x(X)
ED
(E g32)
A
R-GCN
Macroo
(Maene))
A
R-MLR
o(X)
Massoo
Generating adjacency matrices. For x-GCN to work correctly, is important that the adjacency matrix be correctly
normalized. We use standard method, as described in Bachmann et al. (2020), to generate appropriate adjacency matrices.
18

---

Mixed-Curvature Decision Trees and Random Forests
For some adjacency matrix A, do:
A' A AT
(74)
Å = A' +I
(75)
D3 = Σ Aik
(76)
K
Á D A/AAD-1 1/2
(77)
Le. A' is A made symmetrie, A has self-connections, D is diagonal matrix of row-wise degree sums, and A is the properly normalized version of A.
When an adjacency matrix is not provided (i.e., for all benchmarks except the graph embeddings), we compute A via a standard Gaussian kernel on the normalized pairwise distances between points:
_8M(X, Ar = exp mmxx, 8(Xk, xi)
(78)
followed by the transformation from to À as descrihed above. Classification. The stereographic logits described above can be turned into classification targets through standard softmax function.
Regression. For regression problems, we set the output dimension of our k-GCN to and skip the final softmax. We use a mean squared error loss function to train. This variant of k-GCN recapitulates the typical relationship between regression and classification; however, it has not previously been described. our experience, it unfortunately tends to grossly underperform other models.
Link prediction. The link prediction variant of the k-GCN was also described in Bachmann al. (2020). We follow closely related paper, Chami et al. (2019), in the standard choice of applying the Fermi- decoder (Krioukov et al., 2010; Nickel & Kiela, 2017) to predict edges:
5)( E x;) = (exp [6Me(xs (Bx((x;;)2))) +1)
( t
(79)
where the embeddings for points i and j. x; and Xj, may be updated by x-GCN layers.
Shared hyperparameters. For all neural networks, we used learning rate of .0001 and trained for 4,000 epochs. For Euclidean parameters, we used Adam (Kingma Ba. 2017), whereas for non-Euclidean parameters used Riemannian Adam (Becigneul & Ganea, 2018) implemented in Geoopt (Kochurov al., 2020). Both optimizers use the hyperparameters 31 0.9 and 82 0.999. These hyperparameters were chosen on the basis of their convergence and good performance in exploratory hyperparameter sweeps.
D.4. Graph embeddings
Learning embeddings. We reimplement the method in Gu et al. (2018) to learn graph embeddings. In particular, we use the NetworkX package (Hagberg et al., 2008) to load the graph, extract the largest connected component, and compute pairwise distances between nodes using the Floyd-Warshall algorithm. For embedding purposes, we treat all graphs as undirected. Pairwise distances were normalized into the range [o, 1] by dividing by the maximum distance. To prevent train-test leakage, we take non-transductive learning approach and mask out the gradients from the test nodes to the training nodes during the embedding process.
Embedding hyperparameters. Embeddings were learned using Riemannian Adam (Becigneul & Ganea, 2018) imple- mented in Geoopt (Kochurov et al., 2020). For each signature, we train 10 randomly-inttiaiized embeddings for 10,000 epochs each. We treat the first 2,000 epochs as a burn-in period, during which the learning rate is .001 and the curvature of each manifold is fixed. For the remaining epochs, we train embedding coordinates with a learning rate of 0.01 and scale factors with learning rate of 0.001. These hyperparameters were chosen based on their stability and convergence in exploratory experiments.
19

---

Mixed-Curvature Decision Trees and Random Forests
Train-test split. We avoid train-test leakage during embeddings generation by masking the gradients from the test set the training set. Similarly, we performed the train-test split at the node level for all tasks including link prediction, meaning there was not leakage through the adjacency matrix.
Evaluations. Since it was not clear priori which signature would embed each graph the best, we learned 10 embeddings for each candidate signature and took the one with the best Daeg to be the benchmark signature. Our reasoning is that the lowest-distortion embedding of the graph is the most appropriate benchmark for evaluating the geometrical appropriateness of a classifier.
Link prediction. To generate link prediction datasets, we trained 100 randomly initialized sets of node embeddings in S x EE2 x H., If we let X be our original node embeddings and & be the ground-truth edges of the graph, we then generated the following dataset:
XLP (x)) for (x,, Xj, (x,, xj) E X]
(80)
YLP {I{(x,,x)) E E} for (x, X}
(8 1)
The corresponding signature is (P)2 E in the case of our embeddings, that is (S² E² x2) For GCN-based models which use an adjacency matrix, we applied Gaussian kernel to the normalized pairwise distances described in Equation 78; this prevents the labels from leaking into the training process through the adjacency matrix.
D.5. VAE training
Encoder/decoder architectures. Following Tabaghi al. (2021), we use the following encoder/decoder architectures:
Lymphoma dataset: Two 200-dimensional hidden layers, 500 epochs
Blood cell dataset: Three 400-dimensional hidden layers, 200 epochs
Omniglot and MNIST: 400-dimensional latent
CIFAR-100: 4 x convolutional kernels with stride 2 and padding 1. Encoder: 3 CNN layers of 64, 128, and 512 channels. Decoder: 10n8-dimensional dense layer, followed by CNN layers 256, 64, and 3 channels.
Training hyperparameters. Our VAEs were trained using the Adam optimizer (Kingma Ba, 2017) with default parameters (learning rate -001, 31 0.9, 82 0.999. In all models, each layer except the last is followed by ReLU activation function. Curvatures were trained identically, except using a learning rate of .0001, after 100 burn-in epochs. Because some training details were omitted from the original papers, we additionally chose the following hyperparameters:
Batch size: 4,096
Number of samples per point: 64
(weight for KL-divergence in VAE loss):
Train-test split. To minimize the risk of data leakage, trained our VAEs only the training data, then used the trained VAEs to generate embeddings for the training and test data. Embeddings were generated by running points through the VAE encoder and taking the returned mean parameter.
Evaluations, To conserve memory, we randomly subsampled 1,000 points from the training and test sets for each evaluation. We ran 10 trials per dataset in total.
D.6. Empirical datasets
Landmasses. We generated a geospatial classification dataset for land versus water prediction by sampling 1,000 points from an evenly sampled grid of 10,000 longitudes and latitudes, transforming them to 3-dimensional coordinates, and assigning a "land" or "water" label to each point using the Basemap library in Matplotlib (Hunter, 2007). For classification, we associate the 3-dimensional coordinates with the signature $?.
Neural spiking prediction. We patch-clamp electrophysiology datasets downloaded from the Allen Mouse Brain Atlas (Jones et al., 2009). We arbitrarily pick Neurons 33 and 46 for their nontrivial spiking dynamics. We perform a train-test split by taking the first 80% of time points for training, and holding out the rest for testing. To represent signals in product spaces, we apply a Fast Fourier Transform to the training data and take the top 10 Fourier coefficients by magnitude; we label time points as "spiking" (1) or "not spiking" (0) according to whether their amplitude is greater than the median amplitude in the training data. We then take their corresponding frequencies f, and represent each time point in S! via the
20

---

Decision Trees and Random Forests
following transformation:
2 1 ) sin [i
L 10 ) Ji
Φ: IR. - ( (6(1
Cos
(82)
i=1
This yields a product space representation in (S!)10, For each benchmarking trial, we randomly sample 800 points from the training set and 200 points from the test set. We plot both signals, along with their reconstruction using their top 10 Fourier components and the train-test split, in Figure 7.
Global temperature by month. We downloaded list of global average monthly temperatures for the 400 largest cities in the world from Wikipedia (Wikipedia, 2024). We transform longitude and latitude into 3-D coordinates to represent our data in S To convert months to S! valued coordinates, we transform ordinal representations months E [0. 11] via the following transformation:
@: + S x(t)
cos
2nT
sin
2n
(83)
12
12
This yields a product space representation of the data in S x S'. For each benchmarking trial, we randomly sample 1,000 (city, month) pairs, and then apply a standard train-test split.
Traffic prediction. download an automobile traffic prediction dataset from Kaggle (Fedesoriano, 2020). This dataset aggregates readings across four sensors with date and time annotations. We process the date and time annotation into day of year (d), day of week (w), hour (h), and minute (m) labels and transform (S1)a analogously the month timestamps in
Sweep 33
0.025
Voltage (mV)
0.000
-0.025
-0.050
-0.075
Raw signal
Sweep 46
Fourier approx.
Train/Test split
0.025
Spike events
Voltage (mV)
0.000
0.025
-0.050
0.075
400
1000
1200
1400
Time (s)
Figure The "Neuron 33" and "Neuron 46" datasets. Their reconstruction using the top Fourier coefficients is shown in red, and the top half of voltages are colored blue. The first 80% of time points were used determine Fourier coefficients and model training, while the last 20% are for testing.
21

---

Mixed-Curvature Decision Trees and Random Forests
the global temperature data. Letting I be the (numeric) label of the sensor, apply the following transformation data:
Φ: R (S*)S E
(84)
(ld w. h m, 1)
cos
2T
sin
2T
365
365
008
2T
sin
2T
h
008
2n
sin
2T
24
24
m
77L
COS
2TT
sin
2m
(85)
60
60
22

---

Mixed-Curvature Decision Trees and Random Forests
E. Ablations and effects of hyperparameters
For experiments, we sampled 100 mixtures of 32 Gaussians using the signature P @ E² H in an 8-class regression setting (analogous to the multi-K benchmark in Tables and 3, varying one parameter at time. Results are plotted in Figure 8.
Ablation: feature subsampling strategy
0.9
0.8
0.8
Score
0.7
F1 Score
0.7
0.6
0.5
0.4
DT
DT choooaa__2
RF (4)
RF
(8)
(d_choase_2
(sqrt)
(log2)
(none)
(a) Changing the number of features seen by each DT/RF from 9 by including the (21, angle is massively heneficial.
(b) Changing feature subsampling approaches in RFs doesn' appear to do much.
Ablation: maximum depth
Ablatton: midpoint
0.9
0.8
0.8
F1 Soore
Score
0.7
0.4
0.5
0.4
RF (None)
(1)
(6) (None)
(3)
(Paise)
(Trie)
(Palse)
(True)
(c) Increasing the maximum depth of each DT/RF is massively beneficial, and shows no signs of overfitting even unrestricted max depth. All within-predictor differences are significant.
(d) Replacing the midpoint-angle computations with arithmetic means has statistically significant effect on performance for DTs or RFs, surprisingly.
Figure Effects of various hyperparameters on the performance of our algorithms. Asterisks imply that a result statistically significant, as determined by the Wilcoxon test with Bonferroni correction applied; asterisks are omitted for subfigure c, where all changes in depth significant.
23

---

Mixed-Curvalee Decision Trees and Random Forests
F. Detailed results
F.. Global temperature prediction plots
prodidiva
prodities chaiter
Pidkatd neesi 2008
Acdiue EJ SEUSE 2006
Priii DI JA6S s0
ACD acbract (4616160
(a) January temperatures.
(b) April temperatures.
July Inagrette prediction Sendde
Desterumpeeran prodities clartter
(c) July temperatures.
(d) October temperatures.
Figure 9: Decision boundaries for the temperature prediction task for the months of January, April, July, and October, colored by predicted temperature across four trained predictors.
F22 VAE latent space visualizations
opece
Coard Mesecrt
Lin Cas coccos Neat
Noara
Lvepieaa Heabe
Clam
CRARLU
Luque
Lequr Mix shout
Micciro Meat
Siaal Mattrack
Thacs
Figure 10: Visualizations of the latent space for all four of the datasets embed using VAE, colored by class. For visualization purposes, we show @ components in 2-dimensional polar coordinates, and project H2 embeddings to the Poincaré disk.
24

---

Mixed-Curvature Decision Trees and Random Forests
G. Runtimes and complexity
summarize complexities for models used this paper, as well as the pairwise distance preprocessing necessary for operations such as computing nearest neighbors and creating reasonable graph edges for GNNs, in Table 7. Complexity estimates are adapted from Virgolin (2021).
To see that the training time complexity of ProductDT is O(Dnd), observe that we must first preprocess the data into angles, which takes (ndd) operations. From there, the angular comparison is constant-time modification to the decision tree algorithm, so the complexity of ProductDT is Onn + Dnd) = ((nd). For inference, asymptotic performance is slightly slower than decision trees because preprocessing an input requires O(d) operations.
using all (t) 2-D projections, training time complexities are all multiplied by d, and the (dd) preprocessing step is added test time complexities.
Table 7: Complexity comparison machine learning models where: n: number of samples, d: number of features, neurons per layer, T: number of layers, maximum tree depth, 8: number of support vectors. We include the complexity of computing pairwise distance, which are necessary for operating models like k-nearest neighbors and GNNs without topologies, well.
Time
Space
Model
Phase
Worst
Avg =ddd
Avg nt= ni + dh + L(h
Worst
Dists
11
Train
ndh + Luhh²
ndh Lidh²
+ rdt)
MLP
Test
122
1:21
16 L
1.2 L
Train
nd nd
d
Perceptron
Test
d
d
d
SVM
Train n'd
n ndd
n
n²
Test sd
sd
sd
sd
Train n'd
n²d
n²
n²
GNN
Test n²
n²
n
n²
Train
1
nd
nd
k-NN
Test
nd ndnnlogn
logn
nd
Train
Dnd
Dnd
2
2
Decision Tree
Test
D
D
1
Train
Dnd
Dnd
2
2
ProductDT (vanilla)
Test
d d+
d- D
d
d
Train
Dnd'
Dnd'
2
2D
ProductDT splits)
Test
d² +
dt + D
d'
d
Runtime vs. Dataset Size
100
20-4
RcNN Prodicct DTC
RLE Pwroeptron
10-3
10'
10
Figure 11: Runtime comparison for all of our methods
25

---

Mixed-Curvature Decision Trees and Random Forests
H. Interpretability and visualization
Alongside their demonstrated accuracy and efficiency, decision tree algorithms are attractive for their tractability and interpretability. particular, given a trained decision tree T, is possible to:
1. Predict its behavior on the entire space of possible inputs (equivalently: T partitions P in tractable way).
2. Determine the importance of features (for classic decision trees) or feature pairs/components (for ProductDT) by observing how often and how early feature(ppar/coomoneen)) is used in the decision tree procedure. Heuristically, early-splitinng features are more important.
3. Visualize every node using a 2-dimensional projection of the input data and angle
H.1. Submanifold-level attribution experiment
To determine whether our method could accurately distinguish between relevant and irrelevant submanifolds, we drew independent samples from Gaussian mixtures in HH? IE?, and $?, and yielding datasets (Xg: Yx), Vs). We
then concatenated these embeddings together:
Xp X X3 Xg.
(86)
We trained three separate decision tree models on Xp, using Yи Yg: and Yg as labels. Because the labels and embeddings were drawn independently, should be the case that only the component from the same manifold as the labels contains any relevant information, and the other two components are simply noise. Therefore, measuring the fraction of splits that fall in the "correct" manifold is useful proxy for understanding tree models' ability pick out signal that happens in individual component manifolds.
Our results are summarized in Table 8. We found that both product space and ambient decision trees perform well at this task, which is to be expected.
We note that this analysis is unique to tree methods, where the split dimensions are part of the architecture; other methods, such as perceptrons, k-nearest neighbors, or neural networks are harder to query for feature(ccomponent) importances. Therefore, we consider this simple experiment useful demonstration how decision tree learning can reveal aspects of structure in mixed-curvature datasets that other learning algorithms cannot reveal.
Table Intepretability outcomes for Gaussian mixture. Percentages reflect the proportion of splits in the trained decision tree which fell in the non-spurious component manifold.
Model
H2
E22
S2
Product DT
100%
83%
86%
Ambient DT
100%
83%
67%
H.2. Visualization
trained tree gives all of the information need visualize the data and how it is split at every node, since each node looks at a 2-dimensional projection. We display three levels of a decision tree with max depth of 3 in Figure 12. Note that, in this case, the decision tree also gives us relevant information about which 2-dimensional projections are worth looking at on the basis of their feature importances.
26

---

Decision Trees and Random Forests
Figure 12: An example of a visualized decision tree for a Gaussian mixture in P SH?. For each split, we show the 2-dimensional projection under which split angles are parameterized. Greyed-out points are discarded earlier in the tree, and therefore do not participate the split at that level.
27

---

Mixed-Curvatie Decision Trees and Random Forests
I. Additional results
Table 9: Neural network classification benchmarks.
Amblent GCN
Amblent MLP
Amblent MLR
Tangent GCN
Tangent MLP
Tangent
Dataset
Signature
R-GCN
K-MLF
X-MLR
MLL
Synthetic (malti- K)
Gaussion
g
+
2.1
28.2
3.9
27.7
2.7
26.5 2.3
28.2 4..
28.2
E
12.0
5.34
2.3
3.7
4..1
4.33
2.79
30,0 299
2.6
2.2
2.9
2.0
3.7
3.7
(
1222
3.97
4.0
2..2
28.7
2
28.0
33.5
33.3
In?
17.6
2.5
24
1..4
14.1
1.66
2.7
28.6
28.8
g²
26.4
1.6
2.91
2..
8?1ga
2.7
23.7
2.
2..0
27.1
1..1
2..5
2.3
2.
(89ya
1.7
2.1
2.3
15.0
1..1
1..1
2.
22.8
22.7
Graph
CiteSeer
(
2..
13
13
2.5
2..0
2..1
24.8
24.9
24.8
Cora
E
2.85
2..8
2.9
29.6=
0.99
2.97
28.9
2.93
29.4
PolBlogs
92.2209
21.4
13.4
Blood
S (E"""
13.012
0.9
1390
1..4
116408
12.2
12.7
12.1
12.1
VAT
CIFAR 100
(113)4
0.
7.6
7.5
5.7
0.9
0.6
0.8
7.7
7.7
Lymphoma
0.
7.
78.0
66.9 13.9
62.8 14.8
614+1599
7..0
7..1
78.1 0.4
MNIST
2.39
18.0
37.4+62
Other
Landmasses
Neuron 33
(8 (t
84
2.5
76.1220
7..5223
2.0
7.1
2.0
47.6
73.2
09.3+39
46.5
7..3
4..0
4.55
2.41
55.0
7..4+:0
$4.8
0.
Neuron 46
20.7
50.6
50.6 2.0
4..7
50.4
50.9
21.5
$3.7
Table 10: Neural network regression benchmarks. Missing values represent failed runs.
Ambient GCN
Ambient MLLP
Ambient MLR
Tungent GCN
Tangent MLP
Tangent
Dataset
Signoture
K-GCN
x-MLP
>-MLR
MLR
Canssian
g
0.21 0.03
0.25 0.03
0.25 0.83
0.21 0.03
0.25 0.03
0.25 0.83
0.21 +0033
0.25
0.25 0.03
Synthette (male: X)
3.12 2.46
64 9.0E3
4.6 4.7.E3
0.07
0.13 0.02
0.13 0.02
0.17 1.0.2
0.00
0.02
H
2,33 4.00
0 0.78
0.77 0.78
0.17 6.03
0.22+ 0.03
0.34+0.04
0.17 0.03
0.03 0,03=0000
0:33+ 0.00
85.94 152.83
59 1.144
17.79 31.58
0.17 0.07::0.03
0.21 0.00
0.20 0.03
0.17
0.00
0.02
1,96 3.33
7.96 13.50
0.63 0.63
0.23 0.06
0.23
0.28 0.06
0.17 2.03
1:000000
0.02
0.02
0.18
0.19 0.03
0.25 0.04
0.26 0.04
0.04
0.18 0.02
0.00
0.03
g2ga
0.17 0.04
0.22
0.22 0.04
0.22 0.05
0.28 0.06
0.26 0.06
0.17 0.04
0:000000
0,03 1.00
(82)2
0.04
0.21 0.05
0211 0.05
0.30 0.06
0.18 0.02
0.37 0.05
0.18 0.04
0.03 0,0:0000
0:85 = 0.00
3.
39
2.03
Temperature Traffic
g2g1
303.00 25.76
306.03 20.71
305.99 20.08
436.05 + 22.00
407.55 24.70
401.35 24.39
g2 (8
5.17
197
28

---

Mixed-Curvature Decision Trees and Random Forests
Table 11: Comparison of Product RF versus Ambient RF on
mixtures of Gaussians, extending the
dimmensionaa cases Table Asterisks indicate statistical significance (p < .05).
Signature
Product RF
Ambient RF
p-value
H
81.73 = 1.64
79.25 1.33
0.0078
g4
64.38 + 1.42
61.18 1.13
0.0001
(1+2)
69.27 = 1.98
68.15 1.87
0.1191
41.05 + 2.27
40.45
0.3330
60.37 = 1.89
60.17 1.66
0.7323
16
92.02 # 1.12
90.60 1.08
0.0046
g/6
52.58 + 1.68
46.28 1.55
0.0000"
(H*)²
81.45 1.41
80.52 1.44
0.1469
(39)*
66.52 1.53 1.53
64.90 1.73
0.0441
74.50 1.66
73.33 1.45
0.0960
H
93.28
94.22 # 0.80
0.0566
ge4
71.97 1.04
68.37 1.37
0.0000
(1132y2
87.28 + 1.43
87.15 1.43
0.9712
42,97 + 1.81
38.03 + 1.78
0,0000
22
81.98 + 1.58
82.12 1.72
0.8020
94.48 # 1.14
96.00 = 0.68
0.0020
g
69.65 1.39
63.10 1.93
0.0000"
(1H 61222
87.85 = 1.37
88.25 # 1.33
0.5809
2g
84.50 : 1.37
83.55 1.34
0.0949
H 512g512
86.43 # 1.10
83.47 1.05
0.0002
Table 12: Model comparison with decoupled hyperparameters. Because optimal hyperparameters may differ across models, independently selected berf-perrorming hyperparameters using 5-fold cross validation and evaluated these models on held-out test set. All values are accuracy when classifying mixtures of Gaussians in A In general, we find that models converge on similar hyperparameters, and out model consistently outperforms other random forests.
Product RF
Tangent RF
Ambient RF
CV Accuracy
70.74
70.17
69.26
Test Accuracy
71.53
70.07
69.27
max_depth
9
max_features
sqrt
sqrt
None
min_samples.leaf
8
min-samples.splic
8
2
n.estimators
24
24
24
29

---

Mixed-Cure cixed-Curvature Decision Trees and Random Forests
J. Datasets availability
Table 13: All of the datasets used in this paper, with download links and citations. CC-BY-SA is short for the Creative Commons Attribution-SaareAlin license. Allen TOU is the Allen Institute terms of use, found at https: / lenins titute. org/terms-eeeeee of-use/
Datasct
Link
License
Citation
CiteSeer
Network Repository: CiteSeer
CC-BY-SA
Giles et al. (1998)
Cora
Network Repository: CORA
CC-BY-SA
Sen et al. (2008)
Polblogs
Network Repository: Polblogs
CC-BY-SA
Adamic & Glance (2005)
CS PhDs
Pajek datasets: PhD students CS
CC-BY-SA
Johnson (1984)
Adjnoun
Network Repository: Adjnoun
CC-BY-SA
Newman (2006)
Dolphins
Network Repository: Dolphins
CC-BY-SA
Lusseau et al. (2003)
Football
Network Repository: Football
CC-BY-SA
Girvan & Newman (2002)
Karate Club
Network Repository: Karate
CC-BY-SA
Zachary (1977)
Les Mis
Network Repository: Les Mis
CC-BY-SA
Knuth (1993)
Polbooks
Network Repository: Polblooks
CC-BY-SA
Krebs (2004)
Blood
10x Genomies: CD&+ Cytotoxic T-cells
CC-BY-SA
7heng et al. (2017) 17)
Blood
CD&+/CD45RA+ Naive Cytotoxic T
CC-BY-SA
Zheng al. (2017)
Cells
Blood
10x Genomics: CDS6+ Natural Killer
CC-BY-SA
Zheng (2017)
Cells
Blood
10x Genomics: CD4+ Helper T Cells
CC-BY-SA
Zheng al. (2017) Zheng al. (2017)
Blood
10x Genomies: CD4+/CD45RO+ Mem-
CC-BY-SA
ory T Cells
Blood
10x
Genomics: Naive
CC-BY-SA
Zheng et al. (2017)
CDA+/CD4SRAWCI2SS
cells
Blood
CD4+/C22++ Regulatory T Cells
CC-BY-SA
Zheng al. (2017)
Blood
10x Genomics: CD34+ Cells
CC-BY-SA
Zheng (2017)
Blood
CD19+ B Cells
CC-BY-SA
Zheng al. (2017)
Blood
10x Genomics: CD14+ Monocytes Hodgkin's Lymphoma, Dissociated Tu- mor: Targeted, Immunology Panel PBMCs from Healthy Donor:
CC-BY-SA
Zheng et al. (2017)
Lymphoma
CC-BY-SA
10x Genomics (2020a)
Lymphoma
CC-BY-SA
10x Genomics (2020b)
Targeted-Compare Immunology Panel
MNIST
HuggingFace: MNIST
MIT
Lecun et al. (1998) Krizhevsky (2009)
CIFAR-100
HuggingFace: CIFAR-100
None
Landmasses
Basemap 1.4.1: is.land
None
None
Neurons
Allen Brain Atlas
Allen TOU
Jones et al. (2009)
Temperature
Wikipedia: List of cities by average tem-
CC-BY-SA
Wikipedia (2024)
perature
Traffic
Kaggle: Traffic Prediction Dataset
None
Fedesoriano (2020)
30


Reviews:
[{"review_id": "B24DlCdIx2", "reviewer": "Reviewer_5kKt", "summary": "This paper proposes mixed-curvature decision trees (DTs) and random forests (RFs) for data embedded in product manifolds—combinations of hyperbolic, spherical, and Euclidean spaces. The core algorithm selects the geodesic split from three options (hyperbolic, spherical, Euclidean) with highest information gain.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and for acknowledging our “novel integration of mixed-curvature geometries into DTs/RFs” and its “potential for applications in hierarchical or graph-structured data,” and for praising its effectiveness in single-manifold settings.\n\n**Added benchmarks**\nPer the reviewer’s suggestions, we have re-run our benchmarks:\n- We have added ensembles of single-manifold trees, which generally underperform our method\n- We have included F1-macro scores for classification\n- For transparency, we include all neural models.\n\nAccuracies, F1 scores, and MSEs can be found at [https://postimg.cc/gallery/ZbfzMzR](https://postimg.cc/gallery/ZbfzMzR).\n\n**Clarifying benchmark assumptions**\n\nThe reviewer suggests that Table 1 contradicts our performance claims. Table 1 shows that our method does well *both* in problems involving single-curvature and product manifolds; these are different problems, not different methods; therefore, they cannot be contrasted in this way. In particular, our benchmarking setup assumes a manifold is already given. For instance, given a dataset with signature H2 x E2 x S2, our only choices are:\n- Coerce to E8 (Ambient models)\n- Coerce to E6 (Tangent models)\n- Coerce to one of {H2, E2, S2} per tree (single-manifold tree ensemble)\n- Our method\n\nOf the above, ours is *the only method* for training a decision tree *without* coercing to a different geometry.\n\nFor details about our selection of signatures, please refer to our discussion with 3Gbp, subheading “Signature selection.”\n\n**Comparison to single-manifold decision trees**\n\nWe justify our choice not to use single-manifold trees in our paper:\n> Letting a single DT span all components—as opposed to an ensemble of DTs, each operating in a single component—allows the model to independently allocate its splits across components according to their relevance to the task at hand\n\nNonetheless, we agree it is better to verify. In our benchmarks, these ensembles tended to underperform other models, including ours. Furthermore, because designing an ensemble of single-manifold decision trees relies on the constant-curvature decision trees we describe in Sections 3.1–3.3, this method is still a novel contribution of this paper.\n\n**Theoretical justification**\n- For an elaboration on geodesic convexity and decision boundaries, please refer to our discussion with SX2z, subheading “Theoretical justification for splits.” In brief, we explain and commit to updating our manuscript with:\n  - A definition of geodesic convexity\n  - A proof sketch for why geodesic splits partition manifolds into geodesically convex subspaces\n  - The equation for geodesic splits in product manifolds\n  - Citations for each component manifold being split by homogeneous hyperplanes\n  - An explicit connection between geodesic splits and our angular reformulation, clarifying that our splits are fully contained within one of the component manifolds, bypassing the complex interactions between component manifold geometries\n  - A more explicit restatement of the relationship between this and the discussion of geodesic convexity in hyperbolic decision trees (HyperDT)\n- Optimization details:\n  - We modify CART, a greedy algorithm without optimality guarantees \\[1\\]. The overall behavior of the algorithm is left intact. Moreover, constructing optimal DTs is known to be NP-hard \\[2\\].\n- Tangent DTs/RFs:\n  - We directly use the method used in \\[3\\]: apply the log-map at the origin and then train Euclidean DTs/RFs on that. We highlight Section 4.3 for a description.\n  - Eqs. 1 and 12 are necessary for extending the log map to product manifolds, enabling the use of tangent DTs/RFs\n\n**Clarifying writing**\n\n- We commit to rewriting the caption for Table 1 to make it clearer that the manifolds represent aspects of the *problem*, not of the *approach* used, to prevent future confusion\n- We will consolidate Section 2 for clarity; more specific editing suggestions are greatly appreciated.\n- Model details: detailed descriptions for Tangent DTs/RFs, as well as neural net benchmarks, can be found in Appendix E; we will expand Appendix E to be even more detailed.\n- We will add the citations from our reply to SX2z.\n\n**Algorithm 1 curvature handling**\n\nAlg. 1 intentionally omits curvature handling because only the sign of the curvature (i.e. manifold type) is needed to compute midpoint angles under our formulation.\n\n**Clarifying Appendix C**\n\nThe reviewer claims “Synthetic results show unclear performance trends / mixed performance; no ablation on manifold combinations” in Appendix C. However, Appendix C is actually a proof of equivalence for the Euclidean case. We request the reviewer clarify this statement.\n\n**References:**\n\n\\[1\\] Hastie (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction.\n\n\\[2\\] Hyafil and Rivest (1976). Constructing Optimal Binary Decision Trees is NP-Complete.\n\n\\[3\\] Chlenski et al (2024). Fast Hyperboloid Decision Tree Algorithms.", "claims_and_evidence": "- Performance claims are seemingly contradicted by results in Table 1, where product RFs underperform compared to single-manifold RFs in some cases.  \n- No proofs or formal guarantees are provided for the decision boundary derivations or optimization steps to support the theoretical foundation.\n- Motivation for product manifolds not sufficiently justified; theoretically-solid alternatives like ensembles of single-manifold trees are not discussed.", "methods_and_evaluation_criteria": "- Synthetic datasets are not described in detail.  \n- F1/accuracy scores are omitted which clearly reduces interpretability of the results.", "theoretical_claims": "1. The decision boundary is unclear. \n2. It is unclear how geodesic splits are computed for product manifolds (e.g., interactions between geometries?).  \n3. The tangent space definition is presented without context, relevance to the algorithm, and  does not directly lead to a tangent RF.\n4. Theoretical guarantees for convergence or optimality are absent.\n5. No theoretical justification is given\n6. Product-manifold decision boundaries are not defined. The paper relies on a ad-hoc approach instead.", "experimental_designs_or_analyses": "- Product MLP/GCN baselines are confusing and not explained.\n- Tangent RFs are unclear in design.\n- Comparing to tangent RFs is not explained.  \n- Competitors are not well explained\n- Lack motivation for manifold configurations (e.g., why specific curvatures).", "relation_to_broader_scientific_literature": "- Builds on mixed-curvature learning but does not cite __specific__ foundational works or sections, but generally refers to a single source for its foundation without specific attribution.", "essential_references_not_discussed": "./.", "other_strengths_and_weaknesses": "#### Strengths\n  - Novel integration of mixed-curvature geometries into DTs/RFs.  \n  - Potential for applications in hierarchical or graph-structured data.  \n#### Weaknesses\n  - Poor attribution (e.g., Riemannian geometry basics cited via a single source instead of specific papers).  \n  - Ambiguous algorithm description (e.g., tangent RFs).  \n  - Writing lacks clarity, particularly in Sections 2.1–2.2.  \n  - The **benefits over an RF ensemble of single-manifold DTs** (hyperbolic + spherical + Euclidean) is _very unclear_. This RF ensemble approach is much cleaner and supported by reliable theory.", "other_comments_or_suggestions": "- Remove tangential details (e.g., PyTorch `arctan2` footnote).  \n- Condense Sections 2.1–2.2;\n- Add some background on DTs.\n- Define a decision boundary and decision rule for product manifolds for DTs.", "questions_for_authors": "- How are curvature parameters selected for synthetic/data-driven manifolds?   \n- Why compare to tangent RFs? How do they differ from product RFs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "upWrWoYwMj", "reviewer": "Reviewer_4Ga8", "summary": "The manuscript develops a methodology for creating decision trees and random forests (classifiers or regressors) by assuming the data coordinates can be decomposed into products of hyperbolic, hyperspherical, or Euclidean components. It is shown that each of those spaces belongs to a class of \"constant curvature manifolds\" and how, in each, the decision boundary can be written in terms of a threshold on an angle (where the different geometry of the manifolds contribute to different angle equations). Then, the product decision tree (and product random forest) can be expressed in terms of the local decision trees. The manuscript demonstrates the method on a range of benchmarks, with extra effort required to establish how many real-world benchmarks can be expressed as sampled from a product manifold.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their favorable comments on our manuscript, including our “**clear and convincing**” claims, “**good results**,” and the “**extra effort** required to establish” performance on real-world benchmarks\n\n**Selection of benchmarks.**  \nOur work makes the admittedly strong assumption that we are working with product manifold-valued data, and our method relies on having product manifold-valued inputs in hand. Although we have tried to create as broad a set of benchmarks as possible, we acknowledge that this can be a limitation of the product manifold approach to machine learning as a whole.\n\nIn our paper, we try to model what end-to-end approaches to machine learning on product manifolds might look like, including examples of how different types of datasets can be converted to product manifold coordinates. We hope that, in addition to supporting the value of mixed-curvature decision trees, our paper can serve as an instructive guide for researchers who are intrigued by product manifolds but aren't sure how these concepts apply to their own datasets. An additional advantage of product manifolds is they recover single-manifold and Euclidean geometry as special cases: thus, all single-manifold problems can trivially be viewed through the product manifold lens.\n\nMore generally, \\[1\\] describes a method based on sectional curvature to determine whether a graph warrants a product space representation; in such cases, we would expect:\n\n* Product-space representations to model the data substantially better (in terms of distortion, classification accuracy, etc) than single-manifold representations, and  \n* Our methods to be effective for classification/regression\n\nWe always welcome suggestions for any further benchmarks that can be formulated in terms of product manifolds, particularly those from application areas not included in our original manuscript.\n\n**Suggestions for applications.**  \nPer the reviewer’s request, we also provide an incomplete list of some situations in which product manifold approaches can be helpful:\n\n* Many non-Euclidean machine learning problems involve multiple manifolds. For instance, \\[2\\] describes hyperbolic convolutional neural networks, in which each channel is hyperbolic. To combine representations in multiple channels, they propose a variant of vector concatenation that stays on the hyperboloid. An alternative way to view this would be that the channels form a product space of individual hyperbolic representations.  \n* Similarly, pairwise (or higher-order) interactions between data points with non-Euclidean representations can be recast in terms of product manifolds. An example of this is our reformulation of link prediction as binary classification on a product manifold of (outbound node, inbound node) pairs.  \n* To embed pairwise distances or other pairwise dissimilarities, one may use the coordinate learning method described in \\[1\\]. This is a popular approach to embedding graphs, where heterogeneous curvature can make single constant-curvature manifolds inadequate. Our graph benchmarks follow this approach.  \n* To embed features without known pairwise distances, mixed-curvature VAEs can be useful, as in \\[3\\].  \n* We are particularly excited about biological applications of product manifolds, which have been a popular application area for hyperbolic deep learning \\[4\\], and where complicated latent structures can give rise to heterogeneous curvature. For instance, we follow \\[5\\] in using mixed-curvature VAEs to embed single-cell transcriptomics data. We speculate that in such datasets, differentiation trajectories may embed in hyperbolic space, whereas periodic signals (e.g. those pertaining to the cell cycle) embed in hyperspherical space. Empirically, \\[5\\] finds that single-cell data embeds better into product manifolds than single manifolds, and \\[6\\] embeds pathway graphs in product manifolds. \n\n**References.**  \n\\[1\\] Gu et al (2019). Learning Mixed-Curvature Representations in Product Spaces.  \n\\[2\\] Bdeir et al (2024). Fully Hyperbolic Convolutional Neural Networks for Computer Vision.  \n\\[3\\] Skopek et al (2020). Mixed-curvature Variational Autoencoders.  \n\\[4\\] Khan et al (2025). Hyperbolic Genome Embeddings.  \n\\[5\\] Tabaghi et al (2022). Linear Classifiers in Product Space Forms.  \n\\[6\\] McNeela et al (2023). Mixed-Curvature Representation Learning for Biological Pathway Graphs.", "claims_and_evidence": "The claims on constructing decision trees in various constant curvature manifolds are clear and convincing, the the construction of a product decision tree from them looks reasonable.", "methods_and_evaluation_criteria": "The manuscript's evaluation criteria are highly biased toward problems in which the input coordinates can be formulated as products of constant curvature. Indeed, the authors make an effort to demonstrate how to cast many problems into those terms. Still, this bias makes the evaluation tricky and prevents reviewers from asking, \"How does it perform on [reviewers-favorite-benchmark]?\".\n\nThe results are good, but only marginally, i.e., comparable to competing methods.", "theoretical_claims": "Looks valid to me.", "experimental_designs_or_analyses": "No.", "relation_to_broader_scientific_literature": "The background on DT and RF is well presented, and the main contribution is clearly explained.", "essential_references_not_discussed": "Not that I could see.", "other_strengths_and_weaknesses": "Strengths:\n * Presenting a unified view of constant-curvature manifolds and a novel formulation for decision trees in Euclidean, Hyperbolic and Hyperspherical cases.\n * Formulation of product decision trees.\n * Ideas for how to apply the presented theoretical construct to real-world cases\n\nWeaknesses:\n * There is a lack of clarity regarding when the method is applicable and when we can expect results that are competitive with other methods. That is, while the method obviously works well on product spaces, it is not clear when those are useful approximations in real-world problems.", "other_comments_or_suggestions": "None.", "questions_for_authors": "* Can you provide a list of types of problems where the method is applicable per your experience? This would improve the ability to apply your method and might steer future research into extending this list.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lRruNDA4TZ", "reviewer": "Reviewer_SX2Z", "summary": "This paper presents a novel extension of Decision Trees (DTs) and Random Forests (RFs) to product manifolds, which are Cartesian products of hyperbolic, hyperspherical, and Euclidean spaces. The authors introduce an angular reformulation of DTs that respects the geometry of product manifolds, resulting in geodesically convex, maximum-margin, and composable splits. The proposed method generalizes existing Euclidean and hyperbolic algorithms and introduces new hyperspherical DT algorithms. The researchers evaluate their approach on various tasks, including classification, regression, and link prediction, using synthetic data, graph embeddings, mixed-curvature variational autoencoder latent spaces, and empirical data. Their product RFs demonstrate strong performance, ranking first in 25 out of 57 benchmarks and placing in the top 2 for 46 out of 57 when compared to seven other classifiers.\n\n## update after rebuttal\n\nThank you to the authors for their responses. I believe it is essential to incorporate this discussion into the next revision of the paper.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank the reviewer for their attention to our manuscript, and in particular for praising the strong performance of our method, the thoroughness of our benchmarks, and the “clear and convincing evidence” of our claims.\n\n**Motivation for a tree-based method and our contribution**\n\nTree-based methods are extremely popular for Euclidean machine learning and have recently become popular in hyperbolic spaces as well \\[1,2\\]. We believe tree-based methods can be promising as lightweight, simple, easy to train, and interpretable probes on top of learned neural representations or standalone classifiers/regressors in their own right. Additionally, we address a gap in the literature by providing a detailed survey of product space-valued datasets and predictors:\n- We describe and benchmark an end-to-end product space ML pipeline\n- We describe many variants of existing models (e.g. kappa-GCN regression) for the first time\n\nWe also note \\[3\\] is a well-regarded paper that generalizes Euclidean/hyperbolic GCNs to all constant curvatures and product manifolds.\n\n**Theoretical justification for splits**\n\nWe thank the reviewer for pointing out that our paper’s exposition on geodesic convexity (GC) and why it matters for classifiers is too short. We initially believed that it would suffice to defer to other papers describing linear classifiers in non-Euclidean spaces (e.g. \\[3\\], \\[4\\], and especially \\[1\\], which explicitly argues on behalf of GC). We agree a brief but explicit discussion of the geometry of linear splits and its relationship to GC is needed. If accepted, we will use our extra space to expand on this between Sections 2.1 and 2.2 as follows:\n- $S \\\\subset \\\\mathcal{M}$ is **geodesically convex** if any $p, q \\\\in S$ implies the geodesic $\\\\gamma\\_{p,q} \\\\subseteq S$. We take this for granted in Euclidean linear classifiers, as any separating hyperplane preserves this property. However, this is not automatically true in non-Euclidean spaces, which could be a source of misgeneralization for models \\[1,4\\].\n- Building on \\[5\\] Ch. 3.1, if $\\\\mathcal{M}$ is partitioned by GC $G$ into $A$ and $B$, then $A$ and $B$ are also GC. By contradiction, if $\\\\gamma\\_{u,v}$ (where $u, v \\\\in A$) were to cross into $B$, it must take a path $A \\\\to G \\\\to B \\\\to G \\\\to A$. This implies the existence of $p, q \\\\in G$ such that $\\\\gamma\\_{p,q}$ enters $B$, implying $\\\\gamma\\_{p,q} \\\\nsubseteq G$, violating the GC property of $G$.\n- \\[6\\] defines a linear split in product manifolds as $l\\_\\\\mathbf{w}^\\\\mathcal{P} \\= \\\\operatorname{sign}(\\\\langle w\\_\\\\mathbb{E}, x\\_\\\\mathbb{E} + \\\\alpha\\_\\\\mathbb{S} \\\\sin^{-1}( \\\\langle w\\_\\\\mathbb{S}, x\\_\\\\mathbb{S} \\\\rangle ) + \\\\alpha\\_\\\\mathbb{H} \\\\sinh^{-1} ( \\\\langle w\\_\\\\mathbb{H}, x\\_\\\\mathbb{H} \\\\rangle\\_\\\\mathcal{L} + b),$ where $w\\_\\\\mathcal{M}$ means the restriction of $w \\\\in \\\\mathcal{P}$ to some submanifold $\\\\mathcal{M}$.\n- Under our angular reformulation, $w$ is sparse in all but two dimensions lying in the same submanifold, with no bias term; thus, the split simplifies to $\\\\langle w\\_\\\\mathcal{M}, x\\_\\\\mathcal{M} \\rangle \\= x\\_0\\\\cos(\\\\theta) \\- x\\_d\\\\sin(\\\\theta),$ where $\\\\theta$ is our splitting angle, and $d$ is the dimension along which we split. By restricting our attention to two dimensions within a single component manifold at a time, our angular reformulation approach bypasses almost all the complexity of performing geodesic splits in product manifolds.\n\n**Clarifying experimental details**\n\n- We show means with 95% confidence intervals in our tables\n- We use 10 trials for each experiment\n- Seeds are shared across RFs for splitting, but each trial uses a different seed. For instance, in figure 3:\n  - Seeds 0-9 are used for K=-4, 10-19 for K=-2, 20-29 for K=-1, etc\n\n**Low dimensionality**\n\nWe benchmark lower-dimensional embeddings, as these are more challenging and rely more heavily on the classifier’s ability to match the geometry of the space. However, we agree that it is worth evaluating higher-dimensional datasets; therefore, consistent with \\[3\\] we have benchmarked manifolds with 16 total dimensions at [https://postimg.cc/MvR5R8Pp](https://postimg.cc/MvR5R8Pp), which we will add to the Appendix.\n\n**Tangent vector space**\n- Thank you, we have fixed this.\n\n**Missing neural net benchmarks**\n\nThank you for pointing this out, we have a full suite of benchmarks at [https://postimg.cc/gallery/ZbfzMzR](https://postimg.cc/gallery/ZbfzMzR).\nWe will add the full benchmarks to the Appendix in the camera-ready.\n\n**References:**\n\n\\[1\\] Chlenski et al (2024). Fast Hyperboloid Decision Tree Algorithms.\n\n\\[2\\] Doorenbos et al (2024). Hyperbolic Random Forests.\n\n\\[3\\] Bachmann et al (2020). Constant Curvature Graph Convolutional Networks.\n\n\\[4\\] Cho et al (2018). Large-Margin Classification in Hyperbolic Space.\n\n\\[5\\] Urdiste (1994). Convex Functions and Optimization Methods on Riemannian Manifolds.\n\n\\[6\\] Tabaghi et al (2022). Linear Classifiers in Product Space Forms.", "claims_and_evidence": "Overall, the claims made in the submission supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method is a tree-based approach for classification and regression tasks on the product space. While the rationale for choosing a tree-based method in product space is not well-motivated, the author does compare its performance with other non-tree-based methods. Various types of data are considered in the experiments, but the embedding space for the data is relatively low.", "theoretical_claims": "Sec 3, line 191-192, the claim \"we\nobserve that homogenous hyperplanes are geodesically convex in any constant-curvature manifold;\" it is not proved or cited. Moreover, the concept \"geodesically convex\" is not defined and the importance of  \"geodesically convex\" is not clear.", "experimental_designs_or_analyses": "I may have overlooked some details. I assume the numbers and charts indicate the mean scores and 95% confidence intervals as stated in lines 271-273. How many runs has the author conducted for each experiment? I see in line 926 that \"we also employ the same random seed for all RF models.\" Is that same set of seeds applied to all experiments to compute the mean accuracy and its confidence interval?\n\nAdditionally, the results from Product RF and Ambient RF are not statistically significantly different in most cases.\n\nThe performance of DTs and RFs is sensitive to the choice of hyperparameters. It is not clear how the hyperparameters are chosen (section E.2). And as long as the method for choosing hyperparameters for different RFs is the same, it is not necessary to \"set all DT and RF hyperparameters identically\" (line 912).", "relation_to_broader_scientific_literature": "The proposed algorithms seem to be straightforward extensions of some exsiting work, combining the classification method on the product space [Tabaghi et al. Linear classifiers in product space forms.] with the HyperDT framework [Chlenski et al. Fast hyperboloid decision tree algorithms.].\n\nThe author's reformulation of the decision tree in Euclidean space (Section 3.1) seems unnecessary and introduces extra computations.\n\nOverall, the method appears somewhat incremental and does not offer significant novelty.", "essential_references_not_discussed": "Please refer to any references I have mentioned in the answers to other questions. Note that some may have already been cited in the submission.", "other_strengths_and_weaknesses": "Weakness:\n\nIn this paper, the author use product space as representation space, but in real world, it is hard to choose the optimal signiture of the product space. Moreover authors restrict to work on product space with small (1, 2 and 4) dimension component manifold and the dimension of the product space is also relatively small (< 10), which further limit the paper.", "other_comments_or_suggestions": "In Eq.1, $x'$ as a tangent vector should lie in the space $R^d$.", "questions_for_authors": "line 976, \"Table 6: A summary of the neural models benchmarked in our work\", where is the results from tangent/ambient MLP/GNN?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MkN2XJ3hA0", "reviewer": "Reviewer_3Gbp", "summary": "The paper introduces mixed-curvature decision trees (DTs) and random forests (RFs), which can be used to analyse data living on  product manifolds: combinations of Euclidean, hyperspherical and hyperbolic spaces, allowing for heterogeneous curvature. DTs are reformulated using angles to respect the manifold geometry, ensuring that splits are geodesically convex, maximum margin and composable. The methods are evaluated on classification, regression and link-prediction tasks, where they show strong empirical performance. They are also used to analyse graph embeddings and latent spaces from variational autoencoders.\n\n**Update after rebuttal**. The authors clarified that the work is downstream of signature selection. I broadly buy that generalising DTs and RFs to mixed curvature product manifolds might be novel and interesting. I don't think that statistically significant experimental gains across the board are crucial for acceptance, so will stick with my current score. I defer to other reviewers on whether there are problems with the theoretical analysis.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful attention, and are grateful for their favorable assessment of our “**well-written**, clear, convincing text”, our core idea being “**novel and interesting**,” and our method’s potential to be an “**effective, interpretable** tool.” \n\n**Relationship to signature selection and embedding generation.**  \nWe view signature selection and embedding generation as a complementary avenue of work and have tried to keep our discussion focused on classification and regression with existing embeddings. Nonetheless, as these are all parts of one process, we have tried to model simple but realistic end-to-end product manifold learning pipelines, starting e.g. from pairwise distances (for graph benchmarks) or feature tables.  \nMany other papers emphasize the first half of the pipeline—signature selection and embedding generation—and deemphasize approaches to classification and regression. Our work helps complete the picture by expanding and improving ways to use these embeddings once they are generated. We believe this division of focus allows for directed progress in each area and drives the overall field of non-Euclidean machine learning forward. We are optimistic that advances in either of these subproblems can be combined, e.g. by using mixed-curvature decision trees to analyze state-of-the-art product embeddings.\n\n**Algorithm 1 in main body.**  \nWe agree that it would be better to include Algorithm 1 in the main body of the paper. If accepted, we will use the extra page in the camera-ready to include this.\n\n**Signature selection.**  \nWe selected signatures as follows:\n\n- For Gaussians on single manifolds, we try a range of curvatures; on product manifolds, we try signatures with curvatures in $\\\\{-1, 0, 1\\\\}$\n- For graphs, we grid-search over signatures and use the one with the lowest embedding distortion  \n- For VAEs, we follow \\[1\\]  \n- For empirical datasets, the signature is determined by the specific problem.\n\nSelecting a signature is combinatorially difficult because the number of components, and the dimensionality of each component, must be selected. Common heuristics for searching over signatures include:\n\n* The curvature parameter can be learned smoothly in the range $\\[-\\\\infty, \\\\infty\\]$ by using stenographically-projected manifolds, as in mixed-curvature VAE  \n* Signatures can be built up component by component according to a greedy algorithm, as in \\[4\\]  \n* Heuristic-guided Bayesian optimization \\[3\\] is a more principled and sophisticated approach to this  \n* In general, embedding is reasonably fast; training VAEs is slower\n\nRegarding our claim that “product manifolds are not able to represent all patterns in data,” the authors of \\[1\\] make a *theoretical* claim about product manifold limitations, but provide no empirical examples. It is reasonable to speculate that complex graphs with heterogeneous curvature might fall into this category, in which case there is a pragmatic question to ask about fidelity-complexity tradeoffs in the choice of representation geometries.\n\n**Comparison to GCNs.**  \nOne way of using a graph’s adjacency matrix is to generate embeddings via metric learning, as in \\[2\\]; another is simply to use GCNs using some set of features. These are not mutually exclusive: in fact, generating metric embeddings is roughly comparable to pretraining a GCN on link prediction. A promising future direction of work could involve using our decision trees as lightweight, easily trainable, and modestly interpretable probes on representations learned by pretraining GCNs. \n\nIn general, our benchmarks did not generally favor GCNs even in situations where a known graph adjacency matrix was introduced; however, we appreciate that GCNs are in principle capable of substantially more complicated workflows than the ones that we benchmarked. We include the comment about tradeoffs to reflect the geometric deep learning community’s preference for GCNs on complex real-world tasks more than as a reflection of what we see in our own benchmarks: in particular, $\\\\kappa$-GCNs are never the best-performing method in our own Table 2\\. \n\n**References.**  \n\\[1\\] Borde and Kratsios (2023). Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometries.  \n\\[2\\] Gu et al (2019). Learning Mixed-Curvature Representations in Product Spaces.  \n\\[3\\] Borde et al (2023). Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization.  \n\\[4\\] Tabaghi et al (2022). Linear Classifiers in Product Space Forms.", "claims_and_evidence": "The authors convincingly argue that DTs and RFs can be generalised to all constant-curvature manifolds by representing data and splits as angles in two-dimensional subspaces. They show competitive performance on a wide range of benchmarks, including classification, regression and link prediction. The datasets include synthetic data, graph-embeddings, mixed-curvature VAE latent space, and empirical datasets. Their methods offer a nice interpolation between linear classifiers (ineffective but interpretable) and neural networks (effective but uninterpretable).", "methods_and_evaluation_criteria": "The methods and evaluation criteria make sense; see above.", "theoretical_claims": "The paper provides a proof for the theoretical claim that Euclidean DTs using the angular reformulation are equivalent to classical decision trees that perform thresholding in the basis dimensions (see App C). This demonstrates that, while the presentation is unconventional, it is mathematically equivalent to traditional methods. I didn’t check the maths in detail, but this seems reasonable.", "experimental_designs_or_analyses": "I think the experimental designs are reasonable; see above.", "relation_to_broader_scientific_literature": "The paper builds on previous work on machine learning in product manifolds, including Tabaghi et al. (2021, 2024) and Cho et al (2023). This includes linear classifiers, perceptrons, SVMs, PCAs and Transformers. It generalises work by Doorenbos (2023) and Chlenski (2024) on RFs in hyperbolic space, by permitting mixed curvature. Product manifolds have also been used to embed graphs (Wang 2021) and variously in biology.", "essential_references_not_discussed": "The references seem reasonable.", "other_strengths_and_weaknesses": "**Strengths**:\n- Well-written, clear, convincing text\n- The core idea – generalising DTs and RFs to mixed curvature product manifolds – seems novel and interesting. Whilst I wouldn’t expect this to ultimately compete with SOTA deep learning, I buy that this might be an effective, interpretable tool in certain simple cases.\n- Benchmarking on classification, regression and link prediction tasks on synthetic data, graph embeddings, mixed-curvature VAE latent spaces and empirical data is pretty exhaustive\n\n**Weaknesses**:\n- As the authors acknowledge, they view their work as downstream of signature selection and embedding generation. This avoids the fact that good product manifold embeddings might be difficult to find or expensive, and might limit the practicality of their algorithms.", "other_comments_or_suggestions": "I understand that space is limited, but it feels a shame to have Alg 1 in the appendix. I wonder if a shorter version could be included in the main body.", "questions_for_authors": "- Can you comment more on signature selection? How difficult is this in practice? You acknowledge that ‘product manifolds are not able to represent all patterns in data’, referencing Borde & Kratsios (2023). What types of data patterns or structures are difficult for product manifolds to capture, and how might these limitations be addressed in future work?\n- You mention ‘tradeoffs between DTs and RFs and other high-performing methods, especially graph neural networks when topologies are known’. In what specific scenarios do GNNs outperform your method? What characteristics of the data or task contribute to this difference in performance? (I appreciate that this is a big question...)", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper investigates decision trees and random forests across geometries of multiple manifolds: Euclidean, hyperbolic, and hyperspherical. This paper received highly mixed reviews, with two reviewers arguing for acceptance and two for rejection. The positive reviewers find the idea novel, the paper well written, and the results good. The negative reviewers find the benefit of product manifold DT over single-manifold DT unclear and find that the paper provides insufficient theoretical foundations. The AC does not agree with the first concern. The authors provide a great deal of additional numbers in the rebuttal, highlighting the improvements over single-manifold DTs. Ambient DTs seem to remain a strong baseline, but the proposed approach is overall better, only not better in all 20 experiments (16/20, 8 significant). The AC does agree with the second point of the reviewers. The submission lacks insights into why geodesic splits partition manifolds into geodesically convex subspaces, along with the required equations. The authors provide the required insights in the rebuttal however. The point that the Euclidean DTs bring unnecessary steps is valid but deemed small and does unify the manifolds. As such, the AC believes that the major concerns by the negative reviewers are either not a big problem or have been addressed in the rebuttal. Hence the AC recommends weak accept.
Decision: Accept (poster)