Hyperbolic Embeddings of Supervised Models
Authors: Richard Nock
Ehsan Amid
Frank Nielsen
Alexander Soen
Manfred K Warmuth
Venue: NeurIPS 2024 poster (2024)
Keywords: Hyperbolic geometry
supervised model embedding
decision trees
boosting
Primary area: learning_theory
TL;DR: A full-fledged solution to embed supervised *models* in hyperbolic geometry, and more

Abstract: Models of hyperbolic geometry have been successfully used in ML for two main tasks: embedding *models* in unsupervised learning (*e.g.* hierarchies) and embedding *data*. 
To our knowledge, there are no approaches that provide embeddings for supervised models; even when hyperbolic geometry provides convenient properties for expressing popular hypothesis classes, such as decision trees (and ensembles).
In this paper, we propose a full-fledged solution to the problem in three independent contributions. The first linking the theory of losses for class probability estimation to hyperbolic embeddings in Poincar\'e disk model. The second resolving an issue for a clean, unambiguous embedding of (ensembles of) decision trees in this model. The third showing how to smoothly tweak the Poincar\'e hyperbolic distance to improve its encoding and visualization properties near the border of the disk, a crucial region for our application, while keeping hyperbolicity.
This last step has substantial independent interest as it is grounded in a generalization of Leibniz-Newton's fundamental Theorem of calculus.

Hyperbolic Embeddings of Supervised Models
Richard Nock Google Research ri richardnockkgoggle.coom
Ehsan Amid Google Deep eami samiogg com
Frank Nielsen Sony CS Labs, Inc. frank. nielsendacm org
Alexander Soen RIKEN AIP
Manfred K. Warmuth
Google Research manfred@google... com
Australian National University
al alexander.c Seanddersseenanll soen@anu. edu.. au
Abstract
Models of hyperbolic geometry have been successfully used in ML for two main tasks: embedding models unsupervised learning (e.g. hierarchies) and embedding data. To our knowledge, there are no approaches that provide embeddings for supervised models; even when hyperbolic geometry provides convenient properties for expressing popular hypothesis classes, such as decision trees (and ensembles). In this paper, we propose full-fledge full-fledge solution to the problem in three independent contributions. The first linking the theory of losses for class probability estimation to hyperbolic embeddings in Poincaré disk model. The second resolving an issue for an easily interpretable, unambiguous embedding of (ensembles of) decision trees in this model. The third showing how to smoothly tweak the Poincaré hyperbolic distance improve its encoding and visualization properties near the border of the disk, a crucial region for our application, while keeping hyperbolicity. This last step has substantial independent interest as is grounded in a generalization Seibniz-Newton's 's fundamental Theorem of calculus.
Introduction
Models of hyperbolic geometry have been successfully used to embed hierarchies, i.e. tree-based structures [14] 20 38 36). Through the property of low-distortion hyperbolic embeddings (30), symbolic (hierarchies) and numeric (quality metrics) properties of unsupervised learning models can be represented in an accurate and interpretable manner [29]. When comes supervised learning. the current trend involves embedding data in a hyperbolic space, with models trained on the now hyperbolic data [9 15 III. It is important note that none of these supervised methods embed models. Indeed, the focus of the prior literature is to learn supervised models from hyperbolic data, rather than representing supervised models in hyperbolic geometry. This unsupervised usage described, where the (tree-based) structural properties of unsupervised models directly exploited for good embeddings. This hints at benefits which can be used in the supervised case, especially for popular supervised models which are (structurally) tree-based.
stark contrast the
Our paper proposes full-fiedged solution for this problem, in three independent contributions.
The first focuses on the numerical part (quality metrics) of the embedding and its link with supervised losses: to provide natural way of embedding classification and the confidence of prediction, show a link between training with the log-loss (posterior estimation) or logistic loss (real-valued classification) and hyperbolic distances computation in the Poincaré disk.
The second focuses on the symbolic part (hierarchies) of the embeddings for a popular kind of supervised of models: decision trees (and ensembles). Unlike for unsupervised learning, we show
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

that getting an unambiguous embedding of decision tree requires post-processing the model. Our solution extracts its monotonic sub-tree via new class of supervised models we introduce, monotonic decision trees. This is also convenient for explainability purposes [32
The third and most technical one focuses on visualization and the accuracy of the numerical encoding in the Poincaré disk model. The more accurate / confident is a supervised prediction, the closer to the border of the disk it is represented. Thus, the best models will have their best predictions squeezed close to the border. In addition to being suboptimal for visualization, this region is also documented
for being numerically error-prone for hyperbolic distance computation [29]. Two general fixes currently exist: encoding values with (exponentially) more bits or utilizing a trick from Riemannian geometry [19]. As neither is satisfactory for our usage, we propose a third, principled route: like
other distances, Poincaré distance is integral. We generalize Leibniz-Newtong 's fundamental Theorem of calculus using a generalization of classical arithmetic [22]. This generalized "tempered" integral provides parameter to smoothly alter the properties of the "'classical" integral. When defining the Poincarè distance, the tempering controls the hyperbolic constant of the embedding whilst also improving the visualization and numeric accuracy of the embedded models. The generalization of integrals to distances has independent interest as many application in ML rely on integral based distances and distortions, i.e., Bregman divergences, fa f-divergences, integral probability metrics, etc.
Experiments are provided on readily available domains, and all proofs, additional results and addi- tional experiments are given in an Appendix.
2 Related work
Models of hyperbolic geometry have been mainly useful to embed hierarchies, i.e. tree-based
structures [14] 20 38 [6]. with a sustained emphasis coding size and numerical accuracy [19 29
37 In unsupervised learning and clustering, some applications have sought simple representation
of data on the form of tree or via hyperbolic projections 77 6 17 34
Approaches dealing with
supervised learning assumes the data lies in a hyperbolic space: the output visualized is primarily an embedding of the data itself, with additional details linked to classification of secondary importance, either support vector machines (9), neural nets, logistic regression [15), or (ensembles of) decision trees [Σ. We insist the fact that the aforementioned methods do not represent the models in the hyperbolic space, even when those models tree-shaped. The question of embedding classifiers is potentially important improve the state of the art visualization: in the case of decision trees, popular packages stick to a topological layer (the tree graph) to which various additional information about classification are superimposed but without principled links to the "embedding" 7
3 Basic definitions
Training a supervised model starts with a set (sample) of examples, each of which is a pair (æz, v). where X (a domain) and y e {-1, 1} (labels or classes). A decision tree (DT) consists of binary rooted tree H, where at each internal nodes, the two outgoing ares define a Boolean test over observation variables (see Figure [] center, for an example); N(H) is the set of nodes of H. A(H) = N(H) is the set of leaf nodes of H.
The log-loss is best introduced in the context of the theory of losses for class probability estimation
(CPE) 28
We follow the notations of [18] : A CPE loss function, & y [0. - I, is
Ely.p) p)
ly 1] (p) [y = -1 11--__(()))
where [.] are Iverson brackets 16 Functions &,, f- are called partial losses. A CPE loss is symmetric when (p) = l p). Vp e [0, 1]. The log-loss is a symmetric CPE loss with (po(p)) = - log(p). The goal of learning using CPE loss is to optimize the Bayes risk, L(p) = inf, Ey-p,(Y r). In the case of the log-loss, is the criterion used learn DT's in C4.5 (66
(p)
-p- loloog p (1 p) log(1 - pp).
Models like DTs predict empirical posterior Pr|y 1|X] at the leaves: for observation
reaching leaf X, the posterior prediction is the local relative proportion of positive examples in leaf X as estimated by the training sample (noted px)) There exists a duality between CPE classification
"See for instance https: http://ttttuuuucccprr///////////

---

Both Ni anddig are embedded at the center
xin:) 5 61
Zen Peigle E
that www.h Cocti
KKn)
P
1-22
(U5)
[471
pin)
25
3G
47
(19)
410 #7
aha Naggeraaa
fg
#9
99
#117
pin)) K(h))) -1
Dectiion Tiee
Manstosic Decisio
Figure 1: Left pane: subtree of a decision tree (DT) (colors red, green denote the majority class in each node, grey random posterior, tests at ares not shown, 14 is the root, leaves shown) and its embedding following the simple recipe in (3): it is impossible to tell f2 from n, and the tree depicted in BB is not faithful representation of the DT nor of any of its subtrees. Right pane: small DT learned on UCI abalone (left) and its corresponding monotonic decision tree (MDT, right) learned using GETMDT. In each node, the veal-valued prediction (p)aa(p)) is indicated, also in color. Observe that, indeed, H does not grant path path-monotonic classification but H' does (Definition (4.1). In H1', some nodes have outdegree I; also, internal node #6 in the DT, whose prediction worse than its parent, disappears in H' One arc in H' is represented with double width as its Boolean test aggregates both tests it takes go from #3 to # 10 in H. Observations that would be classified by leaf # 11 (resp. #5, resp. #9) in H are now classified by internal node #3 (resp. #2, resp. #4) in H', but predicted labels are the same for H and H' and so the accuracy of H and H' are the same.
and the velled classification setting familiar to e.g. deep learning: optimizing Bayes risk for the posterior is equivalent to minimizing its convex surrogate using the canonical link of the loss to compute real classification [24] Theorem 1J. The canonical link of the log-loss, Venu [0, 1] R is
the famed inverse sigmoid, which has a very convenient form for our purpose,
PPP 1-p)
[rro((p) = log (1=/)
Vloa(p) = log
=
T= 1p- 1/.
(3)
Notably, the absolute value [P)o is a confidence with the sign giving the predicted class. In the case of the log-loss, the convex surrogate is hugely popular in ML: it is the logistic loss.
We now introduce concepts from hyperbolic geometry, particularly those from the Poincaré disk model. Our definitions are simplified, a detailed account can be found in 20 The distance function of a metric space is r-hyperbolic for some т >/ iff for any three geodesic curves Y ,,y?? y linking three points, there exists such that max; y²) & т, where d(z, y) = infz'e d(z, z'). A small hyperbolic constant guarantees thin triangles and embeddings of trees with good properties. The Poincaré disk model, B (negative curvature, 1) is popular model of hyperbolic geometry with the hyperbolic distance between the origin and some = E with Euclidean norm r = | being:
dz = log 1 1-r
(4)
4 Posterior embedding in Poincaré disk and clean embeddings for DTs
Node embedding We exploit the similarity between log-loss confidences (p)) and hyperbolic distances dis dz, 0) when embedding classification models, similarity which is obvious from (3) and
(4) In 33) we see that if 0, the prediction is as bad as a fair coin's. As edges closer to 1, prediction approaches maximum confidence. The connection with the Poincaré distance (4) is
immediate: natural embedding of posterior prediction p is then a point in the Poincaré disk at radius r = ||z|| from the origin 33). The origin of Poincaré disk represents the worst possible confidence. As each leaf X of DT H corresponds a posterior pt the leaves can be embedded as described. In addition, all nodes v N(H) in the tree also have corresponding posteriors, and thus can also be embedded identically to leaf nodes. also good idea to embed them because, as classical DT induction algorithms first proceed by top-down induction, any node in DT was a leaf at some point during training, and has corresponding posterior. This apparently clean picture for node embedding however becomes messy soon as we step to embedding full trees.
3

---

No clean embedding for a full DT A simple counterexample indeed demonstrates that clean embedding of decision is substantially trickier, see Figure[[[eftt) this pathological example, some distinct nodes (resp. ares) of the DT H are embedded in the same node (resp. edge) in B: the depiction has no subgraph relationship to H and some embedded nodes cannot be distinguished between each other without additional information.
Getting a clean embedding: Monotonic Decision Trees To introduce our fix, we define three broad objectives for the embedding in B of DTs
Embedding objective
(A) Embed a tree-based model from H in B, which defines an injective mapping of the nodes of !l and induces subtree of H with each edge in lB corresponding to a path in H; (B) Locally, each node H gets embedded to some 2v such that Tv is close to = 33:
(C) Globally, the whole embedding remains convenient and readable to compare, in terms of confidence in classification, different subtrees in the same tree, or even between different trees.
As we shall see later in this Section, this agenda also carries embedding ensembles of DTs, thus including their leveraging coefficients, exploiting properties of boosting algorithms. For now, our solution for embedding a single DT that can satisfy (A-C) is simple in principle:
Embed the monotonic classification part of a DT,
meaning for each path from the root to leaf in H, we only embed the subsequence of nodes whose absolute confidences strictly increasing. To do so, replace the DT by a "close" path-monotonic approximation model using new class of models we call Monotonic Decision Trees (MDT).
Definition 4.1. A Monotonic Decision Tree (MDT) a rooted tree with a Boolean test labeling each and real- prediction each node. Any sequence of nodes in path starting from the root is strictly monotonically increasing in absolute confidence. At any internal node, no two Boolean tests its outgoing ares be simultaneously satisfied. The classification of an observation is obtained by the bottom-most node's s prediction reachable from the root.
Algorithm 1 GETMDT(2, b,1,1)
Input: Node v N(H) (H =T),,, Boolcan test b. Node vf E N(H') (H' =MDT being build from H), forbidden posteriors
Ic []:
:: ifve E A(H then
2: if P5 E then
3:
V TAGDTLFAF()):
I1 tags v N(H") with info from leaf y M(H)
4: else
5: e" H' NEWEOD)( NEWARCI/ b,²");
will be new leaf in Hf' If adds are -bb 1/ in Hf'
6:
endif
8:
else
9: if
p 4 then
10: 1 H' NEWNODE(V):
1/" = new internal node in H I/ adds are -b 1/ in /I T
NEWARCC 1));
12:
[min(pr. PE maxint,.1 25:]
13:
// 1²:
14:
b.c true;
15:
else
16:
1aw 11 1/ v/: ba.
v yiclds no change in
17:
endif
18:
GETMDT(y.lefchild, b (eTEET((eftchiid) //.. Jeen):
19:
GETMDTiv-ightchidd bis (iTEss(rigttchiid), // huw
20:
endif
We introduce algorithm, GETMDT, which takes input DT Il and outputs an MDT H', which approximates H via the following key property:
(M) For any observation x X, the prediction H'(x) is equal to the prediction in the path followed by in H of its deepest node in the strictly monotonic subsequence starting from the root of H. Figure I (right) presents an example of MDT H' that would be built for some DT (left) and satisfying (M) (unless observations have missing values, H' is unique). Figure I adopts some additional conventions to ease parsing of H', (D1) and (D2):
(D1) Some internal nodes of H' are also tagged with labels corresponding to the leaves of H. If

---

node in H' is tagged with a label of one leaf À of H, it indicates that examples reaching À in the original H are being classified by H'ss tagged node:
(D2) Ares in H' have a width proportional to the number of boolean tests it takes to reach its tail from its head in H. large width thus indicates a long path in H improve classification confidence.
To produce the MDT H' from DT H with Algorithm 1 after having initialized it to root = single leaf, we just run
GETMDT(roo((H) H), true, root( root(H)), [p Priua)
(we let pt = min(p 1 Pi Poiaa = max 1 Pitce J). Upon finishing, the tree rooted at Loce root( H²) is the MDT sought.
We complete the description of GETMDT: When leaf of H does not have sufficient confidence and ends up being mapped to an internal node of the MDT H', TAGDTLEAF is the procedure that tags this internal node with information from the leaf (see (D1) above). We consider tag being just the name of the leaf (Figure [ leaves #5, 9, 11), but other conventions can be adopted. The other two methods we use grow MDT H' by creating new node via NEWNODE and adding new are between existing nodes via NEWARC. We easily check the following.
Theorem 1. H' built by GETMDT satisfies (M) with respect to DT H.
Property (M) is crucial to keep the predictions of the DT and the MDT close to each other: to each node of the MDT indeed corresponds node in the DT with the same posterior prediction. Because the set of nodes of H' corresponds to a subset of nodes in H, predictions can locally change for some observations. This phenomenon is limited by two factors: (i) the set of leaves of the MDT corresponds to set of leaves in the DT (thus, predictions do not change for the related observations), (ii) more often than not in our experiments, is only the confidence that changes, not the sign of the prediction. which means that accuracies of the DT and its corresponding MDT are close to each other (Section 6). It is also worth remarking that the MDT H' always has the same depth the DT H. Finally, any pruning of H is subtree of H and its corresponding MDT is a subtree of the MDT of H. The aforementioned troubles to embed the DT in the Poincaré disk considering (A-C) do not exist anymore for the MDT because the best embeddings necessarily have all arcs going outwards in the Poincaré disk. The algorithm we use to embed the MDT in Poincaré disk a simple alteration of Sarkar's 's embedding [30 which, because of the lack of space, we defer to the Appendix (Section C.
The quality of the embedding is obtained using an embedding error that takes into account all nodes:
|| ds z,)1
p( HH) + 100 (00-E
(%),
(5)
where tha refers to the relevant tho (pt) in (and z, BB is its embedding in B). Two example final representations are in Figure 2
Notice the small errors p in both cases.
Remark 1. Using particular boosting algorithm to boost the logistic loss, can well represent the leveraging coefficients of a DT/MDT in a boosted combination, following a convention sketched in Figure 2 and explained at length, along with the boosting algorithm and its properties, in the Appendix (Section C.IX).
5 Smoothly altering integrals: T-calculus and the t-self of Poincaré disk
It is apparent from Figure 2 (right, lowest red pentagon), that when utilizing standard hyperbolic distances da (z, 0) ( the best MDT nodes are embedded close to the border. In addition to obviously not being great for visualization, these nodes are at risk of having numeric error "push" the nodes to the border O thereby giving false depiction of infinite confidence. In this section, we provide alternative distances to prevent this phenomena. First, we quantify the numerical risk, which has been defined by the critical region where high numeric error can occur 19 29
Definition 5.1. A point x E BB said k-close to boundary AB if |>| = 1 1- 10 k. It is said encodable
iff [ < 1 in machine encoding (it is not "pushed the boundary")
Machine encoding constrains the maximum possible ke in the fouble-precisiie floating-point rep- resentation (Float64), k x 16 [19.. In the case of Poincaré disk, the maximal distance de from the
5

---

node embedding error (posterior)
Se 6.42
Hoot not from the cemer (A) Patr-ppuchaaaee that look more
to (A)
(-7)
"diffrron" from
rito of radi lover agiing confficient ensemole
Tree are libe with feature
aboclute value)
Page Values (B)
root (oquare node)
red I green majority class
Specatic purchessu"
specia
noce going outwards (3))
add (C)
Monotonic Decisioe Tree
Figure 2: Two example MDTs learned on UCI domain online_ aaline_shoppers. intrrssiiteetiioo showing embedding error p ( Left: key parts the embedding. Right: annotations used in Experiments,
Section 6
origin 0 that this authorizes (before numerical error "warps" point the border) is small affine order in k [9]:
d log(2) + log( 10) 0(10**),
Hence, in practice, only ball of radius d4 38 around the origin can be accurately represented. This deep contrast with Euclidean representation, where dg = (2(2)). We now provide a principled solution changing d, while keeping hyperbolicity, which relies crucial generalization of Leibniz-Newton' s fundamental Theorem of calculus.
5.1 T-calculus
We let [n] = (1,2. n for Nzo- At the core of our generalization is the replacement of the addition by the tempered (or t-)addition [22] & 2' = + 2' (1 t)zz', in the notion of Riemann sum. The additional term is scaled saddle point curve 22', positive when the sign of and 2' agree and negative otherwise. Hereafter, f is defined on an interval [a. bj. We define generalization of Riemann integration to t-algebra (see (2) for a preliminary development which does not provide the generalization of Leitniz-Newton's fundamental Theorem of calculus) and for this objective, given an interval [a, 6] and a division A of this interval using reals To =
... In-1 In b,
we define the Riemann f-sum over [a, b] using A, for set of tags E {& E [x;-1, 2;,,,, ai]l
(
(n) = (D)) x.-1) - f(E))
(f) the classical Riemann summation). Let s(A) max; D; I;-1 denote the step of division A. The conditions for t-Riemann integration are the same as for = 1.
Definition 5.2. Fix L TR. A function t-(Riemann) integrable over [a. b) iff 3. R such that
Ve 0,38 O,Y division with s( A) < 6 and associated set of tags s (f) L s. When
this happens, we note
Lh f(z)d<x= = L.
(8)
The case Riemann integration, is denoted using classical notations. We now prove our first main results for this Section, namely the link between (-Riemann integration and Riemann integration.
Theorem 2. Any function is either t- t-Riemann integrable for all E R simultaneously for none. In the former case, we have the relationship: (log, (z1-i 1)/(1 t) for t. + and log, log)
(1) rb
3t (J" ((()da))))
f(u)deu
Vt E IR, with (2) log, exp
For example, if 0, we get + 101g f(u)d, exp Sh (u)du, which Volterra's integral [3 Theorem 5.5.11]. Theorem is important because it allows to compute any t- t-Riemann integral from

---

the classical (f = 1) case. Classical Riemann integration and derivation are fundamental inverse operations. The classical derivation is sometimes called "Euclidean derivative" 4 We now elicit the notion of derivative, which is the "inverse" of I-Riemann integration. Unsurprisingly. it generalizes Euclidean derivative. The Theorem stands as generalization of the classical fundamental Theorem of calculus.
a-dd Theorem 3. Let 2 Or z' T4(1-1)2 denote the tempered subtraction. When it exists, define
Dzf(z)
lim 5-0
5
Suppose f I-Riemann integrable. Then, the function F defined by F(z) = (S) f(u)ddu is such that D,F f. We call F a t-primitive of (which zeroes when = a) and D,F the t-derivative of F.
The function 9, (Figure 3) is key to many of our results; it has quite remarkable properties: in particular, it is strictly increasing for any e I. strictly concave for any t > strictly convex for any t < and such that sign(g, (2)( = sign( ( 22). E R. One can also note that Dzg,(2) 1. The Appendix proves these results (Sections C.I and C.II.
and proper- ties of I-calculus, naturally "percolate" through many properties known for classical integration and the fundamental theorem of calculus (among others, additivity, Chasles' relationship, monotonicity, the hyperbolic Pythagorean theorem, mean value theorem, chain rule, etc.).
Remark 2. The Appendix also proves many more results showing how Theorems 2 and 3
5.2 The t-self of Poincaré disk
Let us now put T-calculus to good use in our context. For a set endowed with function d x X +00 (e.g. Poincaré disk and the Poincaré hyperbolic distance), the t-self of is the set (implicitly) endowed with d = 9, o d. Before going further, let provide the elegant d( associated to Poincaré disk's t-self (with r 2 as in (4)):
1 (1+r)
dgee) (0, 2) = log, exp du (0, z) = logg
(1-;)
(10)
The following Lemma (shown in Appendix) demonstrates the usefulness of T-calculus to address the encoding problem in B.
Lemma 1. In Poincaré disk model, pick any increasing function g(k) > 0. For the choice t 1- f(k) where f(k) E IR is any function satisfying
log (1 + (())
& log(10) k,
(11)
f(k)
get in lieu of (6) the maximal de of (t) satisfying d g( (k), and the new hyperbolic constant ty of the I-self satisfies
Te = exp (J(K)))
(12)
f(k)
Since both log(1 + x)) x x o(x) and exp(x) - = x o(x) in a neighborhood of 0, we check that (11 and (1) get back to properties of the Poincaré model as f(k) - 0 [19]. We then have two cases: we can easily approach back the Euclidean de by picking f(k) > 0: choosing f(k) = gets there and still keep finite hyperbolic constant, albeit exponential in the former one. however we want to improve further the hyperbolic constant, we can pick some admissible f(k) < 0, but then [I] will constrain g j to very small value. Note that depending on the sign of fkk), the triangle incquality can still hold or be replaced by weaker version since we can show dgco (x. z) & dgit) (æ, y) + date (y. 2) + max(0, [kk) dgen (x, y) dgcn (y, 2) in IB())
Let us finally investigate how we can perform general embedding in the t-self B(4) of the Poincaré disk to be (i) more encoding-save and (ii) have points originally close to ²B substantially further away from the t-self ((4) border. We also add third constraint (iii) that the distortions must be fair w.r.t the original the Poincaré disk model embedding; i.e. if some z IB (e.g. the coordinate of a DT node) gets mapped to 2()) (()) (the t-self), then we request
dgas (z 0) dz
(13)
7

---

0.8
0.8
0.4
0.6
g.(2)
0.2
0.4
-2
0.2
0.2
-0.6
0.3-0.2-0.1
0.1 0.2 0.3
0.2
0.4
0.6
0.8
Figure 3: Left: plot of (²)=) for different values of (color map on the right bar), showing where it is convex concave. The case (91 (z) 2) is emphasized in red. Right: suppose = ||||| is the Euclidean norm of point in Poincaré disk Fix t E [0. (color map on the right bar). The plot gives the (Euclidean) norm (()) of point (*) in the t-self such that dgo (2 0) (z, 0)
Remark that even for r very close to we can have p()) substantially smaller (e.g. r() 0.8).
online wetTTeettoprrrsiineeeeo. intersion
MINTORstNta, eiprece
what
ond Wel.nnn,hopperr Extentsor
barduare 70)
-261
Table 1: Left pane: embedding in Poincaré disk of the MDTs corresponding to the DT's learned at the 1st (top row) and 10th (bottom row) boosting iteration, on four UCI domains. Stark differences emerge between these domains from the plots alone. Right pane: comparison between Poincaré disk embedding and its = 0 t-self an MDT learned on UCI online wawiiieesseeseeeeeeeeeee intention (top, boosting coefficients information not shown) and UCI hardware (bottom). The p (0.001, 0.999) isoline (rectangle) is barely distinguishable from 2B but is clearly distinct from (0). Note that in B(O), the center looks similar to a scaling (zoom) of IB; while near the border, the high nonlinearity of ((U) allows us to spot nodes that have high confidence / training accuracy (in orange) but can hardly be distinguished from the bulk of "just good" nodes in Note also the of red nodes in the Poincaré disk for j = 70 (rectangle) that mistakenly look aligned, but not in the t-self.
All three conditions point to a non-lincar embedding v, - IB()) The t-self offers a simple convenient solution with a trivial design for p;: compute (1) by a simple scaling of the Euclidean norm of z in B to ensure that (13) holds. Figure 3 (right) displays the corresponding relationship between = x and (+) ||x(() when t E [0, 1]. clearly achieving (ii) in addition to (iii. For (i),
even for f < very close to and close to ëB (r close to 1), the mapping can send z(1) substantially "back in" the t-self (3: for = 0.7 and r
- 10-4 i.e. k = 4 in Definition 5.1 we get p()) & 0.96, authorizing encoding with a less "risky" visible from Figure 3 (right) is that the distortion is low the center and then merely affine in wide region until near the heavy non-linear changes, typically when r > 0.8.
2. Two additional benefits from (13).
Remark 3. The Appendix details properties of the t-self of the Lorentz model hyperbolic geometry, another popular model (Section C.V.

---

6 Experiments
We summarize number of experiments (Table provided otherwise in extenso in the Appendix. In the top-down induction scheme for DT, the leaf chosen to be split is the heaviest non pure leaf, i.e. the one among the leaves containing both classes with the largest proportion of training examples. Induction is stopped when the tree reaches a fixed size, or when all leaves are pure, or when no further split allows decreasing the expected log-loss. We do not prune DTs. Our domains are public (Cf Appendix).
Poincaré embeddings of DTs/ MDTs See Figure 2 (left) for summary of the visualization. We remind that the center of the disk is "poor" classification (confidence 0), or, in the context of boosting, random classification. A striking observation is the sheer variety of patterns that are plainly obvious from our visualization but would otherwise be difficult to spot using classical tools. Some are common to all domains: as boosting iterations increase, low-depth tree nodes tend to converge to the center of the disk, indicating increased hardness in classification. This is the impact of well known effect of boosting, whose weight modifications make the problem "harder". Among fooainnddepnndeee patterns, highly imbalanced domains get a root predictably initially embedded "far" from the origin (online _twiiiiessppessseeeeeeee citpopper_intentiiinnn _iitentionn,,
supreeee_smem while more balanced domains get their roots embedded near the origin (abalone). Across the experiment, all trees have at most 200 nodes: while this clearly provides very good models after a large number of iterations on some domains, it is not enough to get good models after just few iterations others (anal canalaat saatdata_supreeeee sm)r Interpreting the hyperbolic embeddings can also be telling: consider Figure (right), which a MDT of the (j 3) boosted DT learned on online_ aetoppers_intentaae
Notice the low embedding error (5.4 5..4%). We see in (A) (magenta) that classes are more balanced at the root compared to previous trees (All DTs in Appendix, Table [V) because the root is closer to the center of the disk: thus, hard examples belong to both classes (no class becomes much harder to "learn" than the other). Two clearly distinct subtrees are associated to non-purchase patterns (red, A). The bottom-most subtree achieves very good confidence with several leaves close to the border: these are purchase patterns clearly distinct from purchase patterns (green nodes); the whole subtree dangles from the root via a test on feature Page Values (grey, B) achieving substantial boost in confidence; many nodes are way past posterior isoline p E {0.1, 0.9). Red subtree in (A) is lot closer to purchase patterns (C). One distinct pattern of purchase emerges, built from tests that always strictly increase its prediction's confidence (orange). The full rule achieves very high accuracy (leaf posterior nears 0.99).
Interest of the t-self for visualization As demonstrated by Table [ (right) and Appendix, the t-self is particularly useful to tell the best parts of a tree. Because also manages to "push back" the bulk of nodes from the border (B(() it displays the t-self might also be useful as a standard encoding (use (13) to access Poincaré disk quantities and embedding).
MDT vs DT for classification We traded the hyperbolic visualization of a DT bound to be poor for the visualization of its "monotonic part", an MDT. We have seen that the MDT visualization provides lots of clues about the DT as well. In terms of classification, the fact that the set of leaves of the MDT is a subset of the set of leaves of its DT hints on similarities in classification as well, but so far we have not explored a key question whose scope goes beyond this paper's s: how does the classification of a MDT compares to its DT ? This question is important because in addition to being more amenable to visualization than DT's, it might also be the case that classification with MDTs might just be good alternative to doing it with DTs. The scope of this question goes beyond this paper because of the tremendous popularity of DT's. The left pane of Table[]provides a partial answer on subset of our domains (the full set of results is in the Appendix, Section D.II p-values are that of a Student paired t-test with HO being the identity of the average errors). Clearly, the classification accuracies are similar for a majority of the domains considered, but interestingly, we observe the two polarities when they are not: on domains ionosphere and onl ine______ taaee__sossrr____e intention the DT better than its MDT, but on domain anal catdata_ supr eme it the opposite. This is particularly interesting because the MDT is no larger and can be substantially smaller than its corresponding DT and thus could represent an efficient pruning of its DT.
MDT embedding: visualization error The right pane of @ an example of embedding errors p 55) for the first five MDTs in a boosted ensemble (we remind that the description of the boosting algorithm and the embedding of the leveraging coefficients is provided in Appendix, Section C.IX). The full set of results is in Appendix, Section D.II The results show that errors are small in general, which is good news given our quite heuristic modification to Sarkar's embedding. There is more: the embedding error tends to decrease in some cases very significantly - with the MDT's index in the ensemble, see for example winered, german and qsar. The explanation is simple and

---

domain
test error (DS vs MDT)
MDT
# in ens. + visu. error
DT
MDT
p-val
#1
#2
#3
#4
#5
ionosphere
5.71 + 2.70
9.11 + 5.34
0.0237
15.21
9.84
8.20
6.88
3.45
winered
18.39 2.02
18.32
2.27
0.9227
12.13
11.87
8.56
6.31
4.44
german
24.00 + 4.37
4.20
0.8793
14.74
10.06
7.73
5.05
2.70
a._6.
23.40 + 1.85
22.56
1.75
= 1.75
0.0134
7.11
8.32
8.23
8.67
7.60
abalone
22.15 2.20
=1.14
0.1267
9.37
6.53
8.20
7.61
8.18
qsar
12.98 + 2.63
12.89 + 4.05
0.8772
16.78
12.97
12.86
5.77
4.24
o._ 9.90 0.78
10.67 = 0.54
0.54
0.0057
5.42
3.74
5,46
6.42
7.00
hardware
1.33 - 0.27
1.44 = 0.19
0.0534
7.78
8.03
3.44
3.49
2.48
Table 2: Summary of two experiments for a subset of our domains. Left pane: test errors (%) of DTs vs their corresponding MDTs (averageistd dev). Bold faces on p-values indicate keeping
the identity of averages for a 0.05 first-order risk. Right pane: embedding error (5) (%) for
the visualization of the first five MDTs in an ensemble learned. Shorthands used: o._s._..
a. maalcatdata_suprene See text for details.
due to key boosting feature, the reweighting mechanism underlined above: with weight updates that bring in general the total (new) weight of positive examples closer to that of negative examples, the root of the next MDT gets embedded closer the center of the disk. As the root moves closer to the center, it is much easier to get a good spread of the tree in the rest of the disk at reduced error, in particular the MDT well balanced. For an example, see hardware for j 70 in Figure (p 1%).
7 Conclusion
This paper proposes three separate contributions that altogether provide solution for hyperbolic embedding of (ensembles of) decision trees (DT), via (i) a link between losses for class probability estimation and hyperbolic distances, (ii) the design of clean, unambiguous embedding of DT via its monotonic subtree and (iii) a way to improve visualization and encoding properties of the Poincaré disk model using its t-self, a notion we introduce. Each of these contributions are indeed separate: for example, one could reuse (i) to embed models different from DTs or (iii) to perform hyperbolic embeddings of objects being not even related to supervised models. Contribution (ii) is of independent interest with the introduction of a new kind of tree-based models, monotonic decision trees.
We believe that the way we address (iii) opens general applications even outside hyperbolic geometry. Indeed, we generalize classical integration and derivation context encompassing the concept of additivity, upon which integration is built, extending standard properties of integration and derivation in natural way (see the Appendix for many examples). That such properties can be obtained without additional technical "effort" bodes well for perspectives of developments and applications in other subfields of ML, where such tools could be used, not just in our geometric context, smoothly tweak distortion measures. Many distortion measures used in ML are indeed integrals in nature: Bregman divergences, f f-divergences, integral probability metrics, etc.
One inherent limitation of our work is linked to the use of DTss: they typically fit very well to tabular variables (attribute- value data) but otherwise should be used with caution. Our work is first step towards more sophisticated visualization for supervised models (Section 2 looking for tailored fit geometric spaces to best blend their symbolic and numerical properties. More can be done and more has to be done: at an age of data collection and ML compute still ramping up, seeking model "pictures" worth thousand words a challenge but a necessity for responsible AI and explainability. Finally, introduction of monotonic decision trees begs for a deeper statistical analysis of such models, given the tremendous popularity decision trees that they could complete - or replace on mainstream data science pipelines.
Acknowledgments
Thanks are due Mathieu Guillame-Bal Bert, reviewers and the area chair for many suggestions that helped to improve the paper's content. Work was done while AS was visiting RIKEN AIP.
10

---

References
[1] E. Amid, Nielsen. R. Nock, and M.-K. Warmuth. Optimal transport with tempered exponential measures. In AAAI'24, 2024.
[2] Ehsan Amid, Frank Nielsen, Richard Nock, and Manfred K Warmuth. The tempered Hilbert simplex distance and its application to non-linear embeddings of TEMs. arXiv preprint arXiv:2311. 13459, 2023.
[3] A. Banerjee, S. Merugu, I. Dhillon, and Ghosh. Clustering with Bregman divergences. In
Proc. of the 4th SIAM International Conference on Data Mining, pages 234-245, 2004,
[4] A.-F Beardon and D. Minda. The hyperbolic metric and geometric function theory. In Proc. of the International Workshop on Quasiconformal Mappings and their Applications, 2005.
[5] B.-H. Bowditch. A course geometric group theory. Math. Soc. Japan Memoirs, 2006.
[6] I. Chami, A. Gu, V. Chatziafratis, and C. Ré. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering. In NeurlPS#33, 2020.
[7] I. Chami, A. Gu. D. Nguyen, and C. Ré. Horopea: Hyperbolic dimensionality reduction via horospherical projections. 38th ICML, volume 139 of Proceedings of Machine Learning Research, pages 1419-1429. PMLR, 2021.
[8] P. Chlenski, E. Turok, A. Khalil Moretti, and I. Pe'er. Fast hyperboloid decision tree algorithms. In 12th ICLR, 2024.
[9] H. Cho, B. Demeo, J. Peng, and B. Berger. Large-margh classification in hyperbolic space. In n2nd AISTATS, volume 89 of Proceedings of Machine Learning Research, pages 1832-1840.
PMLR, 2019.
[10] Z. Cranko and R. Nock. Boosted density estimation remastered. In 36th ICML, pages 1416- 1425, 2019.
[11] L. Doorenbos, P. Márquez-Neila, R. Sznitman, and P. Mettes. Hyperbolic random forests. CoRR, abs/2308. 13279, 2023.
[12] D. Dua and Graff. UCI machine learning repository, 2021.
[13] J. Friedman, T. Hastie, and R. Tibshirani. Additive Logistic Regression Statistical View of Boosting. Ann. of Stat., 28:337-374, 2000.
[14] O.-E. Ganea, Bécigneul, and Hofmann. Hyperbolic entailment cones for learning hierar- chical embeddings. In 35th ICML, volume 80 of Proceedings of Machine Learning Research, pages 1632-1641. PMLR, 2018.
[15] O.-E. Ganea, G. Bécigneul, and T. Hofmann. Hyperbolic neural networks. In N31 pages 5350-5360, 2018.
[16] D.-E. Knuth. Two notes on notation. The American Mathematical Monthly, 99(5):403-44) 1992.
[17] M.-T. Law, R. Liao, J. Snell, and R.-S. Zemel. Lorentzian distance learning for hyperbolic representations. In 36th ICML, volume 97 of Proceedings of Machine Learning Research, pages 3672-3681. PMLR, 2019.
[18] Y. Mansour, Nock, and R.-C. Williamson. Random classification noise does not defeat all convex potential boosters irrespective of model choice. In 40th ICML, 2023.
[19] G. Mishne, Z. Wan, Wang, and S. Yang. The numerical stability of hyperbolic representation learning. In 40th ICML, 2023.
[20] M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In NeurIPS#30, pages 6338-6347, 2017.
[21] F. Nielsen and R. Nock. On Rényi and Tsallis entropies and divergences for exponential families. CoRR, abs/1 105.3259, 2011.
[22] L. Nivanen, A. Le Méhauté, and Q.-A. Wang- Generalized algebra within nonextensive statistics. Reports Mathematical Physics, 52:437-444, 2003.
[23] R. Nock, W. Bel Haj Ali, R. D' Ambrosio, F. Nielsen, and M. Barlaud. Gentle nearest neighhors boosting over proper scoring rules. IEEE PAMI, 37(1):80-93, 2015.
[24] R. Nock and A. K. Menon. Supervised learning: No loss no cry. In 37th ICML, 2020.

---

[25] M.-C. Pardo and I. Vajda. About distances of discrete distributions satisfying the data processing
Theorem of Information Theory. IEEE Trans. IT. 43:1288--1992 1997.
[26] J. R. Quinlan. C4.5 programs for machine learning. Morgan Kaufmann, 1993.
[27] J.-G. Ratcliffe. Foundations of Hyperbolic Manifolds. Springer Graduate Texts in Mathematics,
1994.
[28] M.-D. Reid and R.-C. Williamson. Information, divergence and risk for binary experiments. JMLR, 12:731-817, 2011.
[29] F. Sala, C. De Sa, A. Gu, and C.Ré. Representation tradeoffs for hyperbolic embeddings. In 35th ICML, volume 80 of Proceedings Machine Learning Research, pages 4457-4466. PMLR, 2018.
[30] R. Sarkar. Low distortion Delaumay embedding of trees in hyperbolic plane. In GD'II, volume 7034, pages 355-366. Springer, 2011.
[31] R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. MIJ, 37:297-336, 1999.
[32] Andrew Selbst and Solon Barocas. The intuitive appeal of explainable machines. Fordham Rev., 87:1085, 2018.
[33] A. Slavik. Product integration, its history and applications. Matfyzpress, Praze, 2007.
[34] R. Sonthalia and A.-C. Gilbert. Tree! am no tree! am low dimensional hyperbolic embedding. In NewrIPS*33, 2020.
[35] T. van Erven and P. Harremoës. Rényi divergence and kullback-leibler divergence. IEEE Trans.
IT, 60:3797-3820, 2014.
[36] M. Yang, M. Zhou, R. Ying, Y. Chen, and King. Hyperbolic representation learning: Revisiting and advancing. 40th ICML, volume 202 of Proceedings of Machine Learning Research, 2023.
pages
[37] T. Yu and C. De Sa. Numerically accurate hyperbolic embeddings using tiling- tiling-based models. NeurIPS#32, pages 2021-2031, 2019.
[38] T. Yu, T.J..B. Liu, A. Tseng, and C. De Sa. Shadow cones: Unveiling partial orders in hyperbolic space. CoRR, abs/2305. 15215, 2023.
12

---

Appendix
To differentiate with the numberings in the main file, the numbering of Theorems, etc. is letter-based (A, B...
Table of contents
Broader Impact
(-algebra and (-additivity
Pg 14
Supplementary material on proof's
Pg 15
Proof of Theorem
Pg
+
Proof of Theorem
Pg
Additional and helper results for Theorems 2 and
Pg
+
Sets endowed with distortion and their t-self: statistical information
Pg 20
The t-self of the Lorentz model of hyperbolic geometry
Pg
Proof of Lemma
Pg 22
Proof of Theorem
Pg 22
Modifying Sarkar's embedding in Poincaré disk
Pg 23
Boosting with the logistic loss à-la AdaBoost
Pg
Supplementary material on experiments
25
Domains
Pg 25
Visualizing a DT via its MDT and embedding errors
Pg
All Poincaré disk embeddings
Pg 27
Interest of the t-self for visualization.
Pg 27
13

---

A
Broader Impact
This paper presents work whose goal to advance the field Machine Learning. Beyond our proposed tempered calculus which is used to amend the original Poincaré disk model of hyperbolic geometry, the monotonic decision trees and t-self hyperbolic embedding contribute to the field of visualization and explainable AI for ML models. Thus, there are two distinct avenues for broader impact and related care:
1. from the geometry standpoint, the I-self brings a novel coding and visualization advantage over the founding Poincaré disk model for the embedding close to the disk's border, regard- less of the nature of the embedding (whether it is data, models, etc. that are embedded). However, being new, it important to keep the label that the t-self is being used, and also clearly display the relevant value of or ranges, since the properties of the embedding can change depending on them;
2. from the model standpoint and societal perspective, these visualizations will provide better methods for deployed decision tree models to be scrutinized. A potential negative of our approach is the required reduction from normal decision trees monotonic decision trees, where the visualization does not directly correspond to the initial decision tree. We believe that monotonicity adds value to the interpretation of rules that are progressively built from trees, yet practitioners should be careful when extracting inferences from the reduced monotonic decision tree and clearly label them being extracted from such model, to prevent confusion with decision trees, that are overwhelmingly popular. From pure prediction performance standpoint, we show least empirically within the bounds of our settings that monotonic decision trees are similar to their original decision tree counterparts.
B f-algebra and t-additivity
We provide here few more details on the basis of our paper. the t-algebra and the t-additivity of some divergences.
B.I Primer t-algebra
Classical arithmetic over the reals can be used to display duality relationships between operators using the log, exp functions, such for example log( (gg(a/) log log b. cxp(a + b) exp(a) cxp(b), and so on. They can also be used to define one operator from another one. There is no difference between the operators appearing inside and outside functions. In the I-algebra, difference appears and such relationships can be used to define the 1-operators from those over the reals, as indeed one can define the tempered addition
x Bi
(ex,(xxp (x),
the tempered subtraction,
x O y = log, leg,(e (x))<<,()xx(()))
(both simplifying to the expressions we use), and of course the tempered product and division, x E = elog, (x) log,
Øi = exp, (log, log, (v)).
whose simplified expression appears egg. in See also 22
B.II Functional /-additivity
As is well-known, Boltzman-Gibbs (and so, Shannon) entropy additive while Tsallis entropy is (-additive. Note that additivity for BG requires being on the simplex, but t-additivity of Tsallis technically requires only positive measures the simplex restriction ensures the limit exists for and then T-BG.
Divergences can also be f-additive The KL divergence
D
Σ Ep log(ps/4))
14

---

is additive on the simplex but not t-additive: using decomposition of p. product of (independent) distributions (p P2 and 91 92) using the cartesian product of their support, we indeed check (R.(P (plg) Du. (Pr Q1 + DRL. (P2llg)) 92) with Pij PıP22 and lij The22j- On the other hand,
Tsallis divergence 21 f-additive on positive measures with such decomposition, with
Ex (Pkk/////e (p(K/p//)) /xx
Dr(pllg)
1-t
and check that Dr(pllq) Dr(pi 91) Dr(p2 (2)) (1- 6) (T q1) (P2ll2))2)
(additional requirement to be on the simplex for convergence to DXL as t 1). For # (1, 2) and on the simplex. Tsallis divergence is an f-divergence with generator the tempered relative entropy is t-additive on the co co-simplex, where in this case
(22 1)/(1 - t). Similarly,
Σx Phyk
1-t p.qe Am-
D.(plqg
1-t
The tempered relative entropy is Bregman divergence with generator - z zlog, log.-1
C Supplementary material on proofs
C.I Proof of Theorem 2
The Theorem is tautological for t 1 so we prove it for t * Denote P(8) the set of subsets of set
and Pa (8) - P()))()) We first transform ( (f) (index n shown for readability) a better
suited expression:
$ (f) = (+)==14fS(E
= Σ (1 (1-1))P1-1 (I((E) PePa([m))
REP
I ()(()((
1-t PeP ([n)) REP
[
11
-1 1).
= 1-t
-10
(14)
where we have used II; [x;-1, for conciseness. We now need technical Lemma.
Lemma 2. Fix any 0 & < 8/10 and consider any n reals qı, i E [n] such that lqi & u, vi e [n]. Then
(1+ Σ, 1 1 11(1+ ④)) & exp (nw-u) v).
(15)
Proof. Suppose all the g; i E [n] satisfy [u, uj and let be strictly convex differentiable and defined over [u, v.. Then comes from [10 Lemma 9] that we get the right-hand side of
(p(()) (p(u)) ((e) U
E,[((g)] (..l)) & Do (4')
(16)
where we can pick w E {u, /) and D&p is the Bregman divergence with generator (the left-hand side is Jensen's 's inequality). Picking = (assuming wlog 0) and letting
1-
2v 1-
Yo
log
+
(17)
2v
for the choice <p(2) -log(1 2) and ur = u, we obtain after simplification
& log
1
Σ gi
1
Σ
log(1 + q.) < Y, log(Y.) 1.
(18)
n
15

---

Analizing function Y, log(Y.) reveals that it is upperbounded by v² if [-8/10, 8/10). Hence, multiplying by all sides and passing the exponential, we get that
(1 + # ,,
8
1<
exp (πw²), 0A
I1.(1 ( ④))
10
which leads to the statement of the Lemma.
Let us come back to our Riemannian summation setting and let
qs
(1 (1-t) [L]f(E), vi [m].
V max lg:l-
Assume now that f is Riemann integrable, which allows to guarantee that, at least for n large enough and step of division An too large, we have 14i1 8/10, Vi [n] and we can use Lemma 2 We get from (14) and Lemma 2 ift
$1.(1 (n)
[1+ 9:) -1
1-t
exp(-nn -1 -- (1+
-1 -1)d0
1
Σ.
6
1-1
1
Σ
and we permute the bounds if L 1. Importantly, we note that
Σn (1-t). (()
(20)
So suppose lim,. ()) = L1 = Sa f()juis is finite and choose the step of division A, not too large so that
max [gil (1-1) max(m[L)IS((E)) K ((1-0)6 max S((a, bj)),
for some constant K (this possible because f (1-)Riemann integrable; we also note that if the division is regular then we can choose K = 1). The value of K is not important: what is important is that ne remains finite in (19 as increases, while lim
0. Hence, exp(-ne nnv v) 1 as oc and (19) implies, because lim + a/n)" = exp a, the two first identities in
lim $2.
1
lim
1 1+ n
Σ 96
1-t
2-+0
1
Σ 91 - 1
exp
1-t t
(exp ((1 - t) 1)
(21)
I - t
= log, exp(LI)
= log, exp S. f(u)du)
(0
21
follows from 20 and by definition lim Sit ( =
] f(u)deu. So get that
Riemann integration (t = 1) grants
(t)
(u)deu = log, exp
f(u)du,
which, in addition to showing ( (main file) also shows that = I-Riemann integration equivalent to all * 1-Riemann integration, ending the proof of Theorem 2
Remark The absence of affine terms in upperbounding e
log( Y,) the neighborhood
of is crucial to get to our result.
16

---

C.I1 Proof of Theorem 3
Using Theorem
we just have analyze the limit in relationship the Riemannian case:
(8)
w) J. f(u)duu
f(u)dru
DzF(2)
lim
log, exp S 8=+ f(u)du θ log, exp Sr f(u)du
lim 6-0
(22)
exp Sa f(u)du
lim 6-0 8
logg
(23)
exp St f(u)du
(1. f(u)du - -S. f(u)du)
1
f())du-f
lim 6-0 8
log, exp
(f.t0
f(u)du - f(u)du)
lim 8-0 8
(24)
where the last identity is the classical Riemannian case, follows from Theorem 23 follows from a property of log, (log, @ log, b log, (b))), 24 follows from the fact that log, exp(2)
z + (=)). This completes the proof of Theorem
C.III Additional and helper results for Theorems 2 and 3
We first state trivial but important Lemma, whose content is partially used in the main file.
Lemma 3. 9. satisfies the following properties:
1. 9 (2) is strictly increasing for any e TR, strictly concave for any t 1, strictly convex for any t < 1 and such that sign(3, = sign(z), VI e R;
2. (z) = 1;
3. (1-integral mean-value) Suppose f Riemann integrable over [a, b). Then there exists e (a, b) such that
(4)
(b 8--a) 00 gr (cc) [u)dee -1--0(11) a)).
We list series of consequences both theorems.
General properties of t-integrals Some properties generalize those for classical Riemann integra- tion.
Theorem C.1. The following relationships hold for any E lK and any functions f. g I-Riemann integrable over some interval [a, b]:
(+)
f(u) (+ -) = (") f(w)deu (or or)
(4)
(0) ("I g(w)dmy
(additivity),
(*)
(1-(1-1)A)
Af(u)deu = A.
f(w)dau (A
(dilativity),
f(z)dmu = 10 f(u)duu f(u)deu [a, 8])
(0)
(Chasles' relationship),
(0)
(191-10)
f(u)du <
[f(w)(du
(triangle inequality),
(0)
1 J glu)deu (f
f(u)deu
(monotonicity).
17

---

Proof. We show additivity for Bi/+ (the same path shows the result for 0t/-):
J. ul
(+) bb
[" f(u)du Br log, exp Jn g(u)du
g(u)dau
log, exp
f(u)du exp [ a(u)du)
log,
exp
b (f + g)(udd
=
log, exp
= [ (f (u)dd.
We show dilativity, using t' = = - (1---117) t)x:
(4)
b
Nfluddeu
log, exp
(()ddu
log, exp A
f(u)du
Ja
exp A(1 -1) Iha s(u)dm)) -
1-t
1- 1'
exp (1 (1-+) 1 sluaau) -1)
t
1
t) S_e f(w)du) -
X
exp (1 (1-+)
1-t t'
X
logg exp f(ujdu
(t) db
(1-(1-0A) b
[
x
fujdd-= A.
fuuddu.
The triangle inequality follows from the relationship:
log, exp(x)]
exp |=|, -
V2, te IR,
from which we use the fact that Riemann integration satisfies the triangle inequality and log, 10g1-11-01 is monotonically increasing on R, in the penultimate line of:
(4)
b I'
f(u)deu = log, exp
f(u)du
10g1-11-11 exp
[u)du
1og1-11--1 exp
[f())ddu
(1-11-())
[f
18

---

We show Chasles' relationship:
me ( J. f(u)deu = log, exp
J. f(u)da
[ujdd Br
I (u)du Br log, exp
f(u)du exp J f(u)du
log.
exp
I(u)du+ [ M(x)du)
log, exp
= log, exp
[u)du
(1)bb J. f(u)du,
where the second identity uses the property log, log, b log, lab). Monotonicity follows immediately from the fact that 2 - log, exp 2 is strictly increasing:
(+) b
f(u)d,u
log, exp
f(u)du
(1) [ g(u))du.
log, exp
g(u)du
This ends the proof Theorem C.1
Computing I-integrals Next, classical relationships to compute integrals do generalize to t- integration. We cite the case of integration by part.
Lemma 4. Integration by part translates to t-integration by part as:
(0)
(0) bb
J. f'ydu,
Sg'du
(0) [')' O
where we let
(0) [M]] =
log, exp(h(b)) Or log, exp(h(a)).
(Proof immediate from Theorem
Geometric properties based on t-integrals This is more specific result, important context of hyperbolic geometry: the well-known Hyperbolic Pythagorean Theorem (HPT) does translate to tempered version with the same relationship to the Euclidean theorem. Consider a hyperbolic right triangle with hyperbolic lengths a, b, c, being the hyperbolic length of the hypothenuse. Let a, be, ct denote the corresponding tempered lengths, which are therefore explicitly related using 9, as
a log, expa, by = log, cxpb,
= log, cxpc.
Define the tempered generalization of cosh:
exp; z + exp,(-2)
cosh
(25)
2
The HPT tells that cosh c cosh cosh b. It is simple matter of plugging 9c, using the fact that log, and exp, are inverse of each other and simplifying to get the tempered HPT, which we call (-HPT for short.
Lemma 5. (t-HPT) For any hyperbolic triangle described above, the tempered lengths are related as
cosh; C cosh, o cosh, by.
19

---

Now, remark that for any t * O, a series expansion around 0 gives
t²² (x) = 1 +
+ o(2²)
2
(exp, is always infinitely differentiable around O, for any t) So the t-HPT gives
tot 1 1+
2 + +())) = (1+ tap
2 ooa;)) (1+ the + o(b))
which simplifies, if we multiply both sides by 2/t and simplify it into
+ = of bi + o(a)) o(b)).
which for infinitesimal right triangle gives oi of bj, i.e. Pythagoras Theorem, does the HPT one gives in this case (e. a² 1²), which is equivalent of the particular t 1-HPT case.
(-mean value Theorem The t-derivative yields a generalization of the Euclidean mean-value theorem.
Lemma 6. Let t and be continuous over an interval [a, b), differentiable on (a, b) and such that - 1/(11 f) $ f([a. 6)). Then Bee (a, b) such that
Dif(c) = (S(b) O (()) - (f(a) O S()) -
Proof. We can obtain a direct expression of D.f by using the definition of θ and the classical derivative:
Dxf(x) lim f(x + 6) f() 6-0
f(x + 8) f(x))
f((x)
lim 8-0
1+ (1- (1-t))((z)
It (1 t)(())
8
I+ (1 ()f(2) (26)
From here, the mean-value theorem tells us that there exists e [a, b] such that
('(c) [(b) - f(a)
Dividing by + t)f(c) (assuming f(c) # 1/(1 t)) and reorganising, we get
1
f(b) f(a)
D.f(c)
b-a a 1+ (1 ()f(c)
f(a) f(c) b-a 1+ 1+ (1-t)f(( b-a 1+ t)f(c)
1
f(b) - f(c)
1
(f(b) f(c)) (f(a) O f())
b
which completes the proof of the Lemma.
In fact, the t-derivative at some c "just" an Euclidean derivative for an affine transformation of the function, namely - f(z) e f()), also taken at This "proximity" between t-derivation and derivation is found in the tempered chain rule (proof straightforward). Lemma 7. Suppose g differentiable at 2 and f differentiable at g(2). Then
Di(f ()) D.(f)(g(z)) g2(2).
C.IV
Sets endowed with distortion and their f-self: statistical information
Here, contains probability distributions or the parameters of probability distributions: f can then be an f-divergence (information theory) or a Bregman divergence (information geometry). Tsallis divergence and the tempered relative entropy are examples (-additive information theoretic and information geometric divergences. A key property of information theory is the data processing inequality (D), X being a probability space, which says that passing random variables through Markov chain cannot increase their divergence as quantified by [25] 35). A key property of
20

---

information geometry is the population minimizer property (P), which elicits a particular function of of points as the minimizer of the expected distortion to the set, as quantified by f [3]. We let (J) denote the joint convexity property, which would state for f and any 2;,, 222 11 3/2 that
f(A- 21 + (1 X) 22, yı + (1 (1-X), X) 42) & Af(11, v1) + (1 2) U2),
(27)
and convexity (C), which amounts to picking Y1 = Y2 (convexity in the left parameter) xor #1 22 (in the right parameter).
Lemma 8. For any t I. the following holds true:
(D) f satisfies the data processing inequality iff f()) satisfies the data processing inequality;
(P) 2 E arg ming E: (ais 1e) iff
bb E arg min (Bx), ( (ess:)): M));
(28)
(J) f satisfies (7 iff the following (t,t', -'j-jo convexity property holds:
(A- x1
A) 22 y1 + (1 A) y2) E ( (x1. + (1- {) (222. y2), (29)
with t' = mi 1 - A At and t" = min A + (1 A)t].
Proof. (D) and (P) are immediate consequences of Lemma (point [1.], main file) and properties of loge. We prove (J). 9, being strictly increasing for any t, we get for t & 1
(( x1 + (1 - A) x2, X y + (1 () .))
< (Af(21, v1) + (1 - fx2 U2))
X. - f(x1 81 + (1- 1)- - o f 92) X0 v1) (1 ()) (22) (30)
because is convex. Ift > 1, we restart from the first inequality and remark that 9e (Af(21. (1 f)x((r2) 822)
9. (1)( 9, ((1 (22(
& (Af(z11, + ((1 (),,
A A -1ー f( (1 A) (1+ o f(22, U2)
2 (1 (1 (A+ V2). (31)
The inequality is due to the fact that a D b = a + b + (1 aab & a + b ab > and / 1. The
last equality holds because, for l' = 1 (1 1)b, we have
(1-13b
1- t' a' -1
log,
(a")
b. log
1 1-1²
Putting altogether 30pm and 31,, we get that for any l R.
f(0 2i + (1 (1- X) x22 A v1 + (1 A) V22
(min(t.11 (æ) 91) (1 A) (11-)))niitt 92),
(32)
as claimed for (J). This ends the proof of Lemma
We note that 29 also translates into a property for (C); also, if & 1, then t t= t' = in (9
C.V The t-self of the Lorentz model of hy hyperbolic geometry
As an additional example of use, we consider the Lorentz model for a simple illustration in which one creates approximate metricity in the t-self. This d-dimensional manifold is embedded in Rd+ via the hyperboloid with constant -c < curvature and defined by H. (æ E d xo >
0 A x o = -1/c), with æ y = Zool + 1 24y; the Lorentzian inner product (27] Chapter 3]. In ML, two distortions are considered on H. [7]. one of which is the Lorentzian "distance", ( y) = - -(//c) 2 xr y. It is notoriously not a distance because it does not satisfy (T), yet we show that we can pick and the curvature in such way that the t-self is arbitrarily close to a metric space.
21

---

Lemma 9. 4g > O, pick and curvature as a = 1 (1/8),c= 2/8. Then the t-self of H. is approximately metric: d satisfies (R), (S), (I), and the ò-triangle inequality,
d (x, y) + di (y, =) Yz, y. z Hc.
<
(33)
Proof. de still obviously satisfies reflexivity, the identity of indiscernibles, and non-negativity, so we check the additional property now satisfies, the weaker version of the triangle inequality, Given any 2; y: z in H.. condition di, (zx x) & d (x.y) y) + d (y, + for t > 1 is
exp (-(t- 1) (- 2. 2))
exp (- ( -22 y))
t t-1
t t- 1
1 cxp exp (-
1) (- - 22-yyy0 yo 2)) 8,
t 1-1
which simplifies to
2(t -
(-2 2(t - 1)
exp (2(t - 1) y) + exp (2(t 1) 1). y 0oz)
exp
+ (t 1)8exp
+ exp (2(t - 1).1)).. x
We have x o & -1/e by definition, so sufficient condition to get the inequality have exp (2(t - 1) y 2)
(- 2
< (t - (t-1)8exp
(34)
c
Function ((=) = z6 exp( is maximum over R. for c/2, for which it equals (1(2)) = c2/(2e). Fix t (c/2). We then have exp (2(t - y 2) < 1/e so get (34) for this choice t, it is sufficient to pick curvature c = 2/8, yielding relationship = 1 + (1/8).
C.VI Proof of Lemma
We start by proving condition
g(k)
(35)
in the Lemma. Using the proof of boundary (Definition 5.1 satisfies
19
Proposition 3.1], we know that any point æ k-close to the
+ x
d (xx(0) 0)
(x.
log,
log, (2 10% -1),
so to get (35). we want log, (2 - 1) 7 g(k). Letting t = 1- f(k) with f(k) E R, we observe
log, (2- 10 - 1) = ((2 10th 1)5(()) - 1) ff(), so we want, after taking logs,
log (1 + f(k))(k))
log(2 10 11) >
(36)
f(k)
(this also holds if fkk) < because (+)())g(kk)
1), and there remains observe 1og(2-10* -1) vk > Hence, get (36) it is sufficient to
A log(10) log(2 1/100) with log(2 1/10*)
request
log (1 + (k))
log(10) k
f(k)
which is 35 For such t, the new hyperbolic constant satisfies
log, expr
f(k) (exp 1)
C.VII Proof of Theorem
Take any observation æ E Trivially, H'(x:) is real value that belongs to the path followed by in H. Furthermore, if we consider any path from the root to leaf in H, all the vertices in its strictly monotonically increasing subsequence of absolute confidences appear in H' (conditions 4, in GETMDT), which guarantees the condition on prediction in (M). Hence (M) is satisfied.
22

---

children (?)
reflect
add arcs nodes
reflect
children added
Sarkar
Us
Figure I: Schematic description of our modification (bottom) of Sarkar's algorithm (up). Our modification solely changes the step which the children of node (here, a) are computed, this node having been reflected back to the origin. Instead of using fixed angle and length to position ares (and thus children of the node), the angle depends on the number of leaves that can be reached from a child, and the length depends on the difference between absolute confidence between the node and the corresponding child. We also define an angle which represents the domain (before reflecting back) in which the embedding is going to take place, shown with the thick dashed line (here, this angle is TT.
C.VIII Modifying Sarkar's embedding in Poincaré disk
gives clean low-distortion embedding when the tree is binary or the are Things different in case: MDT nodes have arbitrary out-degrees, and lengths depend on the absolute confidence at the corresponding MDT nodes. Plus, a direct implementation of Sarkar's algorithm would violate the constraint for strict path-monotonictty MDT that nodes in path from the root need to progressively come closer to the border equivalently, would create substantial embedding errors for confidences and violate (B). Without focusing on an optimal solution (that we leave for future work), one can remark that all these problems can be heuristically addressed by changing one step of Sarkar's algorithm, replacing the use the total fan out angle for mapping children (Step 5: in Algorithm of (29]) by a variable angle with special orientation the disk.
Sarkar's algorithm
30
length is constant 29
an
We refer to the concise and neat description of Sarkar's embedding in 29 for the full algorithm. Our modification relies changing one step of the algorithm, described in Figure The key step that we change is step 5: in the description of 29 Algorithm 1]. This step embeds the children of a given node (and then the algorithm proceeds recursively until all nodes are processed). Sarkar's algorithm corresponds to the simple case where all ares to/from a node define fixed angle, which does not change during reflection because Poincaré model conformal. Hence, if the is binary, this angle is 2r//3, which provides a very clean display of the tree. In our case however, some children may have just one are leaf while others may support big subtrees. Also, are lengths can vary substantially. We thus design the region of the disk into which the subtrees are going to be embedded by choosing an angle proportional the number leaves reachable from the node, and of course lengths have to match the difference between absolute confidence between node and its children. There is no optimization step to learn a clean embedding, so we rely on set of hyperparameters to effectively compute this new step of the algorithm.
23

---

C.1X
Boosting with the logistic loss à-la AdaBoost
Algorithm 2 LOGGSTICBOOSTTS)
Input: Labeled sample 8 = {(ar,, yi), e [m]], T Nso;
:: Wii < 11/2, Vi [m] initialize all weights (equivalent to maximally unconfident prediction)
2: for 1,2,,
3:
Hj - WEAK-LEARN(S, wy);
request a DT as a "weak hypothesis" // picks leveraging coefficient for H; // weight update, not normalized
4:
aj IR;
5: for =
"(++1 wyj + (1 wj:) ex
(37)
Output: Classifier Hy ay}H(())
The link between distances the Poincaré model of hyperbolic geometry and the canonical link of the log-loss (3) can be extended to leveraging coefficients in boosted combinations (3T]. To get there, we need a tailored boosting algorithm which parallels AdaBoost's design tricks (hence different from LogitBoost [13]). We want boosting algorithm for the liner combination DTs which displays classification that can be easily and directly embedded in the Poincaré disk. Algorithm LLGISTICBoOSS is provided above. For the weight update, we refer to [23.. We already know how to embed DTs via their Monotonic DTs. What boosting algorithm of this kind does is craft
H
Σ uy(), uz(x)) = aj Hy(
(38)
Remark that have merged the leveraging coefficient and the DTs' outputs H. on purpose. To compute the leveraging coefficients o in Step 4, we use AdaBoost's s secant approximation trick applied not to the exponential loss but the logistic loss: for any [-R, R and R,
1 1 v loo(1 + exp(-aR) + log(1 + exp(aR)). (39)
log(1 + exp(-az))
Hence, to compute the leveraging coefficient aj that approximately minimizes the current loss, >, Wji log(1 + cxp(-ay, (x;))) minimize instead the upperbound using (39). Letting
PIA
(Vron)
max XEA(H,)
log
(40)
PJA
(note the slight abuse of notation for readability, since we write (Nhae)) instead of [(Vro))] we remind that A(.) denotes set of leaves tree; the index in the local proportion of positive example P reminds that weights used need to be boosting's weights) which we note can be directly approximated on the Poincaré disk by looking the leaf nearest to the border of the disk. Because the maximal absolute confidence in the DT is also the maximal absolute confidence in its MDT, we obtain the sought minimum,
log 1 1++ra T3
aj
(v)ne)
(1
where ry E [-1, is the normalized edge
Y: (x:))
Tj
Wji
Σx Wiss
max H4 (xx))
st [m
P P))(m)))
log
(41)
(wroa) *
1 PJN(o.))
'Explained in B1 Section 3.1].
24

---

embedeing bolines (posterior)
(-))
$422
(Vi..]
best confidence in DT
(Mlsl)y
normalized DT confidence
Lawraging covflicent
root (square node)
red majority class
arc thicknees Ateata in DT to reach child noda (anzs going outwards B)
Figure II: One MDT learned on UCI domain online_ _et_ppashpppsttteeee iii.eninnnnnnnn reproducing the left of Figure 2 in the main file, with additional details for leveraging coefficients.
where w indicates normalized weights. is not hard show that because we use the local posterior PjA. at each leaf, aj and also ry z 0. Hence, everything is like if we had an imaginary node Vy with Pius (1 + r2)/2 ( 1/2) and positive confidence
Pjeyz Dju,
(vroa)s
viroo (Pju,) log
that we can display in Poincaré disk (we choose to do it as a circle; see Figure || deduce from (38) that
main file). We
(Wion)s llog (1hran)) )
PiM(a)
(z(x))
1- --)))))))
and note that all three key parameters can easily displayed or computed directly from the Poincaré disk:
If we use MDTs directly for prediction, log
Piace) 1-Pjx(m)
Hy(æ)) is read directly from the
MDT's plot; the absolute value can be removed and the true sign guessed from the node's color (or shape); if se use DTs for prediction, the MDT's plot gives approximation of the DT's output which is 100% accurate for all leaves of the MDT (each corresponds to a leaf in the DT);
* If Hj is leveraged (in an ensemble), (Vrm)) is read directly from the plot (40). as well as (vhan)) Their ratio gives the leveraging coefficient. We note that (yroa) ) is the same for both the DT and its corresponding MDT. Only the value of (troa)) can differ for the DT and the MDT.
We note that regardless of whether also fully accurate for the DT for subset of its leaves.
the DT or its MDT to classify, the MDT's representation is
D Supplementary material on experiments
D.I
Domains
The domains we consider are all public domains, from the UCI repository of ML datasets [2] OpenML, Kaggle, see Table
D.II Visualizing a DT via its MDT and embedding errors
All test errors are estimated from a 10-fold stratified cross-validation. One can always choose to directly learn Monotonic DT's instead of DTs, given their natural fit for embedding in the Poincaré disk. In this case, the hyperbolic representation of the MDT can be immediately used for assessment. Suppose stick to learning DTs (because e.g. they have been standard in ML for decades) and wish
25

---

Domain
d
License
breastwisc
699
9
CC BY 4.0
ionosphere
351
33
CC BCBY 4.0
tictactoe
958
9
PDDL
winered
599
11
CC 4.0
german
1 000
20
CC 4.0
analcatdataa_uprreee
4 053
8
CCBY 4.0
abalone
4 177
8
CC 4.0
qsar
055
41
CC 4.0
hillnoise
1 212
100
CY 4.0
firmteacher
800
16
BCBYAB 4.0
onl iiien__ maline_shooee _aaneesshpperrseee interstiiitnniion
12 330
17
CCBY4.0
give_me www_eeee_eeeeeeeeeee_ee__ee___ee some_ccredis
120 269
11
None
Buzz_ in procial_mediz (Tom's hardware)
28 179
96
CC 4.0
in in__ociallll__ccoooo media (twitter)
583 250
78
CC BY 4.0
Table I: UCI, OpenML (Analcatdata ) and Kaggle (Give_me sonesomeeee _eeedit domains con- sidered in experiments (m total number of examples, d number of features), ordered in increasing m x n. Datset licenses listed in last column.
domain breastwisc
DT
MDT
p-val
4.15 2.18
5.00 + 2.38
2.38
0.2408
ionosphere
3.71:227
9.11 5.34
0.0237
tictactoe
2.61
2.09
1.10
0.1390
winered
1
1.22222221
0.9227
german
24.00 4.37
23.90
0.8793
analcatdata_supeee supreme
23.40 1.85
22.56
0.0134
abalone
2.21
2.44
0.1267
qsar
12.99 4.05
0.8772
hillnoise
3..04
45.88
4.72
0.0008
f firmteacher
6.87 + 1.23
7.04 1.17
0.4292
online_ seaiine__shppprsreem
shopperrs_intention intention
9.90 + 0.78
10.67 + 0.54
0.0057
hardware
1.33 + 0.27
+
0.0534
Table II: Results of the experiments checking whether "reducing" the interpretation of DT to that of its MDT (using its hyperbolic embedding) can give accurate information about the tree as well. Numerical columns, from left to right, give the average+ std dev error for DT's and MDT's and provide the p-value of Student paired t-test with HO heing the identity of the average errors. Entries in bold faces correspond keeping HO for 0.05 first-order risk. See text for details.
to use the visualization of its corresponding MDT to make inferences on the original DT itself. Can we make reliable conclusions? We explore this question by observing that the prediction from DT to MDT can only change if the leaf À that an example reaches in the DT satisfies two conditions: (a) does not appear as leaf in the MDT, and (b) is tagged an internal node of the MDT with confidence of the opposite sign (this is (D1). Step 3: in GETMDT). Without tackling the formal aspect of this question, we have designed a simple experiment
a simple assessment of whether / when this is reasonable. We have trained set of T=200 boosted DTTs, each with maximal size 200 (total number of nodes). After having computed the corresponding MDTs, we have compared the test errors of the boosted set of trees and that of the ensemble where each DT is replaced by its MDT, but the leveraging coefficients do not change. Intuitively. if test errors are on par, the variation in classification of DTs vs MDTs (including confidences) is negligible, and can "reduce" the interpretation of the DT to that of its MDT in the Poincaré disk. The results are summarized Table
we can safely say that our hypothesis reasonably holds in many cases, with two important domains for which it does not: ionosphere and hillnoise. For the former domain, we attribute it to the small size of the domain, which prevents training big enough trees; for the latter domain, attribute it to the fact that the domain contains substantial noise, which makes it
From Table [
26

---

domain
MDT
# in ensemble + visualization error
#1
#2
#3
#4
#5
#6
#7
#8
#9
#10
online_ seciine_hopperrseem
inttettinnn
5.42
3.74
5,46
6.42
7.00
7.96
4.70
5.68
7.93
6.22
analcatdata_ssppa
7.11
8.32
8.23
8.67
7.60
6.15
7.85
6.55
6.82
6.63
abalone
9.37
6.53
8.20
7.61
8.18
8.20
5.91
5.88
6.51
5.43
winered
12.13
11.87
8.56
6.31
4.44
3.13
2.09
1.43
3.84
3.03
qaar
16.78
12.97
12.86
5.77
4.24
2.55
1.68
2.92
2.20
4.13
german
14.74
10.06
7.73
5.05
2.70
2.36
1.26
1.19
2.07
1.57
ionosphere
15.21
9.84
8.20
6.88
3,45
5.49
8.34
4.17
5.86
8.82
hardware
7.78
8.03
3.44
3.49
2.48
1.10
0.74
0.81
0.55
0.45
kaggle
4.92
5.84
6.24
7.07
7.87
8.06
8.06
8,70
8.89
7.89
Table III: Emhedding error p (5) for the visualization of the first ten MDTs in ensemble learned.
MDT #1
MDT #2
MDT #3
MDT #1
MDT #5
ww
#6
#8
Table IV: First 10 Monotonic Decision Trees (MDTs) embedded in Poincaré disk, corresponding to several Decision Trees (DTss) with & 200 nodes each, learned by boosting the log logistic-loss UCI onl ine teeiiie__spppppe thoppersssss.... intention. Isolines correspond to the node's 's prediction confidences, and geometric embedding errors (p%) are indicated. See text for details.
difficult to substantially improve posteriors by splitting and thus make many DT nodes, including leaves, "disappear" in the MDT conversion (Steps 2: and 14: in GETMDT). Table Talle|[ll summarizee the visualization errors of the MDT embeddings obtained in Tables IV XII A key feature of the embeddings is that some domains (e.g. german) the error of the embedding dramatically decreases with the index of the MDT the ensemble. The reason why this happens is simple: the weight update of boosting tends to equalize weights between classes (to make the problem "harder"); thus, new trees tend to have their roots embedded closer to the origin, which makes the embedding easier for the rest of the trees keep good visualization at low error.
D.III All Poincaré disk embeddings
They are presented in Table IV through to XII showing the first models learned in fold of the cross-validation experiments. producing Table
D.IV Interest of the t-self for visualization
We now test the experimental impact of switching to the t-self in Poincaré disk model (See Table XIII for clear depiction of how changing can yields better visualization close to the border of the disk). Recall that switching to the t-self moves way model parts close to OB but keeps low non non-linear distortion around the center, which thus roughly only affected by a scaling factor. Figure a few results. For domain online anline_shopperr swwpperss.....p. intention, we note that the part of the tree that is within the isoline defined by posterior p* (0.1, 0.9) gets indeed just scaled: both plots look quite identical. Very substantial differences appear near the border: the best parts of the model could easily
27

---

200
20NN
#1
MDT
#3
MDT
+
+
MDT #6
MDT #T
MDT #8
MDT #9
MDT #10
Table V: First 10 MDTs for UCI anal supreme. Convention follows Table IV
MDT #1
MDT #2
MDT #3
MDT #1
MDT #5
#G
#8
#9
#10
Table VI: First 10 MDTs for UCI abalone. Convention follows Table IV
MDT #11
MDT #22
MDT #3
MDT #4
MDT .45
MDT #6
MDT
MIST
MDT
MDT #10
Table VII: First MDTs for UCI winered. Convention follows Table IV
28

---

#1
MDT #3
MDT
MDT #6
MDT #7
MDT #8
MDT 29
MDT 110
Table VIII: First 10 MDT' for UCI qsar. Convention follows Table IV
MDT #1
MDT #2
MDT #3
MDT #1
MDT #5
#6
#8
#9
#10
Table IX: First 10 MDT' for UCI german. Convention follows Table
SING
MDT #1
MDT #22
MDT #3
MDT #4
MDT #5
MDT #6
MDT
MIST
MDT
MDT #10
Table X: First 10 MDTs for UCI ionosphere. Convention follows Table
29

---

MDT #1
MDT #2
MDT #3
MDT #1
MDT #5
#6
MDT
#8
Table XI: First 10 MDT's for UCT hardware. Convention follows Table
+
00
MDT #1
MDT #2
MDT #3
MDT #1
MDT #5
0.0
MIST
Table XII: Plot of the 10 first MDTs learned on kaggle Le_eemee_SOmee credit (top panel and
bottom panel). In each panel, we plot the embedding in B (top row) and the t-self ( (t = 0, bottom row). Remark the ability for the t-self to display clear difference between the best subtrees, subtrees that otherwise appear quite equivalent in terms of confidence from B alone.
30

---

== (.)) (IB)
0.6
0.3
1 0.0
Table XIII: T-self 1 for Poincaré disk B (left), for several ts. We plot the same set of isolines of IB (and their mapping via p,). parameterized by probability E [0, 1] which gives the norm = |2p 1] in (From the innermost to the outermost, E {0.6, 0.7, 0.8, 0.9, 0.999), or by symmetry for
p' = p. also indicated). Remark the equidistance of isolines in B, approximately kept in BB()) for p up to 0.9 (p' 0.1), while the distortion we need clearly happens near the border: outermost isoline e (0.001, 0.999) (plotted with bigger width) is smoothly and substantially "moved" within the t-self as decreases, guaranteeing good readability and coding convenience (see text).
onl
B
B
hardsare
301
B
B
Figure I: Top pane: comparison hetween Poincaré disk embedding and its t-self for t O for an MDT learned on UCI online beapee_shopprssitteeeee intenniinn (left) and twitter (right). On the left panel, we have not plotted the boosting coefficients information. The isoline distinguished in magenta is the big width one Table XIII Note the difference (non-linear) distortion created in the t-self, which the central part just enjoys scaling. Bottom pane: stark differences in visualization between IB and the t-self do not just appear initially: they can appear at any iteration. We display two MDTs learned on UCI hardware two different iterations (indicated). It would be very hard differentiate the best leaves from just the Poincaré disk embedding, while it becomes obvious from the t-self (orange arrows). Note also the set of red nodes in the Poincaré disk for t 70 that mistakenly look aligned, but not in the I-self (blue rectangle). See text for details.
be misjudged as equivalent from alone (orange rectangle) but there is little double from ( that one of them, which crosses the p. (0.001, 0.009) isoline, is in fact much better than the others. When many subtrees seem to be aggregating near the border as buzz_in_soci att_______iila_mddii_...... aediamimmmda____maaaaaa differences appear the t-self: the hest subtrees rectangles). In between, the t-self makes a much more visible ordering between the best nodes and subtrees, compared to Poincaré disk. hardware demonstrate that such very good nodes that are hard to differentiate from the others in B can appear at any iteration.
stark
immediately spotted from the t-self (orange
31

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims are supported by both the theoretical results presented in the paper or experimental exploration.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
32

---

Limitations of the proposed embedding method are discussed. For instance, the requirement of only embedding the MDT are discussed. Impacts on classification due to this are explored in the "MDT vs DT classification" discussion, with further details in Appendix. last paragraph in the conclusion also discusses limitations.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Assumptions are detailed in formal results and full proofs are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
33

---

Detailed settings of constructing visualizations are presented. Code provided with an example public domain and a resource file README.1x for quick testing and validation.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code provided with an example public domain and resource file + README.xxt for quick testing and validation.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (nttps://nips.. cc/
Suuiides/ooeSubmissttt
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
34

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Details are provided in main text and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error is stated whenever suitable. Multiple results are presented for visualiza- tions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA].
Justification:
The code runs any standard computer.
35

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research of the paper follows the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impact and societal impact is briefly mentioned throughout the main text, with further discussion presented in an Appendix section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
36

---

Answer:
[NA]
Justification:
No release of data or models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
Appropriate credit referencesse are provided. Licenses listed in Appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets provided.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
37

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: No research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
38


Reviews:
[{"review_id": "rslHWsw3fS", "reviewer": "Reviewer_t8YB", "summary": "This paper presents an approach to embed the models (not data) into hyperbolic spaces. Hyperbolic spaces are especially suited for embeddings of hierarchies, but recent works have so far focused on embeddings of data.\nThe paper presents a framework consisting of measures to estimate the confidence of model's predictions and the scheme to embed the decision tree model into hyperbolic space together with the postprocessing procedure. The authors evaluate the proposed approach and notice that more confident and accurate predictions tend to be closer to the border of the Poincare ball.", "questions": "- A small remark: the most part of plots use red/green colours which can be challenging to distinguish for colour blind people. If it is possible, I would encourage authors to change the colour scheme or at least keep that in mind for future submissions. \n- The sentence in line 697 is unclear.\n\nMy main concern regarding the paper is related to the difficulties while evaluating the performance of the approach and it mainly justifies my rating. However, I am open for discussions with the authors.", "limitations": "The authors have adequately addressed the limitations. I see no potential negative societal impact of their work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written.\n- The approach appears very original to me, I am not familiar with the methods similar to the presented work. I also find the angle of embedding the models (and not the data) interesting and non-trivial. \n- The approach is well motivated and backed up by theoretical explanations provided by authors. The mathematical justifications are rigorously presented in Appendix.\n- In my opinion, the methods for embeddings of discrete data into hyperbolic space is less covered in the literature, so I believe that adding more methods covering this topic would benefit the community. The authors show how exactly their approach differs from a baseline by Sarkar.", "weaknesses": "- Some visualizations are hard to interpret. Please also see a remark in 'Questions'.\n- In my opinion, the paper would have benefited from a more clear presentation of quantitative results. E.g.,  there is a discussion of the embedding error in Section 6, but the numbers listed in the text seem not to cover all the datasets considered in the analysis.\nAnother table that seem to provide quantitative results is Table II in Appendix showing difference between DT and MDT (its reduced hyperbolic counterpart). Again, I find it challenging to interpret.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for noting that our approach is “[...] *very original* [...]” – it is indeed the first of its kind and we believe the problem we address is of broad importance. We also appreciate the comments on the formal part of our approach.\n\n## strengths\n\n> In my opinion, the methods for embeddings of discrete data into hyperbolic space is less covered in the literature, [...] adding more methods covering this topic [...]\n\n[t8YB-A] This is an excellent observation that we did not think about and that would surely make our contribution stronger: the approaches [8][11] rely on *real* data embeddings, making it tricky to embed discrete or categorical data. Because we embed models that are naturally fit to handling any kind of numerical / nominal data, our technique encompasses any kind of such data.\n\n## weaknesses\n\n> “[...] In my opinion, the paper would have benefited from a more clear presentation of quantitative results [...]”\n\n[t8YB-B] A fair comment, due to the fact that we mainly focuses on the “visualization part”; fortunately, this can easily be addressed as each visualization also includes an error term: we propose to summarize (mean $\\pm$ std dev) the errors ($\\rho$) made on each dataset, in a separate table, that would hopefully fit in the +1 page allocated for camera-ready. \n\n[t8YB-C] Reading this comment from the reviewer, we came up with another idea to test one of the benefits of hyperbolic embeddings to our setting, which would be trivial to code and run: it is well known that, as points move closer to the border, the hyperbolic distance $d_{B}(\\textbf{z}, \\textbf{z}’)/2$ approaches the average $(d_{B}(\\textbf{z}, \\textbf{0})+d_{B}(\\textbf{z}’, \\textbf{0}))/2$ (see [29] Figure 1). For our application, the benefit is substantial: the expectation is indeed the expectation of absolute confidences of the two nodes mapped on $\\textbf{z}$ and $\\textbf{z}’$ (the nodes’ quality) and what the property says is that it can be approximated by just looking locally at (half) the distance between the two points (in terms of visualization, think zooming on a part of the disk or having a hovering tool providing the information of this distance “on the fly”). We would be happy to compute a discrepancy between the two terms above, as a function of the localization of the nodes on the disk.\n\n## questions\n\n> “[...] A small remark: the most part of plots use red/green colours which can be challenging to distinguish for colour blind people. [...]”\n\nThe reviewer is absolutely right: we propose, in addition to the color, to differentiate using shapes (e.g. circle for red, square for green, for all nodes). This would be a one-liner change in the code.\n\n> “[...] The sentence in line 697 is unclear. [...]”\n\nOur experiments involve a 10-folds stratified cross validation (L672) to compute average errors $\\rho$. In our visualizations, we can only show one of the ten models. In the spirit of fairness, we display the model learned on the first fold (instead of e.g. picking the best model out of the ten). This is what is summarized in L697 and we would be happy to be more specific.\n\n> “[...] My main concern regarding the paper is related to the difficulties while evaluating the performance of the approach and it mainly justifies my rating. However, I am open for discussions with the authors.[...]”\n\nWe are glad the reviewer signals their openness for discussion. Regarding evaluation, we hope that the suggestion we make in [t8YB-B][t8YB-C] would lead to a purely quantitative evaluation of our embeddings that would satisfy the reviewer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i0YC14SIXx", "reviewer": "Reviewer_Ry4X", "summary": "This paper introduces a method to embed decision trees into hyperbolic space, linking class probability estimation with hyperbolic distances. It proposes extracting monotonic subtrees and using a \"tempered\" integral for better visualization. The approach aims to enhance model interpretability and analysis without changing model performance.", "questions": "How does the method perform on regression trees as opposed to classification trees? Are there any significant differences in the representation or interpretation?\n\nHow does this hyperbolic embedding method perform on highly imbalanced datasets, particularly in visualizing minority class decision boundaries?", "limitations": "NA", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an approach to embedding decision trees in hyperbolic space, offering a novel perspective on model visualization and interpretation. By leveraging the properties of hyperbolic geometry, particularly the Poincaré disk model, the authors provide a mathematically grounded method for representing complex tree structures and confidence levels in a unified, visually intuitive manner, potentially enhancing our ability to analyze and understand sophisticated machine learning models.", "weaknesses": "**Limited empirical validation**: The authors present their method primarily on small-scale UCI datasets, which may not fully demonstrate the scalability and effectiveness of their approach on **large, real-world machine learning problems**. The lack of comparison with existing tree visualization techniques, such as those in popular packages like scikit-learn or XGBoost, makes it difficult to gauge the practical advantages of this method.\n\n**Potential computational overhead**: The paper does not adequately address the **computational complexity** of transforming decision trees into their hyperbolic representations. For large ensembles or deep trees, this transformation could potentially introduce significant overhead, which might limit its applicability in **time-sensitive or resource-constrained environments**.\n\n**Ambiguity in interpretability gains**: While the authors claim improved interpretability, they do not provide a **quantitative measure or user study** to substantiate this claim. It remains unclear whether domain experts or data scientists would find these hyperbolic representations more intuitive or actionable than traditional tree visualizations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing a uniform “good” rating on all three paper metrics, highlighting that our approach is novel, our method is mathematically grounded and visualization is intuitive.\n\n## weaknesses \n\n> The authors present their method primarily on small-scale UCI datasets [...]  lack of comparison with existing tree visualization techniques, [...] scikit-learn or XGBoost, \n\n[Ry4X-A] We believe there is a misunderstanding here, see [ALL-2] above. The software suggested work in the same way as the one we quote in our footnote ”*” (bottom page 2). It is not a criticism: all softwares proceeds in the same way – embed a graph (binary rooted tree), superimpose the classification information. The only differences between softwares are “cosmetic” details to represent the full models: use charts or pies or boxes to represent local class probabilities, add text / color, etc. . We write cosmetic because it all end up to the same conclusion when one realizes what we write in [ALL-1]: it is largely impossible to represent a tree with 200+ nodes (that we use in our experiments) and get any meaningful information of how classification changes as we descend the tree, how nodes / leaves in different parts of the tree compare in term of confidence, etc . Monotonic Decision Trees (that we introduce) do not solve the problem because the issue is not on the “cosmetic” details (that is why we use the term “cosmetic”): the issue is that the backbone comes from the graph theory representation and is suboptimal to grasp models that big when applied to ML. We believe the hyperbolic embedding is very helpful to address the issue. We refer the reviewer to the attached file above to see for themselves.\n\n> [...] Potential computational overhead: The paper does not adequately address the computational complexity of transforming decision trees into their hyperbolic representations. [...]\n\n[Ry4X-B] We would be happy to add in the camera-ready (using the +1 page) the fact that the complexity of computing the MDT is linear in the size of the tree (DT: see algorithm getMDT in our paper; class method PROCESS_TREE_GRAPHS in file MonotonicTreeGraph.java) and our variant of Sarkar’s algorithm is a cheap as Sarkar’s, so there is *no* computational overhead compared to a standard visualization of a DT.\n\n> [...] Ambiguity in interpretability gains: While the authors claim improved interpretability, they do not provide a quantitative measure or user study [...]\n\n[Ry4X-C] While we hope that our argument in [ALL-2] makes the case for “not small” models at least, we emphasize that the whole trend of hyperbolic geometry in ML has been built from the *mathematical properties and benefits* of tree/hierarchical representations in the model. All the papers that we cite (either published in ICML or NeurIPS) make this case. We quote, in order of our reference numbering, the first ones:\n\n[6]: “[...]  hyperbolic embeddings [...] are more aligned with the geometry of trees than standard Euclidean embeddings [...]”;\n\n[7]: “[...] Learning representations of data in hyperbolic spaces has recently attracted important interest in Machine Learning (ML) due to their ability to represent hierarchical data with high fidelity in low dimensions [...]”\n\n[8]: “[...] The adoption of hyperbolic geometry for graph embeddings has sparked a vibrant and rapidly growing body of machine learning research [...] driven by the compelling advantages offered by hyperbolic spaces, particularly in capturing hierarchical and tree-like structures [...]”\n\n(and so on) The motivation for our approach is no different from these.\n\n## questions\n\n> [...] How does the method perform on regression trees as opposed to classification trees? [...]\n\nOne would have to change the first step (the node embedding) of our approach because the losses we use are fundamentally for supervised classification. Then, the rest could be just about the same because of the similar tree structure (getMDT, modified Sarkar).\n\n> [...] How does this hyperbolic embedding method perform on highly imbalanced datasets, particularly in visualizing minority class decision boundaries? [...]\n\nWe suspect there is a misunderstanding here: we do not represent data but a model; therefore there is no such boundaries because there is no “training point cloud” mapped. **However** there is an effect of class imbalance, which is to (predictably) move the root of the tree far from the center of the disk. Thus class imbalance can be spotted “at lightning speed”, see L273-L274.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uFLUb96eqy", "reviewer": "Reviewer_j9dG", "summary": "This work proposes a hyperbolic implementation of decision tree models, providing a link providing a link between class probability estimation and hyperbolic distances. In addition the authors identify key objectives/criteria for enabling “clean” hyperbolic embeddings, which is achieved by maintaining monotonic part of the tree thus avoiding shared root nodes at the origin of the poincare ball. The authors also address the inherent collapse of embeddings to the boundary of the ball by the introduction of a t-self method which also improved the qualitative visualisations of embeddings on the poincare ball.", "questions": "- On Line 60 you state insist that these models do not represent models in hyperbolic space, why? This needs elaborating.\n- How does the proposed method compare to Euclidean models? What is the practical benefit of employing your method?\n- You suggest the t-self may be useful as a standard encoding in hyperbolic space, have you made any empirical findings to support this?\n- The limitations of only embedding the monotonic DT are not discussed? It may assist for clean embeddings but what are the drawbacks?", "limitations": "Some limitations are questioned above, while broader impacts of the work are addressed.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- This work addresses many fundamental issues in hyperbolic learning presenting some original concepts linking class probability estimation and hyperbolic distances and addressing boundary collapse, thus presenting an interesting direction and findings for the community.\n- The proposed t-self method addresses a key problem across hyperbolic distance metrics that seemingly helps avoid collapse to the boundary of the ball.\n- Significant theoretical proofs are provided to support many of the claims and assumptions of the work.", "weaknesses": "- It is not clear that the statement made that supervised models are not embedded in hyperbolic space. Some examples of fully hyperbolic classification models include [1,2]. I may be misunderstanding, therefore, correct me if so. In addition, to clarify this point, the authors could provide some evidence to back up this claim as currently it is weak. In addition, supervised hyperbolic geometry is well established which the authors claim otherwise, and hyperbolic decision trees have previously been proposed in [2]\n- The presentation and writing overall is generally poor and hard to follow, notably the bold headings in section 4. The use of mathematical notation is poorly integrated in some settings and largely impedes the readability of the work.\n- The experimental results are purely demonstrated as visual comparisons, and no quantitive measures are presented.\n- Arguably the t-self method you propose to improve visualisation is a distance metric that helps avoid collapse to the boundary of the ball. While helping visualisation should not be positioned in the contributions as a method for visualisation.\n- Experimental/implementation settings are omitted from the work limiting the ability of replication. \n\n\nMinor:\n- The presentation is poor in many places notably the experimental analysis, while the style changes significantly throughout the work.\n- The related work is missing key works and is generally vague in relation to the problem statement.\nGeneral grammar and writing:\n- Line 61 \n- Line 106 is this incomplete? \n- $d^*$ and $d_*$ are used interchangeably.\n\n[1] Doorenbos, L., Márquez-Neila, P., Sznitman, R. and Mettes, P., 2023. Hyperbolic Random Forests. arXiv preprint arXiv:2308.13279.\n\n[2] Chlenski, P., Turok, E., Moretti, A. and Pe'er, I., 2023. Fast hyperboloid decision tree algorithms. arXiv preprint arXiv:2310.13841.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate the comment that our method “[...] addresses many fundamental issues in hyperbolic learning presenting [...]”. This is a strong selling point for our method, which unfortunately comes with a challenging presentation problem. We hope our rebuttal helps in clarifying views and questions.\n\nWe also really appreciate the comment that one part of our contribution (the t-self) indeed addresses a general, “[...] key problem across hyperbolic distance metrics that seemingly helps avoid collapse to the boundary of the ball [...]” (indeed, it avoids collapse).\n\n## weaknesses\n\n> “[...] It is not clear [...] Some examples of fully hyperbolic classification models include [1,2]. I may be misunderstanding [...]”\n\n[j9dG-A] Notations and naming can be misleading. Indeed, there is a misunderstanding: please have a look at Figure 4 in [1] and Figure 2 in [2]. The legend makes it clear that it is **data** which is indeed embedded. Note that some details of the model can be embedded as well, like decision boundaries (Figure 2 in [2]), but at the end of the day, the hyperbolic space is the data’s in those papers, while it is the model’s in ours. We also refer to [t8YB-A] for a limitation of [1][2] in the data space.\n\n> “[...] The presentation and writing overall [,,,] impedes the readability of the work. [...]”\n\nWe apologize for the inconvenience. We believe that we can make use of the +1 page in the camera ready to expand some of the technical material’s explanation in the main file. \n\n> “[...] The experimental results are purely demonstrated as visual comparisons, and no quantitive measures are presented. [...]”\n\nThere are actually qualitative measures of the quality of the embeddings displayed on each, the error $\\rho$, but we do understand the point made, to perhaps instead (or in addition) make a separate quantitative analysis. We have made such a proposal, that would be straightforward to implement using our code, in [t8YB-B][t8YB-C].\n\n> “[...] Arguably the t-self method you propose to improve visualisation [...] should not be positioned in the contributions as a method for visualisation.[...]”\n\nIt is right, it is a transformation that can be used in *any* use of the Poincaré  disk model. We were hoping that our remarks in L305-L308 would help grasp this. To improve this part of the readability, we would be happy to include part of it in the introduction as well, using part of the +1 page camera ready.\n\n> “[...]Experimental/implementation settings are omitted from the work limiting the ability of replication[...]”\n\nWe strongly disagree on this point: the code we provide contains a UCI dataset and the resource file to run experiments exactly as we did for this dataset (all is explained in the README). Using another dataset requires changing just one line in the resource file (“@PREFIX,abalone”).\n\nOn minor comments, we appreciate the suggestions and we are sorry that the small space allocated probably caused potential misunderstandings (ex: L106 goes with L113 for the bold face statement), we are confident that the +1 page can allow us to improve readability.\n\n## questions\n\n> “[...]On Line 60 you state insist that these models do not represent models in hyperbolic space[...]”\n\nThis is related to [j9dG-A] above and we refer to it. In short, in all those papers,  the hyperbolic space is the data’s in those papers, while it is the model’s in ours.\n\n> “[...]How does the proposed method compare to Euclidean models? [...]”\n\nThe substantial benefits come from the properties of the hyperbolic space, and is common to all papers using such geometric models. See for example [t8YB-C] for a concrete example, that was also part of a paper [29], albeit in a different context. There is, however, an additional benefit in our case, which comes from the singular property that there is a relationship between the *link function of the log loss* and the Poincaré distance which makes a tree node embedding not just natural: it provides a direct visualization of classification confidences in the disk ! (though simple, we have not seen this connection elsewhere).\n\n> “[...]You suggest the t-self may be useful as a standard encoding in hyperbolic space, have you made any empirical findings to support this? [...]”\n\nExcellent question: at the moment, with our paper cramming so much content, the sole observation that we have is experimental in our context (the remark appears in L292-L293), and its usefulness hopefully becomes clear in Table 1. We use the conditional (*might*) instead of *may* because we indeed suspect a much more thorough analysis can be carried out, which would require not just digging in experiments but also in various encodings and against additional geometric properties of the Poincaré disk model, to establish the potential for a broader standard. This is out of the scope (and size !) of our paper.\n\n> “[...]The limitations of only embedding the monotonic DT are not discussed? It may assist for clean embeddings but what are the drawbacks?[...]”\n\nAs we understand the question, we see three interpretations for “limitations” that specialize in three different questions:\n\n- *Are there limitations in terms of accuracy for models (MDT vs DT) ?* We kindly refer the reviewer to Appendix Section D.II Line 689 and Table II. Here the effects of reducing a DT to its monotonic form is explored. Here we find that in most cases the accuracy / behaviour of the MDTs are similar in performance to the original DTs. Note that we suspect there is a *formal rationale* to this observation, but it is out of the scope of this paper.\n\n- *Are there limitations in terms of computational cost for creating the MDT ?* The answer is clearly no, see [Ry4X-B].\n\n- *Are there limitations for the user, any pricey “entry ticket” for using MDTs (vs DTs) ?* This question is about the easiness of using MDTs vs DTs. Knowing DTs, the entry ticket to get to MDTs would arguably be minimal compared e.g. to get to kernels.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ukmY14UOqv", "reviewer": "Reviewer_5SUh", "summary": "Summary\nThis paper proposes a novel approach to embed supervised models, specifically decision trees and their ensembles, in hyperbolic space using the Poincaré disk model. The paper contributes three main advancements: linking loss functions for class probability estimation to hyperbolic embeddings, providing a method for unambiguous embedding of decision trees by extracting their monotonic subtrees, and introducing a tempered integral to improve the visualization and numerical accuracy of embeddings near the border of the Poincaré disk.\n\nStrengths\n1. The approach to embedding supervised models, particularly decision trees, in hyperbolic space is novel. The focus on supervised models rather than just the data is a significant contribution.\n2. This paper provides a robust theoretical framework linking class probability estimation losses with hyperbolic embeddings and introducing the concept of monotonic decision trees (MDTs) for clean embeddings.\n3. The introduction of the tempered integral to handle numerical issues near the border of the Poincaré disk demonstrates a deep understanding of the challenges in hyperbolic geometry and offers a principled solution.\n4. This paper is well-structured, with clear definitions, thorough explanations of the proposed methods, and detailed experimental results. The figures and tables effectively support the textual descriptions.\n\nWeaknesses and comments\n1. The theoretical aspects, particularly the tempered integral and its application, might be challenging for practitioners to fully grasp and implement without further guidance or examples.\n2. While the paper demonstrates the applicability to decision trees and their ensembles, it would benefit from a discussion on how the methods could be adapted or extended to other types of supervised models.\n3. While the experiments are thorough, including more diverse use cases, such as different types of datasets or real-world applications, would strengthen the demonstration of the method's versatility and robustness.\n4. The related work section is comprehensive, covering both hyperbolic embeddings and supervised learning, which helps to contextualize the contributions.\n5. The methodology sections are detailed and well-explained. The separation into three main contributions helps in understanding the different components of the proposed approach.", "questions": "N/A", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths\n1. The approach to embedding supervised models, particularly decision trees, in hyperbolic space is novel. The focus on supervised models rather than just the data is a significant contribution.\n2. This paper provides a robust theoretical framework linking class probability estimation losses with hyperbolic embeddings and introducing the concept of monotonic decision trees (MDTs) for clean embeddings.\n3. The introduction of the tempered integral to handle numerical issues near the border of the Poincaré disk demonstrates a deep understanding of the challenges in hyperbolic geometry and offers a principled solution.\n4. This paper is well-structured, with clear definitions, thorough explanations of the proposed methods, and detailed experimental results. The figures and tables effectively support the textual descriptions.", "weaknesses": "Weaknesses and comments\n1. The theoretical aspects, particularly the tempered integral and its application, might be challenging for practitioners to fully grasp and implement without further guidance or examples.\n2. While the paper demonstrates the applicability to decision trees and their ensembles, it would benefit from a discussion on how the methods could be adapted or extended to other types of supervised models.\n3. While the experiments are thorough, including more diverse use cases, such as different types of datasets or real-world applications, would strengthen the demonstration of the method's versatility and robustness.\n4. The related work section is comprehensive, covering both hyperbolic embeddings and supervised learning, which helps to contextualize the contributions.\n5. The methodology sections are detailed and well-explained. The separation into three main contributions helps in understanding the different components of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for a review whose strengths (1-3) perfectly capture our key contributions, in particular the fact that we embed *models*, not *data*.\n\nWe would like to highlight the common point to all three weaknesses mentioned (1-3): they all come down to putting substantial material in a space that was extremely constrained at submission time. **Fortunately**, there would be a +1 page in the camera ready. We believe we could at least partially cover (3) on more diverse use cases, assuming we can label our proposals [t8YB-B][t8YB-C] as such (see above).\n\nWeakness (2) (extension to other supervised models) could be partially addressed in conclusion: for example, one can use our technique to embed branching programs *without any modification* (apart from a few tweaks in the Sarkar part, because we replace binary trees by directed acyclic graphs).\n\nWeakness (1) would require a lot more space to deliver the reviewer’s suggestions, we are afraid. We would propose to use part of the +1 space to slightly expand Section 5, making a better connection with Figure 3 (right).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## To all reviewers

[ALL-1] We would like to thank all reviewers for granting unanimous approval on our paper from the “soundness” (“good” or “excellent”) and “contribution” (all “good”) standpoints. We understand the presentation is the bottleneck of our paper, which we attribute to the fact that we are the first to tackle **model** embedding (new problem) and that a full-fledged solution requires several technical steps (mathematical background). We hope the reviewers concur that the +1 page for the camera ready can be used to address those concerns (details in each rebuttal).

[ALL-2] We have had requests to compare graphical representations with other software, such as scikit-learn [Ry4X-A]. There is one crucial thing to remark: all those packages have the same blueprint as the one we mention in our footnote "*" (bottom of page 2): they adopt a graph-based representation. For our purpose, this would just map nodes’ confidences on top of the vertices of a (rooted binary tree) graph. The issue that is relevant to our paper is **not** whether the DT learned is accurate, but whether the representation easily brings meaning. We attach to this general rebuttal a pdf file representing, on domain online_shoppers_intentions, three *representations* of one tree learned during one CV fold (not necessarily the same for each representation):
- We first represent (left) a decision tree using an ASCII-”art” representation of the rooted binary tree. We chose this because the software we submitted can actually output these at no cost and we believe the picture is telling **and highlight the same issue as would be observed from all public packages**: it is useless to represent a tree (graph) with **200+** nodes to grasp how classification is achieved / changes throughout the tree, compare node predictions in different part of the tree, etc. (with our ASCII representation, the full graph would have 200+ lines; in scikit-learn, it would have 200+ boxes for nodes, etc.).
- We then represent (middle) a monotonic DT using the same ASCII-art formalism (this one representation is not included in our software but is easy to implement). The lesson ? The graph is much smaller than for the DT **but** it is still hard to grasp when it can have dozens of nodes and as many prediction confidences = real numbers to mix in the interpretation.
- We finally present (right) our hyperbolic embedding of the MDT. We believe the reviewers will concur that the “global picture” of the MDT is a “snapshot” clean and interpretable in a way that would not have been possible for both other representations.
Should our paper be accepted, we would be happy to make this point very clear in the camera-ready by using a slick version of our attached file, using part of the +1 page.

Tags are spread throughout rebuttals for easy search / referencing (e.g. [ALL-1]).

Meta-review: This paper introduces a method to embed decision tree models into hyperbolic spaces, particularly using the Poincaré ball model. The approach links class probability estimation with hyperbolic distances, focusing on enhancing interpretability without altering model performance. Key techniques include extracting monotonic subtrees and introducing a "tempered" integral to improve visualization and prevent embeddings from collapsing to the ball's boundary. The authors demonstrate that more confident predictions tend to be closer to the Poincaré ball's border, providing a novel way to analyze and visualize model decisions within hyperbolic geometry.

The reviewers agree that the approach is both interesting and useful. From an algorithmic perspective, this work is novel and would make a strong contribution to NeurIPS. However, one of the main concerns is the experimental evaluation. While the paper includes numerous experiments on benchmark datasets, it lacks experiments with more realistic data. Incorporating real-world datasets would greatly enhance the paper’s impact, and I strongly encourage the authors to add such experiments. Despite this, I find the algorithm to be innovative and compelling, and I recommend the paper for acceptance.
Decision: Accept (poster)