Title: Hyperbolic Embeddings of Supervised Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Hyperbolic geometry

supervised model embedding

decision trees

boosting
Abstract: Models of hyperbolic geometry have been successfully used in ML for two main tasks: embedding *models* in unsupervised learning (*e.g.* hierarchies) and embedding *data*. 
To our knowledge, there are no approaches that provide embeddings for supervised models; even when hyperbolic geometry provides convenient properties for expressing popular hypothesis classes, such as decision trees (and ensembles).
In this paper, we propose a full-fledged solution to the problem in three independent contributions. The first linking the theory of losses for class probability estimation to hyperbolic embeddings in Poincar\'e disk model. The second resolving an issue for a clean, unambiguous embedding of (ensembles of) decision trees in this model. The third showing how to smoothly tweak the Poincar\'e hyperbolic distance to improve its encoding and visualization properties near the border of the disk, a crucial region for our application, while keeping hyperbolicity.
This last step has substantial independent interest as it is grounded in a generalization of Leibniz-Newton's fundamental Theorem of calculus.

Hyperbolic Embeddings of Supervised Models

Abstract

Models of hyperbolic geometry have been successfully used in ML for two main tasks: embedding models in unsupervised learning ( e.g. hierarchies) and embedding data . To our knowledge, there are no approaches that provide embeddings for supervised models; even when hyperbolic geometry provides convenient properties for expressing popular hypothesis classes, such as decision trees (and ensembles). In this paper, we propose a full-fledged solution to the problem in three independent contributions. The first linking the theory of losses for class probability estimation to hyperbolic embeddings in Poincaré disk model. The second resolving an issue for an easily interpretable, unambiguous embedding of (ensembles of) decision trees in this model. The third showing how to smoothly tweak the Poincaré hyperbolic distance to improve its encoding and visualization properties near the border of the disk, a crucial region for our application, while keeping hyperbolicity. This last step has substantial independent interest as it is grounded in a generalization of Leibniz-Newton's fundamental Theorem of calculus.

1 Introduction

Models of hyperbolic geometry have been successfully used to embed hierarchies, i.e. tree-based structures [14, 20, 38, 36]. Through the property of low-distortion hyperbolic embeddings [30], symbolic (hierarchies) and numeric (quality metrics) properties of unsupervised learning models can be represented in an accurate and interpretable manner [29]. When it comes to supervised learning, the current trend involves embedding data in a hyperbolic space, with models trained on the now hyperbolic data [9, 15, 8, 11]. It is important to note that none of these supervised methods embed models . Indeed, the focus of the prior literature is to learn supervised models from hyperbolic data, rather than representing supervised models in hyperbolic geometry. This is in stark contrast to the unsupervised usage described, where the (tree-based) structural properties of unsupervised models are directly exploited for good embeddings. This hints at benefits which can be used in the supervised case, especially for popular supervised models which are (structurally) tree-based.

Our paper proposes a full-fledged solution for this problem, in three independent contributions.

The first focuses on the numerical part (quality metrics) of the embedding and its link with supervised losses: to provide a natural way of embedding classification and the confidence of prediction, we show a link between training with the log-loss (posterior estimation) or logistic loss (real-valued classification) and hyperbolic distances computation in the Poincaré disk.

The second focuses on the symbolic part (hierarchies) of the embeddings for a popular kind of supervised of models: decision trees (and ensembles). Unlike for unsupervised learning, we show

Manfred K. Warmuth

that getting an unambiguous embedding of a decision tree requires post-processing the model. Our solution extracts its monotonic sub-tree via a new class of supervised models we introduce, monotonic decision trees. This is also convenient for explainability purposes [32].

The third and most technical one focuses on visualization and the accuracy of the numerical encoding in the Poincaré disk model. The more accurate / confident is a supervised prediction, the closer to the border of the disk it is represented. Thus, the best models will have their best predictions squeezed close to the border. In addition to being suboptimal for visualization, this region is also documented for being numerically error-prone for hyperbolic distance computation [29]. Two general fixes currently exist: encoding values with (exponentially) more bits or utilizing a trick from Riemannian geometry [19]. As neither is satisfactory for our usage, we propose a third, principled route: like other distances, Poincaré distance is integral. We generalize Leibniz-Newton's fundamental Theorem of calculus using a generalization of classical arithmetic [22]. This generalized 'tempered' integral provides a parameter to smoothly alter the properties of the 'classical' integral. When defining the Poincarè distance, the tempering controls the hyperbolic constant of the embedding whilst also improving the visualization and numeric accuracy of the embedded models. The generalization of integrals to distances has independent interest as many application in ML rely on integral based distances and distortions, i.e., Bregman divergences, f -divergences, integral probability metrics, etc.

Experiments are provided on readily available domains, and all proofs, additional results and additional experiments are given in an Appendix.

2 Related work

Models of hyperbolic geometry have been mainly useful to embed hierarchies, i.e. tree-based structures [14, 20, 38, 36], with a sustained emphasis on coding size and numerical accuracy [19, 29, 37]. In unsupervised learning and clustering, some applications have sought a simple representation of data on the form of a tree or via hyperbolic projections [7, 6, 17, 34]. Approaches dealing with supervised learning assumes the data lies in a hyperbolic space: the output visualized is primarily an embedding of the data itself, with additional details linked to classification of secondary importance, either support vector machines [9], neural nets, logistic regression [15], or (ensembles of) decision trees [8, 11]. We insist on the fact that the aforementioned methods do not represent the models in the hyperbolic space, even when those models tree-shaped. The question of embedding classifiers is potentially important to improve the state of the art visualization: in the case of decision trees, popular packages stick to a topological layer (the tree graph) to which various additional information about classification are superimposed but without principled links to the 'embedding' * .

3 Basic definitions

Training a supervised model starts with a set (sample) of examples, each of which is a pair p x , y q , where x P X (a domain ) and y P tGLYPH<1> 1 , 1 u (labels or classes). A decision tree (DT) consists of a binary rooted tree H , where at each internal nodes, the two outgoing arcs define a Boolean test over observation variables (see Figure 1, center, for an example); N p H q is the set of nodes of H , Λ p H q GLYPH<132> N p H q is the set of leaf nodes of H .

The log-loss is best introduced in the context of the theory of losses for class probability estimation (CPE) [28]. We follow the notations of [18]: A CPE loss function, glyph[lscript] : Y GLYPH<2> r 0 , 1 s GLYPH<209> R , is

where J . K are Iverson brackets [16]. Functions glyph[lscript] 1 , glyph[lscript] GLYPH<1> 1 are called partial losses. A CPE loss is symmetric when glyph[lscript] 1 p p q GLYPH<16> glyph[lscript] GLYPH<1> 1 p 1 GLYPH<1> p q , @ p P r 0 , 1 s . The log-loss is a symmetric CPE loss with glyph[lscript] LOG 1 p p q . GLYPH<16> GLYPH<1> log p p q . The goal of learning using a CPE loss is to optimize the Bayes risk, L p p q . GLYPH<16> inf π E Y GLYPH<18> p glyph[lscript] p Y , π q . In the case of the log-loss, it is the criterion used to learn DTs in C4.5 [26]:

Models like DTs predict an empirical posterior ˆ Pr r Y GLYPH<16> 1 | X s at the leaves: for an observation x reaching leaf λ , the posterior prediction is the local relative proportion of positive examples in leaf λ , as estimated by the training sample (noted p GLYPH<0> λ ). There exists a duality between CPE classification

Decision Tree

H

Monotonic Decision Tree

H

1

p（n1)=.5

n1

Bothniandngare

embedded atthe

Shel/Weight <=0.24

L#

=-0.01

#1

= -0.01

center

0.210.8

0.110.9

true

false

absolute confidence|ul

Shell_Weight <= 0.24

Shell_Weight >0.24

p(n2)=.9

m2

0.310.7

-1.07

#2

#3

1.27

-1.07{#5}

#2

#3

#11}1.27

9.54

0.410.6

Posterior

isolines

Sex in()

ShelLWeight<= 0.35

1

Sex in (1)

Shell_Weight > 0.35

p（n3)=.5

m3

-2.04

#4

#5

#6

#7

-2.04{#9}

#4

#10

#7

Shell_Weight <=/0.14

1/

-0.33

ShuckedWeighit <= 0.35

0.80

2.38

Shell_Weight <= 0.14

2.05

(ShelLWeight <= 0.35)

2.38

4

Gu

14

#8

#9

#10

#11

#8

(Shucked_Weight <= 0.35)

AND

p(n4)=.05

p(ns)=.9

-3.11

-0.95

2.05

0.62

-3.11

and the real-valued classification setting familiar to e.g. deep learning: optimizing Bayes risk for the posterior is equivalent to minimizing its convex surrogate using the canonical link of the loss to compute real classification [24, Theorem 1]. The canonical link of the log-loss, ψ LOG : r 0 , 1 s GLYPH<209> R is the famed inverse sigmoid, which has a very convenient form for our purpose,

Notably, the absolute value | ψ LOG p p q| is a confidence with the sign giving the predicted class. In the case of the log-loss, the convex surrogate is hugely popular in ML: it is the logistic loss.

We now introduce concepts from hyperbolic geometry, particularly those from the Poincaré disk model. Our definitions are simplified, a detailed account can be found in [5, 20]. The distance function d of a metric space is τ -hyperbolic for some τ ¥ 0 iff for any three geodesic curves γ 1 , γ 2 , γ 3 linking three points, there exists z such that max i d p z , γ i q / τ , where d p z , γ q . GLYPH<16> inf z 1 P γ d p z , z 1 q . A small hyperbolic constant guarantees thin triangles and embeddings of trees with good properties. The Poincaré disk model, B (negative curvature, GLYPH<1> 1 ) is a popular model of hyperbolic geometry with the hyperbolic distance between the origin and some z P B with Euclidean norm r . GLYPH<16> } z } being:

4 Posterior embedding in Poincaré disk and clean embeddings for DTs

Node embedding We exploit the similarity between log-loss confidences | ψ LOG p p q| and hyperbolic distances d B p z , 0 q when embedding classification models, similarity which is obvious from (3) and (4). In (3), we see that if r GLYPH<16> 0 , the prediction is as bad as a fair coin's. As r edges closer to 1 , prediction approaches maximum confidence. The connection with the Poincaré distance (4) is immediate: a natural embedding of a posterior prediction p is then a point z in the Poincaré disk B at radius r . GLYPH<16> } z } from the origin (3). The origin of Poincaré disk represents the worst possible confidence. As each leaf λ of a DT H corresponds to a posterior p GLYPH<0> λ , the leaves can be embedded as described. In addition, all nodes ν P N p H q in the tree also have corresponding posteriors, and thus can also be embedded identically to leaf nodes. It is also a good idea to embed them because, as classical DT induction algorithms first proceed by top-down induction, any node ν in a DT was a leaf at some point during training, and has a corresponding posterior. This apparently clean picture for node embedding however becomes messy as soon as we step to embedding full trees.

No clean embedding for a full DT A simple counterexample indeed demonstrates that a clean embedding of a decision tree is substantially trickier, see Figure 1 (left). In this pathological example, some distinct nodes (resp. arcs) of the DT H are embedded in the same node (resp. edge) in B : the depiction has no subgraph relationship to H and some embedded nodes cannot be distinguished between each other without additional information.

Getting a clean embedding: Monotonic Decision Trees To introduce our fix, we define three broad objectives for the embedding in B of DTs

Embedding objective

(A) Embed a tree-based model from H in B , which defines an injective mapping of the nodes of H and induces a subtree of H with each edge in B corresponding to a path in H ;

(B) Locally , each node ν of H gets embedded to some z ν such that r ν is close to } z ν } (3); (C) Globally , the whole embedding remains convenient and readable to compare, in terms of confidence in classification, different subtrees in the same tree, or even between different trees.

As we shall see later in this Section, this agenda also carries to embedding ensembles of DTs, thus including their leveraging coefficients, exploiting properties of boosting algorithms. For now, our solution for embedding a single DT that can satisfy (A-C) is simple in principle:

Embed the monotonic classification part of a DT ,

meaning for each path from the root to a leaf in H , we only embed the subsequence of nodes whose absolute confidences are strictly increasing. To do so, we replace the DT by a "close" path-monotonic approximation model using a new class of models we call Monotonic Decision Trees (MDT).

Definition 4.1. A Monotonic Decision Tree (MDT) is a rooted tree with a Boolean test labeling each arc and a real-valued prediction at each node. Any sequence of nodes in a path starting from the root is strictly monotonically increasing in absolute confidence. At any internal node, no two Boolean tests at its outgoing arcs can be simultaneously satisfied. The classification of an observation is obtained by the bottom-most node's prediction reachable from the root.

Algorithm 1 GETMDT( ν, b , ν 1 , I )

We introduce an algorithm, GETMDT, which takes as input a DT H and outputs an MDT H 1 , which approximates H via the following key property:

(M) For any observation x P X , the prediction H 1 p x q is equal to the prediction in the path followed by x in H of its deepest node in the strictly monotonic subsequence starting from the root of H .

Figure 1 (right) presents an example of MDT H 1 that would be built for some DT H (left) and satisfying (M) (unless observations have missing values, H 1 is unique). Figure 1 adopts some additional conventions to ease parsing of H 1 , (D1) and (D2) :

(D1) Some internal nodes of H 1 are also tagged with labels corresponding to the leaves of H . If a

node in H 1 is tagged with a label of one leaf λ of H , it indicates that examples reaching λ in the original H are being classified by H 1 's tagged node;

(D2) Arcs in H 1 have a width proportional to the number of boolean tests it takes to reach its tail from its head in H . A large width thus indicates a long path in H to improve classification confidence. To produce the MDT H 1 from DT H with Algorithm 1, after having initialized it to a root = single leaf, we just run

(we let p GLYPH<0> root . GLYPH<16> min t p GLYPH<0> root , 1 GLYPH<1> p GLYPH<0> root u , p GLYPH<0> root . GLYPH<16> max t p GLYPH<0> root , 1 GLYPH<1> p GLYPH<0> root u ). Upon finishing, the tree rooted at root p H 1 q is the MDT sought.

We complete the description of GETMDT: When a leaf of H does not have sufficient confidence and ends up being mapped to an internal node of the MDT H 1 , TAGDTLEAF is the procedure that tags this internal node with information from the leaf (see (D1) above). We consider a tag being just the name of the leaf (Figure 1, leaves #5 , 9 , 11 ), but other conventions can be adopted. The other two methods we use grow MDT H 1 by creating a new node via NEWNODE and adding a new arc between existing nodes via NEWARC. We easily check the following.

Theorem 1. H 1 built by GETMDT satisfies (M) with respect to DT H .

Property (M) is crucial to keep the predictions of the DT and the MDT close to each other: to each node of the MDT indeed corresponds a node in the DT with the same posterior prediction. Because the set of nodes of H 1 corresponds to a subset of nodes in H , predictions can locally change for some observations. This phenomenon is limited by two factors: (i) the set of leaves of the MDT corresponds to a set of leaves in the DT (thus, predictions do not change for the related observations), (ii) more often than not in our experiments, it is only the confidence that changes, not the sign of the prediction, which means that accuracies of the DT and its corresponding MDT are close to each other (Section 6). It is also worth remarking that the MDT H 1 always has the same depth as the DT H . Finally, any pruning of H is a subtree of H and its corresponding MDT is a subtree of the MDT of H . The aforementioned troubles to embed the DT in the Poincaré disk considering (A-C) do not exist anymore for the MDT because the best embeddings necessarily have all arcs going outwards in the Poincaré disk. The algorithm we use to embed the MDT in Poincaré disk is a simple alteration of Sarkar's embedding [30] which, because of the lack of space, we defer to the Appendix (Section C.VIII).

The quality of the embedding is obtained using an embedding error that takes into account all nodes:

where ψ ν refers to the relevant ψ LOG p p GLYPH<0> ν q in (3) (and z ν P B is its embedding in B ). Two example final representations are in Figure 2. Notice the small errors ρ in both cases.

Remark 1. Using a particular boosting algorithm to boost the logistic loss, we can as well represent the leveraging coefficients of a DT/MDT in a boosted combination, following a convention sketched in Figure 2 and explained at length, along with the boosting algorithm and its properties, in the Appendix (Section C.IX).

5 Smoothly altering integrals: T-calculus and the t-self of Poincaré disk

It is apparent from Figure 2 (right, lowest red pentagon), that when utilizing standard hyperbolic distances d B p z , 0 q (4), the best MDT nodes are embedded close to the border. In addition to obviously not being great for visualization, these nodes are at risk of having numeric error 'push' the nodes to the border B B , thereby giving a false depiction of infinite confidence. In this section, we provide alternative distances to prevent this phenomena. First, we quantify the numerical risk, which has been defined by the critical region where high numeric error can occur [19, 29].

Definition 5.1. A point x P B is said k -close to boundary B B if } x } GLYPH<16> 1 GLYPH<1> 10 GLYPH<1> k . It is said encodable iff } x } GLYPH<160> 1 in machine encoding (it is not 'pushed to the boundary').

Machine encoding constrains the maximum possible k : in the double-precision floating-point representation (Float64), k GLYPH<19> 16 [19]. In the case of Poincaré disk, the maximal distance d GLYPH<6> from the

Monotonic DT representation - UCI online_shoppers_intentions

[p: 5.46 %]

ratio of radii =

leveraging coefficient

in ensemble

(in absolute value)

node embedding error

isolines (posterior)

Monotonic Decision Tree

red / green = majority class

arc thickness = #tests in

DT to reach child node

(arcs going outwards in     )

(

j

=7)

j

= #boosting

iteration

Root not far from the center (A)

Non-purchases that look more

0.110.9

similar to purchases (A)

,Non-purchases that look

0.210.8

!“different”from purchases(A)

Tree arclabeledwithfeature

PageValues(B)

Specific“ruleforpurchase”that

canbespecializedwithmore

testsfromthepathandgets

strictlymonotonicallybetteras

we add them(C)

[p: 4.70 %]

0.1|0.9

0.2|0.8

0.3|0.7

0.4|0.6

Confidential + Proprietary

origin 0 that this authorizes (before numerical error 'warps' a point to the border) is a small affine order in k [19]:

Hence, in practice, only a ball of radius d GLYPH<6> GLYPH<19> 38 around the origin can be accurately represented. This is in deep contrast with Euclidean representation, where d GLYPH<6> GLYPH<16> Ω p 2 k q . We now provide a principled solution to changing d GLYPH<6> while keeping hyperbolicity, which relies on a crucial generalization of Leibniz-Newton's fundamental Theorem of calculus.

5.1 T-calculus

We let r n s . GLYPH<16> t 1 , 2 , ..., n u for n P N ¡ 0 . At the core of our generalization is the replacement of the addition by the tempered (or t-)addition [22], z ' t z 1 . GLYPH<16> z GLYPH<0> z 1 GLYPH<0> p 1 GLYPH<1> t q zz 1 , in the notion of Riemann sum. The additional term is a scaled saddle point curve zz 1 , positive when the sign of z and z 1 agree and negative otherwise. Hereafter, f is defined on an interval r a, b s . We define a generalization of Riemann integration to t-algebra (see [2] for a preliminary development which does not provide the generalization of Leibniz-Newton's fundamental Theorem of calculus) and for this objective, given an interval r a, b s and a division ∆ of this interval using n GLYPH<0> 1 reals x 0 . GLYPH<16> a GLYPH<160> x 1 GLYPH<160> ... GLYPH<160> x n GLYPH<1> 1 GLYPH<160> x n . GLYPH<16> b , we define the Riemann t -sum of f over r a, b s using ∆ , for a set of tags Ξ . GLYPH<16> t ξ i P r x i GLYPH<1> 1 , x i su i Pr n s ,

( S p 1 q ∆ p f q is the classical Riemann summation). Let s p ∆ q . GLYPH<16> max i x i GLYPH<1> x i GLYPH<1> 1 denote the step of division ∆ . The conditions for t -Riemann integration are the same as for t GLYPH<16> 1 .

Definition 5.2. Fix t P R . A function f is t -(Riemann) integrable over r a, b s iff D L P R such that @ ε ¡ 0 , D δ ¡ 0 , @ division ∆ with s p ∆ q GLYPH<160> δ and associated set of tags Ξ , GLYPH<7> GLYPH<7> GLYPH<7> S p t q ∆ p f q GLYPH<1> L GLYPH<7> GLYPH<7> GLYPH<7> GLYPH<160> ε . When this happens, we note

The case t GLYPH<16> 1 , Riemann integration, is denoted using classical notations. We now prove our first main results for this Section, namely the link between t -Riemann integration and Riemann integration.

Theorem 2. Any function is either t -Riemann integrable for all t P R simultaneously, or for none. In the former case, we have the relationship: ( log t . GLYPH<16> p z 1 GLYPH<1> t GLYPH<1> 1 q{p 1 GLYPH<1> t q for t GLYPH<24> 1 and log 1 . GLYPH<16> log )

For example, if t GLYPH<16> 0 , we get 1 GLYPH<0> p 0 q ‡ b a f p u q d t u GLYPH<16> exp ‡ b a f p u q d u , which is Volterra's integral [33, Theorem 5.5.11]. Theorem 2 is important because it allows to compute any t -Riemann integral from

the classical ( t GLYPH<16> 1 ) case. Classical Riemann integration and derivation are fundamental inverse operations. The classical derivation is sometimes called 'Euclidean derivative' [4]. We now elicit the notion of derivative, which is the 'inverse' of t -Riemann integration. Unsurprisingly, it generalizes Euclidean derivative. The Theorem stands as a generalization of the classical fundamental Theorem of calculus.

Theorem 3. Let z a t z 1 . GLYPH<16> z GLYPH<1> z 1 1 GLYPH<0>p 1 GLYPH<1> t q z 1 denote the tempered subtraction. When it exists, define

Suppose f t -Riemann integrable. Then, the function F defined by F p z q . GLYPH<16> p t q ‡ z a f p u q d t u is such that D t F GLYPH<16> f . We call F a t -primitive of f (which zeroes when z GLYPH<16> a ) and D t F the t -derivative of F .

The function g t (Figure 3) is key to many of our results; it has quite remarkable properties: in particular, it is strictly increasing for any t P R , strictly concave for any t ¡ 1 , strictly convex for any t GLYPH<160> 1 and such that sign p g t p z qq GLYPH<16> sign p z q , @ t P R . One can also note that D t g t p z q GLYPH<16> 1 . The Appendix proves these results (Sections C.I and C.II).

Remark 2. The Appendix also proves many more results showing how Theorems 2 and 3, and properties of t-calculus, naturally 'percolate" through many properties known for classical integration and the fundamental theorem of calculus (among others, additivity, Chasles' relationship, monotonicity, the hyperbolic Pythagorean theorem, mean value theorem, chain rule, etc.).

5.2 The t-self of Poincaré disk

Let us now put T-calculus to good use in our context. For a set X endowed with function d : X GLYPH<2> X GLYPH<209> r 0 , GLYPH<0>8s ( e.g. Poincaré disk and the Poincaré hyperbolic distance), the t-self of X is the set (implicitly) endowed with d p t q . GLYPH<16> g t GLYPH<5> d . Before going further, let us provide the elegant d p t q associated to Poincaré disk's t -self (with r . GLYPH<16> } z } as in (4)):

The following Lemma (shown in Appendix) demonstrates the usefulness of T-calculus to address the encoding problem in B .

Lemma 1. In Poincaré disk model, pick any increasing function g p k q ¥ 0 . For the choice t GLYPH<16> 1 GLYPH<1> f p k q where f p k q P R is any function satisfying

we get in lieu of (6) the maximal d p t q GLYPH<6> of d p t q satisfying d p t q GLYPH<6> ¥ g p k q , and the new hyperbolic constant τ t of the t -self satisfies

Since both log p 1 GLYPH<0> x q GLYPH<16> x GLYPH<0> o p x q and exp p x q GLYPH<1> 1 GLYPH<16> x GLYPH<0> o p x q in a neighborhood of 0 , we check that (11) and (12) get back to properties of the Poincaré model as f p k q GLYPH<209> 0 [19]. We then have two cases: we can easily approach back the Euclidean d GLYPH<6> by picking f p k q ¡ 0 : choosing f p k q GLYPH<16> 1 gets there and we still keep a finite hyperbolic constant, albeit exponential in the former one. If however we want to improve further the hyperbolic constant, we can pick some admissible f p k q GLYPH<160> 0 , but then (11) will constrain g p k q to a very small value. Note that depending on the sign of f p k q , the triangle inequality can still hold or be replaced by a weaker version since we can show d B p t q p x , z q / d B p t q p x , y q GLYPH<0> d B p t q p y , z q GLYPH<0> max t 0 , f p k qu GLYPH<4> d B p t q p x , y q GLYPH<4> d B p t q p y , z q in B p t q .

Let us finally investigate how we can perform a general embedding in the t -self B p t q of the Poincaré disk to be (i) more encoding-savvy and (ii) have points originally close to B B substantially further away from the t -self B p t q border. We also add a third constraint (iii) that the distortions must be fair w.r.t. the original the Poincaré disk model embedding; i.e. if some z P B ( e.g. the coordinate of a DT node) gets mapped to z p t q P B p t q (the t-self), then we request

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-0.3-0.2-0.1

0

0.1  0.2  0.3

g t

(z)

z

-6

-4

-2

0

2

4

6

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

r

(t)

r

0

0.2

0.4

0.6

0.8

1

online_shoppers_intention

analcatdata_supreme

abalone

j

GLYPH<16>

1

j

GLYPH<16>

10

online_shoppers_intention

p

j

GLYPH<16>

1

q

Confidential + Proprietary

Poincaré disk vs t-self - UCI online_shoppers_intentions

~just scaling

hardware

p

j

GLYPH<16>

70

q

Poincaré disk vs t-self - UCI hardware, t=71

[p: 5.42 %]

[% 11°2 :0]

[% 266 :0]

IB

0010.9

0.1/0.9

0.1)0.9

0.10.9

0.0010.99

0.2)0.8

307

0.2)0.8

0.2)0.8

0.210

0.1(0.9

.4/0.6

A.4/0.6

0.4|0.%

[p: 6.22 %]

[p: 6.63 %]

[p: 5.43 %]

[o: 0.80 %]

IB

0.4|0.6

0.3|0.7

0.2)0.8

0.110.3

0.4(0.6

A.310.7

0.2)0.8

0.1)0.9

0.2(0.8

0.1/0.9

0.3(0.7

2)0.8

0.1)0.9

0.001J0.999

p.3Id

0.210

0.1(0.9

0.0eg)0.999

Confidential + Proprietary

All three conditions point to a non-linear embedding p t : B GLYPH<209> B p t q . The t -self offers a simple convenient solution with a trivial design for p t : We compute z p t q by a simple scaling of the Euclidean norm of z in B to ensure that (13) holds. Figure 3 (right) displays the corresponding relationship between r . GLYPH<16> } z } and r p t q . GLYPH<16> } z p t q } when t P r 0 , 1 s , clearly achieving (ii) in addition to (iii). For (i), even for t GLYPH<160> 1 very close to 1 and z close to B B ( r close to 1), the mapping can send z p t q substantially 'back in' the t-self B p t q : for t GLYPH<16> 0 . 7 and r GLYPH<16> 1 GLYPH<1> 10 GLYPH<1> 4 -i.e. k GLYPH<16> 4 in Definition 5.1 - we get r p t q GLYPH<19> 0 . 96 , authorizing encoding with a less 'risky' k GLYPH<16> 2 . Two additional benefits from (13), visible from Figure 3 (right) is that the distortion is low near the center and then merely affine in a wide region until near the heavy non-linear changes, typically when r ¡ 0 . 8 .

6 Experiments

We summarize a number of experiments (Table 1), provided otherwise in extenso in the Appendix. In the top-down induction scheme for DT, the leaf chosen to be split is the heaviest non pure leaf, i.e. the one among the leaves containing both classes with the largest proportion of training examples. Induction is stopped when the tree reaches a fixed size, or when all leaves are pure, or when no further split allows decreasing the expected log-loss. We do not prune DTs. Our domains are public ( Cf Appendix).

Poincaré embeddings of DTs / MDTs See Figure 2 (left) for a summary of the visualization. We remind that the center of the disk is 'poor' classification (confidence 0), or, in the context of boosting, random classification. A striking observation is the sheer variety of patterns that are plainly obvious from our visualization but would otherwise be difficult to spot using classical tools. Some are common to all domains: as boosting iterations increase, low-depth tree nodes tend to converge to the center of the disk, indicating increased hardness in classification. This is the impact of a well known effect of boosting, whose weight modifications make the problem "harder". Among domain-dependent patterns, highly imbalanced domains get a root predictably initially embedded 'far' from the origin ( online_shoppers_intention , analcatdata_supreme ) while more balanced domains get their roots embedded near the origin ( abalone ). Across the experiment, all trees have at most 200 nodes: while this clearly provides very good models after a large number of iterations on some domains, it is not enough to get good models after just a few iterations on others ( analcatdata_supreme ). Interpreting the hyperbolic embeddings can also be telling: consider Figure 2 (right), which a MDT of the ( j GLYPH<16> 3 ) boosted DT learned on online_shoppers_intention . Notice the low embedding error (5.46 % ). We see in (A) (magenta) that classes are more balanced at the root compared to previous trees (All DTs in Appendix, Table IV) because the root is closer to the center of the disk: thus, hard examples belong to both classes (no class becomes much harder to 'learn' than the other). Two clearly distinct subtrees are associated to non-purchase patterns (red, A). The bottom-most subtree achieves very good confidence with several leaves close to the border: these are non-purchase patterns clearly distinct from purchase patterns (green nodes); the whole subtree dangles from the root via a test on feature PageValues (grey, B) achieving a substantial boost in confidence; many nodes are way past posterior isoline p P t 0 . 1 , 0 . 9 u . Red subtree in (A) is a lot closer to purchase patterns (C). One distinct pattern of purchase emerges, built from tests that always strictly increase its prediction's confidence (orange). The full rule achieves very high accuracy (leaf posterior nears 0 . 99 ).

Interest of the t-self for visualization As demonstrated by Table 1 (right) and Appendix, the t-self is particularly useful to tell the best parts of a tree. Because it also manages to 'push back' the bulk of nodes from the border B B p t q , it displays the t-self might also be useful as a standard encoding (use (13) to access Poincaré disk quantities and embedding).

MDT vs DT for classification We traded the hyperbolic visualization of a DT - bound to be poor - for the visualization of its 'monotonic part', an MDT. We have seen that the MDT visualization provides lots of clues about the DT as well. In terms of classification, the fact that the set of leaves of the MDT is a subset of the set of leaves of its DT hints on similarities in classification as well, but so far we have not explored a key question whose scope goes beyond this paper's: how does the classification of a MDT compares to its DT ? This question is important because in addition to being more amenable to visualization than DTs, it might also be the case that classification with MDTs might just be a good alternative to doing it with DTs. The scope of this question goes beyond this paper because of the tremendous popularity of DTs. The left pane of Table 2 provides a partial answer on a subset of our domains (the full set of results is in the Appendix, Section D.II; p-values are that of a Student paired t-test with H0 being the identity of the average errors). Clearly, the classification accuracies are similar for a majority of the domains considered, but interestingly, we observe the two polarities when they are not: on domains ionosphere and online_shoppers_intention , the DT is better than its MDT, but on domain analcatdata_supreme it is the opposite. This is particularly interesting because the MDT is no larger and can be substantially smaller than its corresponding DT and thus could represent an efficient pruning of its DT.

MDTembedding: visualization error The right pane of Table 2 provides an example of embedding errors ρ (5) for the first five MDTs in a boosted ensemble (we remind that the description of the boosting algorithm and the embedding of the leveraging coefficients is provided in Appendix, Section C.IX). The full set of results is in Appendix, Section D.II. The results show that errors are small in general, which is good news given our quite heuristic modification to Sarkar's embedding. There is more: the embedding error tends to decrease - in some cases very significantly - with the MDT's index in the ensemble, see for example winered , german and qsar . The explanation is simple and

due to a key boosting feature, the reweighting mechanism underlined above: with weight updates that bring in general the total (new) weight of positive examples closer to that of negative examples, the root of the next MDT gets embedded closer to the center of the disk. As the root moves closer to the center, it is much easier to get a good spread of the tree in the rest of the disk at reduced error, in particular if the MDT is well balanced. For an example, see hardware for j GLYPH<16> 70 in Figure 1 ( ρ GLYPH<160> 1% ).

7 Conclusion

This paper proposes three separate contributions that altogether provide a solution for hyperbolic embedding of (ensembles of) decision trees (DT), via (i) a link between losses for class probability estimation and hyperbolic distances, (ii) the design of a clean, unambiguous embedding of a DT via its monotonic subtree and (iii) a way to improve visualization and encoding properties of the Poincaré disk model using its t-self, a notion we introduce. Each of these contributions are indeed separate: for example, one could reuse (i) to embed models different from DTs or (iii) to perform hyperbolic embeddings of objects being not even related to supervised models. Contribution (ii) is of independent interest with the introduction of a new kind of tree-based models, monotonic decision trees.

We believe that the way we address (iii) opens general applications even outside hyperbolic geometry. Indeed, we generalize classical integration and derivation to a context encompassing the concept of additivity, upon which integration is built, extending standard properties of integration and derivation in a natural way (see the Appendix for many examples). That such properties can be obtained without additional technical 'effort' bodes well for perspectives of developments and applications in other subfields of ML, where such tools could be used, not just in our geometric context, to smoothly tweak distortion measures. Many distortion measures used in ML are indeed integrals in nature: Bregman divergences, f -divergences, integral probability metrics, etc.

One inherent limitation of our work is linked to the use of DTs: they typically fit very well to tabular variables (attribute-value data) but otherwise should be used with caution. Our work is a first step towards more sophisticated visualization for supervised models (Section 2), looking for tailored fit geometric spaces to best blend their symbolic and numerical properties. More can be done and more has to be done: at an age of data collection and ML compute still ramping up, seeking model "pictures" worth a thousand words is a challenge but a necessity for responsible AI and explainability. Finally, our introduction of monotonic decision trees begs for a deeper statistical analysis of such models, given the tremendous popularity of decision trees that they could complete - or replace - on mainstream data science pipelines.

Acknowledgments

Thanks are due to Mathieu Guillame-Bert, reviewers and the area chair for many suggestions that helped to improve the paper's content. Work was done while AS was visiting RIKEN AIP.

References

E. Amid, F. Nielsen, R. Nock, and M.-K. Warmuth. Optimal transport with tempered exponential measures. In AAAI'24 , 2024.

Ehsan Amid, Frank Nielsen, Richard Nock, and Manfred K Warmuth. The tempered Hilbert simplex distance and its application to non-linear embeddings of TEMs. arXiv preprint arXiv:2311.13459 , 2023.

A. Banerjee, S. Merugu, I. Dhillon, and J. Ghosh. Clustering with Bregman divergences. In Proc. of the 4 th SIAM International Conference on Data Mining , pages 234-245, 2004.

A.-F. Beardon and D. Minda. The hyperbolic metric and geometric function theory. In Proc. of the International Workshop on Quasiconformal Mappings and their Applications , 2005.

B.-H. Bowditch. A course on geometric group theory . Math. Soc. Japan Memoirs, 2006.

I. Chami, A. Gu, V. Chatziafratis, and C. Ré. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering. In NeurIPS*33 , 2020.

I. Chami, A. Gu, D. Nguyen, and C. Ré. Horopca: Hyperbolic dimensionality reduction via horospherical projections. In 38 th ICML , volume 139 of Proceedings of Machine Learning Research , pages 1419-1429. PMLR, 2021.

P. Chlenski, E. Turok, A. Khalil Moretti, and I. Pe'er. Fast hyperboloid decision tree algorithms. In 12 th ICLR , 2024.

H. Cho, B. Demeo, J. Peng, and B. Berger. Large-margin classification in hyperbolic space. In 22 nd AISTATS , volume 89 of Proceedings of Machine Learning Research , pages 1832-1840. PMLR, 2019.

Z. Cranko and R. Nock. Boosted density estimation remastered. In 36 th ICML , pages 14161425, 2019.

L. Doorenbos, P. Márquez-Neila, R. Sznitman, and P. Mettes. Hyperbolic random forests. CoRR , abs/2308.13279, 2023.

D. Dua and C. Graff. UCI machine learning repository, 2021.

J. Friedman, T. Hastie, and R. Tibshirani. Additive Logistic Regression : a Statistical View of Boosting. Ann. of Stat. , 28:337-374, 2000.

O.-E. Ganea, G. Bécigneul, and T. Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. In 35 th ICML , volume 80 of Proceedings of Machine Learning Research , pages 1632-1641. PMLR, 2018.

O.-E. Ganea, G. Bécigneul, and T. Hofmann. Hyperbolic neural networks. In NeurIPS*31 , pages 5350-5360, 2018.

D.-E. Knuth. Two notes on notation. The American Mathematical Monthly , 99(5):403-422, 1992.

M.-T. Law, R. Liao, J. Snell, and R.-S. Zemel. Lorentzian distance learning for hyperbolic representations. In 36 th ICML , volume 97 of Proceedings of Machine Learning Research , pages 3672-3681. PMLR, 2019.

Y. Mansour, R. Nock, and R.-C. Williamson. Random classification noise does not defeat all convex potential boosters irrespective of model choice. In 40 th ICML , 2023.

G. Mishne, Z. Wan, Y. Wang, and S. Yang. The numerical stability of hyperbolic representation learning. In 40 th ICML , 2023.

M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In NeurIPS*30 , pages 6338-6347, 2017.

F. Nielsen and R. Nock. On Rényi and Tsallis entropies and divergences for exponential families. CoRR , abs/1105.3259, 2011.

L. Nivanen, A. Le Méhauté, and Q.-A. Wang. Generalized algebra within a nonextensive statistics. Reports on Mathematical Physics , 52:437-444, 2003.

R. Nock, W. Bel Haj Ali, R. D'Ambrosio, F. Nielsen, and M. Barlaud. Gentle nearest neighbors boosting over proper scoring rules. IEEE Trans.PAMI , 37(1):80-93, 2015.

R. Nock and A. K. Menon. Supervised learning: No loss no cry. In 37 th ICML , 2020.

M.-C. Pardo and I. Vajda. About distances of discrete distributions satisfying the data processing Theorem of Information Theory. IEEE Trans. IT , 43:1288-1293, 1997.

J. R. Quinlan. C4.5 : programs for machine learning . Morgan Kaufmann, 1993.

J.-G. Ratcliffe. Foundations of Hyperbolic Manifolds . Springer Graduate Texts in Mathematics, 1994.

M.-D. Reid and R.-C. Williamson. Information, divergence and risk for binary experiments. JMLR , 12:731-817, 2011.

F. Sala, C. De Sa, A. Gu, and C.Ré. Representation tradeoffs for hyperbolic embeddings. In 35 th ICML , volume 80 of Proceedings of Machine Learning Research , pages 4457-4466. PMLR, 2018.

R. Sarkar. Low distortion Delaunay embedding of trees in hyperbolic plane. In GD'11 , volume 7034, pages 355-366. Springer, 2011.

R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. MLJ , 37:297-336, 1999.

Andrew D Selbst and Solon Barocas. The intuitive appeal of explainable machines. Fordham L. Rev. , 87:1085, 2018.

A. Slavik. Product integration, its history and applications . Matfyzpress, Praze, 2007.

R. Sonthalia and A.-C. Gilbert. Tree! I am no tree! I am a low dimensional hyperbolic embedding. In NeurIPS*33 , 2020.

T. van Erven and P. Harremoës. Rényi divergence and kullback-leibler divergence. IEEE Trans. IT , 60:3797-3820, 2014.

M. Yang, M. Zhou, R. Ying, Y. Chen, and I. King. Hyperbolic representation learning: Revisiting and advancing. In 40 th ICML , volume 202 of Proceedings of Machine Learning Research , pages 39639-39659, 2023.

T. Yu and C. De Sa. Numerically accurate hyperbolic embeddings using tiling-based models. In NeurIPS*32 , pages 2021-2031, 2019.

T. Yu, T.-J.-B. Liu, A. Tseng, and C. De Sa. Shadow cones: Unveiling partial orders in hyperbolic space. CoRR , abs/2305.15215, 2023.

Appendix

To differentiate with the numberings in the main file, the numbering of Theorems, etc. is letter-based (A, B, ...).

Table of contents

A Broader Impact

This paper presents work whose goal is to advance the field of Machine Learning. Beyond our proposed tempered calculus which is used to amend the original Poincaré disk model of hyperbolic geometry, the monotonic decision trees and t -self hyperbolic embedding contribute to the field of visualization and explainable AI for ML models. Thus, there are two distinct avenues for broader impact and related care:

from the geometry standpoint, the t -self brings a novel coding and visualization advantage over the founding Poincaré disk model for the embedding close to the disk's border, regardless of the nature of the embedding (whether it is data, models, etc. that are embedded). However, being new, it is important to keep the label that the t -self is being used, and also clearly display the relevant value of t or ranges, since the properties of the embedding can change depending on them;

from the model standpoint and a societal perspective, these visualizations will provide better methods for deployed decision tree models to be scrutinized. A potential negative of our approach is the required reduction from normal decision trees to monotonic decision trees, where the visualization does not directly correspond to the initial decision tree. We believe that monotonicity adds value to the interpretation of rules that are progressively built from trees, yet practitioners should be careful when extracting inferences from the reduced monotonic decision tree and clearly label them as being extracted from such a model, to prevent confusion with decision trees, that are overwhelmingly popular. From a pure prediction performance standpoint, we show - at least empirically within the bounds of our settings - that monotonic decision trees are similar to their original decision tree counterparts.

B t -algebra and t -additivity

We provide here a few more details on the basis of our paper, the t -algebra and the t -additivity of some divergences.

B.I Primer on t -algebra

Classical arithmetic over the reals can be used to display duality relationships between operators using the log , exp functions, such as for example log p a { b q GLYPH<16> log a GLYPH<1> log b , exp p a GLYPH<0> b q GLYPH<16> exp p a q GLYPH<4> exp p b q , and so on. They can also be used to define one operator from another one. There is no difference between the operators appearing inside and outside functions. In the t -algebra, a difference appears and such relationships can be used to define the t -operators from those over the reals, as indeed one can define the tempered addition

(both simplifying to the expressions we use), and of course the tempered product and division,

whose simplified expression appears e.g. in [1]. See also [22].

B.II Functional t -additivity

As is well-known, Boltzman-Gibbs (and so, Shannon) entropy is additive while Tsallis entropy is t -additive. Note that additivity for BG requires being on the simplex, but t -additivity of Tsallis technically requires only positive measures - the simplex restriction ensures the limit exists for t GLYPH<209> 1 and then T GLYPH<209> BG.

Divergences can also be t -additive The KL divergence

the tempered subtraction,

is additive on the simplex but not t -additive: using a decomposition of p , q as product of (independent) distributions ( p 1 , p 2 and q 1 , q 2 ) using the cartesian product of their support, we indeed check D KL p p } q q GLYPH<16> D KL p p 1 } q 1 q GLYPH<0> D KL p p 2 } q 2 q with p ij GLYPH<16> p 1 i p 2 j and q ij GLYPH<16> q 1 i q 2 j . On the other hand, Tsallis divergence [21] is t -additive on positive measures with such a decomposition, with

and we check that D T p p } q q GLYPH<16> D T p p 1 } q 1 q GLYPH<0> D T p p 2 } q 2 q GLYPH<0> p 1 GLYPH<1> t q GLYPH<4> D T p p 1 } q 1 q GLYPH<4> D T p p 2 } q 2 q (additional requirement to be on the simplex for convergence to D KL as t GLYPH<209> 1 ). For t R p 1 , 2 q and on the simplex, Tsallis divergence is an f -divergence with generator z GLYPH<222>GLYPH<209> p z 2 GLYPH<1> t GLYPH<1> 1 q{p 1 GLYPH<1> t q . Similarly, the tempered relative entropy is t -additive on the co-simplex , where in this case

The tempered relative entropy is a Bregman divergence with generator z GLYPH<222>GLYPH<209> z log t z GLYPH<1> log t GLYPH<1> 1 z .

C Supplementary material on proofs

C.I Proof of Theorem 2

The Theorem is tautological for t GLYPH<16> 1 so we prove it for t GLYPH<24> 1 . Denote P p S q the set of subsets of set S and P GLYPH<6> p S q . GLYPH<16> P p S qztHu . We first transform S p t q ∆ n p f q (index n shown for readability) in a better suited expression:

where we have used I i . GLYPH<16> r x i GLYPH<1> 1 , x i s for conciseness. We now need a technical Lemma.

Lemma 2. Fix any 0 / v GLYPH<160> 8 { 10 and consider any n reals q i , i P r n s such that | q i | / v, @ i P r n s . Then

Proof. Suppose all the q i , i P r n s satisfy q i P r u, v s and let ϕ be strictly convex differentiable and defined over r u, v s . Then it comes from [10, Lemma 9] that we get the right-hand side of

where we can pick w P t u, v u and D ϕ is the Bregman divergence with generator ϕ (the left-hand side is Jensen's inequality). Picking u . GLYPH<16> GLYPH<1> v (assuming wlog v ¡ 0 ) and letting

for the choice ϕ p z q . GLYPH<16> GLYPH<1> log p 1 GLYPH<0> z q and w . GLYPH<16> GLYPH<1> v , we obtain after simplification

Analizing function v GLYPH<222>GLYPH<209> Y v GLYPH<1> log p Y v q GLYPH<1> 1 reveals that it is upperbounded by v GLYPH<222>GLYPH<209> v 2 if v P rGLYPH<1> 8 { 10 , 8 { 10 s . Hence, multiplying by n all sides and passing to the exponential, we get that

which leads to the statement of the Lemma.

Let us come back to our Riemannian summation setting and let

Assume now that f is Riemann integrable, which allows to guarantee that, at least for n large enough and a step of division ∆ n not too large, we have | q i | / 8 { 10 , @ i P r n s and so we can use Lemma 2. We get from (14) and Lemma 2, if t GLYPH<160> 1

and we permute the bounds if t ¡ 1 . Importantly, we note that

So suppose lim n GLYPH<209>GLYPH<0>8 S p 1 q ∆ n p f q . GLYPH<16> L 1 . GLYPH<16> ‡ b a f p u q u is finite and choose the step of division ∆ n not too large so that

for some constant K ¥ 1 (this is possible because f is (1-)Riemann integrable; we also note that if the division is regular then we can choose K GLYPH<16> 1 ). The value of K is not important: what is important is that nv remains finite in (19) as n increases, while lim n GLYPH<209>GLYPH<0>8 v GLYPH<16> 0 . Hence, exp pGLYPH<1> nv GLYPH<4> v q GLYPH<209> 1 as n GLYPH<209> GLYPH<0>8 and (19) implies, because lim n GLYPH<209>GLYPH<0>8 p 1 GLYPH<0> a { n q n GLYPH<16> exp a , the two first identities in

((21) follows from (20)) and by definition lim n GLYPH<209>GLYPH<0>8 S p t q ∆ n p f q . GLYPH<16> p t q » b a f p u q d t u . So we get that Riemann integration ( t GLYPH<16> 1 ) grants

which, in addition to showing (9) (main file) also shows that t GLYPH<16> 1 -Riemann integration is equivalent to all t GLYPH<24> 1 -Riemann integration, ending the proof of Theorem 2.

Remark 4. The absence of affine terms in upperbounding v GLYPH<222>GLYPH<209> Y v GLYPH<1> log p Y v qGLYPH<1> 1 in the neighborhood of 0 is crucial to get to our result.

C.II Proof of Theorem 3

Using Theorem 2, we just have to analyze the limit in relationship to the Riemannian case:

where the last identity is the classical Riemannian case, (22) follows from Theorem 2, (23) follows from a property of log t ( log t a a log t b GLYPH<16> log t p a { b q ), (24) follows from the fact that log t exp p z q GLYPH<16> 0 z GLYPH<0> o p z q . This completes the proof of Theorem 3.

C.III Additional and helper results for Theorems 2 and 3

We first state a trivial but important Lemma, whose content is partially used in the main file.

Lemma 3. g t satisfies the following properties:

g t p z q is strictly increasing for any t P R , strictly concave for any t ¡ 1 , strictly convex for any t GLYPH<160> 1 and such that sign p g t p z qq GLYPH<16> sign p z q , @ t P R ;

D t g t p z q GLYPH<16> 1 ;

( t -integral mean-value) Suppose f Riemann integrable over r a, b s . Then there exists c P p a, b q such that

We list a series of consequences to both theorems.

General properties of t -integrals Some properties generalize those for classical Riemann integration.

Theorem C.1. The following relationships hold for any t P R and any functions f, g t -Riemann integrable over some interval r a, b s :

Proof. We show additivity for ' t {GLYPH<0> (the same path shows the result for a t {GLYPH<1> ):

We show dilativity, using t 1 . GLYPH<16> 1 GLYPH<1> p 1 GLYPH<1> t q λ :

The triangle inequality follows from the relationship:

from which we use the fact that Riemann integration satisfies the triangle inequality and log 1 GLYPH<1>| 1 GLYPH<1> t | is monotonically increasing on R GLYPH<0> in the penultimate line of:

We show Chasles' relationship:

where the second identity uses the property log t a ' log t b GLYPH<16> log t p ab q . Monotonicity follows immediately from the fact that z GLYPH<222>GLYPH<209> log t exp z is strictly increasing:

This ends the proof of Theorem C.1.

Computing t -integrals Next, classical relationships to compute integrals do generalize to t -integration. We cite the case of integration by part.

Lemma 4. Integration by part translates to t -integration by part as:

(Proof immediate from Theorem 2)

Geometric properties based on t -integrals This is a more specific result, important in the context of hyperbolic geometry: the well-known Hyperbolic Pythagorean Theorem (HPT) does translate to a tempered version with the same relationship to the Euclidean theorem. Consider a hyperbolic right triangle with hyperbolic lengths a, b, c , c being the hyperbolic length of the hypothenuse. Let a t , b t , c t denote the corresponding tempered lengths, which are therefore explicitly related using g t as

Define the tempered generalization of cosh :

The HPT tells us that cosh c GLYPH<16> cosh a cosh b . It is a simple matter of plugging g t , using the fact that log t and exp t are inverse of each other and simplifying to get the tempered HPT, which we call t -HPT for short.

Lemma 5. ( t -HPT) For any hyperbolic triangle described above, the tempered lengths are related as

where we let

Now, remark that for any t GLYPH<24> 0 , a series expansion around 0 gives

( exp t is always infinitely differentiable around 0 , for any t ) So the t -HPT gives

which simplifies, if we multiply both sides by 2 { t and simplify it into

which for an infinitesimal right triangle gives c 2 t GLYPH<19> a 2 t GLYPH<0> b 2 t , i.e. Pythagoras Theorem, as does the HPT one gives in this case ( c 2 GLYPH<19> a 2 GLYPH<0> b 2 ), which is equivalent of the particular t GLYPH<16> 1 -HPT case.

t -mean value Theorem The t -derivative yields a generalization of the Euclidean mean-value theorem.

Lemma 6. Let t P R and f be continuous over an interval r a, b s , differentiable on p a, b q and such that GLYPH<1> 1 {p 1 GLYPH<1> t q R f pr a, b sq . Then D c P p a, b q such that

Proof. We can obtain a direct expression of D t f by using the definition of a t and the classical derivative:

From here, the mean-value theorem tells us that there exists c P r a, b s such that

Dividing by 1 GLYPH<0> p 1 GLYPH<1> t q f p c q (assuming f p c q GLYPH<24> GLYPH<1> 1 {p 1 GLYPH<1> t q ) and reorganising, we get

which completes the proof of the Lemma.

In fact, the t -derivative of f at some c is "just" an Euclidean derivative for an affine transformation of the function, namely z GLYPH<222>GLYPH<209> f p z q a t f p c q , also taken at z GLYPH<16> c . This "proximity" between t -derivation and derivation is found in the tempered chain rule (proof straightforward).

Lemma 7. Suppose g differentiable at z and f differentiable at g p z q . Then

C.IV Sets endowed with a distortion and their t -self: statistical information

Here, X contains probability distributions or the parameters of probability distributions: f can then be an f -divergence (information theory) or a Bregman divergence (information geometry). Tsallis divergence and the tempered relative entropy are examples of t -additive information theoretic and information geometric divergences. A key property of information theory is the data processing inequality (D) , X being a probability space, which says that passing random variables through a Markov chain cannot increase their divergence as quantified by f [25, 35]. A key property of

information geometry is the population minimizer property (P) , which elicits a particular function of a set of points as the minimizer of the expected distortion to the set, as quantified by f [3]. We let (J) denote the joint convexity property, which would state for f and any x 1 , x 2 , y 1 , y 2 that

and convexity (C) , which amounts to picking y 1 GLYPH<16> y 2 (convexity in the left parameter) xor x 1 GLYPH<16> x 2 (in the right parameter).

Lemma 8. For any t P R , the following holds true:

(D) f satisfies the data processing inequality iff f p t q satisfies the data processing inequality;

(J) f satisfies (27) iff the following p t, t 1 , t 2 q -joint convexity property holds:

Proof. (D) and (P) are immediate consequences of Lemma 3 (point [1.], main file) and properties of log t . We prove (J) . g t being strictly increasing for any t , we get for t / 1

because g t is convex. If t ¡ 1 , we restart from the first inequality and remark that

The inequality is due to the fact that a ' t b GLYPH<16> a GLYPH<0> b GLYPH<0> p 1 GLYPH<1> t q ab / a GLYPH<0> b if ab ¥ 0 and t ¥ 1 . The last equality holds because, for t 1 . GLYPH<16> 1 GLYPH<1> p 1 GLYPH<1> t q b , we have

Putting altogether (30) and (31), we get that for any t P R ,

as claimed for (J) . This ends the proof of Lemma 8.

We note that (29) also translates into a property for (C) ; also, if t / 1 , then t GLYPH<16> t 1 GLYPH<16> t 2 in (29).

C.V The t-self of the Lorentz model of hyperbolic geometry

As an additional example of use, we consider the Lorentz model for a simple illustration in which one creates approximate metricity in the t-self. This d -dimensional manifold is embedded in R d GLYPH<0> 1 via the hyperboloid with constant GLYPH<1> c GLYPH<160> 0 curvature and defined by H c . GLYPH<16> t x P R d GLYPH<0> 1 : x 0 ¡ 0 ^ x GLYPH<5> x GLYPH<16> GLYPH<1> 1 { c u , with x GLYPH<5> y . GLYPH<16> GLYPH<1> x 0 y 0 GLYPH<0> GLYPH<176> d i GLYPH<16> 1 x i y i the Lorentzian inner product [27, Chapter 3]. In ML, two distortions are considered on H c [17], one of which is the Lorentzian 'distance", d L p x , y q . GLYPH<16> GLYPH<1>p 2 { c q GLYPH<1> 2 GLYPH<4> x GLYPH<5> y . It is notoriously not a distance because it does not satisfy (T) , yet we show that we can pick t and the curvature in such a way that the t-self is arbitrarily close to a metric space.

Lemma 9. @ δ ¡ 0 , pick t and curvature c as t GLYPH<16> 1 GLYPH<0> p 1 { δ q , c GLYPH<16> 2 { δ . Then the t -self of H c is approximately metric: d p t q L satisfies (R) , (S) , (I) , and the δ -triangle inequality,

Proof. d p t q L still obviously satisfies reflexivity, the identity of indiscernibles, and non-negativity, so we check the additional property it now satisfies, the weaker version of the triangle inequality. Given any x , y , z in H c , condition d p t q L p x , z q / d p t q L p x , y q GLYPH<0> d p t q L p y , z q GLYPH<0> δ for t ¡ 1 is

which simplifies to

We have x GLYPH<5> y / GLYPH<1> 1 { c by definition, so a sufficient condition to get the inequality is to have

Function h p z q . GLYPH<16> zδ exp pGLYPH<1> 2 z { c q is maximum over R GLYPH<0> for z GLYPH<6> GLYPH<16> c { 2 , for which it equals h p z GLYPH<6> q GLYPH<16> cδ {p 2 e q . Fix t GLYPH<16> 1 GLYPH<0> p c { 2 q . We then have exp p 2 p t GLYPH<1> 1 q GLYPH<4> y GLYPH<5> z q / 1 { e so to get (34) for this choice of t , it is sufficient to pick curvature c GLYPH<16> 2 { δ , yielding relationship t GLYPH<16> 1 GLYPH<0> p 1 { δ q .

C.VI Proof of Lemma 1

We start by proving condition

in the Lemma. Using the proof of [19, Proposition 3.1], we know that any point x k -close to the boundary (Definition 5.1) satisfies

so to get (35), we want log t p 2 GLYPH<4> 10 k GLYPH<1> 1 q ¥ g p k q . Letting t GLYPH<16> 1 GLYPH<1> f p k q with f p k q P R , we observe log t p 2 GLYPH<4> 10 k GLYPH<1> 1 q GLYPH<16> GLYPH<0> p 2 GLYPH<4> 10 k GLYPH<1> 1 q f p k q GLYPH<1> 1 GLYPH<8> { f p k q , so we want, after taking logs,

(this also holds if f p k q GLYPH<160> 0 because 1 GLYPH<0> f p k q g p k q GLYPH<160> 1 ), and there remains to observe log p 2 GLYPH<4> 10 k GLYPH<1> 1 q GLYPH<16> k log p 10 q GLYPH<0> log p 2 GLYPH<1> 1 { 10 k q with log p 2 GLYPH<1> 1 { 10 k q ¥ 0 , @ k ¥ 0 . Hence, to get (36) it is sufficient to request

which is (35). For such t , the new hyperbolic constant satisfies

C.VII Proof of Theorem 1

Take any observation x P X . Trivially, H 1 p x q is a real value that belongs to the path followed by x in H . Furthermore, if we consider any path from the root to a leaf in H , all the vertices in its strictly monotonically increasing subsequence of absolute confidences appear in H 1 (conditions 4, 8 in GETMDT), which guarantees the condition on prediction in (M) . Hence (M) is satisfied.

add children (?)

reflect

addarcs/nodes

reflect

childrenadded

2x/3

Sarkar

Us

C.VIII Modifying Sarkar's embedding in Poincaré disk

Sarkar's algorithm [30] gives a clean low-distortion embedding when the tree is binary or the arc length is constant [29]. Things are different in our case: MDT nodes have arbitrary out-degrees, and lengths depend on the absolute confidence at the corresponding MDT nodes. Plus, a direct implementation of Sarkar's algorithm would violate the constraint for strict path-monotonicity in an MDTthat nodes in a path from the root need to progressively come closer to the border - equivalently, it would create substantial embedding errors for confidences and violate (B) . Without focusing on an optimal solution (that we leave for future work), one can remark that all these problems can be heuristically addressed by changing one step of Sarkar's algorithm, replacing the use of the total 2 π fan out angle for mapping children (Step 5: in Algorithm 1 of [29]) by a variable angle with a special orientation in the disk.

We refer to the concise and neat description of Sarkar's embedding in [29] for the full algorithm. Our modification relies on changing one step of the algorithm, as described in Figure I. The key step that we change is step 5: in the description of [29, Algorithm 1]. This step embeds the children of a given node (and then the algorithm proceeds recursively until all nodes are processed). Sarkar's algorithm corresponds to the simple case where all arcs to/from a node define a fixed angle, which does not change during reflection because Poincaré model is conformal. Hence, if the tree is binary, this angle is 2 π { 3 , which provides a very clean display of the tree. In our case however, some children may have just one arc to a leaf while others may support big subtrees. Also, arc lengths can vary substantially. We thus design the region of the disk into which the subtrees are going to be embedded by choosing an angle proportional to the number of leaves reachable from the node, and of course lengths have to match the difference between absolute confidence between a node and its children. There is no optimization step to learn a clean embedding, so we rely on a set of hyperparameters to effectively compute this new step of the algorithm.

C.IX Boosting with the logistic loss à-la AdaBoost

Algorithm 2 LOGISTICBOOST( S )

Input: Labeled sample S . GLYPH<16> tp x i , y i q , i P r m su , T P N ¡ 0 ;

1 :

2 :

3 :

4 :

5 :

un confident prediction)

// request a DT as a "weak hypothesis" // picks leveraging coefficient for H j // weight update, not normalized

1

{

j

2

,

GLYPH<16>

H

@

1

i

,

j

-

α

j

for

P r

m

s

; // initialize all weights (equivalent to maximally

2

, ..., T

WEAK-LEARN

P

R

i

;

GLYPH<16>

1

,

2

, ..., m

Output: Classifier H T . GLYPH<16> GLYPH<176> j α j H j p . q ;

The link between distances in the Poincaré model of hyperbolic geometry and the canonical link of the log-loss (3) can be extended to leveraging coefficients in boosted combinations [31]. To get there, we need a tailored boosting algorithm which parallels AdaBoost's design tricks (hence different from LogitBoost [13]). We want a boosting algorithm for the liner combination of DTs which displays classification that can be easily and directly embedded in the Poincaré disk. Algorithm LOGISTICBOOST is provided above. For the weight update, we refer to [23]. We already know how to embed DTs via their Monotonic DTs. What a boosting algorithm of this kind does is craft

Remark that we have merged the leveraging coefficient and the DTs' outputs H . , on purpose. To compute the leveraging coefficients α . in Step 4, we use AdaBoost's secant approximation trick † , applied not to the exponential loss but to the logistic loss: for any z P rGLYPH<1> R,R s and α P R ,

Hence, to compute the leveraging coefficient α j that approximately minimizes the current loss, GLYPH<176> i w ji log p 1 GLYPH<0> exp pGLYPH<1> αy i H j p x i qqq , we minimize instead the upperbound using (39). Letting

(note the slight abuse of notation for readability, since we write p ψ LOG q GLYPH<6> j instead of |p ψ LOG q GLYPH<6> j | ; we remind that Λ p . q denotes the set of leaves of a tree; the index in the local proportion of positive example p GLYPH<0> jλ reminds that weights used need to be boosting's weights) which we note can be directly approximated on the Poincaré disk by looking at the leaf nearest to the border of the disk. Because the maximal absolute confidence in the DT is also the maximal absolute confidence in its MDT, we obtain the sought minimum,

where r j P rGLYPH<1> 1 , 1 s is the normalized edge

w

1

i

-

for

p

S

,

w

j

q

;

Monotonic DT representation - UCI online_shoppers_intentions

best confidence in DT

normalized DT confidence

node embedding error

isolines (posterior)

Monotonic Decision Tree

red / green = majority class

arc thickness = #tests in

DT to reach child node

(arcs going outwards in     )

(

j

=7)

Leveraging coefficient

j

= #boosting iteration

Confidential + Proprietary

where ˜ w j indicates normalized weights. It is not hard to show that because we use the local posterior p GLYPH<0> jλ at each leaf, α j ¥ 0 and also r j ¥ 0 . Hence, everything is like if we had an imaginary node ν j with p GLYPH<0> jν j . GLYPH<16> p 1 GLYPH<0> r j q{ 2 ( ¥ 1 { 2 ) and positive confidence

that we can display in Poincaré disk (we choose to do it as a circle; see Figure II, main file). We deduce from (38) that

and note that all three key parameters can easily be displayed or computed directly from the Poincaré disk:

If we use MDTs directly for prediction, log GLYPH<2> p GLYPH<0> jλ p x q 1 GLYPH<1> p GLYPH<0> jλ p x q GLYPH<10> GLYPH<16> H j p x q is read directly from the MDT's plot; the absolute value can be removed and the true sign guessed from the node's color (or shape); if se use DTs for prediction, the MDT's plot gives an approximation of the DT's output which is 100% accurate for all leaves of the MDT (each corresponds to a leaf in the DT);

If H j is leveraged (in an ensemble), p ψ LOG q GLYPH<6> j is read directly from the plot (40), as well as p ψ LOG q j . Their ratio gives the leveraging coefficient. We note that p ψ LOG q GLYPH<6> j is the same for both the DT and its corresponding MDT. Only the value of p ψ LOG q j can differ for the DT and the MDT.

We note that regardless of whether we use the DT or its MDT to classify, the MDT's representation is also fully accurate for the DT for a subset of its leaves.

D Supplementary material on experiments

D.I Domains

The domains we consider are all public domains, from the UCI repository of ML datasets [12], OpenML, or Kaggle, see Table I.

D.II Visualizing a DT via its MDT and embedding errors

All test errors are estimated from a 10-fold stratified cross-validation. One can always choose to directly learn Monotonic DTs instead of DTs, given their natural fit for embedding in the Poincaré disk. In this case, the hyperbolic representation of the MDT can be immediately used for assessment. Suppose we stick to learning DTs (because e.g. they have been standard in ML for decades) and wish

to use the visualization of its corresponding MDT to make inferences on the original DT itself. Can we make reliable conclusions? We explore this question by observing that the prediction from DT to MDT can only change if the leaf λ that an example reaches in the DT satisfies two conditions: (a) λ does not appear as a leaf in the MDT, and (b) it is tagged to an internal node of the MDT with a confidence of the opposite sign (this is (D1) , Step 3: in GETMDT). Without tackling the formal aspect of this question, we have designed a simple experiment for a simple assessment of whether / when this is reasonable. We have trained a set of T =200 boosted DTs, each with maximal size 200 (total number of nodes). After having computed the corresponding MDTs, we have compared the test errors of the boosted set of trees and that of the ensemble where each DT is replaced by its MDT, but the leveraging coefficients do not change. Intuitively, if test errors are on par, the variation in classification of DTs vs MDTs (including confidences) is negligible, and we can 'reduce" the interpretation of the DT to that of its MDT in the Poincaré disk. The results are summarized in Table II.

From Table II, we can safely say that our hypothesis reasonably holds in many cases, with two important domains for which it does not: ionosphere and hillnoise . For the former domain, we attribute it to the small size of the domain, which prevents training big enough trees; for the latter domain, we attribute it to the fact that the domain contains substantial noise, which makes it

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[p: 5.42 %]

[o: 3.74 %]

[p: 5.46 %]

[p: 6.42 %]

[p: 7.00 %]

0.110.9

0.110.9

0.110.9

0.110.9

0.110.9

0.210.8

.20.8

.210.8

0.2/0.8

0.210.8

0.30.

0.3)0.7

0.3)0.7

0.300.7

0.3(0.7

0.4)0.6

0.4|0.6

.4|0.6

[% 96'2 :0]

[o: 4.70 %]

[p: 5.68 %]

[p: 7.93 %]

[p: 6.22 %]

0.110.9

0.110.9

0.110.9

0.110.9

Q.10.3

0.210.8

0.210.8

0.210.8

0.210.8

.2/0.8

0.3(0.7

0.3|0.7

0.310.7

0.300.7

0.3(0.7

0,4)0.6

0.4)0.6

0.4|0.6

difficult to substantially improve posteriors by splitting and thus make many DT nodes, including leaves, 'disappear" in the MDT conversion (Steps 2: and 14: in GETMDT). Table III summarizes the visualization errors of the MDT embeddings obtained in Tables IV - XII. A key feature of the embeddings is that on some domains ( e.g. german ) the error of the embedding dramatically decreases with the index of the MDT in the ensemble. The reason why this happens is simple: the weight update of boosting tends to equalize weights between classes (to make the problem "harder"); thus, new trees tend to have their roots embedded closer to the origin, which makes the embedding easier for the rest of the trees to keep good visualization at low error.

D.III All Poincaré disk embeddings

They are presented in Table IV through to XII, showing the first models learned in a fold of the cross-validation experiments. producing Table II

D.IV Interest of the t-self for visualization

We now test the experimental impact of switching to the t-self in Poincaré disk model (See Table XIII for a clear depiction of how changing t can yields better visualization close to the border of the disk). Recall that switching to the t-self moves way model parts close to B B but keeps a low non-linear distortion around the center, which is thus roughly only affected by a scaling factor. Figure I presents a few results. For domain online_shoppers_intention , we note that the part of the tree that is within the isoline defined by posterior p GLYPH<0> . P t 0 . 1 , 0 . 9 u gets indeed just scaled: both plots look quite identical. Very substantial differences appear near the border: the best parts of the model could easily

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[% 1°2 :d]

[o:8.32 %]

[0:8.23 %]

[p: 8.67 %]

[p: 7.60 %]

0.110.9

.110.9

0.10.9

0.10.9

0.110.9

0.210.8

p.20.8

0.210.8

0.210.8

Q.210.8

0.3)0.7

0.3)0.7

0.3(0.7

0.3(0.7

0,4)0.6

0.4|0.6

0.4)0.6

0.4]0.6

0.4|0.6

[p: 6.15 %]

[p: 7.85 %]

[p: 6.55 %]

[p: 6.82 %]

[p: 6.63 %]

0.110.9

.1)0.9

0.1)0.9

0.1(0.9

0.1(0.9

0.210.8

.210.8

0.210.8

0.2)0.8

0.210.8

0.310.7

0.310.7

0.3)0.7

0.3(0.7

0.3(0.7

0.4)0.6

0.4)0.6

中

0.4)0.6

0.40.6

中

0.4|0.6

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[p: 9.37 %]

[p: 6.53 %]

[0: 8.20 %]

[p: 7.61 %]

[p: 8.18 %]

6'011'

199.9

1.2)0.8

d)0.7

p.310.7

10.7

.110.7

[p: 8.20 %]

[p: 5.91 %]

[p: 5.88 %]

[p: 6.51 %]

[p: 5.43 %]

0.10,9

0.110.9

0.1|0.9

0.1)0.9

0.1(0.9

0.2(0.8

10.7

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[p: 12.13 %]

[p: 11.87 %]

[p: 8.56 %]

[p: 6.31 %]

[p: 4.44 %]

0.10.9

0.110.9

0.110.9

0.210.8

.2(0.8

0.310

[p: 3.13 %]

[p: 2.09 %]

[p: 1.43 %]

[p: 3.84 %]

[p:3.03 %]

0.1)0.9

.1|0.9

0.1)0.9

0.10.9

0.2)0.8

0.2)0.8

0.2)0.8

0.2(0.8

D310.7

p.3)0.7

0.3)0.7

0.3(0.7

1.4)0.6

0.4|0.6

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[% 8/'91 :d]

[o: 12.97 %]

[p: 12.86 %]

[p: 5.77 %]

[% 2 :d]

0.110.9

0.10.9

0.10.9

0.110.9

0.110.9

0,2)0.8

0.210.8

0.2)0.8

0.2)0.8

0.2(0.8

0.3)0.7

.30.7

folt'o

0.3(0.7

p.3(0.7

0.4|0.6

[p: 2.55 %]

[p: 1.68 %]

[p: 2.92 %]

[p: 2.20 %]

[p: 4.13 %]

0.1)0.9

0.1)0.9

V.1/0.9

0.10.9

0.1(0.9

0.2)0.8

@.2)0.8

0.2)0.8

0.2)0.8

0.2(0.8

0.3)07

310.7

0.3(0.7

0.3/0,7

0.30.7

0.4)0.6

4/DU

0.4|0.6

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[p: 14.74 %]

[p: 10.06 %]

[0: 7.73 %]

[0: 5.05 %]

[p: 2.70 %]

0.110.9

0.110.9

0.110.9

0.110.9

0.110.9

0.210.8

.210.8

0.210.8

0.210.8

0.210.8

0.30.7

0.110.7

0.30.7

0.9.7

0.3(0.7

0.4)0.6

0.4|0.6

0.4)0.6

0.4|0.6

[0: 2.36 %]

[% 971 :0]

[p: 1.19 %]

[p: 2.07 %]

[p: 1.57 %]

0.10.9

0.1|0.9

0.1|0.9

0.10.9

0.10.9

0.2)0.8

.210.8

0.210.8

0.2)0.8

0.2(0.8

0.319.7

0.310,7

0.3)0.7

0.3(0.7

0.4)0.6

4)0.6

0.4)0.6

400.6

0.4|0.6

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[p: 15.21 %]

[p: 9.84 %]

[0: 8.20 %]

[p: 6.88 %]

[p: 3.45 %]

0.10.9

0.110.9

0.110.9

0.110.9

0.110.9

0.210.8

0.210.8

0.210.8

0.210.8

0.2104

0.3)0.7

0.3|0.7

0.3)0.7

0.30.7

.30.7

0.490.6

0.4|0.6

0.4)0.6

0.400.6

0.4|0.6

[p: 5.49 %]

[p: 8.34 %]

[0: 4.17 %]

[p: 5.86 %]

[p: 8.82 %]

0.110.9

0.1/0.9

0.1)0.9

0.1/0.9

0.10.9

.20.8

.210.8

210.8

0.2)0.8

0.2(0.8

0.3(0.7

0.3)0.7

0.3)0.7

0.3(0.7

0N0.7

-0.4)0.6

0.4)0.6

0.4)0.6

0.4)0.6

0.4(0.6

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[% 82°2 :0]

[p: 8.03 %]

[p: 3.44 %]

[p: 3.49 %]

[p: 2.48 %]

0.00110.999

0.001|0.999

0.0010.999

0.0010.999

66601100

0.10.2

.110.9

Q.1)0.9

0.10.9

0.10.9

0.210.8

0.210.8

0.2)0.8

0.2)0.8

.2/0.8

0.3)0.7

Q.3)0.7

0.30.7

0.3(0.7

0.4)0.6

0.4)0.6

0.4)0.6

0.4]0.6

0.4|0.6

[p: 1.10 %]

[p: 0.74 %]

[p: 0.81 %]

[p: 0.55 %]

[p: 0.45 %]

0.001|0.999

0.001)0.999

0.00110.999

0.00110.999

0.00110.999

0.110.9

.1/0.9

0.110.9

0.110.9

Q.10.9

0.210.8

.210.8

0.210.8

0.210.8

0.210.8

0.3(0.7

0.310.7

0.300.7

0.3(0.7

0.4/3.6

0.4|0.5

0.4/0.6

9.4:0.6

0.4(0.6

t

GLYPH<16>

1

.

0

t

GLYPH<16>

0

.

0

MDT

#1

MDT

#2

MDT

#3

MDT

#4

MDT

#5

t

GLYPH<16>

1

.

0

t

GLYPH<16>

0

.

0

MDT

#6

MDT

#7

MDT

#8

MDT

#9

MDT

#10

[o: 4.92 %]

[p: 5.84 %]

[% 9:0]

[p: 7.07 %]

[p: 7.87 %]

0.0010.999

0.001)0.999

1)0.999

.2/0.9

N0.110.9

.1/0.9

0.1(0.9

0.2(0.8

NQ.2(0.8

0.2(0.8

0.2(0.8

0.367

0.3)9.7

[o: 4.92 %]

[: 5.84 %]

[p: 6.24 %]

[% 202:0]

[0: 7.87 %]

0.001)0.99

[p: 8.06 %]

[p: 8.06 %]

[p: 8.70 %]

[p: 8.89 %]

[p: 7.89 %]

0.1/0.9

[p: 8.06 %]

[0: 8.06 %]

[0:9.13 %]

[p: 8.89 %]

[0:7.89 %]

770

t

GLYPH<16>

1

.

0

(

B

)

t

GLYPH<16>

0

.

6

t

GLYPH<16>

0

.

3

t

GLYPH<16>

0

.

0

0.110.9

0.00110.999

0.00110.999

0.00110.999

0.00110.999

0.310.7

0.210.8

0.310.7

0.210.8

0.110.9

0.310.7

0.210.8

0.110.9

0.3Q.7

0.2i0.8

0.110.9

0.410.6

0.410.6

0.4106

0.410,6

Poincaré disk vs t-self - UCI online_shoppers_intentions

online_shoppers_intention

p

t

GLYPH<16>

0

q

twitter

p

t

GLYPH<16>

0

q

Confidential + Proprietary

~just scaling

↗ distortion

Confidential + Proprietary

hardware

p

t

GLYPH<16>

70

q

hardware

p

t

GLYPH<16>

139

q

Poincaré disk vs t-self - UCI hardware, t=71

Poincaré disk vs t-self - UCI hardware, t=140

[p: 9.90 %]

B

B(0)

8.

0.0010.9

0.0010.999

0.1(0.9

0.00110.999

6660100

0.3(0.7

0.2(0.8

0.3107

0.2108

0.1(0.9

0.3/0.7

0.210.8

0.3107

0.2106

0.1|0.9

0.4(0.6

0.4(0.6

0.410.

[p: 0.80 %]

B

B

0.0010.999

0.0010.999

0.1|0.9

0.00110.999

0.1|0.9

0.0010.999

0.3(0.7

0.2(0.8

0.210)6

0.1|0.9

0.3(0.7

0.210.8

0.2106

0.1(0.9

0.310

0.3)0

0.4(0.6

400

,.4(0.6

Poincaré disk vs t-self - UCI twitter

be misjudged as equivalent from B alone (orange rectangle) but there is little double from B p t q 1 that one of them, which crosses the p GLYPH<0> . P t 0 . 001 , 0 . 009 u isoline, is in fact much better than the others. When many subtrees seem to be aggregating near the border as in buzz_in_social_media , stark differences can appear on the t-self: the best subtrees are immediately spotted from the t-self (orange rectangles). In between, the t-self makes a much more visible ordering between the best nodes and subtrees, compared to Poincaré disk. hardware demonstrate that such very good nodes that are hard to differentiate from the others in B can appear at any iteration.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Claims are supported by both the theoretical results presented in the paper or experimental exploration.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations of the proposed embedding method are discussed. For instance, the requirement of only embedding the MDT are discussed. Impacts on classification due to this are explored in the 'MDT vs DT classification' discussion, with further details in Appendix. A last paragraph in the conclusion also discusses limitations.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Assumptions are detailed in formal results and full proofs are provided in the Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Detailed settings of constructing visualizations are presented. Code provided with an example public domain and a resource file + README.txt for quick testing and validation.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code provided with an example public domain and a resource file + README.txt for quick testing and validation.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Details are provided in main text and Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Error is stated whenever suitable. Multiple results are presented for visualizations.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA] .

Justification: The code runs on any standard computer.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research of the paper follows the code of ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Broader impact and societal impact is briefly mentioned throughout the main text, with further discussion presented in an Appendix section.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: No release of data or models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Appropriate credit / referencesse are provided. Licenses listed in Appendix.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets provided.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: No research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary\nThis paper proposes a novel approach to embed supervised models, specifically decision trees and their ensembles, in hyperbolic space using the Poincaré disk model. The paper contributes three main advancements: linking loss functions for class probability estimation to hyperbolic embeddings, providing a method for unambiguous embedding of decision trees by extracting their monotonic subtrees, and introducing a tempered integral to improve the visualization and numerical accuracy of embeddings near the border of the Poincaré disk.\n\nStrengths\n1. The approach to embedding supervised models, particularly decision trees, in hyperbolic space is novel. The focus on supervised models rather than just the data is a significant contribution.\n2. This paper provides a robust theoretical framework linking class probability estimation losses with hyperbolic embeddings and introducing the concept of monotonic decision trees (MDTs) for clean embeddings.\n3. The introduction of the tempered integral to handle numerical issues near the border of the Poincaré disk demonstrates a deep understanding of the challenges in hyperbolic geometry and offers a principled solution.\n4. This paper is well-structured, with clear definitions, thorough explanations of the proposed methods, and detailed experimental results. The figures and tables effectively support the textual descriptions.\n\nWeaknesses and comments\n1. The theoretical aspects, particularly the tempered integral and its application, might be challenging for practitioners to fully grasp and implement without further guidance or examples.\n2. While the paper demonstrates the applicability to decision trees and their ensembles, it would benefit from a discussion on how the methods could be adapted or extended to other types of supervised models.\n3. While the experiments are thorough, including more diverse use cases, such as different types of datasets or real-world applications, would strengthen the demonstration of the method's versatility and robustness.\n4. The related work section is comprehensive, covering both hyperbolic embeddings and supervised learning, which helps to contextualize the contributions.\n5. The methodology sections are detailed and well-explained. The separation into three main contributions helps in understanding the different components of the proposed approach.\n\nSTRENGTHS:\nStrengths\n1. The approach to embedding supervised models, particularly decision trees, in hyperbolic space is novel. The focus on supervised models rather than just the data is a significant contribution.\n2. This paper provides a robust theoretical framework linking class probability estimation losses with hyperbolic embeddings and introducing the concept of monotonic decision trees (MDTs) for clean embeddings.\n3. The introduction of the tempered integral to handle numerical issues near the border of the Poincaré disk demonstrates a deep understanding of the challenges in hyperbolic geometry and offers a principled solution.\n4. This paper is well-structured, with clear definitions, thorough explanations of the proposed methods, and detailed experimental results. The figures and tables effectively support the textual descriptions.\n\nWEAKNESSES:\nWeaknesses and comments\n1. The theoretical aspects, particularly the tempered integral and its application, might be challenging for practitioners to fully grasp and implement without further guidance or examples.\n2. While the paper demonstrates the applicability to decision trees and their ensembles, it would benefit from a discussion on how the methods could be adapted or extended to other types of supervised models.\n3. While the experiments are thorough, including more diverse use cases, such as different types of datasets or real-world applications, would strengthen the demonstration of the method's versatility and robustness.\n4. The related work section is comprehensive, covering both hyperbolic embeddings and supervised learning, which helps to contextualize the contributions.\n5. The methodology sections are detailed and well-explained. The separation into three main contributions helps in understanding the different components of the proposed approach.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a hyperbolic implementation of decision tree models, providing a link providing a link between class probability estimation and hyperbolic distances. In addition the authors identify key objectives/criteria for enabling “clean” hyperbolic embeddings, which is achieved by maintaining monotonic part of the tree thus avoiding shared root nodes at the origin of the poincare ball. The authors also address the inherent collapse of embeddings to the boundary of the ball by the introduction of a t-self method which also improved the qualitative visualisations of embeddings on the poincare ball.\n\nSTRENGTHS:\n- This work addresses many fundamental issues in hyperbolic learning presenting some original concepts linking class probability estimation and hyperbolic distances and addressing boundary collapse, thus presenting an interesting direction and findings for the community.\n- The proposed t-self method addresses a key problem across hyperbolic distance metrics that seemingly helps avoid collapse to the boundary of the ball.\n- Significant theoretical proofs are provided to support many of the claims and assumptions of the work.\n\nWEAKNESSES:\n- It is not clear that the statement made that supervised models are not embedded in hyperbolic space. Some examples of fully hyperbolic classification models include [1,2]. I may be misunderstanding, therefore, correct me if so. In addition, to clarify this point, the authors could provide some evidence to back up this claim as currently it is weak. In addition, supervised hyperbolic geometry is well established which the authors claim otherwise, and hyperbolic decision trees have previously been proposed in [2]\n- The presentation and writing overall is generally poor and hard to follow, notably the bold headings in section 4. The use of mathematical notation is poorly integrated in some settings and largely impedes the readability of the work.\n- The experimental results are purely demonstrated as visual comparisons, and no quantitive measures are presented.\n- Arguably the t-self method you propose to improve visualisation is a distance metric that helps avoid collapse to the boundary of the ball. While helping visualisation should not be positioned in the contributions as a method for visualisation.\n- Experimental/implementation settings are omitted from the work limiting the ability of replication. \n\n\nMinor:\n- The presentation is poor in many places notably the experimental analysis, while the style changes significantly throughout the work.\n- The related work is missing key works and is generally vague in relation to the problem statement.\nGeneral grammar and writing:\n- Line 61 \n- Line 106 is this incomplete? \n- $d^*$ and $d_*$ are used interchangeably.\n\n[1] Doorenbos, L., Márquez-Neila, P., Sznitman, R. and Mettes, P., 2023. Hyperbolic Random Forests. arXiv preprint arXiv:2308.13279.\n\n[2] Chlenski, P., Turok, E., Moretti, A. and Pe'er, I., 2023. Fast hyperboloid decision tree algorithms. arXiv preprint arXiv:2310.13841.\n\nQUESTIONS:\n- On Line 60 you state insist that these models do not represent models in hyperbolic space, why? This needs elaborating.\n- How does the proposed method compare to Euclidean models? What is the practical benefit of employing your method?\n- You suggest the t-self may be useful as a standard encoding in hyperbolic space, have you made any empirical findings to support this?\n- The limitations of only embedding the monotonic DT are not discussed? It may assist for clean embeddings but what are the drawbacks?\n\nLIMITATIONS:\nSome limitations are questioned above, while broader impacts of the work are addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a method to embed decision trees into hyperbolic space, linking class probability estimation with hyperbolic distances. It proposes extracting monotonic subtrees and using a \"tempered\" integral for better visualization. The approach aims to enhance model interpretability and analysis without changing model performance.\n\nSTRENGTHS:\nThe paper presents an approach to embedding decision trees in hyperbolic space, offering a novel perspective on model visualization and interpretation. By leveraging the properties of hyperbolic geometry, particularly the Poincaré disk model, the authors provide a mathematically grounded method for representing complex tree structures and confidence levels in a unified, visually intuitive manner, potentially enhancing our ability to analyze and understand sophisticated machine learning models.\n\nWEAKNESSES:\n**Limited empirical validation**: The authors present their method primarily on small-scale UCI datasets, which may not fully demonstrate the scalability and effectiveness of their approach on **large, real-world machine learning problems**. The lack of comparison with existing tree visualization techniques, such as those in popular packages like scikit-learn or XGBoost, makes it difficult to gauge the practical advantages of this method.\n\n**Potential computational overhead**: The paper does not adequately address the **computational complexity** of transforming decision trees into their hyperbolic representations. For large ensembles or deep trees, this transformation could potentially introduce significant overhead, which might limit its applicability in **time-sensitive or resource-constrained environments**.\n\n**Ambiguity in interpretability gains**: While the authors claim improved interpretability, they do not provide a **quantitative measure or user study** to substantiate this claim. It remains unclear whether domain experts or data scientists would find these hyperbolic representations more intuitive or actionable than traditional tree visualizations.\n\nQUESTIONS:\nHow does the method perform on regression trees as opposed to classification trees? Are there any significant differences in the representation or interpretation?\n\nHow does this hyperbolic embedding method perform on highly imbalanced datasets, particularly in visualizing minority class decision boundaries?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an approach to embed the models (not data) into hyperbolic spaces. Hyperbolic spaces are especially suited for embeddings of hierarchies, but recent works have so far focused on embeddings of data.\nThe paper presents a framework consisting of measures to estimate the confidence of model's predictions and the scheme to embed the decision tree model into hyperbolic space together with the postprocessing procedure. The authors evaluate the proposed approach and notice that more confident and accurate predictions tend to be closer to the border of the Poincare ball.\n\nSTRENGTHS:\n- The paper is well written.\n- The approach appears very original to me, I am not familiar with the methods similar to the presented work. I also find the angle of embedding the models (and not the data) interesting and non-trivial. \n- The approach is well motivated and backed up by theoretical explanations provided by authors. The mathematical justifications are rigorously presented in Appendix.\n- In my opinion, the methods for embeddings of discrete data into hyperbolic space is less covered in the literature, so I believe that adding more methods covering this topic would benefit the community. The authors show how exactly their approach differs from a baseline by Sarkar.\n\nWEAKNESSES:\n- Some visualizations are hard to interpret. Please also see a remark in 'Questions'.\n- In my opinion, the paper would have benefited from a more clear presentation of quantitative results. E.g.,  there is a discussion of the embedding error in Section 6, but the numbers listed in the text seem not to cover all the datasets considered in the analysis.\nAnother table that seem to provide quantitative results is Table II in Appendix showing difference between DT and MDT (its reduced hyperbolic counterpart). Again, I find it challenging to interpret.\n\nQUESTIONS:\n- A small remark: the most part of plots use red/green colours which can be challenging to distinguish for colour blind people. If it is possible, I would encourage authors to change the colour scheme or at least keep that in mind for future submissions. \n- The sentence in line 697 is unclear.\n\nMy main concern regarding the paper is related to the difficulties while evaluating the performance of the approach and it mainly justifies my rating. However, I am open for discussions with the authors.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations. I see no potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}