Original: L ORENTZIAN D ISTANCE L EARNING
Revised: L ORENTZIAN D ISTANCE L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: This paper introduces an approach to learn representations based on the Lorentzian distance in hyperbolic geometry.
Revised: This paper introduces an approach to learn representations based on the Lorentzian distance in hyperbolic geometry.

Original: Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world.
Revised: Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world.

Original: Current hyperbolic representation learning methods compare examples with the Poincar´e distance metric.
Revised: Current hyperbolic representation learning methods compare examples with the Poincar´e distance metric.

Original: They formulate the problem as minimizing the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes.
Revised: They formulate the problem as minimizing the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes.

Original: This formulation produces node representations close to the centroid of their descendants.
Revised: This formulation produces node representations close to the centroid of their descendants.

Original: We exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form.
Revised: We exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form.

Original: We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases.
Revised: We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases.

Original: This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children.
Revised: This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children.

Original: Our approach obtains state-of-the-art results in retrieval and classiﬁcation tasks on different datasets
Revised: Our approach obtains state-of-the-art results in retrieval and classiﬁcation tasks on different datasets

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generalizations of Euclidean space are important forms of data representation in machine learning.
Revised: Generalizations of Euclidean space are important forms of data representation in machine learning.

Original: For instance, kernel methods rely on Hilbert spaces that possess the structure of the inner product and can therefore be used to compare examples.
Revised: For instance, kernel methods rely on Hilbert spaces that possess the structure of the inner product and can therefore be used to compare examples.

Original: The properties of such spaces are well-known and closed-form relations are often exploited to obtain efﬁcient, scalable, and interpretable training algorithms.
Revised: The properties of such spaces are well-known and closed-form relations are often exploited to obtain efﬁcient, scalable, and interpretable training algorithms.

Original: While representing examples in a Euclidean space is appropriate to compare lengths and angles, non-Euclidean representations are useful when the task requires speciﬁc structure.
Revised: While representing examples in a Euclidean space is appropriate to compare lengths and angles, non-Euclidean representations are useful when the task requires speciﬁc structure.

Original: A common and natural non-Euclidean representation space is the spherical model ( e.g. (Wang et al., 2017)) where the data lies on a unit
Revised: A common and natural non-Euclidean representation space is the spherical model ( e.g. (Wang et al., 2017)) where the data lies on a unit

Original: hypersphere S d = { x ∈ R
Revised: hypersphere S d = { x ∈ R

Original: d : (cid:107) x (cid:107) = 1 } and angles are compared with the cosine similarity function.
Revised: d : (cid:107) x (cid:107) = 1 } and angles are compared with the cosine similarity function.

Original: Recently, some machine learning approaches (Nickel & Kiela, 2017; 2018; Ganea et al., 2018) have considered representing hierarchical datasets with the hyperbolic model.
Revised: Recently, some machine learning approaches (Nickel & Kiela, 2017; 2018; Ganea et al., 2018) have considered representing hierarchical datasets with the hyperbolic model.

Original: The motivation is that any ﬁnite tree can be mapped into a ﬁnite hyperbolic space while approximately preserving distances (Gromov, 1987).
Revised: The motivation is that any ﬁnite tree can be mapped into a ﬁnite hyperbolic space while approximately preserving distances (Gromov, 1987).

Original: Since hierarchies can be formulated as trees, hyperbolic spaces can be used to represent hierarchically structured data where the high-level nodes of the hierarchy are represented close to the origin ( i.e. with small Euclidean norm) whereas leaves are further away from the origin.
Revised: Since hierarchies can be formulated as trees, hyperbolic spaces can be used to represent hierarchically structured data where the high-level nodes of the hierarchy are represented close to the origin ( i.e. with small Euclidean norm) whereas leaves are further away from the origin.

Original: Since their ﬁrst formulation in the early nineteenth century by Lobachevsky and Bolyai, hyperbolic spaces have been used in many domains.
Revised: Since their ﬁrst formulation in the early nineteenth century by Lobachevsky and Bolyai, hyperbolic spaces have been used in many domains.

Original: In particular, they became popular in mathematics ( e.g. space theory and differential geometry (Ratcliffe, 2006)), and physics when Varicak (1908) discovered that special relativity theory (Einstein, 1905) had a natural interpretation in hyperbolic geometry.
Revised: In particular, they became popular in mathematics ( e.g. space theory and differential geometry (Ratcliffe, 2006)), and physics when Varicak (1908) discovered that special relativity theory (Einstein, 1905) had a natural interpretation in hyperbolic geometry.

Original: Various hyperbolic geometries and related distances have been studied since then.
Revised: Various hyperbolic geometries and related distances have been studied since then.

Original: Among them are the Poincar´e metric, the Lorentzian distance (Ratcliffe, 2006), and the gyrodistance (Ungar, 2010).
Revised: Among them are the Poincar´e metric, the Lorentzian distance (Ratcliffe, 2006), and the gyrodistance (Ungar, 2010).

Original: In the case of hierarchical datasets, machine learning approaches that learn hyperbolic representations designed to preserve the hierarchical similarity order have typically employed the Poincar´e metric.
Revised: In the case of hierarchical datasets, machine learning approaches that learn hyperbolic representations designed to preserve the hierarchical similarity order have typically employed the Poincar´e metric.

Original: Usually, the optimization problem is formulated so that the representation of a node in a hierarchy should be closer to the representation of its children and other descendants than to any other node in the hierarchy.
Revised: Usually, the optimization problem is formulated so that the representation of a node in a hierarchy should be closer to the representation of its children and other descendants than to any other node in the hierarchy.

Original: Based on Gromov (1987), the Poincar´e metric is a sensible dissimilarity function as it satisﬁes all the properties of a distance metric and is thus natural to interpret.
Revised: Based on Gromov (1987), the Poincar´e metric is a sensible dissimilarity function as it satisﬁes all the properties of a distance metric and is thus natural to interpret.

Original: In this paper, we propose to exploit the squared Lorentzian distance instead of the usual Poincar´e distance.
Revised: In this paper, we propose to exploit the squared Lorentzian distance instead of the usual Poincar´e distance.

Original: The squared Lorentzian distance is linear in the hyperbolic representations and differentiable on the whole domain, which avoids numerical instabilities and exploding gradients.
Revised: The squared Lorentzian distance is linear in the hyperbolic representations and differentiable on the whole domain, which avoids numerical instabilities and exploding gradients.

Original: Moreover, the order of the Poincar´e distances between pairs of points is the same as the order of the (squared)
Revised: Moreover, the order of the Poincar´e distances between pairs of points is the same as the order of the (squared)

Original: Lorentzian distances in the unit hyperboloid model.
Revised: Lorentzian distances in the unit hyperboloid model.

Original: We may therefore directly adapt existing approaches by replacing Poincar´e distances with squared Lorentzian distances.
Revised: We may therefore directly adapt existing approaches by replacing Poincar´e distances with squared Lorentzian distances.

Original: The main property of interest is that the center of mass w.r.t.
Revised: The main property of interest is that the center of mass w.r.t.

Original: the squared Lorentzian distance can be written in closed form and is then easy to interpret.
Revised: the squared Lorentzian distance can be written in closed form and is then easy to interpret.

Original: In particular, we show that the Euclidean norm of the center of mass of a set of different points decreases as the curvature of the exploited hyperboloid model 1 decreases and tends to −∞ .
Revised: In particular, we show that the Euclidean norm of the center of mass of a set of different points decreases as the curvature of the exploited hyperboloid model 1 decreases and tends to −∞ .

Original: This property makes it ideal to represent hierarchical structures since we would like the root of a subtree to minimize the distance to all of its descendants ( i.e. it should be their centroid w.r.t.
Revised: This property makes it ideal to represent hierarchical structures since we would like the root of a subtree to minimize the distance to all of its descendants ( i.e. it should be their centroid w.r.t.

Original: to the chosen distance function) and have smaller Euclidean norm than them.
Revised: to the chosen distance function) and have smaller Euclidean norm than them.

Original: Our contributions are both analytic and experimental.
Revised: Our contributions are both analytic and experimental.

Original: We study properties of the chosen dissimilarity function and explain why they make it appropriate for the task of interest.
Revised: We study properties of the chosen dissimilarity function and explain why they make it appropriate for the task of interest.

Original: In particular, we show that the Euclidean norm of the center of mass in the Poincar´e ball representation decreases as the curvature −
Revised: In particular, we show that the Euclidean norm of the center of mass in the Poincar´e ball representation decreases as the curvature −

Original: 1 /β ∈ R − of the hyperboloid model decreases ( i.e. as β > 0 decreases).
Revised: 1 /β ∈ R − of the hyperboloid model decreases ( i.e. as β > 0 decreases).

Original: Curvature can then be used as a hyperparameter to implicitly enforce the root of a subtree to have smaller Euclidean norm than the other nodes.
Revised: Curvature can then be used as a hyperparameter to implicitly enforce the root of a subtree to have smaller Euclidean norm than the other nodes.

Original: We experimentally show on different problems that exploiting the squared
Revised: We experimentally show on different problems that exploiting the squared

Original: Lorentzian distance improves retrieval and classiﬁcation performance.
Revised: Lorentzian distance improves retrieval and classiﬁcation performance.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: In this section, we provide some technical background about hyperbolic geometry and introduce relevant notation.
Revised: In this section, we provide some technical background about hyperbolic geometry and introduce relevant notation.

Original: The interested reader may refer to (Ratcliffe, 2006) for more detail.
Revised: The interested reader may refer to (Ratcliffe, 2006) for more detail.

Original: 2. N OTATION AND DEFINITIONS
Revised: 2. N OTATION AND DEFINITIONS

Original: To simplify the notation, we consider that vectors are row vectors and (cid:107) · (cid:107) is the (cid:96) 2 -norm.
Revised: To simplify the notation, we consider that vectors are row vectors and (cid:107) · (cid:107) is the (cid:96) 2 -norm.

Original: In the following, we consider three important spaces.
Revised: In the following, we consider three important spaces.

Original: Poincar´e ball: The Poincar´e ball P d is deﬁned as the set of d -dimensional vectors with Euclidean norm smaller than 1 ( i.e. P d = { x ∈ R d : (cid:107) x (cid:107) < 1 } ).
Revised: Poincar´e ball: The Poincar ´ e ball P d is deﬁned as the set of d -dimensional vectors with Euclidean norm smaller than 1 ( i.e. P d = { x ∈ R d : (cid:107) x (cid:107) < 1 } ).

Original: Its associated distance is the Poincar ´ e distance metric deﬁned in Eq.
Revised: Its associated distance is the Poincar ´ e distance metric deﬁned in Eq.

Original: Hyperboloid model: We consider some speciﬁc hyperboloid models H d,β deﬁned as follows:
Revised: Hyperboloid model: We consider some speciﬁc hyperboloid models H d,β deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: where (cid:107) a (cid:107) 2 L = (cid:104) a , a (cid:105) L is the squared Lorentzian norm of a .
Revised: where (cid:107) a (cid:107) 2 L = (cid:104) a , a (cid:105) L is the squared Lorentzian norm of a .

Original: The squared Lorentzian norm is derived from the Lorentzian inner product deﬁned as:
Revised: The squared Lorentzian norm is derived from the Lorentzian inner product deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: It is worth noting that (cid:104) a , b (cid:105) L = − β iff a = b .
Revised: It is worth noting that (cid:104) a , b (cid:105) L = − β iff a = b .

Original: Otherwise, (cid:104) a , b (cid:105) L < − β for all pair ( a , b ) ∈
Revised: Otherwise, (cid:104) a , b (cid:105) L < − β for all pair ( a , b ) ∈

Original: ( H d,β ) 2 .
Revised: ( H d,β ) 2 .

Original: Vectors in H d,β are a subset of positive time-like vectors 2 .
Revised: Vectors in H d,β are a subset of positive time-like vectors 2 .

Original: Moreover, every vector a ∈ H d,β satisﬁes a 20 + β =  (cid:80) di =1 a 2 i .
Revised: Moreover, every vector a ∈ H d,β satisﬁes a 0 = (cid:113) β + (cid:80) di =1 a 2 i .

Original: We note H d :=
Revised: We note H d :=

Original: H d, 1 the space obtained when β = 1 ; it is usually called the  unit hyperboloid model and is the main hyperboloid model considered in the literature.
Revised: H d, 1 the space obtained when β = 1 ; it is usually called the (positive sheet of the) unit hyperboloid model and is the main hyperboloid model considered in the literature.

Original: Model space: Finally, we note F d  the output space of our model ( e.g. the output representation of some linear model or neural network).
Revised: Model space: Finally, we note F d ⊆ R d the output space of our model ( e.g. the output representation of some linear model or neural network).

Original: In the following, we consider that F d = R d .
Revised: In the following, we consider that F d = R d .

Original:  O PTIMIZING THE P OINCAR ´ E DISTANCE METRIC
Revised: 2. O PTIMIZING THE P OINCAR ´ E DISTANCE METRIC

Original: Most methods that compare hyperbolic representations (Nickel & Kiela, 2017; 2018; Ganea et al., 2018; Gulcehre et al., 2018) consider the Poincar ´ e distance metric deﬁned as:
Revised: Most methods that compare hyperbolic representations (Nickel & Kiela, 2017; 2018; Ganea et al., 2018; Gulcehre et al., 2018) consider the Poincar´e distance metric deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: which satisﬁes all the properties of a distance metric and is therefore natural to interpret.
Revised: which satisﬁes all the properties of a distance metric and is therefore natural to interpret.

Original: Direct optimization of problems using the distance formulation in Eq.
Revised: Direct optimization of problems using the distance formulation in Eq.

Original: (3) is numerically instable for two main reasons (see for instance (Nickel & Kiela, 2018) or (Ganea et al., 2018, Section 4)).
Revised: (3) is numerically instable for two main reasons (see for instance (Nickel & Kiela, 2018) or (Ganea et al., 2018, Section 4)).

Original: First, the denominator depends on the norm of examples, so optimizing over c and d when either of their norms is close to 1 leads to numerical instability.
Revised: First, the denominator depends on the norm of examples, so optimizing over c and d when either of their norms is close to 1 leads to numerical instability.

Original: Second, elements have to be re-projected onto the
Revised: Second, elements have to be re-projected onto the

Original: Poincar´e ball at each iteration with a ﬁxed maximum norm.
Revised: Poincar´e ball at each iteration with a ﬁxed maximum norm.

Original: To increase the numerical stability of their solver, Nickel & Kiela (2018) propose to use an equivalent formulation of d P in the unit hyperboloid model.
Revised: To increase the numerical stability of their solver, Nickel & Kiela (2018) propose to use an equivalent formulation of d P in the unit hyperboloid model.

Original: They use the fact that there exists an invertible mapping h : H d,β → P d deﬁned as:
Revised: They use the fact that there exists an invertible mapping h : H d,β → P d deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: When β = 1 , we have the following equivalence:
Revised: When β = 1 , we have the following equivalence:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which corresponds to the (Lorentzian) angle between a and b .
Revised: which corresponds to the (Lorentzian) angle between a and b .

Original: It is shown in (Nickel & Kiela, 2018) that optimizing the formulation in Eq.
Revised: It is shown in (Nickel & Kiela, 2018) that optimizing the formulation in Eq.

Original: (5) is more stable numerically.
Revised: (5) is more stable numerically.

Original: Nonetheless, the gradient of Eq.
Revised: Nonetheless, the gradient of Eq.

Original: (5) still tends to + ∞ as d H ( a , b ) tends to 0.
Revised: (5) still tends to + ∞ as d H ( a , b ) tends to 0.

Original: Interestingly, one can observe from Eq.
Revised: Interestingly, one can observe from Eq.

Original: (5) that preserving the order of Poincar ´
Revised: (5) that preserving the order of Poincar ´

Original: e distances is equivalent to preserving the order of the opposite of Lorentzian inner products since the arcosh function is monotonically increasing on its domain [1, + ∞ ).
Revised: e distances is equivalent to preserving the order of the opposite of Lorentzian inner products since the arcosh function is monotonically increasing on its domain [1, + ∞ ).

Original: In the same way as kernel methods that consider inner products in Hilbert spaces as similarity measures, we consider in this paper the Lorentzian inner product.
Revised: In the same way as kernel methods that consider inner products in Hilbert spaces as similarity measures, we consider in this paper the Lorentzian inner product.

Original:  P ROPOSED METHOD
Revised:  P ROPOSED METHOD

Original: We present in this section the (squared) Lorentzian distance function d L (Ratcliffe, 2006) and mention the properties that make it appropriate for our task.
Revised: We present in this section the (squared) Lorentzian distance function d L (Ratcliffe, 2006) and mention the properties that make it appropriate for our task.

Original: We give the formulation of its center of mass and observe that its Euclidean norm depends on the curvature − 1 /β which can then be used as a hyper-parameter of the model to implicitly enforce centroids to have small  norm.
Revised: We give the formulation of its center of mass and observe that its Euclidean norm depends on the curvature − 1 /β which can then be used as a hyper-parameter of the model to implicitly enforce centroids to have small Euclidean norm.

Original: We also discuss optimization details.
Revised: We also discuss optimization details.

Original: 3. L ORENTZIAN DISTANCE AND MAPPINGS
Revised: 3. L ORENTZIAN DISTANCE AND MAPPINGS

Original: The squared Lorentzian distance (Ratcliffe, 2006) is deﬁned as:
Revised: The squared Lorentzian distance (Ratcliffe, 2006) is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: It satisﬁes all the axioms of a distance metric except the triangleinequality. It actually satisﬁes the reversed triangle inequality ( i.e. d L ( a , b ) >
Revised: It satisﬁes all the axioms of a distance metric except the triangle inequality ( i.e. non-negativity, symmetry, identity of indiscernibles).

Original: d L ( a , c ) + d L ( c , b ) if a , b and c are different points in H d,β ).

Original: One interesting property is that d 2 L is linear in both a and b .
Revised: One interesting property is that d 2 L is linear in both a and b .

Original: Moreover, when β = 1 , it deﬁnes the hyperbolic length of a curve in H d (Ratcliffe, 2006).
Revised: Moreover, when β = 1 , it deﬁnes the hyperbolic length of a curve in H d (Ratcliffe, 2006).

Original: Mapping: Current hyperbolic machine learning models re-project at each iteration their learned representations onto the Poincar´e ball (Nickel & Kiela, 2017) or use some exponential map (Nickel & Kiela, 2018; Gulcehre et al., 2018) to directly optimize on the unit hyperboloid model ( i.e. when β = 1 ).
Revised: Mapping: Current hyperbolic machine learning models re-project at each iteration their learned representations onto the Poincar´e ball (Nickel & Kiela, 2017) or use some exponential map (Nickel & Kiela, 2018; Gulcehre et al., 2018) to directly optimize on the unit hyperboloid model ( i.e. when β = 1 ).

Original: Since our approach does not necessarily consider that β
Revised: Since our approach does not necessarily consider that β

Original: = 1 , we consider the invertible mapping g β : F d → H d,β deﬁned as:
Revised: = 1 , we consider the invertible mapping g β : F d → H d,β deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: As mentioned in Section 2. [Equation] F d is the space of our model; it is R d in practice.
Revised: As mentioned in Section 2.1, F d is the space of our model; it is R d in practice.

Original: To be fair with baselines, the examples in F d in our experiments are non-parametric embeddings.
Revised: To be fair with baselines, the examples in F d in our experiments are non-parametric embeddings.

Original: F d may also be the output space of some parametric model such as a neural network
Revised: F d may also be the output space of some parametric model such as a neural network

Original: [Equation]
Revised: [Equation]

Original: We compare two examples f 1 ∈ F d and f 2 ∈ F d with d 2 L by calculating:
Revised: We compare two examples f 1 ∈ F d and f 2 ∈ F d with d 2 L by calculating:

Original: [Equation]
Revised: [Equation]

Original: Preserved order of Euclidean norms: Although examples are compared with d 2 L in the hyperbolic space H d,β where all the points have the same Lorentzian norm, it is worth noting that the order of the Euclidean norms of examples is preserved along the three spaces F d , H d,β and P d with the  g β and h .
Revised: Preserved order of Euclidean norms: Although examples are compared with d 2 L in the hyperbolic space H d,β where all the points have the same Lorentzian norm, it is worth noting that the order of the Euclidean norms of examples is preserved along the three spaces F d , H d,β and P d with the mappings g β and h .

Original: The preservation with g β is straightforward: ∀ f 1 , f 2 ∈ F d , (cid:107) f 1 (cid:107) < (cid:107) f 2 (cid:107) ⇐⇒ (cid:112) 2 (cid:107) f 1 (cid:107) 2 + β = (cid:107) g β ( f 1 ) (cid:107) <
Revised: The preservation with g β is straightforward: ∀ f 1 , f 2 ∈ F d , (cid:107) f 1 (cid:107) < (cid:107) f 2 (cid:107) ⇐⇒ (cid:112) 2 (cid:107) f 1 (cid:107) 2 + β = (cid:107) g β ( f 1 ) (cid:107) <

Original: (cid:107) g β ( f 2 ) (cid:107) .
Revised: (cid:107) g β ( f 2 ) (cid:107) .

Original: The proof of the following theorem is given in the appendix:
Revised: The proof of the following theorem is given in the appendix:

Original: Theorem 3.1 (Order of Euclidean norms) .
Revised: Theorem 3.1 (Order of Euclidean norms) .

Original: The following order of norms is preserved with h ◦ g :
Revised: The following order of norms is preserved with h ◦ g :

Original: [Equation]
Revised: [Equation]

Original: In conclusion, the Euclidean norms of examples can be compared equivalently in any space.
Revised: In conclusion, the Euclidean norms of examples can be compared equivalently in any space.

Original: This is particularly useful if we want to study the Euclidean norm of centroids w.r.t.
Revised: This is particularly useful if we want to study the Euclidean norm of centroids w.r.t.

Original: examples.
Revised: examples, and also understand the signiﬁcance of the following section.

Original: 3. C ENTROID PROPERTIES
Revised: 3. C ENTROID PROPERTIES

Original: We now study the center of mass w.r.t.
Revised: We now study the center of mass w.r.t.

Original: the squared Lorentzian distance.
Revised: the squared Lorentzian distance.

Original: Ideally, we would like the centroid of node representations to be (close to) the representation of their least common ancestor.
Revised: Ideally, we would like the centroid of node representations to be (close to) the representation of their least common ancestor.

Original: Lemma 3.2 (Center of mass of the Lorentziann inner product) .
Revised: Lemma 3.2 (Center of mass of the Lorentzian inner product) .

Original: the point µ ∈ H d,β that maximizes the following problem max µ ∈H d,β (cid:80) ni
Revised: the point µ ∈ H d,β that maximizes the following problem max µ ∈H d,β (cid:80) ni

Original: =

Revised: =

Original: ν i (cid:104) x
Revised: ν i (cid:104) x

Original: i , µ (cid:105) L where ∀ i, x i ∈ H d,β , ∀ i, ν i ≥ 0 , (cid:80) i ν i > 0 is:µ = (cid:112) β (cid:80) ni
Revised: i , µ (cid:105) L where ∀ i, x i ∈ H d,β , ∀ i, ν i ≥ 0 , (cid:80) i ν i > 0 is:

Original: [Equation]
Revised: [Equation]

Original: where |(cid:107) a (cid:107) L | is the modulus of the imaginary Lorentzian norm of the positive time-like vector a .
Revised: where |(cid:107) a (cid:107) L | is the modulus of the imaginary Lorentzian norm of the positive time-like vector a .

Original: The proof is given in the appendix.
Revised: The proof is given in the appendix.

Original: Theorem 3.3 (Centroid of the squared Lorentzian distance) .
Revised: Theorem 3.3 (Centroid of the squared Lorentzian distance) .

Original: the point µ ∈ H d,β that minimizes the problem min µ ∈H d,β (cid:80) ni
Revised: The point µ ∈ H d,β that minimizes the problem min µ ∈H d,β (cid:80) ni

Original: =

Revised: =

Original: ν i d 2 L ( x i , µ ) where ∀ i, x
Revised: ν i d 2 L ( x i , µ ) where ∀ i, x

Original: i ∈ H d,β , ν i ≥ 0 , (cid:80)
Revised: i ∈ H d,β , ν i ≥ 0 , (cid:80)

Original: i ν i > 0 is given in Eq.
Revised: i ν i > 0 is given in Eq.

Original: The proof exploits the formulation given in Eq.
Revised: The proof exploits the formulation given in Eq.

Original: From Eq.
Revised: From Eq.

Original: (11), one can see that the centroid of one example is the example itself.
Revised: (11), one can see that the centroid of one example is the example itself.

Revised: Theorem 3.4.

Original: The Euclidean norm in P d of the center of mass of a set of different examples decreases as β > 0 decreases (see proof  in appendix).
Revised: The Euclidean norm in P d of the center of mass of a set of different examples decreases as β > 0 decreases. The proof is given in the appendix.

Original: Fig.
Revised: Fig.

Original: 1 illustrates the 2-dimensional Poincar ´ e ball representation of the centroid of a set of 10 different points w.r.t.
Revised: 1 illustrates the 2-dimensional Poincar ´ e ball representation of the centroid of a set of 10 different points w.r.t.

Original: to the Poincar ´ e metric and the squared Lorentzian distance for different values of β > 0 .
Revised: to the Poincar´e metric, the gyrodistance (Ungar, 2010), and the squared Lorentzian distance for different values of β > 0 .

Original: The Euclidean norm in P d of the centroid does decrease as β decreases.
Revised: The Euclidean norm in P d of the centroid does decrease as β decreases.

Original: We provide in the following some side remarks that are useful to understand the behavior of the
Revised: We provide in the following some side remarks that are useful to understand the behavior of the

Original: Lorentzian distance.
Revised: Lorentzian distance.

Original: In particular, we explain its behavior in extreme cases when β tends to 0.
Revised: In particular, we explain its behavior in extreme cases when β tends to 0.

Original: Theorem 3.4 (Nearest point of the squared Lorentzian distance) .
Revised: Theorem 3.5 (Nearest point of the squared Lorentzian distance) .

Original: the point a j ∈ A = { a k ∈
Revised: The point a j ∈ A = { a k ∈

Original: H d,β } mk =1 that minimizes thefollowing problem min a j ∈A (cid:80) ni
Revised: H d,β } mk =1 that minimizes the problem min a j ∈A (cid:80) ni

Original: =1 d 2 L ( x i , a j ) where ∀ i, x i ∈
Revised: =1 d 2 L ( x i , a j ) where ∀ i, x i ∈

Original: H d,β is the example in A that is the closest to the centroid  w.r.t.
Revised: H d,β is the example in A that is the closest to the centroid of x 1 , · · · , x n w.r.t.

Original: the squared Lorentzian distance.
Revised: the squared Lorentzian distance.

Original: The theorem can be used to compute medoids.
Revised: The theorem can be used to compute medoids.

Original: It shows that distances with a set of points can be compared with only one point which is the centroid.
Revised: It shows that distances with a set of points can be compared with only one point which is the centroid.

Revised: The main interest of this theorem in our case is that it helps understand the signiﬁcance of studying the properties of the centroid w.r.t.

Revised: the Lorentzian distance.

Revised: Interpreting distances to a set of points can be done by interpreting their centroid.

Original: Moreover, from the Cauchy-Schwarz inequality, as β tends to 0 , the Lorentzian distance in Eq.
Revised: Moreover, from the Cauchy-Schwarz inequality, as β tends to 0 , the Lorentzian distance in Eq.

Original: (9) to f 1 tends to 0 for any vector f 2 that can be written f 2 = τ f 1 with τ ≥ 0 .
Revised: (9) to f 1 tends to 0 for any vector f 2 that can be written f 2 = τ f 1 with τ ≥ 0 .

Original: The distance is greater otherwise.
Revised: The distance is greater otherwise.

Original: Therefore, the Lorentzian distance with a set of different points tends to be smaller along the ray that contains elements that can be written τ µ where τ ≥ 0 and µ is their centroid as can be seen in Fig.1 (d).
Revised: Therefore, the Lorentzian distance with a set of different points tends to be smaller along the ray that contains elements that can be written τ µ where τ ≥ 0 and µ is their centroid as can be seen in Fig.

Original: Decreasing the curvature in the Poincar´e metric formulation?
Revised: Decreasing the curvature in the Poincar´e metric formulation?

Original: As deﬁned in Eq.
Revised: As deﬁned in Eq.

Original: (5), the curvature of the hyperboloid model cannot be decreased in the Poincar´e metric.
Revised: (5), the curvature of the hyperboloid model cannot be decreased in the Poincar´e metric.

Original: Indeed, the domain of arcosh is [1 , + ∞ ) , and the values of the opposite of the Lorentzian inner product are in the interval [ β, + ∞ ) , which is why β has to be greater than or equal to 1 when using the Poincar´e metric. 
Revised: Indeed, the domain of arcosh is [1 , + ∞ ) , and the values of the opposite of the Lorentzian inner product are in the interval [ β, + ∞ ) , which is why β has to be greater than or equal to 1 when using the Poincar ´ e metric. [Figure]

Original: 3. O PTIMIZATION PROBLEM AND SOLVER
Revised: 3. O PTIMIZATION PROBLEM AND SOLVER

Original: The primary contribution of this paper is the study of the squared Lorentzian distance to represent hierarchically-structured datasets.
Revised: The primary contribution of this paper is the study of the squared Lorentzian distance to represent hierarchically-structured datasets.

Original: Even if models that exploit it do not optimize some centroid-based approach such as (Law et al., 2017; Snell et al., 2017), our analysis provides tools to interpret distances between sets of similar points.
Revised: Even if models that exploit it do not optimize some centroid-based approach such as (Law et al., 2017; Snell et al., 2017), our analysis provides tools to interpret distances between sets of similar points.

Original: We replace the Poincar ´ e metric used by current hyperbolic [Figure]
Revised: We replace the Poincar´e metric used by current hyperbolic representation learning models (Nickel &

Original: representation learning models (Nickel & Kiela, 2017; 2018) with the squared Lorentzian distance.
Revised: Kiela, 2017; 2018) with the squared Lorentzian distance.

Original: Unfortunately, none of these approaches formulate their constraints so that the common ancestor of a set of nodes is their centroid.
Revised: Unfortunately, none of these approaches formulate their constraints so that the common ancestor of a set of nodes is their centroid.

Original: Nonetheless, they try to minimize the distances between similar examples to preserve orders of distances so that ancestors are closer to their descendants than to unrelated nodes.
Revised: Nonetheless, they try to minimize the distances between similar examples to preserve orders of distances so that ancestors are closer to their descendants than to unrelated nodes.

Original: This indirectly enforces an example similar to a set of examples to be close to their centroids.
Revised: This indirectly enforces an example similar to a set of examples to be close to their centroids.

Original: We brieﬂy describe the constraints used in Nickel & Kiela (2017), though we note that the Lorentzian distance can be exploited in any metric learning problem.
Revised: We brieﬂy describe the constraints used in Nickel & Kiela (2017), though we note that the Lorentzian distance can be exploited in any metric learning problem.

Original: Constraints: Nickel & Kiela (2017) consider subsumption relations in a hierarchy ( e.g. WordNet nouns).
Revised: Constraints: Nickel & Kiela (2017) consider subsumption relations in a hierarchy ( e.g. WordNet nouns).

Original: They consider the set D =
Revised: They consider the set D =

Original: { ( u, v ) } such that each node u is an ancestor of v in the tree.
Revised: { ( u, v ) } such that each node u is an ancestor of v in the tree.

Original: For each node u , a set of negative nodes N ( u ) is created: each example in N ( u ) is not a descendant of u in the tree.
Revised: For each node u , a set of negative nodes N ( u ) is created: each example in N ( u ) is not a descendant of u in the tree.

Original: Their problem is then formulated as learning the representations { f i } ni =1 that minimize:
Revised: Their problem is then formulated as learning the representations { f i } ni =1 that minimize:

Original: [Equation]
Revised: [Equation]

Original:  where d is the chosen distance function.
Revised: (12) where d is the chosen distance function.

Original: The problem is optimized via projected gradient-based method.
Revised: The problem is optimized via projected gradient-based method.

Original: We replace d ( f u , f v ) by d 2 L ( g β ( f u ) , g β ( f v )) as deﬁned in Eq.
Revised: We replace d ( f u , f v ) by d 2 L ( g β ( f u ) , g β ( f v )) as deﬁned in Eq.

Original: Solver and numerical stability: Our formulation deﬁned in Eq.
Revised: Solver and numerical stability: Our formulation deﬁned in Eq.

Original: (9) is differentiable on the whole domain F .
Revised: (9) is differentiable on the whole domain F .

Original: Moreover it does not require reprojection on the hyperbolic domain due to the mapping g β : F → H d,β .
Revised: Moreover it does not require reprojection on the hyperbolic domain due to the mapping g β : F → H d,β .

Original: We do not need to use Riemannian stochastic gradient descent (RSGD) as in (Nickel & Kiela, 2018) and can thus use any momentum-based method that keeps track of the gradient history.
Revised: We do not need to use Riemannian stochastic gradient descent (RSGD) as in (Nickel & Kiela, 2018) and can thus use any momentum-based method that keeps track of the gradient history.

Original: In our experiments we use standard SGD optimization with momentum.
Revised: In our experiments we use standard SGD optimization with momentum.

Original:  R ESULTS
Revised:  R ESULTS

Original: We evaluate the Lorentzian distance in three different tasks.
Revised: We evaluate the Lorentzian distance in three different tasks.

Original: The ﬁrst one considers similarity constraints based on subsumption in hierarchies as described in Section 3.3.
Revised: The ﬁrst one considers similarity constraints based on subsumption in hierarchies as described in Section 3.3.

Original: The second experiment performs binary classiﬁcation to determine whether or not a test node belongs to a speciﬁc subtree of the hierarchy.
Revised: The second experiment performs binary classiﬁcation to determine whether or not a test node belongs to a speciﬁc subtree of the hierarchy.

Original: The ﬁnal experiment is qualitative in nature and presents a community detection algorithm based on hierarchical clustering with hyperbolic distances.
Revised: The ﬁnal experiment is qualitative in nature and presents a community detection algorithm based on hierarchical clustering with hyperbolic distances.

Original: 4. R EPRESENTATION OF HIERARCHIES
Revised: 4. R EPRESENTATION OF HIERARCHIES

Original: The goal of the ﬁrst experiment is to learn the hyperbolic representation of a hierarchy.
Revised: The goal of the ﬁrst experiment is to learn the hyperbolic representation of a hierarchy.

Original: To this end, we consider the same evaluation protocol as Nickel & Kiela (2018) and the same datasets.
Revised: To this end, we consider the same evaluation protocol as Nickel & Kiela (2018) and the same datasets.

Original: Each dataset can be seen as a Directed Acyclic Graph (DAG) where a parent node links to its children and descendants in the tree to create a set of edges D described in Section 3.3.
Revised: Each dataset can be seen as a Directed Acyclic Graph (DAG) where a parent node links to its children and descendants in the tree to create a set of edges D described in Section 3.3.

Original: The quality of the learned embeddings is measured with the following metrics: the Mean Rank (MR), the Mean Average
Revised: The quality of the learned embeddings is measured with the following metrics: the Mean Rank (MR), the Mean Average

Original: Precision (MAP) and the Spearman rank-order correlation p (SROC) between the Euclidean norms of the learned embeddings and the normalized ranks of nodes.
Revised: Precision (MAP) and the Spearman rank-order correlation p (SROC) between the Euclidean norms of the learned embeddings and the normalized ranks of nodes.

Original: Following (Nickel & Kiela, 2018), the normalized rank of a node u is formulated:
Revised: Following (Nickel & Kiela, 2018), the normalized rank of a node u is formulated:

Original: [Equation]
Revised: [Equation]

Original:  where lp ( u ) is the longest path between u and one of its descendant leaves; sp ( u ) is the shortest path from the root of the hierarchy to u .
Revised: (13) where lp ( u ) is the longest path between u and one of its descendant leaves; sp ( u ) is the shortest path from the root of the hierarchy to u .

Original: A leaf in the hierarchy tree then has a normalized rank equal to 1 , and nodes closer to the root have smaller normalized rank.
Revised: A leaf in the hierarchy tree then has a normalized rank equal to 1 , and nodes closer to the root have smaller normalized rank.

Original: ρ is measured as the SROC between the list of Euclidean norms of embeddings and their normalized rank. 
Revised: ρ is measured as the SROC between the list of Euclidean norms of embeddings and their normalized rank. [Table] [Table]

Original: We learn the representations { f i ∈ F d } ni =1 that minimize the following problem:
Revised: We learn the representations { f i ∈ F d } ni =1 that minimize the following problem:

Original: [Equation]
Revised: [Equation]

Original: where λ ≥ 0 is a regularization parameter.
Revised: where λ ≥ 0 is a regularization parameter.

Original: The ﬁrst term deﬁned in Eq.
Revised: The ﬁrst term deﬁned in Eq.

Original: (12) tries to get similar examples close to each other.
Revised: (12) tries to get similar examples close to each other.

Original: The second term tries to satisfy the order of the embedding norms to match the normalized ranks.
Revised: The second term tries to satisfy the order of the embedding norms to match the normalized ranks.

Original: Using Theorem 3.1, we optimize Euclidean norms in F d .
Revised: Using Theorem 3.1, we optimize Euclidean norms in F d .

Original: Datasets: We consider the following datasets: (1) 2012 ACM Computing Classiﬁcation System: is classiﬁcation system for the computing ﬁeld used by ACM journal.
Revised: Datasets: We consider the following datasets: (1) 2012 ACM Computing Classiﬁcation System: is classiﬁcation system for the computing ﬁeld used by ACM journal.

Original: (2) EuroVoc: is a thesaurus maintained by the European Union.
Revised: (2) EuroVoc: is a thesaurus maintained by the European Union.

Original: (3) Medical Subject Headings (MeSH): (Rogers, 1963) is a medical thesaurus provided by the U.S. National Library of Medicine.
Revised: (3) Medical Subject Headings (MeSH): (Rogers, 1963) is a medical thesaurus provided by the U.S. National Library of Medicine.

Original: (4) Wordnet: (Miller, 1998) is a large lexical database.
Revised: (4) Wordnet: (Miller, 1998) is a large lexical database.

Original: As in Nickel & Kiela (2018), we consider the noun and verb hierarchy of
Revised: As in Nickel & Kiela (2018), we consider the noun and verb hierarchy of

Original: WordNet.
Revised: WordNet.

Original: More details about these datasets can be found in (Nickel & Kiela, 2018).
Revised: More details about these datasets can be found in (Nickel & Kiela, 2018).

Original: Implementation details:
Revised: Implementation details:

Original: Following (Nickel & Kiela, 2017) 3 , we implemented our method in Pytorch 0.3.1.
Revised: Following (Nickel & Kiela, 2017) 3 , we implemented our method in Pytorch 0.3.1.

Original: We use the standard SGD optimizer with a learning rate of 0 . 1 and momentum of 0 . 9 .
Revised: We use the standard SGD optimizer with a learning rate of 0 . 1 and momentum of 0 . 9 .

Original: For the largest datasets Wordnet Nouns and MeSH , we stop training after 1500 epochs.
Revised: For the largest datasets Wordnet Nouns and MeSH , we stop training after 1500 epochs.

Original: We stop training at 3000 epochs for the other datasets.
Revised: We stop training at 3000 epochs for the other datasets.

Original: The mini-batch size is 50, and the number of sampled negatives per example is 50.
Revised: The mini-batch size is 50, and the number of sampled negatives per example is 50.

Original: The weights of the embeddings are initialized from the continuous uniform distribution in the interval [ − 10 − 4 , 10 − 4 ] .
Revised: The weights of the embeddings are initialized from the continuous uniform distribution in the interval [ − 10 − 4 , 10 − 4 ] .

Original: The dimensionality of our embeddings is 10.
Revised: The dimensionality of our embeddings is 10.

Original: To sample[Table] [Table] { ( u, v ) : rank ( u ) < rank ( v ) } in a mini-batch, we randomly sample κ examples from the set of positive and negative examples that are sampled for L neighborhood , we then select 5% of the possible ordered pairs.
Revised: To sample { ( u, v ) : rank ( u ) < rank ( v ) } in a mini-batch, we randomly sample κ examples from the set of positive and negative examples that are sampled for L neighborhood , we then select 5% of the possible ordered pairs.

Original: κ = 150 for all the datasets, except for WordNet nouns where κ = 50 and MeSH where κ = 100 due to their large size.
Revised: κ = 150 for all the datasets, except for WordNet nouns where κ = 50 and MeSH where κ = 100 due to their large size.

Original: Results: We compare in Table 1 the Poincar ´ e distance metric as optimized in (Nickel & Kiela, 2017; 2018) with our method for different values of β and λ (indicated in the table).
Revised: Results: We compare in Table 1 the Poincar´e distance metric as optimized in (Nickel & Kiela, 2017; 2018) with our method for different values of β and λ (indicated in the table).

Original: Here we separately analyze the case where λ = 0 ,  corresponds to using the same constraints as those reported in (Nickel & Kiela, 2018), and where λ > 0 .
Revised: Here we separately analyze the case where λ = 0 , which corresponds to using the same constraints as those reported in (Nickel & Kiela, 2018), and where λ > 0 .

Original: - Case where λ = 0 : Our approach obtains better Mean Rank and Mean Average Precision scores than Nickel & Kiela (2018) for small values of β when we use the same constraints.
Revised: - Case where λ = 0 : Our approach obtains better Mean Rank and Mean Average Precision scores than Nickel & Kiela (2018) for small values of β when we use the same constraints.

Original: The fact that the retrieval performance of our approach changes with different values of β ∈ { 0 . 01 , 0 . 1 , 1 } shows that the curvature of the space has an impact on the distances between examples and on the behavior of the model.
Revised: The fact that the retrieval performance of our approach changes with different values of β ∈ { 0 . 01 , 0 . 1 , 1 } shows that the curvature of the space has an impact on the distances between examples and on the behavior of the model.

Original: As explained in Section 3.2, the squared Lorentzian distance tends to behave more radially ( i.e. the distance tends to decrease along the ray that can be written τ µ where τ ≥ 0 and µ is the centroid) as β decreases.
Revised: As explained in Section 3.2, the squared Lorentzian distance tends to behave more radially ( i.e. the distance tends to decrease along the ray that can be written τ µ where τ ≥ 0 and µ is the centroid) as β decreases.

Original: Children then tend to have larger Euclidean norm than their parents while being close w.r.t.
Revised: Children then tend to have larger Euclidean norm than their parents while being close w.r.t.

Original: the Lorentzian distance.
Revised: the Lorentzian distance.

Original: We evaluate in Table 2 the percentage of nodes that have a Euclidean norm greater than their parent in the tree.
Revised: We evaluate in Table 2 the percentage of nodes that have a Euclidean norm greater than their parent in the tree.

Original: More than 90% of pairs (parent,child) satisfy the desired order of Euclidean norms.
Revised: More than 90% of pairs (parent,child) satisfy the desired order of Euclidean norms.

Original: The percentage increases with smaller values of β , this illustrates our point on the impact on the Euclidean norm of the center of mass.
Revised: The percentage increases with smaller values of β , this illustrates our point on the impact on the Euclidean norm of the center of mass.

Original: On the other hand, our approach obtains worse performance for the SROC metric which evaluates how the order of the Euclidean norms is correlated with their normalized rank/depth in the hierarchy tree.
Revised: On the other hand, our approach obtains worse performance for the SROC metric which evaluates how the order of the Euclidean norms is correlated with their normalized rank/depth in the hierarchy tree.

Original: This result is expected due to the formulation of the constraints of L neighborhood that considers only local similarity orders between pairs of examples.
Revised: This result is expected due to the formulation of the constraints of L neighborhood that considers only local similarity orders between pairs of examples.

Original: The loss L neighborhood does not take into account the global structure of the tree; it only considers whether pairs of concepts subsume each other or not.
Revised: The loss L neighborhood does not take into account the global structure of the tree; it only considers whether pairs of concepts subsume each other or not.

Original: [Table] As a consequence, although the root of a subtree may have a smaller Euclidean norm than all the other nodes of the subtree, this does not necessarily enforce its children to have comparable norms.
Revised: As a consequence, although the root of a subtree may have a smaller Euclidean norm than all the other nodes of the subtree, this does not necessarily enforce its children to have comparable norms.

Original: They can have Euclidean norms smaller or greater than the norms of the descendants of their siblings.
Revised: They can have Euclidean norms smaller or greater than the norms of the descendants of their siblings.

Original: In other words, although our representations do not preserve the global structure of the tree, each subtree preserves its local depth order.
Revised: In other words, although our representations do not preserve the global structure of the tree, most subtrees preserve produce a root with smaller Euclidean norm than their descendants.

Original: - Case where λ > 0 : As a consequence of the worse SROC performance, we evaluate the performance of our model when including normalized rank information during training.
Revised: - Case where λ > 0 : As a consequence of the worse SROC performance, we evaluate the performance of our model when including normalized rank information during training.

Original: As can be seen in Table 1, this greatly improves the SROC performance and outperforms the baselines (Nickel & Kiela, 2017; 2018) for all the evaluation metrics on some datasets.
Revised: As can be seen in Table 1, this greatly improves the SROC performance and outperforms the baselines (Nickel & Kiela, 2017; 2018) for all the evaluation metrics on some datasets.

Original: The Mean Rank and Mean Average Precision performances remain comparable with the case where λ = 0 .
Revised: The Mean Rank and Mean Average Precision performances remain comparable with the case where λ = 0 .

Original: Global rank information can then be used during training without having a signiﬁcant impact on retrieval performance.
Revised: Global rank information can then be used during training without having a signiﬁcant impact on retrieval performance.

Original: In conclusion, we have shown that the curvature of the hyperbolic space has an impact on the retrieval performance of the model.
Revised: In conclusion, we have shown that the curvature of the hyperbolic space has an impact on the retrieval performance of the model.

Original: Moreover, the fact the Euclidean norms of parents tend to be smaller than those of children as β > 0 decreases experimentally demonstrates that the most similar points ( i.e. high level nodes) tend to get closer to the origin as their set of similar points ( i.e. number of descendant nodes) increases.
Revised: Moreover, the fact the Euclidean norms of parents tend to be smaller than those of children as β > 0 decreases experimentally demonstrates that the most similar points ( i.e. high level nodes) tend to get closer to the origin as their set of similar points ( i.e. number of descendant nodes) increases.

Original: 4. B INARY CLASSIFICATION
Revised: 4. B INARY CLASSIFICATION

Original: Another task of interest for hyperbolic representations is to determine whether a given node belongs to a speciﬁc subtree of the hierarchy or not.
Revised: Another task of interest for hyperbolic representations is to determine whether a given node belongs to a speciﬁc subtree of the hierarchy or not.

Original: We follow the same binary classiﬁcation protocol as
Revised: We follow the same binary classiﬁcation protocol as

Original: Ganea et al.
Revised: Ganea et al.

Original: (2018) on the same datasets.
Revised: (2018) on the same datasets.

Original: We describe their protocol below.
Revised: We describe their protocol below.

Original: Ganea et al.
Revised: Ganea et al.

Original: (2018) extract some pre-trained hyperbolic embeddings of the WordNet nouns hierarchy, those representations are learned with the Poincar ´ e metric.
Revised: (2018) extract some pre-trained hyperbolic embeddings of the WordNet nouns hierarchy, those representations are learned with the Poincar´e metric.

Original: They then consider four subtrees whose roots are the following synsets: animal.n.01, group.n.01, worker.n.01 and mammal.n.01.
Revised: They then consider four subtrees whose roots are the following synsets: animal.n.01, group.n.01, worker.n.01 and mammal.n.01.

Original: For each subtree, they consider that every node that belongs to it is positive and all the other nodes of Wordnet nouns are negative.
Revised: For each subtree, they consider that every node that belongs to it is positive and all the other nodes of Wordnet nouns are negative.

Original: They then select 80% of the positive nodes for training, the rest for test.
Revised: They then select 80% of the positive nodes for training, the rest for test.

Original: They select the same percentage of negative nodes for training and test.
Revised: They select the same percentage of negative nodes for training and test.

Original: At test time, they evaluate the F score of the binary classiﬁcation performance.
Revised: At test time, they evaluate the F score of the binary classiﬁcation performance.

Original: They do it for 3 different training/test splits.
Revised: They do it for 3 different training/test splits.

Original: We refer to Ganea et al.
Revised: We refer to Ganea et al.

Original: (2018) for details on the baselines.
Revised: (2018) for details on the baselines.

Original: Our goal is to evaluate the relevance of the Lorentzian distance to represent hierarchical datasets.
Revised: Our goal is to evaluate the relevance of the Lorentzian distance to represent hierarchical datasets.

Original: We then use the embeddings trained with our approach  and classify a test node by assigning it to the category of the nearest  example w.r.t.
Revised: We then use the embeddings trained with our approach (with β = 0 . 01 and different values of λ ) and classify a test node by assigning it to the category of the nearest training example w.r.t.

Original: the Lorentziandistance.
Revised: the Lorentzian

Original: The test performance of our approach is reported in Table 3. It outperforms the classiﬁcation performance of the baselines which shows that our embeddings can also be used to perform classiﬁcation.
Revised: [Table]

Original: 4. C OMMUNITY DETECTION
Revised: 4. C OMMUNITY DETECTION

Original: In our last experiment, our dataset does not correspond to a hierarchy tree.
Revised: In our last experiment [Table] our dataset does not correspond to a hierarchy tree.

Original: It is an undirected graph
Revised: It is an undirected graph

Original: G =
Revised: G =

Original: ( V , D ) where V is the set of vertices/nodes, and D the set of edges.
Revised: ( V , D ) where V is the set of vertices/nodes [Table] and D the set of edges.

Original: We use the co-authorship information of the NIPS 1-17 dataset (Globerson et al., 2007) which contains information from papers published at NIPS from 1988 to 2003.
Revised: We use the co-authorship information of the NIPS 1-17 dataset (Globerson et al., 2007) which contains information from papers published at NIPS from 1988 to 2003.

Original: The graph G is constructed by considering each author as a node, and an edge ( u, v )
Revised: The graph G is constructed by considering each author as a node, and an edge ( u, v )

Original: ∈ D is created iff the authors u ∈ V and v ∈ D are co-authors.
Revised: ∈ D is created iff the authors u ∈ V and v ∈ D are co-authors.

Original: The number of authors is|V| = 2865 and the number of edges is |D| = 9466 .
Revised: The number of authors is 2865 and the number of edges is |D| = 9466 .

Original: Only  2715 authors have at least one co-author and the edges are 
Revised: Only |V| = 2715 authors have at least one co-author and the edges are neither weighted by the number of authors per paper nor by the number of times a pair of authors are co-authors.

Original: The main difference with the hierarchical case is that the transitivity relation ( u, v ) ∈ D , ( v, w ) ∈
Revised: The main difference with the hierarchical case is that the transitivity relation ( u, v ) ∈ D , ( v, w ) ∈

Original: D ⇒ ( u, w ) ∈
Revised: D ⇒ ( u, w ) ∈

Original: D is not necessarily true in the undirected graph case.
Revised: D is not necessarily true in the undirected graph case.

Original: Figures 2 and 3 illustrate the 2-dimensional Poincar´e representations learned with Eq.
Revised: Figures 2 and 4 illustrate the 2-dimensional Poincar´e representations learned with Eq.

Original: (14) for different values of λ ≥ 0 .
Revised: (14) for different values of λ ≥ 0 .

Original: The number of co-authors is written in parenthesis, authors that have at least 20 co-authors are in red, those between 10 and 19 co-authors in blue.
Revised: The number of co-authors is written in parenthesis, authors that have at least 20 co-authors are in red, those between 10 and 19 co-authors in blue.

Original: The MR and MAP are similar between both approaches (the MR is about 20 and the MAP about 55% ) but the SROC with the number of co-authors is 93% for the second ﬁgure.
Revised: The MR and MAP are similar between both approaches (the MR is about 20 and the MAP about 55% ) but the SROC with the number of co-authors is 93% for Fig.

Original: One can observe some communities such as the kernel method researchers (Shawe-Taylor, Vapnik, Smola etc...) are close to the same radius. 
Revised: One can observe some communities such as the kernel method researchers (Shawe-Taylor, Vapnik, Smola etc...) are close to the same radius. [Table] [Table] [Table]

Revised: [Table]

Original: When learning 10-dimensional representations, the MR is 1.0 and the MAP is 100% .
Revised: When learning 10-dimensional representations, the MR is 1.0 and the MAP is 100% .

Original: When λ = 0 , the embeddings with smallest Euclidean norm are in that order: Sejnowski T. (52), Jordan M. (45),
Revised: When λ = 0 , the embeddings with smallest Euclidean norm are in that order: Sejnowski T. (52), Jordan M. (45),

Original: Tishby N. (29), Dayan P. (25), Koch C. (46), Singh S. (27), Hinton G. (32), Scholkopf B. (36), Mozer
Revised: Tishby N. (29), Dayan P. (25), Koch C. (46), Singh S. (27), Hinton G. (32), Scholkopf B. (36), Mozer

Original: [Table]

Original: M. (21), Bialek W. (27), Singer Y. (19), Vapnik V. (23), Bengio Y. (33), Shawe-Taylor J. (22), Zemel
Revised: M. (21), Bialek W. (27), Singer Y. (19), Vapnik V. (23), Bengio Y. (33), Shawe-Taylor J. (22), Zemel

Original: R. (13).
Revised: R. (13).

Original: The authors with a large number of co-authors then tend to have smallest Euclidean norms.
Revised: The authors with a large number of co-authors then tend to have smallest Euclidean norms.

Original: Since hyperbolic distances are appropriate to represent hierarchies, we perform a hierarchical agglomerative clustering with the learned squared Lorentzian distance (with dimensionality d = 10 ) based on complete-linkage clustering (Defays, 1977) ( i.e. linkage uses the maximum distances between all observations of two sets to merge them into a same cluster).
Revised: Since hyperbolic distances are appropriate to represent hierarchies, we perform a hierarchical agglomerative clustering with the learned squared Lorentzian distance (with dimensionality d = 10 ) based on complete-linkage clustering (Defays, 1977) ( i.e. linkage uses the maximum distances between all observations of two sets to merge them into a same cluster).

Original: Some examples of extracted clusters are given in Table 4.
Revised: Some examples of extracted clusters are given in Table 4.

Revised: Fig 5 illustrates how the researchers from the kernel method clusters of Table 4 are represented in Fig 2.

Revised: Most of them are close to the same radius of the Poincar ´ e ball.

Original: This simple example shows that the Lorentzian distance can be applied to datasets that are not only hierarchical.
Revised: This simple example shows that the Lorentzian distance can be applied to datasets that are not only hierarchical.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we proposed a distance learning approach based on the Lorentzian distance to represent hierarchically-structured datasets.
Revised: In this paper we proposed a distance learning approach based on the Lorentzian distance to represent hierarchically-structured datasets.

Original: Unlike most of the literature that considers the unit hyperboloid model, we show that the performance of the learned model can be improved when the chosen hyperboloid model has low curvature.
Revised: Unlike most of the literature that considers the unit hyperboloid model, we show that the performance of the learned model can be improved when the chosen hyperboloid model has low curvature.

Original: We give a formulation of the centroid w.r.t. the squared
Revised: We give a formulation of the centroid w r t. the squared

Original: Lorentzian distance as a function of the curvature, and we show that the Euclidean norm of its projection in the Poincar ´ e ball decreases as the curvature decreases.
Revised: Lorentzian distance as a function of the curvature, and we show that the Euclidean norm of its projection in the Poincar´e ball decreases as the curvature decreases.

Original: Hierarchy constraints are generally formulated such that high-level nodes are similar to all their descendants and thus their representation should be close to the centroid of the descendants.
Revised: Hierarchy constraints are generally formulated such that high-level nodes are similar to all their descendants and thus their representation should be close to the centroid of the descendants.

Original: Using low curvature implicitly enforces high-level nodes to have smaller Euclidean norm than their descendants and is therefore is more appropriate for learning representations of hierachically-structured datasets
Revised: Using low curvature implicitly enforces high-level nodes to have smaller Euclidean norm than their descendants and is therefore is more appropriate for learning representations of hierachically-structured datasets.

Original: A P ROOFS A. P ROOF OF T HEOREM 3.1:
Revised: A P ROOFS A. P ROOF OF T HEOREM 3.1:

Original: • We ﬁrst show ( ⇒ ) : let us note α = (cid:107) a (cid:107) 2 , we have the following equality (cid:107) h ( g β ( a ))
Revised: • We ﬁrst show ( ⇒ ) : let us note α = (cid:107) a (cid:107) 2 , we have the following equality (cid:107) h ( g β ( a ))

Original: (cid:107) 

Revised: (cid:107)

Original: =
Revised: =

Original: α ( √ α +1+1) 2 which is an increasing function in α when α is positive.
Revised: α ( √ α +1+1) 2 which is an increasing function in α when α is positive.

Original: • We now show ( ⇐ ) :
Revised: • We now show ( ⇐ ) :

Original: We give the formulation of g − 1 β ◦ h − 1 :
Revised: We give the formulation of g − 1 β ◦ h − 1 :

Original: ∀
Revised: ∀

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: d )
Revised: d )

Original: [Equation]
Revised: [Equation]

Original: P
Revised: P

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The vector g − 1 β ( h − 1 ( c )) is obtained by removing the ﬁrst element of h − 1 ( c ) :
Revised: The vector g − 1 β ( h − 1 ( c )) is obtained by removing the ﬁrst element of h − 1 ( c ) :

Original: [Equation]
Revised: [Equation]

Original: By using these deﬁnitions, we have the following implication: we recall that ∀ c ∈ P d , (cid:107) c (cid:107) < 1 due to the deﬁnition of P d .
Revised: By using these deﬁnitions, we have the following implication: we recall that ∀ c ∈ P d , (cid:107) c (cid:107) < 1 due to the deﬁnition of P d .

Original: Let us note γ = (cid:107) c (cid:107) 2 < 1 , we have (cid:107) g − 1 β
Revised: Let us note γ = (cid:107) c (cid:107) 2 < 1 , we have (cid:107) g − 1 β

Original: ( h − 1 ( c ))
Revised: ( h − 1 ( c ))

Original: (cid:107) 2 = 4 γ
Revised: (cid:107) 2 = 4 γ

Original: 1 − γ ) 2 which is an increasing function in γ on (0 , 1) .
Revised: 1 − γ ) 2 which is an increasing function in γ on (0 , 1) .

Original: QED
Revised: QED

Original: A. P ROOF OF L EMMA 3. 
Revised: A. P ROOF OF L EMMA 3.

Original: By using the linearity of the Lorentzian inner product, we have the following equality:
Revised: By using the linearity of the Lorentzian inner product, we have the following equality:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Since µ ∈ H d,β and ∀ a ∈ H d,β , b ∈ H d,β , (cid:104) a , b (cid:105) L is maximized if a = b , Eq.
Revised: Since µ ∈ H d,β and ∀ a ∈ H d,β , b ∈ H d,β , (cid:104) a , b (cid:105) L is maximized if a = b , Eq.

Original: (18) is then maximized for the scaling factor γ > 0 that satisﬁes ( γ (cid:80) ni
Revised: (18) is then maximized for the scaling factor γ > 0 that satisﬁes ( γ (cid:80) ni

Original: =

Revised: =

Original: ν i x i ) =
Revised: ν i x i ) =

Original: µ ∈ H d,β .
Revised: µ ∈ H d,β .

Original: We explain in the next paragraph how to construct such a γ .
Revised: We explain in the next paragraph how to construct such a γ .

Original: For any positive time-like vector a , the vector b =
Revised: For any positive time-like vector a , the vector b =

Original: [Equation]
Revised: [Equation]

Original: = [Equation]
Revised: = − 1 .

Original: Therefore, β (cid:104) b , b (cid:105) L =
Revised: Therefore, β (cid:104) b , b (cid:105) L =

Original: (cid:104)√ β b , √ β b (cid:105) L = − β ( i.e. √ β b ∈ H d,β by deﬁnition since √ β b is positive) and we then have γ =
Revised: (cid:104)√ β b , √ β b (cid:105) L = − β ( i.e. √ β b ∈ H d,β by deﬁnition since √ β b is positive) and we then have γ =

Original: [Equation]
Revised: [Equation]

Original: A. P ROOFTHAT THE NORM OF THE CENTROID INCREASES AS THE CURVATURE INCREASES 
Revised: A. P ROOF OF T HEOREM 3.

Original: Let f 1 , · · · , f n be a set of at least two different points in F d , and µ be the centroid of the set g β ( f 1 ) , · · · , g β ( f n ) (with same weighting ∀ i, ν i = 1 ).
Revised: Let f 1 , · · · , f n be a set of at least two different points in F d , and µ be the centroid of the set g β ( f 1 ) , · · · , g β ( f n ) (with same weighting ∀ i, ν i = 1 ).

Original: The squared Euclidean norm of g − 1 β ( µ ) ∈ F d is:
Revised: The squared Euclidean norm of g − 1 β ( µ ) ∈ F d is:

Original: [Equation]
Revised: [Equation]

Original: Since (cid:107) (cid:80) ni
Revised: Since (cid:107) (cid:80) ni

Original: =

Revised: =

Original: f i (cid:107) 2 does not depend on β and the factor
Revised: f i (cid:107) 2 does not depend on β and the factor

Original: [Equation]
Revised: [Equation]

Original: is positive for β > 0 , we only need to show that it is a increasing function in β , or equivalently, that its reciprocal is decreasing in β .
Revised: is positive for β > 0 , we only need to show that it is a increasing function in β , or equivalently, that its reciprocal is decreasing in β .

Original: The reciprocal of
Revised: The reciprocal of

Original: [Equation]
Revised: [Equation]

Original: We then need to show that the function1 β
Revised: We then need to show that the function

Original: [Equation]
Revised: [Equation]

Original: f i

Original: [Equation]

Original: + β

Original: [Equation]

Original: 2 + β [Equation] − (cid:104) f i , f j (cid:105) is decreasing in β if f i (cid:54) =
Revised: is decreasing in β if f i (cid:54) =

Original: f j .
Revised: f j .

Original:  Wethen study the sign of its gradient w.r.t. β .
Revised: For this purpose, We study the sign of its gradient.

Original: The gradient is written:
Revised: The gradient of Eq. w.r

Original: [Equation]
Revised: [Equation]

Revised: t. β is written:

Original: [Equation]
Revised: [Equation]

Original: The denominator is positive.
Revised: The denominator is positive.

Original: If (cid:104) f i , f j (cid:105) is negative, then Eq.
Revised: If (cid:104) f i , f j (cid:105) is negative, then Eq.

Original: (23) is negative.
Revised: (24) is negative.

Original: If (cid:104) f i , f j (cid:105) = 0 , then
Revised: If (cid:104) f i , f j (cid:105) = 0 , then

Original: Eq.
Revised: Eq.

Original: (23) is 0 if f i =
Revised: (24) is 0 if f i =

Original: f j = 0 , and negative otherwise.
Revised: f j = 0 , and negative otherwise.

Original: Otherwise, the Cauchy-Schwarz inequality is used to prove that Eq.
Revised: Otherwise, the Cauchy-Schwarz inequality is used to prove that Eq.

Original: (23) is negative.
Revised: (24) is negative.

Original: In other words, we need to prove that (assuming that (cid:104) f i , f j (cid:105) is positive):[Equation] [Equation] ⇐⇒
Revised: In other words, we need to prove that (assuming that (cid:104) f i , f j (cid:105) is positive):

Original: [Equation]
Revised: [Equation]

Original: 25) ⇐⇒(2 (cid:104) f
Revised: 26) ⇐⇒

Original: [Equation]
Revised: [Equation]

Original: f j [Equation] f [Equation]
Revised: ⇐⇒

Original: [Equation]
Revised: [Equation]

Original: Since

Original: [Equation]
Revised: [Equation]

Original:  by the Cauchy-Schwarz inequality
Revised: Since, by the Cauchy-Schwarz inequality

Original: [Equation]
Revised: [Equation]

Original: we have:
Revised: we have:

Original: [Equation]
Revised: [Equation]

Original: 29) we then have (term by term):[Equation] [Equation]
Revised: [Equation] we then have (term by term):

Original: [Equation]
Revised: [Equation]

Original: Eq.
Revised: Eq.

Original: (34) is explained by β 2 (cid:107)
Revised: (35) is explained by β 2 (cid:107)

Original: f i (cid:107) 4 + β
Revised: f i (cid:107) [Equation] β

Original: [Equation]
Revised: [Equation]

Original: f j (cid:107)
Revised: f j [Equation]

Original: [Equation]
Revised: [Equation]

Revised: β

Original: [Equation]
Revised: [Equation]

Revised: f i

Original: [Equation]
Revised: [Equation]

Original: f i
Revised: f j

Original: [Equation]
Revised: [Equation]

Revised: 2 = β

Original: [Equation]
Revised: [Equation]

Original: fj (cid:107) 2 = β 2 ( (cid:107) f i (cid:107) 2 − (cid:107) f j (cid:107) 2 ) 2 ≥ 0 .
Revised: f i [Equation] 2 − (cid:107) f j (cid:107) 2 ) 2 ≥ [Equation]

Original: Eq.
Revised: Eq.

Original: (34) is then an equality iff (cid:107)
Revised: (35) is then an equality iff (cid:107)

Original: f i (cid:107) 

Revised: f i (cid:107)

Original: = (cid:107) f j (cid:107) 2 , and the Cauchy-Schwarz relation is an equality iff f
Revised: = (cid:107) f j (cid:107) 2 , and the Cauchy-Schwarz relation is an equality iff f

Original: i are linearly f j dependent.
Revised: i are linearly f j dependent.

Original: Both of these conditions are satisﬁed iff f i =
Revised: Both of these conditions are satisﬁed iff f i =

Original: f j (assuming that (cid:104) f i , f j (cid:105) is positive).
Revised: f j (assuming that (cid:104) f i , f j (cid:105) is positive).

Original: However, we assume that there are at least two different points
Revised: However, we assume that there are at least two different points

Original: f i and f j such that f i (cid:54) =
Revised: f i and f j such that f i (cid:54) =

Original: f j , which then implies that Eq.
Revised: f j , which then implies that Eq.

Original: (23) is negative for this choice of f i and f j .
Revised: (24) is negative for this choice of f i and f j .

Original: This proves that the Euclidean norm in F d of the centroid of different points decreases as β >decreases.
Revised: This proves that the Euclidean norm in F d of the centroid of different points decreases as β >decreases.

Original: From Section A.1, the Euclidean norm of a point in P d increases as the Euclidean norm of its projection in F d increases, which completes the proof.
Revised: From Section A.1, the Euclidean norm of a point in P d increases as the Euclidean norm of its projection in F d increases, which completes the proof.

Revised: [Figure]