Hyperbolic Busemann Learning with Ideal Prototypes
Year: 2021

Abstract:
Hyperbolic space has become a popular choice of manifold for representation learning of various datatypes from tree-like structures and text to graphs. Building on the success of deep learning with prototypes in Euclidean and hyperspherical spaces, a few recent works have proposed hyperbolic prototypes for classification. Such approaches enable effective learning in low-dimensional output spaces and can exploit hierarchical relations amongst classes, but require privileged information about class labels to position the hyperbolic prototypes. In this work, we propose Hyperbolic Busemann Learning. The main idea behind our approach is to position prototypes on the ideal boundary of the Poincaré ball, which does not require prior label knowledge. To be able to compute proximities to ideal prototypes, we introduce the penalised Busemann loss. We provide theory supporting the use of ideal prototypes and the proposed loss by proving its equivalence to logistic regression in the one-dimensional case. Empirically, we show that our approach provides a natural interpretation of classification confidence, while outperforming recent hyperspherical and hyperbolic prototype approaches.

heading: 1 Introduction
text: Classification by prototypes has a long tradition in machine learning. Foundational solutions such as the Nearest Mean Classifier [46] represent classes as the mean prototypes in a fixed feature space. Following approaches that learning a metric space for class prototypes [28], deep learning with prototypes as points in network output spaces has gained traction [14, 17, 26, 32, 36, 41, 49, 51]. In line with foundational approaches, the prototypes are positioned by computing the mean over all training examples for each class. Deep learning by mean prototypes has shown to be effective for tasks such as few-shot learning [14, 32, 36, 41] and zero-shot recognition [41, 49, 51].
To avoid the need to re-learn prototypes or to enable the use of prior label knowledge, several recent works have proposed deep networks with prototypes in non-Euclidean output spaces. Hyperspherical prototype approaches initialize prototypes on the sphere or its higher-dimensional generalization, based on pair-wise angular separation [29, 39] or with the help of word embeddings [4, 43] using a cosine similarity loss between example outputs and class prototypes. Hyperspherical prototypes alleviate the need to re-position prototypes continuously and allow to optionally include prior semantic knowledge about the classes. Recently, a few works have extended prototype-based learning to the hyperbolic domain, where prototypes are obtained by embedding label hierarchies [23, 24]. While hyperbolic prototype approaches provide more hierarchically coherent classification results, prior hierarchical knowledge is required to embed prototypes. This paper strives to combine the best of both non-Euclidean worlds, namely efficient low-dimensional embeddings from hyperbolic prototypes and the knowledge-free positioning from hyperspherical prototypes.
We make three contributions in this work. First, we introduce a hyperbolic prototype network with class prototypes given as points on the ideal boundary of the Poincaré ball model of hyperbolic geometry. Second, we propose the penalized Busemann loss, which enables us to compute proximities between example outputs in hyperbolic space and prototypes at the ideal boundary, an impossible task for existing distance metrics, which put the ideal boundary at infinite distance from all other
35th Conference on Neural Information Processing Systems (NeurIPS 2021)
points in hyperbolic space. Third, we provide a theoretical link between our hyperbolic prototype approach and logistic regression. We show that our choices for output manifold, for prototypes on the ideal boundary, and for the proposed loss provide a direct generalization of the logistic regression model. Experiments on three datasets show that our approach outperforms both hyperspherical and hyperbolic prototype approaches. Moreover, our hyperbolic output space provides a natural interpretation of closeness to the ideal boundary as classification confidence, see Figure 1. The code is available at https://github.com/MinaGhadimiAtigh/Hyperbolic-Busemann-Learning.

heading: 2 Related work
text: As an early adaptation of prototypes in deep networks, the center loss [47] served as an additional loss for softmax cross-entropy optimization, where all outputs of a class were pulled to one point to help minimize intra-class variation. Closer to the original Nearest Mean Classifier objective, Prototypical Networks introduce a prototype-only approach where each class is represented as the mean output of its corresponding training examples [41]. Learning with prototypes from class means has shown to be effective for few-shot learning [14, 32, 36, 41], zero-shot recognition [41, 49, 51], and domain adaptation [33]. Defining prototypes as class means is feasible in few-shot learning by sampling all samples of a class in a mini-batch. For general supervised learning, DeepNCM [17] and DeepNNO [26] propose alternating optimization schemes, where the prototypes are fixed to update the network backbone and vice versa. To avoid the need for constant prototype updating, several works have proposed to use the hypersphere as output space instead of the Euclidean space. On the hypersphere, prototypes can be positioned a priori based on angular separation [29, 39], using an orthonormal basis [4], or using prior knowledge from word embeddings [4, 29, 43]. Similar to hyperspherical prototype approaches, we seek to position prototypes at unit distance from the origin. In contrast, we operate on a hyperbolic manifold, which provides a theoretical link to logistic regression and obtains better empirical results than a hypershperical manifold and loss.
In this work, we strive for a prototype-based network on hyperbolic manifolds, which have become a popular choice for embedding data and deep network layers. Hyperbolic embeddings have shown to represent tree-like structures [38, 37], text [1, 44, 55], cells [20], networks [18], multi-relational knowledge graphs [3], and general taxonomies [16, 21, 30, 31, 50] with minimal distortion. In deep networks, hyperbolic alternatives have been proposed for fully-connected layers [15, 40], convolutional layers [40], recurrent layers [15], classification layers [13, 15, 40], and optimizers [5, 6, 52]. Hyperbolic generalizations have also been introduced for graph networks [2, 12, 22, 25, 53, 54] and generative models [7, 27]. Where hyperbolic-based deep learning approaches commonly operate within the Poincaré model represented by the open unit ball, we propose to position class prototypes
on its ideal boundary – the shell of the unit ball – allowing us to benefit from the representational power of hyperbolic geometry without requiring prior knowledge to operate.
A few works have recently proposed prototype-based approaches with hyperbolic output spaces. Hyperbolic Image Embeddings by Khrulkov et al. [19] include a prototype-based setting as a direct hyperbolic generalization of Prototypical Networks [41], allowing for few-shot learning through hyperbolic averaging. Beyond few-shot learning, Long et al. [24] introduce a hyperbolic action network, where action prototypes are projected to a hyperbole based on prior knowledge about hierarchical relations. Videos are in turn projected to the same space and compared to the prototypes using the hyperbolic distance, enabling hierarchically consistent action recognition and zero-shot recognition. Similarly, Liu et al. [23] perform zero-shot recognition using hyperbolic prototypes positioned based on known hierarchical relations. In this work, we also employ prototypes in hyperbolic space, but do not require prior knowledge about hierarchies. Recent approaches with hyperbolic output spaces have shown to be effective in low dimensionalities [19, 24, 23]. Low dimensional embeddings have several practical use cases and benefits. We list two important use cases. Compactness: Hyperbolic Busemann Learning enables us to embed high-dimensional inputs in low-dimensional output spaces. This ability has potential benefits in applications such as data transmission, compression, and storage. Interpretability: Obtaining high performance with few output dimensions (i.e., two or three output dimensions) provides the potential to interpret the model performance and visualize the output space, see Figure 1.

heading: 3 Learning with hyperbolic ideal prototypes

heading: 3.1 Problem formulation
text: We are given a training set {(xi, yi)}Ni=1 with N examples, where xi ∈ RI denotes an input with dimensionality I and yi ∈ {1, .., C} denotes a label from a set of sizeC. We seek to learn a projection of inputs to a hyperbolic manifold in which can compute proximities to class prototypes. We do so through a transformation F in Euclidean space, followed by an exponential map from the tangent space TxM to a hyperbolic manifoldM:
z = expv(F(x; θ)), (1)
where θ denotes the parameters to learn the transformation. The transformation can be any function, e.g., a deep network or a linear layer akin to logistic regression. The hyperbolic manifold in which z operates is described throughout this work by the Poincaré ball model (Bd, gBz ), with open ball Bd = {z ∈ Rd | ||z||2 < 1} and the Riemannian metric tensor gBz = 4(1 − ||z||2)−2Id [11, 34]. Setting its center to v = 0 the exponential map in the Poincaré model is given by:
exp0(x) = tanh(||x||/2) x
||x|| . (2)
Furthermore, the geodesic distance between two points z1, z2 ∈ Bd is given by:
dB(z1, z2) = arcosh
( 1 + 2
||z1 − z2||2
(1− ||z1||2)(1− ||z2||2)
) . (3)
To train our model, we compare the projected inputs to their class labels, represented by the prototypes P = {p1, ..,pC}. That is, the overall training loss is given as the aggregation of distance-based individual losses ` between projected inputs and the prototype of their respective class labels:
L(θ) = 1 N N∑ i=1 `(exp0(F(x; θ)),pyi). (4)

heading: 3.2 Ideal prototypes and the penalized Busemann loss
text: The central idea of our approach is to position class prototypes at ideal points, which represent points at infinity in hyperbolic geometry. In the Poincaré model, the ideal points form the boundary of the ball:
Id = {z ∈ Rd : z21 + · · ·+ z2d = 1}. (5)
Thus, the set of ideal points of hyperbolic space Bd is homeomorphic to the hypersphere Sd. As a consequence, any of the methods used for prototype embedding into Sd can be used to embed prototypes into Id. In particular, for d = 2, prototypes can be placed uniformly on the unit sphere S2, while class-agnostic prototype embedding based on separation can be used for d ≥ 3 [29]. A key issue that needs to be solved, is that ideal points are at infinite geodesic distance from all other points in Bd and hence the geodesic distance can not be used directly for prototype-based learning. Therefore, we propose the penalized Busemann loss for hyperbolic learning with ideal prototypes. The Busemann function, originally introduced in [9] (see also Def. II.8.17 in [8]), can be considered a distance to infinity and may be defined in any metric space. Let p be an ideal point and γp a geodesic ray, parametrized by arc length, tending to p. Then the Busemann function with respect to p is defined for z ∈ Bd as:
bp(z) = lim t→∞
(dB(γp(t), z)− t). (6)
In the Poincaré model, the limit can be explicitly calculated and the Busemann function is given as:
bp(z) = log ||p− z||2
(1− ||z||2) . (7)
The full derivation of (7) is provided in Appendix A. Our proposed loss combines the Busemann function with a penalty term:
`(z,p) = bp(z)− φ(d) · log(1− ||z||2), (8)
with φ(d) a scaling factor for the penalty term which is a function of the dimension of the hyperbolic space. The first term in the penalized Busemann loss steers a projected input z towards class prototype p, while the second term penalized overconfidence, i.e., values close to the ideal boundary of Bd. The prototypes are positioned before training and remain fixed, hence the backpropagation only needs to be done with respect to the inputs. The gradient of `(exp0(x),p) with respect to x is given as:
∇x `(exp0(x),p) = (x−p) tanh(||x||)
||x|| − tanh(||x||)p · x +1d p·x tanh(||x||)/||x|| − 1 ||x|| − tanh(||x||)p · x +tanh(||x||/2),
(9) with 1d a d-dimensional vector of ones. Figure 2 shows a color map of the penalized Busemann loss itself and its radial gradient for a prototype located at ideal point p = (1, 0).
In the penalized Busemann loss, φ(d) governs the amount of regularization. Below, we show that this function should be linear in the number of dimensions.
Theorem 1. For any d ≥ 4 and ideal point p the penalized Busemann loss `(z,p) is the log-likelihood of a density f(z,p) = 1C(d) exp (−`(z,p)) on the Poincaré ball Bd if and only if φ(d) > d− 2.
Proof. To show that f(z,p) is a density, we have to show that the normalization constant
C(d) = ∫ Bd exp (−`(z,p)) dV
is finite. Here, dV is the hyperbolic volume element on Bd, which is equal to 2d(1 − ||z||2)−ddz1 · · · dzd, see [34]. Furthermore, using invariance of the integrand under rotation, we can assume without loss of generality that p is equal to the unit vector e1 = (1, 0, . . . , 0), which leads to
C(d) = 2d ∫ Bd ( (1− ||z||2)φ(d)+1−d 1− 2z1 + ||z||2 ) dz1 · · · dzd.
Switching to (Euclidean) hyperspherical coordinates (r, ϕ1, . . . , ϕd), the integral factorizes into C(d) = 2d · C1(d) · C2(d), where
C1(d) = ∫ 1 0 ∫ π 0 ( (1− r2)φ(d)+1−d 1− 2r cos(ϕ1) + r2 ) rd−1 sin(ϕ1) d−2dr dϕ1
C2(d) = ∫ π 0 sin(ϕ2) d−3dϕ2 . . . ∫ π 0 sin(ϕd−2) 2dϕd−2 · ∫ 2π 0 sin(ϕd−1)dϕd−1.
Clearly, C2(d) is finite, independently of d, such that it remains to consider C1(d). Instead of solving the integral exactly, we use the elementary estimate
sin(ϕ1) 2 ≤ 1− 2r cos(ϕ1) + r2 ≤ 4,
valid for all r ∈ [0, 1]. This estimate allows us to obtain both a lower and an upper bound on C1(d). The upper bound becomes
C1(d) ≤ ∫ 1 0 (1− r2)φ(d)+1−d rd−1dr ∫ π 0 sin(ϕ1) d−4dϕ1,
which (given that d ≥ 4) is finite if and only if φ(d) > d− 2. The lower bound becomes
C1(d) ≥ 1
4 ∫ 1 0 (1− r2)φ(d)+1−d rd−1dr ∫ π 0 sin(ϕ1) d−2dϕ1,
which (given that d ≥ 2) is also finite if and only if φ(d) > d− 2, which completes the proof.
Given a trained model, inference is performed by projecting the test example x and selecting the class with the smallest prototype distance:
y? = argmin p∈P `(exp0(F(x; θ)),p). (10)
The Poincaré ball model of hyperbolic geometry has the conformal property, i.e., angles in the Poincaré ball model coincide with angles in Euclidean space [35]. Thus, the cosine similarity between two ideal points remains a meaningful measure of their similarity, even in hyperbolic geometry. As the penalty term is the same for all prototypes, we can equivalently minimize the Busemann function of the prototypes directly. Moreover, as the Busemann function is a decreasing function of the cosine similarity, prediction is equivalent to hyperspherical prototype inference:
y? = argmax p∈P
z
||z|| · p, z = exp0(F(x; θ)). (11)

heading: 3.3 Relation to logistic regression
text: Logistic regression is a foundational algorithm for classification and a pillar in supervised learning for deep networks with cross entropy optimization. Here, we show that hyperbolic ideal prototype learning is a natural generalization of logistic regression by showing its equivalence when using d = 1 dimensions in our approach. Theorem 2. For dimension d = 1 and with penalty φ(1) = 1, Hyperbolic Busemann Learning with a single linear layer is equivalent to logistic regression.
Proof. Recall that from the machine learning point of view logistic regression can be described as a single linear layer in combination with the logistic activation function and cross-entropy-loss. In one dimension, hyperbolic space B1 becomes the interval (−1, 1). The set of ideal points I1 consists of the two points ±1. To identify these prototypes with 0/1 (the class labels in logistic regression) we introduce a linear change of coordinates to
z′ = z + 1
2 .
Under this change of coordinates B1 becomes the unit interval (0, 1) and the prototypes p′ (the ideal points) become the endpoints 0 and 1. The exponential map, i.e., the embedding of the base learner output y = w>x+ w0 into B1, maps y ∈ R to
z′ = exp0(y) = tanh(y/2) + 1
2 =
1
1 + e−y ,
which is the logistic function. In z′-coordinates, the penalized Busemann loss becomes
`(z′, p′) = 2 log ( |p′ − z′| 2z′(1− z′) ) = 2 log ( 1 2(1−z′) ) if p′ = 0 log (
1 2z′
) if p′ = 1 .
Rewriting this as 1
2 `(z′, p′) + log(2) = −p′ log(z′)− (1− p′) log(1− z′),
shows that the shifted and scaled penalized Busemann loss coincides with the cross-entropy loss in dimension d = 1.
Beyond binary classification, multinomial logistic regression is performed in a one-vs-rest manner with a K-dimensional softmax output for K classes. i.e., in conventional multinomial logistic regression the output space scales one-to-one with the number of classes. With Hyperbolic Busemann Learning, we provide a different generalization of logistic regression, where the number of output dimensions is independent of the number of classes. Most prominently, we find that most of the classification performance can be maintained in output spaces of small dimension, cf., Tables 1 and 4.

heading: 4 Experiments

heading: 4.1 Ablation studies
text: Setup. We first investigate the workings of the penalized Busemann loss. We evaluate the effect of the penalty term on CIFAR-10 and CIFAR-100 using a ResNet-32 backbone. For the experiment, we
use Adam with a learning rate of 5e-4, weight decay of 5e-5, batch size of 128, without pre-training. The network is trained for 1,110 epochs with learning rate decay of 10 after 1,000 and 1,100 epochs. The prototypes are given by [29] without prior knowledge.
Results. In Figure 3, we show the results for CIFAR-100 and CIFAR-10 for various settings of the penalty term in the Busemann loss. The penalty term φ(d) depends on the output dimensionality and should be linear in form as per Theorem 1. In the Figure, we show five settings, corresponding to the slope s of the linear function, i.e., φ(d; s) = s · d. On both datasets the setting with s=0, where the penalty term is excluded, performs worst, highlighting the importance of including a penalty in the Busemann loss. For the other settings, the higher the value, the larger the penalty and the stronger the pull towards the origin of the output space. On CIFAR-10, all non-zero slopes improve the classification accuracy. Slope s=0.75 is slightly preferred with an accuracy of 92.3% versus 91.9% for s=1 and 90.1% for s=0. For CIFAR-100, a too strong penalty negatively impacts the classification results. A penalty of s=0.1 performs best with an accuracy of 65.8% compared to 39.7% and 35.9% for s=1 and s=0. We conclude that the penalty term is an influential component of the proposed loss and with more classes a lower slope is recommended for best performance.
Analysis. To better understand the penalty term and its effect, we have trained a ResNet32 backbone on CIFAR-10 using only two output dimensions. In Figure 4, we visualize the output space for three slope values, along with the ideal prototypes and test examples. The visualization clearly shows the importance of the penalty; without it, examples move closely towards the ideal prototypes leading to over-confidence. On the other hand, a high penalty pulls all examples close to the origin of the Poincaré ball. Especially with many classes,
this leads to smaller inter-class variation and higher confusion. The visualization explains the lower performance for a high penalty on CIFAR-100 compared to CIFAR10. By balancing confidence and inter-class separation, the best classification results are achieved. Figure 1 and Figure 4 show intuitively that the distance to the origin for an example reflects the confidence of the network in classifying the example. We have performed a quantitative analysis to show that our approach provides a natural interpretation of classification confidence, see Figure 5. In general, the closer the example to the origin, the lower the overall confidence, as also indicated in other hyperbolic approaches [19, 42]. The red line in Figure 5 shows that the distance to the origin in our approach also correlates with angular similarity to the correct class prototype. In other words, the larger the distance to the origin, the higher the likelihood of the classification being correct. Again, this provides a parallel to logistic regression, where the
output score can be interpreted as a classification probability, conditioned on the input. We conclude from the analysis that the severity of the linear penalty matters to balance over-confidence and inter-class confusion, while the networks outputs enable a direct interpretation of the classification results.

heading: 4.2 Comparison to hyperspherical prototypes
text: Setup. In the second experiment, we compare our approach to its hyperspherical counterpart, in which the output space is the positively curved hypersphere instead of negatively curved hyperbolic space. We draw a comparison to Hyperspherical Prototype Networks [29], which has shown to be competitive or better than conventional softmax cross-entropy and using Euclidean prototypes. For a fair comparison, we use the same prototypes and the same network backbone. The difference lies in the choice of manifold and accompanying loss function. Following [29], we report on CIFAR-100, CIFAR-10, and CUB Birds 200-2011 [45] across multiple dimensionalities. On CIFAR-100, we use 3, 5, 10, 25, and 50 output dimensions, while we use 2, 3, 4, and 5 output dimensions for CIFAR-10 and 3, 10, 50, and 100 for CUB Birds 200-2011. For CUB Birds 200-2011, we keep the main hyperparameters the same as for the CIFAR experiments and use a weight decay of 0.0001, 2110 epochs, and a slope of 0.05.
Results. We show the classification results for CIFAR-100 and CIFAR-10 on ResNet-32 in Table 1 and CUB Birds 200-2011 in Table 2. Across all dimensions, our approach performs better and the gap increases when using fewer dimensions. On CIFAR-100 for five output dimensions, we obtain 54.6% accuracy versus 28.7% for hyperspherical prototypes and the relative difference grows for three dimensions with 49.0% versus 5.5%. On CUB Birds 200-2011 for ten output dimensions, we obtain an accu-
racy of 43.5% versus 9.5% for hyperspherical prototypes. When scaling down to three dimensions, hyperspherical prototypes obtain 1.0% accuracy and we obtain 36.6%. On CIFAR-10, our performance is hardly affected by using few dimensions. The accuracy for our approach in two dimensions is still 91.2% compared to 92.3% in five dimensions and 30.0% in two dimensions for hyperspherical prototypes. The high scores with few dimensions highlights the effectiveness of our approach and is in line with the effectiveness of hyperbolic embeddings in lower dimensionalities [30].
To show that the obtained results generalize to other network backbones and choice of prototypes, we report additional results in Table 3 for CIFAR-100. Using a DenseNet-121 architecture improves both our approach and the baseline with similar relative performance improvements with lower dimensionalities. We also investigate the addition of prior semantic knowledge in prototypes as per [29]. With these prototypes, the improvements with few dimensions holds and additionally boost the performance in high dimensions. With 50 dimensions, we obtain an
accuracy of 68.0%, compared to 65.8% without using prior knowledge. In Table 4, we show additional comparisons to softmax cross-entropy and Euclidean prototypes, which re-iterate the potential of our approach both with and without prior knowledge for prototypes. Overall, we conclude that our approach is applicable and effective for classification regardless of backbone and choice of prototypes.

heading: 4.3 Comparison to hyperbolic prototypes
text: Setup. The second experiment has shown that Hyperbolic Busemann Learning boosts the classification results compared to its hyperspherical counterpart. In the third experiment, we draw a comparison to the recent hyperbolic action network [24], the only hyperbolic prototype approach to date which is not restricted to few-shot or zero-shot settings. In [24], action classes are embedded as prototypes in the interior of the Poincaré ball based on given hierarchical relations. In contrast, we position prototypes on the ideal boundary of the Poincaré ball, which can be done both with and without prior hierarchical knowledge. To showcase this possibility, we have performed a comparative evaluation on ActivityNet [10] and Mini-Kinetics [48] for the search by action name task. The ActivityNet dataset is a large untrimmed video dataset, which have been trimmed in [24] based on the available temporal annotations. The trimmed dataset contains ∼23K videos and 200 classes, with a ∼15K split for training and ∼8K for validation. Alongside trimming, a modified and more balanced ActivityNet hierarchy with three levels of granularity is provided [24]. The Mini-Kinetics dataset consists of ∼83K trimmed videos and 200 classes, with a ∼78K split for training and ∼5K for validation.We use the hierarchy with three levels of granularity for the Mini-Kinetics as detailed in [24].
We use both the prototypes from prior hierarchical knowledge [24] with entailment cones [15] and knowledge-free prototypes from separation [29]. For our approach, we project the hierarchical prototypes on the ideal boundary through `2-normalization. For both datasets, we use the same pre-trained model for feature extraction and three-layer MLP for final classification as [24], with Adam as optimizer, a learning rate of 1e-4, and learning rate drop after 900 iterations. All other hyperparameters are the same as in previous experiments. Following [24], we report three metrics:
mean Average Precision (mAP), sibling-mAP (s-mAP), which counts a prediction as correct if it is within 2-hops away in the hierarchy, and cousin-mAP (c-mAP), which does the same for 4-hops.
Results. The comparative evaluation on ActivityNet and Mini-Kinetics is shown in Table 5 and Table 6. When hierarchical knowledge is available, we only position the leaf nodes on the boundary and the internal nodes of the hierarchy will be positioned inside the Poincaré ball. Using the prototypes from hierarchical knowledge, we find that both our approach and the baseline favor low-dimensional embeddings. Our highest score for the mAP metric on Activity-Net (70.6%) outperforms the baseline (68.3%) and the same holds for sibling-mAP (95.1% versus 93.8%) and cousin-mAP (98.2% versus 97.5%). The same holds on Mini-Kinetics. These results highlight our ability to employ prototypes with prior knowledge. We note that the baseline performs additional tuning for the curvature in hyperbolic space, while we only use a standard curvature of one. A unique benefit of our approach is the ability to employ hyperbolic spaces without requiring hierarchical priors for prototypes. We find that positioning prototypes solely based on data-independent separation provides a boost in mAP, from 68.3% by [24] to 79.4%. In contrast, the hierarchical metrics score lower with the prototypes from separation. This is a direct result of the lack of hierarchical knowledge during prototype positioning. An explanation for the better performance of hierarchical prototypes over separation prototypes for s-mAP and c-mAP compared to standard mAP comes from how the prototypes are constructed. For the hierarchical prototypes, classes with the same parent will be pulled together, while other classes will be pushed away. This boosts the s-MAP and c-mAP since these metrics allow for confusion with hierarchically related classes. However, the standard mAP considers all other classes as negatives. In that case, separation prototypes are preferred as they push all classes away during prototype construction. Overall, we conclude that Hyperbolic Busemann Learning provides both an effective and general approach for prototype-based learning with hyperbolic output spaces.

heading: 5 Conclusions
text: In this paper, we introduce Hyperbolic Busemann Learning with ideal prototypes. Where existing work on prototype-based learning in hyperbolic space requires prior knowledge to operate, we propose to place prototypes at the ideal boundary of the Poincaré ball. This enables a prototype positioning without the need for prior knowledge. We introduce the penalized Busemann loss to optimize examples with respect to ideal prototypes. We provide both theoretical and empirical support for the idea of ideal prototypes and the proposed loss. We prove its equivalence to logistic regression in dimension one and show on three datasets that our approach outperforms recent hyperspherical and hyperbolic prototype approaches.
Broader impact. Our approach allows for both knowledge-based and knowledge-free prototypes and this choice matters depending on its broader utilization. Knowledge-free prototypes make fewer errors but the remaining ones are less hierarchically consistent. Hence critical applications such as in the medical domain benefit from the former, while search-based applications benefit form the latter.

heading: A Full derivation of Busemann function in Poincaré model
text: In the Poincaré model, the unit-speed geodesic γp(t) from the origin towards the ideal point p is given by
γp(t) = p tanh(t/2). Inserting into (6) and using the hyperbolic distance as given in (3) we obtain the following representation of the Busemann function:
bp(z) = lim t→∞
( arcosh (1 + x(t))− t ) ,
where
x(t) = 2 ‖p tanh(t/2)− z‖2
(1− tanh(t/2)2)(1− ‖z‖2) .
Using that tanh(t/2) = (et − 1)/(et + 1), it follows that
lim t→∞
e−tx(t) = 1
2
‖p− z‖2 1− ‖z‖2 .
In particular, this shows that x(t) grows to infinity as t→∞. Using the representation of the inverse hyperbolic cosine in terms of the logarithm, we obtain (with Landau’s small-o notation)
arcosh (1 + x(t)) = log ( 1 + x(t) + √ x(t)2 + 2x(t) ) = log ( 2x(t) + o(x(t)) ) ,
as x(t)→∞. Thus, it follows that
bp(z) = lim t→∞
{ log ( 2x(t) + o(x(t)) ) − t } = lim t→∞ log ( 2e−tx(t) + e−to(x(t)) ) =
= log ( 2 lim t→∞ e−tx(t) + 0 ) = log ‖p− z‖2 1− ‖z‖2 ,
as claimed in (7).

Review summaries:
The paper proposes Hyperbolic Busemann Learning for hyperbolic prototypes. Specifically, the prototypes are positioned on the ideal boundary of the Poincare ball model and the penalised Busemann loss is used to learn the projections of the input data to the Poincare ball model. The paper also provides a theoretical link between the proposed hyperbolic prototype approach and logistic regression.

Paper proposes an hyperbolic learning method based on ideal prototypes. The originality of the paper lies in the fact that there is no need to include prior information on the hierarchy to position the prototypes. There are located at the boundary of the Poincaré ball, and their position is computed by using algorithm defined in ref [28]. As points located close to the boundary have an infinite distance to the origin, authors propose to use a penalized busemann loss that can be defined in the hyperbolic space. The penalty term allows defining how far the embeddings can be from the origin. Paper also shows that, in 1d, it corresponds to logistic regression. Experiments are provided, considering CIFAR10 and 100 datasets, on classification tasks. Results show that the proposed method provides better results than the baselines when the dimension is low. When no information on the structure of the prototypes is given, the mAP indicator is better than when the prototypes are inferred from the data, while hierarchical metrics such as s-mAP and c-mAP show that the learnt prototypes perform better when privileged information is provided.

This paper introduces a novel model which uses prototypes placed "at infinity" in hyperbolic space for classification. Specifically, ideal prototypes are placed on the surface of the sphere, and the distance from a point inside the sphere the this idealized point is calculated using the Busemann function, which defines a distance to points at infinity using a limit. Thus, the embedded points representing instances can be interpreted as points in hyperbolic space, and prototypes as points at infinity. The authors perform a number of rigorous evaluations and ablation analysis, demonstrating their model performs well in a number of classification settings.