Hyperbolic Busemann Learning with Ideal Prototypes
Authors: Mina Ghadimi Atigh
Martin Keller-Ressel
Pascal Mettes
Venue: NeurIPS 2021 Poster (2021)
Keywords: ideal prototype
hyperbolic space

Abstract: Hyperbolic space has become a popular choice of manifold for representation learning of various datatypes from tree-like structures and text to graphs. Building on the success of deep learning with prototypes in Euclidean and hyperspherical spaces, a few recent works have proposed hyperbolic prototypes for classification. Such approaches enable effective learning in low-dimensional output spaces and can exploit hierarchical relations amongst classes, but require privileged information about class labels to position the hyperbolic prototypes. In this work, we propose Hyperbolic Busemann Learning. The main idea behind our approach is to position prototypes on the ideal boundary of the Poincar\'{e} ball, which does not require prior label knowledge. To be able to compute proximities to ideal prototypes, we introduce the penalised Busemann loss. We provide theory supporting the use of ideal prototypes and the proposed loss by proving its equivalence to logistic regression in the one-dimensional case. Empirically, we show that our approach provides a natural interpretation of classification confidence, while outperforming recent hyperspherical and hyperbolic prototype approaches.

Hyperbolic Busemann Learning with Ideal Prototypes
Mina Ghadimi Atigh
Martin Keller-Ressel Technische Universität Dresden
Pascal Mettes University of Amsterdam
University Amsterdam
Abstract
Hyperbolic space has become a popular choice of manifold for representation learn- ing of various datatypes from tree-like structures and text to graphs. Building on the success of deep learning with prototypes in Euclidean and hyperspherical spaces, a few recent works have proposed hyperbolic prototypes for classification. Such approaches enable effective learning in low-dimensional output spaces and exploit hierarchical relations amongst classes, but require privileged information about class labels position the hyperbolic prototypes. In this work, we propose Hyperbolic Busemann Learning. The main idea behind our approach is to position prototypes on the ideal boundary of the Poincaré ball. which does not require prior label knowledge. To be able to compute proximities to ideal prototypes, we introduce the penalised Busemann loss. We provide theory supporting the use of ideal prototypes and the proposed loss by proving its equivalence to logistic regression the nie-dimnniioal case. Empirically, we show that our approach provides natural interpretation of classification confidence, while outperforming recent hyperspherical and hyperbolic prototype approaches.
Introduction
Classification by prototypes has long tradition in machine learning. Foundational solutions such as the Nearest Mean Classifier [46] represent classes as the mean prototypes in a fixed feature space.
Following approaches that learning metric space for class prototypes (28). deep learning with
prototypes as points in network output spaces has gained traction [14 17
26
32
36
41
49 [5]]. In
line with foundational approaches, the prototypes are positioned by computing the mean over all
training examples for each class. Deep learning by mean prototypes has shown to be effective for and zero-shot recognition
tasks such as few-shot learning [14
51]
To avoid the need to re-learn prototypes or to enable the use of prior label knowledge, several recent works have proposed deep networks with prototypes in non-Fuclidean output spaces. Hyperspherical prototype approaches initialize prototypes on the sphere or its higher-dimensional generalization, based pair-wise angular separation [29 39 with the help of word embeddings [ 43 using
a cosine similarity loss between example outputs and class prototypes. Hyperspherical prototypes alleviate the need to re-position prototypes continuously and allow to optionally include prior semantic knowledge about the classes. Recently, a few works have extended prototype-based learning to the hyperbolic domain, where prototypes are obtained by embedding label hierarchies
(2 24 While
hyperbolic prototype approaches provide more hierarchically coherent classification results, prior hierarchical knowledge required embed prototypes. This paper strives combine the best of both non-Ean non-Euclidean worlds, namely efficient low-dimensional embeddings from hyperbolic prototypes and the knowledge-free free positioning from hyperspherical prototypes.
We make three contributions in this work. First, we introduce a hyperbolic prototype network with class prototypes given as points on the ideal boundary of the Poincaré ball model of hyperbolic geometry. Second, we propose the penalized Busemann loss, which enables us to compute proximities between example outputs in hyperbolic space and prototypes the ideal boundary, an impossible task for existing distance metrics, which put the ideal boundary at infinite distance from all other
35th Conference on Neural Information Processing Systems (NcurIPS 2021)

---

deer
bird
plane
horse
plane
dog
glane
frog
ship
frog
horse
ship
Visualization of the output space for Hyperbolic Busemann Learning in two dimen- sions on CIFAR-10. The penalized Busemann loss optimizes images to be close to their corresponding ideal prototypes while forcing all examples towards the origin to avoid over-confidence, e.g., for the deer, top plane, and ship examples. Ambiguous cases, such as the horse and bottom plane examples are projected closer to the origin, providing natural interpretation of classification confidence.
Figure 1:
points hyperbolic space. Third, we provide a theoretical link between our hyperbolic prototype approach and logistic regression. We show that our choices for output manifold, for prototypes on the ideal boundary, and for the proposed loss provide a direct generalization the logistic regression model. Experiments and hyperbolic prototype approaches. Moreover, our hyperbolic output space provides a natural interpretation of closeness to the ideal boundary as classification confidence, see Figure T The code available at wtwpghhhw.oooo/Maaaaaaniiiiiiiii http:///wwyypbolll--ussaaaa--aaa
three datasets show that approach outperforms both hyperspherical
2 Related work
As an early adaptation of prototypes in deep networks, the center loss [47 served as an additional pulled to point loss for softmax cross-entropy optimization, where all outputs of class Classifier objective, to help minimize intra-class variation. Closer to the original Nearest Mean
Prototypical Networks introduce prototype-only approach where each class is represented as the mean output of its corresponding training examples [1]. Learning with prototypes from class means
has shown to be effective for few-shot learning [4 32
36 4T. zero-shot recognition 41 49 5I],
and domain adaptation [33]. Defining prototypes as class means is feasible in few-shot learning by class mini-batch. For general supervised learning, DeepNCM [17] and DeepNNO [26] propose alternating optimization schemes, where the prototypes are fixed to update the network backbone and vice versa. To avoid the need for constant prototype updating, several works have proposed to use the hypersphere as output space instead of the Euclidean space. On the hypersphere, prototypes can positioned priori based on angular separation 29 [3].
sampling all samples
using orthonormal basis [4], or using prior knowledge from word embeddings 4 29
43 Similar
to hyperspherical prototype approaches, we seek to position prototypes at unit distance from origin. In contrast, we operate on hyperbolic manifold, which provides theoretical link to logistic regression and obtains better empirical results than a hypershperical manifold and loss.
In this work, we strive for prototype- booset based network on hyperbolic manifolds, which have become popular choice for embedding data and deep network layers. Hyperbolic embeddings have shown to represent tree-like structures [38 37), text [1] 44 55 cells [20), networks [18], multi-relational
31 [50] with minimal distortion. In deep networks, hyperbolic alternatives have been proposed for fully-connected layers [[3]
knowledge graphs [3], and general taxonomies 16
40)
convolutional layers [40), recurrent layers [15), classification layers [3 15 40 and optimizers
(22) Hyperbolic generalizations have also been introduced for graph networks 2 12
22
25
53
and generative models 27). Where hyperbolic- deep learning approaches commonly operate within the Poincaré model represented by the open unit ball, we propose to position class prototypes

---

on its ideal boundary - the shell of the unit ball allowing us to benefit from the representational power of hyperbolic geometry without requiring prior knowledge operate.
few works have recently proposed prototype-based approaches with hyperbolic output spaces. Hyperbolic Image Embeddings by Khrulkov et al. [19] include a prototype-based setting direct hyperbolic generalization of Prototypical Networks [4]], allowing for few-shot learning through hyperbolic averaging. Beyond few-shot learning, Long al. [24] introduce hyperbolic action network, where action prototypes are projected to a hyperbole based on prior knowledge about hierarchical relations. Videos using the hyperbolic distance, enabling hierarchically consistent action recognition and zero-shot recognition. Similarly, Liu et al. [23] perform zero-shot recognition using hyperbolic prototypes positioned based on known hierarchical relations. In this work, we also employ prototypes in hyperbolic space, but do not require prior knowledge about hierarchies. Recent approaches with hyperbolic output spaces have shown to be effective in low dimensionalities [T9 24 23) Low dimensional embeddings have several practical use cases and benefit.. We list two important use cases. Compactness: Hyperbolic Busemann Learning enables us to embed high-dimensional inputs in low-dimensional output spaces. This ability has potential benefits in applications such as data transmission, compression, and storage. Interpretability: Obtaining high performance with few output dimensions (i.e., two or three output dimensions) provides the potential to interpret the model performance and visualize the output space, see Figure
turn projected the same space and compared to the prototypes
3 Learning with hyperbolic ideal prototypes
3.1 Problem formulation
We are given a training set (x) with N examples, where x; E TR denotes an input with dimensionality 7 and ys E (1,--,C) C) denotes label from set of size C. We seek learn projection of inputs to a hyperbolic manifold in which can compute proximities to class prototypes. We do so through a transformation F in Euclidean space, followed by an exponential map from the tangent space T.M to hyperbolic manifold M:
z =
(1)
where 0 denotes the parameters to learn the transformation. The transformation can be any function,
e.g., deep network or linear layer akin to logistic regression. The hyperbolic manifold in which
z operates is described throughout this work by the Poincaré ball model (Ba, 9x ) with open ball
B2 = {z E Rd || < }} and the Riemannian metric tensor dh 4(1 ||z
Setting its center to v the exponential map in the Poincaré model is given by:
expo (x) tanh(()x1/2)) ||x||
(2)
Furthermore, the geodesic distance between two points 21, 22 E B. is given by:
(z 22) = arcosh 1 (1+ 2 (1
21 22l12
(3)
|
22
To train our model, we compare the projected inputs to their class labels, represented by the prototypes P {Pi..., .., }c}. That is, the overall training loss is given the aggregation of distance-based individual losses between projected inputs and the prototype of their respective class labels:
1 N
(())=
(Fx;)
3.2 Ideal prototypes and the penalized Busemann loss
The central idea of our approach is position class prototypes at ideal points, which represent points at infinity in hyperbolic geometry. In the Poincaré model, the ideal points form the boundary of the ball:
La = {z E Rd :27
2 = 1}.
(5)
3

---

Figure 2: Visualization of the penalized Busemann loss. The left plot shows a color map of the penalized Busemann loss function, see (8). in dimension two for single ideal prototype located at p (1, 0); contour lines are log-spaced. The right plot shows its gradient with respect to the radial coordinate. During loss minimization, points in the drop-shaped region adjacent to the prototype will move outwards, while points outside of the region will move inwards.
Thus, the of ideal points of hyperbolic space B& is homeomorphic to the hypersphere Sa. As consequence, any of the methods used for prototype embedding into Sy can be used to embed prototypes into Ig. In particular, for while class-agnostic prototype embedding based on separation can be used for d 2 299
prototypes can placed uniformly on the unit sphere S2.
A key issue that needs to be solved, is that ideal points at infinite geodesic distance from all other points in B. and hence the geodesic distance can not be used directly for prototype-basee learning. Therefore, we propose the penalized Busemann loss for hyperbolic learning with ideal prototypes. The Busemann function, originally introduced in [9] (see also Def. 11.8. 17 in [8), can be considered distance infinity and may be defined in any metric space. Let p be an ideal point and %p geodesic ray, parametrized by are length, tending to Then the Busemann function with respect to p is defined for z E Bd as:
bp(2) (z) = lim (da z) t).
(6)
+-
In the Poincaré model, the limit can be explicitly calculated and the Busemann function is given as:
|p 2
(p(2) log
(1 ||
The full derivation of 7 is provided in Appendix Our proposed loss combines the Busemann function with penalty term:
(Z, (p(2) <dd) log(1 ||211²),
with d(d) a scaling factor for the penalty term which is a function of the dimension of the hyperbolic space. The first term in the penalized Busemann loss steers projected input towards class prototype p. while the second term penalized overconfidence, i.e., values close to the ideal boundary of Ba. The prototypes positioned before training and remain fixed, hence the backpropagation only needs to be done with respect to the inputs. The gradient of ((expo(x), p) with respect to x is given as:
lanh[[|x|]]
||nh(|/x()/||||||| -
Vx l(expo (x), p) (x-p)
11 p-x
tanh([]x|]] x
X tanh( xx| p-x
(9)
shows a color map the penalized Busemann loss with d-dimensional vector of ones. Figure 2 itself and its radial gradient for prototype located at ideal point p (),
In the penalized Busemann loss, ((d) governs the amount of regularization. Below, we show that this function should be linear the number of dimensions.
Theorem 1. For any d 2 4 and ideal point p the penalized Busemann loss l(2, p) the log-likelihood of density fz. p) c(aj exp (-E(2, p)) the Poincaré ball B if and only if ((d) d 2.

---

Proof. To show that f(z.p) is density, we have to show that the normalization constant
C(d) = / cxp(-((z.p)) dV
is finite. Here, dV is the hyperbolic volume element on Ba, which equal 2º(1 [zll))_ddzz ... dza, [34]. Furthermore, using invariance of the integrand under rotation, assume without loss of generality that is equal to the unit vector é1 = (1,0, .), which leads to
C(d) C(d)= =2 JB. ( 221 121
/ [1
d21 .zd.
|2|2
Switching to (Fuclidean) hyperspherical coordinates (r, (r,41,, .r......... pa), the integral factorizes into C(d) 2d Ci(d) C2(d), where
Ci(d) S.' J. 1- 1- c0s((()) + ) ,d sin(()))))) o 2r dpd
(1 (1- -2)000001++11
C2(d) //¹ sin( ... (
2x
sin(pa-1)dda-
Clearly, (d) is finite, independently of d, such that it remains to consider Ci(d). Instead of solving the integral exactly, we use the elementary estimate
sin( S
2r cos((21) + S 4,
valid for all [0, 1j. This estimate allows to obtain both lower and upper bound Ci (d). The upper bound becomes
Ci(d) < 1/' (1- (1- a - [ sin((p1) ( A
which (given that 2 4) is finite if and only if (dd) > d - 2. The lower bound becomes
(i) 2 1/400 / (1 -1- -- dr Lo sin({1) ( dip,,
1
4
which (given that 2 2) is also finite and only if (dd > d - 2, which completes the proof.
Given a trained model, inference is performed by projecting the test example and selecting the class with the smallest prototype distance:
1f" = arg min (Fxx pEP
(10)
The Poincaré ball model of hyperbolic geometry has the conformal property, i.e., angles in the Poincaré ball model coincide with angles in Euclidean space [35) Thus, the cosine similarity between two ideal points remains meaningful measure of their similarity, even in hyperbolic geometry. As the penalty term is the same for all prototypes, we can equivalently minimize the Busemann function of the prototypes directly. Moreover, the Busemann function is a decreasing function of the cosine similarity, prediction is equivalent to hyperspherical prototype inference:
y arg max PEP
Z
z = expo 8)).
(11)
Z
3.3 Relation to logistic regression
Logistic regression is foundational algorithm for classification and pillar in supervised learning for deep networks with cross entropy optimization. Here, we show that hyperbolic ideal prototype learning is a natural generalization of logistic regression by showing its equivalence when using = 1 dimensions our approach.
Theorem 2. For dimension d and with penalty ((1) Hyperbolic Busemann Learning with single linear layer is equivalent logistic regression.
5

---

Test Accuracy
50-
Accuracy(
40
0.25
0.25
10
0.5
0.5
0.75
200 400
600
800
1000
200 400
600
800
1000
Number of Training epochs
Number of Training epochs
(a) CIFAR-100 (50D).
(b) CIFAR-10 (5D).
Figure 3: Ablation study the penalized Busemann loss for CIFAR-100 and CIFAR-10. On both datasets, adding non-negative penalty (0.1 to 1) improves the classification results over the Busemann loss without penalty (0). With more classes, lower penalty is preferred. The y-axis CIFAR-10 starts from 60% to better visualize differences between settings.
Proof. Recall that from the machine learning point of view logistic regression can be described as single linear layer in combination with the logistic activation function and cross-entropy- dimension. hyperbolic space 13 becomes the interval (-1, 1). The set of ideal points 11 consists of the two points 1.. To identify these prototypes with 0/1 (the class labels in logistic regression) we introduce a linear change of coordinates to
one
2 ++
Under this change of coordinates B2 becomes the unit interval (0, 1) and the prototypes p' (the ideal points) become the endpoints 0 and 1. The exponential map, i.e. the embedding of the base learner output y = wt x + wy into B1, maps E R to
expo (v) (anh(y) ( +1
2
1-e=2
which is the logistic function. In 2' e'-coordinates, the penalized Busemann loss becomes
1p' 1p'-211 2z'(1 z¹)
2 log 2(1=²))
if
((z'v1) = 2og
log 2=
pf =
Rewriting this
'(=', logg(2 = -p' log(z') ((1-'))lor) log(1 z'),
shows that the shifted and scaled penalized Busemann loss coincides with the cross-entropy loss in dimension d = 1.
Beyond binary classification, multinomial logistic regression performed in a one-vs-rest manner with a K K-dimensional softmax output for K classes. i.e., in conventional multinomial logistic regression the output space scales one-to-one with the number of classes. With Hyperbolic Busemann Learning, provide a different generalization of logistic regression, where the number of output dimensions independent of the number classes. Most prominently, we find that most of the classification performance be maintained in output spaces of small dimension, cf., Tables and
Experiments
4.1 Ablation studies
Setup. We first investigate the workings of the penalized Busemann loss. evaluate the effect of the penalty term CIFAR-10 and CIFAR-100 100 using ResNet-32 backbone. For the experiment,

---

deer
car
deer
deer
dog
dog
dog
plane
plane
truck
truck
truck
trog
Trog
trog
horse
Ship
horse
ship
horse
'hhp
(a) slope
(b) slope 0.75
(c) slope
Figure 4: Visualization of the penalty term and its effect on the output space with a ResNet-32 on CIFAR-10. Without a penalty, examples are close to the boundary, leading to overconfidence. Contrarily, a high penalty forces examples towards the origin of the Poincaré ball, leading to inter-class confusion. With the right slope, these factors are balanced, resulting in improved classification.
use Adam with learning rate of 5e-4, weight decay of 5e-5, batch size of 128, without pre-training. The network is trained for 1,110 epochs with learning rate decay of 10 after 1,000 and 1,100 epochs. The prototypes given by [29] without prior knowledge.
show the results for CIFAR- 100 and CIFAR-10 for various tings of the penalty term in the Busemann loss. The penalty term ((d) depends on the output dimensionality and should be linear in form as per Theorem 1. In the Figure, show five settings, corresponding to the slope of the linear function, i.e., ó(d; 8) 8 d. On both datasets the setting with 0. where the penalty term is excluded, performs worst, highlighting the importance of includ- ing a penalty in the Busemann loss. For the other settings, the higher the value, the larger the penalty and the stronger the pull towards the origin of the output space. On CIFAR-10. all non-zero slopes improve the classification accuracy. Slope s = 0.75 is slightly preferred with an accuracy of 92.3% versus 91.9% for and 90.1% % for s 0. For CIFAR- 100, too strong penalty negatively impacts the classification results. A penalty of = 0.1 performs best with an accuracy of 55,8% compared to 39.7% and 35.9% for
Results. In Figure 3
and s We conclude that the penalty term is influential com- ponent of the proposed loss and with more classes a lower slope is recommended for best performance.
Analysis. To better understand the penalty term and its effect, we have trained ResNet- 32 backbone on CIFAR-10 using only two output dimensions. In Figure 3 we visualize the output space for three slope values, along with the ideal prototypes and test examples. The visualization clearly shows the importance of the penalty; without it, examples move closely towards the ideal prototypes leading to over-confidence. On the other hand, high penalty pulls all examples close to the origin of the Poincaré ball. Especially with many classes, this leads to smaller inter-class variation and higher con- fusion. The visualization explains the lower performance for a high penalty on CIFAR-100 compared CIFAR- 10. By balancing confidence and inter-class separation, the best classification results are achieved. Figure and Figure show intuitively that the distance to the origin for example reflects the confidence of the network in classifying the example. We have performed a quantita- tive analysis show that our approach provides natural interpretation of classification confidence, see Figure(5] In general, the closer the example the origin, the lower the overall confidence, as also indicated in other hyperbolic approaches 119 42 The red line in Figure shows that the distance to the origin in our approach also correlates with angular similarity to the correct class prototype. other words, the larger the distance to the origin, the higher the likelihood of the classification being correct. Again, this provides parallel to logistic regression, where the
Euclidean disaance the oriiin
0.30
0.25
0.15
0.10
0.00
0.2
Cosine similarity to own class
Figure Confidence interpretation. The larger the distance to the origin, the more likely the example is correctly clas- sified, showing that hyperbolic ap- proach provides a natural interpretation of classification confidence.
7

---

Table Comparison Hyperspherical Prototype Networks on CIFAR- 100 and CIFAR- 10 using the same prototypes and ResNet-32. denotes results from code provided by 22 in case the number was not reported the original paper. For both datasets, our approach obtains high accuracy, especially with few output dimensions.
CIFAR-100
CIFAR-10
Dimensions
3
5
10
25
50
2
3
4
5
Mettes et al.
5.5
28.7
51.1
63.0
64.7
30.0*
84.9*
88.8*
89.5'
This paper
49.0
54.6
59.1
65.7
65.8
91.2
92.2
92.2
92.3
output score can be interpreted as classification probability, conditioned on the input. We conclude from the analysis that the severity of the linear penalty matters to balance over-confidence and inter-class confusion, while the networks outputs enable a direct interpretation of the classification results.
4.2 Comparison to hyperspherical prototypes
Setup. In the second experiment, compare our approach to its hyperspherical counterpart, in which the output space the positively curved hypersphere instead of negatively curved hyperbolic space. We draw comparison to Hyperspherical Prototype Networks (29). which has shown to be competitive or better than conventional softmax cross entsoyy and using Euclidean prototypes. For a fair compari- son, use the same prototypes and the same network backbone. The difference lies in the choice of manifold and accompanying loss function. Following [29], we report on CIFAR- 100, CIFAR- 10, and 45 across multiple dimensionalities. On CIFAR-100,
use 3, 5, 10, 25, and 50 output dimensions, while we use 2, 3, 4, and 5 output dimensions for CIFAR- 10 and 3, 10, 50, and 100 for CUB Birds 200-2011, I. For CUB Birds 200-2011, 1, keep the main hyperparameters the same as for the CIFAR experiments and use a weight decay of 0.0001, 21 10 epochs, and a slope of 0.05.
CUB Birds 200-2011
Results. We show the classification results for CIFAR- 100 and CIFAR-I0 on ResNet-32 in
Table 2:
Comparison to Hyperspherical Proto- type Networks on CUB Birds 200-2011 using the same prototypes and ResNet-32. denotes re- sults from code provided by [29 in case the num- ber was not reported in the original paper.
Table D and CUB Birds 200-2011 in Table 2 Across all dimensions,
approach performs better and the gap increases when using fewer dimensions. On CIFAR- 100 for five output di- mensions, we obtain 54.6% accuracy versus 28.7% for hyperspherical prototypes and the rel- ative difference grows for three dimensions with 49.0% versus 5.5%. On CUB Birds 200-2011 for ten output dimensions, we obtain an accu-
CUB Birds 200-2011
Dimensions
3
10
50
100
Mettes et al. (29
1.0*
9.5"
41.5"
45.6*
This paper
36.6
43.5
43.5
45.6
racy of 43.5% versus 9.5% for hyperspherical prototypes. When scaling down to three dimensions, hyperspherical prototypes obtain 1.0% accuracy and we obtain 36.6%. On CIFAR- 10, our perfor- mance hardly affected by using few dimensions. The accuracy for our approach in two dimensions is still 91.2% compared to 92.3% in five dimensions and 30.0% in two dimensions for hyperspherical prototypes. The high scores with few dimensions highlights the effectiveness of our approach and is in line with the effectiveness of hyperbolic embeddings in lower dimensionalittis
30
Table 3: Comparison with varying architecture and prototypes on CIFAR-100. denotes results from code provided by (29] in case the number was not reported in the original paper. Our approach is applicable and effective for any choice of network backbone and choice of prototypes.
DenseNet-122
ResNet-32 w/ prior knowledge
Dimensions
3
10
25
50
3
10
25
50
Mettes al.
29
11.5*
45.1* 1*
60.5
69.1
71.1*
11.5
37.0
57.0
64.0
65.2"
This paper
66.1
69.0
71.1
72.7
71.8
52.2
56.4
62.4
65.9
68.0
8

---

To show that the obtained results generalize to other network backbones and choice of proto- types, we report additional results in Table 3 for CIFAR-100. Using a DenseNet- 121 architecture improves both our approach and the baseline with similar relative performance improvements with lower dimensionalities. We also investi- gate the addition of prior semantic knowledge in prototypes as per [29]. With these prototypes, the improvements with few dimensions holds and additionally boost the performance in high dimensions. With 50 dimensions, we obtain an accuracy of 68.0%, compared 65.8% without using prior knowledge. In Table 4 we show addi- tional comparisons to softmax cross-entropy and Euclidean prototypes, which re-iterate the potential of our approach both with and without prior knowledge for prototypes. Overall, we conclude that our approach is applicable and effective for classification regardless of backbone and choice of prototypes.
Table 4: Comparative evaluation to softmax cross-entropy, Euclidean prototypes, and hyper- spherical prototypes on CIFAR- 1-FAR-100 with ResNet- 32. Baseline numbers taken from 29
Manifold
Prior knowledge
w/o
w/
Softmax CE
62.1
Guerriero el al. 17
R 100
62.0
Mettes et al. This paper
29
5
65.0
65.2
B
65.8
68.0
4.3 Comparison to hyperbolic prototypes
Setup. The second experiment has shown that Hyperbolic Busemann Learning boosts the classifica- tion results compared to its hyperspherical counterpart. In the third experiment, we draw comparison to the recent hyperbolic action network [24], the only hyperbolic prototype approach to date which not restricted to few-shot zero-shot settings. In [24], action classes are embedded as prototypes contrast, we position prototypes the ideal boundary of the Poincaré ball, which can be done both with and without prior hierarchical knowledge. To showcase this possibility, we have performed a comparative evaluation on ActivityNet [T0] and Mini-Kinetic [48] for the search by action name task. The ActivityNet dataset is a large untrimmed video dataset, which have been trimmed in (24) based on the available temporal annotations. The trimmed dataset contains ~23 videos and 200 classes, with 155 split for training and ~8K for validation. Alongside trimming, a modified and more balanced ActivityNet hierarchy with three levels of granularity is provided (24). The Mini-Kinetics dataset consists of ~83K trimmed videos and 200 classes, with a 78K split for training and ~5K for validation. We
in the interior of the Poincaré ball based on given hierarchical relations.
the hierarchy with three levels of granularity for the Mini-Kinetics as detailed in (4
We use both the prototypes from prior hierarchical knowledge (24 with entailment cones [5 and knowledge-free prototypes from separation (29]. For approach, project the hierarchical prototypes on the ideal boundary through & Ez-normalization. For both datasets, we use the same pre-trained model for feature extraction and three-layer MLP for final classification as (24), with Adam as optimizer, a learning rate of le-4, and learning rate drop after 900 iterations. All other hyperparameters are the same in previous experiments. Following [24 report three metrics:
Table 5: Hyperbolic prototypes comparison on ActivityNet for the search by video name task. When using prototypes from hierarchical knowledge. we score best across the three metrics when using 10 output dimensions. Where [24] is restricted to prototypes from hierarchies, our approach also employ fnowledge-free prototypes, e.g., from separation, to boost the standard mAP metric.
Prototypes from hierarchies
Prototypes from separation
Dimensions
10
20
50
2
10
20
50
Long al. (24
mAP
26.9
68.3
66.9
67.1
S-mAP
72.0
93.8
92.7
93.1
c-mA
91.9
97.7
97.2
97.5
This paper
mAP
40.8
70.6
66.3
65.6
44.1
78.0
79.4
78.5
S-MAP
79.2
95.1
94.5
94.7
48.9
88.6
86.2
86.7
C-MAP
92.9
98.0
97.8
98.2
60.8
84.0
90.6
91.7

---

Table 6: Hyperbolic prototypes comparison on When comparing the models trained on the prototypes from hierarchies, our model outperforms when using low dimension embeddings.
for the search by video name task.
Prototypes from hierarchies
Prototypes from separation
Dimensions
10
20
50
2
10
20
50
Long et al.
mAP
28.8
64.4
63.5
64.5
S-mAP
79.0
94.2
94.1
94.5
C-mAP
91.0
96.5
96.3
96.5
This paper
mAP
51.7
67.5
65.0
64.0
56.4
70.7
71.6
71.7
s-mAP
88.3
95.6
95.1
95.0
62.0
76.8
78.8
80.0
c-mAP
94.1
97.1
97.1
97.0
71.0
82.5
84.5
85.6
mean Average Precision (mAP), sibling-mAP (s-mAP), which counts prediction correct if within 2-hops away the hierarchy, and cousin-mAP cousin-mAP(e-mAP), which does the same for 4-hops.
Results. The comparative evaluation on ActivityNet and Mini-Kinetics is shown in Table 5 and Table When hierarchical knowledge is available, we only position the leaf nodes the boundary and the internal nodes of the hierarchy will be positioned inside the Poincaré ball. Using the prototypes from hierarchical knowledge, we find that both our approach and the baseline favor low-dimensional embeddings. Our highest score for the mAp metric on Activity-Ner (70.6%) outperforms the baseline (68.3%) and the same holds for sibling-mAP (95. 1% versus 93.8%) and cousin-mAP (98.2% versus 97.5 97.5%). The same holds on Mini- Kinetikns These results highlight our ability to employ prototypes with prior knowledge. We note that the baseline performs additional tuning for the curvature in hyperbolic space, while we only use standard curvature of one. A unique benefit of our approach the ability to employ hyperbolic spaces without requiring hierarchical priors for prototypes. We find that positioning prototypes solely based on data-independent separation provides a boost in mAP, from 68.3% by (24] to 79.4%. contrast, the hierarchical metrics score lower with the prototypes from separation. This is direct result of the lack of hierarchical knowledge during prototype positioning. An explanation for better performance of hierarchical prototypes separation prototypes for s-mAP and c-mAP compared standard mAP comes from how the prototypes are constructed. For the hicrarchical prototypes, classes with the same parent will be pulled together, while other classes will be pushed away. This boosts the s-MAP and c-mAP since these metrics allow for confusion with hierarchically related classes. However, the standard mAP considers all other classes negatives. In that case, separation prototypes are preferred as they push all classes away during prototype construction. Overall, we conclude that Hyperbolic Busemann Learning provides both an effective and general approach for prototype-based learning with hyperbolic output spaces.
5 Conclusions
In this paper, we introduce Hyperbolic Busemann Learning with ideal prototypes. Where existing work on prototype-based learning in hyperbolic space requires prior knowledge to operate, we propose to place prototypes at the ideal boundary of the Poincaré ball. This enables a prototype positioning without the need for prior knowledge. We introduce the penalized Busemann loss to optimize examples with respect to ideal prototypes. We provide both theoretical and empirical support for the idea of ideal prototypes and the proposed loss. We prove its equivalence to logistic regression in dimension one and show on three datasets that our approach outperforms recent hyperspherical and hyperbolic prototype approaches.
Broader impact. Our approach allows for both knowledge-based and knowledge-free prototypes and this choice matters depending on its broader utilization. Knowledge-free prototypes make fewer errors but the remaining ones are less hierarchically consistent. Hence critical applications such as in the medical domain benefit from the former, while search-basse applications benefit form the latter.
10

---

References
[1] Rami Aly, Shantanu Acharya, Alexander Ossa, Arne Köhn, Chris Biemann, and Alexander Panchenko. Every child should have parents: taxonomy refinement algorithm based on hyperbolic term embeddings. ACL, 2019.
[2] Gregor Bachmann, Gary Bécigneul, and Octavian Ganea, Constant curvature graph convolu- tional networks. In ICML, 2021.
[3] Ivana Balazevic, Carl Allen, and Timothy Hospedales. Multi-relational poincaré graph embed- dings. NeurlPS, 2019.
[4] Bjorn Barz and Joachim Denzler. Deep learning on small datasets without pre-training using cosine loss. In WACV, 2020.
[5] Gary Bécigneul and Octavian Eugen Ganea. Riemannian adaptive optimization methods. ICLR, 2019.
[6] Silvere Bonnabel. Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control, (1809):2217-2225 2013.
[7] Joey Bose, Ariella Smofsky, Renjie Liao, Prakash Panangaden, and Will Hamilton. Latent variable modelling with hyperbolic normalizing flows. In ICML, 2020.
[8] Martin Bridson and André Haefliger. Metric spaces of non-positive curvature. Springer Science & Business Media, 2013.
[9] Herbert Busemann. The geomelry of geodesicy. Academic Press, 1955.
[10] Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: large-scale video benchmark for human activity understanding. In Proceedings of the ieee conference computer vision and pattern recognition, pages 961-970, 2015.
[11) W. James Cannon, William J. Floyd, Richard Kenyon, and Walter R. Parry. Hyperbolic geometry. In Silvio Levy, editor, Flavors of Geometry, pages 59-115, MSRI Publications, 31 edition, 1997.
[12] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. In NeurIPS, 2019.
[13] Hyunghoon Cho, Benjamin DeMeo, Jian Peng, and Bonnie Berger. Large-margin classification in hyperbolic space. In AISTATS, 2019.
[14] Kaize Ding, Jianling Wang. Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph prototypical networks for few-shot learning on attributed networks. In CIKM, 2020.
[15] Dutavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. In ICML, 2018.
[16] Octavian Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. NeurlPS, 2018
[17] Samantha Guerriero, Barbara Caputo, and Thomas Mensink. Deep nearest class mean classifiers. In ICLRw, 2018.
[18) Martin Keller-Ressse and Stephanie Nargang. Hydra: method for strain-minimizing hyperbolic embedding of network-an distance- based data. Journal of Complex Networks, 8(1), 2020.
[19] Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lem- pitsky. Hyperbolic image embeddings. In CVPR, 2020.
[20] Anna Klimovskaia, David Lopez-Pazz Léon Bottou, and Maximilian Nickel. Poincaré maps for analyzing complex hierarchies in single-cell data. Nature communications, 11(1):1-9, 2020.
[21] Mare Law, Renjie Liao, Jake Snell, and Richard Zemel. Lorentzian distance learning for hyperbolic representations. In ICML, 2019.

---

Representation tradeoffs for [22] Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks. NeurIPS, 2019.
[23] Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong- Wah Ngo, Tat-Seng Chua, and Yu Yu-Gang Jiang. Hyperbolic visual embedding learning for shot recognition. In CVPR, 2020.
[24] Teng Long, Pascal Mettes, Heng Tao Shen, and Cees GMM Snock. Searching for actions on the hyperbole. CVPR, 2020.
[25] Aaron Lou, Isay Katsman, Qingxuan Jiang. Serge Belongie, Ser-Nam Lim, and Christopher De Sa. Differentiating through the fréchet mean. In ICML, 2020.
[26] Massimiliano Mancini, Hakan Karaoguz, Elisa Ricci, Patric Jensfelt, and Barbara Caputo. Knowledge is never enough: Towards web aided deep open world recognition. In ICRA, 2019.
[27] Emile Mathieu and Maximilian Nickel, Riemannian continuous normalizing flows. NeurIPS, 2020.
[28] Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classification: Generalizing to new classes at near-zero cost. TPAMI, 252224222422 2013.
[29] Pascal Mettes, Elise van der Pol, and Cees G.M. Snoek. Hyperspherical prototype networks. In NeurIPS, 2019.
[30] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical represen- tations. In NeurIPS, 2017.
[31] Maximillian Nickel and Douwe Kiela. Learning continuous hierarchies the lorentz model of hyperbolic geometry. In ICMI., 2018.
[32] Frederik Pahde, Mihai Puscas, Tassilo Klein, and Moin Nabi. Multimodal prototypical networks for few-shot learning. In WACV, 2021.
[33] Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wa Wah Ngo, and Tao Mei. Transferrable prototypical networks for unsupervised domain adaptation. In CVPR, 2019.
[34] John Ratcliffe. Foundations of hyperbolic manifolds, volume 149, Springer Science & Business Media, 2006.
[35] John G Rateliffe, Axler, and KA Ribet. Foundations of hyperbolic manifolds, volume 149. Springer, 1994.
[36] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenen- baum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classification. ICLR, 2018
[37] Frederic Sala, Chris De Sa, Albert Gu, and Christopher Ré. hyperbolic embeddings. In ICML., 2018.
[38] Rik Sarkar. Low distortion delaunay embedding of trees in hyperbolic plane. In ISGD, 2011.
[39] Jiayi Shen, Zehao Xiao, Xiantong Zhen, and Lei Zhang. Spherical zero-shot learning. TCSVT, 2021.
[40] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. ICLR, 2021.
[41] Jake Snell, Kevin Swersky, and Richard $ Zemel. Prototypical networks for few-shot learning. NewrIPS, 2017.
[42] Didac Surís, Ruoshi Liu, and Carl Vondrick. Learning the predictability of the future. In CVPR, 2021.
[43] William Thong, Pascal Mettes, and Cees G M Snoek. Open cross-domain visual search. CVIU, 2020.
12

---

[44] Alexandru Tifrea, Gary Bécigneul, and Octavian-Euger Eugen Ganea. Poincaré glove: Hyperbolic word embeddings. In ICLR, 2019.
[45] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011 1-0011 California Institute of Technology. 2011.
[46] Andrew R Webb. Statistical pattern recognition. John Wiley & Sons, 2003.
[47] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In ECCV, 2016.
[48] Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy, Rethinking spa- tiotemporal feature learning: Speed accuracy trade-offs in video classification. In Proceedings of the European conference on computer vision (ECCV), pages 305-321, 2018.
[49] Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, and Zeynep Akata. Attribute prototype network for zero-shot learning. NeurlPS, 2020.
[50] Tao Yu and Chris De Sa. Numerically accurate hyperbolic embeddings using tiling-based based models, In NeurlPS, 2019.
[51] Yunlong Yu, Zhong Ji, Jungong Han, and Zhongfei Zhang. Episode-based prototype generating network for zero-shot learning. In CVPR, 2020.
[52] Hongyi Zhang, Sashank Reddi, and Suvrit Sra. Riemannian svrg: Fast stochastic optimization on riemannian manifolds. In NeurlPS, 2016.
[53] Yiding Zhang. Xiao Wang, Chuan Shi, Nian Liu, and Guojie Song. Lorentzian graph convolu- tional networks. In WWW, 2021.
[54] Shichao Zhu, Shirui Pan, Chuan Zhou, Jia Wu, Yanan Cao, and Bin Wang. Graph geometry interaction learning. In NeurlPS, 2020.
[55] Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, and Qun Liu. Hypertext: Endowing fasttext with hyperbolic geometry. In EMNLP, 2020.
A Full derivation of Busemann function in Poincaré model
In the Poincaré model, the unit-speed geodesic (t) from the origin towards the ideal point is given by
(t) p tanh(t/2).
Inserting into 66) and using the hyperbolic distance as given in we obtain the following represen- tation of the Busemann function:
(x) = lim (arcosh (1 + xxt)) - t)) 1000
where
lp (aanh(t/2)-22) =||²
x()) = 2
(1 tanh(t/2)))) ||
Using that tanh(t/2)- = (e' - 1)/(e' + 1), it follows that
|pp 2 lim ez((()) 2 - || 1000
In particular, this shows that rr(t) grows to infinity as & -+ oc. Using the representation of the inverse hyperbolic cosine in terms of the logarithm, we obtain (with Landau's small-o notation)
arcosh (1 + x())) log (1 + xx() + v Vz(t)=+ 20(1)) log (2x(1)) +(xx())))
as (t) + co. Thus, it follows that
bp(2) 1+00
lim {log (2r(t) o(x(())) - lim log 10
+ te +==oo((())
||p =|| 2
log 22 lim ex((( + +0) = =log 1
|||| 2
as claimed in 7
13


Reviews:
[{"review_id": "ngpT4TblB0X", "reviewer": "Reviewer_HYUM", "summary": "This paper introduces a novel model which uses prototypes placed \"at infinity\" in hyperbolic space for classification. Specifically, ideal prototypes are placed on the surface of the sphere, and the distance from a point inside the sphere the this idealized point is calculated using the Busemann function, which defines a distance to points at infinity using a limit. Thus, the embedded points representing instances can be interpreted as points in hyperbolic space, and prototypes as points at infinity. The authors perform a number of rigorous evaluations and ablation analysis, demonstrating their model performs well in a number of classification settings.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Overall\n\n**Originality:** This paper is reasonably novel, leveraging the Busemann loss function to provide a hyperbolic analog for prototype classification methods.\n\n**Quality:** The submission is technically sound. The approach is reasonable, claims made are proven directly in the paper and appear to be correct.\n\n**Clarity:** The paper is very clearly written and well organized.\n\n**Significance:** The results are significant, extending tools and methods researchers are interested in to the hyperbolic setting.\n\n---\n### Specific Comments / Questions\n\n- (Comment) I was initially prepared to discard the connection to logistic regression in one dimension, as I assumed this would simply be an uninteresting degenerate case, however the connection the authors drew to higher-dimensional models with prototypes being an alternative to multinomial logistic regression did provide interesting motivation.\n- The ablation for the penalized Busemann loss suggest that a larger number of classes will be much more sensitive to the weight of the penalty term. This seems as though it may be inherent to the nature of hyperbolic space, for which volumes of spheres increase exponentially with respect to the radius and thus (in some sense) the penalty term is severely limiting the capacity of the model. Some theoretical exploration or additional experiments on datasets with larger number of labels would provide more confidence that this is not an issue.\n- Much of the claimed benefit for hyperbolic embeddings comes from their inductive bias toward modeling hierarchical data. Placing the prototypes for hierarchical data at the boundary would seemingly discard this essential property. In short, it seems as though you would not *want* to use idealized prototypes, placed on the boundary of the sphere, in the case where the prototypes themselves conform to a hierarchy. I would appreciate it if the authors could clarify this concern.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Some limitations are touched on. While the paper does not oversell it's methods, it is also not particularly self-critical, and some of the questions I raise above do suggest various limitations of the model which might be explored further.\n\nThe work is of a theoretical nature, and does not present any direct negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "e4clHRgAW1w", "reviewer": "Reviewer_2AsY", "summary": "Paper proposes an hyperbolic learning method based on ideal prototypes. The originality of the paper lies in the fact that there is no need to include prior information on the hierarchy to position the prototypes. There are located at the boundary of the Poincaré ball, and their position is computed by using algorithm defined in ref [28]. As points located close to the boundary have an infinite distance to the origin, authors propose to use a penalized busemann loss that can be defined in the hyperbolic space. The penalty term allows defining how far the embeddings can be from the origin. Paper also shows that, in 1d, it corresponds to logistic regression.\nExperiments are provided, considering CIFAR10 and 100 datasets, on classification tasks. Results show that the proposed method provides better results than the baselines when the dimension is low. When no information on the structure of the prototypes is given, the mAP indicator is better than when the prototypes are inferred from the data, while hierarchical metrics such as s-mAP and c-mAP show that the learnt prototypes perform better when privileged information is provided.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is very clear and well written. The introduction of the busemann loss is smart and allows defining an elegant loss, with a link with logistic regression in 1d.\nMy main concern is about the significance of the paper: the algorithm is shown to be effective in low dimensional embedding spaces (as often noticed in the litterature) and the performances come close to the spherical version of [28] when the dimension increases. The paper should emphasize some use cases on which it is important to have such low dimensional spaces (when the classification performance is sought, why considering low dimensional spaces that degrades the performances?). In Table 4, one can notice that ref [23] gives comparable performances when the dimension reaches 50 and one may wonder how the proposed method behaves in larger spaces (one can see that the performances seem to degrade when the dimension increases?).\nIn addition, some results in Table 2 seem surprising. While s-map and c-map perform better when hierarchical prototypes rather than separation prototypes are used, it is not the case for the mAP metrics. It would be interesting to give an explanation. \n\nMinor comments:\n- Code was not made available to re-run the experiments\n- Table 4: ref [23] instead of [28]\n- it would strengthen the conclusion if an other dataset could be considered (not only CIFAR).\n- results are only given for lower dimensional spaces (e.g. 50 instead of 99 or 100) in table 3: why not considering the same dimension?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Pljg-orIFj1", "reviewer": "Reviewer_28qA", "summary": "The paper proposes Hyperbolic Busemann Learning for hyperbolic prototypes. Specifically, the prototypes are positioned on the ideal boundary of the Poincare ball model and the penalised Busemann loss is used to learn the projections of the input data to the Poincare ball model. The paper also provides a theoretical link between the proposed hyperbolic prototype approach and logistic regression.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed hyperbolic prototype network is novel and well motivated. The usage of Busemann function to compute the distance between the ideal boundary point and the point in the Poincare ball model is practical. The empirical evaluation demonstrates the effectiveness of the proposed approach.\n\nSome questions:\n1. The derivation of the Busemann function in the Poincare ball model Eq. (7) is missing. How to obtain Eq. (7) from Eqs. (3) and (6) is not trivial. The derivation process or the corresponding reference should be given.\n2. I have confusion on the penalty term. As mentioned in Section 3.2, the penalty term is the same for all prototypes, thus it is equivalent with minimizing the Busemann function of the prototypes directly. Then why does the penalty term affect the test accuracy in Section 4.1? \n3. Another confusion is in Line 145-146. It said that the Busemann function is a decreasing function of the cosine similarity, but there is no such concept of 'cosine similarity' in the hyperbolic space.\n\nSome revision suggestions (which do not influence the overall evaluation of the paper):\n1. Although hyperbolic space has shown many improvements, it still has some limitations, the 'arbitrary data' in the abstract may be inappropriate and too absolute.\n2. In Line 167, there is an extra comma.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new approach for classification in hyperbolic space by combining ideal prototypes at the boundary of hyperbolic space and a penalized Busemann loss. The paper is well written, clear, and relevant to the NeurIPS community. All reviewers and the AC support acceptance, especially due to the novel and interesting ideas that advance research on hyperbolic neural networks and their applications, as well as the promising empirical results of the method. Reviewers highlighted also that the approach is technically sound, claims are supported in the paper, and the good empirical evaluation. Furthermore, questions and concerns related to the derivation of the approach, motivation, and significance of results where resolved after rebuttal. When preparing the camera ready version, please incorporate the overall feedback and suggestions of reviewers into the new revision.
Decision: Accept (Poster)