Hyperbolic Feature Augmentation via Distribution Estimation and Infinite Sampling on Manifolds
Authors: Zhi Gao
Yuwei Wu
Yunde Jia
Mehrtash Harandi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Hyperbolic Space
Feature Augmentation
Distribution Estimation
Neural ODE
Infinite Augmentation

Abstract: Learning in hyperbolic spaces has attracted growing attention recently, owing to their capabilities in capturing hierarchical structures of data. However, existing learning algorithms in the hyperbolic space tend to overfit when limited data is given. In this paper, we propose a hyperbolic feature augmentation method that generates diverse and discriminative features in the hyperbolic space to combat overfitting. We employ a wrapped hyperbolic normal distribution to model augmented features, and use a neural ordinary differential equation module that benefits from meta-learning to estimate the distribution. This is to reduce the bias of estimation caused by the scarcity of data. We also derive an upper bound of the augmentation loss, which enables us to train a hyperbolic model by using an infinite number of augmentations. Experiments on few-shot learning and continual learning tasks show that our method significantly improves the performance of hyperbolic algorithms in scarce data regimes.

Hyperbolic Feature Augmentation via Distribution Estimation and Infinite Sampling on Manifolds
Zhi Gao', Yuwei Wu Yunde Jia Mehrtash Harandì
Beijing Lab of Intellegent Information Technology, School of Computer Science, Beijing Institute of Technology, China
Guangdong Lab of Machine Perception and Intelligent Computing,
Shenzhen MSU-BIT University, China
3 Department of Electrical and Computer Systems Eng..
Monash University, and Data61, Australia
(gaozhi 2017, wuyuwel, jiayunde) edu. en, mehrtash. Baraadi@@onash.com edu
Abstract
Learning in hyperbolic spaces has attracted growing attention recently, owing their capabilities in capturing hierarchical structures of data. However, exist- ing learning algorithms the hyperbolic space tend to overfit when limited data is given. In this paper, we propose hyperbolic feature augmentation method that generates diverse and discriminative features in the hyperbolic space to com- bat overfitting. We employ wrapped hyperbolic normal distribution model augmented features, and use a neural ordinary differential equation module that benefits from meta-learning estimate the distribution. This is reduce the bias of estimation caused by the scarcity of data. We also derive an upper bound of the augmentation loss, which enables us train hyperbolic model by using an infinite number of augmentations. Experiments on few-shot learning and continual learning tasks show that our method significantly improves the performance of hyperbolic algorithms in scarce data regimes.
Introduction
Several recent studies reveal that real-world data, such images, videos, graphs, and sentences in natural language, usually endows hierarchical structure [ contrast to the Euclidean space with zero curvature, hyperbolic space characterized by constant negative curvature well capture such hierarchical structures, since the volume of the hyperbolic space grows exponentially with respect to the radius 2 This property leads to rich representations and encourages successful developments of hyperbolic algorithms for various applications. Examples include image retrieval 3 action recognition 4] graph classification (5) and machine translation [
Although existing hyperbolic algorithms have achieved impressive performances, they often rely on the availability of sufficient data for training. In many practical applications, very limited data is given, as collecting and labeling data costly. For example, in few-shot animal recognition [], animal images have hierarchical structures according to their species information. With few images (say less than five) in each class, training a model (e.g., a ResNet combined with hyperbolic geometry 19 3) suffers from the undesirable overfitting problem. This motivates to study combating overfitting with limited data in the hyperbolic space.
In this paper, we propose hyperbolic feature augmentation (HFA) method that overcomes overfitting features in the hyperbolic space. Compared with augmentation
by generating
"Corresponding author: Yuwei Wu
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

in the original space of data (e.g., image space). augmentation in the feature space requires less memory footprint [O TT. Moreover, feature augmentation is capable of providing useful intra-class variants by capturing diverse and discriminative augmentation directions, enabling the model to learn the invariance of each class [2 13]. Therefore, feature augmentation offers a feasible way to overcome overfitting in the hyperbolic space.
Performing feature augmentation in the hyperbolic space involves two challenges. (1) The aug- mentation directions should be discriminative and reflect the variations of intra-class in the curved hyperbolic space rather than a flat space, The scarce data and complicated hierarchical structures make producing suitable augmentation directions non-trivial. (2) Augmentation in the hyperbolic space is computationally expensive (as compared the Euclidean space), as the process of augmenta- tion requires complex hyperbolic operations (e.g., parallel transport and exponential map) to preserve the non-Fuclidean geometry of augmented features.
We introduce a meta-neural-ODDS distribution estimation scheme to solve the first challenge. We employ wrapped normal distribution for augmented features, where the augmentation directions are sampled in the tangent bundle and projected to the hyperbolic space. We estimate the distribution by solving continuous optimization process with neural ordinary differential equation (ODE), and utilize meta-learning to acquire prior knowledge in a gradient flow network for the estimation. The continuous optimization with prior knowledge leads to more reliable distribution for scarce data and complicated structures. We derive an upper bound of the augmentation loss to solve the second challenge, which enables us train a hyperbolic model using an infinite number of sampled features. As a result, our method brings enhanced diversity of data and does not need to perform data augmentation explicitly, making an efficient augmentation algorithm without complex hyperbolic operations. We evaluate the proposed HFA method on few-shot learning and continual learning tasks, and results show that the proposed method significantly improves the performance of hyperbolic algorithms when limited data is given. The code of HFA is available at https 7/gi com/ Zhi Gaomcislal Hyperbolic..... www.eeeuaeeuuuueeeeeeeeeeeeeeeee
In summary, our contributions are threefold. (1) To the best of our knowledge, we are the first to perform feature augmentation in the hyperbolic space. The proposed HFA method is capable of generating diverse and discriminative hyperbolic features, addressing the overfitting problem. (2) We introduce a meta-neural-On ODE distribution estimation scheme, whose continuous optimization process with prior knowledge leads to precise approximation of the real distribution in scarce data regimes. (3) We derive an upper bound of the augmentation loss to train a hyperbolic model using infinite augmentation, which creates an efficient augmentation algorithm without explicit augmentation and hyperbolic operations.
2 Related Work
Data augmentation. Data augmentation techniques, widely used to improve generalization, can be divided into two categories: 1. augmentation in the original space of data (e.g., image space) and 2. augmentation in the feature space. Hand designed transformations (e.g., masking and cropping) are popular examples from the first category 14 15
Automatic augmentation is a promising technique to automatically choose hand-crafted transformations [16] 17 To produce more intra-class variants, some methods train generative model for augmentation in the original space [8] 19]. One drawback of generating augmentation in the original space is the computational load of the algorithms. In contrast, feature augmentation not only brings rich intra-class variants but also involves less computational complexity and memory footprint. The benefits of feature augmentation are demonstrated for many applications, including but not limited to contrastive learning [20). domain adaptation [21], few-shot learning (22), long-tailed recognition [13. and instance segmentation (I]. Feature augmentation is usually achieved by manifold mixup (23 24.. sampling from captured distributions [25] 12 10, M. utilizing GAN model 26 27 Different from existing feature
augmentation, we focus on augmentation in the hyperbolic space.In In doing so, we introduce a meta- neural-ODS distribution estimation scheme, instead of directly computing distribution from given data. In this case, our method can better adapt to complex distributions in scarce data regimes.
Hyperbolic geometry. Learning in the hyperbolic space has become an alternative to the Euclidean space, since natural data usually exhibits hierarchical structures. Existing efforts of exploiting the hyperbolic space can be roughly divided into two categories: learning hyperbolic embeddings and

---

designing hyperbolic neural networks. Methods of the first category add several hyperbolic operations on the top of conventional neural network obtain discriminative hyperbolic embeddings. This
scheme has been used in many applications, such as image retrieval [ few-shot learning [28].
metric learning [29], and image segmentation [3]. The second category focuses on extending the
entire architecture of neural network to the hyperbolic space, such as hyperbolic convolutional
network [6]. hyperbolic graph network [31 [5) hyperbolic attention network [32], and hyperbolic
variational autoencoder [33 (4). These methods mainly focus the model level of hyperbolic
algorithms (i.e., designing/improving hyperbolic models). In contrast, our method studies the
data level of hyperbolic algorithms and efficiently improves the generalization of models via data augmentation.
Preliminaries
A hyperbolic space is smooth Riemannian manifold with constant negative curvature [35]. Hy- perbolic space has five isometric models, and we use the Poincaré ball model [36] to work with. A Poincaré ball is defined as M = { E R -c||=||| 1}. where d is the dimension of vectors in the space, || is the Euclidean norm, and the curvature < 0 represents the deviation of M from a flat space. For u E MMde,, its tangent space, denoted by TuMilo, contains all tangent vectors to M at u. We will make of the following operations in work;
Addition. For a, y E Md.e their addition is
(1 2c(x, y) c||B|| (1 c||x=|13) 20(x;, y) +
wdD.y
(1)
1/
Distance measure. For two vectors æ, y E Madi their distance is
| - yl? 2c (1+ [| +cll))))))
(x, = Vid cosh -1
(1-20-
(2)
Exponential map. The exponential map expmm (8) projects vector from the tangent space To MaMdd to the manifold Md.e
(tanhi( expm = x.e (tanh( v 1|||=|1
2
Vc[|ss|
where X = 2/(1 + c||=/|
(2) is the conformal factor. The exponential map in the tangent space
To.Mdic at the origin simplified to expm6 (s) = tanh( tanh(V/c||s|])) Vidlol|
Logarithmic map. The logarithmic map ( maps vector y from the manifold to the tangent space T2 Md.c
2 tanh -1 (VTell x0.yy)l))))))..)) B..
login
vlclAg
x
The exponential map logmn in Mo simplified to (ogm6(y) = tanh -1 '(Vcl|l
y
vl
Parallel transport The parallel transport operation PT: p Ta My moves vector from the tangent space T. to another one My.Md. along the geodesic,
X X
PT
(
gyr kyrly. y
(5)
where gyrly, -
-ar (. (2 (1())))))))) This operation in ToMde is PTC ())
Method
4.1 Formulation
In this paper, we propose the hyperbolic feature augmentation (HFA) method. We employ a wrapped normal distribution P(c, (.c.p.u p. E) to model features in scarce data regimes, and meta-learn neural

---

Distributions
Classifier with Infinite Augmentation
Input data
Hyperbolic Features
Neural
Σ D3
GFN F
ODE Salver
Pl
Σ
Eg
Feature Extractor
GFN P
ODE Salver
P1
Ple, Mai Ma!
GFN F%
GDD Solver
Feature of training data
Augmented
Plc, Enl
Figure 1: A conceptual diagram of HFA. GFN denotes the proposed gradient flow network.
ODE estimate distribution for each class. Then, we derive an upper bound E of the loss function with infinite augmentation to train the classifier. An illustration of main components of HFA provided in Figure I
4.2 Wrapped Normal Distribution in Scarce Data Regime
The wrapped normal distribution has been successfully used in various studies to model distributions in Riemannian manifolds (37 38 39]. The underlying idea is model a Riemannian distribution as projection of a normal distribution on a tangent space to the manifold. Here, we introduce it to scarce data regimes. The density of wrapped normal distribution given by
(pl 2)
P( E) -N (AAglogm Σ)
(6)
sinh( / Ic (p. 2))
where e is the curvature of the underlying space, p E Mdde is the prototype of given data, and μ and Σ E Raxd are the mean and the covariance matrix of augmentation directions, respectively. Since few data is given in scarce data regimes, the prototype p may biased.
Generating feature from P(c, ,t, Σ) in- cludes three steps. (1) Sample a vector from the normal distribution (n, E) in the tangent space ToMe.. (2) Transport 6 from MoMe. to v the tangent space Tp.M..d, = PTo (7). (3) Map to the manifold by expm (v). In the step (1), sampling from (M, Σ) is not differentiable process, and we can back propagate gradients to μ and Σ. We use reparameterization trick to solve this issue. We sample vector from the standard normal distribution (0 I) and trans- form to e by è The sampling process is summarized in Algo- rithm I
Algorithm 1 Sampling process in HFA.
Input: A wrapped normal distribution P(c, E),
where is decomposed by LL =.
Output: z MMdec sampled from
I: Sample e from a standard normal distribution ((0,)) T).
2: Use the reparameterization trick obtain ē μ +
3: Transport ē from T.d to TpMMad by PTo
Le, where LL" E.
4: Map to the manifold by expm),
We apply one wrapped normal distribution to each class. For the distribution P(c, Pj. Mj. Σ,) of the j-th class, its has its own prototype Pj. mean faj. and covariance matrix Ej. All classes share common curvature c.
4.3 Distribution Estimation by Neural ODE
Neural ODE, For the wrap distribution P(c, Pj. Mí Ej), the prototype p, can be computed by averaging given samples (e.g., the Einstein midpoint (40). We neural ODE estimate c, 14,-
and Lj. by viewing an iteration process as an Euler discretization of an ODE 41 42 43). E, is
computed by Σ₁ LJ Concretely, denote the distribution parameter (it can be Mlj or

---

L2) and the estimation of & can be cast as an iterative optimization process using given data (e.g., maximizing the likelihood), ed+1 = 5² + Vg', where Vg' is the gradient at step t. We regard the iterative optimization process as a continues-time process of an ODE, and propose a gradient flow network F to estimate the gradient flow 4s' at time 4, V dr F(g", l). Thus, given the initial or
value 5 and F, the distribution parameter is obtained by solving the ODE at the last time T with an integral term: 5" 5 + f= tt),t).
t)dt.
Here, we utilize the Runge- method [44) denoted by ODESolvee-) to solve the integral term,
(7) three gradient flow networks Fi. F2, and F3 to produce the gradient flow for c, My and Lj. respectively. 5 for 1e, and Lj is obtained by computing the mean of given data and subtracting the mean from given data, and 5" for e is set as a fixed value.
<" =DEoSSolveef" F,T
In implementation,
Gradient flow network. This network takes the distribution param- eter 5 and features of a set of samples as inputs, and generates the We first build es- timation representation e, for each class, where S and the mean T, of features belonging to the j-th class are concatenated as e, = 1st Zz}.
def
gradient de via three layers, shown in Figure 2
d
Output fully-connected layer fo
{e'Hl
We use fully-connected layer fe to embed ejj
multi head attention fe
Embedding
e/ = fe(e))
(8)
{e'Hl
Then, we use self attennttention layer ft to perform interaction among different classes,
Interaction fully connected layer
= (fe)
(9)
where the query, key, and value of the self-attention layer are all {e)} of classes, with classes in total. Finally, we use a fully-
{e,}
xz
connected layer fo to create the gradient,
Figure 2: Architecture the
de' dt = fo((())).
(10) gradient flow network.
Note that, we employ a common curvature e for all classes using Fi, instead of producing class- specific curvatures and unifying them. We send all data of the n classes together to the gradient flow network Fi, concatenate the representation of all classes {em) 1z=1, and use the output fully-connecter layer fo to produce the gradient flow for the common curvature c.
4.4 Learning with Infinite Data Augmentation
We use a distance-base based classifier in the hyperbolic space, and feature is classified by assigning to the closest weight of classes. We represent the weight of the j-th class wy MMds, and the classifier is denoted by W = [Wi... .. ww... The probability that a hyperbolic feature æ belongs to the j-th class is
exp ( (-d" dx, wi))
10 j|æ) Σx= exp - (x)))
(11)
where T is the prediction of the classifier.
We sample features from the estimated distribution to train the classifier. We denote the augmented M4; Σj) (a total of m samples are generated per class). The classifier is trained by minimizing the following cross-entropy loss function,
features for the j-th class by {z,,) where Zj.i ~ Pl(c, P...P......
exp (- d' dz (2),,,;;)
((W) =
log
(12)
Σ exp d (Z2z,,, w.y)
large value of m brings more diversity to the model, promising the model to learn the invariance of each class. Considering m grows infinity, the loss function is defined by the expectation of the cross-entropy,
1
72
exp ( - 1-d'(zj. w)))
Loo
IE:
log Σラ= exp d' (2 w)
(13)
5

---

Computing Eq. 13) is difficult and the computational load of if MCMC methods are used for large value of m. Alternatiely, we derive an upper bound L (W) of Lee (W), which is easy to compute, as shown in Proposition [ In this case, we train the classifier by minimizing the upper bound 2o (W), rather than the original loss function Loe (W).
Proposition 1. Given infinite data 3 ~ P(c, Py+ H. Ej), an upper bound of the loss function Loo in Eq. 133 is given by
I (W) Σ p=1
exp (w) (P; + 1k))) log 2ي' exp (w) (P, + 14) (w) w w/ w,) 3 [X w,))
(14)
where P,- expmn (P,), and D. can be directly computed in the tangent space at the origin.
Proof. We rewrite the loss function of the infinite data augmentation in Eq. 13 as
(-d 2)3)
1
n1
exp
Lox (W)-
Bi log
n1
Ey²=1 exp (- -d (z wyy)
j=1
exp (d (z ()))) - d' dz 20y)
E;
log
/=1
11
log
Σ
E, [exp (z w)) dz dz wyy)
(15a)
12
Σ log
Σ
Eg [exp (1|23,2 - wjll? - || - w 1311
(15b)
17 j=1
3'=1
log
E (exp (2(wg) - w,)² 2z
" + (w.jwy w w
1=1
log
Σ y'=1
E, exp ((w - T - Ō,,²) + (wwy w/ w w,))
(15c)
log
Σ exp
(mm - ww,) (f., + 11,) + (w... - w/ w.) + 1 j(0, - w [, - wo,)
(15d)
exp exp (w,
+ (1,)))
log
11
exp (w + M1y) + (w.wwy - w w,) + (wp w,)
- w,,)
-Co (W).
Eq. 15a) follows from the Jensen's inequality B(log(æ:)] < log E E(x.. For Eq. 15b), we define function f = d' (m y)
x y and compute the derivative of f. We find f is an increasing function
when ||= yll < 1+| Vel In this case, if we have || 10,11² < || my V++ld- vic
Vid
Vid
and | wylll - ز|= wy 1 d' (= wx) d' w.rr) Eq. (5c) is obtained by replac- ing Zj.t with the augmentation process using the parallel transport and exponential map, i.e.
can derive
d' d' wz2 - || w d' w.)) || w
z expme, (PT o-,, ()),)), and adding scale constraints on c, |1@ and |1 Eq. (15d) is
obtained by using the moment-genenn gennent-generatig function: E[[ep[==)] exp(μ + Σ). x ~ N(n, Σ). In our derivation, (wy- w T())) (P, BB..) + { wy w. un;) is Gaussian vector.
Detailed derivation and implementation can be found the supplementary materials. From this derivation, we replace non-Fuclidean loss function with a Euclidean upper bound by adding suitable constraints. The upper bound is easy to compute without explicit augmentation and time-consumption hyperbolic operations, and our experimental results show that it achieves good performance (sec

---

Section 5.. This provides an interesting research direction and feasible way for Riemannian algorithms whose loss functions are difficult or high-cost to compute.
The classifier W is updated by the Riemannian gradient descent algorithm 45 minimizing Cx (W). After several iterations, we obtain the updated classifier W [wj w
in the process of
and test data is classified by Eq.
4.5 Training
The goal of this work is to learn the gradient flow networks Fi, F2, and F3. doing so, we train them in meta-learning framework that simulates the practical setting by using some base data D. In the training stage, we randomly select few data from D as the training data Dr, and the rest is used the validation data D.. F1. F2, and F3 updated via bi-level optimization manner. In the inner-loop, we estimate the distribution of De by using F1, F2,, and Fj, iteratively train the classifier by minimizing the loss function L and obtain the updated classifier W [ui] ... w We update F1, F2. and F3 in the outer-loop by minimizing the following objective that is the loss of W on the validation data D..
n
exp ( (-d dx, (2,,,,,) w1))
min L(W*) =R Fr,
Σ
1 y log
1t==1 exp dx, w.zz))
(16)
s.t. W arg min I (W) W
where y== is the indicator function, meaning that the label y of a is equal to j, 1 1y=j = 1, and 0 otherwise. The pseudo code of training is summarized in Algorithm
In the inference stage, given a task with limited data, our method first estimates the distribution for each class by using the learned Fi, F2, and Fs, and then trains the model by using Loc.
Algorithm 2 Training process of HFA.
Input: Base data 2>.
Output: Updated gradient flow networks FL, F2, Fg. I: while Not converged do
2: Randomly select few training data D. from D, and the rest of D is used as validation data D...
Experiments
3: Estimate the distribution for each class in D, by using Fi. F2, and F3s via Eq. 7 while Not converged do Train the classifier W using the estimated dis- tribution via the loss function Eq. (14). end while
We evaluate the proposed method HFA few- shot learning and continual learning tasks. In few-shot learning, data augmentation is natu- ral idea to overcome the deficiency of training
4:
6:
data 25 26 In continual learning, reply-based methods have shown effectiveness in continual learning by storing data [46 47 yet they cannot store much data due to the memory limitation. We HFA to perform augmentation for stored limited data to solve this issue. We use common backbone networks with exponential map as the feature extractor in the tasks. More experiments (e.g., evaluation on graph data, visualization, and ablation) and experimental details can be found in the supplementary materials.
7:
Compute the loss of the updated classifier W using the validation data D.., and update Fi, F2, and F's via minimizing Eq. (16).
8: end while
5.1 Few-shot Learning
We conducted experiments on four few-shot learning datasets: mini-ImageNet [48). tiered- ImageNet [4)] CUB [7], and CIFAR-FS [50] datasets. The four datasets have hyperbolic structures, and are commonly used evaluate hyperbolic algorithms 29 28
We use ResNet12 and
ResNet18 as the backbone networks, and perform augmentation for the support data.
5.1.1
Main Results
We compare HFA with augmentation--aase few-shot learning methods 21 25 26
55
on the mini-ImageNNet, CUB, and CIFAR-FS datasets. These methods all augment data in Fuclidean space, while HFA performs data augmentation the hyperbolic space. Our method brings improvements them, shown in Table I In particular, VFSL [54] and V1-Net (25] study data
27
7

---

Table Accuracy (%) comparisons with Euclidean augmentation based few-shot learning methods on the
CUB, and CIFAR-FS datasets.
mini-Imagehe ImageNet
CUB
CIFAR-FS
Method
Backhone
I-shot
5-shot
I-shot
VFSL B4
ResNet12
61.23
0.26
77.09 0.17
DIN 55
ResNet12
63.45 0.86
77.91 = 0.62
72.00
85.10
71.50
82, S0
Ours
ResNet12
66.87 0.44
82.08 = 0.31
76, 75 + 0.43
89.19 = 0 0.26
72 52 0.16
85.33 +.333++00 0.33
Della-cncdder
S9
VGG-16
59.9
69.7
69.80 = 0.46
82,60 0.35
66.70
79.80
SalNet [55]
ReaNett01
02.22 0.87
77.93 = 0.63
Dual TriNet 22
ResNet18
58. 18.12=22 1.37
76.92 = 0.69
69.61
81.10
63.41 0.64
78.43 0.61
DDeMc-Net
ResNet18
59.14 0.886 0.86
74.63 0.74
VI-Net
ResNet18
61
74.76
86.84
AFHN
26
ResNet18
02.38 + 0.72
78.16 0.56
70.53 + 1.01
83 96 0.63
08
0.93
81.45 + 0.87
TFH
ResNet18
65.07 0.82
80.81
75.76 0.83
0.47
74.77
0.90
86.88 0.59
HFA (Ours)
ResNet18
68.26 0.46
83.53 1 0.28
77.25 1 0.45
90.77 0.26
75.08 0.47
87.24 0.33
Table 2: Accuracy (%) comparisons with popular few-shot learning methods on the mini-ImageNet and tiered- Meree-et datasets. 'Optim', 'Metric', and 'Aug' denote the optimization-based metric- based, and augmentation-be based few-shot learning methods, respectively. 'Euclidean' and 'Hyperbolic' denote the algorithm is in Euclidean space and hyperbolic space, respectively. means that Hyper-
Proto uses ResNet18 8 as the backbone, while the others use ResNet12.
mini- NetgeNe
tiered dered-Imaged ImageNet
Method
Category
shoo
5-ohot
shot
5-shot
Fuclidean Optim
51.03 0.50
68.26
58.58
0.49
71.24
0.43
L2F
[57
Euclidean Optim
57.48 = 0.49
74,68 = 0.43
63.91 = 0.81
77.61 0 0.11
ALFA 55
Fuclidean Optim
60.06 + 0.49
77.42 + 0.42
64.43
0.49
81.77 + 0.39
McTAL
59
Euclideun Optim
59.64 0.38
76.20
0.19
63.89
= 0.43
80.14 0.40
CurAMI
Hyperholic Optim
63.13 + 0.41
81.04
+ 0.39
0.56 68.46
83.84 + 0.40
ProtoNet
60
Euclidean Metrie
56.52 2: 0.45
74.28
0.20
53.51
0.89
72.69 : 0.74
MetaOptNet
61
Euclidean Metric
62.61 0.61
78.63
65.99
0.72
81.56 + 0.53
DSN (
Euclidean Metrie
78.83 0.45
06.22
0.75
82.79 = 0.48
FEAT g
Euclidean Metric
66.78 0.20
82.05 0.14
0.23 70.80
84.79 = 0.16
HyperProto"
Hyperbolic Metric
59.47 - 0.20
76.81 0.14
w/o Aug
Hyperbolic-
64.75 2: 01.35
79.84 0.20
68.57 : 0.49
82.22 : 0.39
Aug
Hyperholic Aug
65.12
70.22 0.50
84.51 0.37
Neural ODE Aug 5 samples
Hyperbolic Aug
65.54 +: 0.46
80.96 = 0.31
69.32
0.53
63,21
+: 0.35
HFA (Ours) (Neural ODE Inf Aug)
Hyperholic Aug
66.87 : 0.44
82.08 :: 0.31
71.62 0.49
85.47 0.35
augmentation in the Fuclidean feature space, while our method has better performance. For example, on the mini-ImageNet dataset using ResNet12, our method improves VFSL by 5.64% and 4.39% in the 1-shot and 5-shot tasks, respectively. The main reason that augmentation in the hyperbolic space preserves the hierarchical structures of data and avoids the undesirable data distortion. Results on Resnet18 8 indicate the superiority of our method again, demonstrating the significance of data augmentation in the hyperbolic space.
We then compare HFA with popular optimization-bart and metric- few-shot learning methods on the mini ImageNet and ticred- ImageNet datasets. Results are shown in Table 2 L2F [57]. ALFA [58], MeTAL [59], and CurAML 9 are state-of-the-a optimization-base to them, the adaptation to new tasks of HFA is also achieved by optimization process. Our method improves them by 3% on the 1-shot task and 1% - 2% on the 5-shot task. Compared with besric methods, including HyperProto [3]. MetaOptNet [61), DSN (62). and FEAT [8), Our method performs competitively or even exceeds some methods. Note that CurAML and HyperProto both few-shot learning methods in the hyperbolic space, and our better performance suggests that our data augmentation can actually improve the model in scarce data regimes.
methods. Similar
5.1.2 Ablation
We conduct ablation experiments show effectiveness of our neural ODE and the derived upper bound of the augmentation loss. We first denote 'not performing any augmentation' as "w/o aug', where we directly use the given data train the classifier. Then, we perform infinite augmentation. where the curvature is tuned manually, and the mean and covariance matrix are directly computed from the given data, denoted by 'Inf Aug". Next, we use the neural ODE to estimate the distribution, and generate 5 samples for each class, denoted by 'Neural ODE + Aug 5 samples'. Finally, based on the estimated distribution, we perform infinite augmentation, denoted by 'Neural ODE Inf Aug'. Results are shown in Table 2 We find that 'Inf Aug' has better performance than "w/o aug'.
8

---

eftaning data
louan uniyyde mambe
Augmented data
20 30
20 augmentaation
Linnet
dugong
decsion boundary ofter
(a) Feature distribution.
(h) Corresponding images of generated features.
Figure 3: Visualization of augmentation data.
CIFAR-100 (5 phases)
CIFAR-100 (10 phases)
MAS
80
LWF-MC
LwW
Accuracy (96)
MUC
50
(CaRL-CNN
40
(CARL-NME
50
30
EEIL
LUCIR
40
20
H w/o aug
10
5
9
10
11
Number tasks
Number of tasks
(a) 5-phases
(b) 10-phases
Figure Top-1 accuracy on CIFAR 100 continual learning.
showing that the infinite augmentation improves the generalization ability in the hyperbolic space. Using neural ODE further improves the performance on the two datasets. The reason is that directly computing the mean and covariance matrix from limited data results in imprecise distributions. In contrast, our neural ODE makes a better approximation to the real distribution.
5.1.3 Visualization
We first plot the estimated distribution in mini-ImageNet by sampling augmented features, where the MDS and t-SNE dimensionality reduction methods are used. Then, we use a reverse mapping algorithm [2] to search images corresponding to the augmented features for intuitive visualization. Results are shown in Figure 3 In the left panel, when training data is biased, our method well approximates the real distribution, building a robust decision boundary for classification. the right panel, generated data has much diversities (e.g., background and object posture) while preserving the class-identity.
5.2 Continual Learning
We use the CIFAR-101 dataset [50] with 100 classes. We evaluate HFA on two settings: 50 + x 10 and 50 + 10 x For example, 50 10 means that the first task contains 50 classes, and there are 10 following tasks with each one having 5 classes. We use ResNet18 as the backbone, and the model updated by a cross-entropy loss on current data, an infinite augmentation loss on stored data, and distillation loss. HFA stores 20 samples for each class, which is the same as replay-based methods 663
65] Comparisons with MAS [66], LwF-MC [63] LwM [67], MUC [8]. IL2A (69, iCaRL 63 EEIL
64
64 and LUCIR [65] are shown in Figure 4 Meanwhile, an ablation experiment
where augmentation is not performed denoted by 'H w/o aug
Our method has better performance than replay based methods [63] 64| 665.. The reason is that our method is capable of estimating the stored data,
distribution from few data, thus leads to diverse and discriminative augmentation making the model more reliable. In addition, our method performs better than the augmentation-basee method II.A (69]. This suggests that modeling data in the hyperbolic space more discriminative, and performing augmentation in the hyperbolic space can further leads robust model.

---

6 Conclusions
In this paper, we have presented hyperbolic feature augmentation (HFA) method that generates diverse and discriminative features to overcome overfitting in the hyperbolic space. The introduced veeta-neuallODE distribution estimation scheme can precisely approximate the real distribution in scarce data regimes. The derived upper bound of the augmentation loss makes an efficient augmentation algorithm in the hyperbolic space, which not only leads much diversity of data but also reduces much computational load. Experiments in few-shot learning and continual learning tasks show that our method can improve the generalization of hyperbolic algorithms searce data regimes. In this work, we assume that data has a uniform hierarchical structure and use single hyperbolic space for each task. Actually, wal-world data may have complex hierarchical structures with varying local structures. In the future, we will study data augmentation in product manifolds to tackle such complex data.
This work was supported by the Natural Science Foundation of China (NSFC) under Grants No. 62172041 and No. 62176021.
Acknowledgements.
References
[1] Wei Peng. Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, and Guoying Zhao. Hyper- bolic deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (T- (T-PAMI), 2021. 100.1000//AAMM.221..113
[2] Pengfei Fang, Mehrtash Harandi, and Lars Petersson. Kernel methods in hyperbolic spaces. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 10665-10674, 2021.
[3] Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan V. Oseledets, and Victor S. Lempitsky. Hyperbolic image embeddings. In IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 6417-6427, 2020.
[4] Teng Long, Pascal Mettes, Heng Tao Shen, and Cees G. M. Snoek. Searching for actions on the hyperbole. In TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1138-1147, 2020..
2020.
[5] Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A hyyperboli-to--yyerbooic graph convolutional network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 154-163, 2021.
[6] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. In International Conference on Learning Representations (ICLR), 2021.
[7] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
[8] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8808-8817,222 2020.
[9] Zhi Gao, Yuwei Wu, Mehrtash T Harandi, and Yunde Jia. Curvature-adaptive meta-learning for fast adaptation to manifold data. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-MA 2022. doi: 2022
[10] Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M. Hospedales. A simple feature augmentation for domain generalization. In JEEE/CVF International Conference on Computer Vision (ICCV), pages 8866-8875 2021.
(11) Yuhang Zang, Chen Huang, and Chen Change Loy. FASA: feature augmentation and sampling adaptation for long-tailed instance segmentation. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 3437-3446, 2021.
[12] Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmentation for deep networks. Neural Information Processing Systems (NeurIPS), pages 22614-12623, 2019.
10

---

[13] Shuang Li, Kaixiong Gong, Chi Harold Liu, Yulin Wang. Feng Qiao, and Xinjing Cheng. Metasaug; Meta semantic augmentation for long-tailes visual recognition. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5212-5221, 2021.
[14] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2015.
[15] Chengyue Gong, Dilin Wang, Meng Li, Vikas Chandra, and Qiang Liu. Keepaugment: A simple information-ppeservee data augmentation approach. /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1055-1064, 2021.
[16] Denis A. Gudovskiy, L. Rigazio, Shun Ishizaka, Kazuki Kozuka, and Sotaro Tsukizawa. Autodo: Robust autoaugment for biased data with label noise via scalable probabilistic implicit differen- tiation. IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 16596-16605, 2021.
[17] Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim. Fast autoaugment. In Neural Information Processing Systems (NeurIPS), pages 6662-6672, 2019.
[18] Liming Jiang, Bo Dai, Wayne Wu, and Chen Change Loy. Deceive D: Adaptive Pscudo Augmentation for GAN training with limited data. In Neural Information Processing Systems (NeurIPS), pages 21655-21667, 2021.
[19] Liming Jiang. Bo Dai, Wayne Wu, and Chen Change Loy. Deceive d: Adaptive pseudo augmentation for gan training with limited data. In Neural Information Processing Systems (NeurIPS), pages 21655-21667 2021.
[20] Qi Cai, Yu Wang. Yingwei Pan. Ting Yao, and Tao Mei. Joint contrastive learning with infinite possibilities. In Neural Information Processing Systems (NeurIPS), pages 12638-12648 2020.
[21] Shuang Li, Mixue Xie, Kaixiong Gong, Chi Harold Liu, Yulin Wang, and Wei Li. Transferable semantic augmentation for domain adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11516-11522 2021.
[22] Zitian Chen, Yanwei Fu, Yinda Zhang, Yu-Gang Jiang, Xiangyang Xue, and Leonid Sigal. Multi-level semantic feature augmentation for one-shot learning. IEEE Transactions on Image Processing (T-IP), 28944449444 2019.
[23] Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, loannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International Conference on Machine Learning (ICML), pages 6438-6447, 2019.
[24] Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, and Kilian Q. Weinberger. On feature normalization and data augmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12378-12387, 12387, 2021.
[25] Qinxuan Luo, Lingfeng Wang, Jingguo L.v, Shiming Xiang, and Chunhong Pan. Few-shot learning via feature hallucination with variational inference. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3962-3971, 2021.
[26] Kai Li, Yulun Zhang, Kunpeng Li, and Yun Fu. Adversarial feature hallucination networks for few-shot learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13467-13476, 2020.
[27] Michalis Lazarou, Yannis Avrithis, and Tania Stathaki. Tensor feature hallucination for few-shot learning. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 2050-2060, 2022.
[28] Zhi Gao, Yuwei Wu, Yunde Jia, and Mehrtash Harandi. Curvature generation in curved spaces for few-shot learning. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 8671-8680, 2021.
[29] Jiexi Yan, Lei Luo, Cheng Deng, and Heng Huang. Unsupervised hyperbolic metric learning. In JEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12465- 12474, 2021.
11

---

[30] Mina Ghadimi Atigh, Julian Schoep. Erman Acar, Nanne van Noord, and Pascal Mettes. Hyperbolic image segmentation. In /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4443-4452 2022.
[31] Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks. In Neural Information Processing Systems (NeurIPS), pages 8230 8241, 2019.
[32] Çaglar Gülçehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter W. Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. Hyperbolic attention networks. In International Conference on Learning Representations (ICLR), 2019.
[33] Ondrej Skopek, Octavian-Fugen Ganea, and Gary Bécigneul. Mixed-curvature variational autoencoders. In International Conference on Learning Representations (ICLR), 2020.
[34] Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, and Yee Whye Teh. Continuous hierarchical representations with poincaré variational auto-encoders. In Neural Information Processing Systems (NeurIPS), pages 12544-12555, 2019.
[35] John M Lee. Riemannian manifolds: introduction curvature, volume 176. Springer Science & Business Media, 2006.
[36] James W Cannon, William Floyd, Richard Kenyon, Walter R Parry, et al. Hyperbolic geometry. Flavors of geometry, 31(59-115))) 1997.
[37] Yoshihiro Nagano, Shoichiro Yamaguchi, Yasuhiro Fujita, and Masanori Koyama. A wrapped normal distribution on hyperbolic space for gradient- gradient-based learning. International Conference on Machine Learning (ICML), pages 4693-4702, 2019.
[38] Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, and Yee Whye Teh. Continuous hierarchical representations with poincaré variational auto-encoders. In Neural Information Processing Systems (NeurIPS), pages 12544-12555, 2019.
[39] Jianxin Li, Xingcheng Fu, Qingyun Sun, Cheng Ji, Jiajun Tan, Jia Wu, and Hao Peng. Curvature graph generative adversarial networks. In ACM Web Conference (WWW), pages 1528 1537, 2022.
[40] Abraham Albert Ungar. Analytic hyperbolic geometry: Mathematical foundations and applica- tions. 2005.
[41] Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Neural Information Processing Systems (NeurIPS), pages 6572-6583, 2018.
[42] Baoquan Zhang, Xutao Li, Yunming Ye, Shanshan Feng, and Rui Ye. Metanode; Prototype optimization as a neural ode for few-shot learning. arXiv preprint 1rXiv:2103.14341, 2021.
[43] Chengming Xu, Yanwei Fu, Chen Liu, Chengjie Wang, Jilin Li, Feiyue Huang, Li Zhang, and Xiangyang Xue. Learning dynamic alignment via meta-filter for few-shot learning. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5182-5191, 2021.
[44] Ernst Hairer, Syvert P. Nørsett, and Gerhard Wanner. Solving ordinary differential equations i: Nonstiff problems. SIAM Rev,, 32(3):485-486, 1990.
[45] Pierre-Antoine Absil, Robert E. Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds. 2007.
[46] Francisco Manuel Castro, Manuel J. Alahari Karteek. End-to-end incremental learning. In European Conference on Computer Vision (ECCV), pages 241-257, 2018.
Nicolás Guil Mata, Cordelia Schmid, and
[47] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In IEEE/VVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 831, 2019.
12

---

[48] Oriol Vinyals, Charles Blundell, Tim Lillicrap. Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning, In Neural Information Processing Systems (NeurIPS), pages 3630-3638, 2016.
[49] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenen- baum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. International Conference on Learning Representations (ICIRR, 2018.
[50] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[51] Zitian Chen, Yanwei Fu, Yu-Xiong Wang, Lin Ma, Wei Liu, and Martial Hebert. Image deformation meta-networks for one-shot learning. In IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 8680-8689, 2019.
[52] Eli Schwartz, Leonid Karlinsky, Joseph Shtok. Sivan Harary, Mattias Marder, Abhishek Kumar, Rogério Schmidr Feris, Raja Giryes, and Alexander M. Bronstein. Delta-encoder: an effective sample synthesis method for few-shot object recognition. In Neural Information Processing 2018.
Systems (NeurIPS), pages
[53] Hongguang Zhang. Jing Zhang, and Piotr Koniusz. Few-shot learning via saliency- guieey hallucination of samples. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17770-7779. 2019.
[54] Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang. Variational few-shot learning. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 1685-1694, 2019.
[55] Mengting Chen, Yuxin Fang. Xinggang Wang, Heng Luo, Yifeng Geng, Xinyu Zhang, Chang Huang, Wenyu Liu, and Bo Wang. Diversity transfer network for few-shot learning. In AAAI Conference on Artificial Intelligence (AAAI), pages 10559-10566, 2020.
[56] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International Conference on Machine Learning (ICML), pages 1126-1135, 2017.
[57] Sungyong Baik, Seokil Hong, and Kyoung Mu Lee. Learning to forget for meta-learning, In IEEC/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 2379-2387, 2020.
[58] Sungyong Baik, Myungsub Choi, Janghoon Choi, Hee won Kim, and Kyoung Mu Lee. Meta- learning with adaptive hyperparameters. In Neural Information Processing Systems (NeurIPS), pages 20755-20766 2020.
[59] Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaesik Min, and Kyoung Mu Lee. Meta-learning with task-adaptive loss function for few-shot learning. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 9465-9474, 2021.
[60] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Neural Information Processing Systems (NeurIPS), pages 4077-4087, 2017.
[61] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 10657-10665, 2019.
[62] Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi. Adaptive subspaces for few-shot learning. In JEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 4136-4145, 2020.
[63] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5533-5542, 2017.
13

---

[64] Francisco M. Castro, Manuel J. Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In European Conference on Computer Vision (ECCV). pages 241-257, 2018.
[65] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In JEEE/CVF Conference on Computer Vision and Pultern Recognition (CVPR), pages 83 831-839, 2019.
[66] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) forget. In European Conference on Computer Vision (ECCV), pages 144-161, 2018.
[67] Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learn- ing without memorizing. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5133-5141, 2019.
[68] Yu Liu, Sarah Parisot, Gregory G. Slabaugh, Xu Jia, Ale Leonardis, and Tinne Tuytelaars. More classifiers, less forgetting: generic multi-classifier paradigm for incremental learning. In European Conference on Computer Vision (ECCV), pages 699-716, 2020.
[69] Fei Zhu, Zhen Cheng, Xu-yao Zhang, and Cheng-lin Liu. Class-incremental learning via dual augmentation. In Neurul Information Processing Systems (NeurIPS), pages 14306-14318, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See supplementary materials.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See experiments of few-shot learning.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See supplementary materials.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 5.1 and 5.2.
(b) Did you mention the license of the assets? [No] The used datasets are publicly available. and widely used in existing sho-shot learning and continual learning methods. Thus, do not mention the license of the dataset.
(c) Did you include any new assets either in the supplemental material
URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're nou're using/curating? [No] The used datasets are publicly available.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No] The used datasets does not contains personally identifiable information.
14

---

5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "ooRuOV31f2x", "reviewer": "Reviewer_KDoT", "summary": "This paper introduces hyperbolic feature augmentation. This is done by way of introducing a neural-ODE distribution estimation scheme, whose continuous optimization process can approximate the real data distribution reasonably well in scarce data regimes. An upper bound of the augmentation loss is also derived; the paper makes use of this bound to train the final models without explicit augmentation. Experimental results on several few shot datasets give some evidence for the efficacy of the method.", "questions": "### Detailed Comments, Corrections and Questions\n\nOverall the writing in the paper is understandable, but has some grammatical issues.\n\nIn terms of questions, I can perhaps reiterate one of the key points from the weaknesses section here: why is the feature augmentation method focused so much on datasets that don't have hyperbolic structure, and moreover, so focused on the extreme data scarce case (few shot) as opposed to more traditional settings? Data augmentation should be helpful in general, not just at the extremes. \n\nA non exhaustive list of minor corrections is given below:\n\nL13: \"low data regimes\" -> \"scarce data regimes\". I will only highlight this correction once, but it should be applied many times throughouth the paper.\n\nL58: \"in the hyperbolic space\" -> \"in hyperbolic space\"\n\nL305: \"much computational loads\" -> \"much computational load\"", "limitations": "The paper does reasonably address limitations in the final paragraph. Namely, the paper states the assumption that the data used with their method has a uniform hierarchical structure, whereas in actuality, real-world data may have be more complex and have complex hierarchical structure with varying local structures.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n1. The paper is a novel early feature augmentation method in hyperbolic space. Feature augmentation has not been explored much outside of Euclidean space, and I believe this is an interesting area for work.\n\n2. The Table 1 results seem to give reasonable evidence for the efficacy of the introduced method. That is, under the extreme data scarcity seen in the few shot setting, the method does outperform several reasonable baselines, to a statistically significant extent.\n\n### Weaknesses\n\n1. While the method is an early work in hyperbolic feature augmentation, I am unsure as to why the paper focuses so much on few shot learning. All of the test datasets are few shot, and moreover, there is no particular reasons to assume they have an inherently hierarchical structure. This is rather puzzling. If the method is truly a good hyperbolic feature augmentation method, it should be helpful regardless of the few shot setting. I think the experiments are currently quite flawed and incomplete. At the minimum, the paper needs to demonstrate good performance on a reasonably hyperbolic dataset (e.g. the disease dataset from HGCN [b]), and at best, it should do this for several datasets with hyperbolic structure that are outside of the few shot setting.\n\nCertainly, if the margin is the best for the few shot case, so be it. But I would expect cases with actual hyperbolic structure to work better. Moreover, the few shot descriptor should not be an essential test dataset characteristic of a general feature augmentation method.\n\nUltimately, this additional evaluation will help determine if the improvement is truly due to the geometry captured by the method, or if the improvement is mostly due to some unrelated architectural modification.\n\n2. Regular Euclidean Neural ODEs are used to learn the feature densities, despite the fact that the method is clearly about hyperbolic feature augmentation. This should be augmented by using a Neural Manifold ODE [a] for hyperbolic space.\n\n3. The results in Tables 2 and 4 seem to be relatively marginal. Again this increases my suspicion that the evaluation is not being done on sufficiently hyperbolic datasets to see a real significant difference due to the captured geometry (i.e. due to the fact that the method is hyperbolic and not Euclidean). \n\n### Verdict\n\nOverall this paper has some promise as an early work attempting to do hyperbolic feature augmentation. However, lack of proper evaluation, as described above, make me question the significance of the proposed method. As a result, I recommend a borderline reject rating for this paper.\n\n### References\n\n[a] Neural Manifold ODEs. https://arxiv.org/abs/2006.10254\n\n[b] Hyperbolic GCN. https://arxiv.org/abs/1910.12933", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "V5wUN21Wr_", "reviewer": "Reviewer_FKNX", "summary": "This paper proposes a hyperbolic feature augmentation method to avoid the overfitting problem of hyperbolic learning in few data setting, the authors extend the method by using an infinite number of augmentations through an interesting upper bound analysis, promising results on few-shot learning and continual learning are given. ", "questions": "Q1: I think a projection of the sampled noise hat v into the tangent space at the origin is needed? \nQ2: can the code, also the reverse mapping algorithm for the visualization made public? \nQ3: can the method be made in an end-to-end manner for some example tasks? ", "limitations": "discussed above. ", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe motivation of the method is clear and the paper is well-organized/presented, the method is novel/straightforward to understand, the proposal to do learning with infinite data augmentation provides a rigorous and sound way to train the classifier. Experiments on few-shot learning provides promising results in this area. \n\n\nWeaknesses: \nw1. The main weakness is related to the gradient flow network, there are multiple things to model, a practical question is whether these gradient flow networks learn correctly and generalize, especially in a few data setting, it would make the paper stronger if the authors can provide any empirical evaluation of these networks to further verify.\n\nw2: The optimization of the gradient flow networks are a little bit complicated, including both the inner- and outer-loop optimization process + many hyper-parameters, the authors should provide a detailed analysis in this aspect for real usage of this method in practice. \n\nOverall, this paper is novel to me and I'm glad to see hyperbolic models in this setting, the method is clear and straightforward, though a detailed guidance of the method should be made available. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "C-ICPik5ySd", "reviewer": "Reviewer_sUbE", "summary": "This paper proposes a hyperbolic feature augmentation method to prevent overfitting when limited data is given. The authors derive an upper bound of the loss function with infinite data augmentation. Therefore, the classifiers and distribution estimators can be trained via a bi-level optimization manner without complex hyperbolic operations. They also provide extensive experimental results on few-shot learning and continual learning to demonstrate the effectiveness of their proposed method.", "questions": "see weakness and questions", "limitations": "Limitations are not given", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1.\tThis paper uses Neural ODEs to estimate the distribution of hyperbolic features and then sampling augmented features from the learned distribution.\n2.\tThis paper provides an upper bound of the augmentation loss using an infinite number of augmentations. This upper bound allows the authors to make efficient augmentation without complex computation.\n3.\tThe authors conduct extensive comparison experiments, ablation experiments and visualization experiments. The results demonstrate the effectiveness of their proposed feature augmentation method.\n\n\nWeaknesses and Questions\n1.\tFor the distribution estimation, this paper uses three Gradient flow networks to learn different parameters. According to Section 4.3, the networks learn specific parameters for each class using different inputs (i.e., $\\bar x_j$). That is to say, the network $F_1$ will output ${c_j}_{j=1}^n$ for $n$ classes. However, in Line 142, the authors point that the parameter $c$ is shared between all classes. How to unify this $c$?\n2.\tHow to update $F_2$, and $F_3$ via minimizing Eq. (16)? When the classifiers are fixed, it seems that only the network $F_1$ can be trained.\n3.\tSome experimental details are missing.\n3.1.\tWhat is the ratio of training data $D_t$ to validation data $D_v$ in the training stage.\n3.2.\tWhat is the value of initial $c$.\n4.\tIn Table 2, the metric-based baseline FEAT performs similar accuracy to the proposed method. It’s better to discuss the superiority of the method in terms of time consumption. According to Algorithm 2, the upper bound Eq. (14) simplifies the training of classifiers, but Eq. (16) is still difficult to compute.\nTypo: In Table 2, the description does not match the content, e.g., \"Euclidean Metric\" (or \"Hyperbolic Metric\") and \"Model\".\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "AC6wB_38xPf", "reviewer": "Reviewer_xaq1", "summary": "The paper considers the problem of feature augmentation, a special kind of data augmentation, in some low-data regime tasks such as few-shot learning. \nThe main idea is to consider a wrapped normal distribution for each category and use the reparametrization trick (in the tangent space of the hyperbolic manifold) to obtain a differentiable algorithm for sampling augmented data. A Neural ODE is then used to estimate the parameters of the wrap distribution. Since optimizing the problem for a large number of sampled data can be difficult, the paper proposes in Section 4.4 to upper bound the classification loss and optimize a tractable upper bound in the tangent space, which is Euclidean. \nAs explained in Section 4.5, the training process is done by exploiting the validation data in a bi-level optimization manner.\nExperimental results in the few-shot and continual learning tasks show the relevance of the approach.", "questions": "- Have you tried to learn prototypes by using the provided category hierarchy (e.g., on tiered-Imagenet)? It does not seem to be the case.", "limitations": "N/A", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The paper is clear and well written. I think that the approach is original and significant in the hyperbolic manifold learning community. In particular, it proposes different tricks to make the training algorithm tractable so that samples represented on a hyperbolic manifold are high quality (as shown in Figure 3) and improve classification performance. \n\nWeaknesses: The main \"weakness\" of the approach is that it does not quantitatively outperform baselines by a large margin (many baselines return similar performance on many datasets).\nHowever, I still think that the methodology is interesting and can influence future work in the same direction.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept