Title: Hyperbolic Space with Hierarchical Margin Boosts Fine-Grained Learning from Coarse Labels
Venue: NeurIPS 2023
Accepted: yes
Keywords: Fine-grained learning

Coarse-to-fine learning

Hyperbolic space

Hierarchical margin
Abstract: Learning fine-grained embeddings from coarse labels is a challenging task due to limited label granularity supervision, i.e., lacking the detailed distinctions required for fine-grained tasks. The task becomes even more demanding when attempting few-shot fine-grained recognition, which holds practical significance in various applications. To address these challenges, we propose a novel method that embeds visual embeddings into a hyperbolic space and enhances their discriminative ability with a hierarchical cosine margins manner. Specifically, the hyperbolic space offers distinct advantages, including the ability to capture hierarchical relationships and increased expressive power, which favors modeling fine-grained objects. Based on the hyperbolic space, we further enforce relatively large/small similarity margins between coarse/fine classes, respectively, yielding the so-called hierarchical cosine margins manner. While enforcing similarity margins in the regular Euclidean space has become popular for deep embedding learning, applying it to the hyperbolic space is non-trivial and validating the benefit for coarse-to-fine generalization is valuable. Extensive experiments conducted on five benchmark datasets showcase the effectiveness of our proposed method, yielding state-of-the-art results surpassing competing methods.

Hyperbolic Space with Hierarchical Margin Boosts Fine-Grained Learning from Coarse Labels

Abstract

Learning fine-grained embeddings from coarse labels is a challenging task due to limited label granularity supervision, i.e. , lacking the detailed distinctions required for fine-grained tasks. The task becomes even more demanding when attempting few-shot fine-grained recognition, which holds practical significance in various applications. To address these challenges, we propose a novel method that embeds visual embeddings into a hyperbolic space and enhances their discriminative ability with a hierarchical cosine margins manner. Specifically, the hyperbolic space offers distinct advantages, including the ability to capture hierarchical relationships and increased expressive power, which favors modeling fine-grained objects. Based on the hyperbolic space, we further enforce relatively large/small similarity margins between coarse/fine classes, respectively, yielding the so-called hierarchical cosine margins manner. While enforcing similarity margins in the regular Euclidean space has become popular for deep embedding learning, applying it to the hyperbolic space is non-trivial and validating the benefit for coarse-to-fine generalization is valuable. Extensive experiments conducted on five benchmark datasets showcase the effectiveness of our proposed method, yielding state-of-the-art results surpassing competing methods.

1 Introduction

In deep learning, rich supervised information is crucial for the generalization ability of deep models. Fine-grained visual recognition [36] is a fundamental problem in computer vision, which requires accurately identifying subordinate (fine-grained) categories within the same meta (coarse-grained) category. However, in many domains, annotating fine-grained data requires the involvement of domain experts, such as in pipeline failure detection [1], biodiversity recognition [30] and medicine analyses [25], leading to high annotation costs or even infeasibility. In contrast, obtaining the coarse labels, e.g. , the meta category of a species in biodiversity recognition, is much easier. In such cases, we attempt to use coarse-grained labels to train models and apply them to finer-grained recognition tasks. Such a task is challenging as it lacks the detailed supervisory information required for finegrained tasks, preventing us from obtaining sufficiently detailed label information to train models to recognize fine-grained features and distinctions. In this paper, we attempt to train models using coarse-grained labels and apply them to challenging few-shot fine-grained recognition tasks, which are highly demanding and have practical significance.

∗ Corresponding author. This work was supported by National Key R&D Program of China (2021YFA1001100), National Natural Science Foundation of China under Grant (62272231), Natural Science Foundation of Jiangsu Province of China under Grant (BK20210340), the Fundamental Research Funds for the Central Universities (NJ2022028), and CAAI-Huawei MindSpore Open Fund.

To tackle these challenges, we propose a novel method termed Poincaré embedding with hierarchical cosine margins (PE-HCM). Specifically, PE-HCM embeds samples into a Poincaré embedding (which is also known as an embedding in the hyperbolic space [21]) and enhances their discriminative ability with a hierarchical cosine margin manner. Our PE-HCM has two coupled features, i.e. , hyperbolic space and hierarchical cosine margins:

· Hyperbolic space. we first embed the samples into the hyperbolic space instead of the regular Euclidean space, as the hyperbolic space provides stronger expressive power for hierarchical relationships [19]. Since the tasks require an embedding space that can not only capture the coarse-grained information provided during training but also generalize well to fine-grained categories during the test, the embedding space should cover multiple granularities from coarse to fine. The hyperbolic space well satisfies this prerequisite with its natural hierarchical structure.

· Hierarchical consine margins. Based on the hyperbolic space, we further incorporate hierarchical cosine distance constraints to impose a hierarchical proximity relationship among sample pairs. We construct finer-grained labels than the coarse-grained training labels through unsupervised approaches such as data augmentation and clustering. Specifically, similar to SimCLR [4], we apply two sets of data augmentations to a batch of data. For the same sample, different augmentations are assigned with the same instance-level label, while different samples are assigned with different instance-level labels. By clustering the samples within a coarse-grained category, we obtain multiple clusters that represent the fine-grained categories between the coarse-grained and the instance-level categories. As a result, we have instance-level, fine-grained, and coarse-grained labels. Correspondingly, there are four types of pairwise relationships: same instance-level category, same fine-grained category, same coarse-grained category, and different coarse-grained categories.

Moreover, to derive the appropriate target distances that will be used for different levels, we propose an adaptive strategy to update the target distances during training. It utilizes the average distance between sample pairs in each batch to update the target cosine distances with momentum. The adaptive strategy plays a crucial role in ensuring that the target distances reflect the real data distribution. By dynamically adjusting the target distances during training, our method can better capture the underlying relationships and adapt to the fine-grained characteristics of the data. In a batch of training data, we derive the target cosine distance distributions for the pairwise relationships based on their category relationships in the hyperbolic embedding space. By enforcing the consistency between the feature distribution and the category relationship branch, we enhance the hyperbolic embedding space's generalization and discriminative capability for implicit finer-grained categories.

In experiments, we perform PE-HCM on five popular benchmark datasets, i.e. , CIFAR-100 [16] and four sub-datasets { LIVING-17, NONLIVING-26, ENTITY-13, ENTITY-30 } from BREEDS [23]. Our method has achieved state-of-the-art recognition accuracy on these datasets, thereby demonstrating its effectiveness and its potential for practical applications.

In summary, our major contributions are three-fold:

We propose a novel method that addresses the challenging task of fine-grained learning from coarse-grained labels. It bridges the gap between coarse-grained and fine-grained labels and leverages the knowledge learned from coarse-grained categories for fine-grained recognition tasks.

We develop a hyperbolic embedding with hierarchical cosine margins (HCM). HCM enforces relatively large/small similarity margins between coarse/fine classes. The key point of HCM, i.e. , the target similarity margins, are dynamically updated using an adaptive strategy.

We conduct comprehensive experiments on five popular benchmark datasets, and our proposed method achieves superior recognition accuracy over competing solutions on these datasets.

2 Related Work

Coarse and Fine Learning Coarse and fine learning has been an important topic in computer vision and machine learning. Numerous methods [2, 7, 8, 26, 27, 29, 35, 40, 42] and theoretical studies [10] have been proposed to address this problem, with the goal of leveraging coarse-grained labeled data to improve fine-grained recognition. On the one hand, several methods have been proposed to tackle the coarse and fine learning problem. For instance, Stretcu et al. [26] proposed a coarse-to-fine curriculum learning method that can dynamically generate training sample sequences

based on task difficulty and data distribution, thereby accelerating model convergence and improving generalization ability. Xiang et al. [40] proposed a coarse-to-fine incremental few-shot learning method that can use coarse-grained labels to perform contrastive learning on the embedding space and then used fine-grained labels to normalize and freeze the classifier weights, thereby solving the class incremental problem. Sun et al. [27] developed a dynamic metric learning method that can adaptively adjust the metric space according to different semantic scales, thereby improving the performance of multi-label classification and retrieval. Cui et al. [7] designed a coarse-to-fine pseudo-labeling guided meta-learning method that can use coarse labels to generate pseudo-labels, and updated model parameters through meta-optimizer, thereby achieving fast adaptation in few-shot classification tasks. Bukchin [2] proposed a fine-grained angular contrastive learning method with coarse labels that can use coarse labels as prior knowledge to constrain the angular loss function, and constructed positive and negative sample pairs through random sampling and data augmentation, thereby achieving robust feature representation in fine-grained image recognition tasks. On the other hand, Fotakis et al. [10] provided a theoretical analysis of the generalization error of learning with coarse labels, which can recover the true distribution statistically without requiring additional information or assumption conditions.

Overall, previous work on coarse and fine learning demonstrates the significance of this problem and highlights the potential benefits of leveraging coarse-grained labeled data for fine-grained recognition.

Few-Shot Learning Few-shot learning [9, 20, 24, 28, 31, 33, 37-39, 41, 44-46] has become a popular research direction in computer vision due to the difficulty of collecting and annotating large datasets. The goal of few-shot learning is to enable models to quickly acquire knowledge from a small number of new samples, which is particularly suitable for fine-grained recognition as fine-grained level annotations are often costly. To address this problem, various methods have been proposed, which can be broadly categorized into three types: metric-based methods, optimization-based methods, and attention-based methods. In concretely, metric-based methods, e.g. , prototypical networks [24], relation networks [28], matching networks [31], and feature generating networks [39], learn a metric space where the similarity between support and query samples is computed. These methods aim to learn a feature representation that captures the discriminative information of categories and a distance metric that can compare samples in this feature space. Optimization-based methods, e.g. , modelagnostic meta-learning [9] and piecewise classifier mappings [37], aim to learn a model that can quickly adapt to new categories with only a few examples. These methods learn an initialization that can be fine-tuned quickly on a new task with a small amount of data. Attention-based methods, e.g. , MultiAtt [33], MattML [46] and Dual Att-Net [41], use attention mechanisms to identify the most informative parts or regions of the input images. These methods aim to learn a feature representation that is not only discriminative but also informative for few-shot recognition.

In summary, few-shot learning has been extensively studied in recent years, and various methods have been proposed to tackle this problem. While the aforementioned papers have shown success in addressing the problem, they still rely on a considerable amount of labels with the same granularity as the testing granularity during training. In contrast, our work aims to learn fine-grained recognition only from coarse labels during training, which is a more challenging task.

Hyperbolic Geometry In the field of deep learning, the hyperbolic space was first proposed by Nickel and Kiela [19] to learn hierarchical representations of symbolic data such as text and graphs by embedding them into an n -dimensional Poincaré ball, which showed that hyperbolic embeddings can capture the hierarchical relationships in knowledge graphs more effectively than Euclidean embeddings. Similarly, Ganea et al. [11] proposed a hyperbolic neural network for modeling tree-structured data, which was observed to outperform Euclidean-based models. In addition, Sala et al. [22] utilized the hyperbolic space to embed words in natural language processing tasks. Khrulkov et al. [15] proposed a hyperbolic image embedding method to represent hierarchical structures in images.

Compared to the Euclidean space which has a constant sectional curvature of 0 , the hyperbolic space has a constant negative curvature, which enables more efficient low-dimensional embedding for modeling hierarchical structure property data [19]. As illustrated in the left of Fig. 1, in this Poincaré disk, the exponential growth of distances matches the exponential growth of nodes with the tree depth [11]. This property has been shown to be particularly powerful for representing treestructured data, where the hierarchy of the data can be more naturally captured in the hyperbolic space.

C1

C4.2

C4

C4.1

C2

C1

C4

C4.1

C2

C4.2

In contrast, Euclidean space has a flat geometry, which limits its ability to represent hierarchical structures efficiently.

3 Methodology

Task Formulation In the coarse-to-fine setting [2], our goal is to train the model using only coarse-grained labels during training and achieve strong generalization performance on fine-grained categories. To evaluate the model's coarse-to-fine capability, we test it on the challenging few-shot fine-grained recognition tasks, where the model needs to recognize fine-grained categories with limited training samples. In concretely, let Y coarse = { y 1 , y 2 , . . . , y N coarse } be a set of coarsegrained classes ( e.g. , cat , dog , bird ), and Y fine = { y ′ 1 , y ′ 2 , . . . , y ′ N fine } be a set of fine-grained classes ( e.g. , husky , samoyed , burmese ). Given an auxiliary training set B , it contains N B labelled training images B = { ( I 1 , y 1 ) , ( I 2 , y 2 ) , . . . , ( I N B , y N B ) } , where I i is an example image and y i ∈ Y coarse is its corresponding label. Our goal is to use B to learn a model that recognizes different fine-grained patterns when given a small number ( e.g. , 1 for each class, as assumed in this paper) of novel fine-grained labeled samples N = { ( I ′ 1 , y ′ 1 ) , ( I ′ 2 , y ′ 2 ) , . . . , ( I ′ N N , y ′ N N ) } , where I ′ i is a support image and y ′ i ∈ Y fine is its corresponding label. If N contains N categories, each with K support images, then it is usually regarded as an N -way K -shot task.

Recap of the Hyperbolic Space We use the Poincaré ball [19] to perform the hyperbolic space. Formally, an n -dimensional Poincaré ball is defined by a manifold B n = { z ∈ R n : c ∥ z ∥ < 1 , c > 0 } , where ∥ · ∥ denotes the Euclidean norm and the hyperparameter c is used to modify the curvature. B n is an open ball of radius 1 / √ c , and with c → 0 , B n is close to the regular Euclidean space.

In the application of the hyperbolic space, hyperbolic neural networks [11] were first proposed to adopt the formalism of Möbius gyrovector spaces to define the hyperbolic versions of feed-forward networks and Multinomial Logistic Regression (MLR) [11]. In concretely, after using the traditional neural network ( e.g. , CNN) with Multi-Layer Perceptron (MLP) to get an Euclidean representation vector x ∈ R n , we map it to hyperbolic representation using the exponential map:

where exp c ( · ) donates the exponential mapping with curvature c and tanh( · ) refers to the hyperbolic tangent function. Then, we use a hyperbolic hyperplane

determined by two learnable vectors p k , a k ∈ R n as the decision boundary ( e.g. , p 1 and a 1 as shown in the middel of Fig. 1) to predict the category k :

where the ⟨· , ·⟩ represents the inner product, a k is a normal vector to H p k , a k and d ( z , H p k , a k ) is the hyperbolic space distance [11] from point z to the hyperplane H p k , a k .

Motivation We hereby present the motivation behind our work. In recent years, there have been a lot of efforts to explore the hyperbolic space to make it better suitable for a variety of different

mini-batch data

Encoder

Projector

Encoder

Projector

Poincaré  MLR

Logit

Instance labels

Target distance

Pseudo fine labels

Coarse labels

HCM module

· · ·

-means

…

…

…

· · ·

…

…

…

Cosine distance

Mean distance

Momentum update

Memory (Group by coarse labels)

share

ℒ𝑐𝑙𝑠

ℒℎ𝑐𝑚

detach

Q

1

K

Q

K

7

applications [6, 11, 14, 15, 19, 22, 34, 43]. Some efforts strive [6, 34] to obtain large-margin decision boundaries to enhance the expressive power of hyperbolic models. However, since these methods directly perform complex operations in the hyperbolic space and cannot be extended to multi-grained label tasks ( e.g. , an object having Labrador , dog and mammal labels at the same time), these methods cannot make good use of the potential of the hyperbolic space to deal with multi-grained label data.

In the regular Euclidean space, researchers have proposed many methods [17, 32] based on the cosine distance to increase the angular boundary between different categories, which brings a considerable improvement to the performance of the model. Therefore, we introduce the cosine distance constraint into the hyperbolic space, and as far as we know, we are the first to do so in the hyperbolic space. According to Eq. (1), it is known that a vector does not change its direction after being mapped from the Euclidean space to the hyperbolic space by exponential mapping. This property, aka Conformality [21], plays a crucial role in our context. It ensures that the angles between curves or vectors in the Euclidean space remain consistent when projected onto the Poincaré disk. Therefore, as the comparison between the middle and right images in Fig. 1, the introduction of large-margin cosine distance constraint in the hyperbolic space should have similar properties as introducing the cosine distance constraints in the Euclidean space. After that, it could make the decision boundaries have larger / small similarity margins between coarse/fine classes, which enhances the generalization and discriminative capabilities of the hyperbolic space for fine-grained categories.

Overall Framework The method we proposed aims at learning a mapping function f embed ( · ) using only coarse-grained label training data that embeds raw inputs into a hyperbolic space with a hierarchical distance structure for fine-grained visual recognition. Fig. 2 depicts our method at the training time, where our method is termed as Poincaré embedding with hierarchical cosine margins (PE-HCM). Specifically, we first introduce two separate random augmentations for minibatch examples to get two correlated views, and then obtain two groups of representations Q and K by an encoder and a projector. For Q and K these two groups sample points in embedding spaces, we respectively use Poincaré Multinomial Logistic Regression (MLR) and our proposed Hierarchical Cosine Margin (HCM) manner to optimize the embedding space.

similar

target angle

anchor

a sample with ground-truth

/

pseudo label

dissimilar

/

(a) Traditional cosine margin

(b) Hierarchical cosine margin

Concretely, the Poincaré MLR module first predicts the softmax probability distribution of coarsegrained categories. Then, we use the coarse-grained labels y for supervised learning

where CrossEntropy( · ) is the cross-entropy loss and p ( y = k | x ) is the probability prediction distribution as Eq. (3). Based on this, f embed ( · ) can separate different coarse-grained samples in the embedding space.

Hierarchical Cosine Margin Manner On this basis, the HCM manner constrains the distance distribution between sample pairs to be consistent with the hierarchical distribution of sample labels, so that sample pairs with different grained labels have different gaps in the embedding space, thus obtaining a more refined hierarchical embedding space that is conducive to fine-grained recognition. Specifically, we first group features K according to coarse-grained labels and store samples with the same coarse-grained labels in the same group in memory. We keep the latest M samples for each coarse-grained category in memory. Then, we perform k -means clustering on each group of features respectively, dividing each group of sample features into smaller groups of clusters. We regard these more subdivided clusters as fine-grained categories under coarse-grained categories and give different clusters different fine-grained pseudo-labels. From this point on, a feature in the embedding space has three levels of granularity labels: instance-level label (each input sample has its own instance-level label, a sample's different augmentations have the same instance-level label), fine-grained pseudo-labels and coarse-grained labels. Therefore, for a feature vector of an anchor sample in Q , there are four hierarchical relationships in K : same instance-level label, same fine-grained label, same coarse-grained label, and different coarse-grained labels.

The hierarchical cosine distance distribution is demonstrated in Fig. 3(b). From bottom to top, the visual similarity between the samples and the anchor decreases, resulting in larger cosine distances between the samples and the anchor in the embedding space.

This creates a hierarchical cosine distance distribution that follows the same pattern as the visual similarity distribution between categories. Our proposed HCM manner is more refined and complex than the traditional cosine distance constraint, as shown in Fig. 3(a). The traditional cosine distance only needs to consider two types of relationships, i.e. , similar or dissimilar, while the HCM manner needs to consider the four hierarchical similarity relationships mentioned above in our setting.

In order to constrain the sample features by the hierarchical cosine distance, we calculate the cosine distance distribution between each feature { q i } N B i =1 in Q and all features { k j } N B j =1 in K by

where d cos ( u , v ) = 1 -cos ( θ ) = 1 -uv ∥ u ∥∥ v ∥ ∈ [0 , 2] is a cosine distance function, and N B is the training batch size. As shown in Fig. 3(b), the distance between two samples in the embedded space becomes larger as the category gap between them increases. For example, when the relationship between sample pairs is the same physical object, different objects of the same kind, different species of the same genus, and different genera of the same family, the sample pairs become less and less

similar. To simplify the model, we set the distances between sample pairs with the same sample, the same fine-grained category, the same coarse-grained category, and different coarse-grained categories as d 0 , d 1 , d 2 , and d 3 , respectively, where d 0 < d 1 < d 2 < d 3 . We obtain the target semantic distance between each feature { q i } N B i =1 in Q and all features { k j } N B j =1 in K for each sample pairs by

where d t ( u , v ) =      d 0 if u , v are same instance-level labels , d 1 else if u , v are same fine-grained pseudo labels , d 2 else if u , v are same coarse-grained labels , d 3 otherwise . . To ensure that the feature distance distribution of samples in the embedding space aligns with the hierarchical class semantic similarity, we approximate the distributions of the pairwise feature cosine distances of samples and targets using the Kullback-Leibler (KL) divergence

Finally, we combine Eq. (4) and Eq. (7) to obtain the overall loss during the training process

where α is a trade-off hyperparameter.

During testing, we utilize the trained f embed ( · ) to embed the support set and query set samples into the hyperbolic space. In the following, we classify the query samples by computing their distance to the support samples through the k -nearest neighbors method.

Adaptive Hierarchical Cosine Distance In Eq. (6), we use [ d 0 , d 1 , d 2 , d 3 ] in d t ( · ) as the target values of the feature cosine distance distribution, which should be determined by the intrinsic hierarchy in different datasets, so we propose an Adaptive Hierarchical Cosine Distance (AHCD) strategy to update the target values during the training process according to the training data. Specifically, we set the target value d 0 of samples with the same instance-level label to 0 , and the target value d 3 of samples with different coarse-grained labels to 1 which correspond to angles of 90 ◦ . We initialize d 1 and d 2 to 0 . 134 and 0 . 5 which correspond to angles of 30 ◦ and 60 ◦ , respectively, and using d 0 and d 3 as anchor values to uniformly distribute them. During training, we calculate the average feature cosine distance between different examples with the same grained label in each batch of data

where l ∈ { 1 , 2 } respectively represent fine-grained label hierarchy and coarse-grained hierarchy ( i.e. , ¯ d 1 and ¯ d 2 respectively represent average distance of sample pairs with same fine-grained label and same coarse-grained label), Ω l represents a set of the sample pairs that belong to same grained labels, N l is the size of Ω l . Then, we update the target semantic distance by momentum update

We use the momentum update method to dynamically update the target semantic cosine distance of sample pairs with the same fine-grained label and the same coarse-grained label, so that d 1 and d 2 can adjust values adaptively according to the distribution of data in different datasets.

4 Experiments

Benchmark Datasets Table 1 summarizes the benchmark datasets CIFAR100 [16] and BREEDS [23]. The BREEDS dataset includes four subsets derived from ImageNet, namely LIVING17, NONLIVING-26, ENTITY-13, and ENTITY-30, each of which has a class hierarchy calibrated to ensure that classes at the same level have similar visual granularity.

Table 4: Comparisons of intra-class (all fine-grained class recognition within a random coarse class) on the BREEDS [23]. Red and blue bold numbers are the top two best results.

Baselines We compare PE-HCM with the most relevant state-of-the-art models on embedding learning: (1) MoCo-v2 [5], trained on the above datasets; (2) MoCo-v2-ImageNet [5], pre-trained on the official full ImageNet; (3) SWA V-ImageNet, pre-trained by Caron [3]; (4) ANCOR [2], combine supervised and self-supervised contrastive learning with their proposed Angular normalization module; (5) SCGM [18], based on the superclass-conditional Gaussian mixture model. (6) 'Fine upper-bound' [2], natualy trained on the fine-grained labels.

Implementations For fair comparisons with ANCOR [2] and SCGM [18], we use ResNet-12 [12] and ResNet-50 [12] as the backbone network for CIFAR-100 [16] and BREEDS [23]. We use a three-layer MLP as the projector, and the exponential mapping follows it to embed the features in the hyperbolic space. The input, hidden, and output layer dimensions of the MLP are a → a → b , where a is 640 and 2048 for ResNet-12 and ResNet-50, respectively, and the output layer dimension b is 128 , following ANCOR and SCGM. For the hyperparameters, we set c = 0 . 001 in Eq. (1), α = 800 in Eq. (8), β = 0 . 999 in Eq. (10). We used the Adam optimizer to train the model on 4 GeForce RTX 3090 Ti GPUs, and training a total of 200 epochs. For CIFAR-100 and BREEDS, the batch size were 1024 and 256, the initial learning rates were 5 × 10 -4 and 1 . 25 × 10 -4 , the learning rates were reduced by 10 times when the epoch was 120 epoch and 160 epoch. In order to prevent the distance distribution from getting stuck in local optima, we reinitialize d 1 = 0 . 134 and d 2 = 0 . 5 at the 120-th epoch and 160-th epoch. We followed ANCOR [2] to implement random data enhancement using random resized crop, random horizontal flipping, random color Jitter, random grayscale, and random Gaussian smoothing during training. We evaluate the performance using 5-way and all-way 1-shot settings during testing. The evaluation is conducted on 1000 random episodes, and we report the mean accuracy along with the 95% confidence interval.

Main Results Table 2 and Table 3 present the average accuracy rates for fine-grained learning from coarse labels on BREEDS and CIFAR-100. For each dataset, we report both 5-way and all-way fine-grained recognition results during testing. As shown in these tables, our proposed model is significantly better than other baseline methods on the above datasets. Especially on CIFAR-100, we achieve 4.05% and 6.36% improvements over state-of-the-art methods, even surpassing the fine upper-bound baseline significantly. Table 4 evaluates an intra-class case when all fine-grained classes of random coarse-grained classes were sampled in each episode. The results demonstrate that our method can better distinguish fine-grained categories in the embedding space.

Retrieval Task Results In addition to evaluating the ability of learning fine-grained embeddings from coarse labels in few-shot learning tasks, Grafit [29] and MaskCon [8] also investigated the effectiveness of the 'coarse to fine' approach by using retrieval tasks for validation. As shown in Table 5, we conducted fair comparisons of our method with these state-of-the-art approaches, using

the same backbone and epoch settings as MaskCon. Our method still achieves superior results over Grafit and MaskCon (almost 7% improvement on all-way FSFG) under FSFG, and achieves a mean Average Precision (mAP) of 62.04%, surpassing MaskCon's 60.27%. This also reinforces the efficacy of our approach. Additionally, it is worth noting that our method is lower than MaskCon when we used Recall@k as an evaluation metric (by following MaskCon). The reason might be that image retrieval involves identifying relevant images from a gallery of diverse images, usually by utilizing a query image as a reference. In this scenario, the task entails finding any images belonging to the same category as the query image from the gallery, which might contain multiple images per category. On the other hand, few-shot recognition, exemplified by the one-shot scenario, presents a more challenging scenario. In this setting, the goal is to recognize objects/classes with very limited training examples, often relying on only a single image per category. This constraint inherently magnifies the difficulty of the task.

Proposals in Our Method As shown in Table 6, we explored the improvement of each module in our method on LIVING-17 and CIFAR-100. The results of the first two rows show that the hyperbolic space is more suitable for the fine-grained embedding from coarse labels with a hierarchical structure, which is consistent with the conclusion shown in [11, 15]. The results of the last three rows show that the introduction of our HCM manner and AHCD strategy in the hyperbolic space can bring great improvement. This indicates that all proposals of our method can better learn the discriminative patterns between finer-grained categories from coarse labels.

Table 6: Comparisons of our proposals on LIVING17 [23] and CIFAR-100 [16]. E : Coarse-grained supervised learning in the Euclidean space. H : Coarsegrained supervised learning in the hyperbolic space. HCM: Our proposed Hierarchical Cosine Margin manner. AHCD: Our proposed Adaptive Hierarchical Cosine Distance strategy.

Visualization As shown in Fig. 4, it can be observed that compared to training the model using only coarse labels, our method leads to a more concentrated distribution of sample points for each category, while also accentuating the distinctions between different classes. The visualization validates the effectiveness of our method of contributing to the discriminative power from the qualitative aspect.

Adaptive Ability We extensively explored the effective- ness of the AHCD strategy. As shown in Fig. 5, we tracked the update processes of d 1 and d 2 during training on CIFAR-100 and LIVING-17 datasets. In general, there are the same trends and different details. In the early stages of training, as the model lacks discriminative ability, the values of d 1 and d 2 gradually decrease. As the model continues to train, it gradually acquires discriminative ability, resulting in an increase in the values of d 1 and d 2 followed by stabilization. From the final stable results, we observe that the margin between coarse-grained categories ( d 2 ) is larger than the margin between fine-grained categories ( d 1 ), and the margin between fine-grained categories on CIFAR-100 is larger than the margin between fine-grained categories on LIVING-17, which aligns with the true category relationships in the data distribution. This demonstrates that our proposed adaptive strategy can adjust d 1 and d 2 during training according to the actual data distribution.

Hyperparameters As shown in Fig. 6, we changed the trade-off hyperparameter α of the hierarchical cosine loss in Eq. (8) on the CIFAR-100 and LIVING-17 datasets to observe its impact for the final result. It can be seen that, as α increases exponentially, the performance of the model increases

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0

50

100

150

200

value

CIFAR-100

LIVING-17

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

50

100

150

200

value

CIFAR-100

LIVING-17

32

37

42

47

52

50

100

200

400

800

1600

accuracy

trade-off hyperparameter

α

CIFAR-100

LIVING-17

first and then decreases. The value of α is also sensitive, with a smaller value of α limiting the discriminative hierarchical embedding ability of the HCM manner, and a larger value of α suppressing the generalization ability to fine-grained categories.

As shown in Fig. 7, we change the number of k -means clusters for each coarse-grained class in the HCM manner on CIFAR-100 and LIVING-17, and report the recognition results for all fine-grained categories (all-class) and intracoarse class fine-grained categories (intra-class). From the experimental results, it can be observed that the number of clusters has a significant impact on the fine-grained recognition performance. When the number of clusters is small, the model performs poorly in fine-grained recognition tasks. Having a larger number of clusters than the actual number of fine-grained categories leads to better results. Additionally, in our experiments across five datasets, we found a recurring pattern that offers a guideline. Typically, when clustering coarse-grained categories, the number of clusters that yields optimal results is roughly twice the

33

35

37

39

41

43

45

2

4

6

8

10

12

14

accuracy

clusters

CIFAR-100

intra-class

all-class

49

50

51

52

53

54

55

56

57

58

2

4

6

8

10

12

14

accuracy

clusters

LIVING-17

intra-class

all-class

actual number of fine-grained subclasses within that coarse category. While this heuristic emerged consistently across our tested datasets, we acknowledge that it may not be universally optimal. However, it does provide a good starting point for users. In practice, fine-tuning based on validation performance is always recommended to ascertain the best hyperparameter for a specific task.

5 Conclusions

We proposed a novel method (PE-HCM) for fine-grained learning from coarse labels. We used the hyperbolic space to embed the samples and enhanced the discriminative ability with a hierarchical cosine margins manner. On the one hand, we performed supervised learning using coarse-grained labels to distinguish coarse-grained category samples initially. On the other hand, we constructed instance-level labels and fine-grained pseudo-labels using data enhancement and clustering methods, for the hierarchical cosine margins manner constraining the distance between sample pairs of four relationships, which resulted in a more refined hierarchical division of distances for fine-grained learning. By optimizing the hierarchical cosine distance using the hierarchical cosine margins manner, the learned embedding could be well generalized to fine-grained visual recognition tasks. Experiments on five benchmark datasets demonstrated the effectiveness of the proposed method.

Throughout our experiments, we have occasionally observed a particular scenario where our method yields comparatively lower results. This typically occurs when classes within the same hierarchy level exhibit substantial variations in their scopes. For instance, when some classes possess a significantly broader scope while others have a notably narrower scope. We believe this phenomenon arises from the fact that our method employs a uniform margin across all levels. In the future, more refined category-specific semantic hierarchical distance constraints are worth further study, that is, further constraining the distances between sample pairs of different categories within the same grain of granularity to make them more consistent with the real data distribution.

References

Mohammad H. Alobaidi, Mohamed A. Meguid, and Tarek Zayed. Semi-supervised learning framework for oil and gas pipeline failure detection. Scientific Reports , 12(13758), 2022.

Guy Bukchin, Eli Schwartz, Kate Saenko, Ori Shahar, Rogerio Feris, Raja Giryes, and Leonid Karlinsky. Fine-grained angular contrastive learning with coarse labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8730-8740, 2021.

Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems , pages 9912-9924, 2020.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning , pages 1597-1607, 2020.

Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 , 2020.

Hyunghoon Cho, Benjamin DeMeo, Jian Peng, and Bonnie Berger. Large-margin classification in hyperbolic space. arXiv preprint arXiv:1806.00437 , 2018.

Yawen Cui, Qing Liao, Dewen Hu, Wei An, and Li Liu. Coarse-to-fine pseudo supervision guided meta-task optimization for few-shot object classification. Pattern Recognition , 122:108296, 2022.

Chen Feng and Ioannis Patras. MaskCon: Masked contrastive learning for coarse-labelled dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1991319922, 2023.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400 , 2017.

Dimitris Fotakis, Alkis Kalavasis, Vasilis Kontonis, and Christos Tzamos. Efficient algorithms for learning from coarse labels. In Proceedings of Conference on Learning Theory , pages 2060-2079, 2021.

Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. In Advances in Neural Information Processing Systems , pages 5345-5355, 2018.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 770-778, 2016.

Geoffrey E. Hinton and Sam Roweis. Stochastic neighbor embedding. In Advances in Neural Information Processing Systems , pages 857-864, 2002.

Joy Hsu, Jeffrey Gu, Gong Wu, Wah Chiu, and Serena Yeung. Capturing implicit hierarchical structure in 3d biomedical images with self-supervised hyperbolic representations. In Advances in Neural Information Processing Systems , pages 5112-5123, 2021.

Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky. Hyperbolic image embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6418-6428, 2020.

Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto , 2009.

Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolutional neural networks. arXiv preprint arXiv:1612.02295 , 2016.

Jingchao Ni, Wei Cheng, Zhengzhang Chen, Takayoshi Asakura, Tomoya Soma, Sho Kato, and Haifeng Chen. Superclass-conditional gaussian mixture model for learning fine-grained embeddings. In Proceedings of International Conference on Learning Representations , pages 1-12, 2022.

Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In Advances in Neural Information Processing Systems , pages 6338-6347, 2017.

Archit Parnami and Minwoo Lee. Learning from few examples: A summary of approaches to few-shot learning. arXiv preprint arXiv:2007.07350 , 2022.

Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, and Guoying Zhao. Hyperbolic deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):1002310044, 2021.

Frederic Sala, Chris De Sa, Albert Gu, and Christopher Ré. Representation tradeoffs for hyperbolic embeddings. In Proceedings of the International Conference on Machine Learning , pages 4460-4469, 2018.

Shibani Santurkar, Dimitris Tsipras, and Aleksander Madry. BREEDS: Benchmarks for subpopulation shift. arXiv preprint arXiv:2008.04859 , 2020.

Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems , pages 4077-4087, 2017.

Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. In Advances in Neural Information Processing Systems , pages 19339-19352, 2020.

Otilia Stretcu, Emmanouil Antonios Platanios, Tom M Mitchell, and Barnabás Póczos. Coarse-to-fine curriculum learning. arXiv preprint arXiv:2106.04072 , 2021.

Yifan Sun, Yuke Zhu, Yuhan Zhang, Pengkun Zheng, Xi Qiu, Chi Zhang, and Yichen Wei. Dynamic metric learning: Towards a scalable metric space to accommodate multiple semantic scales. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5393-5402, 2021.

Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1199-1208, 2018.

Hugo Touvron, Alexandre Sablayrolles, Matthijs Douze, Matthieu Cord, and Hervé Jégou. Grafit: Learning fine-grained image representations with coarse labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 874-884, 2021.

Grant Van Horn, Elijah Cole, Sara Beery, Kimberly Wilber, Serge Belongie, and Oisin Mac Aodha. Benchmarking representation learning for natural world image collections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12884-12893, 2021.

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Processing Systems , pages 3630-3638, 2016.

Hao Wang, Yitong Wang, Zilei Zhou, Xingang Ji, Dihong Gong, Jingdong Zhou, Zhifeng Li, and Wei Liu. CosFace: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , page 5265-5274, 2018.

Peng Wang, Lingqiao Liu, Chunhua Shen, Zi Huang, Anton van den Hengel, and Heng Tao Shen. Multiattention network for one shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2721-2729, 2017.

Melanie Weber, Manzil Zaheer, Ankit Singh Rawat, Aditya K Menon, and Sanjiv Kumar. Robust largemargin learning in hyperbolic space. In Advances in Neural Information Processing Systems , pages 17863-17873, 2020.

Xiu-Shen Wei, Yang Shen, Xuhao Sun, Peng Wang, and Yuxin Peng. Attribute-aware deep hashing with self-consistency for large-scale fine-grained image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(11):13904-13920, 2023.

Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha, Jianxin Wu, Yuxin Peng, Jinhui Tang, Jian Yang, and Serge Belongie. Fine-grained image analysis with deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(12):8927-8948, 2022.

Xiu-Shen Wei, Peng Wang, Lingqiao Liu, Chunhua Shen, and Jianxin Wu. Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples. IEEE Transactions on Image Processing , 28(12):6116-6125, 2019.

Xiu-Shen Wei, He-Yang Xu, Faen Zhang, Yuxin Peng, and Wei Zhou. An embarrassingly simple approach to semi-supervised few-shot learning. In Advances in Neural Information Processing Systems , pages 14489-14500, 2022.

Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5542-5551, 2018.

Xiang Xiang, Yuwen Tan, Qian Wan, Jing Ma, Alan Yuille, and Gregory D Hager. Coarse-to-fine incremental few-shot learning. arXiv preprint arXiv:2111.14806 , 2021.

Shu-Lin Xu, Faen Zhang, Xiu-Shen Wei, and Jianhua Wang. Dual attention networks for few-shot finegrained recognition. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 2911-2919, 2022.

Yuanhong Xu, Qi Qian, Hao Li, Rong Jin, and Juhua Hu. Weakly supervised representation learning with coarse labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10593-10601, 2021.

Yi Xu, Dongsheng Wang, Bo Chen, Ruiying Lu, Zhibin Duan, Mingyuan Zhou, et al. HyperMiner: Topic taxonomy mining with hyperbolic embedding. In Advances in Neural Information Processing Systems , pages 31557-31570, 2022.

Han-Jia Ye, Lu Han, and De-Chuan Zhan. Revisiting unsupervised meta-learning via the characteristics of few-shot tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(3):3721-3737, 2023.

Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8808-8817, 2020.

Yaohui Zhu, Chenlong Liu, and Shuqiang Jiang. Multi-attention meta learning for few-shot fine-grained image recognition. In Proceedings of the International Joint Conference on Artificial Intelligence , pages 1090-1096, 2020.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper adresses the issue of coarse to fine learning, _i.e._ coarse labels are present during training but the metric are computed on fine-grained labels. The paper introduces a method Poincaré embeddings with hierarchical cosine margins (PE-HCM). The paper suggests embedding the image representation in the hyperbolic space for coarse label classification to take advantage of its natural hierarchical organisation. It has a second branch that clusters with k-means instances that share the same coarse labels to simulate fine-grained pseudo-labels. It uses a hierarchical loss that aims at making the cosine distance between pairs match their relation, i.e. same instance label (same instance with different data augmentation), same cluster label (assigned by k-means), same coarse label (supervision), different coarse label (supervision). The paper validates experimentally the method on several benchmarks, CIFAR-100 and BREEDS - a collection of 4 datasets. The paper conducts ablation studies of the different components of the method.\n\nSTRENGTHS:\nS1. The method is well justified. The adaptive hierarchical distance is interesting and works well experimentally.\n\nS2. Experimental results are encouraging.\n\nS3. Ablation studies are present.\n\nWEAKNESSES:\nW1. Some works are missing in the related works: MaskCon (MaskCon: Masked Contrastive Learning for Coarse-Labelled Dataset, CVPR 2023) and Grafit (Grafit: Learning fine-grained image representations with coarse labels, ICCV 2021) that both tackle the coarse to fine task.  And UnHyp (Unsupervised Hyperbolic Metric Learning, CVPR 2021) that uses hyperbolic space to create a hierarchy.\n\nW2. The paper lacks comparison to state-of-the-art methods such as MaskCon and Grafit. It could conduct experiments on the large scale iNaturalist-18 dataset.\n\nW3. The paper lacks comparison to unsupervised baselines, such as UnHyp or more recent state-of-the-art self-supervised-learning methods, such as Dino (Emerging Properties in Self-Supervised Vision Transformers, ICCV 2021), iBot (iBOT: Image BERT Pre-Training with Online Tokenizer, ICLR 2022) etc.\n\nMinor:\n\nW4. The paper could remind the reader, in the benchmark datasets section, what classes are present in the different datasets, and what are the coarse classes on the different datasets.\n\nQUESTIONS:\nQ1. Line 145: The paper gives notations for coarse and fine-grained labels. Could the method take advantage of a hierarchy of classes rather than solely a coarse label?\n\nQ2. What is the impact of the granularity of the coarse labels, _i.e._ how much will the performance deteriorate when the coarse labels become more and more broad and on the contrary what happens when given the fine-grained labels?\n\nQ3. Line 227: the paper defines a distance that is used in the HCM module. It uses the Euclidian cosine distance, this is surprising as the paper states that it compares instances in the hyperbolic space. The paper should elaborate this design choice.\n\nQ4. Table 2: reports results on the BREEDS benchmark. I am not sure to understand what are the results indicated as “5-way” and “all-way”.\n\nTypos:\n\n- Table 1: Coarase —> Coarse\n- Table 5: European —> Euclidian\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method developed in the hyperbolic space to embed visual embeddings to deal with the task of fine-grained learning from coarse labels. By discovering the advantages of hyperbolic space, the authors design the hierarchical cosine margin manner and an adaptive hierarchical cosine distance, which favour modelling fine-grained objects. Experiments are conducted on five benchmark datasets. Evaluations show that the proposed method achieves superior recognition accuracy over competing solutions on these datasets.\n\nSTRENGTHS:\n+ The proposed method is novel and makes sense, which has great technical contributions, especially for the introduction of hyperbolic space into the fine-grained learning from coarse labels.\n\n+ The fine-grained learning from coarse labels task has good practical potential, which also is a challenging problem in computer vision and machine learning. Such a task deserves further exploration.\n\n+ The hierarchical cosine margins and the adaptive hierarchical cosine distance are interesting. These modules are crucial for the whole framework of the proposed method and they are also tailored designs for the task studied in this paper.\n\n+ The experiments are comprehensive and convincing. Overall, they can validate the effectiveness of the proposed method, as well as the proposed modules.\n\n+ The paper is well-written and easy to follow.\n\nWEAKNESSES:\nAlthough the motivation of Figure 1 is relatively clear, it can be further explained.\n\nQUESTIONS:\n- As stated in the section of Hyperparameters, different numbers of clusters lead to different accuracy. How to choose an appropriate hyperparameter for a specific task for users?\n\n- Some minor issues of paper writing should be fixed, e.g., “Fig” should be fixed as “Fig.” or “Figure”.\n\nLIMITATIONS:\nThis paper does not explicitly discuss its limitations.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel method, PE-HCM, for fine-grained learning from coarse labels. The authors propose the use of the hyperbolic space for sample embedding and introduce a hierarchical cosine margins manner to enhance discriminative ability. The method combines supervised learning with coarse-grained labels and instance-level labels obtained through data enhancement and clustering. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed approach.\n\nSTRENGTHS:\n* The paper addresses an important problem in the field of fine-grained learning, which is the challenge of learning from coarse labels. Fine-grained recognition is crucial in various applications, and the proposed method offers a solution to improve the performance in this context. The significance of the problem adds value to the research presented in the paper.\n* The paper introduces several novel concepts and methods. The use of the hyperbolic space for sample embedding is a unique approach that has shown promise in other related works. Additionally, the introduction of the hierarchical cosine margins manner to enhance discriminative ability is an innovative strategy that contributes to refining the hierarchical divisions for fine-grained recognition. The combination of these concepts and methods in the proposed PE-HCM method demonstrates the originality of the research.\n* The paper exhibits good writing quality, characterized by clarity and soundness. The paper effectively communicates the motivation, methodology, and experimental results to the readers. The descriptions of the proposed method and the experimental setup are clear and well-presented. The writing style contributes to the overall understanding of the research and enhances the readability of the paper.\n* The paper demonstrates a high level of experimental rigor. The authors provide detailed descriptions of the benchmark datasets used, including their characteristics and sizes. They also compare the proposed method with several state-of-the-art models, covering a wide range of relevant approaches in the field of embedding learning. By conducting extensive experiments on multiple benchmark datasets, the authors validate the effectiveness of their proposed method and show its superiority over existing techniques. The inclusion of statistical measures such as mean accuracy and confidence intervals further enhances the reliability of the experimental results.\n* The paper showcases the adaptive ability of the proposed method through the introduction of the Adaptive Hierarchical Cosine Distance (AHCD) strategy. The authors track the update processes of the distance parameters and show that the method adjusts these parameters during training to align with the actual data distribution. Furthermore, the paper provides a thorough analysis of the trade-off hyperparameter α, demonstrating the sensitivity of the method to its value and offering insights into finding an appropriate balance between discriminative hierarchical embedding and generalization ability.\n\nWEAKNESSES:\n* While the paper includes theoretical illustrations of sample pair distances, it would be beneficial for the authors to provide empirical evidence by visualizing the feature distances between real samples.\n* The paper focuses on the evaluation of the proposed method on specific benchmark datasets but lacks a broader discussion on the generalizability of the results. Providing insights into the transferability of the proposed approach to other fine-grained recognition tasks or datasets would strengthen the overall impact of the paper.\n\nQUESTIONS:\nThe paper introduces the use of the hyperbolic space for embedding samples and applies the hierarchical cosine margins to enhance the discriminative ability. While the results demonstrate improved performance in fine-grained recognition, I would like to inquire about the interpretability of the learned embeddings. Can the authors provide more insights into the interpretability of the hyperbolic embeddings and how the hierarchical cosine margins contribute to the separability of fine-grained categories? It would be helpful to include visualizations or case studies that illustrate how the learned embeddings capture meaningful hierarchical relationships and contribute to the discriminative power.\n\nLIMITATIONS:\nThe paper concludes by discussing potential directions for future research, highlighting the importance of refining category-specific semantic hierarchical distance constraints. This suggestion indicates the authors' awareness of the limitations of the proposed method and opens up opportunities for further exploration and improvement in fine-grained learning from coarse labels.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of learning fine-grained embeddings from coarse labels, where detailed distinctions required for fine-grained tasks are often lacking. The authors propose a novel method that embeds visual embeddings into a hyperbolic space and enhances their discriminative ability using a hierarchical cosine margins approach. The hyperbolic space offers advantages such as capturing hierarchical relationships and increased expressive power, making it suitable for modeling fine-grained objects. By enforcing relatively large/small similarity margins between coarse/fine classes in the hyperbolic space, the proposed hierarchical cosine margins approach is introduced. While enforcing similarity margins in the regular Euclidean space is popular for deep embedding learning, extending it to the hyperbolic space is non-trivial and valuable for coarse-to-fine generalization. Experiments are conducted on five benchmark datasets.\n\nSTRENGTHS:\n1.\tNovel Method: The paper introduces a novel method, Poincare embedding with hierarchical cosine margins (PE-HCM), which addresses the challenging task of fine-grained learning from coarse-grained labels. This innovative approach bridges the gap between coarse-grained and fine-grained labels, enabling the transfer of knowledge from coarse-grained categories to fine-grained recognition tasks.\n2.\tTechnical Contribution: The proposed method incorporates key technical contributions. It utilizes the hyperbolic space to capture hierarchical relationships and provide increased expressive power. Additionally, the hierarchical cosine margins enforce proximity relationships among sample pairs, enabling the model to learn fine-grained features and distinctions. The adaptive strategy for updating target distances further enhances the model's ability to capture underlying relationships and adapt to the fine-grained characteristics of the data.\n3.\tPaper Writing: The paper is well-written and effectively communicates the motivations, challenges, and proposed solutions. The introduction provides a clear background and problem statement, leading to a comprehensive description of the proposed method. The technical details, including the use of hyperbolic space and hierarchical cosine margins, are explained concisely, making the paper accessible to readers.\n4.\tExperimental Performance: The proposed method is thoroughly evaluated on five benchmark datasets, demonstrating its superior performance compared to competing solutions. By achieving state-of-the-art recognition accuracy, the experimental results highlight the effectiveness and potential practical applications of the proposed method in fine-grained recognition tasks.\n\nWEAKNESSES:\n1.\tLimited Analysis of Failure Cases: The paper could include a discussion of the failure cases of the proposed method. Analyzing scenarios where the method does not perform well would provide insights into potential weaknesses and opportunities for future improvements.\n\n2.\tMinor Issues: There are several typos throughout this paper, e.g., in Table 1, Coarase should be Coarse.\n\nQUESTIONS:\nI am curious about how does the proposed method perform when the available coarse-grained labels are noisy or contain inaccuracies?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-12"}