SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery
Authors: Zhenqi He
Yuanpei Liu
Kai Han
Venue: NeurIPS 2025 poster ()
Keywords: Generalized Category Discovery; Open World Learning; Hierarchical Learning
Primary area: deep_learning
TL;DR: We introduce SEAL, a semantic-aware hierarchical learning framework that leverages real-world coarse-to-fine taxonomies to provide stronger supervision for GCD.

Abstract: This paper investigates the problem of Generalized Category Discovery (GCD). Given a partially labelled dataset, GCD aims to categorize all unlabelled images, regardless of whether they belong to known or unknown classes. Existing approaches typically depend on either single-level semantics or manually designed abstract hierarchies, which limit their generalizability and scalability. To address these limitations, we introduce a SEmantic-aware hierArchical Learning framework (SEAL), guided by naturally occurring and easily accessible hierarchical structures. Within SEAL, we propose a Hierarchical Semantic-Guided Soft Contrastive Learning approach that exploits hierarchical similarity to generate informative soft negatives, addressing the limitations of conventional contrastive losses that treat all negatives equally. Furthermore, a Cross-Granularity Consistency (CGC) module is designed to align the predictions from different levels of granularity. SEAL consistently achieves state-of-the-art performance on fine-grained benchmarks, including the SSB benchmark, Oxford-Pet, and the Herbarium19 dataset, and further demonstrates generalization on coarse-grained datasets.
Project page: https://visual-ai.github.io/seal/

SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery
Zhenqi He
Yuanpei Liu"
Kai Han"
Visual Al Lab, The University of Hong Kong
(zhenqi_he, he, ypliu0) Sconnect. hku.
kaihanx0hku. hk
Abstract
This paper investigates the problem of Generalized Category Discovery (GCD). Given a partially labelled dataset, GCD aims to categorize all unlabelled im- ages, regardless of whether they belong to known or unknown classes. Exist- ing approaches typically depend on either single-level semantics or manually de- signed abstract hierarchies, which limit their generalizability and scalability. To address these limitations, we introduce SEmantic- aware hierArchical Learning framework (SEAL), guided by naturally occurring and easily accessible hierarchi- cal structures. Within SEAL, we propose a Hierarchical Contrastive Learning approach that exploits hierarchical similarity to generate in- formative soft negatives, addressing the limitations of conventional contrastive losses that treat all negatives equally. Furthermore, Cross Pross-Granularity Consis- tency (CGC) module is designed to align the predictions from different levels of granularity. SEAL consistently achieves state-of-the-art performance fine- grained benchmarks, including the SSB benchmark, Oxford-Pet, and Herbar- ium19 dataset, and further demonstrates generalization coarse-grained datasets. Project page: https: http://visaaaaaaa aai, si.ge .it.ggi............ i0/ /ossel//
Cemmantic-uuidee Soft
Introduction
The field of computer vision has undergone substantial progress in various tasks, including classifi- cation [53, 28], object detection [21, 52], and segmentation [27, 30, 63). Such advancements have largely been driven by access to large-scale, human-annotated datasets [13, 37]. However, models trained on these datasets are constrained
closed-world paradigm, limiting their predictions to the predefined labels within the training set. In contrast, there exists wealth of unlabelled data in the open world. To capitalize on the unlabelled data, a variety of Semi-Supervised Learning (SSL) techniques [10] have been proposed, yielding notable improvements over traditional super- vised learning methods. Despite substantial success in various tasks [3, 68, 1 I1, most existing SSL methods are designed under the closed-set set assumption, wherein the training and test datasets share an identical set of classes. Category discovery, initially introduced as Novel Category Discovery (NCD) [24, 29] and later extended Generalized Category Discovery (GCD) [57, 29], has recently emerged as a compelling open-world problem, attracting significant attention. Unlike SSL, GCD tackles the challenges where the unlabelled subset may include instances from both known and unknown classes. Its primary objective is to utilise knowledge gained from labelled data to effec- tively categorize all samples within the unlabelled data. Concurrently, an equivalent task named Open-world Semi-Supervised Learning (OSSL) [5] has also been introduced.
'Equal contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) GCD With Abstract Hierarchies
Red Fart
Reogle
Rorurt House
Siberian Fioer
Eggasion Mani
Siew
(b) GCD With Semuntic- Hieranchies
anviviee
Combowe
Folidee
Ansent Aarora
songleer
Fillpe
Wod Fox
Cagapos
Mesple
Boosst House
Appyetoo Man
Siherian Figer
Rengef
Comparison of SEAL and previous methods [50, 64] using hierarchical learning. (a) In previous attempts, several upper and lower levels as well as abstract concepts are defined around the ground-truth level, which may cause errors in the hierarchical structure. (b) In our method, we propose to utilise the semantic information at different levels to enhance the GCD performance.
Figure 1:
The effectiveness of GCD is rooted in the efficient transfer of knowledge from known categories to cluster samples of both known and novel categories. As a transfer clustering task [24), hierarchical information has been demonstrated to be effective in GCD [50, 26] and similarly in the parallel task of OSSL [64], particularly with fine-grained datasets [58]. In [50]. the hierarchical structure is com- posed of abstract concepts as implicit binary tree, where each node represents increasingly abstract concept derived from shared binary code prefixes, and in [51], the hicrarchical structure is implicitly formed by incrementally halving the category count as the hierarchy level increases with hierarchical pseudo-labeling to provide soft supervision for the training. Similarly, CiPR [26] con- structs abstract hierarchies by iteratively merging data partitions through semi-supervised clustering. The hierarchical tree in [64] consists of manually defined upper and lower levels that represent dif- ferent granularities, with the number of categories per level controlled by hyperparameters. More recently, HypCD [39] implicitly models hierarchies via hyperbolic embeddings, achieving strong performance and underscoring the importance of hierarchical information for GCD. These methods build hierarchical levels from abstract, weakly supervised structures that may introduce noise and errors, ultimately affecting GCD performance. As illustrated in Fig. (a), the 'Siberian Tiger'. 'Ben- gal' and 'Egyptian Mau' can be merged into a single category while the 'Red Fox' can be divided into multiple categories. Additionally, the high similarity among categories may result in some im- ages of the "Basset Hound' being incorrectly merged with those of the 'Beagle.. This observation naturally prompts us to consider: whether the intrinsic, semantically grounded taxonomies present in the reul world can serve as more reliable guides. In botanical research, taxonomists commonly use labelled specimens of known species to classify newly collected, unlabelled samples into exist- ing taxonomic hierarchies or to identify unseen species [40, 33]. Similarly, studies in closed world visual classification [9, 65, 15] have shown that hicrarchical structures enhance classification. From an information- tthooreticctcctceeeeooooooooeete perspective, we further deduce that such semantic-aware hierarchies yield a tighter mutual information bound, providing a principled foundation for our design.
To this end, we propose the SEmantic-award hierArchical Learning (SEAL) framework for informa- tion [57, 59, 61, 66, 38] or rely on abstract hierarchical cues [50, 26, 64, 51, 391, SEAL effectively leverages the naturally occurring semantic hierarchies without manual design (shown in Fig. 1 (b)) and incorporates several innovative techniques tailored specifically for this task. Firstly, we imple- ment multi-task training paradigm that facilitates the simultaneous discovery of categories across several different semantic levels. Secondly, we introduce a Gross-Granulartty Consistency (CGC)
GCD. Unlike previous approaches that either focus exclusively on

---

module to align the class predictions from different granularities. Thirdly, we propose the Hier- archical Semantic-guide Soft Contrastive Learning capture uncertainty in contrastive learning. ensuring that not all negative samples are treated equally. By effectively integrating these compo- nents into cohesive framework, SEAL can be trained end-to-end in a single stage.
In summary, we make the following contributions in this paper: (i) We propose SEAL, novel framework specifically designed tackle the challenging GCD task by leveraging the inherent se- mantic hierarchies, marking the first exploration of this aspect. (ii) Within the SEAL framework, we develop two novel components: the Semantic- semantic-guided guided Soft Contrastive Learning. These components function synergistically to signifi- cantly enhance the model's category discovery capabilities. (iii) Through extensive experimentation on public GCD benchmarks, SEAL consistently demonstrates its effectiveness and achieves superior performance, especially on fine-gra one-grained datasets.
Consistency (CGC) module and Hierarchical
2 Related Work
Category Discovery. Novel Category Discovery (NCD) was first articulated in [24], establishing a pragmatic framework for transferring knowledge from known categories to clusters of unseen cat- egories, framed as transfer clustering problem. Subsequently, variety of methods have emerged to advance the research domain [22, 23, 31, 72, 74, 18]. Generalized Category Discovery (GCD) extends the NCD framework by relaxing its assumptions, incorporating unlabelled data that features samples from both known and unknown classes [57]. Recent studies [5, 26, 48, 32, 8, 62, 38] have explored range of strategies tackle the challenges introduced by GCD. Notably, InfoSieve [50] and CiPR [26] guide category discovery using abstract hierarchies that are automatically inferred from the data. A similar approach is employed in the OSSL task by TIDA [64], which employs handcrafted abstract hierarchies by constructing prototypes at manually defined levels. Conversely, SimGCD [66] introduces an entropy-regularrized classifier that provides robust baseline. SPT- Net [61] builds upon SimGCD by incorporating spatial prompt tuning to emphasize salient object cetrributionguuidee debiased learning framework address the inherent label bias and semantic shifts in GCD.
parts, while DebGCD [38] proposes
Hierarchical Learning. In the realm of hierarchical learning, numerous studies [9, 49, 65, 15] have explored the use of hierarchical label information to enhance classification performance, par- ticularly in closed-world settings. For instance, [9] employs a multi-task framework that utilises coarse-to-fine labels improve fine-grained recognition, whereas [71] introduces hierarchical con- trastive learning to enrich representations with multi-level semantic cues. More recently, hierarchical learning has been adapted open-set recognition, as demonstrated in [36, 67], where semantic hi- crarchies contribute to improved generalization to unseen classes. To the best of our knowledge. our work is the first to apply semantic-guided hierarchies the GCD task, facilitating the effective discovery and classification of both known and novel categories.
3 Preliminary
Problem Statement: GCD aims to develop models capable of classifying unlabelled samples from known categories while simultaneously clustering those from unknown categories. Formally, we are given a labelled dataset Dj- = (zf, yf) C X 34 and an unlabelled dataset D, = (x v²*) C X Xx where )) C )n. The unlabelled data includes samples from both known and novel categories. The number of known categories is denoted by M D+., and the total number of categories is K 1D) U I. Following prior works [23, 66, 59], assume K is known during training. When K is unknown, it can be estimated using techniques such as [24, 57].
Revisiting Baseline: SimGCD [66] is representative end-to-end baseline for GCD that unifies contrastive representation learning and parametric classification. The model employs a Vision Trans- former [14] backbone pretrained using DINO [7], where the input image is first passed through an embedding layer o and the feature extractor F, followed by a projection head H to produce a nor- malized representation z, H((F((((x))//HH((((((tt))) The representation learning objective
3

---

Crep is based on the InfoNCE loss [44]]
(xe)(
log a oz :
(I)
(P(x,)
where P(x;)) denotes the set of positive features (e.g., different views of the same image), and ( (::7) the softmax with temperature т. For labelled samples, additional positives from the same class used to enable supervised contrastive learning.
For the parametric classification, SimGCD adopts a cosine-based classifier [20] with a learnable prototype set C {CI,.... ..... Cк) where each prototype C₁ lz-normalized and the output probability for the k-th category is given by (k) oz. T). Given the pseudo-label q, obtained from sharpened prediction different view, the classification loss is:
Lats
lee (g. (H))).
B
IEB
where B is current image batch and (p) denotes the entropy of the mean prediction p. Specifically, for each æ; in the labelled batch B,, an additional y: the one-hot ground-truth vector is also used for supervised classification loss written as L |8 Lies, (p., y:). Then, the overall classification loss is formulated as Lots (1 A,) + AoLiis where As is a balance factor. The final training objective combines both representation and classification terms: Lbs Lets Erep-
4 Method
Before delving into methodological details, we begin with an intuitive hypothesis that underlies our framework: Leveraging structured semuntic hierarchies across multiple levels can facilitate more informative and robust feature learning for GCD setting. To support this intuition, we first present theoretical justification grounded in information theory, demonstrating that the incorporation of hicrarchical labels yields a tighter bound on mutual information. This theoretical insight lays the foundation for the design of our approach, which we detail in the subsequent sections.
4.1 Theoretical Motivation
From the perspective of information theory, with denoting model parameter as data X, and label as y, we write Z = ff((') as the deterministic representation of X once model 0 is fixed. The optimisation objective is then maximize the mutual information between Z and Y [4], which can 1o Zr; Vi) + B Ho((Y, X.) HHd { with detailed proof provided in the Appendix, where Y, is the model prediction for unlabelled data, and (3 is the weight factor. Assuming the coarse graiine semantic hierarchical labels 20 ,.. (1--)) are accessible for all labelled samples, the objective naturally extends to:
be re-formulated minę
min {
To Zd 11 ................................ >(")) 3 Ho (. (x.)) Ho(S)
(3)
By applying the chain rule of mutual information, the supervised part satisfies:
H-1 To ((
IdZ = - +
, )...)) 2 Id(Zi>)11))
(4)
Analogously, for the unsupervised component, we obtain:
#-1
Hd (.Y(1:1) Xx) Ho({():))) 1o(xa; p((1)) Σ
Id: (1) p((+1:st))
(5)
< - Lo( ((1) = (Ho( + Ho (x).

---

Imsag
Challe
Cress Cross-grsmilin gramularity Comsatrocy
P
weight- sharing
Pline MM
Emage
Eleerrctical
Pint
Semantic giided
Contrastive Learning
Figure Overview of the proposed SEAL framework.
Combining the supervised and unsupervised parts, we have:
+ "(""").)))")) - -Io(Z;: y1
Hol (1:H) (aa) Ho(((11:0)
(6)
I-IdZ; ) + 3 Ho H ( + HY() {.)]]
Therefore, from the perspective of information theory, incorporating semantic hierarchical labels provides strictly tighter upper bound the mutual information, which motivates to introduce the semantic-guided hierarchical learning framework for GCD.
4.2 SEAL: Semantic-Aware Hierarchical Learning for GCD
Building the advantages of semantic hierarchies, we propose the SEmantic-aware hierArchical Learning (SEAL) framework for GCD. The overall framework is outlined in Fig. 2. contrast to prior GCD approaches that either rely solely depend on abstract hierarchies [50, 64], we embed explicit semantic structure via three key elements: (I) a semantic-aware multi-task framework; (2) a cross-granularity consistency module align predictions across levels; and (3) hierarchical soft contrastive learning strategy to mitigate the "equivalent negative" assumption by weighting dissimilarity according semantic proximity.
granularanulrrity information [57, 59, 61, 66]
4.2.1 Semantic-aware Hierarchical Learning
We first introduce the semantic-aware multi-task framework. Inspired by [9], we advocate for joint learning framework across multiple semantic levels, allowing information across the hierar- chies to guide and strengthen representation learning at the target granularity. We define H as the number of semantic levels, with corresponding ground-truth labels yı...., . ordered from coarse to fine. Our multi-task architecture couples shared image encoder F followed by projection layer to disentangle features for various granularities, which can formulated as of F(x)) [[z Zzi :ZH where :;' denotes concatenation. Following the observa- tion in [9] that fine-grained features can benefit searse-grained predictions but not vice versa, we reuse lower-level features when computing coarse-level outputs. To avoid training bias towards coarse branches, we adopt gradient controller to include fine-level features without allow- ing gradient backpropagation. Formally, the aggregated feature of sample æ; at the h-th level is 2, = [i::.. Z +) ... [[(z )}. where I'(-) stops gradient propagation during training. We train a GCD classifier each level. For h-th level, the classification loss denoted Ch
2
4.2.2 Cross-Granularity Consistency Self Distillation
Although multi-level classification has been widely studied in closed-wond settings [9, 65], prior methods often treat each semantic level in isolation, leading to inconsistencies such as assigning labels like 'Shiba" and 'Cat' at different granularities for the same instance. This lack of cross-level interaction weakens the benefits of hierarchical learning. We address this with Consistency (CGC) module that distills information between granularities to keep predictions mutu- ally coherent. Coneretely, we add a self-distillation term that minimises the KI. divergence between the coarse-level posterior p(æ, (0,) and pseudo-coarse distribution obtained by mapping the target
crossaGranllartty

---

posterior p(z where 0 denotes the model parameters at granularity h. Specifically, we define dynamic transition matrix M, E R x* at granularity where n,, denotes the number of cate- gories at that level. Fach row of M, encodes how a fine-grained class distributes over coarse classes. For known fine-grained categories, this with uniform distribution and iteratively refine it during training (See Algo. Dynamic Update of MA). The pseudo-coarse probability thus can be computed as p(æ; (OH) x Mn and the hierarchical consistency loss at level h is defined as Dx (xl(p( On. (1))p((;)) (0)) x Ms). Summing across levels, the overall CGC loss becomes:
fixed one-hot vector; for novel classes, we initialize
Ma), where p(z; 0m) = ofoo, (a;); Te) with ot-) denoting the softmax operation and To be the consis- tency temperature and for (z;) being logits computed for granularity h.
Loge
Algorithm 1: Dynamic Update of Mr.
Input: Model f. number of class at level h, ny, known fine fine-grained classes Chase
Dynamic Update:
Compute logits I, f(D)
Compute prediction probability ph: Prr o1,, Te), TT)
for each fine class index not in Chase do
Compute fine-grained predictions yи argmax(ps)
Compute average probability distribution for samples predicted as fine class A: avg_h b ___p_bbb______ mean( ph [yH == =>))
Momentum update Mr [k] as follows: M,[A] + A M, kk + (1 A) avg hgghhppo h_pb
M, lk]
Normalize M, [k]: Mr [k] + Σ M
Output: Mn.
4.2.3 Hierarchical Semantic-guided Soft Contrastive Learning
To strengthen the discriminative capacity of representations in GCD, we propose Hicrarchical Semantic-guides Soft Contrastive Learning approach, addressing key limitations of existing con- trastive learning approaches. Prior GCD methods [23, 66, 59, 38] treat each non non-positive in mini- batch as an equally hard negative, ignoring semantic relatedness. We instead leverage the hierarchy in our multi-level framework to compute similarity-awan targets, assigning softer negative weights to semantically closer samples and preserving full penalties for unrelated ones. We compute pair- wise similarities within each mini-batch every semantic level, yielding similarity matrices S₁ at ZZ.(Z.)"
the h-th granularity, where S₁ = [[.. [ E IR BxB with Z₁ being the features of the mini-batch at granularity h and being the mini-batch size. Each fine-level matrix then fused with its coarser counterpart, yielding a hierarchical similarity matrix Sp. We then generate semantic-aware soft la- bels as a matrix: Ysoft, = (1 - Aa) + X, Sp, where I is the identity matrix and X controls the smoothness of the semantic-aware soft labels. The resulting semantic-guided hierarchical soft contrastive loss is defined as:
Ysofta (i,5) log exp(sim(z;; 23))
Chac
ΣΣΣ ==1
(8)
B
Im exp(sim(z;, 2%m))
where sim(-) represents the similarity metric between feature z; from æ; and feature z/ from the augmented view of wj, and Yooft, (i.j) refers to the (i.j) element of the soft label matrix. Unlike prior works [66, 61] that rely solely angle or distance-based measure, we adopt hybrid metric de-
z - where A is the weighting coefficient that linearly gradually decays during training. This design implements a carriculum learning strategy: begins with easier angle-based cues and gradually adds distance terms to refine representations. More ablation studies about the decay schedule are in the Appendix.
fined as sim (2,, 2/.) = A.2, - (1- Xxe 1 |

---

Uniee Land
Foendrr
Hennetied Sreverie gridel
Aonaddo Fed Iorahhe
Figure 3: Overview of Hierarchical cenaaticc-guiden Soft contrastive learning.
4.2.4 Overall Objective
Based on the baseline SimGCD [66] classifier, our framework designed to be trained a multi- task manner. We first replace the original InfoNCE loss [44] in the baseline representation loss Lepp introduced in Sec. by proposed hierarchical soft contrastive loss Ch denoting the resulting training objective at each granularity as L softrer"
The final training objective can be formulated
H
Call Σ (Choftrre + Cois) + Lege
Experiment
5.1 Experimental Setup
Datasets. We conduct comprehensive evaluation our method across variety of benchmarks. The main paper reports results the Semantic Shift Benchmark (SSB) [58]. which covers fine- grained datasets-CUP [60]. Stanford Cars [34]. and FGVC-Aircraft [42]-plus Oxford-Pet [46] and the more challenging Herbarium19 [55]. To gauge generalization
standard recognition tasks, (0/100(35) [35] and ImageNet-100 [13] in the
also include results on the generic benchmarks CIFAR 10/1 Appendix. For all datasets, follow the class split protocol of [57]. where subset of classes is selected as the known ('Old') label set )/. From these known classes, 50% the samples are used to construct the labelled set Di, and the remaining images with instances from novel classes form the unlabelled set D... Dataset statistics are summarized Tab. 1.
Evaluation metrics. We evaluate GCD perfor- mance using clustering accuracy (ACC), following standard practice [57]. Specifically, given ground- truth labels y, and predicted labels ÿ; for the unla- belled set D., the ACC is computed as:
Table 1: Overview of dataset, including the classes in the labelled and unlabelled sets
(V)) (Pa)) and counts images ( (Dr)), [P,)]
Dataset
Balance
[r
[
[P.]
ACC 1 D
CUB [60]
1.5K
100
4.5K
200
[Dal =
Σ
(10)
Stanford Cars [34]
2.0K
6.1K
196
FGVC-Aircrat [42]
1.7K
50
5.0K
100
Oxford-Pet [46]
0.9K
19
2.7K
37
where h denotes the optimal one-to-one mapping be- tween predicted clusters and true class labels. For a comprehensive evaluation, we report ACC sepa- rately for all classes ('All',, known classes ('Old"), and novel classes ("New').
Herbarium [55]
8.9K
341
25.4K
Implementation details. Following prior works [51, 66, 57], we adopt the ViT-B backbone [14],
initialized with pretrained weights from either DINO [7] or DINOv2 [45]. The model is trained for 200 epochs using a batch size of 128 and a cosine learning rate schedule, starting from an initial learning rate of 10-1 and decaying to 10-4 All experiments are performed on single NVIDIA L40S GPU with 24GB of memory. More details are provided in Appendix.

---

Table 2: Comparison of oftte-of-the-art GCD methods on SSB [58] benchmark. Results are reported in ACC across the 'All', 'Old' and 'New' categories. The highest and second-highest scores are indicated in bold and underline respectively.
CUB
Stanford Cars
FGVC-Aireraf
Average
Method
Venue
All
Old
New
All
Old
New
All
Old
New
All
k-means [41]
34.3
38,9
32.1
12.8
10.6
13.8
16.0
14.4
16.8
21.1
RankStatst [23]
ICLR20
33.3
51.6
24.2
28.3
61.8
12.1
26.9
36.4
22.2
29.5
UNO+ [18]
ICCV2/
35.1
49.0
28.1
35.5
70.5
18.6
40.3
56.4
32.2
37.0
ORCA [5]
CVPR22
35.3
45.6
30.2
23.5
50.1
10.7
22.0
31.8
17.1
26.9
GCD [57]
CVPR22
51.3
56.6
48.7
39.0
$7.6
29.9
45.0
41.1
46.9
45.1 1
XCon [17]
BMVC22
54.3
51.0
40.5
58.8
31.7
47.7
44.4
49.4
OpenCon [54]
TMLR23
54.7
63.8
54.7
49.1
78.6
32.7
PrompiCAL. [70]
CVPR23
64.4
62.1
50.2
52.2
$2.3
DCCL [48]
CVPR23
63.5
60.8
64.9
43.1
55.7
36.2
GPC [73]
ICCV23
52.0
55.5
47.5
38.2
58.9
27.4
43.3
40.7
44.8
44.5
DINOvI
PIM [12]
ICCV23
62.7
75.7
56.2
43.1
66.9
31.6
SimGCD [66]
ICCV23
60.3
65.6
57.7
53.8
71.9
45.0
54.2
59.1
$1.8
56.1
,GCD [59]
NeurlPS23
65.7
68.0
64.6
56.5
68.1
50.9
53.8
55.4
53.0
58.7
InfoSieve [50]
NewrIP523
69.4
77.9
65.2
55.7
74.8
46.4
56.3
63.7
52.5
60.5
TIDA [64]
NeurlP523
54.7
72.3
46.2
54.6
61.3
52.1
CIPR [26]
TMLR24
57.1
58.7
55.6
47.0
61.5
40.1
SPTNet [61]
ICLR24
68.8
65.1
59.0
79.2
61.8
58.1
Yang et al. [69]
ECCV24
61.3
60.8
62.1
44.3
58.2
39.1
AMEND [2]
WACV24
75.6
59.6
$2.8
61.8
48.3
73.3
48.2
LegoGCD [6]
CVPR24
63.8
71.9
59.8
57.3
75.7
48.4
55.0
61.5
51.7
58.7
MSGCD [16]
IF25
63.6
70.7
60.0
57.7
75.5
49.9
56.4
64.1
52.6
59.2
DebGCD [38]
ICLR25
66.3
71.8
63.5
65.33
81.6
$7.4
61.7
63.9
60.6
64.4
Ours
66.2
72.1
63.2
65.3
79.3
58.5
62.0
65.3
60.4
64.5
k-means [41]
71.1
29.4
16.9
19.9
38.6
GCD [57]
CVPR22
71.9
71.2
72.3
65.7
67.8
64.7
55.4
47.9
59.2
64.3
DINOV2
CiPR [26]
TMLR24
78.3
73.4
80.8
66.7
77.0
61.8
59.2
65.0
56.3
68.
SimGCD [66]
ICCV23
71.5
78.1
68.3
71.5
81.9
66.6
63.9
69.9
60.9
69.0
,GCD [59]
NewlPS23
74.0
75.9
73.1
76.1
91.0
68.9
66.3
68.7
65.1
72.1
SPTNet [6]]
ICLR24
76.3
79.5
74.6
DebGCD [38]
ICLR25
77.5
80.8
75.8
75.4
87.7
69.5
71.9
76.0
69.8
74.9
78.3
75.2
77.7
88.7
72.4
73.2
75.3
76.3
5.2 Main Results
We present benchmark results of our method and compare it with nineteen tfat--of-the-ar techniques in GCD as well as three robust baselines derived from novel category discovery. All methods are based on the DINO [7] and DI- NOv2 [45] pre-trained backbone. This comparative eval- uation encompasses performance on the fine-grained SSB
Table 3: Comparison with state-of-the-
art GCD methods on Herbarium [55]
and Oxford [46] on DINOvI.
Oxford-Pet
Hcrbarium19
Method
All Old New
All
Old
New
5-means [41]
77.1
70.1 80.7
13.0
12.2
13.4
benchmark [58], Oxford-Ped Pet [46] and Herbarium19 [55], as shown in Tab. and Tab. 3.
RankStutst [23]
27.9
55.8
12.8
UNO+ [18]
28.3
14.7
ORCA [5]
24.6
26.5
23.7
As shown in Tab. 2, our method consistently achieves state-of-the-ars performance on the SSB benchmark [58] based on both DINO [7] and DINOv2 [45] pretrained backbones. Specifically, under the DINOv2 setting, our approach reaches average 'All' accuracy of 76.3%, out- performing the previous best method, DebGCD [38], by 1.4% margin. Our framework demonstrates strong and stable improvements on both the Stanford Cars [34] and FGVC-Aireraft [42] datasets, achieving the highest accuracy under both backbone settings. This highlights the effectiveness of semantic- guided hicrarchical design and contrastive learning strategy, particularly in domains where the se- mantic hierarchy aligns closely with the underlying structure of man-made categories, such as vehi- cles and aircraft. On the CUB [60] dataset, although our method slightly lags behind DebGCD [38] and the non-pan con-marmetticc method InfoSieve [50], we attribute this gap the nature of bird taxonomy based on annotatee semantics, which may introduce inconsistencies absent in more system- atically defined hierarchies like those in artificial object domains.
GCD [57]
80.2
85.1
77.6
35.4
51.0
27.0
XCon [17]
86.7
91.5
84.1
OpenCon [54]
39.3
58.9
28.6
DCCL [48]
88.1
88.2
88.0
SimGCD [66]
91.7
83.6
96.0
44.0
58.0 36.4
μGCD [59]
45.8
37.2
InfoSieve [50]
90.7
95.2
88.4
40.3
59.0
30.2
DebGCD [38]
93.0
86.4
96.5
44.7
59.4
36.8
Ours
92.9
88.9
95.0
46.9
45.8
48.2
As shown in Tab. 3, our method achieves competitive performance on the relatively easier Oxford- Pet dataset [46], outperforming the bascline. More notably, the more challenging Herbarium19
8

---

Table 4: Ablations. The results regarding the different components in our framework on SSB Bench- mark [58]. ACC of All',, 'Old' and 'New' categories are listed. Red numbers indicate the improve- ment over the baseline.
Hirrachical Leurning
Consistency
Semanic guided
SCars
CUB
Airenit
Selt Distillation
HSCL
All
Did
New
All
Old
New
All
Old
New
baseline
53.8
71.9
45.0
60.3
65.6
57.7
54.2
59.1
51,8
(1)
57.5
67.1
52.9
57.0
57.8
56.6
52.8
56.4
51.0
(2)
62.6
78.3
55.0
57.8
56.6
57.5
57.0
63.5
53.8
64.4
77.5
58.1
62.5
67.5
60.0
57.4
58.5
56.8
Ours
65.3--1..51
79.36-740 5855-133)
66.26-599
63.244551 62,0(-780
65.3--63) 60.--80)
benchmark [55], it sets a new state-of-the-art by surpassing the previous best method, fGCD [59],
by 1.1% on the 'All' accuracy. These results highlight the robustness of our approach across both simple and complex open-world discovery scenarios.
5.3 Analysis
Component Analysis. We conduct ablation studies to analyse the contributions of each major component in our framework: Hierarchical Learning, Consistency Self-Distillatonn, and Hierarchi- cal Semantie-Guided Soft Contrastive Learning (HSCL). As shown in Tab. 4, we report results on the SSB benchmark [58], including Stanford Cars [34], CUB [60], and FGVC-Aireraft [42] datasets, evaluated over 'All', 'Old', and 'New" categories. Starting from the baseline trained solely with the GCD loss, we incrementally integrate the proposed components. Incorporating hicrar- chical learning alone (Row (1)) yields a modest improvement, particularly on the old categories. Adding consisenncy-bsse self-distillation (Row (2)) further improves alignment and stability, while mmnantic--uidee HSCL (Row (3)) significantly boosts performance novel classes by leveraging cross-instance semantic similarity. When all components are combined, the full framework achieves substantial gains with 11.5% on Stanford Cars, 7.8% on FGVC-Aircraft, and 5.7% CUB.
line with the practices in [66, 57]. we perform hyperparame- ter tuning using a held-out validation split from the labelled data. Specifically, we tune the con- sistency temperature Te and the soft negative controller A₂ based their performance the Stanford Cars [34] dataset. Detailed results across different hyperparameter values, evalu- ated on both the unlabelled training set and the validation split, are provided assess their im- pact on model performance. As shown in Tab. 5, we conduct a detailed grid search over the con- sistency temperature Te and the soft negative controller A on the Stanford Cars dataset. No- tably, the trends across both evaluation sets are highly consistent, with optimal performance achieved when T. tings yield the best balance between old and new class performance, highlighting the importance of carefully tuning both the consistency strength and the soft negative ratio in our framework.
Hyperparameter Tuning.
Table 5:
Experimental results regarding consis- tency temperature T. and ratio X, control the soft negative ratio on the unlabelled set and vali- dation set of Stanford Cars [34] dataset.
Unlabelled Set
Validation Set
Puram.
All
Old
New
All
Old
New
r. = 0.5
62.9
77.5
55.9
65.3
77.4
53.6
T. 0.75
65.3
79.3
58.5
66.4
77.3
55.9
1.0
61.6
73.9
55.7
63.7
75.3
52.6
7. 1.25
79.5
54.7
65.2
78.4
52.6
63.6
78.9
36.3
63.6
78.1
53.5
À. 0.1
63.9
78.5
56.9
65.2
78.0
$2.9
80.8
56.9
66.3
78.3
A. 0.8
64.4
78.1
57.8
66.
78.4
54.2
1.0
65.3
79.3
58.5
66.4
77.3
55.9
0.75 and X, 1.0. These set-
Semantic Dimensions. Semantic hierarchies are not re- stricted single dimension. To further demonstrate the flexibility of our framework, we additionally adopt LLM-generated labels along an alternative semantic di- mension, e.g., complementing the vehicle type hierar- chy (SUV/Van/Coupe) with brand-based hierarchy (Au- di/BMW). Tab. 6 demonstrates that our approach achieves consistently strong performance under both semantic hier- archies. This highlights the robustness and flexibility of our proposed use of semantie-guided hierarchies across different semantic dimensions, and underscores their im- portance for GCD, whether sourced from curated taxonomies, generated by L.LMs, or defined along alternative semantic structures.
Table Results on Scars with alterna- tive semantic hierarchies (vehicle brand vs. vehicle type) with DINOv2 pre- trained backbone.
Scars
Parum.
All
Old
New
SimGCD [66]
71.5
81.9
66.6
,GCD [59]
76.1
91.0
68.9
DebGCD [38]
75.4
87.7
69.5
SEALiVehicle Brand)
77.1
89.0
71.3
(Vehiie Type)
77.7
88.7

---

Visualization. We present t-SNE [56] visualization comparing the feature representations learned by the baseline and ours. For clarity, we randomly select 20 categories, including 10 from the "Old' set and 10 from the 'New" set. As shown in Fig. 4, our method yields tighter, better-separated clus- ters, indicating stronger inter-class discrimination. The zoomed view further reveals that the model preserves coarse-to-fine semantics: visually diverse subcategories within the broader 'Cab' group li close together, yet each remains distinct. This confirms that our method captures hierarchical structure while retaining fine-grained separability.
128
127
110
105
96
Cab
12
10
Baseline
Ours
Figure 4: i-SNE visualization of 20 classes randomly sampled from the Stanford Cars [34] dataset.
6 Conclusion
In this paper, we introduce a semantic-aware hierarchical learning framework for Generalized Cat- egory Discovery, composed of three key components. Firstly, design multi-task architecture that leverages naturally occurring semantic hierarchies to jointly learn coarse-to- category struc- tures. Secondly, propose Cross Cross-Granulan Granularity Consistency (CGC) module that distils information between levels, eliminating label conflicts across the hierarchy. Thirdly, we develop a Hierarchical Soft Contrastive Learning strategy that incorporates semantic similarity into the contrastive objec- tive, enabling fine-grained representation learning guided by structured semantic relationships. Our framework theoretically motivated by information theoretic principles, which highlight the ben- efit of incorporating hierarchical supervision to achieve tighter theoretical bounds. Evaluations on diverse fine-grained and generic benchmarks confirm consistent, state-of-the-art gains, demonstrat- ing both theoretical soundness and strong empirical performance.
information-theoretie
7 Discussion
Limitations. It is important to acknowledge a limitation concerning the scale of validation within our study. The dataset used for model evaluation includes fewer than 700 instances, which constrains the breadth of category coverage. This constrained sample size may not fully represent the diversity of categories encountered real-world scenarios. Consequently, the application of our model to category discovery in more complex and varied situations could be restricted. Further research with larger, more comprehensive datasets is warranted to validate the robustness our findings across a wider range of categories.
Broader Impacts. This work presents a feasible method for discovering novel categories in un- labelled data, potentially benefiting variety of applications such as robotics, healthcare, and au- tonomous driving, etc. However, there is a potential risk of misuse. The technology could be applied in surveillance to cluster unknown individuals, raising significant privacy concerns. Therefore, it is imperative to carefully consider ethical guidelines and legal compliance to address concerns regard- ing individual privacy. Additionally, to mitigate potential negative social impacts, the development of robust security protocols and systems crucial to protect sensitive information from cyberattacks and data breaches.
10

---

Acknowledgements
This work is supported by National Natural Science Foundation of China (Grant No. 62306251) 1),, Hong Kong Research Grant Council Early Career Scheme (Grant No. 27208022), Hong Kong Research Grant Council General Research Fund (Grant No. 17211024), and HKU Seed Fund for Basic Research.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.06777, 2023.
[2] Anwesha Banerjee, Liyana Sahir Kallooriyakath, and Soma Biswas. Amend: Adaptive margin and expanded neighborhood for efficient generalized category discovery. WACV, 2024.
[3] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: holistic approach to semi-supervised learning. In NeurIPS, 2019.
[4] Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Pi- antanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: cross- entropy vs. pairwise losses. In ECCV, 2020.
[5] Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. In ICLR, 2022.
[6] Xinzi Cao, Xiawu Zheng, Guanhong Wang, Weijiang Yu, Yunhang Shen, Ke Li, Yutong Lu, and Yonghong Tian. Solving the catastrophic forgetting problem in generalized category dis- covery. In CVPR, 2024.
[7] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
[8] Fernando Julio Cendra, Bingchen Zhao, and Kai Han. Promptced: Learning gaussian mixture prompt pool for continual category discovery. In ECCV, 2024.
[9] Dongliang Chang, Kaiyue Pang, Yixiao Zheng, Zhanyu Ma, Yi-Zhe Song, and Jun Guo. Your flamingo is my bird: Fine-grained, or not. In CVPR, 2021.
[10] Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks, 2009.
[11] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In CVPR, 2021.
[12] Florent Chiaroni, Jose Dolz, Ziko Imtiaz Masud, Amar Mitiche, and Ismail Ben Ayed. Para- metric information maximization for generalized category discovery. ICCV, 2023.
[13] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li. Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
[14] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
[15] Ruoyi Du, Jiyang Xie, Zhanyu Ma, Dongliang Chang, Yi-Zhe Song, and Jun Guo. Progressive learning of category-consistes multi aalti-granularity features for fine-grained visual classification. IEEE TPAMI, 2021.

---

[16] Yu Duan, Zhanxuan Hu, Rong Wang, Zhensheng Sun, Feiping Nie, and Xuelong Li. Mutual- support generalized category discovery. Information Fusion, 2025.
[17] Yixin Fei, Zhongkai Zhao, Siwei Yang. and Bingchen Zhao. Xcon: Learning with experts for fine-grained category discovery. In BMVC, 2022.
[18] Enrico Fini, Enver Sangineto, Stéphane Lathuiliere, Zhun Zhong, Moin Nabi, and Elisa Ricci. unified objective for novel class discovery. ICCV, 2021.
[19] GBl..org. Gbif occurrence download. https://doi. org/ 10tg/1..350000 35035/d9pk- 1162, 2025.
[20] Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In CVPR, 2018.
[21] Ross Girshick. Fast r-cnn. In 2015.
[22] Kai Han, Sylvestre-Alviss Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisser- man. Automatically discovering and learning visual categories with ranking statistics. In ICLR, 2020.
[23] Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zis- serman. Autonovel: Automatically discovering and learning novel visual categories. IEEE TPAMI, 2021.
[24] Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In ICCV, 2019.
[25] Qi Han, Zhibo Tian, Chengwei Xia, and Kun Zhan. Infomatch: Entropy neural estimation for semi-supervised image classification. In IJCAI, 2024.
[26] Shaozhe Hao, Kai Han, and Kwan Yee Wong. Cipr: An efficient framework with cross- instance positive relations for generalized category discovery. TMLR, 2024.
[27] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.
[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[29] Zhenqi He, Yuanpei Liu, and Kai Han. Category discovery: An open-world perspective. arXiv preprint arXiv:2509.22542. 2025.
[30] Zhenqi He, Mathias Unberath, Jing Ke, and Yiging Shen. Transnuseg: A lightweight multi-task transformer forănuclei segmentation. In MICCAI, 2023.
[31] Xuhui Jia, Kai Han, Yukun Zhu, and Bradley Green. Joint representation learning and novel category discovery on single-and multi-modal data. In ICCV, 2021.
[32] KJ Joseph, Sujoy Paul, Gaurav Aggarwal, Soma Biswas, Piyush Rai, Kai Han, and Vinceth N Balasubramanian. Novel class discovery without forgetting. In ECCV, 2022.
[33] Kevin Karbstein, Lara Køsters, Ladislav Hodač, Martin Hofmann, Elvira Hörandl, Salvatore Tomasello, Natascha D Wagner, Brent C Emerson, Dirk C Albach, Stefan Scheu, al. Species delimitation 4.0: integrative taxonomy meets artificial intelligence. Trends in Ecology & Evo- lution, 2024.
[34] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In ICCV workshop, 2013.
[35] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[36] Nico Lang, Vésteinn Snæbjarnarson, Flijah Cole, Oisin Mac Aodha, Christian Igel, and Serge Belongic. From coarse to fine-grained open-set recognition. In CVPR, 2024.
12

---

[37] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, 2014.
[38] Yuanpei Liu and Kai Han. Debged: Debiased learning with distribution guidance for general- ized category discovery. In JCLR, 2025.
[39] Yuanpei Liu, Zhenqi He, and Kai Han. Hyperbolic category discovery. In CVPR, 2025.
[40] Georgina M Mace. The role of taxonomy in species conservation. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 2004.
[41] James MacQueen et al. Some methods for classification and analysis of multivariate observa- tions. In Proceedings of the fifth Berkeley symposium mathematical statistics and probabil- ity, volume I, pages 281 291- Oakland, CA, USA, 1967.
[42] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.515// 2013.
[43] George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 1995.
[44] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. ArXiv e-prints, 2018.
[45] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khali- dov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint mrXiv:2304.07193, 2023.
[46] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, 2012.
[47] Jizong Peng, Marco Pedersoli, and Christian Desrosiers. Mutual information deep regulariza- tion for semi-supervised segmentation, In Medical imaging with deep learning, 2020.
[48] Nan Pu, Zhun Zhong, and Nicu Sebe. Dynamic conceptional contrastive learning for general- ized category discovery. In CVPR, 2023.
[49] Yanyun Qu, Li Lin, Fumin Shen, Chang Lu, Yang Wu, Yuan Xie, and Dacheng Tao. Joint hi- erarchical category structure learning and large-scale image classification. IEEE Transactions on Image Processing, 2017.
[50] Sarah Rastegar, Hazel Doughty, and Cees Snoek. Learn to categorize or categorize learn? self-coding for generalized category discovery. In NeurlPS, 2023.
[51] Sarah Rastegar, Mohammadreza Salehi, Yuki M Asano, Hazel Doughty, and Cees G M Snoek. Selex: Self-expertise in fine fine-grained generalized category discovery. ECCV, 2024.
[52] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NewrIPS, 2015.
[53] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[54] Yiyou Sun and Yixuan Li. Opencon: Open-world contrastive learning. TMLR, 2022.
[55] Kiat Chuan Tan, Yulong Liu, Barbara Ambrose, Melissa Tulig, and Serge Belongie. The herbarium challenge 2019 dataset. arXiv preprint 1rXiv:1906.0$372, 2019.
[56] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008.
[57] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In CVPR, 2022.
[58] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. The semantic shift benchmark. In ICML workshop, 2022.
13

---

[59] Sagar Vaze, Andrea Vedaldi, and Andrew Zisserman. No representation rules them all in category discovery. In NeurIPS, 2023.
[60] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- uesd birds-200-2011 dataset. 2011.
[61] Hongjun Wang, Sagar Vaze, and Kai Han. Sptnet: An efficient altemnatvee framework for generalized category discovery with spatial prompt tuning. In ICLR, 2024.
[62] Hongjun Wang, Sagar Vaze, and Kai Han. Hilo: A learning framework for generalized category discovery robust domain shifts. In ICLR, 2025.
[63] Xinlong Wang. Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li. Solo: Segmenting objects by locations. In ECCV, 2020.
[64] Yu Wang, Zhun Zhong, Pengchong Qiao, Xuxin Cheng, Xiawu Zheng, Chang Liu, Nicu Sebe, Rongrong Ji, and Jie Chen. Discover and align taxonomic context priors for open-world semi- supervised learning. In NeurlPS, 2023.
[65] Xiu-Shen Wei, Yi-Zhe Song, Oisin Mac Aodha, Jianxin Wu, Yuxin Peng, Jinhui Tang, Jian Yang, and Serge Belongie. Fine-grained image analysis with deep learning: survey. IEEE TPAMI, 2022.
[66] Xin Wen, Bingehen Zhao, and Xiaojuan Qi. Parametric classification for generalized category discovery: A baseline study. In ICCV, 2023.
[67] Tz. Ying Wu, Chih-Hui Ho, and Nuno Vasconcelos. Protect: Prompt tuning for taxonomic open set classification, In CVPR, 2024.
[68] Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In ICCV, 2021.
[69] Fengxiang Yang. Nan Pu, Wenjing Li, Zhiming Luo, Shaozi Li, Nicu Sebe, and Zhun Zhong Learning to distinguish samples for generalized category discovery. In ECCV, 2024.
[70] Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, and Fa- had Shahbaz Khan. Promptcal: Contrastive affinity learning via auxiliary prompts for gen- eralized novel category discovery. CVPR, 2023.
[71] Shu Zhang, Ran Xu, Caiming Xiong, and Chetan Ramaiah. Use all the labels: A hierarchical multi-label contrastive learning framework. In CVPR, 2022.
[72] Bingchen Zhao and Kai Han. Novel visual category discovery with dual ranking statistics and mutual knowledge distillation. In NeurlPS, 2021.
[73] Bingchen Zhao, Xin Wen, and Kai Han. Learning semi-supervised gaussian mixture models for generalized category discovery. In ICCV, 2023.
[74] Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, and Nicu Sebe. Neigh- borhood contrastive learning for novel class discovery. In CVPR, 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main contributions and scope introduce comantic-guidden hierar- chical framework for generalized category discovery motivated by theoretical motivation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We provide the discussion of limitations in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, a facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: We provide theoretical result in the main paper, and detailed proofs and full set of assumptions are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proof' can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data provided not)?
Answer: [Yes]
Justification: All the experimental results in this paper are reproducible. We will release the codes and guidelines for reproducing the results after acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
16

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We will release the codes in the attached link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for details.
While we encourage the release of code and data, understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips wwtpccccppcccccec.cccc.cccccccct htt///gddds/eeeeeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide details of experimental settings including type of optimizer, model architecture used, data splitting, hyperparameters ,and how they were chosen in Sec. 5 and provide more details the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [No]
Justification: No. Our paper does not report error bars statistical significance metrics.
This decision is consistent with the standard practice in the Generalized Category Discovery (GCD) literature.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details about the computer resources used in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http//eeurpp..p//ee//ee
Answer: [Yes]
Justification: Ethics.
All research conducted in this paper conform with the NeurIPS Code of
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide the Broader impacts in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (c.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not introduce or release any models or datasets that present high risk of misuse. All models are trained publicly available, curated datasets (e.g., CIFAR, ImageNet, CUB) with clear licenses, and our contributions are limited to standard classification or contrastive learning techniques without generating sensitive content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
allow for controlled use of the model, for example by re-
Datasets that have been scraped from the Internet could pose safety risks. The authors
should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets used in our workineluding datasets (c.g.. CUB, Stanford Cars) and pretrained models (e.g., DINOv2, DINOv2)are publicly available. We have cited the corresponding papers and sources in our references.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
19

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license. copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datascts. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer:
[NA]
Justification:
We do not release new assets in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not include any crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //nurruupp.... http:///aeeeeeccc/eeeeeeeee///// cccConnferenccc//225////////3// for what should or should not be described.
21

---

Appendix
A Theoretical Perspective
A.I Notations & Definitions
A.2 Assumptions
A.3 Theoretical Motivations
Additional Details
B.1 Additional Implementation Details
B.2 Additional Dataset Details
Experiments under Realistic Situation
D Analysis on using randomly generated coarse-level labels
Results on Generic Datasets
F Analysis on the Depth of Semantic Hierarchies
Analysis of Computational Costs
Analysis on the Curriculum Learning Schedule
22

---

A Theoretical Perspective
In this section, we provide an information-theorehle proof motivating the design of SEAL.
A.1 Notations & Definitions
Mulual Information (MI) quantifies the reduction in uncertainty of one random variable given knowl- edge of another. We have the following definitions for MI:
Definition. The MI between two continuous random variables X and Y is formulated as.
dy dz 1(x = ISaw Pxy (2, y) log (x py (v)
(x, y)
(11)
Definition, The MI between two discrete random variables X and Y is formulated as.
((;Y)= = Exxy log Pxy (x,
(12)
(x (1
2.y
The notation we used and the related formulas are given Tab. A1 Table A1: Definition of the random variables and information measures used in this paper.
General
Labelled dataset
Dj {(x; Vi )
Unlabelled dataset
Du = {(I;, Vi) mm
Image data space
X'
Embedded feature space
Z IRd
Label/Prediction space
y/5 C IRKK
Euclidean distance
Dij |1;
3t ll
Cosine distance
Dooa,s
21. xy 22
Model
Encoder
fo X + 7
Soft-classifier
H 2 10, 11
Random variables (RVs)
Data
X (X,, X.). Y = VY) Z|X'' fo(X)
Embedding
Prediction
81z H(Z))
Information measures
Mutual information between Z and Y
I(Z;Y) = H(YY) - H((
Entropy of Y
H(Y) = Epv
logpy (Y)
Conditional entropy of given Z
H(YY Z) Epyz
log iz (YIZ))
Cross entropy (CE) between Y and V
H(Y:Y) = Epy
log (Y))
Conditional CE given Z
H(Y: = Eppr
log pvi z (Y|2)
A.2 Assumptions
The following assumptions are made in our proof.

---

A.1 Independent sampling between (X,, Yi) and (Xu, Yu), which is written as (X,, TL (Xu, Yu). A.2 Same data distribution for (X,, Yi) and (Xu, Y.) the labelled and unlabelled data follow the same underlying data distribution e-g. same domain.
A.3 The representation mapping Z (X) is deterministic and per-sample independent given parameters
A.3 Theoretical Motivations
As shown in [4], from the view of information theory, the optimization objective of discriminative tasks is equivalent to maximising the MI between the learned latent features Z and Y, which is:
max (Z( < min Io)
(13)
While this objective operates under the closed-world assumption, which assumes the availability of all annotations for the training data-in the GCD setting, both labelled and unlabelled data are present during training. Therefore, we further decompose the learning objective for GCD as follows. Given the chain rule for MX; Y1, YI,...,,,, .Y.. Ei= I(X; Yi Y1s-1), we can extend To (Z;YY) as:
Io(Z;Y) Io (Z, Zui D1,
Is Zu: 21) + Io Zu: V. 31) Io( + Io (233) + Io Zui Vu (V)
(14)
(z( (z(2. + Jo (1)) + Lz Zz Vu. Vi).
where as (Z, Yi) 1L (Zu, Y.) + p(a, Mr, Zu, Ba) p(z,, an) p(zu, vn). we have:
1 [[ Vu (34) By: (L((p(2) Mo 180) p wi) p(uu m))
p(21 31 Uu)
Eu KI(
p((2 ()) p(va w))
p(v))
pl
2I, M1 (((
KT
pp() 31) p(a w))
p(())
(15)
-E
KT(p(z 31) p(yn)
p(=) ai) p(y. u)) (By Bayes Rule)
= Ey KL(p(z p(y.) p(a wi) p(Hw)) (By Independency)
=0
As the two arguments of the KL divergence are identical, we finally have T(Z;; (Z)iiiD)i)) = 0. Similarly, for 10( L.. Jt), we have:
I(Z Z(:))) I(Di; Z. Zt) = Ezi KK(p(e, zu 2x) || p(z, 21))
KT p 21, =u)
|| 21) p(zu =i)]
E.,
p(()
E.g KT p(u, 21 ((2u)
p(v P(uu =1)
p(z))
(16)
E2,
KL(p1m (l)
p 2)) p(z, 21)
(By Bayes Rule)
E., KL(p(1 p))())))))))))))))))))) p(un 21) p)))(())) (By Independency)
= 0.
By the independency assumption, have Therefore, the optimization objective is decomposed to:
I(Z.:
min Io(Zi;31) Io Zw; V.))
(17)

---

We further introduce weight factor , [47, 25] to balance between the supervised and unsupervised part formulate the final objective as:
min To(Z 3I (To(Za: D.).
(18)
where for Io(Zu: Vu unknown, we introduce a variational label distribution based on model prediction qo (x.) - pe (Y.......) where Y is the softmaxed model prediction. By the data- processing incquality that information passes through transformation, mutual information with the source cannot increase, we have Io Zo(Zu:Ja, 2 (X u; Pu). From which, we can rewrite:
min (Zn; 7 min - (x P.) - min Ho()))) + Ho(Sin X.).
(19)
Thus, the overall optimization objective can be reformulated as:
- To. 74; V1) + B + -HHH.) + Hd 12.)
min 8
(20)
where 3 is the weight factor to balance labelled and unlabelled parts.
Assuming the coanse-rrineed semantic hierarchical labels 21 ..... 1("11)) are accessible, the objec- tive naturally extends to:
min 8
Zz: y
) + u
X..) - Ho(P
(21)
supervised purt
unsupervised part
By applying the chain rule, first decompose the supervised part as:
10 (((1:))221)))
( 21(
Σ Lo( Z1; ((()
((10)))
(22)
H-1
2 [d(Z))))))) as
Σ
lol (( | y((utt))
)( 2 0.
h=1
where ΣØE l0 (o(Zi:)(())
0 comes from the below. For h E {1... H --1..
we have:
1(11;)(1)) | so(ht2) ((t
KT(P 2 vf 100D000
P Pzipa n:00 me Pygn)) wtth 112000
(23)
0. (Non-negutivity of KL divergence)
Similarly, for the unsupervised part, we can also obtain:
#-1 Σ h=1
Ho(Y! Xx)) Ho(Y(i To( ((
No( (1h) (((h1im))
(24)
< To(:u; V(H)))((H)
- -Ho(Pa) + HolV., Xx).
From the above, we now have:
min { Id 1n )...)) + (P( | HdP
(25)
min { Zo V1) + 3 { (H)) + Ho()i, {.
|
where that the semantic-guides hierarchies provide tighter bound the mutual infor- mation, which motivates us to introduce the semantic-guider hierarchical learning framework for GCD.
3

---

B
Additional Details
B.1 Additional Implementation Details
We adopt the class splits of labelled ('Old") and unlabelled ('New") categories in [37] for generic object recognition datasets (including CIFAR- 10 [35] and CIFAR-100 [35]) and the fine aie-grained Se- mantic Shift Benchmark [58] (comprising CUB [60]. Stanford Cars [34], and FGVC-Aireraft [42]), Oxford-Pet [46] and Herbarium19 [55]. Specifically, for all these datasets except CIFAR-100, 50% of all classes are selected 'Old' classes (3)), while the remaining classes are treated 'New' classes (2 (.). For CIFAR-100, 80% the classes are designated as 'Old' classes, while the remaining 20% 'New' classes. Moreover, following [57] and [66], the model's hyperparameters are chosen based on its performance on hold-out validation set, formed by the original test splits workstation with Nvidia L40s GPUs. The models are trained with a batch size of 128 on a single GPU for all datasets.
of labelled classes in each dataset. All experiments utilize the PyTorch framework
For the hierarchical information required by our framework, we rely exclusively on publicly avail- able taxonomies well-established datasets rather than any manual annotation. For the fine-grained SSB benchmarks [58], we follow the closed-workee hierarchies of [9]: CUB [60] is organised into 13 orders, 38 families, and 200 species; Stanford Cars [34] is structured into car types (e.g., 'Cab', "SUV) and 196 specific models; FGVC-Aircraft [42] is arranged into 30 makers (e.g., 'Boeing', "Douglas); 70 families (e.g., 'Boeing 767'), and 100 models. Oxford Pets [46] is re-cast into two- level hierarchy with the coarse level "Cat' vs. 'Dog', while Herbarium 19 [55] is grouped by coarser- grained genus using the GBIF botanical database [19]. For generic benchmarks, CIFAR-10 [35] is split into the super-classes 'Vehicle' and 'Animal', CIFAR- 100 [35] adopts its built-in 20 super- classes, and ImageNet- 100 [13] leverages the WordNet [43] taxonomy to form coarse categories. All hierarchies are obtained via public code, openly accessible biological and lexical databases or can be generated by LLMs, ensuring that our experiments reflect realistic usage without bespoke
curation.
B.2 Additional Dataset Details
Table A2: Overview of datasets we use, including the classes in the labelled and unlabelled sets (D))) DaD and counts of images ((D:), (Pa)). The 'FG' indicates whether the dataset fine-grained.
Dataset
FG
[Di]]
Di
[Du
CIFAR- 10 [35]
X
12.5K
5
37.5K
10
CIFAR- 100 [35]
X
20.0K
80
30.0K
100
ImageNet- 100 [13]
X
31.9K
50
95.3K
100
CUB [60]
V
1.5K
100
4.5K
200
Stanford Cars [34]
/
2.0K
98
6.1K
196
FGVC-Aireraft [42]
/
1.7K
50
5.0K
Oxford-Pet [46]
0.9K
19
2.7K
37
Herbarium19 19 [55]
/
8.9K
341
25.4K
683
We further introduce the details of the datasets used in our paper. The statistics for the commonly
used datasets are summarized in Tab. A2.
Generic Datasets. (1) ImageNet- 100 [13] is a widely used dataset for natural image classification in
computer vision, which is constructed by randomly subsampling 100 classes from ImageNet-1K: K. (2) CIFAR-10 & CIFAR-100 [35] are both natural images sized in 32 x 32. CIFAR-10 contains 50, 000 images spanning across 10 different classes and CIFAR-100 includes 100 classes, with each class containing 500 images.
Fine-grained Datasets. The most widely used fine- orainee benchmark is SSB [57]. which includes
three datasets: CUB [60], Stanford Cars (SCars) [34], and FGVC Aircraft [42]. (1) CUB [60] is a widely used benchmark dataset for fine-grained visual classification tasks, particularly focused on bird species recognition. (2) Stunford Cars [34] is a large-scale dataset designed for fine-grained vehicle classification tasks. It contains 196 different car models, primarily spanning various makes, models, and years. (3) FGVC-Airerufi [42] is a fine-grained visual classification dataset focused
on aircraft recognition. It contains 10,000 images spanning 100 different aircraft model variants,

---

with each image labelled by its corresponding model. (4) Oxford-Per [46] is a large, fine-grained dataset designed for pet image classification and segmentation tasks. (5) Herbarium19 19 [55] is a large- scale image collection focused on plant species identification, particularly for herbarium specimen recognition.
5

---

C Experiments under Realistic Situation
Following the majority of the literature, we conduct experiments mainly using the ground-truth category numbers. In this section, we test SEAL under more realistic conditions where neither coarse-granularity labels nor the number of classes are known. We adopt the same constraints used in earlier GCD works [57, 66]: only the known fine-grained classes are revealed. We first estimate the total number of ungeted-granulare automatically derive coarse-level names and the fine-to-coarse mapping using ChatGPT-40 [1] with the following prompt: " ([Tareeted-granee
categories with an off-the-shelf method [57]. Next, we
Category Names}" I provide these (Number of known category) fine -inennr class names, please generate the corresponding coarse-graines labels for me. After obtaining the coarse-granularity labels, we run the estimator [57] to infer the number coarse categories. We test under such realistic condition for one fine-gr ane-grained dataset (Stanford Cars) and one generic datasets (CIFAR 100) and report the estimated class number about different granularities Tab. A3. We compare SEAL with SimGCD [66], ,GCD [59], and GCD [57] in Tab. A4, Even in this realistic scenario with an unknown number categories and automatically generated granulargaallrrtt labels, our method outperforms existing approaches across both datasets. These results demonstrate that SEAL can be effectively deployed without any manual access to higher-level labels or class counts, while still achieving state-of-the-an -hr accuracy.
Table A3: Estimated class numbers the unlabelled data using the method proposed in [57] for both target granularity and coarse granularity.
SCars (Target) SCars (Coarse) CIFAR- 100 (Target)
CIFAR-100 (Coarse)
Gmund-truth Estimated
100
100
20
Table A4:
Results under the realistic scenario where neither grarse-granularity labels nor the number
of classes are known. The estimated class numbers in Tab. A3 are adopted for all methods.
Stanford Cars
CIFAR-100
Method
Old
All
New
[S7]
35.0
36.0
24.8
73.0
76.2
66.5
SimGCD [66]
49.1
65.1
41.3
80.1
81.2
77.8
,GCD [59]
56.3
66.8
51.1
Ours
62.4
78,9
54.5
82.1
81.7
83.0
D Analysis on using randomly generated coarse-level labels
To further substantiate our motivation that incorrect hierarchies may introduce misleading supervi- sion, we conduct experiment in which the true coarse-level labels are replaced with randomly generated hierarchies. Specifically, we evaluate under two settings: 100% random and 50% random. As shown in Tab. A5, performance drops sharply across CUB, SCars, and Aircraft under both vari- ants of randomly assigned hierarchical labels. This observation indicates that our gains arise from the semantic alignment of the hierarchy, not from the mere presence of a hierarchical structure.
Table A5: Ablation on randomly generated coarse-level labels.
CUB
SCars
Airerafi
All
Old
New
All
Old
New
All
Old
New
100% Random
30.3
31.6
29.7
29.6
31.4
28.7
33.2
31.3
34.2
50% Random
51.2
50.3
51.7
48.5
50.1
47.7
40.7
39.6
41.3
SEAT
66.2
72.1
63.2
65.3
79.3
58.5
62.0
65.3
60.4

---

E
Results on Generic Datasets
Table A6: Comparison of state-of-the-arr GCD methods on generic datasets. It includes CIFAR-
10 [35],
CIFAR-100 [35]. ImageNet-100 [13], and the average ACC on All categories.
CIFAR-I0
CIFAR-I00 100
ImageNet-100 100
Average
Method
All
Old
New
All
Old
New
All
Old
New
All
k-means [41]
83.6
85.7
82.5
52.0
52.2
50.8
72.7
75.5
71.3
69.4
RunkStats+ [23]
46.8
19.2
60.5
58.2
77.6
19.3
37.1
61.6
24.8
47.4
UNO+ [18]
68.6
98.3
53.8
69.5
80.6
47.2
70.3
95.0
57.9
69.5
ORCA [5]
69.0
77.4
52.0
73.5
92.6
63.9
81.8
86.2
79.6
74.8
GCD [57]
91.5
97.9
88.2
73.0
76.2
66.5
74.1
89.8
66.3
81.1
XCon [17]
96.0
97.3
95.4
74.2
81.2
60.3
77.6
93.5
69.7
82.6
DINOv!
OpenCon [54]
84.0
81.2
PrompiCAL.
97.9
96.6
98.5
81.2
84,2
75.3
83.1
92.7
78.3
87.4
DCCL [48]
96.3
96.5
96.9
75.3
76.8
70.2
80.5
90.5
76.2
84.0
GPC [73]
90.6
97.6
87.0
75.4
84.6
60.1
75.3
93.4
66.7
80.4
SimGCD [66]
95.1
98.1
80.1
81.2
77.8
83.0
93.1
77.9
86.7
InfoSieve [50]
94.8
97.7
93.4
78.3
82.2
70.5
80.5
93.8
73.8
84,5
CiPR [26]
97.7
97.5
97.7
81.5
82.4
79.7
80.5
84.9
78.3
86.6
SPTNet [61]
97.3
95.0
98.6
81.3
84.3
75.6
85.4
93.2
81.4
88.0
DebGCD [38]
97.2
94.8
98.4
83.0
84.6
79.9
85,9
94.3
81.6
88.7
Ours
97.2
94.7
98.4
82.1
81.7
83.0
84.6
90.9
81.3
88.0
GCD [57]
97.8
99.0
97.1
79.6
84.5
69.9
78.5
89.5
73.0
85.3
DINO22
CIPR [26]
99.0
98.7
99.2
90.3
89.0
93.1
88.2
87.6
88.5
92.5
SimGCD [66]
98.7
96.7
99.7
88.5
89.2
87.2
89.9
87.1
92.4
SPTNet [61]
90.1
96.1
87.1
DebGCD [38]
98.9
97.5
99.6
90.1
90.9
88.6
93.2
97.0
91.2
94.1
Ours
98.9
98.1
99.3
89.8
90.4
89.5
91.3
93.3
90.3
93.3
Tab. A6 shows that SEAL remains effective even when only shallow hierarchies are available. With using both DINO [7] abd DINOv2 [45] pre-trained backbones, SEAL surpasses the strong SimGCD [66] baseline on all three datasets- datasets-CIFAR-I CIFAR-I00 and ImageNet-100. For generic datasets, they provide only coarse and heterogeneous groupings (e.g., 'Animal' vs. 'Vehicle' in C1), so the hierarchy does not converge to a common parent class. By contrast, fne-grainedd datasets like CUB [60] share a clear taxonomic root (e.g., the class 'Aves' for all bird species), al- lowing method to exploit deeper and more coherent semantic structure. Even under this less favourable condition, SEAL still delivers competitive performance, confirming the robustness of hierarchical design.
F Analysis on the Depth of Semantic Hierarchies
Sec. B.1 notes that framework uses different hierarchical depths depending dataset availabil- ity. To quantify the effect of depth, we conduct an ablation study on the two datasets that provide three explicit levels, including CUB [60] and FGVC-Aireraft [42]. For each dataset we compare: (i) a single-level baseline that uses only the target granularity, (ii) two-level version that adds one parent level, and (iii) the full three-level setting adopted in the main paper. Tab. A7 shows that SEAL remains effective irrespective of the number of available semantic levels. When compared to the giaggllgaaullrrtty baseline across both datasets, the incorporation of just one additional coarse- granularity level yields improvements of approximately 2% and 4%. These results demonstrate the robustness of our design, which can leverage richer hierarchies when they are present, while still providing significant benefits regardless of the number of hierarchies utilized.
Table A7: Ablations analysis on the depth of semantic hierarchies. ACC of 'All', 'Old' and 'New' categories on Stanford Cars and FGVC-Aireraft are listed.
Depth of Hierarchies
Coarser Hierachyy
CUB
RGVC-Airerai
All
Old
New
All
Old
New
(i) Baseline
60.3
65.6
57.7
54.2
59.1
51.8
Gi)
2
Family
63.5
73.9
58.3
58.4
63.6
55.8
Order Maker
62.3
72.3
57.3
58.6
60.7
58.3
Gi)) SEAL
3
Order/Maker + Family 66.2(+5.9) 72,1 (+6.5)
63.2 (45.5)
62.0 (+7.8)
(46.22
60.4 (48.6)
7

---

G Analysis of Computational Costs
Tab. A8 presents a comprehensive analysis of the computational cost associated with our method compared the SimGCD baseline [66] for the training stage. Despite incorporating additional multi-level supervision, our approach introduces minimal computational overhead during training. Specifically, the number of parameters increases by less than 9% across all datasets (from approxi- mately 630 MB to 688 MB), and the GFLOPs remain virtually identical, showing only a marginal increase from 17.59 to 17. .660 Importantly, the training efficiency is largely preserved, with time per epoch on the unlabelled dataset increasing by no more than 0.7 seconds in all cases. These results clearly demonstrate that our framework achieves its performance improvements without sacrificing computational efficiency in training stage. At inference, however, we discard the coarse granulagaall branches and keep only the classifier for the target granularity. The cost breakdown in Tab. A re- veals that our model actually uses fewer parameters than the SimGCD baseline [66]. Runtime on the unlabelled test sets reduced for all three datasets Stanford Cars [34], CUB [60], and FGVC- Aircraft [42]. This economy stems from our design: single MLP projector separates features across levels without enlarging the overall feature dimension, so the target-level head compact at test time. Consequently, our method introduces almost no overhead during training and even lowers the computational footprint at inference, while still boosting accuracy.
Table A8: Computational cost analysis with baseline during training.
Params (MB))
GFLOPs|
Time per Epoch (s)
Method
SCars
CUB
Aircratt
SCars
CUB
Aireraft
SCars
CUB
Aircraft
SimGCD [66]
630.9
630.9
630.6
17.59
17.59
17.59
59.2
25.0
74.9
660.5
688.0
688.1
17.60
17.60
17.60
59.8
25.1
75.6
Table A9:
Computational cost analysis with baseline at inference time.
Params (MB)
GFLOPs]
Time per Epoch (s)..
Method
SCars
CUB
Aircraft
SCars
CUB
Aireraft
SCars
CUB
Aircraft
SimGCD [66]
630.9
630.9
630.6
17.59
17.59
17.59
56.9
34.1
53.1
Ours
629.0
627.6
627.5
17.59
17.59
17.59
56.8
34.0
52.9
H Analysis on the Curriculum Learning Schedule
As introduced in Sec.4.2.3, we employ a linear decay schedule for Ae. Tab. A10 reports an abla- tion study the decay strategy for the curriculum weighting coefficient Xe. We compare fixed values (A₂ = 0, 0.5, 1), and exponential decay schedule, and our proposed SEAL with linear de- cay. The results consistently show that decaying schedules outperform fixed baselines, validating the effectiveness of progressively shifting focus from coarse semantic alignment to finer positional discrimination. In particular, SEAL achieves the best comparable performance across three fine- grained datasets (CUB, Stanford-Cars and Aircraft), demonstrating that the linear decay schedule provides more stable and effective curriculum learning design.
Table A A10: Ablation on curriculum decay strategies.
CUB
SCars
Aircraft
All
New
Old
New
Old
New
Ae = 0
64.6
72.2
60.8
64.5
80.8
56.6
59.3
61.8
58.0
Xe = 0.5
65.1
72.8
61.3
64.1
77.1
57.8
58.9
62.5
57.1
X 1.0
64.9
72.4
60.9
63.2
80.2
55.0
58.7
65.4
55.3
Exp Decay
66.1
71.7
63.3
66.3
81.2
60.3
61.2
63.1
60.2
SEAL
66.2
72.1
63.2
65.3
79.3
58.5
62.0
65.3
60.4
8


Reviews:
[{"review_id": "t6BlJOVO4m", "reviewer": "Reviewer_8DjF", "summary": "This paper proposes the Semantic-aware Hierarchical Learning framework (SEAL), marking the first utilization of naturally biological classification system to guide Generalized Category Discovery (GCD) tasks. It addresses the limitations of existing approaches that rely on manually designed abstract hierarchies through three components: Semantic-aware Hierarchical Learning, Cross-Granularity Consistency Self Distillation, and Hierarchical Semantic-guided Soft Contrastive Learning.", "questions": "See Major Weaknesses", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Paper Strengths：**\n\n1.\tThe first framework to introduce biological classification systems into GCD, replacing error-prone manually designed abstract hierarchies. \n2.\tMoreover, compared with the similar work DebGCD, the effect is significantly better. \n3.\tThis work will have a good effect in data with a clear natural hierarchy\n4.\tThe ablation experiment is sufficient.\n\n**Major Weaknesses:**\n\n1.\tStatic Hierarchy Dependency: \nThe reliance of static biological classification on predefined semantics limits its adaptability to the open world\n\n2.\tLabel Sensitivity:\nI notice that: 'When the real semantic structure conflicts with the preset hierarchy, performance drops sharply (e.g., non-standard biological classification)'. This also constitutes a significant limitation.\n\n*Minor Weaknesses:*\n\nI would recommend beautifying the images in the article.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thoughtful and positive feedback. We greatly appreciate the reviewer's recognition of the value  of  **first introducing biological classification systems into GCD**,  **clear advantage over similar work DebGCD**, and **sufficient ablation studies**. Our detailed responses to the listed concerns are provided below. We will also further strengthen the final manuscript based on the reviewers' concluding feedback. A well-documented code together with all trained models will be made public.\n\n**W1 Static Hierarchy Dependency**\n\nWe would like to clarify that the semantic hierarchies leveraged in our framework are **adaptable** and **well-aligned with real-world structures**, rather than rigid or artificially imposed. \nIn biological sciences, hierarchical classification arises naturally and serves as a foundational principle in taxonomy. For example, in botanical research, taxonomists commonly use labelled specimens of known species to classify newly collected, unlabelled samples into existing taxonomic hierarchies or to identify novel species [a,b]. \n\nSimilarly, in our work, we assume that hierarchical labels are available **ONLY** for the seen categories, which is a widely adopted and realistic assumption. Such labels are commonly used and readily available in many prior studies[9,15,40]. In practice, they can be obtained directly from curated resources (e.g., the GBIF botanical database [Supp.9]) or automatically generated using large language models (LLMs), as shown in our supplementary material B.1. This design not only aligns with the natural structure of biological classification systems but also enhances the interpretability of open-world learning.\n\nTo further demonstrate the practicality of our setting, we test the use of LLM-generated hierarchical labels. Specifically, we apply ChatGPT to generate taxonomic structures, and observe that the outputs align with the ground-truth hierarchies. This shows that modern LLMs are capable of producing reliable semantic hierarchies with minimal effort, and our method still achieves strong performance under this setup, validating the effectiveness and accessibility of such automatically constructed hierarchies in real-world discovery scenarios.\n\n| Method       | CUB         |             |             | SCars |             |             | Aircraft |             |             |\n|--------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|\n|              | All         | Old         | New         | All         | Old         | New         | All         | Old         | New         |\n| SEAL(LLM-generated hierarchies)     | 66.2        | 72.1       | 63.2        | 65.3          | 79.3       | 58.5        | 62.0           | 65.3        | 60.4        |\n\n\n[a] *The role of taxonomy in species conservation* - 2004\n\n[b] *Species delimitation 4.0: integrative taxonomy meets artificial intelligence* - 2024\n\n**W2 Label Sensitivity**\n\nThanks for pointing out label sensitivity. While we would like to clarify that we do *not* state that performance drops sharply when the real semantic structure conflicts with a preset hierarchy. Rather, our concern is that incorrect or arbitrarily defined hierarchies such as the abstract superclasses manually designed in prior work [55] may introduce *misleading supervisory signals* due to their lack of semantic grounding.\n\nTo further support this point, we provide an additional experiment using randomly generated hierarchical labels, which indeed resulted in performance degradation. This highlights the importance of **semantically meaningful hierarchies** rather than abstract self-defined hierarchies.\n\n| Method       | CUB         |             |             | SCars |             |             | Aircraft |             |             |\n|--------------|-------------|-------------|-------------|---------------|-------------|-------------|----------------|-------------|-------------|\n|              | All         | Old         | New         |       All           | Old         | New         | All            | Old         | New         |\n| 100%  Random   | 30.3        | 31.6      | 29.7        | 29.6          | 31.4       | 28.7        | 33.2           | 31.3        | 34.2       |\n| 50%  Random     | 51.2        | 50.3       | 51.7        | 48.5         | 50.1       | 47.7        | 40.7           | 39.6        | 41.3        |\n| SEAL     | **66.2**        | **72.1**       | **63.2**        | **65.3**          | **79.3**       | **58.5**        | **62.0**           | **65.3**        | **60.4**        |\n\n**W3 Recommend beautifying the images in the article**\n\nThanks for the suggestion. We will try to further beautify the images. If more detailed suggestions could be provided, we would be happy to adopt."}, {"review_id": "3w6U6RcFZG", "reviewer": "Reviewer_mwGr", "summary": "This paper introduces SEAL, a new framework for Generalized Category Discovery. The core problem is to classify images from a dataset where some classes are known (labeled) and others are novel (unlabeled). The authors argue that existing methods are limited by their reliance on single-level semantics or manually created hierarchies. To address this, SEAL leverages naturally occurring semantic hierarchies to guide the learning process. The framework has two key components: 1) a Hierarchical Semantic-Guided Soft Contrastive Learning loss, which uses the hierarchy to select more informative negative samples, and 2) a Cross-Granularity Consistency module, which ensures that predictions are consistent across different levels of the semantic hierarchy (e.g., fine-grained classes and their coarse-grained superclasses). The method is evaluated on several fine-grained and coarse-grained benchmarks, where it reportedly achieves state-of-the-art results.", "questions": "1. Could you elaborate on the relationship between SEAL and DebGCD? Given their competitive performance in Table 2, a more direct comparison of their architectural and methodological differences would be helpful to understand the trade-offs and the specific scenarios where SEAL offers an advantage.\n2. What is the potential explanation for the lower performance of SEAL on the CUB dataset? Does this suggest a limitation of the automatically-derived semantic hierarchy for this particular data distribution, or are there other aspects of the framework that may be less effective here?\n3. Regarding the related work, could you provide a more detailed differentiation from the key methods cited in lines 75-85? Specifically, for those that also employ contrastive learning, what is the key advantage of your hierarchical semantic-guided approach?", "limitations": "The authors provide a limitations section, but it is located in the appendix. According to the NeurIPS Call for Papers and reviewing guidelines, which emphasize the importance of transparency, a discussion of limitations is a key part of a strong submission. The most relevant limitations should be summarized in the main paper to provide readers with a more balanced and complete understanding of the work's scope and applicability.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\nThe paper is generally well-written and easy to follow. The motivation for using semantic hierarchies in GCD is clear and compelling, and the proposed SEAL framework is described in sufficient detail.\n\nThe authors provide a theoretical motivation for their approach in the supplementary material, grounding their method in information-theoretic principles.\n\nThe supplementary material includes a valuable analysis of the computational cost of SEAL compared to baseline methods, showing consideration for the practical aspects of the proposed framework.\n\n\n\nWeaknesses\n\nThe related work section could be strengthened. In several instances (e.g., lines 75, 77, 85), multiple prior works are cited as a group without a clear explanation of their individual contributions or how SEAL specifically improves upon them. This makes it difficult to precisely understand the originality and significance of the proposed method in the context of the literature.\n\nThe empirical results, while strong on some datasets, raise questions that are not fully addressed in the paper:\n\n1. In table 2, the improvement in DINO (average) wrt to the second best method is only 0.1 (DebGCD). Indeed, DebGCD matches or outperforms the proposed method in 2 of the 3 datasets for All classes. The results for this backbone are not discussed, and an explanation for this difference is not given.\n2. The performance on the CUB dataset is a notable concern. Across the \"All,\" \"New,\" and \"Known\" class settings, SEAL consistently lags behind methods like DebGCD and SimGCD. This underperformance on a key fine-grained benchmark is not discussed.\n3. It is not immediately clear from the main text which backbone was used for the results in Table 3 (DINO or DINOv2?). Why only one for these two datasets?", "was_revised": true, "final_justification": "The authors have successfully addressed all my concerns, including new experiments and explanations.", "rebuttal": "We sincerely thank the reviewer for the thoughtful feedback.\nWe greatly appreciate the reviewer's recognition of the **paper's clarity**, the **clear and compelling motivation for using semantic hierarchies in GCD**, and the **detailed presentation** of the SEAL framework. We're also glad that the **theoretical justification** and **computational analysis** in the supplementary material were found to be **valuable**. Our detailed responses are provided below. We will also further strengthen the final manuscript based on the reviewers' concluding feedback. A well-documented code together with all trained models will be made public.\n\n**Q1&W1 Difference between DebGCD**\n\nThanks for the valuable question. DebGCD addresses general *label bias*, while SEAL targets the core challenge of leveraging **semantic hierarchy** in GCD, leading to more **scalable** and **lightweight** solution, especially under **strong** representation.\n\nDebGCD tackles the *label bias* issue by introducing an auxiliary debiased classifier trained on pseudo-labels, along with an OOD-aware regularization strategy. This approach is particularly effective under weaker backbones such as DINO, where label bias is more severe and novel-class separation is more difficult. By explicitly mitigating overfitting to known classes, DebGCD improves robustness in such *less discriminative* feature spaces.\n\nIn contrast, SEAL focuses on a space-agnostic challenge: how to effectively leverage **inherent semantic hierarchies** when only a subset of the data is labeled. Grounded in an information-theoretic framework, SEAL introduces two key components—Cross-Granularity Consistency and Semantic-Guided Hierarchical Soft Contrastive Learning—to provide **structure-aware supervision** across multiple levels of semantic granularity. These components enable the model to capture fine-grained semantic relationships and improve generalization to novel categories.\n\nWhile SEAL and DebGCD achieve comparable performance under DINO (**64.5** vs. 64.4 avg.), SEAL consistently outperforms DebGCD, and the advantage becomes more pronounced under the stronger DINOv2 backbone (**76.3** vs. 74.9). This suggests that while DebGCD is highly effective at correcting label bias under *weak initialization*, SEAL excels in utilizing **hierarchical structure**, especially when the feature space is less noisy.\n\nFurthermore, we perform a direct comparison of model complexity in terms of model parameters and FLOPs, and find that SEAL introduces significantly fewer parameters and lower computational overhead, making it a more **lightweight** solution.\n\n|Method|#Params (MB) ↓ |||GFLOPs ↓ || |\n|-|-|-|-|-|-|-|\n||SCars|CUB|Aircraft| SCars|CUB|Aircraft   |\n|DebGCD|705.1|704.4|705.1| 17.6| 17.6|17.6|\n|Ours|660.5| 688.0 | 688.1| 17.6|17.6|17.6|\n\n\n**Q2&W2 Performance on CUB**\n\nThanks for this insightful question. We attribute the relatively lower performance gain on CUB not to a limitation of our method, but rather to the **unique data distribution** of the CUB dataset, which presents specific challenges that affect certain types of label assignment strategies.\n\nSpecifically, we observe that CUB exhibits a higher degree of **intra-class multi-modality**—that is, individual classes tend to contain **multiple distinct sub-clusters**—compared to other datasets such as Scars and Aircraft. To empirically validate this, we conduct an analysis using Gaussian Mixture Models (GMMs) on PCA-reduced, L2-normalized features, selecting the optimal number of components M within each class via BIC (Such operation is standard for mixture selection [a], used in [b] to estimate the multimodality of a given distribution and used in class‑number estimation [c]).\nOur results show that **145 out of 200** classes in CUB require $M\\geq2$, indicating multi-modal structure, whereas only 7 out of 196 in SCARS and 20 out of 100 in Aircraft show such behavior. This confirms that CUB classes are often fragmented into multiple sub-modes. We report the average number of components per class and the percentage of classes with $M\\geq2$.\n\n|CUB||Aircrafts||Scars||\n|-|-|-|-|-|-|\n|Avg.M|% ($M\\geq2$)|Avg.M|% ($M\\geq2$)|Avg.M|% ($M\\geq2$)|\n|**2.07**|**72.5**|1.2|20|1.04|3.57|\n\nUnder such conditions, non-parametric clustering method like k-means—as used in Infosieve [41]—are better suited to capture these sub-clusters. In contrast, our method uses a parametric classifier trained with cross-entropy, following the widely adopted SimGCD [57] framework. While this design is effective in many settings, it may be structurally under-specified for modeling multi-modal distributions, leading to suboptimal label assignment on CUB.\n\nTo further examine this, we re-evaluate our model using the semi-supervised k-means in [41] clustering (averaged over 3 runs) on extracted features. Under this semi-supervised k-means setting, our method outperforms Infosieve [41] on CUB, especially when using the strong DINOv2 backbone, which reduces initialization noise and yields more structured features. Additionally, applying this to the SimGCD baseline yields a similar performance boost, further confirming that the lower performance is not due to the learned representations, but rather due to a mismatch between clustering method and the underlying data structure.\n\nWe will clarify this in the final version and include the k-means evaluation results to present a more complete picture of our method’s effectiveness.\n\n[a] Bishop, Pattern Recognition and Machine Learning, Sec. 9 (2006).\n\n[b] Wasserstein Distances, Neuronal Entanglement, and Sparsity, ICLR(2025)\n\n[c] X‑means: Extending k‑means with efficient estimation of the number of clusters, ICML(2000).\n\n| Method||CUB||  \n|-|-|-|-|\n||**All**|**Old**| **New** |\n|CiPR| 78.3 | 73.4  | 80.8 |\n|SimGCD (Reproduce)|72.3| 74.1| 71.4    |\n|SimGCD + KMeans (3runs)| 76.8$\\pm$0.4  | 76.4$\\pm$1.0     | 77.1$\\pm$0.3    |\n|InfoSieve(Reproduce) |80.7| 84.5    | 78.8 |\n|DebGCD  |77.5| 80.8| 75.8 |\n|SEAL| 76.7| 78.3 | 75.0|\n|SEAL + KMeans (3runs)| **81.2$\\pm$0.1**| **79.8$\\pm$0.4** | **81.9$\\pm$0.3**|\n\n**Q3 Difference with related work**\n\nThanks for the helpful suggestion. We will revise the presentation accordingly with a more detailed discussion.\n\n**Key Advantages of Our Hierarchical Semantic-Guided Approach** : \n\n1. **Semantic-Guided Hierarchical Supervision**  \n   Unlike Infosieve [41], which encodes category structure using binary paths in an implicit decision tree (thus assuming a fixed latent tree structure), and TIDA [55], which relies on handcrafted artificial hierarchies, which may introduce erroneous supervision as evidenced by the performance gap between TIDA and SEAL (e.g., **54.7 vs. 66.2** on CUB, **54.6 vs. 62.0** on Aircraft). SEAL is the first to incorporate **semantically guided** hierarchical labels into the learning process. These labels are readily available in many real-world scenarios and provide meaningful structural cues. \n2. **Semantic-Aware Soft Contrastive Learning**  \n   Prior contrastive learning methods (e.g., GCD [48], SimGCD [57], DebGCD [33]) treat all negative samples equally. In contrast, SEAL introduces **semantic-aware soft contrastive learning** by leveraging the class hierarchy to impose graded penalties.  \n   For instance, samples from semantically similar categories—such as **Siamese** and **Egyptian Mau** (both domestic cats)—should be penalized less than distant ones like **Beagle** (a dog breed).  \n   This structure-aware contrastive learning improves representation quality and allows more nuanced inter-class separation.\n \nMethods like DTC [23], RS [21, 22], UNO [18], NCL [65], DualRS [63], and JOINT [27] operate under the *Novel Category Discovery* setting, assuming all unlabelled samples are from unseen classes.  These methods leverage clustering, rank statistics, and label balancing, but are not directly applicable to GCD.\n\nGCD [48] generalizes NCD by assuming that unlabelled data contains both seen and novel classes. Under this setting, ORCA [5] introduces an uncertainty-adaptive margin to dynamically balance the learning speed between seen and novel categories. CiPR [24] constructs pseudo-labels from confident unlabelled samples through clustering and enforces consistency via a contrastive loss. DCCL [39] further improves pseudo-label quality by leveraging local structure mining in a teacher-student contrastive learning framework. SimGCD [57] establishes a strong end-to-end baseline by jointly training a feature extractor and a parametric classifier using self-distillation and entropy regularization, while SPTNet [52] enhances SimGCD with spatial prompt tuning to emphasize discriminative object regions. Infosieve [41] encodes each sample using binary paths in an implicit decision tree, which assumes a fixed latent hierarchical structure. TIDA [55], on the other hand, introduces handcrafted abstract hierarchies by constructing prototypes at manually defined levels.\n\n**W3 Backbone choose for Tab.3**\n\nThanks for pointing this out, and we will fix the order typo of Tab.2 and Tab.3 in the final version. For Tab.3 (Pets and Herbarium), we report results using DINO, which is a common practice in the GCD community for these datasets in prior works [41,48,52,57]. To further address your concern, we have additionally included the results of our method using DINOv2 for reference, where SEAL achieves consistent improvements when switching from DINO to DINOv2, especially on the more challenging Herb19 dataset (**+12.1** for All), highlighting the scalability of our method under stronger visual representations.\n\n| Method| Oxford-Pet ||| Herb | ||\n|-|-|--|-|--|--|-|\n| **SEAL**|All| Old| New| All| Old| New|\n|DINO|92.9| 88.9|95.0|46.3|45.8 | 48.2 |\n|DINOv2|95.7| 96.3|95.4 | 58.4 | 57.1|59.91|\n\n**Limitations Section**: \n\nThanks for pointing this out. We will revise the paper to move the limitations from Appendix to the main paper to include a concise summary of the most relevant limitations."}, {"review_id": "IIYyLLsAlB", "reviewer": "Reviewer_9SBL", "summary": "This paper addresses the Generalized Category Discovery (GCD) problem, where the goal is to categorize both known and unknown classes given a partially labeled dataset. The authors propose a new method SEAL, which leverages hierarchical structures to improve semantic representation. Concretely, the method introduces a Hierarchical Semantic-Guided Soft Contrastive Learning strategy that generates informative soft negatives and a Cross-Granularity Consistency (CGC) module to align predictions across different semantic levels. The proposed approach experiments with three fine-grained datasets.", "questions": "Please refer to the weaknesses.", "limitations": "The authors haven't address the limitations and potential negative societal impact of their work.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths:\n+ Incorporating hierarchical information into the GCD setting is an interesting direction, as it aligns well with how semantic relationships are structured in real-world data.\n+ The proposed method demonstrates sound experiment performance across three datasets.\n### Weannesses:\n+ The problem setting presented in this paper is not sufficiently clear, particularly in terms of the hierarchical information being leveraged. For example:\n   + In Lines 54–55,  it's unclear what \"naturally occurring semantic hierarchies\" is supposed to refer to? Furthermore, how does the proposed setting differ from prior works such as [41, 55]? \n   + The caption of Figure 1 claims that prior methods define upper and lower levels as well as abstract concepts around the ground-truth level, while the proposed method leverages semantic information at different levels. However, the figure does not clearly illustrate how the two approaches differ in their use of hierarchical information.\n\n+ In Lines 41-43, it states that \"All existing methods construct hierarchical levels based on handcrafted structures, which may introduce errors, thus affecting the GCD performance\" This is a strong claim. Are there any empirical results to support this statement? This is the motivation of this paper, so I think it is necessary to provide concrete evidence that handcrafted hierarchies significantly introduce errors and affect the GCD performance.\n\n+ In Section 4.1, the theoretical analysis assumes access to all coarse-grained hierarchical labels. However, this assumption seems unrealistic in GCD settings, where only a subset of classes is labeled.\n\n+ It is unclear how the number of hierarchical levels $H$ is determined in Lines 152–153. Is it predefined based on the ground-truth hierarchy? Similarly, how is the number of classes per level obtained in Line 162? A more detailed explanation is necessary to clarify whether this structure is assumed given or derived from data.\n\n+ In Lines 173–175, the description of updating the transition matrices $M_h$ for novel classes lacks clarity. The paper mentions initializing with a uniform distribution and refining during training, yet it is unclear how this refinement is implemented. While Algorithm 1 appears to be introduced for updating $M_h$, the method of obtaining $(p_h|\\mathbf y_H == k)$ is not clearly explained.\n\n+ In Lines 189–190, the process of fusing each fine-level similarity matrix with its coarser counterpart to obtain $\\tilde{S}_h$ is insufficiently described. What specific operation or function is used in this fusion?\n\n+ In Line 191, the variable $\\lambda_s$ is introduced without explanation. Please define its meaning in the overall objective.\n\n+ The paper states that the weighting coefficient $\\lambda_c$ decays during training as part of a curriculum learning strategy in Lines 196-197. However, no details are provided regarding how this decay is implemented. Please clarify this component for a better understanding of the training dynamics.", "was_revised": true, "final_justification": "I will maintain my original score, based on the following reasons:\n+ The paper requires substantial revision or even rewriting, due to many unclear and ambiguous descriptions throughout. These issues make it difficult to fully understand the core ideas and assess the contributions.\n+ I believe the assumption of a predefined hierarchical structure and fixed number of position categories in the GCD setting to be highly unrealistic and problematic. This setup does not align with the goals or practical constraints of real-world GCD scenarios.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and positive feedback.\nWe greatly appreciate the reviewer's recognition of the **value of incorporating hierarchical information into GCD** and acknowledgment of our **sound experiment performance across three datasets**.\nOur detailed responses to mentioned concerns are provided below. We‘ll also further strengthen the final manuscript based on the reviewers' feedback. A well-documented code will be made public.\n\n## 'Naturally occurring' & Difference with [41, 55]:\nThanks for raising this point. In Lines 54–55, by \"naturally occurring semantic hierarchies\", we refer to domain-inherent structures that are **readily available** without manual design. For instance, taxonomic hierarchies in species classification (e.g., family, genus, species) are well-established and widely used in scientific databases. These reflect real-world semantic relationships and require no additional annotation. Our method leverages such **domain-native** hierarchies as semantically meaningful supervision. We will revise the wording to avoid ambiguity in the final version.\n\nMoreover, our approach differs from prior works-Infosieve [41] and TIDA [55] in both the source and use of hierarchical information. Infosieve [41] encodes category structure via binary paths in an implicit *decision tree*, assuming a fixed latent hierarchy that may not match *true semantics*—leading to suboptimal performance (e.g., SEAL outperforms Infosieve: Avg. **64.5** vs. 60.5). TIDA [55], meanwhile, uses manually defined hierarchies for prototype-level supervision, which risks introducing noisy supervision and lacks scalability. In contrast, SEAL employs readily available, **semantic-guided** hierarchies and integrates them through an **information-theoretic** framework to provide consistent, structured supervision across multiple levels, ultimately improving both representation quality and category discovery.\n\n## Justification of motivation:\nThanks for highlighting this point. We acknowledge that the statement in Lines 41-43 may come across as strong, and we will revise the final version to smooth our tone. \n\nTo support our motivation, we provide two forms of empirical evidence illustrating the potential risks of handcrafted or semantically misaligned hierarchies:\n1. TIDA [55] adopts manually defined **abstract hierarchies**, which, while aiming to introduce structure, may not reflect true semantic relationships. This can lead to suboptimal supervision signals. In our experiments, SEAL significantly outperforms TIDA on multiple datasets, for example, CUB (66.2 vs. 54.7) and Aircraft (62.0 vs. 54.6), suggesting that misaligned or artificial hierarchies can adversely affect GCD performance.\n\n|Method|CUB|||Aircraft|||\n|-|-|-|-|-|-|-|\n|| All  | Old  | New| All   | Old  | New|\n| TIDA[55]    | 54.7  | **72.3**       | 46.2        | 54.6           | 61.3        | 52.1    |\n| SEAL  | **66.2**(+11.5)       | 72.1       | **63.2**        | **62.0** (+7.4)         | **65.3**   | **60.4**     |\n\n2.  To further validate this, we conduct an additional experiment using **randomly generated coarse-level labels** as a substitute for semantically meaningful hierarchies. In this setting, performance **drops substantially** compared to using true semantic hierarchies, indicating that the effectiveness of our method is not simply due to hierarchy, but rather the semantic alignment of the hierarchical information.\n\n|Method|CUB||| SCars ||| Aircraft |||\n|-|-|-|-|-|-|-|-|-|-|\n||All|Old | New|  All  | Old| New  | All| Old | New|\n| 100%  Random   | 30.3  | 31.6      | 29.7| 29.6| 31.4       | 28.7  | 33.2  | 31.3        | 34.2       |\n| 50%  Random     | 51.2 | 50.3       | 51.7 | 48.5   | 50.1  | 47.7   | 40.7  | 39.6        | 41.3        |\n| SEAL | **66.2** | **72.1** | **63.2** | **65.3** | **79.3**  | **58.5** | **62.0** | **65.3** | **60.4** |\n\n## Assumption validity:\nThanks for the thoughtful comment. We assume **ONLY** the availability of hierarchical labels for labelled data, which is **fully aligned** with the Generalized Category Discovery task and reflects realistic conditions in many domains. \n\nIn Sec. 4.1, we leverage available hierarchical labels to decompose the mutual information via the chain rule (Line 135), showing that incorporating coarse-grained labels provides more informative and semantically structured supervision.\n\nFor unlabelled data, we make no assumptions about label availability. Instead, we apply the variational lower bound:\n$I(X, Y)\\geq\\mathbb{E}_X \\left[ \\mathrm{KL}(q(Y \\mid X) \\parallel p(Y)) \\right]$ where $q(Y \\mid X)$ is approximated by the model's predictive distribution. \n\nThis leads to a tractable lower bound expressed as$\\mathcal{H}_\\theta(\\hat{Y}_u \\mid X_u) - \\mathcal{H}(\\hat{Y}_u)$ where $\\hat{Y}_u$ refers to the model's predicted label distribution for the unlabelled data, as detailed in Appendix, Lines 43–50.\n\nWe will revise the manuscript to clarify this assumption and make the notation between labelled and unlabelled data more explicit in our theoretical motivation.\n\n## Number of hierarchies:\nThanks for the insightful question. The number of hierarchical levels $H$ is known, given the ground-truth hierarchies of labelled categories. In the main paper, we assume the number of classes at the target level is known, following common practice in the GCD community [23, 24, 33, 41, 52, 55], which ensures fair comparison under controlled settings. For the category numbers of other levels, we estimate them using the estimation method from GCD [23]. Actually, they are consistent with the ground-truth numbers (see Appendix Tab. 3). We will offer a more detailed clarification in the final version.\n\nTo evaluate more realistic cases where the number of classes at the target level is also unknown, we also provide results in Appendix C (Line 110 onward), where class numbers are estimated using the method from GCD [23] with only labeled hierarchical data. As shown in Appendix Tab. 4, SEAL maintains strong performance, outperforming other baselines even without ground-truth category numbers. \n\n## Transition Matrix Update Mechanism:\nThanks for pointing this out. We refine the transition matrices by averaging the coarse-level model-predicted distribution ($p_h$) of samples predicted to the corresponding fine-grained class, enabling bottom-up propagation of semantic structure.\n\nGiven model predictions at two granularity levels—$p_H$ (fine-grained) and $p_h$ (coarse)—we assign pseudo-labels at the fine level via $y_H = \\arg\\max(p_H)$ (Algo.1, line 6). For each novel class $k$ (i.e., not matching any base category), we collect samples predicted as class $k$ where $y_H == k$ and compute their average predicted distribution at the coarse level: $\\text{mean}(p_h[y_H = k])$.\n\nThis average captures how the fine-level class $k$ relates to higher-level categories, enabling bottom-up refinement of the hierarchy. The underlying intuition is that samples grouped under the same fine-level class should also exhibit consistent coarse-level semantics.\n\nWe will improve the presentation to clarify this refinement process and its motivation.\n\n## Fusion Mechanism & $\\lambda_{s}$:\nThanks for the insightful comment. \nIn our implementation, the fused similarity matrix is computed as$\\tilde{S_h} =\\frac{1}{h} \\sum_{n=h}^1 S_n$where $S_n$ denotes the similarity at level n. This top-down fusion integrates coarse-level semantics into fine-grained supervision.\n\nUnlike prior methods that treat all non-positive samples *equally*, we leverage hierarchical structure to impose **semantic-aware soft supervision**. For example, samples from semantically related categories (e.g., Siamese (Cat) vs. Egyptian Mau (Cat) are penalized less than those from distant ones (e.g., Siamese (Cat) vs. Beagle(Dog) ). This enables the model to learn more nuanced, structure-aware decision boundaries.\n\nThe variable $\\lambda_s$ in line 191 is a hyperparameter used to control the smoothness of our proposed semantic-aware soft labels. \nWe further provide an additional ablation study to demonstrate the effectiveness and impact of this component. We will define its meaning in the final version.\n\n|Method|CUB||| SCars ||| Aircraft |||\n|-|-|-|-|-|-|-|-|-|-|\n||All|Old | New|All| Old| New  | All| Old | New|\n| $\\lambda_{s}=0$| 64.9| 70.4 | 59.4 | 63.8| 76.4 | 51.3|60.2|61.5|59.6|\n| SEAL| **66.2**  | **72.1** | **63.2** | **65.3** | **79.3**| **58.5** | **62.0**  | **65.3**  | **60.4** |\n\n## Curriculum Learning Schedule:\nThanks for the helpful comment. The weighting coefficient $\\lambda_c$ follows a **linear decay schedule**: $\\lambda_c = 1 - \\frac{\\text{epoch}}{\\text{TotalEpochs}}$, gradually shifting focus from semantic alignment (via cosine similarity) to finer positional discrimination as training progresses.\n\nAn ablation study comparing fixed values ($\\lambda_c = 0, 0.5, 1$) and exponential decay($1 - \\exp\\left(-\\frac{\\text{epoch}}{\\text{TotalEpochs}} \\right)$) confirms that decaying schedules consistently outperform fixed baselines, validating the effectiveness of this curriculum design.\n|Method|CUB||| SCars ||| Aircraft |||\n|-|-|-|-|-|-|-|-|-|-|\n||All|Old| New|All| Old| New| All| Old | New|\n| $\\lambda_{c}=0$ |64.6| 72.2| 60.8 | 64.5|80.8|56.6|59.3|61.8|58.0|\n| $\\lambda_{c}=0.5$| 65.1| 72.8|61.3| 64.1|77.1| 57.8  |58.9|62.5|57.1|\n| $\\lambda_{c}=1$| 64.9| 72.4  | 60.9|63.2|80.2| 55.0  |58.7|65.4 |55.3|\n| Exp Declay|66.1| 71.7| 63.3 | **66.3**| **81.2**|**60.3**|61.2| 63.1 | 60.2|\n| SEAL| **66.2**| **72.1**| **63.2**|65.3| 79.3| 58.5|**62.0**| **65.3**| **60.4**|\n\nWe will provide more clarification regarding this strategy in the final version."}, {"review_id": "zCM2yeee1P", "reviewer": "Reviewer_PVY1", "summary": "This paper presents SEAL, a semantic-aware hierarchical learning framework for GCD, which leverages natural semantic hierarchies to address the limitations of existing methods relying on single-level semantics or manual hierarchies. It introduces a Hierarchical Semantic-Guided Soft Contrastive Learning approach and a Cross-Granularity Consistency module to enhance representation learning and prediction alignment across different granularities. SEAL achieves good performance on fine-grained benchmarks.", "questions": "**Motivation:** How are these hierarchical data obtained? Does the method rely on predefined structures in the dataset?  \n**Novelty:** Multi-level semantic learning was first proposed in [41]. What is the key novelty of SEAL compared to [41]?  \n**Experiments:**  \n- Why is the improvement on CIFAR-100 and ImageNet-100 smaller than that on fine-grained datasets?  \n- In Table 4, why does the accuracy drop on CUB and Aircraft after applying hierarchical learning?  Similarly, why does Semantic-Guided HSCL hurt CUB but not other datasets?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n- The paper is well-organized, guiding readers smoothly from motivation to methodology and experiments. Each section builds on the previous one, and visual aids like Fig. 1 help illustrate the core ideas of the hierarchy-aware design.\n- Well-presented content. The proposed modules, such as Cross-Granularity Consistency (CGC) and hierarchical contrastive learning, are clearly explained. The combination of theoretical insights and implementation details makes the method easy to follow. \n- Good performance. SEAL demonstrates strong performance on fine-grained datasets and consistent improvements across both known and novel categories. Ablation studies and t-SNE visualizations provide further evidence of the method’s effectiveness and the contribution of each component.\n\n**Weaknesses:**\n- The authors claim state-of-the-art performance in the abstract, but the CUB accuracy in Table 1 (66.2%) is lower than that of [41] (69.4%).\n- The code has not been released.\n- Missing Relevant Works: The paper overlooks important recent works, such as LegoGCD, which was accepted at CVPR 2024.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's thoughtful and positive feedback. We greatly appreciate the recognition that the paper is **well-organized** from motivation to methodology and experiments, the proposed modules are **clearly explained**, and SEAL demonstrates **strong performance** across fine-grained datasets. Our detailed responses to the concerns are provided below. We‘ll also further strengthen the final manuscript based on the reviewers' feedback. A well-documented code together with all trained models will be made public.\n\n**Q1 Hierarchical Data Availability**\n\nThanks for the valuable question. We would like to highlight that the hierarchical information used in our framework is **easily obtainable** and **naturally aligned with real-world structures**, rather than being rigid or manually imposed.\nAs detailed in our supplementary material, the hierarchical labels used in our framework are either constructed based on biological taxonomies (e.g., for CUB and Herbarium), or derived from existing internal dataset structures (e.g., Aircraft, where models are already organized in the dataset by manufacturer (e.g., Boeing or Douglas) and further grouped into families such as 767 or 777.), or can be generated using large language models (LLMs) in cases where external hierarchies are unavailable.\n\nWe highlight that these hierarchies are not artificially constructed, while they naturally exist in the corresponding domains and align with how taxonomic knowledge is organized in biological and real-world settings. \nFor instance, in botanical research, taxonomists commonly use labelled specimens of known species to classify newly collected, unlabelled samples into existing taxonomic hierarchies or to identify unseen species [a,b]. Similarly, our method assumes access to hierarchical labels only for seen categories, which is both **practical** and **realistic**, and reflects real-world conditions where semantic structure is available for known data but not for novel categories.\n\n[a] *The role of taxonomy in species conservation* - 2004\n\n[b] *Species delimitation 4.0: integrative taxonomy meets artificial intelligence* - 2024\n\n**Q2 Distinction from [41]**\n\nThanks for the insightful question. While Infosieve [41] introduces multi-level learning by encoding each sample using a binary path in an implicit decision tree, it inherently assumes the existence of a meaningful and fixed tree structure. However, such a structure may not accurately reflect the true semantic organization of categories, which is further supported by the observed performance gap in our experiments. For example, SEAL outperforms Infosieve by a significant margin on Scars (**65.3** vs. 55.7) and on Aircraft (**62.0** vs. 56.3).\n\nIn contrast, SEAL emphasizes the use of **semantic-guided hierarchical supervision**, which is both flexible and interpretable, and does not rely on predefined or artificially imposed tree structures. Our framework is grounded in a solid information-theoretic analysis, which motivates the design and ensures that the introduced hierarchy enhances rather than misleads the learning of target-level categories. This makes SEAL more robust and better aligned with real-world semantic structures compared to tree-based approximations.\n\n**Q3 Discussion of performance on generic datasets**\n\nThanks for the thoughtful question. The relatively smaller improvements observed on CIFAR-100 and ImageNet-100 can be attributed to the nature of their semantic structure. These datasets are **coarse-grained**, and the hierarchical relationships between classes are often **broad and loosely defined** (e.g., grouping semantically distant categories like \" computer keyboard\" and \"clock\" under a general class as \"household electrical devices\"), resulting in weaker and less informative supervision signals from the hierarchy. \nIn contrast, fine-grained datasets such as CUB, Aircraft, and Scars feature tighter semantic groupings (e.g., species within the same genus or models from the same manufacturer), which allow our framework to more effectively exploit semantic hierarchies for representation learning and category separation. This difference naturally leads to more pronounced gains in fine-grained settings.\nWe appreciate the reviewer for pointing this out, and we will further explore strategies to enhance the utility of hierarchical information in coarse-grained scenarios as part of our future work.\n\n**Q4 Ablation Studies**\n\nThanks for the insightful question. As shown in Tab. 4 (Line 1), directly incorporating hierarchical supervision leads to performance degradation on CUB and Aircraft. While our theoretical motivation (Section 4.1) suggests that hierarchical information can be beneficial, naïvely applying it in the GCD setting can cause the training to converge to **suboptimal local minima** due to inconsistent or noisy supervision. To address this, we introduce Cross-Granularity Consistency (CGC) to ensure alignment between hierarchical levels, and Semantic-Guided Hierarchical Soft Contrastive Learning (HSCL) to better model inter-class relationships by relaxing the equally-negative assumption used in prior work. The improvements observed after applying these modules (Tab.4, *Ours*) further support their effectiveness in **robustly leveraging hierarchical signals** under the challenges of GCD.\n\nRegarding the Semantic-Guided HSCL, we clarify that it does not hurt performance on CUB. In fact, comparing Tab.4 (line 2 without HSCL) with the final _Ours_ row shows that incorporating this module improves CUB accuracy from 57.8 to __66.2__, demonstrating its effectiveness.\n\n**W1 Performance on CUB**\n\nThanks for pointing this out. We will revise the abstract to avoid overstatement. We attribute the relatively lower performance on CUB not to a limitation of our method, but rather to the **unique data distribution** of the CUB dataset, which presents specific challenges that affect certain types of label assignment strategies.\n\nSpecifically, we observe that CUB exhibits a higher degree of **intra-class multi-modality**—that is, individual classes tend to contain multiple distinct sub-clusters—compared to other datasets such as Scars and Aircraft. To empirically validate this, we conduct an analysis using Gaussian Mixture Models (GMMs) on PCA-reduced, L2-normalized features, selecting the optimal number of components M within each class via BIC (Such operation is standard for mixture selection [c], used in [d] to estimate the multimodality of a given distribution and used in class‑number estimation [e].\nOur results show that **145 out of 200** classes in CUB require $M\\geq2$, indicating multi-modal structure, whereas only 7 out of 196 in SCARS and 20 out of 100 in Aircraft show such behavior. This confirms that CUB classes are often fragmented into multiple sub-modes. We report the average number of components per class and the percentage of classes with $M\\geq2$.\n\n|CUB||Aircrafts||Scars||\n|-|-|-|-|-|-|\n|Avg.M|% ($M\\geq2$)|Avg.M|% ($M\\geq2$)|Avg.M|% ($M\\geq2$)|\n|**2.07**|**72.5**|1.2|20|1.04|3.57|\n\nUnder such conditions, non-parametric classifiers like k-means—as used in Infosieve [41]—are better suited to capture these sub-clusters. In contrast, our method uses a parametric classifier head trained with cross-entropy, following the widely adopted SimGCD [57] framework. While this design is effective in many settings, it may be structurally under-specified for modeling multi-modal distributions, leading to suboptimal label assignment on CUB.\n\nTo further examine this, we re-evaluate our trained model using the same semi-supervised k-means in [41] clustering (averaged over 3 runs) on extracted features. Under this semi-supervised k-means setting, our method outperforms Infosieve [41] on CUB, especially when using a stronger DINOv2 backbone, which reduces initialization noise and yields more structured features. Additionally, applying semi-supervised k-means to the SimGCD baseline yields a similar performance boost, further confirming that the lower performance is not due to the learned representations, but rather due to a mismatch between classifier type and the underlying data structure.\n\nWe will clarify this in the final version and include the k-means evaluation results to present a more complete picture of our method’s effectiveness.\n\n[c] Bishop, Pattern Recognition and Machine Learning, Sec. 9 (2006).\n\n[d] Wasserstein Distances, Neuronal Entanglement, and Sparsity, ICLR(2025)\n\n[e] X‑means: Extending k‑means with efficient estimation of the number of clusters, ICML(2000).\n\n\n| Method            | |             CUB                  |     |  \n|------------------|-------------------------------|------|-----|\n| **DINOv2**        | **All**   | **Old**  | **New** |\n| SimGCD (Reproduce)| 72.3     | 74.1     | 71.4    |\n| SimGCD + KMeans (3runs)   | 76.8$\\pm$0.4     | 76.4$\\pm$1.0     | 77.1$\\pm$0.3    |\n| InfoSieve (Reproduce)      | 80.7     | 84.5    | 78.8|\n| SEAL             | 76.7      | 78.3     | 75.0    |\n| SEAL + KMeans (3runs)   | **81.2$\\pm$0.1**  | **79.8$\\pm$0.4** | **81.9$\\pm$0.3**|\n\n\n**W2 Code**\n\nThanks for the suggestion. We will release our code upon acceptance.\n\n**W3 LegoGCD**\n\nThanks for the suggestion. We would like to clarify that we have **cited** LegoGCD[6] and compared our method with LegoGCD[6] in Tab.2, where our method achieves stronger results. \nLegoGCD mitigates catastrophic forgetting via entropy regularization and dual-view KL divergence, whereas our method emphasizes the role of semantic hierarchies in generalized category discovery. These complementary approaches address GCD from distinct perspectives, jointly highlighting the value of structural guidance. We will include a discussion of LegoGCD in the Related Work section in the revised version."}]

Author rebuttal:

Meta-review: The paper proposes SEAL, a semantic-aware hierarchical learning framework for Generalized Category Discovery (GCD). SEAL is motivated by the observation that prior work often relies on single-level semantics or manually defined abstract hierarchies, which may be error-prone. Instead, SEAL leverages naturally occurring semantic hierarchies (e.g., biological taxonomies) to guide learning. The framework combines Hierarchical Semantic-Guided Soft Contrastive Learning, which improves negative sampling, with Cross-Granularity Consistency, which aligns predictions across coarse and fine levels. Extensive experiments on multiple benchmarks show strong results, with ablations and visualizations highlighting the contributions of each component.

Reviewers acknowledged several contributions: SEAL is the first framework to explicitly introduce biological classification systems into GCD (R1), its motivation is “clear and compelling” and theoretically grounded (R2), and incorporating hierarchical information aligns well with real-world semantics (R3). The paper was described as well-written and well-organized (R2, R4), and the experimental evaluation was supported by sufficient ablation studies and computational analysis (R1, R2, R4). SEAL shows competitive or state-of-the-art results on fine-grained datasets, outperforming methods such as DebGCD and TIDA in several cases (R1, R4).

Concerns were raised about clarity (R3), the assumption of fixed hierarchies and class counts (R1, R3, R4), and novelty relative to prior works such as DebGCD and LegoGCD (R2, R4). Performance inconsistencies (e.g., lower accuracy on CUB) were also noted (R2, R4). However, the authors clarified that hierarchies are derived only from labeled data, a practice consistent with real-world taxonomy and with common assumptions in related work (e.g., SimGCD, InfoSieve, DebGCD).  They also provided strong new experiments showing SEAL’s robustness under LLM-generated hierarchies, random hierarchies, and scenarios without known class counts. These additional results convincingly addressed the realism of assumptions and demonstrated that SEAL is not tied to a single predefined hierarchy, and that semantic-guided hierarchies -- rather than abstract, hand-crafted ones -- are key for GCD, whether sourced from curated taxonomies, generated by LLMs, or defined along different semantic dimensions.  Given these clarifications, the main reservations about applicability and realism are resolved.

The paper received final ratings of 2x borderline rejects (R1, R3) and 2x borderline accepts (R2, R4).  While initial consensus leaned negative, the authors’ thorough rebuttal -- including results on random/LLM-generated hierarchies and clarification of practical assumptions -- demonstrated that SEAL is both realistic and impactful.  Since natural hierarchies are abundant in real-world applications, SEAL makes a timely and meaningful contribution to GCD research.
Decision: Accept (poster)