DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery
Authors: Yuanpei Liu
Kai Han
Venue: ICLR 2025 Poster (2025)
Keywords: Generalized Category Discovery
Semi-supervised Learning
Out-of-distribution Detection
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose the Debiased Learning with Distribution Guidance framework for the generalized category discovery task.

Abstract: In this paper, we tackle the problem of Generalized Category Discovery (GCD). 
Given a dataset containing both labelled and unlabelled images, the objective is to categorize all images in the unlabelled subset, irrespective of whether they are from known or unknown classes. 
In GCD, an inherent label bias exists between known and unknown classes due to the lack of ground-truth labels for the latter. 
State-of-the-art methods in GCD leverage parametric classifiers trained through self-distillation with soft labels, leaving the bias issue unattended. 
Besides, they treat all unlabelled samples uniformly, neglecting variations in certainty levels and resulting in suboptimal learning. 
Moreover, the explicit identification of semantic distribution shifts between known and unknown classes, a vital aspect for effective GCD, has been neglected. 
To address these challenges, we introduce DebGCD, a Debiased learning with distribution guidance framework for GCD. 
Initially, DebGCD co-trains an auxiliary debiased classifier in the same feature space as the GCD classifier, progressively enhancing the GCD features. Moreover, we introduce a semantic distribution detector in a separate feature space to implicitly boost the learning efficacy of GCD. Additionally, we employ a curriculum learning strategy based on semantic distribution certainty to steer the debiased learning at an optimized pace. 
Thorough evaluations on GCD benchmarks demonstrate the consistent state-of-the-art performance of our framework, highlighting its superiority. Project page: [https://visual-ai.github.io/debgcd/](https://visual-ai.github.io/debgcd/)

Published as a conference paper at ICLR 2025
DEBGCD: DEBIASED LEARNING WITH DISTRIBUTION GUIDANCE FOR GENERALIZED CATEGORY DISCOV- ERY
Yuanpei Liu
Kai Han
Visual Lab, The University of Hong Kong ypliud@connect. hku.hk
kaihanx@hxu. hk
ABSTRACT
In this paper, we tackle the problem of Generalized Category Discovery (GCD). Given dataset containing both labelled and unlabelled images, the objective is to categorize all images in the unlabelled subset, irrespective of whether they are from known or unknown classes. In GCD, an inherent label bias exists between known and unknown classes due to the lack of ground-truth labels for the latter. State-of-the-arr methods in GCD leverage parametric classifiers trained through self-distillation with soft labels, leaving the bias issue unattended. Besides, they treat all unlabelled samples uniformly, neglecting variations in certainty levels and resulting in suboptimal learning. Moreover, the explicit identification of semantic distribution shifts between known and unknown classes, a vital aspect for effective GCD, has been neglected. To address these challenges, we introduce DebGCD, a Debiased learning with distribution guidance framework for GCD. Initially, De- bGCD co-trains an auxiliary debiased classifier in the same feature space the GCD classifier, progressively enhancing the GCD features. Moreover, we intro- duce semantic distribution detector in separate feature space implicitly boost the learning efficacy of GCD. Additionally, we employ a curriculum learning strat- egy based on semantic distribution certainty to steer the debiased learning an optimized pace. Thorough evaluations on GCD benchmarks demonstrate the con- sistent state-of-the-are performance of our framework, highlighting its superiority. Project page: https: /tt/vvvialllllaeeeeaa -ai github io/ /ddgged///////
INTRODUCTION
Over the years, the field of computer vision has witnessed remarkable progress in diverse tasks such as object detection Girshick (2015); Ren et al. (2015), classification Simonyan & Zisserman (2015); He et al. (2016), and segmentation He et al. (2017); Wang et al. (2020). These advancements have predominantly stemmed from the availability of expansive labelled datasets Deng et al. (2009); Lin et al. (2014). However, the prevalent insufficiency of training data in real-world scenarios is a note- worthy concern. This has engendered a surge in research on semi-supervised learning Chapelle et al. (2009) and self-supervised learning Jing & Tian (2020), yielding promising outcomes in com- parison supervised learning approaches. Recently, the task of category discovery. which was initially studied as novel category discovery (NCD) Han et al. (2019) and subsequently extended to its relaxed variant, generalized category discovery (GCD) Vaze et al. (2022b), has emerged as a research task attracting increasing attention. GCD considers partially-labelle dataset, where the unlabelled subset may contain instances from both labelled and unseen classes. The objective is to learn to transfer knowledge from labelled data to categorize unlabelled data.
In GCD, there exists an inherent label bias between known and unknown classes due to the absence of ground-truth labels for the latter. This label bias has the potential cause the model to inadver- tently develop a decision rule making confident predictions that inclined to known classes. Similar problem has been identified in the area of long-tailed recognition Tang et al. (2020); Yang et al. (2022). Besides, in other fields such as object classification Choi et al. (2019); Bahng et al. (2020); Geirhos et al. (2020), it is widely known that model performance suffers from task-specific bias.
"Corresponding author.

---

Published as a conference paper at ICLR 2025
labelled sample
unlabelled sample
Known Classes
Unknown Classes
biased labels
Image Eneoder
CLS
It-
debiased labels
(a) parametric GCD classifier
biased labels
(c) label debiasing
ground-truth label
pscudo label
GCD classifier
CLS
dibinsed classifier
high scure
CLS
biased labels
high OOD score
Image Encoder
debiasing
known class sample
CLS
debiused labels
unknown class sample
TD boundary
(b) auxiliary debiased learning
(d) distribution guidanec
OOD boundary
(a) The parametric GCD classifier Wen et al. (2023) is trained on labelled and unlabelled images using ground-trum hard labels and soft labels, respectively. (b) The auxiliary debiased learn- ing: training another classifier using debiased labels. (c) The process of label debiasing: keep the hard labels unchanged and transform soft labels to one-hot hard labels; samples that do not meet the threshold are removed. (d) The illustration of distribution guidance: if sample receives high in-distribution/uut-o wowpo//ooooooooooooooooooooo of-distribution score, its weight in GCD training will be increased accordingly.
Figure I:
State-of-the-art parametric classifier methods GCD, such as those proposed by Wen et al. (2023); Zhao et al. (2023); Vaze et al. (2023), leverage the self-distillation Caron et al. (2021) mechanism based on soft labels generated from the model's predictions of another image view. While these methods have shown promising results, they still rely biased labels for training (as shown in Fig. (a)). The issue of label bias remains an unattended problem in the realm of GCD. Addition- ally, existing approaches uniformly handle all unlabelled samples without explicitly accounting for their different certainty, which may introduce noise to the model training due to unreliable samples. Moreover, they do not explicitly address semantic shifts, especially in scenario like GCD involv- ing both known and unknown classes within unlabelled data. Notably, these concerns have been demonstrated to provide significant advantages related tasks, such as open-world world -uperviie learning Cao et al. (2022). In this area, OpenCon Sun & Li (2022) has attempted to identify novel samples based on their proximity to known prototypes. However, its performance is heavily contin- gent on predefined distance thresholds, ultimately yielding suboptimal accuracy.
We propose DebGCD, novel framework designed to tackle the challenges of GCD. DebGCD introduces Debiased learning with distribution guidance for GCD, incorporating several innovative techniques specifically tailored for this task. Firstly, we introduce a novel auxiliary debiased learning paradigm for GCD (as shown in Fig. 1(b) and (c)). This method entails training an auxiliary debiased classifier in the same feature space as the GCD classifier. Unlike the GCD classifier, both labelled and unlabelled data trained using one-hot hard labels prevent label bias between known and unknown classes. Secondly, to discern the semantic distribution of unlabelled samples, we propose learn a semantic distribution detector in a decoupled normalized feature space, which we empir- ically find it enhance the learning effect of GCD implicitly. Furthermore, we propose to measure the certainty of sample based on its semantic distribution detection score. This certainty score then enables the gradual inclusion of unlabelled samples from both known and unknown classes during training, allowing the auxiliary debiased learning to function curriculum learning ap- proach (as shown in Fig. (d)), thus further enhancing its performance. We develop our framework upon the parametric baseline Wen et al. (2023). By effectively incorporating these components into unified framework, DebGCD can be trained end-to-end in single stage while not introduc- ing any additional computational burden during inference. Despite its simplicity, DebGCD attains unparalleled performance on the public GCD datasets, including the generic classification datasets CIFAR-10 10 Krizhevsky et al. (2009), CIFAR- 100 Krizhevsky et al. (2009), and ImageNet Deng et al. (2009), as well as the fine-grained SSB Vaze et al. (2022a) benchmark.
We make the following key contributions in this work: (1) We propose DebGCD, a novel framework that addresses the challenging GCD task by considering both label bias and semantic shift, marking
2

---

Published as a conference paper at ICLR 2025
the first exploration of these aspects for the challenging GCD task. (2) Within DebGCD, propose a novel auxiliary debiased learning paradigm to optimize the clustering feature space, in conjunction with the distribution shift detector in distinct feature space. They work tightly to enhance the model's discovery capabilities. (3) We introduce a curriculum learning mechanism that steers the debiased learning process using distribution certainty score, effectively mitigating the negative impact of uncertain samples. (4) Through extensive experimentation on public GCD benchmarks, DebGCD consistently demonstrates its effectiveness and achieves superior performance.
2 RELATED WORK
Category Discovery. This task is initially studied as novel category discovery (NCD) Han et al. (2019), aiming to discover categories from unlabelled data consisting of samples from novel cate- gories. by transferring the knowledge from the labelled categories. Many methods have been pro- posed to tackle NCD, such as Han et al. (2019; 2020; 2021); Fini et al. (2021); Zhao & Han (2021); Joseph et al. (2022). Vaze et al. (2022b) extends NCD to more relaxed task, generalized category discovery (GCD), wherein unlabelled datasets encompass both known and unknown categories. A baseline method is presented for this task, incorporating self-supervised representation learning and semi-supervised k-means clustering, and extending popular NCD methods such as RankStats Han al. (2020) and UNO Fini al. (2021) to GCD. CiPR Hao et al. (2024) proposes to bootstrap the representation by leveraging cross-instance positive relations in the partially labelled data for contrastive learning. Cao et al. (2022) addresses similar problem to GCD from the perspective of semi-supervised learning. SimGCD Wen et al. (2023) introduces a strong parametric baseline achieving promising performance improvements. In Vaze et al. (2023), a new dataset is introduced to illustrate the limitations of unsupervised clustering in GCD. To address these limitations, method based on the 'mean-teachers' approach proposed. In Rastegar et al. (2023), a category coding ap- the outcome of an optimization problem.
proach is introduced, considering category prediction Recently, SPTNet Wang et al. (2024b) is proposed to consider the spatial property of images and presents a spatial prompt tuning method, enabling the model to better focus on object parts for knowledge transfer. Moreover, an increasing number of efforts are focused on addressing category discovery from diverse perspectives. For example, Jía et al. (2021) concentrates multi-modal cat- egory discovery, whereas Zhang et al. (2022), Ma et al. (2024), and Cendra et al. (2024) investigate continual category discovery. Additionally, Pu et al. (2024) explores federated category discovery. Furthermore, Wang et al. (2025) studies category discovery in the presence of domain shifts.
Debiased Learning. The issue of bias in data and the susceptibility of machine learning algorithms to such bias have been widely recognized as crucial challenges across diverse tasks. Numerous methodologies have been developed to address and alleviate biases inherent in training datasets or tasks. The studies by Ponce (2006); Torralba & Efros (2011) elucidate that many training sets impose regularity conditions that are impractical in real-world settings, leading to machine learning models trained on such data failing to generalize in the absence of these conditions. Furthermore, recent research by Hendrycks et al. (2021); Xiao et al. (2021); Li et al. (2021) demonstrate biases in state-of- of-ttheart object recognition models towards specific backgrounds textures associated with object classes. Additionally, Sagawa et al. (2020) investigate the vulnerability of overparametrizee models to spurious correlations, resulting in elevated test errors for minority groups Notably, large language models also exhibit biased predictions towards certain genders or races, as indicated by Cheng et al. (2021). Furthermore, the severity of biased predictions and fairness concerns related to deployed models are extensively explored across various tasks Zemel et al. (2013); Noble (2018) 8); Bolukbasi al. (2016). In this paper, we examine the inherent label bias in GCD, representing the initial exploration of this issue.
Out-of-d didtrriiittooon Detection. In the realm of out-of-distribution (OOD) detection, the objective is to identify samples or data points that originate from a distribution distinct from the one on which the model was trained, encompassing both semantic and domain distributions Yang et al. (2021); Wang et al. (2024a). The simplest method in this area involves utilizing the predicted softmax class probability to detect OOD samples Hendrycks & Gimpel (2017). ODIN Liang et al. (2018) further enhances this approach by introducing temperature scaling and input pre-processing. Additionally, Bendale & Boult (2016) proposes an alternative approach by calculating the score for an unknown class using weighted average of all other classes. OOD detection has been applied in various open-set tasks, such as open-set semi-supervised learning Yu et al. (2020) and universal domain adaptation Saito & Saenko (2021), where it is utilized select in-distribution data during training.
3

---

Published as a conference paper at ICLR 2025
In contrast, our focus lies in the exploration of semantic shift detection considering the specific challenges of GCD. OpenCon Sun Li (2022) has attempted to explore the semantic shift for open- world semii-sueerised learning. However, its reliance predefined distance threshold to rigidly distinguish inliers and outliers leads to suboptimal accuracy. In contrast, our method takes a distinct approach by avoiding rigid separation. We subtly utilize the predicted OOD score by our model as a guiding factor for debiased learning, further enabling curriculum learning scheme.
PRELIMINARIES
3.1 PROBLEM STATEMENT
Generalized category discovery (GCD) aims to learn model that can not only correctly classify the unlabelled samples of known categories but also cluster those of unknown categories. Given an unlabelled dataset P, = {(x," }²)} E X x Vi and a labelled dataset Dj = {[<r,, b() E X x 3% where Vn and 3/2 are their label sets respectively. The unlabelled dataset contains samples from both known and unknown categories, ie., 3i C Va. The number of labelled categories is M = [D%]. We assume the number of categories
= 1 U Vil to be known following previous works Han et al. (2021); Wen et al. (2023); Vaze et al. (2023). When it is unknown, methods like Han et al. (2019); Vaze et al. (2022b) can be applied to provide a reliable estimation.
3.2 BASELINE
Wen al. (2023) introduces robust parametric GCD baseline, which has been widely adopted in the field ever since Vaze et al. (2023); Wang et al. (2024b). It employs a parametric classifier, implemented in self-distillation manner Caron al. (2021). The classifier is randomly initialized with normalized category prototypes C = (cr..., ex}. ). For the randomly augmented view of an image æ; and its corresponding normalized hidden feature vector h, = =(2:)//l6(x;))] the output probability for the ktth category is given by:
p; (k) exp(h: Ca/Ts)
Σ5 exp(h; c5/Ts))
where тa is the scaling temperature for this 'student" view. The soft label qi is produced by the 'teacher' view with sharper temperature тr using another augmented view in the same fashion. The self-distillation loss of the two views is then simply calculated following the cross-entropy loss lee (4' p)) - ΣØ= q log pú), Given a mini-batch B containing both labelled samples B, and unlabelled images Bu. the self-distillation loss calculated using all the samples in the mini-batch:
C =
Σ iEB
Cen Co. p.) - (H
(2)
B
where p 2TB Eies(p; + pt) denotes the mean prediction within a batch and its entropy H()) -ド=z 100 g() weighted by ç. For the labelled samples, the supervised classification loss is written as Lit. |8, EiEB, lee Mi). where y: represents the one-hot vector corresponding to
god Lds + ged Cats- Combining with the representation learning loss Lrep adopted from Vaze et al. (2022b), the overall training objective becomes:
the ground-truth label y/. The whole classification objective Lats = (1 Ag
Gged = Leta + Lrep-
(3)
Through training with Lyed on both D; and D... the classifier can directly predict the labels for unlabelled samples after training.
4 DEBIASED LEARNING WITH DASTRIBUTION-GUIDAA
FOR GCD
In this section, we present our debiased Learning with distribution guiddacce framework for GCD (see Fig. 2). First, in Sec. 4.1, present the semantic distribution learning the GCD task. Next, in Sec. 4.2, we demonstrate the training paradigm of the debiased classifier. Finally, we describe the joint training and inference process of our full framework in Sec. 4.3.
4

---

Published as a conference paper at ICLR 2025
Debiased Learning with Distrbbutin-GGuidance
B
sofimax
nom
Bu
Image Encoder
OVA
L
distribution detector
B
stop gradient
MLP
distribution feature space
softmax
weight- sharing
CLS
Ladl
debiased
classifier
nom
stop gradient
B
Image Encoder
debiasing
softmax
B
vision transformer
CLS
Lood
clustering feature space
features
GCD
classifier
Figure 2: Overview of the DebGCD framework. In the upper branch, raw features are transformed using an MLP and then normalized. These normalized features are used for semantic distribution learning with one-vs-all classifier. In the lower branch, GCD classifier is trained the normal- ized raw features. The predictions from both branches are combined to train the debiased classifier. As DebGCD aligns with prior work in representation learning, it's not explicitly depicted here.
4.1 LEARNING SEMANTIC DISTRIBUTION
OOD detection methods have been employed in tasks like universal domain adaptation Saito & Saenko (2021) and open-set semi-supervised learning Yu et al. (2020), obtaining improved perfor- mance. In these tasks, the identified OOD samples avoid affecting the recognition tion shifts can be of any type. In GCD, we are particularly interested in identifying the semantic shifts. The instances from the labelled classes are considered in-distribution (ID) samples, while the instances from the novel classes are considered OOD samples. However, the potential of effec- tively introducing OOD techniques for GCD remains under-explored. An intuitive approach for OOD detection or logit score of closed-set classifier can serve as good indicator of OOD Vaze et al. (2022a); Wang et (2024a). However, this is not suitable for the common GCD classifier, which contains an mean entropy regularization term in the loss function to prevent biased predictions. Nevertheless, we find that it also results the classifier's predictions on known categories being less confident, thereby degrading the OOD detection performance. Moreover, these OOD methods need to carefuly select a threshold Geng et al. (2020) for rejecting "unknown" samples, which relies on validation pre-defined ratio of "unknown" samples, making them impractical for the GCD due the ab- sence of such validation samples. One-vs-all (OVA) classifier Saito & Saenko (2021), which has consistently shown promise in the literature Saito al. (2021): Fan et al. (2023); Li al. (2023), can be a more suitable option. Moreover, in the context of OOD, the objective is not to differentiate between multiple distinct unknown categories, we do in GCD; rather, we aim distinguish all unknown classes from the known classes, effectively framing this as a binary classification problem. This calls for the need of a different feature space that is better suited for this task. Therefore, as depicted in Fig. 2, we introduce an additional multi-layer perceptron (MLP) projection network Ps, to project raw features into another embedding space, followed by fz-normalization to attain the embedding space for distribution discrimination. Differrnt from the prior works applying OOD in the magnitude-awar aware feature space for other tasks Yu et al. (2020); Saito et al. (2021); et al. (2023), empirically found that the fz-normalized feature space aligns more seamlessly with the DINO pre-trate pre-trained weights in GCD. Subsequently, we devise M fy-normalized binary classifiers, denoted as X (x1, X2. XM for semantic OOD detection in GCD.
single background class to unlabelled samples from the labelled classes, and the distribu-
treated
examine the class prediction probabilities. Generally, the maximum softmax
Given an augmented image view 2;, its fo-oomallzzed feature in the semantic distribution feature space is fü (=(((m)))/llo (o(=:))) Subsequently, the output of the k-th binary classifier is
5

---

Published as a conference paper at ICLR 2025
o softmax(x*(f))). where os,k ( and of t 1. For labelled samples, multi-
binary cross-entropy loss with hard-negative sampling strategy Saito et al. (2021) is employed:
Σ iEB,
log
min log(o kFy.
(4)
1
where y; represents the ground-truth category label of the sample æ;. For unlabelled samples, an en- tropy minimization technique Saito & Saenko (2021) is applied to improve low-density separation:
Sat
Oi log(oty) + 0i log(o,,))
(5)
Bu BEB.
where B, denotes the unlabelled subset in current mini-batch. The loss function for the semantic distribution learning is defined as:
Ludl Lidt + L%
By optimizing Ludt, our detector distinctly segregates the feature distributions between known and unknown categories. Additionally, generates predicted score based on the maximum output from all M binary classifiers, denoted as:
Ly, Up arg max %
This score will serve as a crucial cue for the debiased learning be introduced next.
4.2 AUXILIARY DEBIASED LEARNING
As depicted in Fig. 2, the raw features are normalized to the clustering feature space in the lower branch, wherein novel categories are discovered. In order minimize the unintended negative im- pact of biased labels while maintaining the basic probability constraints Assran et al. (2022) and consistency regularization Caron et al. (2021) in the GCD classifier, we propose an auxiliary debi- ased learning mechanism. Specifically, a parallel debiased classifier t)s, initialized with K normal- ized prototypes C =fcf, .c., ck } trained in the same embedding space using debiased labels. Note that in our experiment, we only finetune the last two transformer blocks of the DINO Caron al. (2021) pre-trained ViT backbone. The k-th softmax score of sample æ; is given by:
(a(k) exp(h; ct/TTa
Σ Ej1exp((h ca//Ta)
where T, is the scaling temperature. The maximum classification score has demonstrated promising performance in several semi-supervised learning methods and we find it also a good indicator of sample quality in the context of GCD task. For an augmented view æ, and its GCD classifier prediction pa, debiasing threshold is set on the max(p.). with only samples surpassing being utilized to train the debiased classifier, expressed as 1 (max(p;; > T). Additionally, given that the semantic distribution detector and the GCD classifier paradigms, is essential to ensure the alignment of their predictions. Consequently, we introduce a function indicate the task consistency of these two tasks, defined
learned in different feature spaces and
(9) where y arg max(p;) represents the predicted category index by the GCD classifier, and gi: denotes its corresponding one-hot vector. This function aims selectively filter out samples with identical distribution predictions across the two tasks.
F(S. s.) = [í E P, A > 0.5) V 1 [ E JA 0.5),
Furthermore, as previously stated, given the inclusion of both known (in-distribution) and unknown -iut-of-distribution) samples in the unlabelled data, it is imperative to devise learning strategy based on semantic distribution information. With the training progresses, the semantic OOD scores gradually approach the two extremes (i.e., 0 and 1). The score of the unknown class sample steadily increases to 1, while the score of the known class gradually decreases to 0. Prior techniques Saito et al. (2021); Li et al. (2023) simply employ threshold to determine whether the sample belongs to the known or unknown. Such a naive method is unreliable and may introduce many noises to the model training for GCD. In approach, prioritize samples with distinct distributions for

---

Published as a conference paper at ICLR 2025
Algorithm End-to-end Training Algorithm for DebGCD.
Input: Set of labelled data D, = {( v/) set of unlabelled data D. (x;; v²)). Data aug- mentation function Model parameters w, learning rate 17, epoch Emax- iteration manx trade-off parameters, A Aadi; for Epoch= to Emarx do for Iteralãon = to Iaaz do
Sample labelled data B₁, unlabelled data Bui E B,,
Compute model prediction P.. Pf loss function Lgeds add Compute debiased label y4; task consistency Flyic, si)
// Eq.3,6,8
Eq.9
Compute loss function Lad Cadll Ladt
// E/q.11,12,13
Compute loss function Call = Lged + Cadl + Cadi
Update model parameters
1i V m fall
end end
Output: Model parameter w.
self-training, aligning with the principles of curriculum learning. To establish consistent metric for assessing sample discriminability, we introduce a normalized distribution certainty score:
d; 2 - 1/:
(10)
which approaches the value 0 for ambiguous samples and the value for certain samples. This score, to certain extent, indicates the learning status of samples and can serve as a crucial cue for our debiased classifier. Therefore, the auxiliary debiased learning loss for unlabelled samples is written as:
1 Σ ((max(p)) 7) F(Je, 8t) d, x tee (p 9.).
Cudt
(11)
Bu iEB.
In this manner, the training of the debiased classifier transforms into a curriculum learning process, where easily identifiable samples that are clearly semantic in-distribution or out-of-distribution are given higher priority for learning. Moreover, debiased classifier also retains the prior knowledge from the labelled data. For the labelled samples, it's is simply trained with the cross-entropy loss:
Ciall
Σ Loe (p.: y:).
(12)
BI
FEB.
Finally, the overall training loss for the debiased classifier is:
Ladi = Ladll + Ludt-
(13)
In this manner, all the samples are trained using one-hot hard labels, irrespective their belongings known unknown categories. Operating within the feature space, debiased classi- fier collaborates closely with the GCD classifier, thereby facilitating the joint optimization of the clustering feature space.
4.3 LEARNING AND INFERENCE FRAMEWORK
Based on the baseline GCD classifier, our framework is designed to be trained in multi-task man- ner. Different from previous approaches in the open-set literature Yu et al. (2020), our DebGCD framework employs a one-stage training process, climinating the necessity for task-specific warm- up phases. Consequently, the three tasks can be jointly trained end-to-end with the overall loss:
Call Eyed Xedi + Audi
(14)
where Aed and Aud denote the loss weights for the semantic distribution detector and debiased classifier, respectively. The complete training pipeline of the framework is illustrated in Algorithm 1.
Throughout the joint training process, the three branches are collectively optimized an end-to- end manner. During inference, only the GCD classifier is retained. This indicates that method does not impose any additional computational overhead compared to the baseline approach during inference, further emphasizing its simplicity and efficiency.
7

---

Published as a conference paper at ICLR 2025
5 EXPERIMENTS
In this section, we present a comprchensive evaluation of the proposed DebGCD framework and fur- ther perform meticulous ablation studies to showcase the effectiveness of its individual components. More results and analysis can be found in the Appendix.
5.1 EXPERIMENTAL SETUP
Datasets. We conduct comprehensive evaluation of our method across diverse benchmarks, en- compassing the generic image recognition benchmark (CIFAR-10/100 Krizhevsky et al. (2009), ImageNet- 100 Deng et al. (2009)), the Semantic Shift Benchmark (SSB) Vaze al. (2022c) com- prising fine-grained datasets CUB Wah et al. (2.11). Stanford Cars Krause et al. (2013), and FGVC- Aircraft Maji et al. (2013), along with the challenging ImageNet- IK Deng et al. (2009). For each dataset, we adhere to the data split scheme detailed in Vaze et al. (2022b). The method involves sampling a subset of all classes as the known ('Old") classes Vt- Subsequently, 50% of the images from these known classes are utilized to construct Dr, while the remaining images are designated as the unlabelled data Da. The statistics can be seen in Tab. 1.
Evaluation metries. We assess the GCD performance using the cluster- ing accuracy (ACC) in accordance with established conventions Vaze et al. (2022b). For evaluation, the ACC on Dj is computed as follows, given the ground truth y; and the pre-
Table I: Overview of dataset, including the classes in the la- belled and unlabelled sets (1%)) (D and counts of images ((Prl, [PuD. 'FG' denotes fine-grained.
Dataset
FG
[D]
[D,]
[Duu
CIFAR-10 Krizhevsky et al. (2009)
X 12.5K
37.5K
dicted labels U::
CIFAR-100 Krizhevsky et (2009)
X
20.0K
80
30.0K
100
ImageNet- 100 Deng et al. (2009)
X
31.9K
50
95.3K
100
Duul
CUB Wah et al. (2011)
/
1.5K
100
4.5K
200
ACC
1 [Dal
Σ
11y = h(j)),,
Stanford Cars Krause al. (2013)
/
2.0K
6.1K
FGVC-Aircraft Maji al. (2013)
1.7K
50
5.0K
100
ImageNet-IK Deng al. (2009)
X 321K
500
960K
1000
(15) where represents the optimal per- mutation that aligns the predicted
cluster assignments with the ground-truth class labels. ACC for 'All' classes, 'Old' classes and 'New' classes are reported for comprehensive assessment.
Implementation details. Following previous attempts in GCD Vaze et al. (2022b); Wen et al. (2023), our model is structured with ViT-B/16 Dosovitskiy et al. (2021) backbone pre-trained using DINO Caron et al. (2021), and the feature space centers around the 768-dimensional classification token. The projection networks for representation learning and semantic distribution detection com- prise three-layer and five-layer MLPs, respectively. The model is trained with batch size of 128, initiating with an initial learning rate of 10 which decays to 10 using a cosine schedule over 200 epochs. Notably, the loss weights Xeat and Aad! are set to 0.01 and 1.0, while the loss balancing weight Aged is assigned to 0.35 following Wen et al. (2023). Regarding the temperature parameters, the initial temperature Ti is established at 0.07, subsequently warmed up to 0.04 employing a cosine schedule during the first 30 epochs, whereas the other temperatures are set to 0.1.
5.2 BENCHMARK RESULTS
We present benchmark results of our method and compare it with state-of-the-art techniques in gen- eralized category discovery (including ORCA Cao et al. (2022), GCD Vaze al. (2022b), XCon Fei et al. (2022), OpenCon Sun & Li (2022), PromptCAL Zhang et al. (2023), DCCL Pu et al. (2023), GPC Zhao et al. (2023), CiPR Hao al. (2024), SimGCD Wen al. (2023), pGCD Vaze et al. (2023), InfoSieve Rastegar et al. (2023), and SPTNet Wang et al. (2024b)). as well as robust base- lines derived from novel category discovery (RankStats+ Han et al. (2021), ), UNO+ Fini al. (2021), and k-means MacQueen (1967)). All methods are based on the DINO Caron et al. (2021) pre-trained backbone. This comparative evaluation encompasses performance on the fine-grained SSB bench- mark Vaze et al. (2022c) and generic image recognition datasets Krizhevsky et al. (2009); Deng al. (2009), as shown in Tab. and Tab. 3.
Results on SSB. As shown in Tab. 2, DebGCD demonstrates superior performance across the three datasets, achieving an average ACC of 64.4 on 'All' categories. surpassing the second-best SPT- Net Wang et al. (2024b) by 3%. It maintains the best both Stanford Cars and FGVC-Aireraft
8

---

Published as a conference paper at ICLR 2025
Table 2: Comparison of state-of-the-art GCD methods SSB Vaze et al. (2022c) benchmark.
Results are reported in ACC across the 'All', 'Old' and 'New' categories.
CUB
Stanford Cars
FGVC-Aircraft
Average
Method
All
Old
New
All
Old
New
All
Old
New
All
k-mcans MacQueen (1967)
34.3
38.9
32.1
12.8
10.6
13.8
16.0
14.4
16.8
21.1
RankStats+ Han al. (2021)
33.3
51.6
24.2
28.3
61.8
12.1
26.9
36.4
22.2
29.5
UNO+ Fini al. (2021)
35.1
49.0
28.1
35.5
70.5
18.6
40.3
56.4
32.2
37.0
ORCA Cao al. (2022)
35.3
45.6
30.2
23.5
50.1
10.7
22.0
31.8
17.1
26.9
GCD Vaze et al. (2022b)
51.3
56.6
48.7
39.0
57.6
29.9
45.0
41.1
46.9
45.1
XCon Fci ct al. (2022)
52.1
54.3
$1.0
40.5
58.8
31.7
47.7
44.4
49.4
46.8
OpenCon Sun & Li (2022)
54.7
63.8
54.7
49.1
78.6
32.7
PrompiCAL Zhang
(2023)
62.9
64.4
62.1
50.2
70.1
40.6
52.2
52.2
52.3
55.1
DCCL, Pu et al. (2023)
63.5
60.8
64.9
43.1
55.7
36.2
GPC Zhao et al. (2023)
52.0
55.5
47.5
38.2
58.9
27.4
43.3
40.7
44.8
44.5
SimGCD Wen et al. (2023)
60.3
65.6
$7.7
53.8
71.9
45.0
54.2
59.1
51.8
56.1
2GCD Vaze et al. (2023)
65.7
68.0
64.6
56.5
68.1
50.9
53.8
55.4
53.0
58.7
InfoSieve Rastegar et (2023)
69.4
77,9
65.2
55.7
46.4
56.3
63.7
60.5
CiPR Hao et al. (2024)
57.1
58.7
55.6
47.0
61.5
40.1
SPTNet Wang al. (2024b)
65.8
68.8
65.1
59.0
79.2
49.3
59.3
61.8
58.1
61.4
DebGCD
66.3
71.8
63.5
65.3
81.6
57.4
61.7
63.9
60.6
64.4
Table 3: Comparison of state-of-the-art GCD methods on generic datasets. It includes CIFAR-
10 Krizhevsky et al. (2009), CIFAR- 100 Krizhevsky et al. (2009), ImageNet- Deng et al. (2009), and ImageNet- IK Deng al. (2009) dataset.
CIFAR 10
CIFAR-10 100
ImageNet- 100
ImageNet-IK
Method
All
Old
New
All
Old
New
All
Old
New
All
Old
New
k-meuns MacQueen (1967)
83.6
85.7
82.5
52.0
52.2
50.8
72.7
75.5
71.3
RankStatst Han al. (2021)
46.8
19.2
60.5
58.2
77.6
19.3
37.1
61.6
24.8
UNO+ Fini (2021)
68.6
98.3
69.5
80.6
47.2
70.3
95.0
57.9
ORCA Cao
(2022)
69.0
77.4
52.0
73.5
92.6
63.9
8188
86.2
79.6
GCD Vaze et al. (2022h)
91.5
97.9
88.2
73.0
76.2
66.5
74.1
89.8
66.3
52.5
72.5
42.2
XCon Fei et al. (2022)
96.0
97.3
95.4
74.2
81.2
60.3
77.6
93.5
69.7
OpenCon Sun Li (2022)
84.0
93.8
81.2
PrompiCAL, Zhang al. (2023)
97.9
96.6
98.55
81.2
84.2
75.3
83.1
92.7
78.3
DCCL Pu al. (2023)
96.3
96.5
96.9
75.3
76.8
70.2
80.5
90.5
76.2
GPC Zhao ct al. (2023)
90.6
97.6
87.0
75.4
84.6
60.1
75.3
93.4
66.7
SimGCD Wen
(2023)
97.1
95.1
98.1
80.1
81.2
83.0
93.1
77.9
57.1
27.3
InfoSieve Rastegur et al. (2023)
94.8
97.7
93.4
78.3
82.2
70.5
80.5
93.8
73.8
CiPR Hao et al. (2024)
97.7
97.5
97.7
81.5
82.4
79.7
80.5
84.9
78.3
SPTNet Wang et al. (2024b)
97.3
95.0
98.6
81.3
84.3
75.6
85.4
93.2
814
DebGCD
97.2
94.8
98.4
83.0
84.6
79.9
85.9
94.3
81.6
65.0
$2.0
56.5
dataset, while ranking second on CUB, where is outperformed only by InfoSieve Rastegar et al. (2023), hicrarchical encoding method specifically designed for fine-grained GCD. In contrast, De- bGCD aims for broader improvements across both generic and fine-grained datasets. These results reveal DebGCD's exceptional ability to uncover new categories, while also showcasing remarkable performance in recognizing known categories.
Results generic datasets. In Tab. 3, report results three widely used generic datasets (CIFAR-10, CIFAR- 100 and ImageNet-100 100) in GCD, as well as the challenging ImageNet-IK. K. Our method attains superior performance in terms of ACC across 'All' categories, establishing the new state-of-the--at, except CIFAR-10, on which the performance is nearly saturated (over 97% ACC) for our method and other most competitive methods. On the challenging ImageNet-IK, IK, containing 1, 000 classes with diverse images, DebGCD also establishes the new state-of-the-art, surpassing the previous pess-performing method by 7.9%. These results validate the effectiveness and robustness of our method for generalized category discovery on generic datasets.
5.3 ANALYSIS
In this section, we provide ablations regarding the key components within our framework. Besides, study the impact of the debiasing threshold 7 and labelled data.
Framework components. Starting with the baseline method trained using Lyed (Row (1)), we gradually incorporate proposed techniques the Stanford Cars dataset, as depicted in Tab. 4. An intuitive approach is to apply debiased learning to the original classifier as in Row (2). However, this still produces biased supervision signal because it relies the original GCD loss for that

---

Published as a conference paper at ICLR 2025
Table 4: Ablations. The results regarding the different components in our framework Stanford
Cars Krause et al. (2013), ACC of 'All', 'Old' and 'New' categories are listed.
Debiased Learning
Auxiliary
Semantic Dist.
Dist. Guidance
Stanford Cars
Classifier
Learning
All
Old
New
53.8
71.9
45.0
51.3
72.8
40.9
58.5
78.7
48.8
X
56.5
73.3
48.3
V
60.7
78.1
52.3
65.3
81.6
57.4
classifier. It turns that such naive approach may even hurt the performance. Rows (1) and (2) indicate that directly applying debiased learning to the GCD classifier can lead to a decrease in performance, particularly affecting novel categories. The introduction of auxiliary classifier in Row (3) demonstrates significant performance enhancements. Similarly, our semantic distribution learning alone results in 2.7% improvement across all categories in Row (4). Row (5) highlights that co-training the debiased classifier and semantic distribution detector further boosts performance.
Notably, guiding the debiased learning with semantic distribution certainty and task consistency function yields notable 4.6% performance increase in Row (6).
Loss function. addition, we explore the impact of the data and the respective loss functions employed during the train- ing of debiased classifier, denoted as Caa and Loa targeting the labelled and unlabelled datasets, respectively. The results are shown in Tab. 5. These experiments are undertaken on the FGVC-Aircraft Maji et al. (2013) using various subset com- binations. Solely training with Ledt introduces bias towards known categories, leading to notable performance decline. Conversely, exclusive training with L"a fails to reach optimal performance levels, underscoring the essential role of knowl- edge derived from labelled data. These outcomes demonstrate the vital significance of both Cid and L in optimizing the debiased classifier.
Table 5: Experimental results distillation data by using different loss functions.
FGVC-Aireraft
Cu
Cun
54.2
51.8
53.1
60.5
49.4
57.9
60.1
56.9
61.7
63.9
60.6
Debiasing threshold т. Similar to self- olf-trraiing approaches Sohn et al. (2020); Zhang et (2021), the selection of the threshold for gen- erating pseudo-labels also plays crucial role in approach. Consistent with the methods outlined Wen et al. (2023) and Vaze et al. (2022b), we calibrate the threshold based on its performance on separate validation set of the labelled data. Detailed results regarding differ- ent thresholds on the FGVC-Aireraft Wah et al. (2011) dataset, covering performance on both the unlabelled training dataset and the validation set, are presented. As shown in Tab. 6, the threshold is incrementally adjusted in intervals of 0.05. No- tably, the performance trends for both datasets align, with optimal performance achieved when the threshold is set to 0.85.
Table 6: Experimental results regarding thresh- old the unlabelled set and validation set of FGVC-Aircrafi Maji et al. (2013) dataset.
Unlabelled Set
Validation Set
All
Old
New
All
Old
New
0.90
59.4
64.7
56.7
58.9
61.1
56.8
0.85
61.7
63.9
60.6
61.1
62.0
60.3
0.80
60.7
61.5
60.3
60.6
61.6
59.6
CONCLUSION
This paper presents DebGCD, a distribution- guided debiased learning framework for GCD, com- prising three primary components. Firstly, we introduce an auxiliary debiased learning mechanism by concurrently training parallel classifier with the GCD classifier, thereby facilitating optimiza- tion in the GCD feature space. Secondly, semantic distribution detector is introduced to explicitly identify semantic shifts and implicitly enhance performance. Lastly, we propose a semantic distribu- tion certainty score that enables curriculum-basee learning approach, promoting effective learning for both seen and unseen classes. Despite its simplicity, DebGCD showcases superior performance, evidenced by comprehensive evaluation
public benchmarks.
10

---

Published as a conference paper at ICLR 2025
Acknowledgements. This work is supported by National Natural Science Foundation of China (Grant No. 62306251), Hong Kong Research Grant Council Early Career Scheme (Grant No. 27208022), and HKU Seed Fund for Basic Research.
REFERENCES
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. In ECCV, 2022.
Hyojin Bahng. Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In ICML, 2020.
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In CVPR, 2016.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer woman is to homemaker? dehiasing word embeddings. In NeurIPS, 2016.
Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi-supervised learning. In ICLR, 2022.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in sule-vsd vision transformers. In ICCV, 2021.
Fernando Julio Cendra, Bingchen Zhao, and Kai Han. Prompteed: Learning gaussian mixture prompt pool for continual category discovery. In ECCV, 2024.
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi-supervised learning. IEEE Trans- actions on Neural Networks, 2009.
Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. Fairfil: Contrastive neural debiasing method for pretrained text encoders. In ICLR, 2021.
Jinwoo Choi, Chen Gao, Joseph CE Messou, and Jia-Bin Huang. Why can't i dance in the mall? learning to mitigate scene bias in action recognition. In NeurIPS, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
Yue Fan, Anna Kukleva, Dengxin Dai, and Bernt Schiele. Ssb:
Simple but strong baseline for boosting performance of open-set semi-supervised learning. In ICCV, 2023.
Yixin Fei, Zhongkai Zhao, Siwei Yang, and Bingchen Zhao. Xcon: Learning with experts for fine- grained category discovery. In BMVC, 2022.
Enrico Fini, Enver Sangineto, Stéphane Lathuiliere, Zhun Zhong, Moin Nabi, and Elisa Ricci. A unified objective for novel class discovery. In ICCV, 2021.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis. Matthias Bethge, Felix A Wichmann, and Wieland Brendel. mmagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. In ICLR, 2020.
Chuanxing Geng, Sheng-jun Huang, and Songean Chen. Recent advances in open set recognition: A survey. IEEE TPAMI, 2020.
Ross Girshick. Fast r-enn. In ICCV, 2015.
Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In ICCV, 2019.
11

---

Published as a conference paper at ICLR 2025
Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Automatically discovering and learning new visual categories with ranking statistics. In ICLR, 2020.
Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Autonovel: Automatically discovering and learning novel visual categories. IEEE TPAMI, 2021.
Shaozhe Hao, Kai Han, and Yean-Yee K Wong. Cipr: An efficient framework with cross-instance positive relations for generalized category discovery. TMLR, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog-
nition. In CVPR, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, 2017.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021.
Xuhui Jia, Kai Han, Yukun Zhu, and Bradley Green. Joint representation learning and novel category discovery on single-and multi-modal data. In ICCV, 2021.
Longlong Jing and Yingli Tian. Self-supervised visual feature learning with deep neural networks: A survey. IEEE TPAMI, 2020.
KJ Joseph, Sujoy Paul, Gaurav Aggarwal, Soma Biswas, Piyush Rai, Kai Han, and Vineeth N Balasubramanian. Novel class discovery without forgetting. In ECCV, 2022.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV workshop. 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, and Cihang Xie. Shape-texture debiased neural network training. In ICLR, 2021.
Zekun Li, Lei Qi, Yinghuan Shi, and Yang Gao. lomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization. In ICCV, 2023.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of diist---ist image detection in neural networks. In ICLR, 2018
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In
Shijie Ma, Fei Zhu, Zhun Zhong, Wenzhuo Liu, Xu-Yao Zhang, and Cheng-Lin Liu. Happy: A debiased learning framework for continual generalized category discovery. In NeurIPS, 2024.
James MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability Volume 1: Statistics, volume 5, pp. 281 281-298. University of California press, 1967.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.515/ 2013.
Safyy Umoja Noble. Algorithms of oppression: How search engines reinforce racism. In Algorithms of oppression, 2018.
Jean Ponce. Toward category-level object recognition, volume 4170. Springer Science & Business Media, 2006.
12

---

Published as a conference paper at ICLR 2025
Nan Pu, Zhun Zhong, and Nicu Sebe. Dynamic conceptional contrastive learning for generalized category discovery. In CVPR, 2023.
Nan Pu, Wenjing Li, Xingyuan Ji, Yalan Qin, Nicu Sebe, and Zhun Zhong. Federated generalized category discovery. In CVPR, 2024.
Sarah Rastegar, Hazel Doughty, and Cees Snoek. Learn to categorize or categorize to learn? self- coding for generalized category discovery. In NeurIPS, 2023.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS, 2015.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations. In ICML, 2020.
Kuniaki Saito and Kate Saenko. Ovanet: One-vs-all network for universal domain adaptation. In ICCV, 2021.
Kuniaki Saito, Donghyun Kim, and Kate Saenko. Openmatch: Open-set consistency regularization for semi-supervised learning with outliers. In NewrIPS, 2021.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. ICLR, 2015.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun- Li. Fixmatch: Simplifying semi- supeesupeee learning with consistency and confidence. In NewrlPS, 2020.
Yiyou Sun and Yixuan Li. Opencon: Open-world contrastive learning. TMLR, 2022.
Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. Long-tailed classification by keeping the good and removing the bad momentum causal effect. In NeurlPS, 2020.
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR, 2011.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: good
closed-set classifier is all you need? In ICLR, 2022a.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In CVPR, 2022b.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. The semantic shift benchmark. In ICML workshop, 2022c.
Sagar Vaze, Andrea Vedaldi, and Andrew Zisserman. No representation rules them all in category discovery. In NeurlPS, 2023.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Hongjun Wang, Sagar Vaze, and Kai Han. Dissecting out-of-distribution detection and open-set recognition: A critical analysis of methods and benchmarks. IJCV, 2024a.
Hongjun Wang, Sagar Vaze, and Kai Han. Sptnet: An efficient alternative framework for generalized category discovery with spatial prompt tuning. In ICLR, 2024b.
Hongjun Wang, Sagar Vaze, and Kai Han. Hilo: A learning framework for generalized category discovery robust to domain shifts. In ICLR, 2025.
Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, and Lei Li. Solo: Segmenting objects by locations. ECCV. 2020.
Xin Wen, Bingchen Zhao, and Xiaojuan Qi. Parametric classification for generalized category dis- covery: A baseline study. In ICCV, 2023.
13

---

Published as a conference paper at ICLR 2025
Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise signal: The role of image backgrounds in object recognition. In ICLR, 2021.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distributioo detection: A survey, arXiv preprint arXiv:2110.1333 2021.
Lu Yang, He Jiang, Qing Song, and Jun Guo. A survey on long-tailed visual recognition. IJCV, 2022.
Qing Yu, Daiki Ikami, Go Irie. and Kiyoharu Aizawa. Multi-task curriculum framework for open-set semi-supervised learning. In ECCV, 2020.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In ICML, 2013.
Bowen Zhang, Yidong Wang. Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi eemi-supervised learning with curriculum pseudo la- beling. In NeurIPS, 2021.
Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, and Fahad Shah- baz Khan. Promptcal: Contrastive affinity learning via auxiliary prompts for generalized novel category discovery. In CVPR, 2023.
Xinwei Zhang, Jianwen Jiang, Yutong Feng, Zhi-fan Wu, Xibin Zhao, Hai Wan, Mingqian Tang, Rong Jin, and Yue Gao. Grow and merge: unified framework for continuous categories discov- ery. In NeurIPS, 2022.
Bingchen Zhao and Kai Han. Novel visual category discovery with dual ranking statistics and mutual knowledge distillation. In NeurIPS, 2021.
Bingchen Zhao, Xin Wen, and Kai Han. Learning supervipe gaussian mixture models for generalized category discovery. In ICCV, 2023.
14


Reviews:
[{"review_id": "wFLhNt5Xri", "reviewer": "Reviewer_YCCi", "summary": "The paper claims that existing GCD methods suffer from label bias, fail to account for differences in uncertainty, and do not address semantic distribution shifts. To address these issues, the author proposes D2G framework, which comprises Semantic Distribution Detection and Auxiliary Debiased Learning. The Semantic Distribution Detection module treats each labeled category as a separate binary classification, using the prediction confidence score obtained to filter and scale the debiased loss. The additional loss introduced by these components can be directly integrated with SimGCD and these modules can be entirely discarded during inference.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation behind addressing label bias is sound to me. Previous methods apply soft supervision to unlabeled data, which results in weaker supervision for unknown classes. The proposed method aligns well with this motivation.\n2. The approach achieves performance improvements demonstrating its effectiveness.\n3. The framework is efficient in inference.\n4. The writing is clear and easy to follow.", "weaknesses": "1. The distribution detector functions as multiple independent binary classification, so there is no competition between categories. It serves two purposes: first, it uses negative class confidence scores to filter out likely unknown classes in the final $L^u_{adl}$; second, it imposes stronger supervision on samples with higher uncertainty. For the first purpose, is there a significant difference in effectiveness compared to using self-entropy to filter unknown samples? Self-entropy would seem a more natural and straightforward metric, yet the author does not analyze the benefits of this one-vs-all design. For the second, Equation 10 imposes stronger pseudo one-hot supervision on samples deemed uncertain by the distribution detector. For example, if an unknown class is close to a known class, the loss will be reduced by $d_i$. The ablation study indicates that this yields significant performance gains, but lacks detailed analysis and discussion.\n2. The D2G framework finetunes more parameters than SimGCD, which only trains the last block. Since D2G builds on SimGCD, it would be more meaningful to compare performance under the same training setup. The authors did not provide this.\n3. There is a performance drop compared to the baseline on Herbarium19.\n4. Since all introduced modules can be discarded during inference, I think the key to performance improvement likely lies in the enhancement of the discriminability of DINO CLS token. However, the authors provide minimal discussion on this aspect.\n5. The description of the ablation studies is not sufficiently clear, and there is a lack of discussion between experiments. Specifically, regarding debiased learning, all debiased losses could theoretically be applied directly to the original classifier in SimGCD. It is unclear why the addition of a second classifier is necessary for effective performance. Additionally, I would like to know the impact on performance of removing the MLP prior to the OVA module.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cHNjOxoh87", "reviewer": "Reviewer_M5TQ", "summary": "This paper propose D2G a novel framework that addresses the challenging GCD task. Several new paradigms and mechanisms like debias learning in this framework enhance the model’s performance. Combined with these, the method proposed demonstrates its effectiveness and achieves superior performance on broad benchmark.", "questions": "please refer to weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper writing is clear and easy to understand.\n2. The topic of the article is of significant theoretical and practical importance, addressing a gap in the existing literature.\n3. The paper clearly outlines the shortcomings of previous studies and results section is logically organized.\n4. The proposed framework for DCG is novelty. Various incremental mechanisms make sense to me.", "weaknesses": "1. There is a error in fig.1 (d), the brown dish line is invisible.\n2. The hyperparameters in Eq. 14 are empirical values or obtained through experiments? If latter, I believe the authors should include some ablation studies for clarification.\n3. In Tab. 2, the performance of D2F is suboptimal compared to InfoSieve, but it lacks a specific analysis. Could the authors provide further details on this?\n4. The impact of the various method proposed, such as Debias learning and Auxiliary Classifier, should be evaluated through ablation studies on a broader dataset. The paper currently reports results only on the Stanford dataset. Do authors validate only on this dataset?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mLWSluDKZB", "reviewer": "Reviewer_59SW", "summary": "This paper presents the D2G (Debiased Learning with Distribution Guidance) framework for addressing the Generalized Category Discovery (GCD) problem. GCD is challenging due to label biases and semantic shifts between known and unknown categories. The D2G framework introduces a debiased learning paradigm, a semantic distribution detector, and a curriculum learning approach based on distribution certainty to address these issues. Extensive experiments demonstrate D2G’s superiority over existing GCD methods on various benchmarks.", "questions": "1. What strategies do authors envision to reduce potential overfitting during assisted debiasing learning, especially when utilizing limited unlabeled data on fine-grained datasets?\n2. Based on Table 4, it can be concluded that the effect of label debiasing is not very good. Have the authors considered not using the label debiasing strategy? For example, what are the results for \"w/o debiased learning, w/ auxiliary classifier, w/o semantic dist. learning, w/o dist. guidance\"?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-organized, with clear explanations of technical details.\n2. The introduction of a debiased learning framework specific to GCD with a multi-feature distribution approach is innovative.\n3. The technical contributions are well-structured and effectively evaluated. The integration of auxiliary debiased learning, semantic detection, and curriculum learning reinforces the model's performance.", "weaknesses": "1. The variation in results from the GCD benchmarks can be very large, so it is important to report all results as well as the error bars from the three independent runs, as SimGCD does in its Supplementary Information.\n2. While the authors claim that D2G does not introduce additional computational burdens during inference, a more detailed analysis of the training time and computational costs associated with the auxiliary components would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rvzOxOiIuc", "reviewer": "Reviewer_Q4NM", "summary": "The authors propose a novel framework called Debiased Learning with Distribution Guidance (D2G) for the GCD task, which introduces a debiased learning paradigm to optimize the clustering feature space and learns a semantic distribution detector to enhance the learning effect of GCD. Besides, D2G propose a curriculum learning mechanism that steers the debiased learning process to effectively mitigate the negative impact of uncertain samples. The authors evaluate the method on the public GCD benchmarks to demonstrate the effectiveness.", "questions": "1. I wonder whether the GCD method is sensitive to certain categories, resulting in limited performance improvement on some datasets. Perhaps the authors can design some experiments to test it.\n2. Is it better to use vision-language pre-trained large models such as CLIP to solve label bias problems, as CLIP contains a lot of pre-trained knowledge for new categories.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "D2G considers both label bias and semantic shift to address the challenging GCD task. It’s a novel idea to mark the first exploration of these aspects.\nD2G effectively incorporates all components into a unified framework and can be trained in a single stage without any additional computation burden.\nThe authors conduct extensive experimentation on public GCD benchmarks to demonstrate its effectiveness.", "weaknesses": "1. The reason for using OOD techniques to solve GCD task is not clear because the objectives of these two tasks are different. The motivation for using MLP projection network to solve this problem needs further explanation.\n2. From the experimental results, the performance improvement of the method is not significant, especially on the CUB dataset. Besides, there are few comparison methods on the ImageNet-1K dataset, which can lead to unreliable comparison results.\n3. Some hyperparameters lack ablation experiments to verify that the experimental method is optimal, including the number of layers in MLPs, the loss weights, and so on.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)