Happy: A Debiased Learning Framework for Continual Generalized Category Discovery
Authors: Shijie Ma
Fei Zhu
Zhun Zhong
Wenzhuo Liu
Xu-Yao Zhang
Cheng-Lin Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: novel category discovery
generalized category discovery
continual category discovery
continual generalized category discovery
Primary area: machine_vision
TL;DR: We propose a debiased framework to continually discover novel categories with minimal forgetting of old ones.

Abstract: Constantly discovering novel concepts is crucial in evolving environments. This paper explores the underexplored task of Continual Generalized Category Discovery (C-GCD), which aims to incrementally discover new classes from *unlabeled* data while maintaining the ability to recognize previously learned classes. Although several settings are proposed to study the C-GCD task, they have limitations that do not reflect real-world scenarios. We thus study a more practical C-GCD setting, which includes more new classes to be discovered over a longer period, without storing samples of past classes. In C-GCD, the model is initially trained on labeled data of known classes, followed by multiple incremental stages where the model is fed with unlabeled data containing both old and new classes. The core challenge involves two conflicting objectives: discover new classes and prevent forgetting old ones. We delve into the conflicts and identify that models are susceptible to *prediction bias* and *hardness bias*. To address these issues, we introduce a debiased learning framework, namely **Happy**, characterized by **H**ardness-**a**ware **p**rototype sampling and soft entro**py** regularization. For the *prediction bias*, we first introduce clustering-guided initialization to provide robust features. In addition, we propose soft entropy regularization to assign appropriate probabilities to new classes, which can significantly enhance the clustering performance of new classes. For the *harness bias*, we present the hardness-aware prototype sampling, which can effectively reduce the forgetting issue for previously seen classes, especially for difficult classes. Experimental results demonstrate our method proficiently manages the conflicts of C-GCD and achieves remarkable performance across various datasets, e.g., 7.5% overall gains on ImageNet-100. Our code is publicly available at https://github.com/mashijie1028/Happy-CGCD.

Happy: A Debiased Learning Framework for Continual Generalized Category Discovery
Shijie Ma 1.2 Fei Zhu Zhun Zhong*. Wenzhuo Liu! Xu-Yao Zhang Cheng-Lin Liu
MAIS, Institute of Automation, Chinese Academy of Sciences, China
2 School of Artificial Intelligence, University of Chinese Academy of Sciences, China Centte for Artificial Intelligence and Roboties, HKISI-CAS, China Schooo of Computer Science and Information Engineering, Hefei University of Technology, China School of Computer Science, University of Nottingham, NG8 IBB Nottingham, UK jashijie202188 ac. xyzθnlpr aa..ac..cn
Abstract
Constantly discovering novel concepts is crucial in evolving environments. This pa- per explores the underexplored task of Continual Generalized Category Discovery (C-GCD), which aims incrementally discover while maintaining the ability to recognize previously learned classes. Although several settings are proposed study the C-GCD task, they have limitations that do not reflect real-world scenarios. We thus study a more practical C-GCD setting, which includes more new classes be discovered over a longer period, without storing samples of past classes. In C-GCD, the model is initially trained on labeled data known classes, followed by multiple incremental stages where the model is fed with unlabeled data containing both old and new classes. The core challenge involves two conflicting objectives: discover new classes and prevent forgetting old ones. We delve into the conflicts and identify that models are susceptible prediction bias and hardness bias. address these issues, we introduce a debiased learning framework, namely Happy, characterized by Hardness aware prototype sampling and soft entropy regularization. For the prediction bias, we first introduce cesstrringguudee initialization to provide robust features. In addition, we propose soft entropy regularization to assign appropriate probabilities classes, which can significantly enhance the clustering performance of new classes. For the harness bias, we present the hardness aware prototype sampling, which effectively reduce the forgetting issue for previously seen classes, especially for difficult classes. Experimental results demonstrate our method proficiently manages the conflicts of C-GCD and achieves remarkable performance across various datasets, e.g., 7.5% overall gains on ImageNet- 100. Our code is publicly available https: // g thub. com mwwhi mashijie1022 Hatp:iie//1//////000000 Happp-CGCD.
classes from unlabeled data
new
Introduction
In the open world [1, 2, 3]. visual concepts are infinite and evolving and humans can cluster them with previous knowledge. It is also important to endow AI with such abilities. In this regard, Novel Category Discovery (NCD) [4, 5, 6] and Generalized Category Discovery (GCD) [7, 8. 9, 10] endeavor to transfer [4, 11] the knowledge from labeled classes to facilitate clustering new classes. However, they are constrained to static settings where models only learn once, which contradicts ever-changing world. Thus, extending them to the temporal dimension is important. In the literature, Continual Novel Category Discovery (C-NCD) [12, 13, 14] and Continual Generalized Category
"Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Initial (Labeled)
Continual Generalized Category Discovery (Unlabeled)
Training
Stage-O
Stage-1
Stage-2 2
Stage-T
Data
Oldd
panda
tiper
panda
tiger
panda
tiger
discover-3 discoven
pands
tiper
discoven1 discover
discoven-T
Classifier
Old
New
Old
New
Old
New
besoms oid in stage
Cae
Figure I: The diagram of Continual Generalized Category Discovery (C-GCD). In this paper, we focus on more pragmatic setting with (1) more continual stages and more novel categories, (2) rehearsal-free learning, and (3) no prior knowledge of the ratio of new class samples.
Discovery (C-GCD) [15, 16, 17, 18] aim to discover novel classes continually. C-NCD assumes all data come from new classes, while C-GCD further considers the coexistence of old and new ones. However, C-GCD still has some limitations, e.g., some works [15, 17] store labeled data of old classes, causing storage and privacy [19] issues. Others [16, 18] consider limited incremental stages and novel categories or assume a prior ratio of known samples [15], failing to reflect practical cases.
In this paper, we tackle the task of C-GCD, but with more realistic considerations: (1) More learning stages with more new classes. (2) At each stage, data from previous stages are inaccessible [20] for storage and privacy concerns. (3) Unlabeled data contain samples from old classes but are fewer than new ones in each class, and the ratio of them is unknown. The C-GCD setting is illustrated in Figure I with two phases: (1) Initial supervised learning (Stage-O). The model is trained on labeled classes to acquire general knowledge. (2) Continual unsupervised discovery (Stage-1 ~ 7'), At each stage, the model learns from unlabeled data containing both new and old classes. Note that, old classes include initially labeled classes well as those discovered in previous stages. The core challenge is managing the conflicts between discovering new classes and preventing forgetting old ones.
To explore the nature of the conflicts, we conducted preliminary experiments (Section 3.2) which reveal two issues: Models (I) tend to misclassify new classes as old, leading to collapsed accuracy of new classes, and (2) exhibit catastrophic forgetting of old classes. We summarize them as underlying issues to be addressed: (1) Models display overconfidence in old classes and severe prediction bias. (2) The features of old classes are disrupted when learning novel classes. Meanwhile, the similarity between clusters varies, leading to biased hardness across classes for classification.
To address these issues, we propose a debiased framework, namely Happy, which is characterized by Hardness- aware prototype sampling and soft entropy regularization. Specifically, on the one hand, to better discover new classes during incremental stages, we utilize clustering guided initialization for new classes, ensuring a reliable feature distribution. More importantly, to mitigate the prediction bias between new and old classes, we introduce soft entropy regularization to allocate necessary probabilities the classification head of new classes, which is essential for new class discovery. On the other hand, to prevent catastrophic forgetting in rehearsal- Saeeee C-GCD, we model the class-wiss distribution in the feature space for old classes, and sample them when learning novel classes, which significantly mitigates catastrophic forgetting. Furthermore, we devise metric to quantify the hardness of each learned class, and prioritize sampling features from categories with greater difficulty. This helps the model to consolidate difficult knowledge accordingly and thus improves the overall performance. Consequently, these designs enable our model to specifically address the challenges in C-GCD, i.e., effectively discover new classes while preventing catastrophic forgetting of old classes.
In summary, our contributions are: (I) We extend Continual Generalized Category Discovery (C- GCD) to realistic scenarios. In addition, we propose debiased learning framework called Happy, which excels effectively discovering new classes while preventing catastrophic forgetting with reduced bias in the introduced C-GCD settings. (2) We propose cluster-guided initialization and soft entropy regularization for collectively ensuring stable clustering of new classes. On the other hand, we present hardness-aware prototype sampling to mitigate forgetting. (3) Comprehensive experiments show that our method remarkably discovers new classes with minimal forgetting of old classes, and outperforms of-te-of-hheartt methods by large margin across datasets.

---

2 Related Works
Category Discovery. Novel Category Discovery (NCD) [4, 5, 21] is firstly formalized as deep transfer clustering [4], i.e., transferring the knowledge from labeled classes to help cluster new ones. Early works employ robust rank statistics [5, 22] for knowledge transfer. UNO [6] proposes a unified objective with Shinkhorn- Knopp algorithm [23]. Later works [24, 25, 26] exploit relationships between samples and classes. NCD assumes unlabeled data only contain new classes. Instead, Generalized Class Discovery (GCD) [7, 27] further permits the existence of old classes. Thus models need to classify old classes and cluster new ones in the unlabeled data. Recent works handle GCD with non-parametric contrastive learning [8, 28, 10] or parametric classifiers with self-training [9. 29, 30]. More recent works explore GCD in other settings, e.g., active learning [31] and federated learning [32]. In summary, both NCD and GCD are limited to static settings where models only learn once.
Continual Category Discovery. Pioneer works [12, 13, 14] study the incremental version of NCD, assuming unlabeled data only contain new classes, and we call them C-NCD. Recent works [15. 16, 17, 18] explore the incremental version of GCD, we collectively refer to them as Continual Generalized Category Discovery (C-GCD). GM [15] proposes a framework of growing and merging. In the growing phase, the model performs novelty detection and implements clustering on the novelties. Then GM integrates the newly acquired knowledge with the previous model in the merging stage. Kim et al. [16] utilize noisy label learning and the proxy and anchor scheme to split the data in C-GCD. Zhao et al. [17] propose a non-parametric soft nearest-neighhor classifier and a density-based sample selection method. Orthogonally, Wu et al. [18] argue that the initial labeled data are not fully exploited and present a meta-learning [33] framework to learn a better initialization for continual discovery. Despite effecttiveness, C-GCD settings studied by the above methods still have some limitations, e.g., the number of stages is very few with limited new classes, and the assumption of prior ratio of old classes storing previously labeled samples is unrealistic [19, 34]. In this paper, we extend C-GCD to more pragmatic scenarios, as shown in Figure 1.
3 Preliminaries
We first formalize Continual Generalized Category Discovery (C-GCD) (Section 3.1). To delve deeper into the issues, we conduct preliminary experiments (Section 3.2). Results reveal that models susceptible two types of bias, which significantly degrade the performance and motivate to propose the debiased learning framework in Section 4.
3.1 Problem Formulation and Notations
Task Definition. As shown in Figure 1, C-GCD has two phases: (1) Initial supervised learning (Stage-O). The model is trained on labeled data D {a0% 1)) of initially labeled classes Co C to learn general knowledge and representations. We denote Chew = None. (2) Continual unsupervised discovery (Stage-1 ~ T). At Stage-l (1 < ( S T), the model is fed with unlabeled Fi=p which contains both old and new classes. We denote the categories UC new- K ICial K IChool and K = K + Ktt new denote the number of "old", "new" and "all" classes respectively. Note that, after the first stage, i.e., when t 2 2, "old" classes include initially labeled classes Ch and all new classes discovered in previous
dataset D Di = {xx) 1.N*
in Di as C = C. U
stages, i.e., C. = CD U (Chow) 1-1 and "new" classes refer to the classes unseen before. At the next stage, new classes from the current stage become the subset of old classes, i.e., Ch = Ct-1 UC The number of novel classes K at stage t is known a-prior estimated using off-the-shelf methods [5, 7, 35] in advance. After training of each stage, the model will be evaluated on the disjoint test set Diest = ((x,, 1 Ni containing all seen classes Caa U C Chew-
Realistic Considerations. Our C-GCD is more realistic than prior arts [15, 16, 18) in that: (1) More stages with more new classes to be discovered. (2) Rehearsal -free. Previous samples are inaccessible for storage and privacy issues. (3) At each continual stage, old classes have fewer samples per class than new classes in the unlabeled data, and the proportion of old samples is unknown.
Notations.
At Stage-t, we decompose the model into encoder ff( and parametric classifier g'; Kt Kin
(o Ki : for
[j=1] with head of old and new classes. The classifier is Eg-normalized without 1. The encoder maps the input 2; to feature vector z; f/(x)) E R.. Here,
bias term, Le., ||
3

---

8
32
CA8. [b] Reason
S0-MA
(x)
Issue
Figure 2: Preliminary results. We identify two issues and underlying causes, including (a) Issue 1: performance gap between old and new classes, caused by (b) Reason 1: model's overconfidence in old classes, i.e., prediction bias. (c) Issue 2: accuracy fluctuations in new class across various stages, caused by (d) Reason 2: different categories have varying levels of difficulty, i.e., hardness bias.
we use normollle hyperspectral feature space, ie., 2; = = The classifier finally produces probability distribution p. (z)((=)))T)) RRK using softmax function (())
3.2 Preliminary Experiments: Two Bias Problems
We conduct preliminary experiments on CIFARIO0 100 [36] using the model described in Section 3. which is initially trained on D and continually discovers new classes on Di using unsupervised self-training scheme [9]. Results reveal that models are prone to the following two types of bias.
Prediction bias in probability space. As illustrated in Figure 2 (a), the model's accuracy for new classes has collapsed. The reason is that old classes Cold are trained under full supervision while new classes are under unsupervised self-training [9, 37]. which brings about overconfidence [38,39, 40] in old classes, as in (b). In this case, prediction bias could occur, where some new classes are incorrectly predicted as old ones, which motivates to constrain the model to give necessary attention and predictive probabilities to new classes to compensate for this intrinsic gap, as discussed in Section 4.2.
Hardness bias in feature space. After adding constraints to ensure learning new classes (Section 4.2), their accuracies significantly fluctuate across incremental stages, leading unstable performance, as shown in Figure 2 (c). The underlying cause is that some clusters are more similar to others in the feature space, resulting in lower accuracy of these difficult classes. in (d), hardness bias (defined in Section 4.3) is obvious across classes. This paper focuses on the hardness of previously learned categories Ca and addresses how to avoid these biases in preventing forgetting in Section 4.3.
4 The Proposed Framework: Happy
Overview of the Method. As shown in Figure 1, C-GCD has two phases: (1) Initial supervised
learning (Stage-O). The model trained on labeled samples of C (Section 4.1). Our contribution mainly lies in (2) Continual unsupervised discovery (Stage- 1~ T). Motivated by the conflicts between new class discovery and the forgetting of old classes, as well as the two types of bias discussed in Section 3.2, we propose the debiased learning framework Happy as illustrated in Figure 3. Specifically, for category discovery, we propose initialization of new heads and soft entropy regularization to resist prediction bias (Section 4.2). To mitigate forgetting, we consider hardness bias and present hardness-awand aware prototype sampling (Section 4.3). The overall objective is derived in Section 4.4.
4.1 Supervised Training at the Initial Stage
At Stage-O, the model is trained on labeled data D from large number of classes CO to learn general representations, which serves the foundation for subsequent continual category discov- ery. We use standard supervised cross-entropy loss the batch B: Lets T Eien -V, log P.. where p. = o 0g (F:():)))/ denotes the prediction. To reduce overfitting, we further employ supervised [41] and sulf-supe contrastive learning [42] in the normmalle projection space:
log Eng, exp(h, h',
exp(h, (g/TT))
1
Σ iEB
log
exp(h, h(/Te)
B
Ende exp(h; h/
(FP(
(1)

---

loss for
-+ 58
Self-train
Soft Entropy Regularization
old class head class head
Encoder
old,in new,in
Σ
Pc prototypes
class-ovis
Dall
batch-ovise merage
P
SE: stop grad
Pi predictive prob
Feature Distillation
Classifier
Prototype Sampling
Hardness-aware
9.
prob
marginal
Clustering-gue guided
old,new
he class-wise
Initialization
he
Phariess hardness distribution
Clustering-guider Initialization
Hardness-awe awwrddeess-wwree Prototype Sampling
class sampling Pursdocw
large similarity
Рraniuae
h
small similarity
coof Pai
My
bg
init new heed
N(
Classifior
cluster centroid
My
feature sampling
Figure 3: Illustration of the proposed Happy framework. Top: Overall learning pipeline for continual stages. Bottom Left: Clustering-gau guided Initialization, together with Soft Entropy Regularization (Section 4.2) ensures effective novel class category. Bottom Right: Hardness-aware Prototype Sampling (Section 4.3) remarkably mitigates catastrophic forgetting of old classes.
where P(i) is the positive set with the same label and is temperature. The overall loss function is:
Linitial
Lets + A + (1 Ao)
(2)
4.2 Classifier Initialization and Soft Entropy Regularization
Continuously discovering unlabeled new classes is challenging, as prediction bias towards old classes could collapse new class accuracy (Section 3.2). Therefore, we need to constrain the model to pay more attention to new classes to ensure effective category discovery.
Clustering-guide Initialization. Randomly initialized classifiers bring about unstable training. We argue that clustering could provide good initialization for new classes. Specifically, Stage-t, we employ KMeans [43] on Diaiin and obtain Kt K + K fz-normalized cluster centroids { Among them, the Kiew centroids least similar to old heads, as measured by maximum cosine similarity with them, serve as the potential initialization for new class heads:
(t = topk, (- max C,2 gold), i= =, Kald-
Ct.y j =,1,- Ktt new
Group-wise Soft Entropy Regularization. Entropy regularization [9. 29, 30] is common to avoid trivial solutions of clustering in static settings. However, at each stage of C-GCD, there generally more old classes. Directly employing it equally across all classes will allocate most of the probability old classes, leading to prediction bias and collapsed performance (as in Figure (a, b)). To address this, we need to constrain the model explicitly. Considering that at each stage, there are fewer new classes but more samples per new class, and old classes have been well-learned previously, we propose to treat all old classes as a whole and the new classes as another, and derive C-GCD binary classification. Specifically, we first compute the marginal probability in the batch TE ΣiEB P:- Thus, Pold E E.Ec, (e) and Pnew E R Eeecy (()) are scalars indicating the marginal distribution on old and new classes respectively, where the superscript (c) denotes class indices and Pold + Puew 1. Then we propose soft entropy regularization on marginal distribution of the old and the new:
e R
Centropy Pota log Pold + Prew log Pnew
(4)
In this way, the model could focus more on each new class, ensuring reliable learning in new classes. We also employ entropy regularization within the new and old classes to avoid trivial solutions:
coldin
Σ cEC
p (og
log (c),
newwin entropy
Σ (e9) log (c).
(5)
GEC!
5

---

To sum up, the soft entropy regularization is employed in group-wise manner on three groups. i.e.,
"inter old-new" (Eq. (4)), "intra old" and "intra new" (Eq. (5)), and we add them together:
enntropy reg Centropy +
Coldinn
entropp entropy
The soft regularization ensures effective learning of new classes. See Section 5.4 for more discussions.
Overall Loss for New Class Discovery. To achieve self-training on unlabeled data, perform self-distillation [9, 37]. Specifically, we use another augmented view zy to produce sharpened qj with smaller temperature Tt. < Tp and employ cross-entropy loss supervise the prediction p;: 2 EieB ((g/ pa) ) + f(g.; The overall objective for new category discovery is:
Lnew Lsetf-train + X Centopy-rrg
where \ controls the importance of the proposed regularization loss.
4.3 Hardness-aware Prototype Sampling
Modeling Learned Classes. Catastrophic forgetting [44, 45, 46] is notorious problem in continual learning, especially when previous samples are inaccessible. Instead of storing seen samples, we can model the feature distribution for learned classes. Since the data in each incremental stage are unlabeled, at the end of each incremental stage, we perform class-wise Gaussian distribution in the feature space using models' predictions on Dirain
1h
Σ fa))), No 6ime
1
E.
E(t((e)) - (f)(s(e))) me) c=1, K" No
(8)
8:mc
where O arg maxe p denotes the prediction, Me and E. mean and covariance. Note that, for Stage-O, we directly use the ground-trute labels instead of predictions in Eq. (8). We call pa as prototypes. When learning new knowledge, one sample features from old classes N Ee), and classify them correctly to mitigate forgetting. We find a shared diagonal matrix [47] empirically works fine, i..,, D. rI, where is computed at Stage-O as r²
Eeeco Tr(E.)/d.
Incorporating Hardness to Learned Classes. As in Figure 2 (c), accuracy fluctuations across classes are significant, and treating all classes equally leads to hardness bias and suboptimal results. Intuitively, difficult classes should receive more attention during sampling. Here, we propose unsupervised metric, considering the samples with higher similarity to others more prone be confused and therefore more difficult. We define hardness h; and obtain hardness distribution as:
exp(he/Th)
h;
coS( 14:)
(5) Phondness ahha/Th)
(9)
KL
Σ Kis exp(h:/Tx)
where i = 1, Kit and Phardness is the categorical distribution to sample classes. Those with higher hardness are more likely be sampled, which better suppresses the forgetting of hard classes.
Sequential Sampling. We first sample categories from categorical distribution c Phanlness and then sample class-wise features from Gaussian distribution of the sampled classes 2, ~ - N rI) for classification. The loss for hardness-aware prototype sampling is
Cnpp Eowpheite Eg,
- (( - Ye log o (g (Tp)
(10)
Overall Loss for Mitigating Forgetting. As training proceeds, the feature space becomes outdated for previous prototypes, we thus apply knowledge distillation [48] using the last stage model and current training dataset, i.e., Lad [BT EIER cos( (f) (ax)) f6 f (ax)). The overall loss
Lotd
Lhap + A2
(11)
where Aa controls the weight of the knowedge distillation.
4.4 Overall Learning Objective
To continually discover new classes without forgetting old ones, we combine the losses for new (Eq. (7)) and old classes (Eq. (11)), and contrastive learning (Eq. (1)) formulate the final objective:
Litppy
Loew
Losd Cu
(12)

---

Table Performance of Ststage Continual Generalized Category Discovery (C-GCD) on CIFAR (C100), ImageNet- 100 (IN100), TinylmageNe (Tiny) and CUB. All methods have similar Stage-0 (S-0) ACC, which fair for evaluation on continual stages. Here denotes adjusted results.
5-0
Stage-1
Stage-2
Stage-3
Stage
Stage-5
Datasets
Methods
All
New
Old
New
Old
All
New
Old
New
KMcans [43]
66,
40.27
41,76
32,80
37.
38.33
30.00
36,20
37.6 63.63
26.20
36.66
38.30
23.50
35.69
36.79
25.80
VanillaGCD [7]
90.82
72.32
78.50
41.40
67.04
72.50
34.30
57.99
62.26
28.10
56.60
59,55
33.00
51.36
53.70
30.30
SimGCD [9]
90.36
73.37
86.44
8.00
62.56
72.43
3.30
54.17
61.61
2.10
47.62
53.37
1.60
43.53
47.86
4.60
C100
SimGCD+ (44)
90.36
75.93
87.04
67.07
75.33
17.50
64.33
54.31
58.71
19.10
50.49
53.90
19.80
FRoST [12]
90.36
76.87
79.58
63.30
65.31
68.88
43.90
58.01
61.09
36.50
49.27
50.90
36.20
48.03
48.17
46.80
GM [15)
90.36
76.58
79.80
60.50
71.10
74.52
50.60
63.51
68.16
31.00
59.74
62,51
37.60
54.11
54.74
48.40
MetaGCD [18]
90.82
76.12
83.60
38.70
69.40
72.82
48.
90
61.95
65.76
35.30
58.22
61.21
34,30
55.78
58.47
31.60
Happy (Ours)
90.36
(80.40
85.26
74.13
78.27
49.
68.23
70.86
49.50
62.26
63.75
50.30
59,99
60.96
51.30
KMcans (43]
85.56
54:90
57.04
44.20
54.73
56.37
44.90
54.67
56.66
40.80
54.63
56.25
41.70
53,92
56.18
33.60
VamillaGCD [7]
95.96
70.13
72.92
56.20
69.37
73:47
44,80
68.50
70.63
53.60
65.56
67.85
47.20
64.54
67.44
38.40
SimGCD [9]
96.20
79.67
91.68
19.60
70.23
78.83
18.60
61.90
67.43
23.20
56.67
60,92
22.60
52.90
56.40 21:40
SimGCD+ [44]
96.20
83.07
16 22.60
74.57
83.47
21.20
67.60
73.57
25.80
02.09
66.83
24.20
57.62
61.47 23.00
IN100
FRoST [12]
96.20
87,50
92.96 60.20
69.63
83.37
57.20
76.78
77.00
75.20
66,
68.65
46.4
63.82
66.40 40.60
GM [15]*
96.20
89.53
95.04
62.00
K2.34
86,93
$4,80
77.97
79.17
69.60
72.80
74.65
58.00
71.08
71.76 65.00
MetaGGD [18]
95.96
[75.27
78.20
60.60
73.79
75.93
54,90
69.35
72,20
49.40
67 22.22
70.
10
44.20
66.68
69.31
43.00
Happy (Ours)
96.20
91.22
95.36
70.40
87.83
90.83
69.80
85,22
86,40
77.00
81.93
83.00
73.40
78.58
79.11
73.80
KMeuns [43]
61.70
35.42
35.46
35.20
34.99
35.75
30.40
34.80
36.07
34.77
35,90
24.90
34.62
35.63
25.50
VanillaGCD [7]
84.20
55.93
58.92
41,00
54.96
58.58
33.20
52.82
55,74
32,40
48.81
51.46
27.60
45.94
48.06
26.90
SimGCD [9]
85.86
66.95
79.94
2.00
57.81
66.98
2,80
52.70
59.83
2.77
45.01
50.29
2.80
41.59
45.79
3.80
Tiny
SimGCD+ [44]
85.86
70.38
81.80
13.30
62.47
70.75
12.80
54.55
60:46
13:20
47.98
52.49
11.90
42.98
46.46
12.70
FRoST [12]
85.86
75.15
78.5
52.50
54.31
30:40
48.22
16.90
42.73
16:90
GM [15]*
85.86
76.42
82.40
46,50
68.87
73.82
39.20
58.68
63.43
25.40
52.86
57.21
18.10
46,90
50.62
13.40
MetaGCD [18]
84.20
60.88
64.90
40.80
57.20
61.03
34.20
54.36
57.19
34.60
50.83
53.59
28.80
48.
50.16
30.00
Happy (Ours)
85.86
78.85
82.40
61.10
71.34
76.18
42.30
64.68
68.70
36.50
58.49
60.64
41.30
54.56
56.66
35.70
KMeans [43]
43.93
32..54
30.76
41.18
31.19
30.53
35.20
29.28
42.09
28.
37.61
28, 17
27.01
38.53
VanillaGCD [7]
89:20
64.47
67.06
$1.93
58.15
60.65
42.91
54,10
56.40
37.91
49.98
51.33
39.32
46.84
46.58
49.14
SimGCD [9]
90.26
73.84
84.54
22.02
63.36
72.35
8.58
55.63
61.95
11.13
49.31
54.55
7.86
44.72
48.69
9.25
CUB
SimGCD+ [44]
90.26
75.62
85.55
25.97
66.32
73.93
13.68
57.40
63.28
16.26
51.11
55.72
14.27
45.79
49.29
14:28
FRoST [12]
90.26
77.03
83.95
43.53
50.77
53.46
34.33
46.42
49.31
26.09
39.40
41.47
23.08
34.55
35.12
29.45
GM [15]]
90.26
76.17
80.23
56.51
67.91
73.38
34.58
61.12
66.53
23.00
55.90
57.49
43.38
51,96
54,40
30.10
MetaGiC [18]
89.20
67.08
70.21
51.92
60.77
62.39
50.86
57,53
59.33
37.78
$1.90
52.22
49.40
49.60
49.96
46.38
Happy (Ours)
90.26
88.40
85.06
63.70
74.27
76.03
63.57
67.09
71.06
39.13
62.25
63,83
49.74
59.39
60.49
49.52
5 Experiments
5.1 Experimental Setup
Datasets. We construct C-GCD on four datasets: CIFARI00 [36] (C100), ImageNet- 100 [49] (IN100),
Table 2: Dataset splits of C-GCD setting. We show #classes and #images per class of different stages. #old denotes all previously learned classes.
[50] (Tiny) and CUB [51], each is split into two subsets: (1) Stage-O, where 50% of classes serving as int constitute initial labeled data. (2) Stage- 1 (T = 5 by default). At each stage, the new
Stage-0
Each Stage-t 1,... 5)
Datatset #class
Wimg/Wclass
#new #ing//hew Wimg//oold
remaining classes are evenly sampled classes, along with all previously learned classes to constitute continual unlabeled data. Detailed dataset statistics are shown in Table 2.
C1000
IN100
50
50
~1,000 11,0000880% (80%)
400
10
10
1,000
400
60
Tiny CUB
100
400
20
25
100
25 (80%)
20
25
Evaluation Protocol. At each stage, after training on Di Di the model is evaluated on disjoint test Des i.e,, inductive setting, which contains both new Chi and old classes Cla-- The accuracy is cal- culated using ground truth yi and models' predictions ÿJ as: ACC = max,pP(c.) 37 ΣM I (yx- = p())), where M- = |Desel and P(C') is the set of all permutations across all classes Ctu U The optimal permutation could be computed once using Hungarian algorithm [52] on all classes, and we report "All", "Old" and "New" accuracies as evaluate metrics.
Implementation Details. Following the convention 17, 10, 18, 31], we use ViT-B/16 [53] pre- trained by DINO [37] as the backbone, and fine-tune only the last transformer block for all experi- ments. The output [CLS] token is chosen feature representation. At Stage-0, models are trained
7

---

Table Forgetting & discovery.
Table 4: 'All' ACC of C-GCD across 10 continual stages.
Methods
C100
Tiny
Data
Methods
9
10
M, 4 Mat
M,
Mat
VanillaOCD
90.82 78.42 75.68
70.35
66.64
64.29
61.05
58.33
57.14
56.23
55.15
C100
MetaGCD
90.82
81.07 76.55
74.26
67.64
64.45
61.58
59.13
60.13
56.91
56.51
VanillaGCD
17.10
33.42
20.20
32.22
Happy
85.62
62.14
61.38
57.81
FRoST
22.82
45.34
21.62
34.96
MetaCIDD
16.56
37.76
19.30
33.68
VanillGGC
84.20
65.15
64.63
60.94
59.46
56.52
55.47
51.65
50.66
49.83
48
Thny
MetaGCD
84.20 68.87 65.48
62.92
60.81
58.21
56.16
54.68
52.58
50.57
48.92
Happy
11.22
51.36
9.75
43.38
Happy
62.75
57.56
54.73
53.02
50.69
Table 5: Ablations on the main components. Average accuracies of 5 stages are reported.
ID Category Discovery
Mitigating Forgetting
CIFARI00
CUB
enmmpy-rrgg
inil
Chap
Lxd
All
Old
New
All
Old
New
(a)
X
X
50.95
58.66
1.96
53.28
60.75
4.70
(b)
X
X
57.67
65.58
7.44
59.26
65.99
14.91
/
X
X
58.26
65.33
12.84
63.11
68.62
27.33
(d)
/
/
X
X
60.51
67.39
16.36
64.53
69.34
32.91
(e)
X
J
57.75
66.32
1.96
57.67
66.05
3.69
(f)
X
66.89
69.98
47.94
66.36
70.94
37.15
g
69.00
71.82
51.36
68.88
71.29
53.13
with 100 epochs. Subsequently, we train models 30 epochs at each continual stage with a batch size of 128 and learning rate of 0.01. We set (A1, ) as and temperature (Tp, Th J as 0.1 while Tr as 0.05. All experiments are run on NVIDIA Geforce RTX 4090 GPUs.
5.2 Comparison with State-of-the-Arts
We compare our methods with (1) Kmeans [43] on pre-trained features, (2) GCD methods: Vanil- laGCD [7], SimGCD [9], SinGCD+LwF [44], and (3) recent continual category discovery works: FROST [12], GM [15] and MetaGCD [18]. Since GM [15] requires storing exemplar samples, we adjust it to sampling features. For fair comparison, all methods use the same objective (Bq. (2)) to pre-train the model at Stage-O. Results are reported in Table 1, Table 3, and Table 4.
Happy outperforms prior methods by large margin. For example in Table 1, on IN100, compared to MetaGCD [18] and GM [15], our approach achieves an improvement of 1 11.90% and 7.50% for 'All' accuracy, respectively. On C100, Happy improves the previous state-of-the-art by 3.45% and 13.60% for old and new classes across 5 stages. Besides, our method produces more balanced accuracy between 'Old' and 'New". These improvements benefit from our consideration of underlying bias in the task of C-GCD and the tailor-made debiased components in Happy.
Happy effectively balances discovering new classes with mitigating forgetting old classes. To decouple and analyze the two confficiigg objectives, we use M, and Ma in [15] to evaluate the overall forgetting of labeled classes and the discovery of new classes respectively. Table shows that VanillaGCD [7] and MetaGCD [18] struggle with category discovery due to the weak supervision of contrastive learning. In addition, FRoST [12] focuses solely new classes, the expense of old class performance. In contrast, our method effectively balances both, achieving improvements of 6~-2% in two metries.
C-GCD with more continual stages. To explore more realistic and challenging scenarios, we conduct C-GCD with 10 continual stages. Results in Table 4 demonstrate that Happy consistently outperforms other counterparts, showcasing Happy a competent long-term novel category discoverer.
5.3 Ablation Study
Here, we conduct extensive ablations each main component (Table 5) and analyze how our method handles the conflicting goals between discovering new classes and mitigating forgetting old ones. Finally, we delve into the mechanism of hardness in framework.
How does Happy achieve remarkable category discovery? In Table (a), we observe that models trained with only Leelt-min are collapsed in 'New' ACC. (b) and (c) incorporate soft entropy regular- ization and the designed initialization, respectively. In addition, (d) combines both of them and brings significant improvements for new classes, e.g., 28.21 % on CUB. From (a) to (d), the initialization
8

---

70
Gap (New Ralio) Belween GT Pred
wio hardness hardnees
weo prior vol prior
80
IF
Accuracy (%6
prg new milo
70
All Accurroo (%)
75
prod now rafo
770
65
64
66
60
60
66
60
C100
Tiny
CUB
C100
Tiny
C100
Tny
Figure 4: Effect hardness.
Figure 5: Analysis: Acc.
Figure 6: Analysis: Ratio.
produces robust and desirable feature location, and Centrpy mitigates prediction bias and ensures necessary learning of new classes. Additionally, mitigating the forgetting of old classes also helps (gd)) as ensures the preservation of general representations for most classes, which in turn benefits the clustering of new classes.
How does Happy mitigate catastrophic forgetting? (f) includes hardness- sampling based on (d), which improves 'Old' ACC by 2.59% on CIFAR 100. However, without Led, the feature space could drift significantly when learning new classes and become misaligned with the learned classifier, which degrades the performance. As a whole, (g) incorporates Lkd to remarkably improve 'Old' by 4.43% and 1.95% on CIFAR 100 and CUB. Similarly, better clustering of new classes also benefits old ones because incorrectly classifying new as old can hinder the learning of old classes. In this sense, the learning of new and old classes is mutually reinforcing.
How does hardness-awareness help C-GCD? To delve into the effective- ness of hardness-aware modeling, we conduct ablations with and without it. Results (average 'All" accuracy across 5 stages) in Figure 4 show that hardness- awareness consistently improves performance across various datasets. We also present sensitivity analysis on temperature Th in Eq. (9). As Table 6 shows,
Table 6: Sensitivity analysis Th.
Th
C100
Tiny
Th 0.1 is proper choice. When 7h is too large, Phardness convergences the uniform distribution, which is similar to the one without hardness modeling. A small Tı also brings suboptimal results. In such cases, Phadness becomes overly sharp, resulting in the sampling of only a very limited number of hard classes, which exacerbates the forgetting of remaining categories.
0.01
66.40
62.36
0.05
68.06
64.95
0.
69.00
65.56
68.01
64.76
10
67.59
63.93
5.4 Further Analysis
Does incorporating class prior into regularization necessarily improve results? Without any prior knowledge about the proportion of new and old class samples, we employ soft entropy regularization in Eq. (4) to prevent bias. A natural question arises: Can the introduction of information about the ratio of new to old class samples at each stage further enhance performance? ? To explore this issue, we directly use the ground truth ratio of old and new samples T Fhe Fhew as prior and modify Eq. (4) nold.eww = -Pola old log phew log Pnew. That is, using cross-entropy supervise the model's predictive probabilities Pola- Pnew. which surprisingly degrades performance shown in Figure 5. The reason lies in the gap between the model's predicted ratio new class samples (pred new ratio) and the prior ratio of new classes Pnew (gt ratio), as revealed in Figure 6, which is caused by the confidence gap between old and classes (Figure 2). This gap ultimately causes the predicted ratio of new samples exceed phe bringing about degraded performance than using Eq. (4) without any prior.
Unknown class number scenarios. Previous experiments assume the number of new classes Kt Kew is known, which often does not hold in reality. At the start of each stage, we need to first estimate the number of new classes before instantiating the classifier. Prior arts [5, 7] query some labeled data when estimating the class number, which is not applicable the purely unsupervised setting of C-GCD. kew an unsupervised manner. Specifically, we compute silhouette score using mean intra-cluster distance and mean nearest-cluster distance
Table 7: Unknown class number results on C100.
Methods All
Old
New
GCD
58.72 62.66 32.92
Instead, we employ off-the-shed silhouetle score [35] to estimate Kt
MetaGCD
63.28
67.65
34.94
Ours
68.80
72.40
45.74

---

Table 8: Effectiveness of proposed reg and hardness-aware modeling for bias mitigation.
CTFARI00
CUB
CIFARIO0
CUB
Ap 4
Ar
Ap 4
Ar
Varo + Aooa,
Varo
Accy. t
w/o Lentropy-reg veg
81,50
63.25
83.20
65.80
w/o hardness
23.04
65.10
21.77 9.28
62.65 68.40
w/ Centropyreg
5.76
10.20
10.25
11.05
w/ hardness
10.33
70.23
(a) Mitigation probability bias.
(b) Mitigation of hardness bias.
and select the number of classes corresponding the highest score value as the estimation. Then we utilize the estimated number for training and evaluation. Average accuracies across 5 stages on CIFARI00 are reported in Table 7. Our method outperforms others when K is not known a-prior.
Happy could effectively mitigate two types of bias in C-GCD, As elaborated Section 3.2, models in C-GCD are susceptible to prediction bias and hardness bias. To validate the effectiveness of the proposed method in bias mitigation, we design metrics to quantitatively measure these biases. Specifically, for prediction bias, we provide two metrics: (1) Ap(L) Pold Pnew denotes the difference in marginal probabilities between old and new classes (see Section 4.2). (2) Ar(L) denotes the proportion of new classes" samples misclassified as old classes. Both Ap and Ar are calculated on the test data after Stage-1. 1. The results Table 8a from two datasets demonstrate that Lentrop-reg effectively reduces prediction bias, with significantly lower marginal probability gap and fewer new class samples misclassifidd as the old. For hardness bias, we also present two metries: (1) ('aro(L) denotes the variance in accuracy of the initial labeled classes Cini (2) Accn (1) denotes the accuracy of the hardest class in CO Both metrics are calculated after 5 stages. Results in Table 8b demonstrate that hardness-aware sampling effectively reduces hardness bias, with lower accuracy variance and higher hardest accuracy. In this regard, the proposed modules competently alleviate both types of bias, which is consistent with our motivation.
Conclusive Remarks
We tackle the pragmatic but underexplored task of Continual Generalized Category Discovery (C- GCD), which involves conflicting goals of continually discovering unlabeled new classes while preventing forgetting old ones. We further identify prediction bias and hardness bias hinder the effective learning of both old and new classes. To overcome these issues, propose debiased framework namely Happy. The clustering-guie guided initialization and soft entropy regularization collectively alleviate prediction bias and ensure the clustering of new classes. On the other hand, by modeling the hardness of learned classes, we propose hardness-aware prototype sampling to dynamically place more attention difficult classes, which significantly prevents the forgetting of old classes. Overall, our method achieves better discovery of new classes with minimal forgetting of old classes, which is validated by extensive experiments across various scenarios.
Limitations and Future Works. Due to the imbalanced labeling conditions between the initial and continual stages in C-GCD, the model's confidence is not calibrated and there is an obvious confidence gap between old and new classes, in these cases, incorporating prior information even degrades performance (Section 5.4). Future work should incorporate confidence calibration [38] into C-GCD to further mitigate potential biases. Another promising direction is to devise competent class number estimation methods for C-GCD, because in the unsupervised setting, class number estimation becomes significantly challenging. Additionally, this paper primarily discusses classification tasks, while future works could extend the C-GCD learning paradigm to object detection [54], segmentation [55] and multi-mode moodal learning [56, 57, 58].
Acknowledgments and Disclosure of Funding
This work has been supported by the National Science and Technology Major Project (2022)D01 16500), National Natural Science Foundation of China (U20A20223, 62222609, 62076236), CAS Project for Young Scientists in Basic Research (YSBR-083), Key Research Program of Frontier Sciences of CAS (ZDBS-LY-7004), and the InnoHK program.
10

---

References
[1] Fei Zhu, Shijie Ma, Zhen Cheng, Xu- Yao Zhang, Zhaoxiang Zhang, and Cheng-Lin Liu. Open-world machine learning: review and new outlooks. arXiv preprint arXiv:2403.01153 2024.
[2] Chuanxing Geng, Sheng-jun jun Huang, and Songcan Chen. Recent advances in open set recognition: A survey. IEEE transactions pattern analysis and machine intelligence, 43(10))3614-3633 2020.
[3] Zhi-Hua Zhou. Open-environment machine learning. National Science Review, 98)) 223 2022.
[4] Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8401 1840, 2019.
[5] Kai Han, Sylvestre- Alvise Rebufli, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Autonovel: Automatically discovering and learning novel visual categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10)=6777-678) 2021.
[6] Enrico Fini, Enver Sangineto, Stéphane Lathuilière, Zhun Zhong, Moin Nabi, and Elisa Ricci. unified objective for novel class discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2021.
[7] Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7492-7501, 2022.
[8] Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Nascer, Guangyi Chen, and Fahad Shahbaz Khan. Promptcal: Contrastive affinity lcarning via auxiliary prompts for generalized novel category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3479-3488, 2023.
[9] Xin Wen, Bingchen Zhao, and Xiaojuan Qi. Parametric classification for generalized category discovery: A bascline study. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 11690-6600,,01 2023.
[10] Bingchen Zhao, Xin Wen, and Kai Han. Learning semi-supervised gaussian mixture models for generalized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16623-16633, 2023.
[11] Sinno Jialin Pan and Qiang Yang. A survey transfer learning. IEEE Transactions knowledge and data engineering, 22 22(10))1345-133 1359, 2009.
[12] Subhankar Roy, Mingxuan Liu, Zhun Zhong, Nicu Sebe, and Elisa Ricci. Class-incrente incremencrl novel class discovery. In European Conference Computer Vision, pages 317-335 Springer, 2022.
[13] KJ Joseph. Sujoy Paul, Gaurav Aggarwal, Soma Biswas. Piyush Rai, Kai Han, and Vineeth Balasubra- manian. Novel class discovery without forgetting. In European Conference on Computer Vision, pages 570-586. Springer, 2022.
[14] Mingxuan Liu, Suhhankar Roy, Zhun Zhong, Nicu Sehe, and Elisa Ricci. Large-scale pre-trained models surprisingly strong in incremental novel class discovery. arXiv preprint arXiv:2303. /5975, 2023.
[15] Xinwei Zhang, Jianwen Jiang, Yutong Feng. Zhi-Fan Wu, Xibin Zhao, Hai Wan, Mingqian Tang, Rong Jin, and Yue Gao. Grow and merge: A unified framework for continuous categories discovery, Advances in Neural Information Processing Systems, 35:27455-27468 2022.
[16] Hyungmin Kim, Sungho Suh, Daehwan Kim, Daun Jeong, Hansang Cho, and Junmo Kim. Proxy unchor-based unsupervised learning for continuous generalized category discovery. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2023.
[17] Bingchen Zhao and Oisin Mac Aodha. Incremental generalized category discovery. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 19137-19147, 2023.
[18] Yanan Wu, Zhixiang Chi, Yang Wang, and Songhe Feng. Metaged: Learning to continually learn in generalized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1655-1665, 2023.
[19] Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Ann Toh, and Zhiping Lin. Acil: Analytic class inseemnncemttall learning with absolute memorization and privacy protection. Advances in Neural Information Processing Systems, 35:11602- 11614, 2022.

---

[20] Fei Zhu, Zhen Cheng, Xu- Yao Zhang, and Cheng-lin lin Liu. Class-incremental learning via dual augmentation. Advances in Neural Information Processing Systems, 143188 2021.
[21] Peiyan Gu, Chuyu Zhang, Ruijie Xu, and Xuming He. Class-relation knowledge distillation for novel class discovery. In 2023 IEEF/CVF International Conference on Computer Vision (ICCV), pages 16428-16437. IEFE, 2023.
[22] Bingchen Zhao and Kai Han. Novel visual category discovery with dual ranking statisties and mutual knowledge distillation. Advances in Neural Information Processing Systems, 34:22982 222994, 2021.
[23] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
[24] Wenbin Li, Zhichen Fan, Jing Huo, and Yang Gao. Modeling inter-class and intra-class constraints in novel class discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14499-458,, 2023.
[25] Zhun Zhong. Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang. and Nicu Sebe. Openmix: Reviving known knowledge for discovering novel visual categories in an open world. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9462-9470 2021.
[26] Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, and Nicu Sebe. Neighborhood contrastive learning for novel class discovery. In Proceedings of the IEEEJCVF conference on computer vision and pattern recognition, pages 10867-10875, 2021.
[27] Kaidi Cao, Maria Brbic, and Jure Leskovec. Open-world semi omi-suuprvised learning. In International Conference on Learning Representations, 2022.
[28] Nan Pu, Zhun Zhong, and Nicu Sebe. Dynamic conceptional contrastive learning for generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7579-7588, 2023.
[29] Florent Chiaroni, Jose Dolz, Ziko Imtiaz Masud, Amar Mitiche, and Ismail Ben Ayed. Parametric infor- mation maximization for generalized category discovery. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 1729-1739, 2023.
[30] Sagar Vaze, Andrea Vedaldi, and Andrew Zisserman. No representation rules them all in category discovery. Advances in Neural Information Processing Systems, 36, 2024.
[31] Shijie Ma, Fei Zhu, Zhun Zhong, Xu-Yao Zhang, and Cheng-Lin Liu. Active generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16890- 16900, 2024.
[32] Nan Pu, Wenjing Li, Xingyuan Ji, Yalan Qin, Nicu Sche, and Zhun Zhong, Federated generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28741-28750, 2024.
[33] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta- meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1 1135. PMIR, 2017.
[34] Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1310-1321, 2015.
[35] Peter Roussceuw, Silhouettes: a graphical aid to the interprctation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53-65 65, 1987.
[36] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[37] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660, 2021.
[38] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017.
[39] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of -nt-o--diitributtin examples in neural networks. In International Conference on Learning Representations, 2017.
[40] Shijie Ma, Fei Zhu, Zhen Cheng, and Xu- Yao Zhang. Towards trustworthy dataset distillation. Pattern Recognition, 157: 10875,075500000 2025.
12

---

[41] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot. Ce Liu, and Dilip Krishnan. Supervised contrastive learning, Advances neural information processing systems, 13:11861-18677 2020.
[42] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[43] James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281-297. Oakland, CA, USA, 1967.
[44] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12))2935 29442)9993-2297, 2017.
[45] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings the national academy of sciences, 14((1))3521-3356, 2017.
[46] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366 33355,333,,,, 2021.
[47] Fei Zhu, Xu-Yau Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu, Prototype augmentation and self- supervision for incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5871-5880, 2021.
[48] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint 1rXiv:1503.025:: 2015.
[49] Jia Deng, Wci Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hicrarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[50] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
[51] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 201
[52] Harold W Kuhn. The hungarian method for assignment problem. Naval research logistics quarterly. 2(1-2):8 97, 1955.
[53] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[54] Jiyang Zheng, Weihao Li, Jie Hong, Lars Petersson, and Nick Barnes. Towards open-ser set object detection and discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
pages
[55] Yuyang Zhao, Zhun Zhong, Nicu Sebe, and Gim Hee Lee. Novel class discovery in semantic segmentation. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4340- 4349, 2022.
[56] Yuxin Guo, Shijie Ma. Hu Su, Zhiqing Wang, Yuhao Zhao, Wei Zou, Siyang Sun, and Yun Zheng. Dual mean-teacher: An unbiased semi eunissupervssed framework for audio visual source localization. Advances in Newral Information Processing Systems, 36:48639-48666 48661, 2023.
[57] Yuxin Guo, Shijie Ma, Yuhao Zhao, Hu Su, and Wei Zou. Cross pseudo-labeling for semi-supervised audio-visual source localization. In ICASSP 2024-2024 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 8356 8360. IEEE, 2024.
[58] Haokun Lin, Haoli Bai, Zhili Liu, Hou, Muyi Sun, Lingi Song, Ying Wei, and Zhenan Sun. Mope- clip: Structured pruning for efficient lision-language models with module wise pruning error metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27370- 27380, 2024.
13

---

[59] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference Learning Representations, 2019.
[60] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization, In Proceedings of the IEEE international conference on computer vision workshops, pages 554-561. 2013.
[61] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- Fine-grained visual classification of aireraft, arXiv preprint arXiv:1306.513: 2013.
[62] Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang. Linzhan Mou, Lingi Song. Zhenan Sun, and Ying Wei. Duquant: Distributing outliers dual transformation makes stronger quantized Ilms. arXiv preprint wxxi:2406.0172.. 2024.
[63] Yuxin Guo, Siyang Sun, Shuailei Ma. Kecheng Zheng, Xiaoyi Bao, Shijie Ma, Wei Zou, Yun Zheng. Crossmae: Cross- Cross-modality masked autoencoders for region- audio-visual pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26721-26733 2024.
14

---

A More Discussions about the Task of C-GCD
In this section, we first provide a detailed explanation of the task of Continual Generalized Category Discovery (C-GCD) and a comparison with caass-incremettll learning Then we illustrate the practicality of C-GCD studied in this paper through some examples.
A.1 Comparison with Class-ineremental Learning
The core differences between C-GCD and caas-innrremetal Learning [20, 44] (CIL) lie in that training data is fully unlabeled at each stage of C-CGD, by contrast, conventional CIL adopts fully-supervised setting. On the other hand, at each stage of C-GCD, the unlabeled training data Dirain contains samples from previously seen classes, which makes the task more challenging because models need to implicitly or explicitly split the samples from old and new classes and then discover novel categories. While in rehearsal-free CIL, at each stage, the labeled training dataset typically does not contain samples of previous classes, otherwise becomes the replay-based sitting and will simplify the problem, because the training data is fully labeled.
A.2 Realistic Considerations of C-GCD
As mentioned in the main manuscript, we study a more pragmatic setting of C-GCD, whose specific manifestations of realistic considerations are listed as follows:
More continual stages with more novel categories to be discovered. Prior works [15, 16, 18] mainly implement C-GCD with 3 stages given nearly 70% of all the classes serving as labeled classes. This simple setting does not reflect real-world scenarios. Humans are lifelong learners over the course of their entire lives, and our setting closely aligns with this situation. Specifically, the default setting in this paper has 5 continual stages with 50% of all the classes serving as novel classes.
Rehearsal- -free setting without storing previous samples. Several works [15, 17] in C-GCD require the storage of previous samples to construct a non parametric classifier or mitigate catastrophic forgetting. This store-and-replay manner could cause privacy and storage issues, especially in cases with very long learning periods. While we study the rehearsal-free C-CGD.
The ratio of new class samples is unknown. Some works study C-GCD by assuming that the proportion of new class samples per stage is known, which facilitates the design of novelty detection, owing to the fact that novelty detection [15, 39] typically relies on a threshold to determine whether a sample is from novel classes. In our setting, we lift this restrictive assumption and our framework Happy does not rely on the ratio. Instead, our method does not explicitly perform novelty detection, but instead implicitly learns with soft entropy regularization and self-distillation.
At each stage, the number of samples of each old class is significantly fewer than the number of samples in each new one. If at each stage, the number of class-wise samples of old and new classes roughly the same or both have plenty of samples, then C-GCD degenerates the static setting of GCD where the catastrophic forgetting is inherently avoided because there are plenty of samples for each class, which bring about desirable outcomes even using baseline methods. This also contradicts the reality. Imagine a scenario where a student is growing up and entering different stages of learning. For example, he is currently in college where he needs to self-learn many new subjects like calculus and linear algebra. However, he occasionally encounters some old knowledge from his high school days, such as trigonometry and plane geometry. In this case, new knowledge is mixed with old knowledge, but the quantity of old knowledge is quite small.
B More Implementation Details
Fair training in Stage-0. We train all the methods using similar objectives at Stage-O, specifically. for methods with parametric classifiers [9, 12, 15], we employ Lint in Eq. (2), while for methods with contrastive learning and non-parametric classifiers [7, 18], we employ supervised and self-supervised contrastive learning on the labeled data, i.e., the last two terms in Eq, (2). The results of different methods at Stage-O are similar, as shown in Table 1, ensuring fair comparisons of subsequent continual learning stages.
15

---

Model Details. Following the convention of the literature [7, 10, 18], we use VIT-B/16 [53] pre-trained with DINO [37] as the encoder, and fine-tune only the last transformer block for all experiments. The output [CLS] token is chosen as feature representation. For the parametric classifier, we use & weight normed prototypical classifier [9, 30] without the bias term. The dimensionality of feature space and projection space for contrastive learning is 768 and 65,536, respectively. Note that all the feature vectors in the 768-dimensional feature space are l2 f2-normalized, i.e., hyperspherical feature space, including the feature representation 2; of each sample E, the head of each class in the classifier Oi, the KMeans [43] cluster centroids c; in Eq. (3), the bliss-wise prototypes μe in Eq. (8) and the sampled features &e in Eq. (10).
Training Details. We train the models in Stage-O for 100 epochs with learning rate of 0.1, and 30 epochs with a learning rate of 0.01 for each of the continual stages. We use a cosine annealed schedule for the learning rate.
Hyper-parameters and implementation details of Happy. For the weights of loss terms, empirically set Àu = 0.35 and A = X2 = 1, and detailed hyper- paremerrr analysis is elaborate in Section E.5. For the temperature, probability p; and the 7; in the sharp soft qt- We set 7h in hardness distribution Phardness as 1. As for the temperature in the contrastive learning term, follow prior arts [7, 9] and set T. as 0.07 and for supervised and self- sulf--uper contrastive learning, respectively. When we compute Coldinn and Exniropy in Eq. (5), the distribution within old p( e E Cola and new classes p()), c E C should be firstly normalized whose summation across the class indices equals to 1.
set the main temperature Tр 0.1 in model predictive
C Algorithm of the Proposed Method Happy
In this section, we give detailed algorithm of Happy in Algorithm 1, including both (1) Initial supervised learning (Stage-O) and (2) Continual unsupervised discovery (Stage-1 T).
Algorithm 1 Training Pipeline of Happy
Input: Initial laheled dataset Do Piainn {(ær), ( of K classes Ch and training epochs Fg for Stage-O.
Input:
Number of continual stages T.
Input:
Continual stages datasct (Dimin )7=1 of classes C³ = Caa U Ciew and training epochs E for cach stage. Number of new classes Kin. at each stage, which could be ground truth or estimated.
Input:
Input: The model h g fo(() where f)(.) is encoder and g is parametric classifier. Stage-0
I:
2: for epoch e = + Eg do
3: Train the model he on Di using the loss function Cuinal in Eq. (2).
4: end for
5: > Compute class-wise prototypes the (c 1, KD) in Eq. (8) using ground truth labels
6: > Compute class-shared radius nd バū Legey Tr(E.)/d
7: > Model hardness distribution Phanines on all the prototypes using Eq. (9)
8:
9: #
Continual Stages
10: for epoch t T do Clustering-guided initialization current new heads ( 1 for epoch e = 1 + F. do
Ki.. 1j=1 using Eq. (3)
12:
13:
> T'aain the model h' on Dinnn using the overall loss function Lrmmy in Eq. (12)
14:
end for
15:
Compute class-wist prototypes p. (c = Kt-1 + 1.... KC) in Eq. (8) using model predictions
16:
b Append new prototypes to the existing set
17:
b Model hardness distribution Phardness on all the prototypes using Eq. (9)
18:
end for
Output: The trained model hT 9% ff(()) that could perform classification on all seen classes.
16

---

oid Now
Did
2.5
Did
New
New
Dankky
Dersky
1.0
2
8.0
6.3
Confidence: MSP
Confidence Max
Confidence: Margin
Confidence: <Entropy
(a) Max Softmax Prob.
(b) Max Logit.
(c) Margin.
(d) Negative Entropy.
Figure 7: Confidence gap of various metrics between old and new classes of baseline models.
D Metrics of C-GCD
C-GCD essentially a clustering problem, specifically for unlabeled new classes. Following [7, 5,9, 10, 18], the accuracy is calculated using ground truth y; and models' predictions
M
1
ACC
max PEP(CE)
$ 1 = p(o:)),
(13)
M
here, .M = [D is the number of samples in the test dataset and P(C') represents the set of all permutations across all classes Cala U.eww The optimal permutation could be computed once using Hungarian algorithm [52], and subsequently 'All', 'Old' and 'New' are computed on corresponding indices of classes. C-GCD utilizes inductive evaluation, i.e., models are evaluated on a disjoint test dataset containing all of the seen classes.
To decouple and analyze the objectives of novel class discovery and preventing forgetting, GM [15] designed new metrics, ie.,, the maximum forgetting metric M, and the final discovery metric Ma- They are defined as follows:
Mj max (ACC ACC
(14)
Ma = ACC new
(15)
However, old classes at different stages are changing and expanding. in the above definitions, M does not truly quantify the forgetting of the initial classes. On the other hand, Ma only measures the category discovery performance at the last stage, which overlooks measuring the accuracy of new categories throughout the process. As result, re-define these two metrics as follows:
Mj !
Aax ACC
(16)
T
Ma
Σ ACC
(17)
In our metrics, My quantify the forgetting of fixed classes set Co which is more reasonable, and M2 measures category discovery of each new classes, which could more comprehensively reflect the ability to cluster new classes. In the main manuscript, we use the re-defined M, and Ma to evaluate models in Table 3.
E
More Experimental Results
E.1
Confidence Gap with More Confidence Metrics
Here, similar the preliminary experiments in Figure train baseline models and provide more metrics, iee., maximum softmax probability, maximum logit value, margin and negative entropy, of confidence distribution new and old classes, illustrated in Figure 7. The results consistently reveal the severe confidence gap between old and new classes, which is the underlying cause of prediction bias.
E.2 Performance of C-CGCD with Longer Stages
In the main paper, we conduct experiments with 5 continual stages by default. To evaluate models in more realistic scenarios with longer continual learning stages, provide more detailed results of
17

---

Table 9: Performance of 10 continual stages on CIFAR100.
Methods
Stage
0
2
3
5
6
7
8
10
90.82
78.42
75.68
70.35
66.64
64.29
61.05
58.33
57.14
56.23
55.15
VanillaGCI
Old
82.86
80.65
73.52
69.09
68.
63.32
60.42
59.20
57.93
56.75
New
34.00
33.00
32.40
34.80
26.00
27.00
24.80
22.20
25.60
24.80
All
90.82
81.07
76.55
74.26
67.64
64.45
61.58
59.13
60.13
56.91
56.51
MetaGCD
Old
84.16
80.35
77.32
70.09
67.16
63.88
61.20
61.99
58.76
58.01
New
50.20
34.80
37.60
35.80
26.60
27.00
26.00
28.60
23.60
28.00
All
90.36
85.62
81.88
79.82
74.01
71.81
68.46
64.05
62.14
61.38
57.81
Happy (Ours)
Old
85.46
81.67
79.53
76.60
73.06
71.12
66.53
63.58
61.74
59.36
New
87.20 84.20 83.20 40.40 54.40 28.60
24.40
37.80
54.80
28.40
Table 10: Performance of 10 continual stages on Imngeee.
Methods
Stage
2
3
4
5
8
10
All
84.20
65.15
64.63
60.94
59.46
56.52
55.47
51.65
50.66
49.83
48.56
VanillaGCD
Old
68.20
67.36
63.28
61.51
58.66
57.07
53.39
52.07
51.32
49.63
New
34.60
34.60
32.80
32.80
26.60
31.60
23.80
26.60
23.00
28.20
All
84.20
68.87
65.48
62.92
60.81
58.21
56.16
54.68
52.58
50.57
48,92
MetaGCD
Old
72.00
68.24
65.13
63.02
60.23
57.96
56.46
54.21
52.02
49.85
New
37.60
35,20
36.80
32.20
30.00
29.20
26.20
24.80
24.40
31.20
All
85.86
80.75
76.92
73.34
69.77
66.33
62.75
57.56
54.73
53.02
50.69
Happy (Ours)
Old
84.04
79.76
75.02
72.12
67.44
64.37
59.44
55.29
53.92
50.95
New
47.80
45.60
53.20
39.20
50.80
38.40
27.60
45.20
36.80
45.80
VanillaGCD
MetaGCD
Happy (Ours)
60
Accuracy (%)
50
40
30
gauss
shot
impulse
defocus
glass
motion
zoom
snow
frost
fog
brightness
contrast
elastic
pixelate
jpeg
Figure All accuracy on 15 unseen shifted distributions of CIFAR100-C with severity=2.
10-stage C-GCD CIFARI00 and TinylmageNet, as shown in Table 9 and Table 10. Our method still consistently outperforms others over the whole course of continual stages.
E.3 Performance under Unseen Distributions
We conduct experiments the distribution-shift dataset. Specifically, we train models the original CIFAR100 dataset, and test the model on all 100 classes after 5 stages training. Models are directly evaluated on the unseen distributions of CIFARIO0-4 [59], e.g., gaussian blur, snow and frost, as shown in Figure 8. Our method consistently outperforms others across several unseen distributions, showcasing its strong robustness and generalization ability.
E.4 Performance under Fine-grained Datasets
Furthermore, we have also conducted experiments on two more fine-grained datasets, i.e., Stanford Cars [60] and FGVC Aircraft [61]. We adopt the default setting C-GCD described Section 5.1, i.e., 5 continual stages and 50% of classes serving as Ch initially labeled classes. Average accuracies
18

---

Table 11: Performance of C-GCD on two more fine-grained datasets.
Stanford Cars
FGVC Aircraft
Methods
All
Old
New
All
Old
New
VanillaGCD
47.00
47.73
42.61
42.95
44.35
33.38
MetaGCD
54.67
55.28
50.95
47.16
48.61
38.23
Happy (Ours)
62.79
63.68
57.34
53.10
53.81
48.71
over five continual stages are reported in Table 11. Happy also achieves remarkable performance on these fine-grained datasets.
E.5 eyppe-pparameer Sensitivity Analysis
We fix the weights of Leert-rain and Chap as 1, considering they are the main objectives for new and old classes. As a result, our method mainly contains three loss weights λ1, λ2, Às for Lentrpy-reg- Ckd and Cim: respectively. Here, we give a sensitivity analysis on CIFAR 100 and report average 'All' Acc in Table 12. As shown above, the model is relatively insensitive to λs. whereas X1 and X have a more significant impact. Overall, the optimal values for each hyper-parament are close to In our experiments, we simply set all weights to 1, which shows remarkable results across all datasets. Thus, our method does not require complex tuning of parameters and exhibits strong generalization capabilities and practicability.
Table 12: Sensitivity analysis of hyper-parameters A1 X2 and A3-
A
0
0.5
1.0
3.0
5.0
Aa
0
0.5
1.0
3.0
5.0
A
0
0.5
0.7
1.0
3.0
Acc.
60.60
69.04
69.00
63.98
59.23
Acc.
65.31
66.98
69.00
69.30
68.90
Acc.
68.74 68.94
69.16 69.00 68.92
(a) Sensitivity of A1-
(b) Sensitivity of A2.
(c) Sensitivity of Xa-
F Potential Societal Impacts
This paper focuses on Continual Generalized Category Discovery (C-GCD) and primarily addresses the classification issues. From a more intrinsic perspective, represents a paradigm of transferring existing knowledge continuously generalize and learn new information. Therefore, it can be applied to a wide range of tasks and scenarios, such as reasoning abilities in LLMs [62] and multi- modal models [58, 63], continuous pre-training and instruction tuning, and large generative models' generalization abilities to novel concepts. In the fields of biology and health sciences, the principle of C-GCD assist the discovery of new species and drugs, which will help human beings understand the ecosystem better and facilitate timely diagnosis and treatment of new diseases.
At its core, C-GCD involves leveraging and transferring knowledge learned from old categories to learn new information better, embodying the principle of applying learned concepts to new situations. From this perspective, old knowledge significantly determines the model's ability to discover new knowledge. biases or unfairness are learned from old knowledge, these issues can also manifest in the newly discovered knowledge. As result, future works should pay attention to the bias and fairness issues, specifically when learning old classes, and the scrutiny of newly learned knowledge.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, we present the studied task, and motivation, together with the proposed method and contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations including the confidence calibration issues and the scope regarding the classification task in the Conclusion, i.e., Section 6.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer:[ Yes]
Justification: We have presented implementation details in Section 5.1 and also the algorithm pipeline in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
21

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our code is publicly available https: /tt. Happy-CGOL CGCD.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
cwaeinisssoooooiiiii onPolicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wttpp.ccpppuiiiieeeeeeeepppppppp wctpublic//aeccccccccccccccc /gwpi/gg///////eeeeee /Ca///eoeubbiiicc sateeuubbisiion.. ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We present the dataset splits Table 2, and give details about the experimental details in Section 5.1 and the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We report all the experimental results as the average over 5 runs, but we do not report the error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We introduce the experimental computational resources in Section 5.1, all our experiments run on NVIDIA GeForce RTX 4090 GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that NeurIPS Code of Ethics https /tterr///ee.....eeeeeeeee cwtpppcccc/eeeeeccceece wtwteetttiiiiieeeeiieeeeeeeeeeee
didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer:
[Yes]
Justification:
We have carefully read the NeurlPS Code of Ethics and make sure preserve anonymity.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., s if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include the discussions in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
23

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all the comparative methods and necessary adjustments in Section 5.2.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We are currently organizing the codes and will release them as soon as possible.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "p9xCNbeHCZ", "reviewer": "Reviewer_TDQJ", "summary": "The article presents a method for Continual Generalized category discovery. A de-biasing learning framework for the Category Discovery (C-GCD) task is designed to address the challenge of continuously discovering new concepts in an ever-changing environment while maintaining recognition of known categories. Traditional C-GCD studies have some limitations, such as storage and privacy issues caused by storing samples of past classes, and only considering limited incremental stages or assuming a proportion of known samples, which are not in line with practical application scenarios. Therefore, the study focuses on a more realistic C-GCD setup that includes more learning phases, more new categories, and after each phase, data from previous phases is not accessible.", "questions": "1. In the experimental part, how do you make sure that in the unsupervised increment phase, the model not only finds new categories, but also accurately distinguishes them without confusing them with the old ones? Are there specific metrics or experimental Settings to measure this ability to differentiate?\n2. How does the soft entropy regularization realize the reasonable distribution of the probability of the new class? Can you explain this process in detail and how it is combined with cluster-guided initialization to improve the clustering performance of new categories?\n3. In the hardness-aware prototype sampling strategy, how do you define and quantify the \"hardness\" of a class, and how do you sample effectively against this hardness to mitigate forgetting? Are there concrete examples of how this strategy helps the model remember old categories that are difficult to classify?\n4. The paper mentions that in preliminary experiments it was found that the model tends to misclassify new categories into old ones, and that the features of the old categories are disturbed when learning the new ones. In addition to the proposed solutions, have other technologies, such as meta-learning or memory enhancement networks, been considered to further improve these problems?", "limitations": "The authors discussed the limitations of the work in the paper, including the confidence calibration problem and the scope of the study mainly focused on the classification task.\nConfidence Calibration problem: In the Continuous Generalized Category Discovery (C-GCD) task, the confidence of the model is not calibrated due to an imbalance in the labeling conditions between the initial phase and the sustained phase. This leads to a clear gap between the old category and the new category, and even degrades performance when incorporating prior information.\nThe scope of application is limited to classification tasks: Although this paper mainly discusses the C-GCD learning paradigm under classification tasks, the application of this method has not been extended to other fields, such as object detection and image segmentation.\nThe authors note that future work should consider how confidence calibration techniques can be incorporated into C-GCD to further reduce potential bias. At the same time, they encourage the extension of the C-GCD learning framework to more types of visual tasks in order to expand its applicability and impact. These discussions reflect a clear understanding of the limitations of the current findings and suggest possible directions for future research.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: In this paper, the authors propose a novel de-biased-learning framework, \"Happy, \"specifically for the continuous Generalized Category Discovery (C-GCD) task, which is a relatively underexplored research area. It has designed a unique set of methods to handle the challenge of incrementally discovering new categories in unlabeled data while preserving the ability to recognize old categories, which traditional machine learning and deep learning approaches have overlooked. In particular, the proposed cluster-guided initialization, soft entropy regularization and hardness-aware prototype sampling strategies are innovative solutions to the unique problems of C-GCD.\nQuality: The quality of the paper is reflected in its detailed theoretical analysis and experimental verification. The authors not only elaborate the design principle and motivation behind the method, but also conduct extensive experiments on multiple datasets to prove the effectiveness of the \"Happy\" framework achieving significant performance improvement, showing its strong generalization ability and practicality. In addition, the paper also reflects on the limitations and assumptions of the method, indicating that the author has deeply considered the comprehensiveness and rigor of the study.\nClarity: The structure of the paper is clear and the logic is coherent. From the introduction, which clearly explains the research background and motivation, to the method, which analyzes the components of the framework and their working principles in detail, to the presentation and analysis of the experimental results, every step is well organized. In addition, the authors also provide detailed implementation details and algorithm flow in the appendix, which increases the readability and reproducibility of the paper.", "weaknesses": "1. Although the authors acknowledge and briefly discuss the social impact of technology, the paper does not detail specific negative impacts that a \"Happy\" framework could bring, such as potential bias transmission, fairness issues, privacy violations, or security risks. For an algorithm intended for application in the open world, the lack of a comprehensive social impact analysis may limit its acceptance and ethical application in practice.\n2. It is clearly stated that this study does not include theoretical results, which means that a complete hypothesis set and theoretical proof are not provided to support the validity of the proposed method. While empirical research has shown the effectiveness of \"Happy, \"the lack of a solid theoretical foundation may diminish its persuasive power in academia.\n3. The authors acknowledge that there are some assumptions and limitations to the study, such as overfitting the model, performance in noisy environments, and testing on specific datasets. These factors may limit the method's general applicability and ability to generalize, especially in different data distributions or more complex real-world environments.\n4. It is pointed out that even after the introduction of the debias mechanism, the recognition accuracy of the new class still fluctuates at different incremental stages, which indicates that the model may have stability problems when processing data of different class difficulty. This volatility can affect model reliability and user trust in real-world deployments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful advice and valuable questions, we will respond to your concerns point by point.\n\n> W1:  Lack of a comprehensive social impact analysis.\n\n* We summarize the negative impacts: (1) **Bias/error accumulation**. Given the stringent unlabeled conditions of C-GCD, the initial learning bias may continuously accumulate, leading to a greater bias in later stages. (2) **Spurious correlations**. Category discovery relies on knowledge learned from labeled classes. If incorrect correlations are learned, e.g., over-reliance on the background, these may interfere with the learning of new ones.\n\n* We will discuss them in detail in the final version.\n\n>  W2: Lack of a theoretical foundation.\n\n* Happy is essentially self-learning on unlabeled data, and the theoretical foundation is conditional entropy maximization.\n* Specifically, the objective can be expressed as $\\max I(Y, Z) = H(Y) - H(Y|Z)$, where $Y$ represents the labels and $Z$ the features, with $I$ and $H$ denoting mutual information and entropy, respectively. The first term is equivalent to the proposed marginal entropy regularization $L_{entropy-reg}$, while the second one $H(Y∣Z)$, can be computed by the posterior probability $p(y|z)$, which we aim to minimize through self-distillation $L_{self-train}$.\n\n* We will include this theoretical analysis.\n\n> W3: There are assumptions that may limit the general applicability especially in different distributions.\n\n* Our method achieves sota not only on four datasets (table 2) but also on the distribution-shift CIFAR100-C. (**See General Common Concern 2 for more details**). The results demonstrate that our method has a stronger generalization ability.\n\n> W4: The accuracy of the new class still fluctuates at different incremental stages with the debias methods.\n\n* The fluctuation in new categories is an inherent issue in C-GCD. However, our method exhibits significantly less variability compared to others, e.g., on CIFAR100, the maximum new accuracy difference across stages is $\\Delta_{new}=6.3$ (56.1-49.8), which is less than the 17.3 (48.9-31.6) of MetaGCD. Similar patterns can be observed in other datasets as well.\n\n> Q1: How to make sure the model not only finds new categories but also avoids confusing them with the old ones? Are there specific metrics?\n\n* We follow SimGCD and employ a unified classification head for both old and new classes in a joint prediction space. So our method could classify both new and old categories together, which not only implicitly differentiates between new and old classes but also discovers new ones.\n\n* **Specific metrics**. We calculate the proportion of new class samples misclassified as old classes (new$\\to$old) and old classes misclassified as new (old$\\to$new) after stage-1 training. Results are shown below:\n\n  | misclassify ratio | new$\\to$old (C100) | old$\\to$new (C100) | new$\\to$old (CUB) | old$\\to$new (CUB) |\n  | :---------------: | :----------------: | :----------------: | :---------------: | :---------------: |\n  |      SimGCD       |       85.75        |        6.78        |       84.18       |       6.65        |\n  |      MetaGCD      |       30.65        |        6.64        |       32.36       |       6.50        |\n  |       Happy       |     **10.20**      |      **4.35**      |     **11.05**     |     **4.95**      |\n\n  Results show Happy can more effectively distinguish between new and old classes with a lower misclassify ratio, and achieves a higher accuracy for new classes (e.g., 51.3>31.6 at stage-5 in table 2).\n\n> Q2: Explain soft entropy regularization and cluster-guided initialization.\n\n* Soft entropy regularization. See Sec 3.2 line 128, as the model tends to bias the predictions to old classes, so we implement an explicit constraint to ensure necessary predictive probabilities for new classes to facilitate learning. Considering that at each stage the prior ratio of old and new classes is inaccessible, we employ a soft regularization, assuming a general balance between the probabilities for new and old classes ($p_{old} = p_{new} = 0.5$), which is achieved by the marginal entropy maximization.\n\n* Cluster-guided initialization ensures a good **initialization** for new class heads, while entropy regularization improves **training** of new classes with less prediction bias.\n\n> Q3: How to define \"hardness\" and how to sample against this hardness? Concrete examples of how it helps remember difficult old classes?\n\n* As in Sec 3.4 Eq (9), we define class-wise **hardness** $h_c$ as the average cosine similarity between the class center $\\mu_c$ and other class center $\\mu_j$, where $\\mu_j$ is computed in Eq (8). Intuitively, the greater the similarity to other classes, the more likely it is that its feature space will be confused with others, thereby the class is more difficult.\n\n* To sample against hardness, considering more difficult categories tend to be forgotten more readily, we sample harder classes more frequently. So we treat $h_c$ as logits and model hardness distribution with softmax function in Eq (9).\n\n* **Concrete samples**. After learning 5 stages, we evaluate the model on initial labeled classes $C_{init}^0$ and the accuracy of the hardest class (lowest acc) is reported:\n\n  | hardest Acc |   C100    |    CUB    |\n  | :---------: | :-------: | :-------: |\n  |   SimGCD    |   62.85   |   59.68   |\n  |   MetaGCD   |   65.31   |   61.26   |\n  |    Happy    | **70.23** | **68.40** |\n\n  Results show that Happy could remember difficult classes better with higher hard class accuracy.\n\n> Q4: Have other technologies been considered to further improve the task?\n\n* MetaGCD actually utilizes meta-learning, but the results are weaker than ours (see table 2), because MetaGCD only simulates the C-GCD process without any debiasing mechanism.\n* In essence, meta-learning and memory enhancement can be integrated with our debiasing mechanism to further enhance the results. We leave this aspect to future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MJ2Nxsn2cJ", "reviewer": "Reviewer_TLvc", "summary": "The paper points out two-bias issues in CGCD: prediction bias in probability space and hardness bias in feature space. To tackle those two issues, they propose cluster-guided initialization and soft entropy regularization to mitigate prediction bias, and they propose hardness-aware prototype sampling to mitigate hardness bias and forgetting.", "questions": "See weaknesses, especially 2 and 4. I will raise my score if the 2 and 4 are well resolved.", "limitations": "None", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* The paper is easy to follow. I really appreciate this writing idea: explain the problem and solve the problem.\n* The experiment show great improvement over existing method.", "weaknesses": "1. Overclaim. The paper argues that they extend CGCD to realistic scenarios. But from the experiment setup, I am not convinced that the scenarios they consider are more realistic than existing work [16,18]. The stage and new classes are still limited.\n2. Too many hyper-parameters and loss terms. There are too many hyper-parameters to adjust different loss weights, hindering its generalization. In addition, there are too many loss terms, hindering the understanding of the method. For example, what is the effect of the proposed Eq 4? \n3. In Table 5, why KD improve New class by a large margin? What does the new class mean? The total novel class or the new class in the last task?  \n4. As the paper points out two issues: prediction bias and hardness bias, and proposes methods to mitigate the two issues, they should provide evidence to show that two issues are mitigated.\n\n\nMinor:\n1. The rightmost one in Figure 2 is (d) instead of (b).\n2. $L_{hap}$ in eq 10 and 11 should be consistent.\n3. citation 23 is a little strange. It seems that it was published in ICCV2023. The citation should be corrected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful advice and valuable questions, we will respond to your concerns point by point.\n\n> W1: Overclaim the CGCD setup.\n\n* **Differences from [16,18]**.  We consider (1) more continual stages (5>3 and 10>3 in table 4) with more novel classes to discover (50%>30% of total classes are new) (2) rehearsal-free, i.e., we do not save class-wise labeled samples for old classes, which alleviate privacy and memory issues.\n* Thanks for pointing out the issue. We acknowledge that the settings of our study still have some gaps compared to real-world scenarios and it is a basic exploration towards real-world applications.\n* We will revise the statement as you suggested.\n\n> W2: Too many hyper-parameters and loss terms. \n\n* **Explain of each loss function**. Overall, our method can be divided into three parts: $L_{Happy}=L_{new}+L_{old}+\\lambda_3L_{con}^u$. Each of them is important, see table 5 for ablation studies.\n  * $L_{new}$: discover new classes (cluster). It has two parts (1) self-distillation $L_{self-train}$ with sharpen targets for self-training. (2) soft entropy regularization $L_{entropy-reg}$ to alleviate prediction bias, wherein eq (4) is regularization between old and new classes as a whole.\n  * $L_{old}$: mitigating forgetting of old classes. It also has two parts (1) hardness-aware prototype sampling $L_{hap}$ eq (10) and (2) KD loss $L_{kd}$.\n  * $L^u_{con}$: unsupervised contrastive learning (i.e., SimCLR) for basic representation.\n\n* The effect of eq (4) is to reserve the necessary probability for learning new classes and mitigate prediction bias towards old classes.\n* **About hyper-parameters**.\n  * In our method, we fix the weight of $L_{self-train}$ and $L_{hap}$ as 1, considering they are related to the core part of learning new and old classes. (just like cross-entropy loss in standard deep learning)\n  * We only need to tune the weights $\\lambda_1,\\lambda_2,\\lambda_3$ for $L_{entropy-reg}$, $L_{kd}$ and $L_{con}^u$, respectively. For the detailed sensitivity analysis, see the table in **General Response to Common Concern 1**. The optimal weight for the three loss are all near to 1.\n  * In practice, we set all the loss weights to 1 for all datasets without fussy tuning, which works fine and generalizes well to several datasets. As a result, we do not need to extensive search, **just set all the loss weights to 1 and it works fine**.\n\n> W3: In table 5, what does new class mean? Why KD improve new class by a large margin?\n\n* Good question. In table 5, new class means the average accuracy of new classes over 5 stages, i.e., $C_{new}^1,\\cdots, C_{new}^5$. i.e., we report the average of new accuracy across five continual tasks.\n* In C-GCD, the classification performance is determined by both the feature space and the classification head.\n  * Without KD loss, the feature space may shift and become mismatched with the head, causing the model to misclassify some new classes as old, which significantly reduces the accuracy of new classes.\n  * Additionally, since the evaluation of C-GCD relies on Hungarian matching across all classes to achieve the highest overall accuracy (see line 224), there are generally more old classes, which tends to stabilize the old classes while new classes experience greater fluctuation.\n\n> W4: Provide evidence to show that two issues are mitigated.\n\n* Very good suggestion. We will append the following results to the revised manuscript.\n\n* **Prediction bias**. We provide two metrics: (1) $\\Delta p=\\overline p_{old}-\\overline{p}_{new}$: the difference in marginal probabilities between old and new classes. (2) the proportion of new classes' samples misclassified as old classes (new$\\to$old). The results are as follows (after stage-1):\n\n  |           (in %)           | $\\Delta p \\downarrow$ (on C100) | new$\\to$old $\\downarrow$ (on C100) | $\\Delta p \\downarrow$ (on CUB) | new$\\to$old $\\downarrow$ (on CUB) |\n  | :------------------------: | :-----------------------------: | :--------------------------------: | :----------------------------: | :-------------------------------: |\n  | Ours w/o $L_{entropy-reg}$ |              81.50              |               63.25                |             83.20              |               65.80               |\n  | Ours w/ $L_{entropy-reg}$  |            **5.76**             |             **10.20**              |           **10.25**            |             **11.05**             |\n\n  The results from two datasets demonstrate that $L_{entropy-reg}$ effectively reduces prediction bias, with a significantly lower marginal probability gap and fewer new class samples misclassified as the old.\n\n* **Hardness bias**. We also present two metrics: (1) $Var_0$: variance in accuracy of the initial labeled classes $C_{init}^0$. (2) hardest Acc: accuracy of the hardest classes in $C_{init}^0$. Results are as follows (after training on 5 stages):\n\n  |           (in %)           | $Var_0 \\downarrow$ (on C100) | hardest Acc $\\uparrow$ (on C100) | $Var_0 \\downarrow$ (on CUB) | hardest Acc $\\uparrow$ (on CUB) |\n  | :------------------------: | :--------------------------: | :------------------------------: | :-------------------------: | :-----------------------------: |\n  | Ours w/o hardness sampling |            23.04             |              65.10               |            21.77            |              62.65              |\n  | Ours w/ hardness sampling  |          **10.33**           |            **70.23**             |          **9.28**           |            **68.40**            |\n\n  Results show that hardness-aware sampling effectively reduces hardness bias, with lower accuracy variance and higher hardest Acc.\n\n> Minor typos.\n\n* Thanks for your detailed review. We will correct all the typos carefully.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bimRzs6jD2", "reviewer": "Reviewer_jUDg", "summary": "The paper proposes a novel method for the Continual Generalized Category Discovery task, addressing the challenges of discovering new classes and preventing forgetting. The approach introduces Clustering-guided Initialization and Group-wise Soft Entropy Regularization for class discovery, as well as Hardness-aware Prototype Sampling for mitigating forgetting. Rigorous experimentation across several datasets demonstrates a significant improvement in performance.", "questions": "Please see the Weaknesses", "limitations": "No", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The organization, presentation, and writing of the paper are very clear, and the figures are attractive and easy to understand.\n\n2. The paper analyzes and proposes solutions for several important issues in Continual Generalized Category Discovery (C-GCD), demonstrating strong innovation.\n\n3. The experimental analysis is thorough, with excellent results. The sufficient ablation studies confirm the contribution of each proposed innovation.", "weaknesses": "1. In the C-GCD setting described in the paper, during the continuously discovering stage, the samples of old classes are available. Why not use these old class data directly instead of the Hardness-aware Prototype Sampling method? Some explanation or experimental validation should be added.\n\n2. How is the margin probability computed in line 173 if all the labels are not available?\n\n3. The proposed method uses DINO pretrained ViT-B/16 as the backbone, which, as I understand, is pretrained on the ImageNet dataset. This results in information leakage for CIFAR-100, ImageNet-100, and TinyImageNet datasets in the continual learning process, as all classes are already known in the pretrained model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful advice and valuable questions, we will respond to your concerns point by point.\n\n> W1: Why not use these old class data directly instead of hardness-aware prototype sampling?\n\n* Good question. At each continual stage of C-GCD, all training data are unlabeled, i.e., **unlabeled** old and unlabeled new classes samples are mixed together, and the model does not know which samples belong to old categories, thus they cannot be directly utilized.\n* We employ hardness-aware prototype sampling to prevent forgetting, without the need to store class-wise labeled samples of old classes.  By contrast, previous methods store class-wise labeled data, which could cause privacy issues.\n\n> W2: How to compute margin prob in line 173 without labels?\n\n* For the marginal probability, we compute the average of model **predictive** probabilities across a batch, namely $\\overline p\\in\\mathbb{R}^{K^t}=\\frac{1}{|B|}\\sum_{i\\in B}p_i$, which does not require any labels, and **only uses model predictions**.\n\n> W3: DINO pretrained ViT-B/16 backbone results in information leakage for CIFAR100 and TinyImageNet.\n\n* Very good question. We will address this point in detail and incorporate them into the revision.\n* Firstly, DINO is an **self-supervised (unsupervised)** pre-training scheme without any labels. So there is no label leakage in DINO. The main objective of DINO is to learn good and general initial representations for the visual encoder. DINO does not directly learn classification heads for downstream tasks.\n\n* Second, since using a DINO-pretrained ViT has become a standard practice in the literature of GCD [R1, R2, R3], we just followed this setup to ensure a fair comparison.\n\n* Third, we acknowledge your concerns. To thoroughly eliminate the influence of information leakage, we adopted the settings in [R4, R5] and conducted experiments using a DeiT pretrained on 611 ImageNet categories (**explicitly excluding those from CIFAR and TinyImageNet**). Average accuracy over 5 continual stages are shown below:\n\n  |             |           | CIFAR100  |           |           | TinyImageNet |           |\n  | :---------: | :-------: | :-------: | :-------: | :-------: | :----------: | :-------: |\n  |             |    All    |    Old    |    New    |    All    |     Old      |    New    |\n  | VanillaGCD  |   42.50   |   44.57   |   28.16   |   29.28   |    30.04     |   24.02   |\n  |   MetaGCD   |   46.64   |   48.63   |   32.96   |   35.24   |    36.48     |   26.80   |\n  | Happy(Ours) | **69.34** | **71.37** | **57.44** | **53.50** |  **55.67**   | **39.40** |\n\n  As it shows, in scenarios without any information leakage, our method still consistently outperforms previous methods by a large margin. In contrast, methods like MetaGCD, which rely on non-parametric clustering, suffer significant performance declines due to unstable clustering with less pretrained classes.\n\n\n\n\n\n**References**:\n\n[R1]. Generalized Category Discovery. CVPR 2022.\n\n[R2]. Parametric Classification for Generalized Category Discovery: A Baseline Study. ICCV 2023.\n\n[R3]. MetaGCD: Learning to Continually Learn in Generalized Category Discovery. ICCV 2023.\n\n[R4]. Learnability and algorithm for continual learning. ICML 2023.\n\n[R5]. Class Incremental Learning via Likelihood Ratio Based Task Prediction. ICLR 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EXWNvFrTx0", "reviewer": "Reviewer_cVL6", "summary": "The paper presents a novel approach to the task of Continual Generalized Category Discovery (C-GCD). The proposed framework, named Happy, aims to address the challenges of continuously discovering new classes from unlabeled data while preventing the forgetting of previously learned classes. The authors identify two primary issues in C-GCD: prediction bias and hardness bias. To mitigate these, they introduce clustering-guided initialization, soft entropy regularization, and hardness-aware prototype sampling. Experimental results show that the Happy framework significantly improves performance across various datasets, including notable gains on ImageNet-100.", "questions": "Generalizability to Other Domains: While the framework is validated on multiple vision datasets, it is unclear how well it would generalize to other domains, such as natural language processing or other types of sequential data.\n\nAdapt to the SoTA methods: Is it possible to adopt the method to several significant recent works in the field, such as SPTNet, TIDA, and InfoSieve?", "limitations": "NA", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Clear Contributions: The paper clearly outlines its contributions, making it easier for readers to understand the novelty and significance of the work.\n\nExperimental Validation: The framework is validated through extensive experiments on multiple datasets, demonstrating significant improvements over state-of-the-art methods. The 7.5% gain on ImageNet-100 is particularly impressive.\n\nAddressing Biases: The identification and mitigation of prediction and hardness biases are well-articulated and experimentally supported. This shows a deep understanding of the underlying issues in continual learning.", "weaknesses": "Complexity of the Framework: While the combination of multiple techniques (clustering-guided initialization, soft entropy regularization, and hardness-aware prototype sampling) is innovative, it also adds complexity to the framework. This could make it challenging to implement and tune in practice.\n\nScalability Concerns: The paper does not extensively discuss the scalability of the proposed framework. As the number of classes and stages increases, the computational and memory requirements could become prohibitive.\n\nEvaluation Metrics: The paper primarily focuses on accuracy improvements but does not delve deeply into other potential evaluation metrics, such as computational efficiency, memory usage, or robustness to different types of data distributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful advice and valuable questions, we will respond to your concerns point by point.\n\n> W1: Complexity of the Framework.\n\n* **Effect of each component**. Considering that C-GCD is a challenging task, each component is essential and addresses specific issues. (1) Learning new class: Cluster-guided initialization provides a robust initialization for new class heads, while entropy regularization mitigates prediction bias and allocates necessary probabilities for discovering new classes. (2) Preventing forgetting old classes: hardness-aware prototype sampling effectively controls the forgetting of difficult old classes.\n* **Implementation**. (1) Cluster-guided only runs once at each continual stage. (2) Soft entropy regularization, (3) hardness modeling could be implemented on the fly. All of them consume less than 1% of total memory and time, see W3 for more details.\n\n* **Hyper-parameter tuning**.\n  * We fix the loss weights of self-training $L_{self-train}$ and hardness-aware prototype sampling $L_{hap}$ as 1, considering they are the main loss for new and old classes. And our method basically has three weight parameters $\\lambda_1,\\lambda_2,\\lambda_3$ for $L_{entropy-reg}$, $L_{kd}$ and $L_{con}^u$ respectively. Please see **General Response to Common Concern 1** for more details and sensitivity analysis.\n  * Overall, the optimal values for each hyperparameter are close to 1. In our experiments, **we simply set all weights to 1, which shows remarkable results across all datasets.** Thus, our method does not require complex tuning of parameters and exhibits strong generalization capabilities.\n\n> W2: Scalability of the proposed method.\n\n* The time consumption increases linearly with the number of classes and stages, which is the **same for all continual learning methods**.\n* Disk Memory consumption. To mitigate forgetting of old classes, we save one prototype feature per class (768-dim) using about 3KB, whereas previous methods store multiple labeled images, with one image 224× requiring 588KB and ten images requiring over 5000KB. Thus, our method consumes **less than 1/1000** of previous replay-based methods.\n\n> W3: Evaluation Metrics: computational efficiency, memory usage, or robustness to different distributions.\n\n* **Computational efficiency**. The proposed three components are computationally efficient. We run experiments on a single 4090 GPU. For CIFAR100, we train 5 continual stages with 30 epochs for each, it takes $\\sim$2.5h in total. (1) Cluster-guided initialization is run only once at the start of each stage, it takes $\\sim$15s, only 0.83% of total time. (2) Hardness-aware sampling and (3) entropy regularization occupy less than 1% of the training time. While KD loss takes relatively the most time, around 30% of the time, however, it is necessary and widely used in continual learning.\n* **Disk memory consumption**. As in W2, we only save features for old classes with only <1/1000 memory cost compared with replay-based continual learning. (3KB/5000KB) and our method alleviates privacy issues.\n* **GPU memory consumption**. Training costs around 14192MB GPU memory in total. Within it, hardness-aware sampling and entropy regularization occupy less than 0.1%, and KD loss cost 5672MB, whose main objective is to mitigate forgetting, it consumes even less memory than replay-based methods ($\\sim$8000MB) with the same batch size.\n* **Robustness to different distributions**. We have evaluated various methods on several corrupted distributions, see **General Response to Common Concern 2**.\n\n> Q1: Generalizability to Other Domains.\n\n* Good question. This paper primarily explores visual tasks of visual category discovery. The underlying spirit could also be adapted to NLP data, we leave it for future work.\n* For other distributions, we conduct experiments on CIFAR100-C with various corrupted distribution, see **General Response to Common Concern 2**.\n\n> Q2: Adapt to the SoTA methods.\n\n* Great suggestion. We implemented SPTNet [R1] and adapted it to the C-GCD (by incorporating alternate prompt learning), results on CIFAR100 are as follows:\n\n  |   CIFAR100   |    All    |    Old    |    New    |\n  | :----------: | :-------: | :-------: | :-------: |\n  |    SPTNet    |   59.54   |   67.84   |   8.56    |\n  |  SPTNet+LwF  |   63.89   |   70.33   |   24.31   |\n  |    Happy     |   69.00   |   71.82   |   51.36   |\n  | SPTNet+Happy | **70.81** | **73.58** | **54.18** |\n\n  we also adapt InfoSieve [R2] (by incorporating self-supervised code extraction into contrastive learning) as you suggested, results on CUB are shown below:\n\n  |       CUB        |    All    |    Old    |    New    |\n  | :--------------: | :-------: | :-------: | :-------: |\n  |    InfoSieve     |   57.23   |   65.28   |   7.75    |\n  |  InfoSieve+LwF   |   63.18   |   69.30   |   25.60   |\n  |      Happy       |   68.88   |   71.29   |   53.13   |\n  | InfoSieve +Happy | **69.80** | **72.32** | **54.30** |\n\n  Results indicate that adapting SoTA GCD methods to C-GCD with Happy could further enhance the performance.\n\n* We will cite related papers and add them to the revised manuscript.\n\n\n\n**References**:\n\n[R1]. SPTNet: An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning. ICLR 2024.\n\n[R2]. Learn to Categorize or Categorize to Learn? Self-Coding for Generalized Category Discovery. NeurIPS 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their dedication and insightful comments, and we believe these comments are significant for improving the overall quality of this paper.

We are pleased that the reviewers appreciate our paper from various aspects, including the novelty of the method [TDQJ,jUDg,cVL6], clear writing [TLcv, jUDg, cVL6], and remarkable performance [TDQJ, TLvc, jUDg, cVL6].

In this paper, we explore the task of Continual Generalized Category Discovery (C-GCD). At each continual stage, data from new and old classes are mixed together without labels. Compared to previous settings, our setup is more realistic, (1) encompassing more stages and new classes to discover, (2) does not preserve class-wise labeled samples of old classes to replay.

Here, we respond to some common concerns:

> Common Concern 1: The method is complex with many hyper-parameters.

* Our approach can be divided into three parts: (1) $L_{new}$: employs self-distillation $L_{self-train}$ and $L_{entropy-reg}$ to reduce prediction bias to learn new classes. (2) $L_{old}$: hardness-aware prototype sampling $L_{hap}$ and $L_{kd}$ to further mitigate hardness bias and forgetting of old classes. (3) $L_{con}^u$: contrastive learning to ensure basic representations. Each of them is important.
* **Hyper-parameters**. We fix the weight of $L_{self-train}$ and $L_{hap}$ as 1, considering they are the main objectives for new and old classes. As a result, our method mainly contains three loss weights $\lambda_1,\lambda_2,\lambda_3$ for $L_{entropy-reg}$, $L_{kd}$ and $L_{con}^u$ respectively. Here, we give sensitivity analysis on CIFAR100 as follows: (average All Acc over 5 stages)

  | $\lambda_1$ ($L_{entropy-reg}$) |   0   |    0.5    |  1.0  |  3.0  |  5.0  |
  | :---------------------------------------: | :---: | :-------: | :---: | :---: | :---: |
  |                  All Acc                  | 60.60 | **69.04** | 69.00 | 63.98 | 59.23 |

  | $\lambda_2$ ($L_{kd}$) |   0   |  0.5  |  1.0  |    3.0    |  5.0  |
  | :------------------------------: | :---: | :---: | :---: | :-------: | :---: |
  |             All Acc              | 65.31 | 66.98 | 69.00 | **69.30** | 68.90 |

  | $\lambda_3$($L_{con}^u$) |   0   |  0.5  |    0.7    |  1.0  |  3.0  |
  | :--------------------------------: | :---: | :---: | :-------: | :---: | :---: |
  |              All Acc               | 68.74 | 68.94 | **69.16** | 69.00 | 68.92 |

  As shown above, the model is relatively insensitive to $\lambda_3$, whereas $\lambda_1$ and $\lambda_2$ have a more significant impact. Overall, the optimal values for each hyperparameter are close to 1.

  In our experiments, **we simply set all weights to 1 which shows remarkable results across all datasets.** Thus, our method does not require complex tuning of parameters and exhibits strong generalization capabilities.

* **Computational costs**. The proposed three components: cluster-guided initialization, hardness-aware sampling and entropy regularization bring very small computation overhead, and the consumption (both time and GPU memory) of each part is **less than 1%**. More details are in **Response to Reviewer [cVL6]**.

> Common Concern 2: Generalization to different distributions.

* We conduct experiments on the distribution-shift dataset CIFAR100-C [R1] with severity=2, and test the model on all 100 classes after 5 stages of training. The results are shown below:

  | Distribution | Original  | +gauss noise |   +snow   |   +fog    | +motion blur | +pixelate |
  | :----------: | :-------: | :----------: | :-------: | :-------: | :----------: | :-------: |
  |  VanillaGCD  |   51.36   |    23.36     |   44.38   |   51.12   |    48.52     |   46.77   |
  |   MetaGCD    |   55.78   |    23.53     |   44.80   |   53.63   |    48.69     |   48.72   |
  | Happy (Ours) | **59.99** |  **36.51**   | **52.61** | **56.77** |  **51.60**   | **52.13** |

  Our method consistently outperforms others across several unseen distributions, showcasing its strong robustness and generalization ability.

* **See the Rebuttal PDF for all 15 distributions of CIFAR100-C**.

> Common Concern 3: Evidence to show prediction bias and hardness bias are mitigated.

* **Prediction bias**. We present two metrics (after stage-1): (1) $\Delta p=\overline p_{old}-\overline{p}_{new}$: the gap of marginal predictive probability on old and new classes, (2) the proportion of new classes samples misclassified as old ones. Both metrics are low in our method, which shows that the bias is mitigated. **See response to Reviewer [TLvc] for more details**. 
* **Hardness bias**. We also show two metrics (after stage-5): (1) $Var_0$: the variance of accuracy among the initially labeled classes $C_{init}^0$, (2) $acc_{hard}$: accuracy of the hardest class. The former decreases and the latter increases with our method, which means that the bias is mitigated. **See response to Reviewer [TDQJ, TLvc] for more details**.

For other specific questions, we respond to each reviewer point by point as below. We will carefully revise all comments from the four reviewers and incorporate them into the revised paper. Thanks again to all reviewers for their valuable suggestions!


**References**:
[R1]. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. ICLR 2019.

Meta-review: This paper studies a new task called Continual Generalized Category Discovery (C-GCD), which aims to incrementally discover more new classes from unlabeled data over a long period while maintaining the ability to recognize previously learned classes without storing samples of past classes. There are two challenges in C-GCD, including the prediction bias in probability space and the hardness bias in feature space. To address these challenges, this paper proposes cluster-guided initialization and soft entropy regularization to mitigate prediction bias, and proposes hardness-aware prototype sampling to mitigate hardness bias and forgetting. Experimental results demonstrate the effectiveness of the proposed method.

This paper finally received 1x Borderline reject, 2x Borderline accept, 1x Weak accept. Reviewer TDQJ has raised concerns about the negative impacts of the proposed method, lack of theoretical results, the limited applicability, and the stability of the method. After the authors' rebuttal, Reviewer TDQJ indicated that some concerns were addressed. Reviewer TLvc thinks that the setting is over-claimed, there are too many hyper-parameters and loss terms, more evidence should be provided to show that the two issues are mitigated. The authors have provided corresponding feedback, and Reviewer TLvc confirmed that these concerns were resolved and would keep the original score (Borderline accept). Reviewer jUDg mainly has concerns about the C-GCD setting and the potential information leakage issue. After discussing with the authors, Reviewer jUDg acknowledged that these concerns were addressed and increased the score from 6 (Weak accept). Reviewer cVL6 (Borderline reject) has concerns about the complexity of the framework, scalability concerns, evaluation metrics, generalizability to other domains, and adaptation to state-of-the-art methods. Reviewer cVL6 has also been involved in the discussion with the authors, but indicated that most of the concerns were not addressed. Reviewer cVL6 still has concerns about the hyperparameters and theoretical contributions. I have checked all the comments from Reviewer cVL6 and the authors, I would agree with Reviewer cVL6 regarding the two concerns, but I feel that the hyperparameter issue is not a fatal problem. Besides, the lack of theoretical contributions also may not be a very strong reason to reject this paper, because this paper studies a new practical task with an innovative method achieving satisfactory performance. I personally think that whether a paper should be accepted should not simply depend on whether this paper has some minor problems. We should focus on the key merits of the paper. As most reviewers acknowledged, this paper indeed has merits in a new task, an innovative method, and good performance.

The current average score of this paper is 5.0, which means that the decision of this paper is clearly on the borderline. I have carefully checked the paper, all the comments, and all the discussions. I noticed that the relatively low score is actually caused by some minor problems. After much consideration, I feel that this paper could be accepted.
Decision: Accept (poster)