Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery
Authors: Alireza Fathalizadeh
Roozbeh Razavi-Far
Venue: Accepted by TMLR ()

Abstract: Continual generalized category discovery has been introduced and studied in the literature as a method that aims to continuously discover and learn novel categories in incoming data batches while avoiding catastrophic forgetting of previously learned categories. A key component in addressing this challenge is the model’s ability to separate novel samples, where Extreme Value Theory (EVT) has been effectively employed. In this work, we propose a novel method that integrates EVT with proxy anchors to define boundaries around proxies using a probability of inclusion function, enabling the rejection of unknown samples. Additionally, we introduce a novel EVT-based loss function to enhance the learned representation, achieving superior performance compared to other deep-metric learning methods in similar settings. Using the derived probability functions, novel samples are effectively separated from previously known categories. However, category discovery within these novel samples can sometimes overestimate the number of new categories. To mitigate this issue, we propose a novel EVT-based approach to reduce the model size and discard redundant proxies. We also incorporate a novel experience replay and knowledge distillation mechanisms during the continual learning stage to prevent catastrophic forgetting. Experimental results demonstrate that our proposed approach outperforms state-of-the-art methods in continual generalized category discovery scenarios.

Published in Transactions on Machine Learning Research (02/2026)
Proxy-Anchor and EVT-Driven Continual Learning Method for Generalized Category Discovery
Alireza Fathalizadeh
diirea..atholizddee@@b...c
Faculty of Computer Science University of New Brunswick
Roozbeh Razavi-Far
Pacully of Computer Science
University of New Brunswick
Reviewed on OpenReview: obter:///penreeveu.
openreview. net/ net/forum? id= JJQ
Abstract
Continual generalized category discovery has been introduced and studied in the literature as a method that aims to continuously discover and learn novel categories in incoming data batches while avoiding catastrophic forgetting previously learned categories. key com- ponent in addressing this challenge is the model's ability separate novel samples, where Extreme Value Theory (EVT) has been effectively employed. In this work, propose novel method that integrates EVT with proxy anchors to define boundaries around prox- ies using a probability of inclusion function, enabling the rejection of unknown samples. Additionally, we introduce a novel EVT-based loss function to euhance the learned repre- sentation, achieving superior performance compared to other deep-metric learning methods in similar settings. Using the derived probability functions, novel samples are effectively sep- arated from previously known categories. However, category discovery within these novel samples can sometimes overestinate the number of new categories. To mitigate this issue, propose novel EVT-bbaed approach to reduce the model size and discard redundant proxies. We also incorporate novel experience replay and knowledge distillation mecha- nisms during the continual learning stage to prevent. catastrophic forgetting. Experimental results demonstrate that our proposed approach outperforms state-of-the-ar methods in continual generalized category discovery scenarios. The implementation code available at: https://gitthn cow.///wmOno01//ATGGOOO1011221
1
Introduction
Most traditional machine learning algorithms operate under the closed-world assumption, in which the training and test data are drawn from the same label and feature spaces and no new class is introduced to the model during the test phase. However, in a more practical scenario, the training data lacks complete knowledge of the world and unknown classes may emerge during the test phase. A crucial problem is that model that operates under the closod set assumption usually makes high-confidence prodictions for these novel samples. This is particularly problematic in critical systems like autonomous driving, where misclassification cause serious harm, requiring the model to detect novel samples and integrate potential novel classes into the knowledge base of the model.
Humans have the ability lo identify new entities and group them without prior knowledge, while also recog- nizing them upon subsequent encounters., e.g., one can see new types of bird species that they did not see before, but they can still group and categorize them once they see them and add this new information their knowledge base without affecting what they know about other types of birds, i.c., no forgetting. This has inspired scenario, called Continual Generalized Category Discovery (CGCD) Zhang et al. 2002 Wu

---

Published in Transactions on Machine Learning Research (02/2026)
et al. (2023] Kim et al. 2023 Zhao & Mac Aodha (023), in which a model is trained an initially labeled dataset and after this initial stage, the model is only introduced to unlabeled data and is expected to detect and discover potential novel categories in the data and integrate them into the model without compromising the performance of previously Icarned tasks.
The problem of continual generalized category discovery can be decomposed into three subtasks: 1) Novelty detection: Detecting samples that do not belong to any previously learned and known categories Geng al.
(2020) Yang ct al. (2024). 2) Category discovery: Identifying potential novel categorics in an unlabeled
dataset Han et al.
[2021] Vaze et al. (022)) and 3) Continual learning: Integrating newly discovered
categories into the model without catastrophic forgetting of previously learned categories Rolnick et al.
(2019); Wang et al. (2024). Almost any method that solves each of these sub-tasks can be combined to
handle the CGCD scenario, however, this might result in sub-optimal solutions as each method is trying to solve different task, and balancing these compoting objectives effectively is challenging Zhang al. (2022)
A unified framework tailored to this problem then necessary achieve a promising balance.
A crucial component of such a framework is the ability to clearly distinguish between known and unknown samples. To achieve this, we leverage Extreme Value Theory (EVT), which has demonstrated its effectiveness in addressing the open-set recognition problem Bendale & Boult (2016): Rudd et al. (2017); Geng et al. (2020). EVT is a statistical framework for modeling extreme deviations in data by analyzing the tail ends of the distribution Coles et al. (200I). In this context, EVT is used to model the margin distribution each proxy, which a representative of a class and learned as a part of the network parameters Kim al. (2020), relative to other samples and define boundaries around proxies to reject unknown samples by developing a probability of inclusion function. Building on this, we propose novel loss, called ect loss, which is derived from the EVT analysis of each proxy. In addition to preparing the model for unknown rejection, this loss improves the learned representation.
In the context of novel class discovery, existing methods often rely on clustering techniques, which tend to overostimate the number of classes. We mitigate this issue by utilizing EVT to reduce the mumber of estimated novel categories by discarding redundant ones, leading to improved discovery of new classes, while
minimizing the forgetting of previously learned ones. To avoid catastrophic forgetting of previously learned
data, we employ commonly used methods of knowlodge distillation
Li & Hoiem (017 ; Hou et al. (2019)
and experience replay Chaudhry et al. (2019) : Rolnick et al. (2019) Lin et al. [2023]; ; Zheng et al. (0024)
During experience replay, we employ an EVT-driven distribution to sample features, which outperforms the Gaussian distribution commonly used in related work. In summary, our contribution can be summarized as follows:
We propose a novel approach, called proxy-anghor and EVT-driven continual-learning method for generalized category discovery (CATEGORIZER), Extensive evaluations multiple datasets demonstrate that our proposed method outperforms state-of-the-art approaches in the CGCD set- ting.
We introduce a novel loss function called eet loss, which is derived from proxy anchors Kim et al. (2020) and extreme value theory. This loss outperforms deep metric learning methods used in similar methods.
We propose a novel approach to miligate over-estimaling the number of novel categories in the discovery phase. This has been done by means of extreme value theory, which boosts the performance of the model in discovering categories as well as minimizing the forgetting of previously learned categories.
We propose an EVT--driven Weibull-based feature replay strategy for continual learning and show that it consistently outperforms Gaussian-based approaches in prior work.
2 Related Works
Novelty detection aims identify samples from novel classes not encountered during the training phase. Methods for novelty detection can be categorized into two main groups of open-set recognition (OSR)
2

---

Published in Transactions on Machine Learning Research (02/2026)
Initial Stage
Continmal Learning Stuge
Time-trrp
Time
Japu: donasee
000
A
O O
boldeed Duta
Canegory discovery
tnowe Cougerics
Knewn cotsgories
Oicovered Categories
Figure 1: The general presentation of the Continual Generalized Category Discovery (CGCD) setting. In the initial stage, a labeled dataset is provided train the initial model. After the initial stage, the model enters the continual learning stage, in which no labeled data is provided. The input data in this stage can contain samples belonging to novel or previously known categories. The model is expected to discover polential novel categories in this unlabeled data and integrate them into the model without compromising the performance of previous categorics and making assumptions about the number of novel categorics.
and out-of-distribution (OOD) detection. Despite having small differences, these methods share the goal of detecting samples from unknown categories. OOD detection methods include post-hoc, reconstruction, distance-based, density- and training-staged approaches. Post-hoc met hods do not change the training process Hendrycks & Gimpel (0016); Liang et al. (2017); Hendrycks et al.
2019 Liu et al. (2020); Sun et al.
(2021 9: Sun Li (2022). Reconstruction methods leverage the fact that models trained on ID data cannot
effectively compress or reconstruct OOD data [Denouden et. al. (2018); Jiang et al. (2023 Distance-based
methods use distance measures to ID centroids prototypes Lee et al.
2018
: Sun et al. (2022 Wang et al.
20022; Ming et al. 2022); Kim et al. (2024). Density- based methods rely on probabilistic modeling to flag
samples in low-density regions as OOD Abati et al. (2019)
:; Jiang et al.
(2021): ; Zisselman & Tamar 2020
Training-staged methods modify the training process to separate ID) and
OOD data Hendrycks et al. 2018
Yu & Aizawa 2019 0 Lee et al. [2017a] Du et al. (0022 Hsu et al. (2020).
Open-set recognition methods are categorized as discriminative, generative, or hybrid. Discriminative classi-
fiers are extended to perform open-set recognition either using distance measure (Shu et al. (2020); Hassen
& Chan 2020); Cevikalp (2016) or constructing probability functions for ID/OOD distinction Scheereik &
Rigling (0016); Shu el al. (2017); Rudd et al. (2017); Yoshihashi et al. (2019); Oza & Patel (2019). Gen-
erative models create synthetic unknown samples for training Ge et al. (2017)
: Neal et al. (2018) Hybrid
approaches combine discriminative and generative strategics to learn embeddings jointly Zhang et al. 2020
Zhou et al. 2021 Chen et al. 2001T)
Category discovery methods try to identify novel categories within unlabeled data. Early approaches assume that unlabeled data contains only novel samples and typically follow a multi-stage process, involving
pre-training and fine-tuning on labeled and unlabeled data. These approaches include generating pair- wise
pseudo-labels based on feature magnitudes Han et al. (2021): Zhao & Han (2021] Zhong et al. (2021)
clustering unlabeled data using deep embedding clustering Xie et al. (0016); Han et al. (2019), and assigning pseudo-labels through self-labeling Joseph et al. (2022).
Recent advancements have addressed more realistic scenarios through Generalized Category Discovery (GCD), where unlabeled data includes both previously known and novel classes. GCD Vaze et al. (2022) combines self-supervised pre-training, contrastive learning-based fine-tuning, and semi-supervised k-means clustering for novel class discovery. Gaussian mixture model for generalized category discovery with Protypi- cal Contrastive lcarning (GPC) Zhao et al. (2023a) employs semi-supervised Gaussian Mixture Model with prototypical contrastin learning, enabling joint representation learning, cluster estimation, and label assign- ment without assuming a fixed number of clusters. In contrast to the multi-stage methods, UNified Objective function (UNO) Fini et al. (2021) presents a unified objective function that simultancously addresses fea- ture extraction, labeled data classification, and clustering of unlabeled data by leveraging pseudo-labels and employing multiple clustering and over-clustering strategies. Cross-instance Positive Relations (CiPR) Hao
3

---

Published in Transactions on Machine Learning Research (02/2026)
et al. (2024) proposes a semi-supervised hierarchical clustering algorithm to obtain reliable cross-instance relations in order to bootstrap the representation in contrastive learning.
Continual learning methods aim to address the challenge of catastrophic forgetting, enabling models to retain previously learned knowledge, while adapting to new tasks. Existing approaches can be broadly categorized into the following: Replay-based techniques focus on mitigating forgetting by storing a subset of previous task data and replaying it during training to reinforce memory of the earlier tasks Chaudhry
et al. (2019); Rolnick et al.] (2019); Zhenng et al. (2024) : Lin et al. (2023). Regularization-basse techniques
that aim to preserve prior knowledge by guiding the training process, including parameter regularization
to prevent critical parameters from significant updates Kirkpatrick et al. (2017): Lee et al. (2017b) Zenke
let al. |017);; Wistuba et al. (2023), knowledge distillation, where a "teacher' model transfers knowledge to a "student" model Li & Hoiem (2017;: Wu et al.] (2018); Hou et al. (2019), and data regularization that constrains the optimization process using stored exemplars Lopez-Paz & Ranzato (2017); Chaudhry et al.)
p2018p: Zhao al. (2023b). Meta-learning approaches aim to train models capable of rapidly adapting to new tasks with minimal data, leveraging learned algorithms for efficient generalization [Finn el al. (2017); Gupla
et al. 2018); Ostapenko et al. (2019); Saha & Roy (2025). Architecture- based methods address continual learning by modifying or expanding the model's structure to accommodate new tasks, while preserving existing knowledge Yoon et al. (2017) : Rusu et al. (2016): Hu et al. (2023). Hybrid methods that use the combination of the aforementioned techniques Li et al. (2019); Wu et al. (2021] Douillard et al. 2020)
Continual generalized category discovery methods aim to identify novel categories in unlabeled data, which may include both novel samples and samples from previously known classes. These methods tackle the challenge of discovering new categories incrementally, while retaining knowledge of existing ones. Grow and Merge (GM) Zhang et al. (0022) introduces framework with two models: a static model that pre- serves knowledge of old classes and a dynamic model trained in a self-supervised manner to discover novel classes. Novel samples are identified based on their distance to the prototypes of known classes, and the two modols are merged, when a new task arrives. Incremental Generalized Category Discovery (IGCD) Zhao & Mac Aodha (2023) employs a non-parametric classifier combined with a density-based exemplar selection method that employed to select exemplars samples as well discovering novel classes. Proxy Anchor (PA) Kim et al. (2023) uses proxies learned through proxy anchor loss Kim et al. (2020) to detect novel samples. A non-parametric clustering algorithm clusters and identifies new categories, while catastrophic forgetting is mitigated through experience replay and knowledge distillation. MetaGCD Wu et al. (2023) adopts a meta-learning framework that leverages offline labeled data, to simulate the incremental learning process and utilizes the classic k-means algorithm for novel class discovery. Happy Ma et al. (2024) identifics pre- diction bias and hardness bias as the main challenges in the CGCD scenario, and proposes to address them through clustering-guidke initialization and soft entropy regularization for mitigating prediction bias, along with hardness-aware prototype sampling to tackle hardness bias.
3 Problem Definition
This section presents the problem setting. In the framework of Continual Generalized Category Discovery (CGCD), the model begins with an initial labeled dataset DJ = {(xr, y) E X/ x )) where X1 represents the labeled input data and D denotes the corresponding labels. This dataset is used to train a feature extractor F²= : X + zu, which generates embedding vectors 2", and a classifier CD : 22 + y0, which maps the embeddings to specific classes.
Following this initial phase, no additional labeled data is provided. Instead, the model is soquentially exposod to a series of unlabeled datasets, (DDY= where D - fa E X Here, X contains the unlabeled input data and T denotes the number of time steps. These datasets include examples from previously encountered categories as well as samples from potentially new categorios, i.c., Vin. + 0. No assumption is made about the presence or the number of novel categories in the unlabeled data.
The model's task is to identify and learn new categories, while updating the feature extractor F X - Z' and classifier C² : 2t + )d at each time step t. Over time, yr encompasses all categorics seen so far, i.c., yr yt- U Dieem where View is the newly discovered categories in the current time step t. Figure 1 illustrates this setting.
4

---

Published in Transactions on Machine Learning Research (02/2026)
The key challenge lies in balancing the catastrophic forgetting of previously learned data (stability) with the effective discovering and learning of new categories (plasticity).
Inittal Stage
Continual Learning
step 2
Spiit BVT
Reduuced Model
usttussing EVT
Known
Unicown
Pseudo Labeling
Cle stesttiing
Model Reduction
Pscuda
Cluscring
Apply
Feemolens
Fomplee
Destlation
Pine-tune on loss
Apply EVT
Labeled
Model
Tuskial
Destlation
Upsistdd
Model
Opdined PA
Unlabeled Data Reduced Data
Figure 2: Overview of CATEGORIZER, In the initial stage, the model is first pre-trained on PA loss to derive proxy anchors for different classes. Following this, the EVT analysis is applied to each proxy to compute the Weibull distribution around each proxy and devising a probability of inclusion (PSI) function that is capable of rejecting unknown samples. With the computed distributions, we fine-tune the model on our novel eut loss to get the initial model. In the continual learning stage, the input data containing novel and known samples are separated by thresholding PSIs functions computed in the initial stage into known and unknown samples. Known samples are pseudo-labeled using the current model from the previous step and unknown samples clustered. The model is updated using pseudo-labeled and clustered data, exemplars of previous categories, and distillation loss derived from the previous step model. EVT is applied to the updated model to get updated distribution, where the model is reduced and redundant proxies are discarded. This process repeats for the next steps. The yellow boxes indicate our novel contribution in the proposed scheme.
4 Proposed Method
Our proposed novel method is composed of two main stages: 1) the initial stage and 2) the continual learning stage. In the initial stage, the feature exiractor J and classifier C are trained using the labeled dataset. In the subsequent continual learning stage, the process begins with novelty detection module that divides the data into known and unknown sets. The unknown samples are then subjected to a novel class discovery procedure, which identifies coherent and distinct samples as potential new classes. Finally, the feature extractor and classifier are updated to incorporate the newly discovered samples. Figure[jillusterr the overview of CATEGORIZER. The following sections provide detailed explanations of the main steps involved.
4.1 Initial Stage
To train the feature extractor F we use the Proxy Anchor (PA) loss Kim et al. (2020) to pre-train the model. This PA loss a deep metric learning loss, which is known for its strong performance and fast convergence. Moreover, introducing new classes is straightforward adding proxies. Specifically, it
5

---

Published in Transactions on Machine Learning Research (02/2026)
leverages data-to-data and dlta-to-proxy, relationships to minimize the following loss function:
pa (2) =
log
+
Σ
e
c=((((x)))5))
p
pe
ZEZ
Σ PEP
log
1 Σ
ca(s(2.))+8)
(1)
P
=EZ,
where 5 > O is a margin, a > O is a scaling factor, s(,,,) is the similarity function, P represents the set of all proxies, and pt and p are the set positive and negative proxies in the batch, respectively, For each proxy p, the batch embedding vectors Z split into two sets: 2 stands for positive embedding vectors of p and 2p Z Zz
Algorithm Initial Stage Training Session.
Input: Training dataset Do, learning rate 7) number of epochs F,, batch size B, tail size and threshold
6,
Output: Feature extractor 5 and classifier CO.
Initialize: Proxy set P, feature extractor with random weights.
Step 1: Pro-training on PA loss for epoch
to E do for each Muti-batch B D of size B do Compute PA loss (B) [Eq Update feature extractor F as:
Ol pea
F F 17 OF
end for end for Step 2: Apply EVT for each proxy do
Estimate Weibull parameters Ap and Kp using the tail size T.
Construct probability of inclusion function V (Eq
end for
Construct classifier C using caleulated inclusion functions and threshold (Eq.)) 5
Step 3: Fine-tuning on eut loss for epoch to E do for each mini-batch B C D of size B do
Compute eut loss (B) (Eq 3
Update feature extractor F as:
Hlevi 0F
F x- F
end for end for Return F and Co.
After pretraining, we apply Extreme Value Theory (EVT) to enhance the learned representations and improve the model's ability to detect. novel samples. For each proxy p: E P, identify the nearest. samples, Np. {( )=)) where Vp, * Mj and caleulate their distances in the embedding space mij s(2j, p:) to the proxy Ps. The distribution of these minimal distances follows a Weibull distribution based on EVT Rudd et al. (2017). The probability of a sample x' being within the boundary of proxy pt is given by the inverse Weibull distribution:
V (pi, Ra, As)
(2)

---

Published in Transactions on Machine Learning Research (02/2026)
where 2');: is the similarity between z' and proxy Ps, 2' is the embedding vector of sample x' and R4. are the Weibull shape and scale parameters estimated from the smallest mij values Rudd et al. (2017). This function provides the probability of sample inclusion (PSI) that is capable of rejecting unknown samples.
So far, EVT has been applied as a post-hoc procedure, meaning does not affect the learned representation, potentially leading to suboptimal results. To address this, we propose a novel eut loss fine-tune the model based the estimated Weibull distribution:
(2) =
Σ PE
log
1 Σ
-
((a.e) up
P+
zEZ
+
Σ PEP-
log
1
Σ 2EZp
<(aup) up
(3)
P-
where hi, Ap are the Weibull shape and scale parameters for proxy p. The first term encourages higher probabilities for positive proxies with respect to positive samples, while the second term penalizes high probabilities for negative proxies. This fine-tuning, in addition improving the capability of the model for unknown detection, enhances the learned representation compared to the plain PA loss (see Table 3
Using Eq-
2
the probability of input x' belonging to class I is computed as:
max (p,, 2(8,,,) (i:g= p(122)=
(4)
Using this, the classifier C is defined as:
co arg (1.. P(1 2'), if
(5)
"unknown',
otherwisc,
where M is the total number of classes in the initial stage, and a threshold for rejecting unknown samples. A practical way to select is optimize the trade-off between closed-set accuracy and the rejection of unknown classes through cross-class validation. In experiments, common threshold worked well enough across different datasets. See Algorithm I for the pseudo-code for training the model in the initial stage.
4.2 Continual Learning Stage
After the initial stage and training on the labeled data, the model enters the continual learning stage, where no label will be provided and the input data might contain samples from both novel categories and known categories. The data first goes through a novelty detection module, where it is split into known and unknown samples. The detected unknown samples are then examined using the novel class discovery algorithm to find novel categories. In the end, the model is updated based on these new categories.
4.2.1 Novelty Detection
We utilize the classifier, which was trained using EVT in the initial stage to perform novelty detection. More specifically at time t, we use the classifier learned at the previous step ct-1 to separate unknown samples, as well as assign pscudo labels to known samples
arg ma.. M- 2'),,), it P(1=)22 "unknown',
(6)
otherwise,
where - 1 is the total number of classes accumulated until time - 1, and is the threshold for rejecting unknown samples as discussed before. This way the data split into two sets Di and Dt unknown where Dknown is the set samples belonging to previously known categories (i.e., pseudo labeling) and D Dinknown
samples of novel categories (i.e., input for novelty detection), where D D Dinowa DC unknoww
7

---

Published in Transactions on Machine Learning Research (02/2026)
4.2.2 Novel Class Discovery
The scparated unknown samples D uukkown contain potential novel categories that need to be further dis- covered. To do this the most common approach is clustering these unknown data. Since many clustering algorithms need to know the number of clusters, and estimating this number can add more complexity to the algorithm, we follow Kim et al. (0023) by using non-parametric clustering algorithm. Specifically, we use the affinity propagation method Frey & Dueck (2007) to cluster and discover novel classes. In our experiments, we observed that using affinity propagation for clustering in proposed scheme leads to over-clustering, which leads to overestimating the number of novel categories, degrading the model's perfor- mance in both novel categories and known categories. This problem also exists in other common clustering algorithms (see Appendix A), and is mitigated in the following class incremental learning step.
Algorithm 2 Continual Learning Stage.
Unlabeled data D classifier Ct-1, feature extractor Ft-1, proxy set pl-1, tail size T, learning rate 7), threshold e, number of epochs E, and batch size B.
Input:
Output: Updated feature extractor Ft, updated classifier C³, and updated proxy set pt.
Step 1: Split Unlabeled Data
4: Use classifier Ct-1 ( to split D. into:
Denown and
D unknown'
5: Step 2: Process Known and Unknown Data
6: Pseudo-labeling: For x E D known: assign pseudo-labels using C²-1,
7: Clustering: Apply clustering on D
unknown to group data points into clusters C,,
Initialize the proxy sot p + pt-1
for each cluster C₁ in D Punknown do
10: Initialize proxy p; centroid of C and add it to the proxy set Pt:
pt - Up.).
11: end for
12: Step 3: Train Feature Extractor
13: for epoch e = 1 to E do
14: for cach mini-batch B C Dr, of size B do Compute the total loss L as:
15:
C' = - Cz, + Cha(zo)
C
-
(See Eqs 1 and 7
16:
Update feature extractor Ft as:
Fet Ft-1 dc
OF
17: end for
18: end for
19: Step 4: Recalculate Weibull Distributions and Construct Inclusion Functions
20: for each proxy p & pt do
21: Estimate Weibull parameters Ao and Kp using tail size r.
22: end for
23: Step 5: Model Reduction
24: Remove redundant, proxies of newly discovered categories from the proxy set pt based on the updated
Weibull distributions (Algorithm B
25: Return Updated feature extractor Ft, updated classifier C³ and updated proxy set pt.
8

---

Published in Transactions on Machine Learning Research (02/2026)
4.2.3 Class Incremental Learning
The known samples in the data have been assigned pscudo labels by the previous classes and novel samples have their clustering number as pseudo class. Given this, we can integrale the newly discovered categories into the model. To do this we create new proxies for each discovered cluster, i.e, new category and initialize the proxies on the centroids of clusters, following Kim et al. (2023). To achicve this add the set of proxies of new classes Pnew to set of proxies p {Pt-1 U Puew] Puew and optimize the same proxy anchor loss as in Eq. to improve the performance of the model on the discovered novel categories.
To avoid catastrophic forgetting during this update, the well-known problem of continual learning, we use feature distillation and feature replay methods. More specifically the distillation loss for feature distillation is defined
(xd( = BR,
-
Furthermore, to improve robustinss against. catastrophic forgetting, we sample cosine distances from each proxy's 's Weibull distribution and use spherical interpolation to place unit-norm exemplars on the hypersphere. This preserves the intrinsic geometry of cosine space and aligns more closely with the feature space Iearned by the EVT loss, compared to alternative distributions such as the Gaussian used in Kim al. (2023) (see Table for comparison with Gaussian-based sampling). Based these synthesized features, define the feature replay loss as follows:
(Z) (2).
Ž = {2 SphInterp(p' d), d ~ Weibull(x"-1AA (1111))
(8)
The overall loss for updating the model can be defined as
C L'p ()) + (ha(Eo)
(9)
where one loss optimizes the model based on the current data for discovered categories and two other losses prevent catastrophic forgetting.
After training the model on the Eq. 19), i.c., getting updated feature extractor Ft, we derive the classifier C using the same approach that was used in the initial stage, ie., using EVT and fitting the Weibull distribution using 7 nearest points of opposite class samples for each proxy. The overall pseudo code of the continual learning stage provided in Algorithm 2
As was mentioned in section 4.2.2 the clustering of novel samples leads to over-clustering of the data, i.e., a category might have more than one cluster. After training the model on Eq. (0). the clusters of the same category are likely to be pushed near each other, we employ this fact to reduce the number of proxies following the approach in Rudd et al. (2017) to remove redundant proxies. Let p. be a proxy of a discovered novel class and (p.. p', Ni; Ae) be its corresponding fitted Weibull distribution. To decide the redundancy of the pair (ps. (pc. , Ki, Aa)), i.c., whether to keep it, we define an indicator function I(-) such that
(pp.) {
if (pi, V(pr, Ki, Az)) is kept, otherwise.
(10)
Then, define the optimization problem of selecting a minimum number of proxies such that cach proxy is at least covered by another proxy as follows:
minimize ΣN ENT((p) I(p:)
(11)
subject to Vj3i (p)
where N the number of discovered classes and Ç is the threshold used determine whether proxy is covered by another proxy. In our experiments, set this threshold very high value near to only
9

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 3 Model Reduction using Greedy Set Cover.
Input: Proxy set P {PI,P2. P2.....PP... ,N),, corresponding Weibull parameters (Ae, Ks), inclusion threshold
Output: Reduced proxy set P reduced-
Step 1: Compute Subsets
For each proxy P. E P, compute the set of covered proxies (Eq
S (ps (ps
Kis A1) 2 5}
Step 2: Initialize Greedy Set Cover
Define the universe as all proxics = P, and subsets as S Initialize:
covered k
Preduced 0
Step 3: Greedy Selection
while covered * U do Find the subset S; that covers the maximum uncovered proxies:
arg max |S; \ covered]
Add proxy pe to the reduced proxy set:
Preduced f- {P Preduced Up--
Updale the covered set:
covered <- {covered U S }.
end while Return Reduced proxy set Produced-
reduce proxies that very close to each other. Since the optimization problem in Eq. 11 is special case of the Karp's Set Cover problem Rudd et al. (2017) and is NP-hard, we follow the greedy approach introduced in Slavik (1996) to solve this problem approximately. We begin with defining the universe U as all the proxies, initializing the covered set as an empty set, and finding subsets of covered proxies of each proxy based on 10 In each iteration, we select the subset that covers the most uncovered proxies, add this subset to the covered set, and repeat this process until all proxies are covered (See Algorithm 3).
5 Expermiental Results
This section provides implementation dotails and evaluation metrics, and, then, compares the results obtained by the proposed novel method with those obtained through slate-of-the-art works. finally presents the ablation study for the proposed method.
5.1 Implementation Details
For the sake of fair comparison, utilized ResNet18 He et al. (2016). pre-trained on ImageNet-1k, as the feature extractor across all methods. For the data augmentation, employed commonly used techniques such random crops and horizontal flips. For the proxy anchor loss hyperparameters, is set to 0.1, and 8 is set to 32. The hyperparameters T, e, and Ç of CATEGORIZER are set to 500, 0.75, and 0.999, respectively. The hyperparameter analysis and shown in Table ! and Table [ respectively. This analysis shows that the sensitivity with respoct to these two hyperparameters is low, and a common value has worked well across different datasets, Since is only used during the continual stage and can't be configured in the initial stage, set it to high value make robust across different datasets.
10

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Initial accuracy of the model using different number of neighbours to be used in the EVT modeling. The results show low sensitivity towards this hyperparameter.
DATASET
100
250
500
1000
2000
CUB
80.72
80.91
80.98
80.96
80.76
MIT
72.54
72.45
72.83
72.26
72.54
DOGS
84.86
84.70
84.77
84.67
84.76
CIFAR10
96.07
96.08
96.07
96.15
96.12
CIFAR100
80.90
80.86
80.98
80.83
80.95
AIRCRAFT
65.19
65.22
65.49
65.16
65.01
SVHN
96.50
96.53
96.53
96.54
96.51
MNIST
99.00
98.99
99.03
99.02
98.99
FASHION-MNIS
97.60
97.58
97.55
97.56
97.56
CARS
67.85
67.86
68.30
68.21
68.13
Table 2: Accuracy of the novelty detection module using different thresholds. sensitivity towards this hyperparameter
The result indicates low
DATASET
0.3
0.5
0.75
0.95
CUB
63.76
69.69
70.57
66.03
MIT
63.82
66.19
67.20
63.67
DoGs
61.64
65.55
68.74
65.05
CIFARIO
62.50
66.00
70.00
64.50
CIFARI00
62.30
66.10
69.10
64.80
AIRCRAFT
61.90
65.70
70.20
64.60
SVHN
62.00
66.30
69.50
64.70
MNIST
62.20
66.40
68.30
64.40
FASHION-
61.80
65.80
70.10
64.30
CARS
70.19
72.24
75.32
69.83
We used an RTX 3070 GPU, Ryzon 5 7600x CPU, and 32GB RAM for running the experiments. During the initial training stage, the model was trained using the AdamW optimizer with weight docay of 0.0001 and a learning rate 0.0001 for 60 epochs on PA loss Kim al. (2020) and 60 epochs on our proposed evl loss. The learning rate was reduced by half every five epochs. In the continual learning stage, the model was updated over 10 epochs. In our experiments, we observed that training beyond this point significantly degrade the used
performance novel categories in some datasets (See Figure 5p. To fit the Weibull distributions, torch implementation based on Vastlab] (2024) to estimate the shape and scale parameters of a Weibull distribution. The overhead of this process is negligible, making it fast to compute. For other methods, we adhored to the hyperparametors and network architectures specified their original implementations, referring to the respective papers for details. All reported results represent the average performance over all runs.
5.2 Evaluation Metrics
We employ a clustering accuracy-based measurement, called Hungarian assignment algorithm Kuhn (955 to measure the performance of the model, following previous methods Zhang et al. (2022) Kim et al. 2023
Zhao & Mac Aodha (0023) Wu et al. (2023) This metric is defined as
D
1
M
I(u = 12' (v))).
(12)
D
11

---

Published in Transactions on Machine Learning Research (02/2026)
where [D] is the size of the evaluation dataset, is the indicator function, ys and are the ground-truth label and clustering assignment for the i-th sample, and h" is the optimal assignment. This algorit aligns predicted clusters with true labels to accurately evaluate the clustering performance. Based on this, we use Mau and M, to mcasure the chustering accuracy on all of the categorics and old categories, respectively. To measure the performance drop the previously known classes after discovering and learning new categories, i.c., forgetting, we employ My metric, which proposed Zhang et al. (2022) and defined as
My = max My).
(13)
where My and Mr are the clustering accuracy of old categories, Le., known categories, at the initial stage and time step t.
Table 3: Performance comparison w.r.t. the @e metric on the model trained with different methods on fine-grained datasets. GM Zhang et al. 2022 uses traditional cross-entropy loss, PA Kim et al.
2023
uses proxy anchor loss Kim al. (2020), and TGCD Zhao & Mac Aodha (2023). MetaGGD Wu et al.
20223
combine a supervisod contrastive loss from Khosla al. (2020) and unsupervisod contrastive loss from Chen
al. 2020 and Gutmann & Hyvärinen (2010), respectively, and Happy Ma. al. (2024) combines cross-
entropy loss, supervised KKoslaa al (0020) and unsupervised contrastive loss Chen et al. (2020) Our
proposed evt loss outperforms all mentioned SOTA methods in almost all datasets.
Dataset Method
R@1
R@2
R@4
R@8
GM
55.83
67. 76
75.80
83.20
MetaGCD
59.61
69.91
78.97
83,20
PA
70.57
79.47
86,50
91.16
IGCD
64.01
73.85
81.74
87.36
Happy
42.23
55.28
67.22
77.35
CATEGORIZER
73.69
81.17
87.22
91.66
GM
54.70
65.37
73.06
82.23
MetaGCD
59.32
70.82
78.50
84.47
58.80
69.70
79.70
87.01
MIT
IGCD
63.06
72.01
79.10
84.77
Happy
31.79
43.20
54.92
66.94
CATEGORIZER
65.47
73.77
80.56
88.01
GM
48.20
61.71
72.64
80.49
MetaGCD
62.65
74.15
83.72
89.16
PA
75.77
84.06
91.42
95.60
Dogs
IGCD
64.45
74.46
82.02
87.48
Happy
61.24
75.36
85.30
91.88
CATEGORIZER
77.55
84.96
90.23
93.49
GM
51.29
61.74
71.75
79.74
MetaGCD
52.74
63.58
72.86
80.96
PA
76.13
82.48
88.06
92.21
Cifar100
IGCD
77.70
83.02
86.71
89.30
CATEGORIZER
79.06
83.92
87.18
89.90
GM
72.60
81.27
87.50
91.64
MetaGCD
59.44
71.57
81.33
88.39
PA
69.66
79.12
86.71
92.42
Cars
IGCD
77.08
85.133
90.41
94.03
Happy
60.64
73.81
$3.09
90.23
CATEGORIZER
78.02
85.64
91.08
94.49
To measure the ability of the model to discover and learn novel categories, we employ Ma measure Zhang et al. (0022), which is described as
Ma = T
1
Σ M
(14)
12

---

Published in Transactions on Machine Learning Research (02/2026)
where MA is the clustering accuracy novel categories at the i-th step and T is the total number of learning steps.
0.74
0.66
0.73
0.64
0.62
0.72
0.60
R@
0.71
0.70
0.56
0.69
0.54
0.68
10
20
40
50
60
0
10
20
30
40
50
60
Epoch
Epoch
(a) CUB-200
(b) MIT67
0.78
0.78
0.77
0.76
0.76
0.74
R@1
0.75
R@1
0.72
0.70
0.73
0.68
0.72
0.66
10
50
0
10
20
Epoch
Epoch
(c) Stanford Dogs
(d) Stanford Cars
Figure 3: Performance in Recall@1 versus epoch number for fine-tuning on our eut loss over all runs. The blue dotted line indicates PA accuracy, i.e., the initial accuracy of the model. The accuracy drops at the beginning of training, bouncing back after a few epochs across different datascts, converging after 10-20 epochs and achieving higher performance compared to that of PA.
Comparison of different methods across various datasets w.r.t. various metrics Malts Mo. M and Ma. CATEGORIZER outperforms all other methods in terms of almost all metrics. (4) indicates the metric should have higher (lower) value.
Table 4:
Dataset
Method
Mell
Mo 1
M,
Mg 1
GM
12.13 1 1.10
13.52 1 1.51
37.28
1 1.51
10.5 1 1.34
MetaGCD
48.18 1 0.68
52.08
1.10
27.27
1.10
32.42 3.06
PA
56.52 0
64.32 0.88
14.39
11.39-00 0.88
25.77
= 3.12
CUB
IGCD
56.67 + 0.43
+
9.55 + 1.34
17.28 2.98
Happy
46.44 + 0.43
54.03 1.14
13:27
1.14
16.78 1.89
CATEGORIZER 61.50 + 0.46
68.48 = 0.97
11.72 + 0.97
33.62 + 3.05
CM
18.50 1.92
19.37
41.68 1.84
16.93
MctaGCD
40.45 1.93
43.40 2.04
31.18
2.04
29.29 2.02
PA
52.84 + 1.56
1.92
12.45 1.92
25.00
1.97
MIT
IGCD
44.01 + 1.75
50.27 +: 1.99
20.25
1.99
20.29
2.32
Happy
42.61 1 0.24
51.42 1 1.34
14.90 1 1.34
9.29 1 1.14
CATEGORIZEA
57.10
1.83
64.15
2.02
10.30
2.02
32.87
2.24
GM
11.35 1 1.52
13.42 1 1.81
44.61 1. 1.81
10.23 1 2.73
MctaGCD
54.35 + 1.23
54.9 + 1.45
29.48 +: 1.45
34.45 1 4.42
PA
66.23 + 1.15
74.23 + 1.95
10.69 1.95
34.69
4.62
Dogs
IGCD
33.63 0.93
1.38 39.15
40.11 1.38
11.52 2.34
13

---

Published in Transactions on Machine Learning Research (02/2026)
Dataset
Method
Mall
M,,
Mf
Ma 1
Happy
64.75 0.84
72.06 1.22
10.66 1.22
35.06 1.99
CATEGORIZER
68.46 + 1.05
76.42 = 1.79
6.98 + 1.79
37.09 + 4.50
GM
8.21 = 1.49
20.32 + 1.36
65.37 + 1.36
4.23 + 2.39
MetaGCD
9.55 = 1.33
23.86 + 1.73
49.36 + 1.73
8.47 + 2.62
CIFARIO
PA
64.39 + 1.37
79.07 + 1.89
17.14 1.89
5.65 t 2.68
TGCD
20.13 + 0.67
20.46 + 2.48
45.44 + 2.48
8.80 t 1.94
Happy
63.14 1.23
54.15 1.18
16.96 + 1.18
9.20 = 1.63
CATEGORIZER
73.02 + 1.35
87.90
1.54
3.88 + 1.54
11.73
2.32
GM
8.32 1.39
9.51 + 1.71
69.94 1.71
5.23
2.46
MetaGCD
34.14 1.47
35.60 1.85
49.36 1.85
28.30
2.71
CIFAR100
PA
61.30 1.19
74.45
6.46 1.89
8.70 3.63
IGCD
10.15 0.67
10.86
2.48
71.37 2.48
7.30 1.94
Happy
44.70 1.27
25.09 1.12
14.75 1.12
26.70 1.11
CATEGORIZER
64.01 + 1.35
76.95
1.54
4.51 + 1.63
14.72 + 2.51
GM
14.82 1.42
17.45 + 1.21
46.87 + 1.21
11.23 2.35
MetaGCD
45.65 + 1.36
45.65 + 1.74
26.35 + 1.74
39.28 + 2.63
AIRCRAFT
PA
37.62 + 1.15
42.46 + 1.76
18.34 + 1.76
18.29 3.21
IGCD
42.11 0.57
47.62
1 2.56
29.35 1 2.56
20.09 + 1.73
Happy
24.36 0.67
28.39
1.13
15.98 + 1.13
8,25 + 1.94
CATEGORIZER
45.24 1.29
49.74 = 1.38
13.46 = 1.38
27.29
2.57
GM
10.81 1.31
16.32 1.32
46.87 1.32
10.23
1.34
MetaGCD
20.76
19.11
28.31 1.39
25.20
1.95
SVHN
PA
73.28 1.27
82.66
13.88
45.23
IGCD
11.79 1.32
1.36
1.36
19.52
2.21
Happy
20.13 0.67
20.46 - 2.48
45.44
2.48
8.80 + 1.94
CATEGORIZER
75.21 + 1.39
84.32
1.24
11.46 + 1.24
52.01 + 2.31
GM
58.72 + 1.24
60.27 + 1.21
35.23 + 1.41
40.27 2.11
MetaGCD
73.20 + 1.03
76.90 + 1.15
18:24 + 1.24
45.14 + 1.96
mnist
PA
77.48 1 1.27
85.65 1 1.25
13.47 1 1.25
45.32 1 2.17
IGCD
74.12 1 1.45
83.43 1 1.28
15.92 1 1.28
36.88 1 2.36
Happy
60.99 + 1.46
65.43 1.22
20.62
1.22
43.33
2.73
CATEGORIZER
79.90 + 1.29
87.82 + 1.19
11.23 + 1.19
49.15 + 2.28
GM
28.72 1.32
32.16 1.11
66.24 1.11
4.17 + 2.22
MetaGCD
52.76 1.27
61.91 1.14
35.69
25.16 1.86
Rashion-mutt munst
PA
50.13 1.18
1.22
1.22
7.50
IGCD
48.29
57.51
1.17
40.34 1.17
11.40 1 2.21
Happy
51.78 + 1.37
58.86 +: 1.98
27.35 + 1.98
8.80 + 1.94
CATRGORIZER
61.02 + 1.29
65.82
1.13
26.84 + 1.13
27.87 + 2.17
GM
23.52 + 2.32
28.24 + 2.67
43.26 + 2.67
17.23 + 1.96
MetaGCD
47.17 1 2.23
52.99 1 2.34
23.69
1 2.34
24.71 1 4.53
PA
40.46 1 2.09
44.35 1 2.55
24.30
1 2.55
25.44
1 3.62
Cars
IGCD
48.60 +: 1.88
57.82
1 2.57
19.33
1 2.57
12.65 62.63
2.75
Happy
15.06 + 0.38
16.54
+ 1.27
19.17
1.27
9.38
+ 1.28
CATEGORIZER
52.97 1: 2.13
59.34
2.46
17.75
2.46
33.29
1 4.68
5.3 Comparison with State-of-the-ar Methods
We conducted scries of experiments to evaluate CATEGORIZER against state-of-tho-art (SOTA) ap- proaches in the CGCD setting. For comparisons, we used the IGCD Zhao & Mac Aodhal (2023), GM Zhang et al. 2022), PA Kim et al. (2023). MetaGCD Wu et al. (022)). and Happy Ma et al. (2024), which are very recent works and operate the same context. Following the experimental sctup from Kim al. (2023). we used 80% of the classes in the initial stage and introduced the remaining 20% in the continual learning stage. To better reflect world scenarios, where incoming data may include both known and novel classes, only
14

---

Published in Transactions on Machine Learning Research (02/2026)
80
90
80
70
60
50
40
30
20
20
10
10
Methods
CATEGORIZER
Happy
60
MetaGCD
PA
Mr
30
20
10-
CU8
MIT
Dogs
OFAR10
CHAR100
ARCRAFT
5WAW
moist
Fashon-nnist
Cara
CUB
MIT
Dogs
CHAR1O
CRAR100
AIRRAFT
SWW
miit
Fashion-maistt
Cars
Figure Performance comparison of different methods across multiple datasets using four metries: Matts Mo. My, and Ma. Each subplot corresponds to one metrie, where the x-axis represents datasets and the y-axis indicates the metric score. Lines represent the average performance of each method, with shaded regions showing the standard deviation across runs. CATEGORIZER consistently achieves top performance across most metrics and datasets.
80% of the data was utilized during the initial stage, while the remaining 20% of samples from known classes wore mixed with novel samples in the continual learning stage.
For our experiments, we used fine-grained datasets that closely resemble real-world scenarios and some
general datasets: birds Wah et al. (2011), indoor scenes Quattoni & Torralba (2009). cars Krause et al.)
(2013). dogs Khosla al. (2011), CIFAR10 Krizhevsky al. (2009), CIFAR100 Krizhevsky et al. (2009).
SVHN Netzer et al. [0011 mnist Deng (012)), Fashion-mnist Xiao et al. (2017) and aircraft Maji et al.
(2013). According to the experimental setup, the initial and continual stage class splits were as follows:
birds (160/40), cars (156/40), indoor scenes (53/14), dogs (96/24), CIFAR10 (8/2), CIFAR100 (80/20), SVHN (8/2), MNIST (8/2), Fashion-mnist (8/2), and aircraft (80/20). is important to highlight that our proposed framework not limited to a specific data type, such as images. It can be applied to any type of data, as long as a suitable backbone network employed for fcature extraction. However, since prior approaches in the CGCD setting have utilized image datasets for their results, we opted to use them as well.
In addition to accuracy, evaluated the recall at K metric for more fine- fine-grained datasets such CUB (since the difference in performance on other datasots is negligible), to mcasure the learned representation of different methods in the initial stage as it is essential for novelty delection and overall performance. Results for recall at K (K 1, 2, 4, and 8) are reported in Table Our proposed eut loss significantly improves the performance, particularly on the cars dataset, compared to plain PA as shown in Figure 3 which reports the Recall@1 performance versus epoch number. Overall, our proposed approach surpasses other methods in terms of representation learning.
15

---

Published in Transactions on Machine Learning Research (02/2026)
0.7
Clustering accuracy
Clustering accuracy
0.6
0.6
M-all
0.5
M-all M-old
0.5
M-old
0.4
M-new
0.4
M-new
0.3
0.2
10
15
20
25
30
5
10
20
25
Epoch
Epoch
(a) CUB-200
(b) MIT67
0.8
0.6
Clustering accuracy
0.7
Clustering accuracy
M-all M-old
M-all
0.5
0.4
M-old
M-new
M-new
0.4
0.3
0.2
0.2
10
15
20
25
30
10
20
25
Epoch
Epoch
(c) Stanford Dogs
(d) Stanford Cars
Figure 5: Clustering accuracy versus epoch number the continual learning stage over all runs. In the Dogs and Cars datasets, training for longer epochs has led to feature collapse of newly discovered classes, while in CUB and MIT datasets, the accuracy has not changed after certain epochs. Based this observation we limit the nunber of epochs of the model training in the continual learning stage.
Table 4 compares different methods during the continual learning stage. Figure Jillustral the comparative performance of state-of-the-art methods and CATEGORIZER across multiple datasets. CATEGORIZER consistently achieves superior or comparable results compared to other methods across different datasets and evaluation metrics, including novel class discovery (Ma), retention of prior knowledge (M, and M2). and overall performance (Mau). On some datasets like birds, IGCD exhibited better forgetting performance, likely because its initial accuracy due to the fact that it requires the ratio of novel category samples on the new dataset. MetaGCD showed good performance in terms of the Mj metric, demonstrating its ability to learn novel classes effectively, but struggled to retain performance on previously known classes. PA performed well on most dalasets, especially dogs, but showed weaker results for the cars dataset, which can be explained by its relatively poor learned representation, as reported in Table 3 Happy shows strong performance on the Ma metric for the CIFAR100 and Dogs datasets, but does not exhibit notably better results across the other metrics.
lower than ours. GM performs poorly compared to others, which is
Table Ablation study on the effect of eut loss and model reduction (RED).
Levt
Mat
Mo 7
Mr
Ma
X
57.03
65.65
12.70
23.00
CUB
X
59.44
66.05
14.15
33.36
X
59.39
67.78
10.82
26.28
61.50
68.48
11.72
33.62
X
50.93
58.74
13.52
MIT
X
55.82
63.87
10.94
25.71
55.72
64.09
10.72
25.00
16

---

Published in Transactions on Machine Learning Research (02/2026)
Leot
Red
Mall
M. 1
Myy 1
Ma
57.10
64.15
10.30
32.87
62.35
73.12
11.68
23.81
63.36
72.07
12.06
29.05
Dogs
66.40
77.89
6.75
22.09
68.46
76.42
6.98
37.09
X
60.15
76,47
16.31
20.04
66.28
80.35
11.43
24.64
CIFARIO
69.10
83.00
8.78
27.85
73.02
87.90
3.88
11.73
x
55.03
55.03
14.12
18.64
60.00
60.00
9.15
22.08
CIFARI00
62.40
62.40
6.75
24.12
64.01
76,95
4.51
14.72
41.92
47.65
15.55
20.45
44.32
49.90
13.30
22.78
AIRCRAFT
43.00
47.23
15.97
21.36
45.24
49.74
13.46
27.29
x
68.10
72.85
11
33.25
72.10
77.50
18.46
36.25
SVHN
73.40
80:20
15.76
38.10
75.21
84.32
11.64
52.01
60.50
71.10
27.95
35.50
70.50
80.25
18.80
40.20
MNIST
74.15
82.95
16.10
79.90
87.82
11.23
55.80
61.80
30.86
20.00
56.90
62.90
29.76
21.80
FASHION-MNIST
58.50
63.50
29.16
23.50
61.02
65.82
26.84
27.87
45.92
52.37
24, 72
21.05
47.72
52.80
24.29
29.06
Cans
48.46
55.50
21.59
22.18
52.97
59.34
17.75
33.29
Table 6: Effect of the model reduction on the estimated number of categories. With the model reduction, the estimations are significantly closer to the actual number of categories
# OF CATEGORIES w/o REDUCTION
REDUCTION
CUB
200
285
231
MIT
67
134
81
DOGS
120
141
CIFAR10
10
120
37
CIFAR100
100
230
120
AIRCRAFT
100
110
19
SVHN
10
180
21
MNIST
10
175
20
FASHION-MNIST
10
185
24
CARS
196
354
222
5.4 Ablation Study
We study the effect of our eut loss and cluster reduction on the performance of the proposed method, which reported in Table
17

---

Published in Transactions on Machine Learning Research (02/2026)
Effectiveness of the eut loss: As reported in Table the proposed eut loss enhances performance in terms of most metrics, with particularly notable improvement in terms of Ma. highlighting its significance for the novelty detection module. However, on certain datasets, such as Stanford Dogs, the Mo metric shows reduced performance when using this loss, in exchange for a higher Ma. This indicates a suboptimal trade- off between plasticity and stability, which is mitigated by integrating the ert. loss with model-reduction techniques. Additionally, the method exhibits increased forgetting on some datasets due to the eut loss boosting the initial accuracy, making subsequent accuracy drops more pronounced.
15
-10-
-10
-15
(a) MIT67 Before reduction
(b) MNIST Before reduction
(c) SVHN Before reduction
90
60
40
100
20
30
-20
-100
130
-40
-70
20
40
1000
-75
-50
-25
50
75
-700
-150
-100
-50
100
150
(d) MTT67 After reduction
(e) MNIST After reduction
(f) SVHN After reduction
Figure Distribution of proxies the feature space before and after reduction on MIT67, MNIST, and SVHN. The reduction procedure eliminates rodundant proxies, yiclding a closer approximation of the true number of categories, which enhances accuracy and reduces prediction time.
Table Comparison of novelty detection module of CATEGORIZER and PA. The results show improvement across all the datasets.
PA
CATEGORIZER
CUB
59.77
70.57
MIT
60.30
67.20
DOGS
65.26
68.74
CIFAR10
63.20
70.00
CIFAR 100
66.10
69.10
AIRCRAFT
65.23
70.20
SVHN
63.86
69.50
MNIST
62.11
68.30
FASHION-
65.32
70.10
CARS
69.10
71.66
Effecctivnness of model reduction: Per Table
5 model reduction introduced in Section could improve the proposed method in terms of most metrics, particularly the M,, metric. We believe this improvement is
18

---

Published in Transactions on Machine Learning Research (02/2026)
due to the fact that having redundant proxies for the novel categories significantly degrades the performance of old categories. Removing these redundant proxies helps to maintain the model's stability. The combination of the eut loss and model reduction has achieved the most optimal trade-off between maintaining accuracy on the old categories (Mo) and effective discovery and learning novel categories (Ma). Table 6 shows the effect of the model reduction module on the estimated number of categories the discovery phase. Upon performing
model reduction, the estimated number of categories significantly closer to the real number of categories.
Figure 6 shows the removal of redundant proxies in the feature space, visualized with t-SNE Maaten & Hinton (2008), resulting in distribution that more accurately reflects the true number of categories.
Effectiveness on the novelty detection module: Table 7 shows the improvement in the accuracy of the novelty detection module of CATEGORIZER compared to the original PA method Kim et al. (2023) CATEGORIZER has consistently improved the novelty detection accuracy method across different datasets.
Table 8: Performance comparison using Gaussian and WEIBULL distributions across datasets.
DATASET
DISTRIBUTION
Mau
Motd
M;
Ma
DOGS
GAUSSIAN
68.10
76.24
7.89
36.01
WEIBULL
68.46
76.42
6.98
37.09
CUB
GAUSSIAN
60.82
67.72
12.48
33.62
WEIBULL
61.50
68.48
11.72
33.61
CARS
GAUSSIAN
51.52
58.50
18.59
32.15
WEIBULL
52.97
59.34
17.75
33.29
AIR
GAUSSIAN
43.80
48.42
14.78
25.34
WEIBULL
45.24
49.74
13.46
27.29
Mrr
GAUSSIAN
56.42
63.11
11.70
32.86
WEIBULL
57.10
64.15
10.30
32.87
CIFAR10
GAUSSIAN
71.13
86.18
5.60
10.95
WEIBULL
73.02
87.90
3.88
11.73
SVHN
GAUSSIAN
73.86
82.16
13.80
51.42
WEIBULL
75.21
84.32
11.64
52.01
CIFAR100
GAUSSIAN
63.51
75.82
5.64
14.25
WEIBULL
64.01
76.95
4.51
14.72
MNIST
GAUSSIAN
79.40
87.28
11.77
48.63
WEIBULL
79.90
87.82
11.23
49.15
PASHION-MNIST
GAUSSIAN
59.23
64.66
28.00
27.50
WEIBULL
61.02
65.82
26.84
27.87
Effectiveness of EVT-based feature sampling: distribution from EVT and using a Gaussian distribution (used in Kim al.] (2023)) in the feature replay phase. Employing the Weibull distribution leads to improved performance across all metrics and for all datasets.
a comparison between using the Weibull
6 Conclusion
In this paper, we proposed novel method, called CATEGORIZER, to handle the continual generalized category discovery problem. CATEGORIZER combines proxy anchors and extreme value theory to define decision boundaries around each proxy. We proposed a novel loss, called the erl loss which, enhances the learned representation compared to the plain proxy anchors and outperforms deep metric learning loss used in similar SOTA methods in the CGCD scenario. In addition, we demonstratod that employing a Weibull distribution instead of Gaussian distribution in the feature-replay phase consistently leads improved performance. Furthermore, mitigated the problem of over-estimating the number of novel categories in
19

---

Published in Transactions on Machine Learning Research (02/2026)
the discovery phase by resorting to novel method, which is based on EVT. CATEGORIZER outperforms state-of-the-art methods in the CGCD) scenario across different datasets. In future work, we plan to integrate the eut loss during the continual learning stage of the framework. In addition, we will investigate other clustoring methods to be used in the discovery step.
References
Davide Abati, Angelo Porrello, Simone Calderara, and Rita Cuechiara. Latent space antoregression for novelty detection. In Proceedings of the IEEE/CVP conference on computer vision and pattern recognition, pp. 481 490, 2019.
Mihael Ankerst, Markus M Breunig, Hans-Peter Kriegel, and Jörg Sander. Optics: Ordering points to identify the clustering structure. ACM Sigmod record, 28(2):49-60 1999.
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1563-1572, 2016.
Hakan Cevikalp. Best fitting hyperplanes for classification. IEEE transactions on pattern analysis and machine intelligence, 39(6):1076 1088, 2016.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint orXiv:1812.00/ 2018.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint trxiv:1902. 10486, 2019.
Guangyao Chen, Peixi Peng, Xiangqian Wang, and Yonghong Tian. Adversarial reciprocal points learning for open set recognition. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(11):8065-80) 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive Icarning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Stuart Coles, Joanna Bawa, Lesley Tremner, and Pat Dorazio. An introduction to slatistical modeling of eatreme values, volume 208. Springer, 2001.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 1914)114 2012.
Taylor Denouden, Rick Salay, Krzysztof Czarnecki, Vahdat Abdelzad, Buu Phan, and Sachin Vernekar. Improving reconstruction autoencoder out-of-distribution detection with mahalanobis distance. arXiv preprint arXie:1812.0221 2018.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Computer rision-ECCV 2020: 16th European conference, Glasgow, UK, August 23-28, 2020, proceedings, part XX 16, pp. 86-102. Springer, 2020.
Xueleng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. Xixi preprint 22v: 0197,,
197, 2022.
Martin Ester, Hans- Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A Kensity-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pp. 226 231, 1996.
Enrico Fini, Enver Sangincto, Stéphanc Lathuilière, Zhun Zhong, Moin Nabi, and Elisa Ricci. A unificd objective for novel class discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9284 0284-9292, 2021.
20

---

Published in Transactions on Machine Learning Research (02/2026)
Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMT.R, 2017.
Brendan J Frey and Delbert Dueck. Clustering by passing messages between data points. science, 315(5814): 972-976, 2007.
YongYuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi. Generative openmax for multi-class open set classification. arXiv preprint arXiv:1707.0744 2017.
Chuanxing Geng, Sheng-jun Huang, and Songcan Chen. Recent advances in open set recognition: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(10):3614 1-3366111363,,,, 2020.
Abhishek Gupta, Russell Mendonca, YuXuan Liu, Picter Abbeel, and Sergey Levine. Meta-reinforcement learning of structured exploration strategies. Advances in newral information processing systems, 31, 2018.
Michael Gutmann and Aapo Hyvärinen. Toise-contrastive estimation: A new estimation principle for unnor- malized statistical models. In Proceedliggs of the thirleenth international conference on artificial intelligence and statistics, pp. 297 304. JMLR Workshop and Conference Proceedings, 2010.
Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning to discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE/CVF International Conference Computer Viston, pp. 8401-8409, 2019.
Kai Han, Seivestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Autonovel: Automatically discovering and learning novel visual categorics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):6767 0781,, 2021.
Shaozhe Hao, Kai Han, and Kwan- Yee K. Wong. CiPR: An efficient framework with cross-instance positive relations for generalized category discovery. Transactions Machine Learning Research, 2024. ISSN 2835-8856. URL http::///ppaneeeee.eeee///rrttta net::/orrmmm::::1ffcccpp...
Mchadi Hassen and Philip K Chan. Learning a neural-network- based representation for open set recognition. In Proceedings of the 2020 STAM International Conference on Dala Mining, pp. 154-162 SIAM, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In
Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. XrXio preprind 1i1.::6100.02. 2016.
Dan Hendryeks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure.
arXiv preprint Xrxi 2118811.000 04666 2018.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. arXiv preprint 1r111.1911.11199, 2019.
Saihni Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incre- mentally via rebalancing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 831 839, 2019.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out- -itt---distribution image without learning from out-of-distrribttio data. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 10951-10960, 2020.
Zhiyuan Hu, Yunsheng Li, Jiancheng Lyu, Dashan Gao, and Nuno Vasconeelos. Dense network expansion for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11858 11867, 2023.
21

---

Published in Transactions on Machine Learning Research (02/2026)
Dihong Jiang, Sun Sun, and Yaoliang Yu. Revisiting flow generative models for out-of-distribution detection. In International Conference on Learning Represenlations, 2021.
Wenyu Jiang, Yuxin Ge, Hao Cheng, Mingcai Chen, Shuai Feng, and Chongjun Wang. Read: Aggregating reconstruction error into out-of-distribution detection. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 37, pp. 14910 14918, 2023.
KJ Joseph, Sujoy Paul, Gaurav Aggarwal, Soma Biswas, Piyush Rai, Kai Han, and Vineeth N Balasubrama- nian. Novel class discovery without. lorgetting. In Europewn Conference on Compuler Vision, pp. 570 586. Springer, 2022.
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine-graines visual categorization (FGVC), volume 2, 2011.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:18661 18673, 2020.
Hyungmin Kim, Sungho Suh, Daehwan Kim, Daun Jeong, Hansang Cho, and Junmo Kim. Proxy anchor-based unsupervised learning for continuous generalized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16688-16697, 2023.
Jang-Hyun Kim, Sangdoo Yun, and Hyun Oh Song. Neural relation graph: A unified framework for identi- fying label noise and outlier data, Advances in Neural Information Processing Systems, 36, 2024.
Sungyeon Kim, Dongwon Kim, Minsu Cho, and Suha Kwak. Proxy anchor loss for deep metric learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3238 3247, 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaune Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Bareska-aawiikka,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3522 (5213)3:52133333 2017.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained catego- rization. In Proceedings of the IEEE international conference computer vision workshops, pp. 554-561, 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2 (1-2):83 97, 1955.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrateed classifiers for detecting distof-distiibttion samples. arXiv preprint xrXiv:1111.09325 2017a
Kimin Lee, Kibok Lee, Honglak Leo, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Sang-Wo Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Wor Woo Ha, and Byoung-The Tak Zhang. Overcoming calastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017b.
Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual struc- ture lcarning framcwork for overcoming catastrophic forgetting. In International conference on machine learning, pp. 3925 3934. PMLR, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935 2442299997977 2017.
22

---

Published in Transactions on Machine Learning Research (02/2026)
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. xiviv preprint orXio: 1706. 106.02690, 2017.
Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, and Yumming Ye. Per: Proxy-based contrastive replay for online class-incremental continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognátion, pp. 14246-242255 2023.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 88:21464-21477 2020.
David Lopez-Pas Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Shijie Ma, Fei Zhu, Zhun Zhong, Wenzhuo Liu, Ya- Zhang, and Cheng-Lin Liu. Happy: A debiased lcarning framework for continual generalized category discovery. arXiv preprint arXiv:2410006338 2024.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-snc. Journal of machine learning research, 9(Nov):2579 2605, 2008.
S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013.
Leland McInnes, John Healy, Steve Astels, et al. hdbscan: Hierarchical density based clustering. J. Open Source Softw., 2(11):205, 2017.
Yifci Ming, Yiyou Sun, Dia Ousmane, and Yixuan Li. Cider: Exploiting hyperspherical embeddings for out-of-distribution detection, 03 2022.
Lawrence Neal, Malthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. Open set learning with counterfactual images. In Proceedings of the European conference on computer vision (ECCV), pp. 613- 628, 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop deep learning and unsupervised feature learning, volume 2011, pp. 4. Granada, 2011.
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 11321 11329, 2019.
Poojan Oza and Vishal M Patel. C2ac: Class conditioned auto-cncodor for open-set recognition. In Proceed- ings of the BEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2307-2316, 2019.
Ariadna Quattoni and Autonio Torralba. Recognizing indoor scenes. In 2009 IEEE conference on compuler vision and pattern recognition, pp. 413 420. IEEE, 2009.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual lcarning. Advances in neural information processing systems, 32, 2019.
Ethan M Rudd, Lalit P Jain, Walter Scheirer, and Terrance Boult. The extreme value machine. IEEE tronsactions on pottern analysis and machine intelligence, 40(3):762-768, 2017.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXio preprint urXiv:1606.04p00 2016.
Gobinda Saha and Kaushik Roy. Amphibian: A meta-learning framework for rehearsal-free, fast online continual learning. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https:
23

---

Published in Transactions on Machine Learning Research (02/2026)
Matthew D Scherreik and Brian D Rigling. Open set recognition for automatic target classification with
rejection. TEEE Transactions on Acrospace and Electronic Systems, 52(():632-66) 642, 2016.
Lei Shu, Hu Xu, and Bing Liu. Doc: arXiv:1709. 08716, 2017.
Deep open classification of text documents. arxio preprint
Yu Shu, Yemin Shi, Yaowei Wang, Tiejun Huang, and Yonghong Tian. P-odn: poototype-based open deep network for open set recognition. Scientific reports, 10(1):7146, 2020.
Petr Slavik. A tight analysis of the greedy algorithm for set cover. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pp- 435 0355441, 1996.
Viyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision, pp. 691 708. Springer, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. Advances in Newral Information Processing Systems, 34:144-157, 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Oittof-disti detection with deep nearest neigh- PMLR, 2022.
bors. In International Conference on Machine Learning, pp. https://gthubb.ccm/asalaaaaaa 2024.
Vastlab. Vast: A versatile toolkit. for deep learning research.
Accessed: 2024-12-09.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7492-7501, 2022.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongie. The caltech-uesd birds- 200-2011 dataset. 2011.
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distributtin with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4921 4930, 2022.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Joe H Ward Jr. Hierarchical grouping to optimize objective function. Journal of the American statistical association, (((301):236-244) 1963.
Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, and Giovanni Zappella. Continual learning with low rank adaptation. arXiv preprint wxXiv:231.11700, 2023.
Chenshen WU, Luis Herranz, Xialei Liu, Joost Van De Weijer, Bogdan Raducanu, et al. Memory replay gans: Learning to generate new categories without forgetting. Advances in neural information processing systems, 31, 2018.
Yanan Wu, Zhixiang Chi, Yang Wang, and Songhe Feng. Metaged: Learning to continually learn in general- ized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1655- 1665, 2023.
Ziyang Wu, Christina Baek, Chong You, and Yi Ma. Incremental learning via rate reduction. Tn Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1125 1133, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint orXiv:1708.07/0 2017.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. International conference on machine learning, pp. 478-487. PMTR, 2016.
24

---

Published in Transactions on Machine Learning Research (02/2026)
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribation detection: A survey. International Journal of Computer Vision, 132(11))566355000) 2024.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expand- able networks. arXiv preprint arXiv:170.. 150.0011001 2017.
Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto Iida, and Takeshi Naemura. Classification- reconstruction learning for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4016-4025, 2019.
Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribution detection by maximum classifier discrep- ancy. In Proceedings of the IEEE/CVF international conference computer vision, pp. 9518-9526, 2019.
Friodemann Zenke, Ben Poole, and Surya Ganguli. Continual lcarning through synaptic intelligence. In
International conference on machine learning, pp. 3987 3995. PMLR, 2017.
Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. Hybrid models for open set recognition. In Computer Vision ECCV 2020: 16th European Conference, Glasgou, UK, August 23 28, 2020, Proceedings, Part III 16, pp. 102-117. Springer, 2020.
Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustcring method for very large databases. ACM sigmod record, 25(2):103 114, 1996.
Xinwei Zhang, Jianwen Jiang, Yutong Feng, Zhi-Fan Wu, Xibin Zhao, Hai Wan, Mingqian Tang, Rong Jin, and Yue Gao. Grow and merge: A unified framework for continuous categories discovery. Advances in Newral Information Processing Systems, 0127455-274666 2022.
Bingchen Zhao and Kai Han. Novel visual category discovery with dual ranking statistics and mutual knowledge distillation. Advances in Neural Information Processing Systems, 34:22982 22994, 2021.
Bingchen Zhao and Oisin Mac Aodha. Incremental generalized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19137 19147, 2023.
Bingchen Zhao, Xin Wen, and Kai Han. Learning superviee gaussian mixture models for generalized category discovery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16623 16633, 2023a.
Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticis feature space decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023b.
Guangyao Zheng, Samson Zhou, Vladimir Braverman, Michael A Jacobs, and Vishwa Sanjay Parckh. Sc- lective experience replay compression using coresets for lifelong deep reinforcement learning medical imaging. In Medical Imaging with Deep Learning, pp. 1751-1764 PMTR, 2024.
Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, and Nicu Sebe. Neighborhood con- trastive learning for novel class discovery. In Proceedings of the IEEE/CW conference on computer vision and pottern recognition, pp. 10867-10875, 2021.
Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Learning placeholders for open-set recognition. In Proceed- ings of the IEEE/CVF conference computer vision and pattern recognition, pp. 4401-4410, 2021.
Ev Zisselman and Aviv Tamar. Deep residual flow for out of distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 14004,, 2020.
25

---

Published in Transactions on Machine Learning Research (02/2026)
A
Appendix
Table 9 presents the number of categorics estimated by different clustering methods. Our evaluation across multiple clustering algorithms confirms that the overestimation issue persists regardless of the method used. Affinity Propagation Frey & Dueck (2007) and Hierarchical Clustering Ward Jr 1963) exhibit similar and stable behavior, whereas Birch Zhang et al. 1996) consistently overestimates the number of clusters. contrast, density-based methods (DBSCAN Ester al. (1996). HDBSCAN McInnes al. (2017), and OPTICS Ankerst et. al. (1999)) highly sensitive to hyperparameters such min samples, leading to unstable results and rendering them unsuitable for CGCD's continually changing environment.
Table 9: Comparison of the number of clusters estimated by different clustering methods across datasets.
DATASET
ACTUAL
AFFINITY
BIRCH
HIERARCHICAL
DOGS
120
229
437
229
CUB
200
285
496
285
CARS
196
354
250
355
AIR
100
110
115
110
Mrr
67
134
140
134
C1PAR10
10
120
236
120
SVHN
10
180
210
179
CIFAR100
100
230
811
232
MNIST
175
130
175
FASHIO--MNIST
10
185
232
185
26


Reviews:
[{"review_id": "h5M6RuVjxl", "reviewer": "Reviewer_zvi6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript introduces an extreme value theory (EVT)-driven proxy anchoring and reduction framework for continual generalized category discovery. While the proposed approach is novel within this domain and attempts to mitigate over-clustering and cluster drift through EVT-based rejection and proxy reduction mechanisms, the manuscript in its current form suffers from several issues that significantly undermine its clarity, theoretical soundness, and experimental reliability.", "claims_and_evidence_yn": "No", "claims_explanation": "The theoretical foundation of the EVT-based proxy reduction is insufficiently developed. The paper introduces several functions to define coverage relationships between proxies and then formulates a set-cover-like optimization objective. However, no mathematical derivation is provided to support how EVT-based quantities naturally induce these coverage relations, nor is there an explanation of why the proposed thresholding operation leads to a grounded reduction mechanism.", "audience_yn": "No", "audience_explanation": "The major issue concerns the clarity and organization of the framework. The overall methodology is presented in an overly complex manner, where the main system diagram contains many dense arrows, repeated EVT modules, and several intermediate states whose roles are never explicitly explained in the text. Important components in the figure do not correspond clearly to algorithmic steps or mathematical formulations in the paper, making the methodology unnecessarily difficult to follow and leaving the reader uncertain about the precise flow of operations.", "requested_changes": "1. The manuscript mentions several prior works such as GM, IGCD, MetaGCD, and Proxy Anchor, but it does not provide a meaningful comparative analysis to clarify how the proposed method differs from or improves upon these closely related approaches. The authors are advised to provide further elaboration on this aspect.\n2. The experimental evaluation section also contains several weaknesses. Some key claims, such as the effectiveness of the EVT loss or the benefits of proxy reduction, are supported mainly by qualitative visualizations rather than quantitative evidence. Certain performance degradations are attributed to vague “trade-offs” without deeper analysis. Hyperparameters that appear to have significant effect on the results are described only briefly and without sensitivity studies, limiting reproducibility.", "broader_impact_concerns": "N/A", "additional_comments": "Several writing issues further reduce the readability of the paper. Important variables and symbols are introduced without prior definition in Section 4. A detailed examination of potential errors is required to enhance the accuracy and quality of the paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "SvuaCrFfJ3", "reviewer": "Reviewer_GL8n", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Summary\nThe paper introduces \"CATEGORIZER\", a framework designed for Continual Generalized Category Discovery (CGCD) that integrates Extreme Value Theory (EVT) with Proxy Anchor learning. By leveraging Weibull distributions to model the decision boundaries of proxies, this paper proposes a novel evt loss to enhance representation quality and a Probability of Inclusion function for effective novelty detection. To address the over-clustering issue inherent in Affinity Propagation during the discovery phase, an EVT-driven Model Reduction strategy is employed to merge redundant proxies via greedy set cover optimization. Experiments on benchmarks like CUB-200 demonstrate that the proposed method achieves superior clustering accuracy and more accurate category count estimation compared to state-of-the-art methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper has the following three Strengths:\n1.The paper proposes an innovative approach by incorporating Extreme Value Theory directly into the training objective via the evt loss. This effectively shapes the embedding space to better fit Weibull distributions, improving the separation between known and unknown categories during the initial stage.\n2.This paper presents a comprehensive framework that systematically addresses the interconnected sub-tasks of novelty detection, category discovery, and continual learning. This design explicitly targets the challenge of class count over-estimation, providing a unified solution for the CGCD problem.\n3.The Model Reduction module offers a structured mechanism based on Set Cover optimization to refine the granularity of discovered concepts. This effectively mitigates the over-clustering limitations of affinity-based methods, as demonstrated by the improved proxy distributions in the visualization.", "audience_yn": "Yes", "audience_explanation": "The paper proposes a novel Continual Learning Method for Generalized Category Discovery which may inspire the community.", "requested_changes": "The following concerns may impact the final decision of this paper.\n1. The proposed evt loss (Eq. 3) depends on Weibull parameters ($\\lambda_p, \\kappa_p$) derived from the network's feature distribution, but Algorithm 1 indicates these are estimated only once in Step 2 and treated as fixed constants throughout the fine-tuning loop. This results in the model being optimized against outdated statistical boundaries that do not reflect the evolving feature space. The paper lacks theoretical proof or experimental evidence (e.g., loss curves) to demonstrate the convergence of this specific scheme. Furthermore, applying this evt loss in the continual learning stage would likely amplify these optimization inconsistencies.\n2. In Section 4.2.2, the paper employs Affinity Propagation for novel class discovery, explicitly noting that it results in over-clustering. To mitigate this, a Model Reduction module is introduced. However, the paper does not include comparisons with other robust non-parametric clustering algorithms that might inherently avoid such over-segmentation. Without these comparisons, it is difficult to assess whether the proposed reduction module is essential or if the issue could be addressed by employing a different clustering method.\n3. Although the Class Incremental Learning stage is a pivotal component of the framework, it lacks technical innovation by relying on standard feature distillation and replay techniques without modification. Furthermore, the paper fails to verify the actual effectiveness of these adopted components.\n4. Table 6 shows that the estimated category counts still deviate notably from the ground truth. For example, on the Aircraft dataset, the method estimates only 19 categories compared to the actual 100, indicating that the reduction strategy fails to accurately approximate the true class distribution.", "broader_impact_concerns": "No further broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0dfNOZjk4S", "reviewer": "Reviewer_ZHgS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. The paper addresses an important and challenging problem in machine learning.\n2. It introduces a new approach, CATEGORIZER, which achieves superior performance compared to existing state-of-the-art (SOTA) methods.\n3. The proposed method effectively mitigates the tendency to overestimate the number of novel categories during the discovery phase.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The problem setting in the paper is well formulated and the proposed method is well illustrated.\n2. Despite that introducing EVT or Proxy loss into this Continual GCD setting is not entirely new but the pipeline is quite intuitive.\n3. The evaluation is also abundant and convincing from my perspective.", "audience_yn": "Yes", "audience_explanation": "I believe that, given the challenging and important problem setting, along with the clear and comprehensive methodology and evaluations, this work should be valuable to at least a portion of TMLR’s audience.", "requested_changes": "The overall writing is clear and easy to understand. Some captions and characters of Figure 1 is not clear enough.", "broader_impact_concerns": "I think the main limitation of this paper is that, although Continual GCD is an important setting, the specific formulation presented in Section 3 is rather restrictive. In real-world scenarios, it is often unrealistic to assume that the model first encounters labeled data and then unlabeled data sequentially, making this setup somewhat limited in practical applicability.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)