Hard-Negative Prototype-Based Regularization for Few-Shot Class-Incremental Learning
Authors: Seongbeom Park
Hyunju Yun
Daewon Chae
Sungyoon Kim
Suhong Moon
Minwoo Kang
Seunghyun Park
Jinkyu Kim
Venue: Accepted by TMLR ()

Abstract: Few-shot class-incremental learning (FSCIL)---involving abundant base training data followed by novel classes with limited labeled samples---poses challenges such as catastrophic forgetting and overfitting, leading to significant performance degradation across incremental sessions. As a remedy, recent work focuses on minimizing the interference of embeddings between base and incremental classes. However, previous studies have not explicitly considered variation in discriminative difficulty across samples and classes, leaving room for improvement: we observe that hard-negative (i.e., difficult to discriminate from the label) samples and classes significantly affect FSCIL performance, whereas easy ones have little impact. To this end, we propose a hard-negative prototype-based regularization approach that enhances discrimination between similar classes by imposing a penalty margin between each sample and its most similar class prototypes based on cosine similarity. To select hard-negative prototypes, we explore two distinct mining strategies: dynamic selection that leverages the model's decision boundary, and static selection that utilizes a pre-defined class-wise similarity matrix derived from external sources such as pre-trained models. We evaluate our approach on three widely used benchmarks, miniImageNet, CIFAR100, and CUB200, achieving state-of-the-art performance on each. Comprehensive analyses demonstrate that our proposed method enhances intra-class cohesion and inter-class separability of embeddings, both of which are crucial for FSCIL to better accommodate novel classes.

Published in Transactions on Machine Learning Research (11/2025)
Hard-Negative Prototype-Based Regularization for Few-Shot Class-Incremental Learning
Seongbeom Park', Hyunju Yun', Daewon Chae², Sungyoon Kim, Suhong Moon', Minwoo Kang', Seunghyun Park" Jinkyu Kim
Korea Universiby 2 UUiverrsiyy of Michigan, Ann Arbor 3 UUniversity of California, Berkeley SSoongsl University 'Corresponding Authors. Email: sh. sh. parkossk kr and jinkyukimkkoreo.
Reviewed on OpenReview: https: opwp:e/eepprvvee....mmmmm net/ forun? id= dd=2kn000DDa
Abstract
Few-shot class-incremental learning (FSCIL) involving abundant base training data fol- lowed by novel classes with limited labeled samples- posess challenges such as catastrophic forgetting and overfitting, leading to sigmificant performance degradation across incremental sessions. As a remedy, recent work focuses on minimizing the interference of embeddings between base and incremental classes. However, previous studies have not explicitly con- sidered variation in discriminative difficulty across samples and classes, leaving room for improvement: we observe that hard-negative (i.e., difficult to discriminate from the label) samples and classes significantly affect FSCIL performance, whereas casy ones have little impact. To this end, we propose a hard-negative prolotype-based regularization approach that enhances discrimination between similar classes by imposing a penalty margin between each sample and its most similar class prototypes based on cosine similarity. To select hard-negative prototypes, we explore two distinct mining strategies: dynamic selection that leverages the model's decision boundary, and static selection that utilizes pre-defined class-wise similarity matrix derived from external sources such as pre-trained models. We evaluate approach on three widely used benchmarks, minilmageNett, CIFAR100, and CUB200, achieving state-of-the-art performance on each. Comprehensive analyses demon- strate that our proposed method enhances intra-class cohesion and inter-class scparability of embeddings, both of which are crucial for FSCIL to better accommodate novel classes.
1 Introduction
Few-shot class-incremental learning (FSCIL.) has recently gained significant. attention due to potential for variety of real-world applications (Tao al. 2020). FSCIL training process typically consists of two phases: (1) the base session, in which model is trained base classes with large amount of labeled data, and (2) the incremental session, where the model is fine-tuned novel classes using few labeled samples. FSCIL poses two challenging tasks that need to be addressed jointly: (i) class-incremental learning, in which model must learn newly added classes without forgetting previously acquired knowledge (i.e., catastrophic forgetting MMCloskey & Cohen 1989)); ); and (ii) few-shot learning, requiring the model learns new knowledge from a small number of training examples while avoiding overfitting to the new classes and generalizing to unscon test samples.
Various methods have been proposed for FSCIL, with focus minimizing interference of embeddings between base and incremental classes. In this context, two main approaches have been explored prototype learning and embedding space learning-eithe separately or in combination. Prototype learning aims to learn distinguishable and clas--represnttative vectors that can gemeralize to unseen novel classes, as demon-
strated by recent efforts such as human- human-coguition iegnition-inspired prototypes (Yao et al. 2022) and orthogonal pscudo-targets (Ahmed al. 2024).. Additionally, embedding space learning socks to construct compact intra-class clusters while enforcing large inter-class margins (Peng et al. 2022 Zou et al. 2022 Song al. 2023), thereby preserving sufficient. space for incoming novel classes.

---

Published in Transactions on Machine Learning Research (11/2025)
Baseline
Ours
Base Session
push (HN)
Herd- Hord-Negative Negative (HN) Protetype Regulonzation
hard samples
push (HN)
A class prototype sample
3 incorrect sample
Incremental Session
confusion between old and new classes
sufficient space new classes
Figure 1: An overview of the proposed hard-negative prototype regularization approach. Previous methods which overlook variation in discriminative hardness across samples and classes often struggle with hard examples, leading to confusion between old (red, green, blue) and new (purple) classes. Our method leverages hard-negative prototypes of each sample to impose a penalty margin, effectively pushing hard samples away from confusing classes and creating sufficient space to accommodate novel classes in incremental sessions.
Nonetheless, while prior work has effectively mitigated catastrophic forgetting in FSCIL, they often overlook the variation in discriminative difficulty across samples and classes, e.g,, visually similar classes or ambiguous examples near decision boundaries, leaving room for improvement. As illustrated in Figure [ a key factor shaping a model's decision boundary is the presence of hard samples, which reside far away from their respective class centers (i.e., prototypes in our context). These hard samples negatively impact FSCIL in two ways: (1) they cause confusion in the model's predictions, resulting in overall performance degradation, and (2) they occupy regions in the embedding space, potentially hindering the adaptability of novel classes and causing the forgetting of previously learned classes.
To support our motivation, we analyze the variation in sample and class difficulty and observe that hard- negative samples and classes are key to success in FSCIL. Specifically, after training the base classes, we classify their test samples into hard and casy groups by sclecting the top-k samples that are farthest from and closest to their prototypes, respectively. As shown in Figure 2 easy samples not only achieve superior performance but also experience little or no catastrophic forgetting (compare dark vs. light for each color of easy samples). In contrast, hard samples exhibit poor performance, with significant forgetting occurring after learning incremental classes. These tendencies indicate that hard samples are the main cause of confusion and forgetting in FSCIL. To further analyze this confusion, we evaluate the top-k accuracy of models and compute the delta the difference between accuracies for adjacent k values as illustrated in Figure 3 The delta graphs show an exponentially decreasing trend, implying that most confusions are concentrated among small number of classes. We define these classes as hard-negative classes and leverage their prototypes to regularize samples, thereby minimizing interference among emboddings of different classes. detailed analysis of the difficulty variation is provided in Section 3
Inspired by these observations, we propose a novel negatiee prototype regularization (HNPR) approach by identifying the top-k most similar class prototypes for cach sample and impose penalty margin between thei. By applying this penalty margin, the model is trained to focus on distinguishing similar classes, thereby pushing them apart and yielding a larger margin in the embedding space for novel classes. To select these hard-negative prototypes, we explore two distinct mining strategies based on the scope of similarity comparison. Dynamic selection leverages the model's decision boundary during training, while static selection utilizes a pre-defined class- class-wise similarity matrix derived from external sources such as pre-trained visual or textual models. We validate the effectiveness of our approach against three widely used benchmarks minilmageNet (Vinyals al. 2016), CIFAR100 (Krizhevsky al., 2009), and CUB200 (Wah al.) 2011 achieving state-of-the-art performance across all benchmarks. Extensive analysis demonstrates the intra- class cohesion and inter-class separability of the learned embedding space, which supports the superior performance. Our contributions are summarized as follows:
2

---

Published in Transactions on Machine Learning Research (11/2025)
We analyze the variation in difficulty among samples and classes, demonstrating that the poor performance and significant forgetting exhibited by hard samples and classes pose critical challenges for effective FSCIL.
We propose a novel hard-negative prototype regularization approach that imposes a penalty mar- gin on similar classes, thereby guiding the model to focus on their distinctions. Additionally, we explore two mining conntegiisddnnnnnn selection and static selection for identifying hard-negative protolypes based on the scope of similarity comparison.
We conduct extensive experiments on three widely used benchmarks, demonstrating the superiority of our approach and achieving state-of- of-ttf--ttrtttoo performance across all evaluated benchmarks.
2 Related Work
Few-Shot Class-Incremen Incrementell Learning (FSCIL). FSCIL presents a challenging problem that involves
incrementally adapting to a stream of data with few labeled samples (Tao et al. 2020). Various approaches
have been proposed and can be classified as follows: (i) meta-learning- based methods Chi et al.) 2022 Zhu
ct al.
2021), (ii) prototype-ban based methods (Hersche et al. 2022; Zhu et al.) 2021: Zhnang ct al. 2023
(iii) dynamic network-based methods (Tao et al.] 2020 [Kng el al. 2023; Yang el al.] 2022).
p (iv) replay-
based methods (Liu et al.) 2022), and (v) embedding space- based methods (Kim et al. 2023 Cheraghian
ct al.
2021 Song ct al.] 2023] Zhou al., 2022a, Oh et al.] 2024). Among these, for embedding space-
based maethods-white have recently shown remarkable success -im to achieve intra-class compactness and inter-class discriminability to minimize feature interference between old and new classes. In this context, ALICE (Peng et al., (022) utilized an angular margin loss, complemented by augmentation and balanced testing, to obtain well-chustered features. Furthermore, SAVC (Song et al.) 2023) applies predefined trans- formations to generate virtual classes and uses supervised contrastive learning in a multi-semantic fantasy space to improve separation between base and future novel classes. Although these studics validate the effectiveness of a discriminative feature space, variations in data difficulty have not been explicitly consid- ered. Therefore, we propose novel method that encourages the model to better distinguish between similar classes by leveraging an additional penalty margin for hard-negative prototypes.
Metric Learning with Hard-Negatives Unlike traditional learning methods that directly predict la- bels or values, metric learning focuses on capturing the relationships within a data distribution through a similarity-aware distance function. Farlier work, such as contrastive loss |Chopra el al., 2005) and triplet loss (Wang et al.] [2014] [Hoffer & Ailon 2015), utilizes margins to enhance model learning ability by en- suring that negative pairs are sufficiently separated in the embedding space. Additionally, angular distance metrics, employed in large-margin softmax loss (L-Softmax) (Liu et al.
2016) and its improvements, further
refine feature discrimination through angular regularization (Liu et al. 2017| and additive margins (Wang) eet al.] 2018; Deng et al. 2019). Hard-negative mining plays a crucial role in metric learning by target- ing the most challenging negative examples- those closest to the anchor in the embedding space. The use of (ammi-)hard-negattinn
mining has demonstrated significant, performance improvements across various do- mains, including face recognition (Schroff et al., 2015), object detection (Shrivastava et al., 2016), image classification (Suh et al., 2019)) and graph contrastive learning (Xia et al. (022)). Furthermore, similar metric- and tard-example mining- based approaches have been extensively explored in the few-shot learning
literature (Mandalika, 2025 Roy et al. 2022 Li et al. 2021), where prototype- and relation-based methods leverage embedding distances to generalize from only a few examples. While previous works have primarily focused on sample-wise mining within mini-batches (Xuan et al.
2020 Robinson et al., 2020). we extend
this approach by exploring class-wise wise negativee mining based on representative prototypes, which are Zhu et al.) 2021). ). This prototype- based strategy closely aligus with class-level proxy mining principles (Movshovitz- Attias et al. 2017 Kim et al. 2020), offering theoretical foundation. To the best of our knowledge, this is the first approach to leverage the hard-negative concept within the FSCIL literature.
commonly employed in continual learning scenarios (Hersche al. 2022
3 Nard-Negatives in FSCIL
In this section, we demonstrate that hard-negative samples and classes are crucial for effective FSCIL, motivating our hard-negative prototype regularization approach.
3

---

Published in Transactions on Machine Learning Research (11/2025)
3.1 Preliminaries
Problem Formulation. We follow the standard FSCIL experiment selup, which consists of two main phases: (i) a base session and (ii) multiple incremental sessions. In the base session, a model is first trained with sufficient training data. Then limited number of training data (thus called few-shot) is provided in the incremental sessions. Note that, in the incremental sessions, data from previous sessions are inaccessible; this presents the challenges of retaining the knowledge learned from earlier sessions. Formally, we assume m-step FSCIL tasks where we have m + 1 training data splits for the basc and incremental sessions, i.c., D = (D("), D(,), D(m)). }. In session i E {0, 1,.,, m), the goal is to learn a function / X + V. given y(()) is its corresponding label. the training data (z, E D(O), where I E x is an input sample and y E
Note that the label spaces for different sessions are disjoint, i..., y0 n y( 0 for i i j. Thus, model should learn new classes in each incremental session while retaining knowledge of old classes without access to past samples. An incremental session is often organized in C-way K-shot setting, where each session has training samples from C' classes, with K samples per class, providing total of C' x K training samples per session. Concretely, at the i-th incremental session, we optimize the following learning objective:
minimize Dicaa)- [C(f(;; (()) )),
(1)
where we minimize the empirical risk over all the seen classes up to session
Prototype-Based Classification. Prolotype-based classification is a widely used approach in FSCIL literature due to its simplicity and effectiveness in handling limited data for each class (Zhu et al. 2021
Hersche et al. 2022 Zhuang et al. 2023). Following previous work (Peng et al., 2022), we define the prototype Pe of class e as the average feature of training samples. Classification is performed by calculating the cosine similarity between an input feature and the prototypes, which can be formalized as follows:
9 = arg max f(x) P. ||
(2)
where ğ denotes the predicted class, represents the dot product, and |I-I| is the Euclidean norm. During the base session training, instead of recalculating prototypes at every epoch, we introduce randomly initialized, learnable weights W, which act as gseudo-prototypee in place of P. These pseudo- protooyps are updated dynamically through backpropagation, allowing them to adapt continuuusly to the classification task. The training objective is to minimize the loss by optimizing these learnable pseudo-prototypes. After the base session training, we compute the protolypes for both the base and incremental session classes using the frozen backbone.
Baseline. We set baseline approach using angular margin loss, which aims to reduce the overlap between features from different classes. Previous studies (Liu et al.) 2017 Wang et al. 2018 Deng et al. 2019) have proposed various methods to enforce a margin on the angular decision boundary, commonly referred as angular margin loss LAM- Variants of the angular margin loss function can be formulated based on choices of (i) the scale factor s and (ii) margins m1, m2, and my that define the angular dccision boundary:
1 Σ log =ccos(m) +22) ma) + Σ jttn (cos())))
(caos(m Du +m2)) -ma)
LAM
where 0 denotes the angle between the input feature and the class prototype, N is the number of samples, and yi represents the ground-truth class. Note that the additive margin m3 applied to the cosine values is
typically used in the FSCIL task. This is because (i) the multiplicative margin mj often leads to unstable
training issues by making the target logit curve steeper (Deng et al., 2019) and (ii) the additive margins
m2 and my are numerically similar. The angular margin loss for the FSCIL task (Peng et al. 2022] can be
formulated as follows:
1 LAM
Σ log e (cons(0, ) m)) + Ejty. (<(os((0)))
(e(coo(O,,)-m)
(4)
N
4

---

Published in Transactions on Machine Learning Research (11/2025)
100
Duragbase
100
150
Lou
100
burs
109
09.3
100
100
100
Oursgbbase
09.7
60.3
99.
Bastline@base
Baseline@basse
Baseline@basee
Baselineglast
Baselineglast
Baselineg
Accuracy (%)
Accuracy
Accuracy
40
41.
53.0
20
20
20.2
3.7
20-
14.0
hard samples
easy
hard samples
easy samples
hard samples
easy samples
5%
k 10%
20%
100
Ouraghase
100
Durggbase
100
Dursgaase
09.3
27.33
86:1
28.2
Baseline@base
Baseline@base
Bsselinn@basee
80
Baselino@tast
Baseline@last
Baselino@last
Accuracy (%)
Accuracy (95)
Accuracy
2.1
20
20-
20-
hard samples
easy
hard samples
easy samples
hard samples
easy samples
k 30%
k 40%
k 50%
An analysis on the impact of difficulty variation of samples in FSCIL. After training base classes, we divide the test samples these classes into two groups using the cosine distance from their corresponding class prototypes: hard samples (with the top-k largest distances) and easy samples (with the top-k smallest distances). The baseline is trained using angular margin loss (Peng et al. 2022), without hard-negative prototype regularization. We report the accuracy after the base session (dark colors) and the last session (light colors) for each method. Hard samples are observed as the main cause of confusion and forgetting, whereas easy samples have little effect on overall performance.
Figure 2:
3.2 Impact of Hard-Negatives in FSCIL
Easy Samples vs. Hard Samples. By analyzing variations in sample difficulty, we observe that hard samples are key factors in both performance and forgetting in FSCIL. Specifically, after training on the base classes, we divide the test samples of the base classes into two groups: hard samples, defined as the top-k samples with the largest distances from their corresponding prototypes, and easy samples, defined the top-k samples with the smallest distances. We then report the accuracy for cach group after the basc session (i.e., right after learning the classes) and after the last session (i.e., after learning all incremental classes). As shown in Figure 2 easy samples show superior performance and exhibit little or no catastrophic forgetting, whereas hard samples not only perform poorly but also suffer from significant forgetting during incremental sessions. These trends bocome even more pronounced when using smaller values of k. further underscoring that hard samples critical to the success of FSCIL. Note that proposed method reduces the forgetting of hard samples between the base and last sessions, which will be discussed in the following sections.
Confusion in Hard Classes. Given that hard samples are the main source of confusion in FSCIL, we further analyze class-level confusions and show that misclassifccations predominantly occur within small number of challenging classes. To elaborate, evaluate the accuracy with top-k predictions (i.e., Acc@k) and compute the difference between accuracies for adjacent & values (ie.,, A Acc@k-Aco@k- k > 1). The resulting delta value reflects the incremental improvement in accuracy when adding one more prediction, and a larger delta indicates that the model exhibits greater confusion among the challenging classes. As shown in Figure K] in both base and last sessions, the overall trend of delta graphs is exponentially decreasing implying that most of the confusions are concentrated among small number of challenging classes. Note that the delta value does not designate a fixed set of challenging classes: e.g., some sample in class A may be confused with class B, while others in class A may be confused with class C. These varied confusions

---

Published in Transactions on Machine Learning Research (11/2025)
90
12
10% Teeee - Meoee Dled
Delta (Aco@k Acc@k-1 10%
85
10
Accuracy (%6
92.5
Baseline Accuracy
5
Accuracy (%)
80
Baseline Accuracy
8
Ours Accuracy
75
Ours Accuracy
87.5
Baseline
Delta
3
70
Delta
Delta
6
85.0
65
82.5
-2
60
80.0
1
2
4
8
10
2
3
5
8
9
10
(Aco@k)
(Acc@ek)
Base Session
Last Session
Figure An analysis on the confusion among classes in the base and last sessions using top-k accuracy (i.e., accuracy with top-k predictions, denoted as Acc@k) and delta (i.e., the difference between accuracies for adjacent & values for k > 1). The same models in Figure are used for this analysis. The delta graphs (triangle markers) exhibit exponentially decreasing trend, indicating that most confusions are concentrated among a small number of classes.
are aggregated into single delta value, reflecting the overall uncertainty within the class. Therefore, we define these challenging classes hard-negative classes per-sample basis and leverage their prototypes to regularize each sample.
4 Hard-Negative Prototype Regularization
We propose novel approach that emphasizes the distinctions between hard-negative classes while incor- porating schemes for hard-negative prototype mining- In baseline, the model is expected to learn two key aspects to minimize the loss function due to the margin effect: (i) the angle between positive pairs should be minimized to reduce intra-class variance (altractive force, 0y,), and (ii) the angle between negative pairs should be maximized to improve inter-class separation (repulsive force, 0,). A major limitation, how- ever, is that the repulsive force uniformly exerted on all negative pairs, meaning the margin m is applicd equally regardless the pairs. Considering the varying degrees of similarity among classes, this approach may restrict the model's ability to distinguish between similar classes, potentially leading sub-optimal performance. This limitation is further confirmed by our observation Section 3.2 where most confusions are concentrated among the top-k challenging classes. To address this issue, identify similar classes and leverage their prototypes to regularize the learning of challenging classes by imposing an additional margin.
4.1 Hard-Negative Prototype Mining
To enable a model to better distinguish challenging classes, we introduce two hard-nogative mining ap- proaches based on the scope of similarity comparison: dynamic selection and static selection. Our mining process relies the cosine similarity between the input feature and class prototypes. Dynamic selection leverages the modcl's decision boundary during learning, whereas static selection utilizes a pre-defined class- wise similarity matrix from external sources (e.g., pre-trained models). Figure illustrates the overall flow of these different mining strategies.
Dynamic Selection. Assuming that the model is optimized via stochastic gradient descent, the similarity distribution of an input sample varies with each gradient step. Therefore, selecting similar classes through batch-wise similarity calculation can reflect the decision boundary at the current optimization step. Given a set of training batches B and the set of classes C, the similarity score 8! between a sample in mini-batch and the pseudo-prototype W. E IRd of class e can be computed as follows:
f(1 W.
5(0) =
(5)

---

Published in Transactions on Machine Learning Research (11/2025)
select top-k repeat
learnable protatype
embeedding
protstype
A.A.A.
cominesimillrity
class
similarity
pre traitra
meen
model
encoder
batch
class
embedding
select top-k
(a) Dynamic Selection
(b) Static Selection
Overview of proposed hard-negative class mining strategies: (a) dynamic selection, and (b) static selection. The dynamic selection approach repeatedly selects the top-k similar classes in a batch-wise manner, considering the continuous shift of similarity distribution at each gradient step. In contrast, the static selection approach leverages pre-trained encoder to compute similarities, assuming fixed similarity distribution for selecting hard-negative classes.
Figure 4:
where ( Rd denotes the feature vector. Finally, the top-k similar classes of an input sample are dynamically defined in batch-wise manner follows:
C (c, = {c E C I TopK(s!") k)):
(6)
where ) E R|Cl denotes the similarity vector of 2;, and the TopK function returns the classes with the k largest similarity scores, excluding the true class yt- Note that under this strategy, similar classes for input sample may vary across optimization steps because the learnable pseudo-prototyper W are periodically updated. Furthermore, the similar classes for two inputs from the same class differ, i.e., C + (j.k) for i F j and y; Vý. since the similarity is measured at the instance level. We empirically observe that this variation helps improve the model's generalizability, as discussed in detail in the Section 5
Static Selection. Apart from dynamic selection, which is based on the current decision boundary, selecting similar classes by leveraging the semantics of a pre-trained encoder on large-scale data could be another option. In this work, we explore two static selection strategies based on the data modality: (i) using pre- trained visual encoder, and (ii) employing a pre-trained textual encoder. To quantify similarities between classes, we define class-representative prototypes (denoted as P. for class c) using the outputs from each encoder. For the visual encoder, P. is defined as the average feature of all samples belonging to class c, following the traditional approach described in Section 3.1 For the textual encoder, the class prototype is computed using the natural language label of the class. Once the class prototypes are obtained from either modality, we compute the class similarity matrix S using cosine similarity, S PT P, where P represents the Ln-normalized prototypes. The similarity vector for sample z; is obtained by indexing S with its ground-truth class yi, i.e, S, = S(y))- Finally, the top-k most similar classes are selected as follows:
Caim (c e C | (opK k)}. (t.
(7)
where the Topk function is defined in Equation 6 In the following sections, we use the dynamic selection as our default mining strategy, while also exploring the effect of static selection in Section 5.2
4.2 Hard-Negative Prototype Regularization in Angular Margin Space
Given the top-k similar classes for a sample I;, we assign an extra margin as penalty to accelerate the separation between these challenging classes. Concretely, we decompose the term in the denominator of Equation 3 which imposes repulsive force between the sample and the negative prototypes into two parts: one for hard-negative prototypes and the other for non-hard-negative prototypes. For the hard- negative prototypes, we introduce new regularization term as follows:
=(ccus(O,,) m)
CHNPR-AM = log (c0s(0,,) Imm) + (c(coo(())+t)) Σ ()()))(9,))
(8)
cm
SEC i stu.
(1.6)

---

Published in Transactions on Machine Learning Research (11/2025)
Base classes
Novel classes
Base
Novel
Novel class Prototype
Prototype
Baseline (w/o HNPR)
Ours (w/ HNPR)
visualization comparing the embedding spaces of base and novel classes, alongside the corresponding sample images. Our proposed hard-negative prototype regularization (HNPR) clearly shows better intra-class cohcsion and inter-class distinction.
Figure 5:
t-SNE (Van der Maaten & Hinton 2008
where ni denotes the extra penalty margin applied to challenging classes. For clarity, assume that both m and n are positive numbers. To minimize the overall loss, a model should deal with not only normal class prototypes (0,) but also challenging (hard-negative) class prototypes with a penalty margin (regularized repulsive force, Oc). Notably, increasing the angular separation between the input feature and similar class prototypes leads the model capture the nuanced difference between them, contributing to enhancing the model's discriminative capability.
5 Experimental Results
Benchmarks. We use the following three widely-used FSCIL benchmarks: (1) minilmageNeet (Vinyals et al. 2016). (2) CIFAR100 (Krizhevsky et al. 2009), and (3) CUB200 (Wah et al. 2011). minilmageNee and CIFAR100 benchmarks include 100 classos, with each dataset comprising 60K images, while CUB200 bench- mark contains over 11K finc- ane-grained images across 200 classes of North American bird species. Following existing works (Kim et al., 2023 Wang et al. 2023 Zhao et. al. 2023), we use an 8-step 5-way 5-shot setting for the minilmageNet and CIFAR100 benchmarks, i.e, 60 classes are set as base classes, and the rest is set as new classes used in cight incremental sessions, each of which has five classes and five samples per class. Additionally, we use configuration of 10 steps with 10 classes and 5 samples per class (10-way 5-shot) for the CUB200 benchmark - in the base session (session 0), half of the total 200 classes are utilized. Then, in the incremental sessions (sessions 1 to 10), the remaining 100 classes are evenly divided into ten separate subsets (with 10 classes per session), with each class being represented by 5 randomly selected examples.
Implementation Details. Following the standard setting of prior work (Tao al. 2020 we use ResNet- 18 (He et al. 2016) as the backbone network and train our model from scratch for the minilmageNNet and CIFAR100 datasets, while using an ImageNet-pretraince model for the CUB200 dataset. We utilize the augmentation strategies frequently employed in prior FSCIL research (Peng et al.) 2022 Yang et al. 2023): (1) resized cropping, (2) horizontal flipping, and (3) color jittering. Our model is trained on a single NVIDIA A100 GPU with an SGD optimizer, using cosine learning rate scheduling with warm-up. Hyperparameters are set with grid scarch as follows: is chosen to be 30 across all datascts. For m, we use 0.3 for CIFAR100 and 0.4 for the others. The parameter l (for the top-k selection) is set to 1 for CUB200 and 2 for the others. Lastly, ii is set to 0.05. Other hyperparameters (e.g., learning rate, weight decay, and batch size) and implementation details are provided in the appendix.
8

---

Published in Transactions on Machine Learning Research (11/2025)
Table 1:
Performance comparison with state-of-the-art methods on miniImageNet (Vnyals et al.
2016) benchmark. PD represents the rate of performance drop, calculated as the difference between the accuracies of the base (TO) and last (T8) sessions. All values are presenled as percentages (%). We mark the best value in bold, and the second best with underline.
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
PD(L)
TOPIC
Tao et al.
2020
61.31
50.09
45.17
41.16
37.48
35.52
32.19
29.46
24.42
36.89
CLOM
Lou et al.
2022
73.08
68.09
64.16
60.41
57.41
54.29
51.54
49.37
48.00
25.08
ALICE
Peng el
2022
80.60
70.60
67.40
64.50
62.50
60.00
57.80
56.80
55.70
24.90
FACT
Zhou
2022a
72.56
69.63
66.20
62.77
60.60
57.33
54.34
52.16
50.49
22.07
LIMIT
Zhou el al.
20226
72.32
68.47
64.30
60.78
57.95
55.07
52.70
50.72
49.19
23.13
GKEAL
Zhuang
2023
73.59
68.90
65.33
62.29
59.39
56.70
54.20
52.59
51.31
22.28
WaRP
Kim et at.
2023
72.99
68.10
64.31
61.30
58.64
56.08
53.40
51.72
50.65
22.34
SoftNet
Kang et. a..
2203
76.63
70.13
65.92
62.52
59.49
56.56
53.71
51.72
50.48
26.15
SAVC [Song al..
2023
81.12
76.14
72.43
68.92
66.48
62.95
59.92
58.39
57.11
24.01
CaBD
Zhao et al.
2023
74.65
69.89
65.44
61.76
59.49
56.11
53.28
51.74
50.49
24.16
TEEN Wang a...
2023
73.53
70.55
66.37
63.23
60.53
57.95
55.24
53.44
52.08
21.45
OrCo (Ahmed et al. 2024
83.30
75.32
71.53
68.16
65.63
63.12
60.20
58.82
58.08
25.22
Comp- Comp-FSCIL [Zou et al.
2024
82.78
77.82
73.70
70.57
68.26
65.11
62.19
60.12
59.00
23.78
Bag of Tricks (Rov et CLOSER (Oh et al. 2024
2024
84.30
79.59
75.49
71.40
68.45
65.13
62.20
60.52
59.11
25.19
76.02
71.61
67.99
64.69
61.70
58.94
56.23
54.52
53.33
22.69
Ours
80.90
76.06
72.24
69.92
67.27
64.96
62.07
60.91
59.96
20.94
5.1 Effect of Hard-Negative Prototype Regularization
We compare our method with existing state-of-the- art methods on three FSCIL benchmarks: minilmageNet, CIFAR100, and CUB200. Following previous work (Zhang et al. 2021: Peng al. 2022 Oh et al. 2024), we employ the performance dropping rate (PD) as the evaluation metric, which represents the difference in accuracy between the basc scssion and the final session. As shown in Tables our model outperforms other approaches across all benchmarks, achieving new tatte-of-thh-art performance. Furthermore, to assess generalization to modern transformer-basee architectures, we compare against the state-of-the-arl ASP (Atlention-awware Self-adaptive Prompt) @Lu et al. 2024h method on the ImageNet- R (Hendrycks et al. 2021) benchmark. As detailed in Appendix A.1 HNPR outperforms both vanilla ASP and its eegular-marginn variant even when using a much larger ViT backbone.
Comparison with State-of-the-Art Approaches.
3 and (the extended version of Tables 3 and are in the Appendix),
Effect on Embedding Clusters. Further, as shown in Figure 5 we employ I-SNE (Van der
Table 2: Comparison of average cosine distances be- tween samples and three types of prototypes (positive, negative, and hard-negative).
Maaten & Hinton 2008 visualizations to qualita- tively analyze the impact of our proposed approach. We visualize sample emboddings from both base and novel classes along with their corresponding proto- types (i.e., the average of training sample embed- dings). Importantly, our method generally guides the learned representations to map positive pairs closer together while (hard) negative pairs further apart. This finding is further confirmed by Table 2
Average Cosine Distance
Method
Positive(L)
Nogative(t)
Bard-Nogativoff
Baseline
0.237+0.074
0.190+0.0022
0.386++0.075
Ours
0.23110.074
0.501
0.394+0.077
which measures the average cosine distance between samples and three different types prototypes.
Effect on Hard Samples and Classes. As shown in Figure 2 our proposed method reduces the forgetting of hard samples between the base and last sessions (with values displayed immediately to the right of the arrow) regardless of k. Notably, for & e (5, 10), the last session accuracy achieved by our method (e.g., 15.0% for 5) even exceeds the base session accuracy obtained by the bascline (c.g., 14.0% for = 5). Furthermore, in the hard class analysis presented in Figure 3. our model generally demonstrates superior performance in both sessions across various values (see circle markers), suggesting that hard-negativv
9

---

Published in Transactions on Machine Learning Research (11/2025)
Performance comparison with state-of-the-art methods on CTFAR100 Krizhevsky et al. 2009) benchmark. We mark the best value in bold, and the second best with underline.
Table 3:
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
PD(4)
SAVC Song et al.
2023
78.77
73.31
69.31
64.93
61.70
59.25
57.13
55.19
53.12
25.65
CaBD Zhao et al.
2023
79.45
75.20
71.34
67.40
64.50
61.05
58.73
56.73
54.31
25.14
TEEN Wang
2023
74.92
72.65
68.74
65.01
62.01
59.29
57.90
54.76
52.64
22.28
OrCo Ahmed et al.
2024
80.08
68.16
66.99
60.97
59.78
58.60
57.04
55.13
52.19
27.89
Comp- FSCI- Zou al.
2024
80.93
76.52
72.69
68.52
65.50
62.62
60.96
59.27
56.71
24.22
Bag of Tricks (Roy ot al.
2024
80.25
77.20
75.09
70.82
67.83
64.86
62.73
60.52
58.55
21.70
CLOSER (Oh et al. 2024
75.72
71.83
68.32
64.62
61.91
59.25
57.53
55.43
53.32
22.40
Ours
77.38
73.57
70.74
66.87
64.30
61.71
60.19
58.23
55.98
21.40
Table 4: Performance comparison with state-of-the-art methods on CUB200 We mark the best value in bold, and the second best with underline.
Wah et al.
2011
benchmark.
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
PD(1)
SAVC
Song et al.
2023
81.85
77.92
74.95
70.21
69.96
67.02
66.16
65.30
63.84
63.15
62.50
19.35
CaBD Zhao et al.
2023
79.12
74.99
70.87
67 30
65.89
63.45
61.40
60.11
58.61
58.23
57.48
21.64
TEEN
Wang cl al.
2023
77.26
76.13
72.81
68.16
67.77
64.40
63.25
62.29
61.19
60.32
59.31
17.95
OrCo
Almmdd al. 2024
75.59
71.73
64.48
60.83
60.66
38.80
59.29
58.73
58.01
59.02
58.84
16.75
Comp-FSCIL Zou al.
2024
80.94
77.51
74.34
71.00
68.77
66.41
64.85
63.92
62.12
62.10
61.17
19.77
Bag of Tricks (Roy et al.
2021
82.31
78.03
75.15
70.99
71.06
67.85
67.44
66.05
64.95
64.31
63.60
18.71
CLOSER (Oh et al. 2021
79.40
75.92
73.50
70.17
69.24
67.22
66.73
65.69
64.00
64.02
63.58
15.82
Ours
77.44
74.58
72.27
69.23
68.02
65.88
65.32
63.86
62.43
62.40
61.98
15.46
prototype regularization leads to more reliable predictions. Although the top-1 accuracies in the base session are nearly identical (80.77 for the baseline vs. 80.90 for ours), our approach consistently achieves higher base session accuracies for most values of k (e.g., 95.70 for the baseline vs. 96.50 for our method at k 10), trend also observed in the top-1 last session accuracy (58.31 for the baseline vs. 59.96 for our method).
5.2 Further Analysis
Hard-Negative Prototype Mining Analysis. In this section, we investigate the impact of different hard-negative prototype mining approaches, which are explained in Section 4.1 For the static selection approach, we utilize a frozen ResNet pre-trained on the ImageNet dataset (Russakovsky et al.) 2015 the visual encoder and frozen CLIP-VIT-B/32 (Radford et al. 2021 backbone as the textual encoder. As shown in Table 5 regardless of the selection approach, our proposed method improves performance compared to the baseline. Among the selection approaches, the dynamic selection approach generally demonstrates the best performance, likely due to its tailored adaptation capabilities that better accommodate shifting decision boundarics. Since the model's decision boundary shifts with each gradient step, the classes similar to a given instance may vary. While dominant hard-negative classes persist, the occasional inclusion of unusual hard- negative classes may help regularize the model by introdueing inherent noise, thereby preventing overfitting to dominant classes during gradient estimation (Keskar et al. 2017). To verify this, visualize the probabilities of selecting two classes as hard-negative pairs throughout training. We count the hard-ncgative classes for each anchor class per epoch at the sample level and aggregate these counts into probabilities after training. As shown in Figure 6 both approaches exhibit a similar trend (e.g., square patterns in miniTmageNet) in selection, but the dynamic approach stochastically includes unusual classes introducing variability that regularizes the model.
Diwee Prototype Regularization Analysis. Our proposed approach focuses on increasing inter-class separation by pushing embeddings of similar (i.e., hard-negative) classes farther away. As shown in Table [6] we further conduct an experiment demonstrate the effect of using prototypes of similar classes as hard-negatives in the FSCIL task. To this end, we compare ours with variant models with prototypes from (1) random classes and (2) dissimilar (i.e., eesynnegative) classes. Note that, for the latter (2), we
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 5: Performance comparison different mining approaches for hard-negative prototype regularization
(HNPR). The bascline method is trained using angular margin loss without HNPR.
minilmageNet
CIFAR100
CUB200
Method
HNPR
Base
Last
PD(1)
Base
Last
PD(L)
Base
Last
PD(J)
Baseline
79.30
57.62
21.68
77.37
54.55
22.82
77.30
60.86
16.44
Static selection (visual)
80.15
58.89
21.26
78.02
55.35
22.67
76.71
60.72
15.99
Static selection (textual)
79.97
58.82
21.15
77.03
54.40
22.63
76.75
60.72
16.03
Dynamic selection (ours)
80.90
59.96
20.94
77.38
55.98
21.40
77.44
61.98
15.46
21
21
21
28
28
28
CUB200
35
35
35
42
42
42
49
49
49
56
0.10
56
84
84
84
91
91
91
98
0.00
98
98
14
21
28
35
82
29
56
63
70
77
34
92
98
21
28
36
22
29
26
63
70
16
38
21
28
35
22
20
36
63
70
77
84
92
36
00255
0
Net
15.
10.
0.20
10
10
15
20
20
20
25
0.15
25
0.6
25
30
30
30
35
0.10
35
35
0.4
40
40
45
45
50
0.05
50
50
0.2
55
55
55
0.00
0.0
10
15
20
23
35
40
45
50
22
35
40
50
35
23
30
35
40
Dynamic Selection (Ours)
Static Selection (Visual)
Static Selection (Textual)
Figure 6: Visualization of the selection probabilities of two classes as hard-negative pairs during training, using different mining methods. The y-axis of each heatmap represents an anchor class, where we count the classes of hard-ncgative pairs (x-axis) per epoch. After training, aggregate the counts of hard-negative pairs for each anchor class and normalize these accumulated counts to probabilities. Note that the overall trend in selection is similar across selection approaches, while the dynamic approach selects hard-negative pairs in a stochastic manner. Best viewed in color.
use our top-k function Section but replace the argmax operation with argmin choosing the most. k dissimilar classes excluding their ground-trut class. In Table we obscrve that our prototype regularization with top-k similar class mining indeed improves the overall FSCIL performance, clearly outperforming other of top-k dissimilar classes. Moreover,
design choices, i.e., of randomly-chosen classes the use of dissimilar classes in the FSCIL task is less significant in learning inter-class separation (it even performs worse than simply using randomly chosen classes as negative pairs). This may confirm that our approach to using hard-negative mining is effective in the FSCIL task. We further provide example images the hard-negative and easy-negative classes corresponding to the anchor sample in the Appendix (see Figure A1
argued,
Ablation Studies. We conduct an ablation study to assess the impact of elements composing our proposed method. As shown Figure 7 (a), we compare our proposed method to other loss functions and show that our model outperforms the alternative loss functions including cross-entropy (CE) and angular margin (AM) loss. Further, in Figure (b) and (c), we provide our analysis with different values of two main hyper- parameters, and fn. We observe that (1) models with 2 and m 0.05 show the best performance
11

---

Published in Transactions on Machine Learning Research (11/2025)
Table Performance comparison among different prototype regularization (PR) methods: (i) using random l classes, (ii) using top-k dissimilar classes (i.c., casy-negatives), and (iii) using top-k similar classes (i.c.,
hard-negatives). The baseline method is trained using angular margin loss without PR.
minilmageNNet
CIFAR100
CUB200
Method
PR
Base
Last
PD(L)
Base
Last
PD(I)
Base
Last
PD(I)
Baseline Random &
79.30
57.62
21.68
77.37
54.55
22.82
77.30
60.86
16.44
80.07
58.55
21.52
77.75
55.57
22.18
77.44
61.41
16.03
Gasy-Negative top-k
79.68
57.69
21.99
77.80
55.40
22.40
77.58
61.48
16.10
Hard-Negative top-k (ours)
80.90
59.96
20.94
77.38
55.98
21.40
77.44
61.98
15.46
80
CE
80
k-0
80
m=0.00
CE+DA
m=0.03
75
k-4
m=0:05
Accuracy (%6
70-
HNPR+AM-DA
Accuracy (%)
k=
k-6
Accuracy (%)
m=0.09
f0=0.07
70
70
65
65
50
0
3
4
7
8
0
5
6
7
0
4
5
Session
Session (b)
Session
(a)
(c)
Figure 7: (a) Effect data augmentation (DA) and comparison various loss functions: cross-entropy loss (CE), angular margin loss (AM), and ours (HNPR). (b, c) Analysis of two hyperparameters our proposed approach: (b) & for the top-k value, and (c) in for the penalty margin. A dynamic selection approach is applied to Irain all models the minilmageNet. dataset. Note that the blue line in (b) and (c) represents training without HNPR.
(see orange), and that (2) within a reasonable region of hyperparameters, HNPR clearly outperforms the baseline (compare circle vs. triangle).
6 Discussion
Batch-level hard-negative mining can lead training instability, particularly under small batch sizes or label imbalance. This issue has been widely observed and reported in prior work on hard-negative mining (Yuan) ct al. 2022 Balmaseda et al. 2025). As approach also relics mining hard negatives within mini- batches, it inevitably inherits this limitation. In our experiments, similarly observed signs of training instability when using small batch sizes (e.g., 128 for minilmageNet), which highlights broader challenge inherent to hard-negative-base based methods.
Despite this, one of the key advantages of our method lies in its computational efficiency. Although computational cost. could also be a potential concern, we observe that our proposed method incurs only negligible overhead compared to the baseline. As shown in Table 7 we measure the epoch runtime by executing both methods concurrently on the same device and averaging the results over five independent Irials. This minimal cost arises because our method
Comparison of average Table 7:
epoch runtime during the learning.
Method
CIFAR100
minilmageNet
Baseline
46. 13.22
107.16s 107.19
Ours 46. 4.660
operates on class prototypes, adding only a small amount of computations during training.
7 Conclusion
In this work, we propose a novel method for imposing a penalty margin between the sample and the hard- negative prototypes, which effectively guide the model to focus on the distinctions between challenging classes. We first analyze the impact of difficulty variation of data, demonstrating that hard-negative samples
12

---

Acknowleddgements. This work supported by ITP grant funded by the Korea government (MSIT) 10%, RS-2022- Published in Transactions on Machine Learning Research (11/2025)
and classes are key factors for effective FSCIL. Based this observation, we design hard-negative pro- totype regularization approach and explore two distinct selection strategies for these prototypes. Extensive experiments and analyses on widely used benchmarks demonstrate the superiority of our proposed method, achioving state-of-the-art performance.
LUTTP20025-0 R8S-2024-003370055, 30%, 25-2225-02263554 I1TP-2025-R8 1I220043, Adaptive Personality for Intelligent Agents, 30%). This work was also supported by the Korea Institute of Science and Technology (KIST) Institutional Program (Project No. 2E33611, 30%).
References
Noor Ahmed, Anna Kukleva, and Bernt Schiele. Orco: Towards better generalization via orthogonality and contrast for few-shot class- insemennremettll learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 28762-28771 2024.
Vicente Balmaseda, Bokun Wang, Ching-Long Lin, and Tianbao Yang. Discovering global false negatives on the fly for self-supervised contrastive learning, arXiv preprint or 4rivv:2502.26612 2025.
Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe, Pengfei Fang, Christian Simon, Lars Petersson, and Mehrtash Harandi. Synthesized feature based few-shot class-incremental learning on mixture of subspaces. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8661- 8670, 2021.
Zhixiang Chi, Li Gu, Huan Liu, Yang Wang, Yuanhao Yu, and Jin Tang. Metafscil: A meta-learning approach for few-shot class incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14166-14175, 2022.
Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning similarity metric discriminatively, with application face verification. In 2005 IEEE compuler sociely conference on compuler vision and pallern recogrition (CVPR 005) volume pp. 539-546. IEEE, 2005.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zaleirion. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690-4699, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXio:2010. 11929, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of- distribution generalization. In Proceedings of the LEEE/CVI international conference on computer vision, pp. 8340-88--00 8349, 2021.
Michael Hersche, Geethan Karunaratne, Giovanni Cherubini, Luca Benini, Abu Sebastian, and Abbas Rahimi. Constrained few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9057 9067, 2022.
Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In Similarity- Based Pattern Recog- vitioon: Third International Workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Pro- ceedings 3, pp. 84-92. Springer, 2015.
Hacyong Kang, Jachong Yoon, Sultan Rizky Hikmawan Madjid, Sung Ju Hwang, and Chang D. Yoo. On the soft-subnetwork for few-shot class incremental learning, 2023.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Con- ference on Learning Representations, 2017. URL http:://oppareeis.sse///orurrtep hwt.://////ooooriiioooooo/.r....
YeoYeon Kim, Dong-Jun Han, Jun Seo, and Jaekyun Moon. Warping the space: Weight space rotation for class-nnremmnnte few-shot learning. In The Eleventh International Conference on Learning Representa- tions, 2023. URL https:://ppenreviex. htt.://www....tttoooomooddddddaa
Sungyeon Kim, Dongwon Kim, Minsu Cho, and Suha Kwak. Proxy anchor loss for deep metric learning. In Proceedings of the IEER/CVF conference on computer vision and pattern recognition, pp. 3238 3247, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yiting Li, Haiyue Zhu, Yu Cheng, Wenxin Wang, Chek Sing Teo, Cheng Xiang, Prahlad Vadakkepat, and Tong Heng Lee. Few-shot object detection via classification refinement and distractor retreatment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15395-15403, 2021.
Chenxi Luu, Zhenyi Wang, Tianyi Xiong, Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. Few- shot class incremental learning with attention-aware self-adaptive prompt. In European Conference on Computer Vision, pp. 18. Springer, 2024.
Huan Liu, Li Gu, Zhixiang Chi, Yang Wang, Yuanhao Yu, Jun Chen, and Jin Tang. Few-shot class- incremental learning via entropy-regulariee data-free replay. arXiv preprint arXiv:2207.11213, 2022.
Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolutional neural networks. Tn Proceedings of the 33rd International Conference on International Conference on Machine Learnäng-Voie Volume 48, pp. 507-516, 2016.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on compuler vision and pattern recognition, pp. 212 212-220, 2017.
Sriram Mandalika. Generalizable vision-language few-shot adaptation with predictive prompts and negative learning. arXiv preprint 1xXiv:2555.1158 2025.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Yair Movshovitz-Attias, Alexander Toshev, Thomas K Leung, Sergey Toffe, and Saurabh Singh. No fuss distance metric learning using proxies. In Proceedings of the IEEE international conference on computer vision, pp. 360-366 368, 2017.
Junghun Oh, Sungyong Baik, and Kyoung Mu Lee. Closer: Towards better representation learning for few-shot iaas-inncemental learning. In European Conference on Computer Vision, pp. 18 35.3. Springer, 2024.
Can Peng, Kun Zhao, Tianren Wang, Meng Li, and Brian C Lovell. Few-shot class-incremental learning from an open-set perspective. In Computer Vision-ECCV 2022: 17th European Conference, Tel Avie, Israel, Oclober 23-27, 27, 2022, Proceedings, Part XXV, pp. 382-33 2973977-20 Springer, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763. PMLR, 2021.
Joshua Robinson, Ching Yao Chuang, Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. Xixiv preprint ar 2010:2000.04502, 2020.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Aniket Roy, Anshul Shah, Ketul Shah, Prithviraj Dhar, Anoop Cherian, and Rama Chellappa. Felmi: Few shot learning with hard mixup. Advances in Newral Informalion Processing Systems, 35:24474-24486, 2022.
Shuvendu Roy, Chunjong Park, Aldi Fahrezi, and Ali Etemad. A bag of tricks for few-shot class-incremental learning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https::/openrreviev.. ner/forrumiddDDyyYiikttt
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjoev Sathcesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-25)) 2015, doi: 10.1007 1100//11122-115-08116-1
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 815- 2015.
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training eggion-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 761-769, 2016.
Zeyin Song, Yifan Zhao, Yujun Shi, Peixi Peng, Li Yuan, and Yonghong Tian. Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning, 2023.
Yumin Suh, Bohyung Han, Wonsik Kim, and Kyoung Mu Lee. Slochastic class-based hard example mining for deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few-shot class- incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12183-12192, 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9, 2008
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in newral information processing systems, 29, 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds- 200-2011 dataset. 2011.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingohao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5274, 2018
Jiang Wang, Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. Learning fine-graine grained image similarity with deep ranking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1386-1303, 2014.
Qi-Wei Wang, Da- Wei Zhou, Yi-Kai Zhang, De-Chus Zhan, and Han-Jia Ye. Few-shot class-incremental learning via training-free prototype calibration, 2023.
Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learnóng research, 10(2), 2009.
Jun Xia, Lirong Wu, Ge Wang, Jintao Chen, and Stan Li. Progel: Rethinking hard negative mining in graph contrastive learning. Xiv preprint arXiv:21 21i::210000000 02.02277. 2021.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Hong Xuan, Abby Stylianou, Xiaotong Liu, and Robert Pless. Hard negative examples are hard, but 2020: 16th European Conference, Glasgow, UK, August 23-28 29-28, 2020,
useful. In Computer Proceedings, Part XIV 16, pp. 126 142. Springer, 2020.
Boyu Yang, Mingbao Lin; Yunxiao Zhang, Binghao Liu, Xiaodan Liang, Rongrong Ji, and Qixiang Ye. Dynamic support network for fow-shot class incremental learning. IEEE Transactions on Pattern Analysis and Machine Inlelligence, 45(3):2945 2951, 2022.
Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse in- spired feasure-classffer aligmment for few-shot class-ineremental learning. In The Eleventh International Conference Learning Representations, 2023.
Guangle Yao, Juntao Zhu, Wenlong Zhou, and Jun Li. Few-shot class-nneremental learning based on repre- sentation enhancement. Journal of Electronic Imaging, 3144):043027, 2022.
Zhuoning Yuan, Yuexin Wu, Zi-Hao Qiu, Xianzhi Du, Lijun Zhang, Denny Zhou, and Tianbao Yang. Prov- able stochastic optimization for global contrastive learning: Small batch does not harm performance. In International Conference on Machine Learning, pp. 25760-25782 25782. PMLR, 2022.
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12455 12464, 2021.
Linglan Zhao, Jing Lu, Yunlu Xu, Zhanzhan Cheng, Dashan Guo, Yi NIu, and Xiangzhong Fang. Few- shot class-incremental learning via class-aware bilateral distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11838-11847, June 2023.
Re-Wei Zhou, Fu- Yun Wang, Hin-Jia Ye, Liang Ma, Shiliang Pu, and De- Chuan Zhan. Forward compatible few-shot class-incremennal learning. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 9046 9056, 2022a.
Rei Zhou, Han-Jia Ye, Liang Ma, Di Xie, Shiliang Pu, and De-Chuan Zhan. Few-shot class-incremental learning by sampling multi-phase tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022b.
Kai Zhu, Yang Cao, Wei Zhai, Jie Cheng, and Zheng-Jun Zha. Self-promoted prototype refinement for few-shot class-increrenntal learning. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6801-6814 2021.
Huiping Zhnang, Zhenyu Weng, Run He, Zhiping Lin, and Ziqian Zeng. Gkeal: Gaussian kernel embedded analytic learning for few-shot class incremental task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition, pp. 7746 7755, 2023.
Yixiong Zou, Shanghang Zhang, Yuhua Li, and Ruixuan Li. Margin- based few-shot dlass-incremental leareeennnnalllannnng with class-level overfitting mitigation. Advances in neural information processing systems, 35:27267-27277 -72299, 2022.
Yixiong Zou, Shanghang Zhang, Haichen Zhou, Yuhua Li, and Ruixuan Li. Compositional few-shot class- incremental learning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 21-27 Jul 2024. URL
PMLR,
press/v235/ mlr press/ v235/ ha///////222222222 zoo24cc html
16

---

Published in Transactions on Machine Learning Research (11/2025)
HN Class
Anchor
EN Class
HN Class
Anchor
EN Class
HN Class
Anchor
EN Class
TOBACOO
minilmageNet
CIFAR100
CUB200
Figure Al: Example images of the classes selected as hard-negative (HN) and easy-negative (EN) relative to the anchor sample, across three widely used datasets in few-shot daas-incremennal learning (FSCIL): minilmageNet (Vinyals et al. 2016), CIFAR100 (Krizhevsky et al.) 2009), and CUB200 (Wah et al. 201 Note the strong visual similarity between the hard-negative class and anchor images, and the clear visual dissimilarity between the negative class and anchor images.
A Appendix
A.1 Further Experimental Results
In this section, we provide additional qualitative and quantitative results that further support our technical claims made the main paper.
Loss Ablation Study. Our proposed approach, defined in Equa- tion S in the main paper, consists three main loss terms in the denominator: intra-class cohesion (CA, with fy.), inter-class sep- aration (LB, with fj), and regularized inter-class separation with hard-negative protolype mining (Lc, with 0z). We perform ab- lation study to evaluate the impact of each loss term As shown in Table A1 we obscrve that (i) the model trained with all three loss terms achieves the best performance (compare the last row with others), and (ii) regularized inter-class separation with hard-negative prototype mining significantly improves performance when combined with the inter-class separation loss term. This improvement likely occurs because combining these two terms provides both global and local guidance, enabling the model to better align embeddings across different classes
Table A1: Performance comparison between variant models trained with different loss terms our method. CA: Intra-class cohesion, LB: Inter- class separation, and Le: Regular- ized inter-class separation with hard- negative prototype mining.
Ca LB
Lo Base
Last PD(L)
79.30
57.62
21.68
78.60
57.20
21.40
80.90
59.96
20.94
Negative Class Examples. To qualitatively evaluate our proposed method, we visualize example images from hard-negative and easy-negative classes relative to the anchor sample, using cosine distance the
17

---

Published in Transactions on Machine Learning Research (11/2025)
Table A2: Performance comparison with ViT-based state-of-the-att method, Attention-aware Self-adaptive
Prompt (ASP) (Liu ct al. 2024), on ImageNet-R (Hendrycks et al. margin loss. We mark the best value in bold.
2021
benchmark. AM denotes angular
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
PD(L)
ASP
81.34
78.46
77.96
74.72
73.72
72.73
71.05
70.29
69.98
68.58
67.07
14.27
ASP+AM
81.64
78.49
77.88
75.08
73.70
72.76
71.40
70.89
70.24
69.43
68.27
13.37
ASrs
81.38
78.81
78.60
75.54
74.50
73.64
72.12
71.22
70.51
69.46
68.55
12.83
Table A3: Mean and standard deviation across five independent trials with different random seeds on the miniImageNet benchmark. AM denotes the angular margin loss. We reproduce the TEEN (Wang et al. method using the official code repository.
2023
Method
TO
T1
T2
T3
T4
T5
T6
17
T8
PD(J)
TEEN
75.14=0.29
70.27:10.4
68. 29:1.444
63.1410.49
60.4520.48
57.7010.52
55.00=0.50
53.23:0.54
51.83:10522
23.3110043
AM
79.08-0.40
75.03+0.49
71.144.40
68.56+0.53
65.7840.48
63.07+0.53
60.26-0.36
38.62+0.37
37.78+0.27
21.91+0016
Ouss
79.9610..6
75.3610.50
71.5110.52
08.9320.75
66.31=0.74 63.7420.82
60.9110.71
59.6211.73
58.7310.77
21.24=0.30
similarity metric. As shown in Figure A1 images from hard-negative classes tend to be visually similar to the anchor samples, whereas those from easy-negative classes are noticeably different from the anchor images. These examples illustrate that our method effectively identifies challenging negative classes, highlighting its capability to enhance discriminative learning during model training.
VIT-Based FSCIL Evaluation. Recent work has begun to leverage the generalization capabilities of modern pre-trained Vision Transformer (ViT) (Dosovitskiy et al. 2020) models to FSCIL. To demonstrate the effectiveness of our approach on such architectures, we adopt the state-of-hhe-att ViT-based FSCIL method- Attention-aware Self-adaptive Prompt (ASP) (Liu et al. 2024) as our baseline and directly compare its performance to ours. Following the ASP evaluation protocol, we use the ImageNet-R, (Hendrycks et al., 2021) benchmark, a real-world distribution-shif dataset not previously explored in this context. As shown in the Table A2 hard-negative-bass based prototype regularization outperforms both vanilla ASP and ASP euhanced with an angular margin (AM). Note that use much larger ViT backbone (176M parameters) compared to ResNet-18 18 (20M), which makes our gains even more compelling.
Multi-Run Evaluation miniImageNNt. To validate the effectiveness of our proposed approach, we conducted five independent trials with different random seeds the minilmageNNet benchmark. We compare three different methods: (i) TEEN (Wang et al. 2023). the previous state-of-the-art method; (ii) the baseline (angular margin), as described in Section 3.1.; and (iii) our proposed method, hard-negative-bars prototype regularization. As shown in the Table A3 our method consistently achieves the best performance among the three. Note that we reproduced the TEEN method using its official code repository.
Comparison with Sample-Wis Wise Hard-Exampa Mining Method. To quantify the benefits of our prototype-based hard-negative prototype regularization (HNPR) over a conventional sample-wise hard ex- ample mining (HEM) approach, we compare against the standard triplet-loss (Weinberger & Saul, 2009) framework on the CUB200 benchmark under the FSCIL protocol. Table A4 presents the session-wise accu- racy of both methods: our HNPR consistently outperforms the triplet-loss baseline, demonstrating superior mitigation of class confusion and forgetting by leveraging class prototypes rather than individual hard sam- ples.
Extended Tables. We provide the complete tables comparing our proposed method against stato-of-the- art approaches on the CIFAR100 and CUB200 benchmark. Note that Tables 3 and 4 in the manuscript are the shorter version of Tables A5 and A6
18

---

Published in Transactions on Machine Learning Research (11/2025)
Table A4: Performance comparison with sample-wise hard-vegative mining method, triplet loss (Weineegee & Saul 2009) with different margin (m) values on CUB200 benchmark.
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
PD(J)
Triplet (m 0.3)
76.26
73.34
71.14
67 07.84
66.31
64.45
63.29
62.27
60.70
60.76
60.11
16.15
Triplet (m = 0.4)
76.47
73.59
71.52
68.08
66.34
64.49
63.24
62.25
60.53
60.63
59.91
16.56
Ours
77.44
74.58
72.27
69.23
68.02
65.88
65.32
63.86
62.43
62.40
61.98
15.46
Table A5:
Performance comparison with state-of-the-art methods CIPAR100 (Krizhevsky al. benchmark. We mark the best value in bold, and the second best with underline.
2009
Method
TO
T1
T2
T3
T4
T5
T6
T7
T8
PD(L)
TOPIC
Tao al.
2020
64.10
55.88
47.07
45.16
40.11
11
36.38
33.96
31.55
29.37
34.73
CLOM
Lou1 ct al.
2022
74.20
69.83
66.17
62.39
59.26
56.48
54.36
52.16
50.25
23.95
ALICE
Peng et a
2022
79.00
70.50
67.10
63.40
61.20
59.20
58.10
56.30
54.10
24.90
FACT
Zhoul of
2022a
74.60
72.09
67.56
63.52
61.38
58.36
56.28
54.24
52.10
22.50
LIMIT
Zhou et al.
20226
73.81
72.09
67.87
63.89
60.70
57.77
55.67
53.52
51.23
22.58
GKEAL
Zhuang
al.
2023
74.01
70.45
67.01
63.08
60.01
57.30
55.50
53.39
51.40
22.61
WaRP
Kim ct at.
2023
80.31
75.86
71.87
67.58
64.39
61.34
59.15
57.10
54.74
25.57
NoftNet
Kang et a..
2023
79.88
75.54
71.64
67.47
64.45
61.09
59.07
57.29
55.33
24.55
SAVC
[Song al..
2023
78.77
73.31
69.31
64.93
61.70
59.25
57.13
55.19
53.12
25.65
CaBD
Zhao el al.
2023
79.45
75.20
71.34
67.40
64.50
61.05
58.73
56.73
54.31
25.14
TEEN Wang al.
2023
74.92
72.65
68.74
65.01
62.01
59.29
57.90
54.76
52.64
22.28
OrCo [Ahmeed el al.
2024
80.08
68.16
66.99
60.97
59.78
58.60
57.04
55.13
52.19
27.89
Comp- FSCIL [Zou et al.
Comp-FSCIL
2024
80.93
76.52
72.69
68.52
65.50
62.62
60.96
59.27
56.71
24.22
Bag of Tricks [Rov et CLOSER (Oh et al. 2024
2024
80.25
77.20
75.09
70.82
67.83
64.86
62.73
60.52
58.55
21.70
75.72
71.83
68.32
64.62
61.91
59.25
57.53
55.43
53.32
22.40
Ours
77.38
73.57
70.74
66.87
64.30
61.71
60.19
58.23
55.98
21.40
A.2 Implementation Details
Hyperparameters. For all experiments, we use a ResNet-18 (RN18) backbone with a projection head of dimension 2048. The number of negative pairs varies depending on the dataset: we use k 1 for CUB200, and k for both CIFAR100 and minilmageNet.. We train our model using an SGD optimizer with the following learning rates: 3e-2 for minilmageNet, 6e-2 for CIFAR100, and 3e-3 for CUB200. Cosine scheduling with warm-up is employed for learning rate scheduling. For the warm-up, utilize 3% of the total epochs. The models are trained for 80 epochs on CUB200, 100 epochs on CIFAR100, and 300 epochs on minilmageNet, with batch sizes of 1024, 512, and 512, respectively.
Data Augmentation. We utilize the augmentation strategies frequently employed in prior FSCIL re- search (Yang et al.) 2023 Peng et al., 2022 Zhang et al. 2021 Tao et al. 2020]: (1) resized cropping, (2) horizontal flipping, and (3) color jittering. Given the variation in the size of input images, we adjust the size of the resized crop according to the dataset (224 for CUB200, 32 for CIFAR100, and 84 for minilmageNet). Additionally, horizontal flipping is applied the cropped images with a probability of 0.5. For color jittering, we establish parameter of 0.4 for the adjustment of brightness, contrast, and saturation.
19

---

Published in Transactions on Machine Learning Research (11/2025)
Table A6: Performance comparison with state-of-the-art methods on CUR200 Waah et al. We mark the best value in bold, and the second best with underline.
2011 benchmark.
Method
T0
T1
T2
T3
T4
T5
T6
T7
T8
T9
T10
PD(4)
TOPIC
Tao ct al.
2020
68.68
62.49
54.81
49.99
45.25
41.40
38.35
35.36
32.22
28.31
26.28
42.40
CLOM
Zou et
2022
79.57
76.07
72.94
69.82
67.80
65.56
63.94
62.59
60.62
60.34
59.58
19.99
ALICE
Peng al
2202
77.40
72.70
70.60
67.20
65.90
63.40
62.90
61.90
60.50
60.60
60.10
17.30
FACT
Zhou al.
2022a
75.90
73.23
70.84
66.13
65.56
62.15
61.74
59.83
58.41
57.89
56.94
18.96
LIMIT
Zhou et al.
20226
76.32
74.18
72.68
69.19
68.75 79
65.64
63.57
62.69
61.47
60.44
58.45
17.87
GKEAL
Zhuang et
al.] 2023
78.88
75.62
72.32
68.62
67.23
64.26
62.98
61.89
60.20
59.21
58.67
20.21
WaRP
Kim et al.
2023
77.74
74.15
70.82
66.90
65.01
62.64
61.40
59.86
57.95
57.77
57.01
20.73
NoftNet
Kang et
2023
78.07
74.58
71.37
67.54
65.37
62.60
61.07
59.37
57.53
57.21
56.75
21.32
SAVC
Song el al. 2023
81.85
77.92
74.95
70.21
69.96
67.02
66.16
65.30
63.84
63.15
62.50
19.35
CaBD
Zhao et al. 2023
79.12
74.99
70.87
67.30
65.89
63.45
61.40
60.11
58.61
58.23
57.48
21.64
TEEN
Wang et al.
2023
77.26
76.13
72.81
68.16
67.77
64.40
63.25
62.29
61.19
60.32
59.31
17.95
OrCo
Ahmned et al. 2021
75.59
71.73
64.48
60.83
60.66
58.80
59.29
58.73
58.01
59.02
58.84
16.75
Comp-F Comp-FSCIL FSCIL Zou et al.
2024
80.94
77.51
74.34
71.00
68.77
66.41
64.85
63.92
62.12
62.10
61.17
19.77
Bag of Tricks Rov
2024
82.31
78.03
75.45
70.99
71.06
67.85
67.44
66.05
64.95
64.31
63.60
18.71
CLOSER (Oh et al. 2021
79.40
75.92
73.50
70.47
69.24
67.22
66.73
65.69
64.00
64.02
63.58
15.82
Ours
77.44
74.58
72.27
69.23
68.02
65.88
65.32
63.86
62.43
62.40
61.98
15.46
20


Reviews:
[{"review_id": "OsWV0RevB4", "reviewer": "Reviewer_aYjG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n- **Compelling empirical motivation**: Provides strong empirical analysis showing hard samples cause disproportionate forgetting in FSCIL (Figure 2), with clear delta analysis demonstrating confusion concentration among a few classes.\n\n- **Comprehensive experimental validation**: Achieves consistent state-of-the-art results across three standard benchmarks with thorough ablation studies, meaningful comparison between mining strategies, and good qualitative analysis.\n\n- **Clear mechanistic insights**: Offers insights through t-SNE visualizations and distance measurements demonstrating improved embedding structure, with effective presentation and reproducible methodology.\n\n- **Practical effectiveness**: Explores both dynamic and static hard-negative selection strategies with thorough comparison, showing consistent improvements across different experimental settings.\n\n### Weaknesses\n\nMy main concerns lie in three points:\n\n- **Limited technical novelty**: The core contribution applies well-established hard-negative mining to FSCIL through straightforward modification of angular margin loss. The \"dynamic selection\" does not truly mine hard negatives since changing prototypes makes selection somewhat arbitrary, differing significantly from established hard-negative mining approaches.\n\n- **Weak theoretical foundation**: No theoretical analysis explains why this specific regularization works or why class-level hard-negative selection outperforms sample-level approaches from metric learning. The claim that dynamic selection introduces beneficial \"noise\" lacks proper validation and seems speculative.\n\n- **Experimental limitations**: Missing statistical significance testing, computational overhead analysis, and comparison with recent hard-negative mining advances. The static selection underperformance using pre-trained models suggests the approach may not capture meaningful semantic similarities as claimed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses catastrophic forgetting and overfitting in few-shot class-incremental learning (FSCIL) by proposing Hard-Negative Prototype Regularization (HNPR). The authors identify that hard-negative samples and classes significantly impact FSCIL performance through empirical analysis, then develop a regularization approach that imposes penalty margins between samples and their most similar class prototypes. Two mining strategies are explored: dynamic selection using model decision boundaries and static selection using pre-trained encoders. Extensive experiments on miniImageNet, CIFAR100, and CUB200 demonstrate state-of-the-art performance with improved intra-class cohesion and inter-class separability.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the Weakness.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rXy6vl1wj6", "reviewer": "Reviewer_XGo1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n1. Clean idea, easy to implement. The method just adds one more margin term into a known loss (angular margin loss), so it doesn’t require new architectures or fancy backbones.\n\n\n**Weakness:**\n1. **Motivation is overstated.** The idea of focusing on hard-negative examples is far from new. It’s a well-established concept in metric learning, face recognition (e.g., FaceNet, ArcFace), and even in some FSCIL methods like[3] [4] that implicitly push apart similar classes.\nAlso, related work section lists metric-learning papers but does not discuss why their hard-negative strategies cannot be directly applied , weakening the novelty argument.\n\n[1]FaceNet: A Unified Embedding for Face Recognition and Clustering (CVPR 2015)\n\n[2] ArcFace: Additive Angular Margin Loss for Deep Face Recognition\n\n[3] Learning with Fantasy: Semantic‑Aware Virtual Contrastive Constraint for Few‑Shot Class‑Incremental Learning (CVPR 2023)\n\n[4] Rethinking Few‑shot Class‑incremental Learning (ECCV 2024)\n\n2. Adding a uniform extra margin for all hard-negative classes ignores that some pairs may already be well-separated, potentially over-penalising and harming calibration. No analysis of failure cases or calibration metrics is given.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Hard-Negative Prototype Regularization (HNPR) for FSCIL. The method adds an extra angular margin between a sample and its most similar (hard-negative) class prototypes to reduce confusion. Two mining strategies are explored, dynamic (from model predictions) and static (from pretrained encoders).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify novelty: The use of hard-negative mining is well-established in metric learning and has appeared in FSCIL. The paper must clearly state what is actually new here. e.g., prototype-level margin within the FSCIL angular-margin framework, and avoid overstating “first” claims.\n\n2. Improve related work discussion: Expand the related work section to properly acknowledge prior literature on hard-negative mining in metric learning, few-shot learning, and FSCIL. Show how your method differs technically and conceptually.\n\n3.Discuss potential drawbacks of dynamic mining: Batch-level hard-negative selection could introduce noise or instability. Include a discussion (or experiment) on whether this affects training dynamics, especially under small batch sizes or label imbalance.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wp36HAynEq", "reviewer": "Reviewer_4yva", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS:\nThe approach is straightforward and well motivated.\n\nThe results are on par with other methods, although I would disagree with the claim that the method has been demonstrated to be superior than the others.  \n\nThe writing is clear and easy to follow. \n \nWEAKNESSES:\nWhile the application to FSCIL may be novel, the importance and use of hard example learning for few shot learning is well established.  It would be helpful to articulate early on why hard example learning within FSCIL is sufficiently novel and impactful as opposed to an application of a standard framework (HEM) to a specific newer method (FSCIL).  Grounding this approach in other HEM works applied to other few shot methods would help with this.  Or for example, a sentence like \"We... observe that hard-negative samples and classes are key to success in FSCIL\"- this observation is seen in other few shot approaches so it would be helpful to articulate (and cite) that.  Works within metric learning are referenced, but expanding to the broader one-shot and few-shot learning literature would be helpful.\n\nThe datasets selected are relatively small by today's standards.  Additionally, they are all classification focused whereas much of the interest in FSL approaches is for detection and segmentation.  While these are the same datasets that were used in FSCIL, that work was more theoretically driven/motivated so these smaller, simpler, toy datasets are more justified.  \n\nSimilarly all experiments are conducted using Resnet-18, which is a very small model by today's standards.  For theoretical proof, a small model is fine (i.e. to demonstrate that a method produces correct results).  However, to demonstrate the usefulness of a method, particularly something like this where the primary application would be in large scale, real-world datasets, likely leveraging substantially larger models, using a more relevant model would make more sense.  ResNet-18 and toy datasets establish that the approach works, but something more substantive is needed to demonstrate that the approach is useful for this particular task.  \n\nThe results in Table 2 suggest that while negative samples, especially hard-negative samples, lead to improvement (a result which has been seen with other approaches in metric learning, HEM, etc.), the baseline and proposed approaches are not meaningfully different.  \n\nError bars are provided with Table 2, but not elsewhere.  Changing the classes which are novel, or changing which specific samples are used could be done to show significance on the other results as well.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a hard-negative prototype regularization approach which imposes a penalty margin between samples and their similar class prototypes.  They explore dynamic and static training selection strategies.  The approach is validated on miniImageNet, CIFAR100, and CUB200.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please run repeat trials on at least one of the datasets (Tables 1,3,4) so we can see the significance of the approaches.\n\nPlease address any compute considerations which may make the proposed method advantageous/disadvantageous, especially compared to other methods.  Are there requirements around memory, compute time, total number of classes, etc.?  Speaking to limitations of the current method is also important in general.  \n\nI think demonstrating results on a more modern model, much larger than ResNet, is important for demonstrating usefulness of the approach.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)