Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration
Authors: Qi-Wei Wang
Da-Wei Zhou
Yi-Kai Zhang
De-Chuan Zhan
Han-Jia Ye
Venue: NeurIPS 2023 poster (2023)
Keywords: Few-Shot Class-Incremental Learning
Continual Learning
Class-Incremental Learning

Abstract: Real-world scenarios are usually accompanied by continuously appearing classes with scare labeled samples, which require the machine learning model to incrementally learn new classes and maintain the knowledge of base classes. In this Few-Shot Class-Incremental Learning (FSCIL) scenario, existing methods either introduce extra learnable components or rely on a frozen feature extractor to mitigate catastrophic forgetting and overfitting problems.  However, we find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. In other words, the strong discriminability of base classes distracts the classification of new classes. To figure out this intriguing phenomenon, we observe that although the feature extractor is only trained on base classes, it can surprisingly represent the *semantic similarity* between the base and *unseen* new classes. Building upon these analyses, we propose a *simple yet effective* Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes by fusing the new prototypes (i.e., mean features of a class) with weighted base prototypes. In addition to standard benchmarks in FSCIL, TEEN demonstrates remarkable performance and consistent improvements over baseline methods in the few-shot learning scenario. Code is available at: https://github.com/wangkiw/TEEN

Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration
Qi-Wei Wang, Da-Wei Zhou, Yi-Kai Zhang, De-Chuan Zhan, Han-Jia Ye*
State Key Laboratory for Novel Software Technology
Nanjing University, Nanjing, 210023, China
{wangqiwel zhoudw, zhangyk, yehj) .nju edu. cn
Abstract
Real-world scenarios usually accompanied by continuously appearing classes with scarce labeled samples, which require the machine learning model to in- crementally learn new classes and maintain the knowledge of base classes. In this Few-Shot Class-Incremental Learning (FSCIL) scenario, existing methods either introduce extra learnable components rely frozen feature extractor to mitigate catastrophic forgetting and overfitting problems. However, we find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. In other words, the strong discriminability of base classes distracts the classification of new classes. To figure out this intriguing phenomenon, we observe that although the feature extractor is only trained on base classes, it can surprisingly represent the semantic similarity between the base and unseen new classes. Building upon these analyses, we propose simple yel effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes by fusing the new prototypes (i.e., mean features of class) with weighted base prototypes. addition standard benchmarks in FSCIL TEEN demonstrates remarkable performance and consis- tent improvements over bascline methods in the few-shot learning scenario. Code is available at:
com wwwom//wagkiw//eee TEEN
Introduction
Deep Neural Networks (i.e., DNNs) have achieved impressive success in various applications [16 5), but they usually rely heavily on static and pre-collected large-scale datasets (e.g., ImageNet (11) to achieve this success. However, the data in real-world scenarios usually arrive continuously. For example, the face recognition system is required to authenticate existing users, and meanwhile, new users are continually added [30]. The scene where the model is required to continually learn new knowledge and maintain the ability on old tasks is referred to as Class- Incremental Learning (CIL) M.. The main challenge of CIL is the notorious catastrophic problem 14] where the model forgets old knowledge as it learns new ones.
19
36 55
54
Many methods have been designed to overcome catastrophic forgetting in CIL from different perspectives, e.g., knowledge distillation [35), parameter regularization [22 [33] and network expan- sion (50 47 [66] These methods require
tasks contain sufficient labeled data for supervised training. However, collecting enough labeled data in some scenarios is challenging, making the conventional CII. methods hard to deploy (30). For example, the face recognition system can only collect very few facial images of a new user due to privacy reasons. Therefore, more realistic and practical incremental learning paradigm, Few-Shot Plass--ncremental Learning (FSCIL) [42] is
'Han-Jia Ye is the corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

proposed to address the problem of class incremenllll learning with limited labeled data. Figure la gives a detailed illustration of FSCIL.
In addition to the catastrophic forgetting prob- lem [14), FSCIL will suffer the overfitting prob- lem because the model can easily overfit the very few labeled data of new tasks. Previous works [57 58 6 have adopted prototype-based methods 40 to conquer the limited data problem. These methods freeze the feature ex- tractor trained on base classes when dealing with new classes and use the prototypes of new classes as the corresponding classifier weights. The frozen feature extractor can alleviate the catastrophic forgetting problem and the proto- type classifier can circumvent the overfitting problem. However, the inherent problem of learning with few-shot data is challenging to depict precisely the semantic information of new category with limited data, making pro- totypes of new classes inevitably biased (27]. An intuitive reason for the biased prototypes is that the feature extractor has not been opti- mized for the new classes. For example, the feature extractor trained on "cat" and "dog" can not precisely depict the feature of a new class "bird" especially when the instances of "bird" incredibly scarce Many prototype adjust-
Base Classes
Novel Classes
Class:
Class:
Class: 6
Class:
Class: 3
Class:
Class: 7
Base Session
Session
Session 2
(a) An illustration of FSCIL setting.
"Plate"
"Bowi"
"Clock""
"Peecspine"
Novel Classes
(b) An illustration of the semantic similarity.
Figure I: (a) The base session contains sufficient sam- ples of base classes for training. The incremental ses- sions (i.e., sessions after the base session) only contain few-shor samples of new classes. FSCIL aims obtain a unified classifier over all seen classes. Notably, the data from previous sessions are unavailable. (b) For ex- ample, considering the new classes "Plain", "Plate" and "Porcupine" in CIFAR 100 [23], the corresponding most similar base classes selected by the similarity can depict the really semantic similarity. The three most similar base classes are selected by computing the cosine sim- ilarity between base prototypes and novel prototypes. The results show the feature extractor only trained on the base classes can also well represent the semantic similarity between the hase and new classes.
I (24] are dedicated rectifying the biased representa- tions of the new classes. These methods usually focus on designing complex pre-training algo- rithms to enhance the compatibility of represen- tations [58 59 10], or complicated trainable modules that better adapt the representation of the new classes (57 15 M1 all of which rely
ment methods [27 [5
58
15
61
on significant training costs in exchange for improved model performance.
However, we find that although existing methods perform well on the widely used performance measure (i.e., the average accuracy across all classes), they usually exhibit poor performance on new classes, which suggests that the calibration ability of existing methods could be improved. In other words, existing methods neglect the performance of new classes. A direct reason for this negligence the current unified performance measure is easily overwhelmed by the dominated base classes (e.g.. 60 base classes in the 100 dataset). However, the more recent tasks are usually more crucial in some real-worke applications. For example, the new users in the recommendation system are usually more important and need more attention [45). The important yet neglected new classes' performance inspires us to pay more attention to it.
To better understand the performance of current FSCIL methods 42 6.
57], we explicitly evaluate and analyze the low performance of existing methods on new classes and empirically demonstrate the instances of new classes are prone to be predicted into base classes. To further understand this intriguing phenomenon, we revisit the representation of the feature extractor and find that the feature extractor trained only on base classes can already represent the semantic similarity between the base and new classes well. As shown in Figure [bb] although the novel classes are unavailable during the training of the feature extractor, the semantic similarity between the base and novel classes can also be well-represented. However, existing methods have been obsessed with designing complex learning modules and disregard this off-the-shelf semantic similarity.
58
2 "new class" and "novel class" in this paper mean the classes emerging after the base session.
2

---

Based on the above analysis, we propose to leverage the overlooked semantic similarity to explicitly enhance the discriminability of new classes. Specifically, we propose simple yet effective Training- frEE prototype calibratioN (i.e. TEEN) strategy for biased prototypes of new classes by fusing the biased prototypes with weighted base prototypes, where the weights for the hase prototypes are class-specific and semantic-aware. Notably, TEEN does not rely any extra optimization procedure or model parameters once the feature extractor is trained on the base classes with a vanilla supervised optimization objective. Besides, the excellent calibration ability and training-free property make it blug-and-play module.
Our contributions can be summarized as follows:
We empirically find that the lower performance of new classes is due to misclassifying the samples of new classes into corresponding similar base classes, which is intriguing and missing in existing studies.
We propose a simple yet effective training- free calibration strategy for new prototypes, which not only achieved a higher average accuracy but also improved the accuracy of new classes (10.02% ~ 18.40% % better than the runner-up FSCIL method).
We validate TEEN on benchmark datasets under the standard FSCIL scenario. Besides, TEEN shows competitive performance under the few-shot learning scenario. The consistent improvements demonstrate TEEN's 's remarkable calibration ability.
2 Related Works
2.1 Class Incremental Learning
The model in the Ilass-nncremental Learning (i.e., CIL) scenario is required to learn new classes without forgetting old ones. Save representative instances in old tasks (i.e., exemplars) and replay simple and effective way to maintain the model's discriminability ability on
them in new tasks Furthermore, knowledge distillation 17 K5
old tasks 20 2 37
25 49) is widely used to maintain the knowledge of the old model by enforcing the output logits between the old model and the new one to be consistent. iCaRL (35] uses the knowledge distillation
regularization item and replays the exemplars when learning the feature representation. Many methods follow this line and design more elaborate strategies to replay exemplars [29] and distill knowledge [2] Recently, model
expansion [28 50 47 46 39 60 have been confirmed to be effective in CIL. The most representative
method [50] saves single backbone and freezes it for each incremental task. The frozen backbone substantially alleviates the catastrophic forgetting problem.
2.2 Few-Shot Learning
The model in Few-Shot Learning (i.e., FSL.)
48 scenario is required to learn new classes with limited labeled data, Existing methods usually achieve this goal either from the perspective of optimization 13 31 or metric learning 40 43 26
27 56). The core thought of aptimization-basee methods is to equip the model with the ability to fast adaptation with limited data. The metric-base methods focus on learning unified and general distance measure to depict the semantic similarity between instances. Besides, (5T) proposes to use Gaussian distribution to model each feature dimension of a specific class and sample augmented features from the calibrated distribution. Based on these augmented features, (5T] train logistic regression and achieve competitive performance. However, TEEN can outperform [5l] in most settings and without any training cost when recognizing new classes.
2.3 Few-Shot Class-Incremental Learning
The model in Few-Shot Innse-Innremettll Learning (i.e., FSCIL)
42 scenario is required to incre- mentally learn new knowledge with limited labeled data. Many existing methods are dedicated to designing learning modules to train more powerful feature extractor [58 or adapting the representa-
42 utilizes a neural gas network to alleviate the challenging problems in FSCIL. 8 adopts word embeddings as semantic information and introduces distillation-based FSCHL method. IDLVQ 6
tion of the instances of new classes [57 9 61 I.. Besides, TOPIC
proposes to utilize quantized reference vectors to compress the old knowledge and improve the performance in FSCIL. However, all these
3

---

methods overlook the abundant semantic information in base classes and the poor performance in new classes. In this study, we aim take a small step toward filling this gap. The most related work to TEEN is [T] However, it relies on optimizing a eggularization-aase objective function to implicitly utilize the semantie information. As opposed to [ TEEN takes advantage of the empirical observation and gets rid of the optimization procedure and is thus more efficient and effective.
3 Preliminaries
3.1 Definition and notations
In FSCIL 422. we assume there exists 7 sessions in total, including a base session (i.e., the first ses- sion) and - incremental sessions (i.e. sessions after the first session). We denote the training data in the base session as Do and the training data in the incremental sessions as (D1, D2,..., Dr-1h- For the training data D; in the i-th session, we further notate it with { ) and corresponding label space with C,. Note that only training data DD is available in the i-th session. Accordingly, the testing data and testing label space in session can be denoted as Diest and Ciest To better evaluate the model's discriminability on all seen tasks, the testing label space Crest of i-th session contains all classes during training, i.e., Ctest = -ن= Cy. An incremental session can also be denoted as a N-way K'shot classification task, i.e., N classes and K labeled examples for each class. Note that the training label spaces between different sessions are disjoint, i.e., for any i [0, - 1] and i j.C, nC₁ Ø.
Compared to conventional CIL, FSCIL only requires the model learn new classes with limited labeled data. On the other hand, compared to conventional Few-Shot Learning (FSL), FSCIL requires the model to continually learn the knowledge of new classes while retaining the knowledge of previously seen classes. We introduce the related works on CIL, FSL and FSCIL in supplementary due to the space limitation.
The model in FSCIL can be decoupled into a feature encoder (-) with parameters and a linear classifier W. Given a sample zj E IR,, the feature of z, can be denoted as do(x)) E Rd. For N- class classification task, the output logits of a sample zy can be denoted as O2 = W TT(())))) e R where W R4XN
3.2 Prototypical Network
ProtoNet 40 is a widely used method in few-shot learning problems. It computes the mean feature Ck of class (e.g., class prototype) and uses the class prototype to represent the corresponding class:
Ck
Numg
By=k
Num, denotes the number of samples in class k. For a classification task with N classes, the classifier can be represented by the N prototypes, i.e., W [c1, C CN.. Following the [58] 57 61 [] we freeze the feature extractor d9 trained on the base task and plug the class prototype into the classifier while dealing with new class. The frozen feature extractor can alleviate catastrophic forgetting and
the plug-in updating of the classifier can circumvent the overfitting problem relatively.
4 A Closer Look at FSCIL
In this session, comprehensively analyze current FSCIL. methods from decoupled perspective. Although the previous updating paradigm of extractor-frozen and prototypes-plugee can achieve adequate average accuracy in all classes, there also exist some shorteomings in it. In this section, empirically show that the current methods (e.g., [57 588) are generally not effective in new classes. Furthermore, we take a step toward understanding the reason for the low performance in new classes.
4.1 Understanding the reason for poor performance in new classes
To better understand the performance of existing methods, we first measure the performance by average accuracy on all classes, base classes and new classes, respectively. As illustrated in Figure 2

---

Table Detailed prediction results False Negative Rate/False Positive Rate (%) on CIFAR1O0 23 dataset. The analysis results are from session I because new classes do not exist in session O. Exceedingly high FPR and relatively low FNR show the instances of new classes are easily misclassified into base classes and the instances of base classes are also casily misclassified into base classes. TEEN achieve relatively lower FPR than baseline methods, which demonstrates the validity the proposed calibration strategy. Please refer to the supplementary for results on mindImageNet and CUB200.
Session
2
3
5
6
8 FNR
FNR/FPR
FNR FPR FNR FPR
FNR FPR
FNR
FPR
FNR
FPR
FNR
FPR
FNR
FPR
FPR
ProtoNet 40
2.13 67.40 4.42 67.40 6.68 60.67
8.28 58,90
10.25 56.68
11.40
12.57 51.66
13.78 51.80
CEC [57
2.32 70.40 4.38 66.20 6.18 62.20
7.50
58.65
9.72
56.00
11. 53.60
12.
51.40
13.48 51.78
FACT (58]
22.05 66.60
3.88 61.70
5.58 56.80
7.23
55.05
8.85 53.64
9.83 51.13
10.45
48.83
11.75 49.27
TEEN
|4.03 57.40 7.40 52.50 9.35 45.00) 1.58 40.75
14.75 40.80 15.78 39.233992221 16.33 36.91 18.75 36.65
14.00
Table 2: Detailed prediction results of TNR/TBR (%) on CIFARI00
dataset. The analysis results from session because new classes not exist session For new classes, we only consider the 10 most similar base classes out of 60 base classes. For base classes, we suppose C, new classes exist in the current incremental session i. We only consider the most similar [20% x C.J new classes. Class similarity adopts cosine similarity between different class prototypes. TEEN can achieve relatively lower TBR than baseline methods, which demonstrates the validity of the proposed calibration strategy. Please refer to the supplementary for results on minilmageNet and CUB200.
Session
2
3
4
6
7
8
TNR/TBR
TNR TBR /NNR TBR
TNR TBR
|NNR TBR
TNR TBR
TNN TBR
TNNR TBR
TNR TBR
ProtoNet 40
3.47 73.01| 5.02 61.79 7.06 55.051 52.95
7.14 53.27 51.96
8.53 49.18 9.21 50.91
CEC 57
2.87 71.70
4.53 61.08 5.66 58.26 6.35 55.16 6.07 54.12
6.92
52.53 8.24 49.96
8,20 51.64
FACT 58
2.33 70.00
3.32 61.01
5.50
55.17
6.88 50.55
6.57 50.52
7.72 49.30
8.88 46.99
9.31 48.48
TEEN
2.116 2.53 55.14 3.55 44.87 3.85 37.68 3.57 39.47] 4.02 37.07| 4.76 33.83 4.85 35.04
3.85
our first observation is that the average accuracy across base classes extremely higher than the accuracy in new classes. The inconsistent performance between the base and new classes is caused by the frozen feature extractor and biased new prototypes. The former forces the model to overfit base classes and the latter causes the model to underfit the new classes.
To determine the real cause of the low performance on new classes, we further investigate "What base classes are the new classes incorrectly predicted into?" and get our second observation.
(ancy(y)) (%)
NM,ec
AArgcc
-BAcc
Our second observation is that the prototype-but peabesed classifier misclas- sifies the new classes to their corresponding most similar base classes with high probability, i.e., many instances of new classes are closer to their nearest base prototype than corresponding target prototypes
Accuracy
22
To verify this observation, we first analyze the detailed prediction results from decoupled perspective. Specifically, we treat all base classes as a "positive class" and all new classes as a "negative class" and transform the FSCIL problem into a two-class classification task. We compute the false negative rate (i.e., FNR) and false positive rate (i.e., FPR) of each binary prediction task each incremental session. The FNR and FPR in the confusion matrix is defined as follows:
Incremental Session
Figure 2: The perfor- mance of FACT (58) on CIFARI00 datasct. NAcc,BAcc, AAccA,,AAA.Acc mean the average accuracy on new classes, base classes, and all classes, respectively.
FN
FP
FNR
100%, FPR
X 100%
TP + FN
FP F TN
As shown in Table I the FPR far greater than FNR illustrating new classes are generally mis- classified into base classes but the base classes are generally misclassified into base classes. On the basis of this conclusion, we further explore the details of misclassification. To better illustrate the analysis, we define "misclassified To most similar Base classes Ratio" (i.e., TBR for new classes and "misclassified To most similar New classes Ratio" (i.e. TNR ) for base classes Specifically, considering the misclassified instances in base classes and new classes respectively, the TBR and
5

---

Incremental Session]
Sample lype
UC
W >R R +W| UC W-RR W
Base Class
93.08 8.51 76.92
87.51
68.03 12.49 92.05 31.97
7.95
New Class
6.92 91.49 23.08
(a) An qualitative illustration the calibration
(b) An quantitatiiv illustration the calibration
Figure 3: (a) The O represents the samples of new classes and the A represents the samples of base classes. The dotted lines between samples and prototypes (i.e., c,,c, ..,) represent the corresponding classification results. Blue and purple dotted lines represent samples that are correctly classified. and red dotted lines represent misclassification. The yellow circles (i.e.. W-RR samples) and green triangles (i.e., R- samples) in the right figure are samples with prediction changes after calibration. (b) The detailed ratio (%) of base and new classes with regard to three types of samples(i.e. UC samples, W-R samples, R-W samples). Only two incremental sessions (i.e, session and session 2) of CIFARIO0 are listed here for convenience. Please refer to the supplementary for results more incremental sessions and more datasets.
TNR are defined as follows:
M
Mn. No
TBR
100%, TNR N₂
100%
N₂ means the number of misclassified instances of new classes. M, is the number of instances misclassified into most similar base classes in N,, samples. M,, and N₂ have similar meaning. As shown in Table 2 the TBR is higher and the TNR is relatively low, which strongly underpins our second observation. To our knowledge, we the first to explain the reason for the low performance of new classes in existing methods and observe that the samples of new classes are easily misclassified into the most similar base classes.
To summarize, we empirically demonstrate that existing methods perform poorly on new classes and find that this is because the model tends to misclassify new class samples into the most similar base classes. Combining existing analysis (i.e., the samples of new classes tend to be classified into the most similar base classes) and observations (i.e., the well-trained feature extractor on base classes can also well represent the semuntic similarity between the base and new classes), we propose to use the frozen feature extractor bridge and calibrate the biased prototypes of new classes by fusing the biased new prototypes with the well-calibrated base prototypes.
5 Similarity-based Prototype Calibration
Based on the common assumptions in FSCIL during the training of the feature extractor. We argue the sufficient data in the base session contains abundant semantic information (i.e., sufficient number of classes) and the base prototypes are well-calibrated (i.e., sufficient number of instances for each class). The above analysis leads to natural question:
sufficient instances of base classes are available
Can leverage the well-calibrated prototypes in the base session for new prototype calibration?
As the previous methods overlook the low performance of new classes caused by the corresponding biased prototypes, we propose to explicitly calibrate these biased prototypes with the help of the bridge between
well-calibrated base prototypes. The off off-the-shelf semantic similarity serves the base prototypes and the new ones. In the following sections, we introduce the details of fusing the well-calibrated base prototypes and ill- ill-calibrated new prototypes to calibrate the biased ones. Afterwards, we analyze the effects of this training-free prototype calibration on new classes and base classes, respectively.
5.1 Fusing the biased prototypes with calibration item
We assume there exist B classes in the base session and C classes an incremental session, e.g., B 60, C 5 in the CIFAR 100 dataset. Without loss of generality, we only consider the base session and the first incremental session for simplification. Other incremental sessions can be obtained in the same way. Following the notations in section 3.1 the empirical prototype of i-th class can

---

be notated as c, Therefore, the base prototypes are co B and the new prototypes are C, S n I B 1). As the base session contains sufficient classes and sufficient samples for each class, the model trained on the base session can capture the distribution of base classes and obtain well-calibrated prototypes for base classes. Generally, the empirical prototype of base classes can be regarded as approximately consistent with the expected class representation. However, due to the limited data in incremental sessions, the empirical prototypes the novel classes are considered to be severely biased.
Based on the analysis of prototypes, we only calibrate biased prototypes in incremental sessions. Given a new class prototype en (B I n < + + 1), the calibrated new prototype & can be notated by:
en (1 a)e, + adc.
(4)
The calibration item Ac, is a component of base prototypes. The hyperparameter a controls the calibration strength of biased prototypes. Smaller means the calibrated prototype reflects more of the original biased prototype, while larger a means the calibrated prototype heavily incorporates the base prototypes. Motivated by the observations in section 4.1 the similarity between well-calibrated base prototypes and ill-calibrated new prototypes contains auxiliary side information about the new classes. Therefore, we use weighted base prototypes to represent the calibration item Ac, and enhance the discriminability of biased prototypes. Specifically, we compute the cosine similarity Sh, between new class prototype On. and base prototype es:
Cb Cp.
Sb.
(5)
|aal| ||
||c,]]
where 7 (r 0) is the scaling hyperparameter controlling the weight distribution's 's sharpness. The weight of a new prototype en with respect to a base prototype o is the softmax results over all base prototypes:
So.
Wb.n
B-1 SS...
(6)
Finally, the calibration of biased prototypes of new classes can be formulated as follows:
B Σ
Tn = (1 - a) ca + a Ac, (1 a) en
Wb.n
Notably, the above prototype rectification procedure is training-free calibration strategy because does not introduce any learning component or training parameters. Figure Bal gives an intuitive description of the calibration effect.
5.2 Effect of calibrated prototypes
Intuitively, Wa,r is larger when a new prototype Cn and base prototype cG are more similar. Given new prototype C,, we assume the most similar base prototype with c, the base prototype chase Therefore, given a new prototype Cu. B-1 Wb.nCo Rs baae when the scaling hyperparameter is large enough. From the perspective of a biased prototype, calibrated prototype č, will be aligned to its most similar base prototypes 6, with a proper r.
To further comprehend the effect of TEEN on the predictions of base and new classes respectively, we define three types of test samples according to whether the prediction results change after TEEN: with unchanged predictions (i.e.. UC samples), with prediction going from right to wrong (i.e., R-W samples), with predictions going from wrong to right (i.e., W R samples). We analyze the specifics of these three types of samples in detail.
Intuitively, some calibrated prototypes of new classes are aligned base prototypes and reduce the discriminability of base prototypes. Oppositely, aligning the biased prototype to most similar base prototypes can calibrate the prediction of new classes. As shown in Table we observe the W-R samples are mainly from new classes and the R-W samples are mainly from base classes. Besides, extensive comparison results on the benchmark datasets (i.e., Table and Figure 4 show that the negative effect of TEEN is negligible due to the significance of TEEN.
7

---

Table 3: Detailed average accuracy of each incremental session mindmageNet dataset. Please refer to the supplementary for results on CUB200 and CIFARI00. The results of compared methods means higher accuracy is better. means lower PD is better.
cited from 42 57 58
Accuracy in each session (%) t
Method
PD L
A PD
2
7
iCaRL 35
61.31
46.32
42.94
37.63
30.49
24.00
20.89
18.80
17.21
44.10
+22.65
EEIL [
61.31
46.58
44.00
37.29
33.14
27.12
24.10
21.57
19.58
41.73
+20.28
Rebulancing
18
61.31
47.80
39.31
31.91
25.68
21.35
18.67
17.24
14.17
47.14
+25.69
TOPIC 42
61.31
50.09
45.17
41.16
37.48
35.52
32.19
29.46
24.42
36.89
+15.44
Recoupeed-NeGoosee
26
71.68
66.64
62.57
58.82
55.91
52.88
49.41
47.50
45.81
25.87
+4,42
Decoupled-Conee 43
70.37
65.45
61.41
58.00
54.81
51.89
49.10
47.27
45.63
24.74
+3.29
56
69.77
60.21
56.63
53.16
50.13
47.79
45.42
26,36
+4.91
CBC 153
72.00
66.83
62.97
59,43
56.70
53.73
51.19
49.24
47.63
24.37
+2.92
FACT 58
72.56
69.63
66.38
62.77
60.6
57.33
54.34
52.16
50.49
22.07
+0.62
TEEN
73.53
70.55
66.37
63.23
60.53
$7.95
$5.24
53.44
52.08
21.45
TTTCCC + DuxpEMD
+-CBC FACT
10
EBIL
Aocuracyy (14)
40-
MINCT
Accuracy (%)
g-Rabiizndng
+-BEIL
DDaEED
--TIIN
FMT
2.571
Asouracy
11541
42
20
Incremental Session
Incremental Scasion
Incremental Session
(a) minilmageNet
(b) CUB200
(c) CIFAR100
Figure 4: Top-1 average accuracy on all seen classes in each incremental session. We annotate the performance gap between TEEN and the runner-up method f.
Experiments
In this section, we first introduce the main experiment details of FSCIL in section 6. which include the implementation and performance detail. Subsequently, we introduce the FSL performance of TEEN in section 6.2 which consistently shows the effectiveness of TEEN. Finally, we evaluate TEEN by a comprehensive ablation study in section 66.
6.1
Main experimental details of FSCIL
6.1.1
Implementation Details
we evaluate TEEN on CIFAR100 [33] CUB200-2011 [44, minilmageNet (38). We keep the dataset split consistent with Notably, each benchmark dataset divided into subsets containing nonoverlapping label space. For example, C1FAR100 is divided into 60 classes for the base session and the left 40 classes are divided into eight 5-way 5-shot few-shot classification tasks. To validate the performance of TEEN, 56 (3), and FSCIL methods 42 57 58
Datasets and baseline details: Following previous methods 442 57 58
existing methods (57 58
compare TEEN with popular CIL methods (35 4 [8] FSL methods [26
Please refer to the supplementary material for more datasets
and baseline details.
Training details: All experiments are conducted with PyTorch B2 on a single NVIDIA 3090. The training of the feature extractor uses vanilla cross-entropy loss as the objective function. It does evolve any extra complex pretraining module [58 59 TO 57], thus making TEEN more efficient and elegant. In addition, we adopt the cosine similarity to measure the similarity between the instances and class prototypes. Following 42 157 158), we use ResNet20 [6] for CIFAR
pre-trained ResNet18 18 [16] for CUB200 and randomly initialized ResNet18 8 [16] for minilmageNee.. All compared methods use the same backbone network and initialization for fair comparison. We set = 0.5, 16 for minilmageNet and CUB200, = 0.1, 16 for CIFAR 100. We train the feature extractor on CUB200 with learning rate of 0.004, batch size of 128, and epochs of 400. Please refer to the supplementary for more training details CIFAR 100 and minImmgeNNet.

---

Table 4: Detailed results of HMMen and NAce on mindIaageNei. The best results are bold and the runner-up results are in underlines. The A measures the performance gap between the best and second-best results on the corresponding session. Due to space limitations, the performance on only six incremental sessions is presented. Please refer to the supplementary for more detailed results on CUB200 and CIFARI00.
Session
3
4
6
HMean/NAce
HMean NAcc
HMean
NAcc
HMean
NAce
|HMean
NAce
HMean
NAcc
HMean
NAcc
CEC [57
30.72
19.60
30.05
19.10
29.86
19.00
29.41
18.65
27.15
16.88
27.36
17.07
FACT [58]
30.60
19.20
27.84
17.10
25.89
15.67
23.85
14.20
22.01
12.92
20.65
12.00
TEEN
50.04
38.00
46.67
34.60
44.72
32.67
43.53
31.55
41.75
29.80
39.22
27.37
A
+19.32
+18.4
+16.62
+15.5
+14.86
+13.67
+14.12
+12.9
+14.6
+12.92
+11.86
+10.3
6.1.2 Comparison results
In this section, we conduct overall comparison experiments on different performance measures. These different performance measures focus on different aspects of the methods. For example, the widely-used average accuracy across all classes (i.e., AvgAcc) is difficult to reflect the performance of new classes because the base classes take a large percentage of all classes (e.g.. g.. 60 base classes of 100 classes in CIFAR100). Following (58), the Harmonic mean (i.e., HMMaa) is used to evaluate the balanced performance between the base and new classes. Besides the above performance measures, we also additionally evaluate the different methods of their performance on new classes (i.e., NAcc). Following [4 [57 58], we also measure the degree of forgetting by the Performance Dropping Rate (i.e., PD). The PD is defined as PD = Acco Acc- L.e., the average accuracy dropping between the first session (i.e, Accg) and the last session (i.e., Acc_1). 1) The detailed comparison results of PD and AvgAcc are reported in Table 3 and the detailed comparison results of NAce and HMean are reported in Table These experimental results from different performance measures all demonstrate the effectiveness of TEEN. Besides, significantly lower FPR and TBR in Table and Table 2 also verify the effective calibration of TEEN.
usually design complex pretraining algorithms to enhance the extendibility of feature space, which may harm the diserim- inability of base classes. This elaborate pretraining stage may lead to an inconsistent but negligible performance in the base session. Besides, PDJ and APD in Table B are not affected by the pretrained results and also show TEEN outperforms previous state-of-the-ar methods.
Notably, many previous state-of-the-art FSCIL methods (e.g.. 57 58)
6.2 Comparison results of FSL
Table 5: Few-Shot Leaning performance of classification accu-
FSL can be approximated as measuring only the performance of the new classes the first incremental session of FSCIL. Besides, FSL itself also faces the chal- lenge of biased class prototypes due to the few-shot data. Therefore, we vali- date TEEN in the setting of FSL and report the compared results in Table 5 To ensure a fair comparison, we strictly followed the experimental setup of [51].
racy (%) on minilmageNet and CUB. The results of compared methods are cited from 511. 5w1s and 5w5s mean 5way- I shot and 5way-5shot, respectively. The best results are in bold and the runner-up results are in underlines.
minilmageNet
CUB
Methods
5wls
5w5s
5w
5w5s
ProtoNet [4
54.16
73.68
72.99
86.64
NegCosine 26
62.33
80.94
72.66
89.40
LR with DC (51
68.57
82.88
79.56
90.67
The comparison results in Table show
TEEN
65.70
83.11
81.44
91.04
that TEEN can easily outperform the previous state cuute-of-thee ooethe--rt method [51] in several experimental settings. Notably. TEEN does not involve any additional training cost when recognizing new classes, and the only time cost lies in feature extraction. Therefore, once the sample features extracted, the inference cost of TEEN can be considered negligible compared to previous methods that require heavy training.
6.3 Ablation Study
The influence of a and 7: Notably, the proposed TEEN does not involve additional training-based modules or procedures after pretraining on base classes. When TEEN incrementally learns new classes, only the scaling temperature 7 and the coefficient of calibration item need to be determined.

---

300
Acouncy (%)
21.3
Accurscy (Ng
Accumey (HG)
150
Incremental Session
Incremental Session
(8) AvgAce
(b) word NAce
SimTEEN AvgAcc
(d) SiMTEEN
Figure 5: (a) and (b) show the influence of and the last session's average accuracy in all classes (ie., AvgAcc) and new classes (i.e., NAce), respectively. (c) and (d) show influence similarity- bbased weight in TEEN on the last session's 's average accuracy in all claxses (i.e. AvgAcc) and new classes (L.e. NAcc), respectively. It demonstrates that replacing the softmax-base weight with a hard onc-shot weight will drastically reduce the performance of TEEN.
Specifically, select from {8, 16, 32, 64] and from 0.2, 0.4, 0.6, 0.7, 0.9). Figure [a and Figure 55 show how the hyper-parametery a and r influence the average accuracy improve the
on all classes and new classes, respectively. It also demonstrates that proper performance of new classes, which confirms the base prototypes can benefit the performance of new classes. Compared with larger relatively smaller can smooth the weight distribution. The smaller T with stronger performance further confirm the effectiveness utilizing the abundant semantic information from base classes.
The influence of similarity-base weight: To further validate the effectiveness the semantic similarity between the base and new classes, remove the Equation and directly align the new prototypes to their corresponding K most similar base class prototypes, i.e., Cn = a Cn +
a) Σ R=1 Cj- We denote this Simple version of TEEN without similarity-base bassed weight as SimTEEN. We select K from {1, 3, 5, 7. Figure 5c and Figure show how the similarity-bast -baeed weight (i.e., Equation (6 influences the average accuracy on all classes and new classes, respectively. Notably, as shown Figure 5dl there is significant drop, particularly on new classes, after removing the similarity-basee based weight. This phenomenon confirms that TEEN can achieve calibration of new prototypes with the help of well-represented semantic similarity between the base and new classes.
Conclusion
Few-shot class-incremente learning is of great importance real-world learning scenarios. In this study, we first observe existing methods usually exhibit poor performance in new classes and the samples of new classes easily misclassified into most similar base classes. We further find that although the feature extractor trained on base classes can not well represent new classes directly, can properly represent the semantic similarity between the base and new classes. Based on these analyses, we propose a simple yet effective training-free prototype calibration strategy (i.e., TEEN) for biased prototypes of new classes. TEEN obtains competitive results in FSCIL and FSL scenarios.
The FSCIL methods mentioned this paper all seleet the base and new classes from the same dataset. In other words, current FSCIL methods assume the model pre-trains in the same domain, increasing the restrictions on pre-training data collection. Therefore, more realistic scenario pre-train dataset that is independent of the subsequent data distribution and then perform few-shot class- inaseinncreeenall learning on a target dataset that we expect. The intricate problem of cross-domain few-shot class- incremettll learning will be thoroughly investigated in our future works.
Limitations:
Acknowledgements
This work is partially supported by National Key R&D Program of China (2022ZD0) 14805), NSFC (62376118. 18, 620061 12, 62250069), Young Elite Scientists Sponsorship Program of Jiangsu Association for Science and Technology 2021-020, Collaborative Innovation Center of Novel Software Technology and Industrialization.
10

---

References
[1] Afra Feyza Akyürek, Ekin Akyürek, Derry Wijaya, and Jacob Andreas. Subspace regularizers for few-shot class incremental learning- In ICLR, 2022.
[2] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online
continual learning. In NeurlPS, pages 11816-11825, 2019.
[3] Antreas Antoniou, Harrison Edwards, and Amos J. Storkey, How train your MAML. In ICLR, 2019.
[4] Francisco M Castro, Manuel J Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteck Alahari.
End-to-end incremental learning. In ECCV. pages 233-244, 2018
[5] Wei-Lun Chao, Han-Jia Yc, De-Chuan Zhan, Mark E. Campbell, and Kilian Q. Weinberger. Revisiting meta- meta-learning as supervised learning. CoRR, abs/2002. 00573, 2020.
[6] Kuilin Chen and Chi-Guhn Lee. Incremental few-shot learning via vector quantization in deep embedded space. In ICLR, 2020.
[7] Ting Chen, Simon Korbllith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607, 2020.
[8] Ali Cheraghian, Shafin Rahman, Pengfei Fang, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi. Semantic-aware knowledge distillation for few-shot class-incremental learning. In CVPR, pages 2534 2543,533 2021.
[9] Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe, Pengfei Fang, Christian Simon, Lars Petersson, and Mehrtash Harandi. Synthesized feature hased few-shot class-incremental learning mixture of subspaces. In ICCV, pages 8641-8650. 2021.
[10] Zhixiang Chi, Li Gu, Huan Liu, Yang Wang. Yuanhao Yu, and Jin Tang. Metafseil: A meta-learning approach for few-shot class incremental learning. In CVPR, pages 14146-14155, 2022.
[11] Jia Deng. Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, pages 248-255, 2009.
[12] Arthur Douillard. Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled
outputs distillation for small-tasks incremental learning. In ECCV, pages 86-102, 2020.
[13] Chelsea Finn. Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In JCML. pages 1 1126-1135, 2017.
[14] Robert M. French and Nick Chater. Using noise to compute error surfaces in connectionist networks: A novel means of reducing catastrophic forgetting. Neural Computation, (475):755-1769 2002.
[15] Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In CVPR,
pages 4367-4375 2018
[16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
In CVPR, pages 770-778, 2016.
[17] Geoffrey E. Hinton, Oriol Vinyals, and Jeffiey Dean. Distilling the knowledge in a neural network. CoRR,
abs/1503.0253/ 2015.
[18] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In CVPR, pages 831-839, 2019.
[19] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weiibergerr Densely connected convolutional networks. In CVPR, pages 2261-2269, 2017.
[20] David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In AAAI, pages 3302-3309, 2018.
[21] Minsoo Kang. Jaeyoo Park, and Bohyung Han. Class-incremental learning by knowledge distillation with adaptive feature consolidation. In CVPR, pages 16050011059, 2022.
[22] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Barwinsk,,,,,,,a, et al. Overcoming catastrophic
Kieran Milan, John Quan, Tiago Ramalho, Agnieszka
forgetting in neural networks. PNAS, 11<(13):521-33526 2017.
[23] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009,
[24] Anna Kukleva, Hilde Kuchne, and Bernt Schiele. Generalized and incremental few-shot learning by
explicit learning and calibration without forgetting. In ICCV, pages 9000-9009, 2021.
[25] Zhizhong Li and Derek Hoiem. Learning without forgetting. In ECCV, pages 314-629, 2016.
[26] Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang. Mingsheng Long, and Han Hu. Negative margin
matters: Understanding margin in few-shot classification. In ECCV, pages 438-455. 2020.
[27] Jinlu Liu, Liang Song, and Yongqiang Qin. Prototype rectification for few-shot learning. In ECCV, pages
741-756, 2020.
[28] Yaoyao Liu, Bernt Schiele, and Qianru Sun. Adaptive aggregation networks for class-incrementtl learning.
In CVPR, pages 2544-2553, 2021.
[29] Yaoyao Liu, Bernt Schiele, and Qianru Sun. RMM: reinforced memory management for class-increman incremental
learning. In NeurIPS, pages 3478-3490, 2021.
[30] Suresh Madhavan and Nitin Kumar. Incremental methods in face recognition: survey. Artificial
Intelligence Review, 54(1):253-303, 2021.
[31] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. CORR,
2018
[32] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high--erformanee deep learning library. In
11

---

NewrIPS, pages 8024-8035, 2019.
[33]
Hang Qi. Matthew Brown, and David G. Lowe. Low-shot learning with imprinted weights. In CVPR, pages
2018.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from
natural language supervision. In ICML. pages 1748-8763. 2021.
[35] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental
classifier and representation learning. In CVPR, pages 5532,554,, 2017.
[36] Shaoqing Ren, Kaiming He,
B. Girshick, and Jian Sun. Faster R- R-CNN: towards real-time object
detection with region proposal networks. In NIPS, pages 91-99 1-99, 2015.
[37] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P. Lillicrap, and Gregory Wayne, Experience
replay for continual learning. In NewrlPS, pages 348-358, 2019.
[38] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, ct al. Imagenet large scale visual recognition challenge.
NCV,
(10))2111222222 2015.
[39]
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting
with hard attention the task. In ICML. pages 4548-4557, 2018.
[40] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. In NIPS,
pages 4077-4087, 2017.
[41] Hai-Long Sun, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Pilot: A pre-trained model-based continual
learning toolbox. arXiv preprini (iX::i339.07111111 17, 2023.
[42] Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang. Songlin Dong, Xing Wei, and Yihong Gong. Few-shot
class-incremental learning. CVPR, pages 12180-12189, 2020.
[43] Oriol Vinyals, Charles Blundell, Tim Lillicrap. Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In NIPS. pages 3630-3638, 2016.
[44] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds-200-2011 datasct.
Technical Report CNS-TR-22 1-001, California Institute of Technology, 2011.
[45] Feng Wang. Weike Pan, and Li Chen. Recommendation for new users with partial preferences by integrating product reviews with static specifications. In UMAP, pages 281-288, 2013.
[46] Yun Wang. Da- Zhou, Liu Liu, Han-Jia Yc, Yatao Bian, De-Chuan Zhan, and Pailin Zhao. Beef: Bi-compatible inssemnnrellttal learning via energy-based expansion and fusion. In ICLR, 2023. [47] Yu- Wang. Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression
for class-inerementt learning. In ECCV, pages 398 414, 2022.
[48] Yaqing Wang. Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few examples: A
survey few-shot learning. ACM Computing Surveys, 53(3):1-34, 2020.
[49] Yue Wu, Yinpeng Chen, Lijuan Wang. Yuancheng Ye. Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental lcarning. In CVPR, pages 374-382, 2019.
[50] Shipeng Yan, Jiangwei Xie, and Xuming He. DER: Dynamically expandable representation for class
incremental learning. In CVPR, pages 3014-3023, 2021.
[51| Shuo Yang. Lu Liu, and Min Xu. Free lunch for few-shot leaming: Distribution calibration. In ICLR, 2021.
[52] Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse inspired crature-classifer alignment for few-shot hass-incremental learning. In ICLR, 2023.
[53] Yang Yang. Da-Wei Zhou, De-Chuan Zhan, Hui Xiong, and Yuan Jiang. Adaptive deep models for incremental lcarning: Considering capacity scalability and sustainability, In KDD, pages 74-82, 2019, [54] Han-Jia Ye, De-Chuan Zhan, Yuan Jiang, and Zhi-Hua Zhou. Heterogeneous few-shot model rectification with semantic mapping. IEEF Transactions on Pattern Analysis and Machine Intelligence, 43(1 1):3878-
3891, 2021.
[55]
Han-Jia Yc, De-Chuan Zhan, Nan Li, and Yuan Jiang. Learning multiple local metrics: Global considcration helps. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(7): 169-1122..,, 2020.
[56] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with
differentiable earth mover's 's distance and structured classifiers. In CVPR. pages 12203-12213, 2020. [57] Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning
with continually evolved classificrs. In CVPR, pages 12455-1 12464, 2021.
[58] Da- DeiWei Zhou, Yun Wang, Han-Jia Ye, Liang Ma, Shiliang Pu, and De-Chuan Zhan. Forward compatible
few-shot masseinnremental learning. In CVPR, pages 9036-9046. 2022.
[59] DeiWei Zhou, Han-Jia Ye, Liang Ma, Di Xie, Shiliang Pu, and De-Chuan Zhan. Few-shot glass-incrementa
learning by sampling multi-phase tasks. IEEE Transactions on Pattern Analyxis and Machine Intelligence,
45(1 1):128 6-12831, 2023.
[60] Da- Wei Zhou, Yuanhan Zhang. Jingyi Ning. Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Learning without
forgetting for lssoon-lnnguage models. arXiv preprint orXiv: /9270, 2023.
[61] Kai Zhu, Yang Cao, Wei Zhai, Jie Cheng, and Zheng-Jun Zha. promooro prototype refinement for few-shot class- class-incremental learning. In CVPR, pages 6801-681 10, 2021.
12

---

Supplementary materials
A Discussion
A.1 Implicit Semantic Similarity
Notably, TEEN demonstrates that the off- the-the-shllf shelff extractor already capable of representing semantic similarity without the need for any additional training costs or auxiliary side information (e.g., s. the textual information of the class name (
A.2 Efficiency
The only time cost in TEEN is the pre-training cost the base classes. Furthermore, the training cost of the vanilla cross-entropy loss function is relatively small compared to other more complex training paradigms. Additionally, the calibration in TEEN does not evolve any training cost and update procedure. Therefore, the cost incrementally learning new classes of TEEN can be considered negligible compared with existing methods.
A.3 Combination with better representation
In addition combining with existing prototype-based methods, TEEN can also be combined with more powerful pre-trained models (e.g., self-supervised pre-trained model Notably, the recent popular vision-language models (e.g., CLIP (34)) can also be seen powerful pre-trained model and provide better representation.
A Closer Look FSCIL
In this section, we show the observations (i.e., Section 3 of the main paper) on more datasets. As shown in Table 10 the existing prototype-basee FSCIL methods are extremely prone to misclassify new classes into base classes. Considering the misclassified instances of base classes, we quanti- tatively show that the samples of new classes are usually misclassífied into base classes. These observations in the additional benchmark dataset are consistent with observations in the main paper, which further confirms the universality of our observation.
To further understand the aforementioned observation, we further analyze the question What base second observation: The classes are the new classes incorrectly predicted into? and come corresponding most similar prototype-base classifier misclassifies the new classes to their base classes with high probability. The detailed analysis results in Table further verify the correctness of our observation.
Table 6: Detailed average accuracy of each incremental session CIFAR100 dataset. results of compared methods are cited from 42 better.
means higher accuracy is better. J means lower PD
Accuracy session (%)
Method
PD
A PD
7
iCaRL 35
64.10
53.28
41.69
34.13
27.93
25.06
20.41
15.48
13.73
50.37
+28,09
EEIL 4
64.10
53.11
43.71
35.15
28.96
24.98
21.01
17.26
15.85
48.25
+25.97
Rebulancing
18
64.10
53.05
43.96
36.97
31.61
26.73
21.23
16.78
13.54
50.56
+28.28
TOPIC [42
64.10
55.88
47.07
45.16
40.11
36.38
33.96
31.55
29.37
34.73
+12.45
Sccoupee-NNeCosstiee
26
74.36
68.23
62.84
59.24
55.32
52.88
50.86
48.98
46.66
27.70
+5.42
Decoupled-Cosine
43
74.55
67.43
63.63
59.55
56.11
53.80
51.68
49.67
47.68
26.87
+4.59
Decouplei-DeepEMMD
56
69.75
65.06
61.20
57.21
53.88
51.40
48.80
46.84
44.41
25.34
+3.06
CBC [57
73.07
68.88
65.26
61.19
58.09
55.57
53.22
$1.34
49.14
23.93
+1.65
FACT
58
74.60
72.09
67.56
63.52
61.38
58.36
56.28
54.24
52.10
22.50
+0.22
TEEN
74.92
72.65
68.74
65.01
62.01
59.29
$7.90
54.76
$2.64
22.28
13

---

Table 7: Detailed average accuracy of each incremental session CUB200 dataset. The results of
compared methods are cited from 42 57 58 is better.
1 means higher accuracy is better. + means lower PD
Accuracy in each session (%)
Method
3
5
8
ICaRL
52.65
44.16
36.62
29.52
26.26
23.89
EEIL 4]
68.68
53.63
47.91
44.20
36.30
27.46
25.93
24,70
23.95
24.13
22.11
46.57
+28.44
Rebslancing
18
68.68
57.12
44.21
28.78
26.71
25.66
24.62
21.52
20.12
20.06
19.87
48.81
+30.68
TOPIC 12
68.68
62.49
54,81
49.99
45.25
41.40
38.35
35.36
32.22
28.31
26.26
42.40
+24.27
Cecoupled-NeeCooee
74.96
70.57
66.62
61.32
60.09
56.06
55,03
52.78
51.50
50,08
48.47
26.49
+7.36
Decoupled-Coise 43
75.52
70.95
66.46
61.20
60.86
56.88
55.40
53.49
51,94
50.93
49.31
26.21
+8.08
Decoupled De 56
75.35
70.69
66.68
62.34
59.76
$6.54
54.61
52.52
50.73
49.3
47.60
27.75
+9.62
57
75.85
71.94
58.27
57.73
55.81
53.52
52.28
23,57
+5.44
FACT 58
75,90
73.23
70.84
66. 13
65 56
62.15
61.74
59.83
58.41
57.89
94
18.96
+0.83
TEEN
77.26
76.13
72.81
68.16
67.77
64.40
63.25
62.29
61.19
60.32
59.31
18.13
Table 8: Detailed results of HMean and NAce on mindmageNeet. The best results are in bold and the runner-up results are in underlines. The A measures the performance gap between the best and second sccond-best results on the corresponding session. Due space limitations, the performance on only six incremental sessions presented. Please refer to the supplementary for more detailed results on CUB200 and CIFAR
Session
HMean/NAce
HMMaa NAce HMean
NAce
HMean
NAcc
HMean
[MMean
NAcc HMean
NAce
HMean
NAce
HMean
NAce
CEC [5N
47.04
34,80
40.35
28.10
36.01
24.13
36.49
24.65
36.56
24.84
37.07
25.40
36.41
24.83
35,49
24.08
FACT [88
43.04
30.00
38.77
10
34.70
22.60
34.44
22.45
34.55
22.64
35.44
23.47
33.79
22.06
33.05
21.48
TEEN
47.77
34,82
42.99
30.10
40.19
27.53
39.52
27.00
39.66
27.28
39.65
27.37
37.71
25.57
37.11
25.13
A
+0.73
+0.02
+2.64
+2.00
+4.18
+3.40
+3.03
+2.35
+3.10
+2.44
+2.58
+1.97
+1.30
+0.74
+1.62
+1.05
C
Experiments
C.1 Details of experiments
Dataset details:
Following previous methods 42 57 58), we evaluate TEEN on CIFARIOO 100 23
CUB200-2011 [44), minilmageNeet [38]. CIFAR100 23 contains 100 classes and each class contains 500 images for training and 100 images for testing. The image size of CIFARI00 is 3 x 32 x 32. The CUB200-2011 44 is widely-used fine-graiie tre-grained dataset and also a benchmark dataset of few-shot image classification. It contains 200 classes and all images of these classes belong to birds. The minilmageNet [8 are sampled from the raw ImageNet 11 and contains 100 classes.
In FSCIL, each benchmark dataset is divided into different subsets. Each subset contains specific classes and the label space of different subsets is nonoverlapping. Specifically, CIFARION 100 is divided into 60 classes for the base session and the remaining 40 classes are divided into eight 5-way 5-shot few-shot classification tasks. CUB200 is divided into 100 base classes for the base session and the remaining 100 classes are divided into ten 10-way 5-shot few-shot classification tasks. minilmageNet divided into 60 base classes for the base session, and the remaining 40 classes are divided into eight 5-way 5-shot few-shot classification tasks. The splitting details (i.e., the class order and the
selection of support data in incremental sessions) follow the previous methods 42 57 58 Notably,
no old samples are saved to assist in maintaining the discriminability of previous classes
we compare TEEN with popular CIL methods, FSL methods, and FSCIL methods. For CIL. methods, we select iCaRL 35), EEIL [4] and Rebalancing [18 our baseline methods. For methods based on few-shot, we adopt Decoupled- NegativeCosine (66), Decoupled-Cosin 45 and Serouupeed-DeepE.MM [56] as our baseline methods.
Baseline details:
Following previous methods 42 57 58 611,
For FSCIL methods, we adopt TOPIC methods.
42 CEC [57] and FACT 58 methods as our baseline
Training details:
The training of the feature extractor uses vanilla cross-entropy loss as the objective function. In addition, we adopt the cosine similarity to measure the feature of instances to class
prototypes. Following 42 [5 58), we ResNet20 [66 for CIFAR 100, pre-trained ResNet18 [16] for CUB200 and randomly initialized ResNet18 [16] for minilmageNet. All compared methods use the same backbone network and same initialization for fair comparison.
For the hyperparameters setting, we set a = 0.5, - 16 for minilmageNet and CUB200, a = 0.1, = 16 for CIFAR 100. We train the feature extractor CUB200 with learning rate of 0.004,
14

---

batch size of 128, and epochs of 400. We train the feature extractor on CIFARI00 and minilmageNett with learning rate 0.1 and batch size of 256. The cosine scheduler used adjust the learning rate. Before computing the cross-entropy loss, a widely-used temperature scalar is used to adjust the distribution of output logits. For example, the original output logits of instance Iy are denoted as Oj TR. The logits used to compute cross-entropy loss are denoted as T The To set for the CIFAR100 dataset and 32 for minilmageNet and CUB200 datasets.
C.2 More comparison results
In the main paper, we compare different methods in overall performance measures: performance drop rate, average accuracy across all classes, harmonic mean accuracy and average accuracy on new classes. In this session, report the detailed comparison results of the aforementioned performance measures on CIFARI00 23 and CUB200 44 Besides the main results in Table 6 and Table 7 we also report the accuracy across new classes and harmonic mean accuracy in Table g The consistent performance improvement demonstrates the effectiveness of TEEN's ' calibration ability.
D Empirical Analysis
D.1 Effects of TEEN
In the main paper, we only show the effect of calibration prototypes on the CIFARIO0 dataset. In this supplementary material, we report more detailed results in Table The results further confirm our conclusion in the main paper: the proposed TEEN can well calibrate the biased prototype of new classes. Although TEEN leads to misclassifying some instances of base classes, the average accuracy across all classes is always very competitive.
Table 9: The detailed ratio (%) of base and new classes about three types of samples(i.e. UC samples, W-R samples, R-W samples) of minilmageNet
Incremental Session
2
3
4
Sample type
UC
W-R R R-
UC
W-R
R R-W
UC
W-R
R R--
UC
W-R
R--
Base Class
93.84
0.00
99.99
88.87
0.00
95.78
84.15
0.00
94.20
79.65
0.00
91.03
New Class
6.16
100.00
0.01
11.13
100.00
4.22
15.85
100.00
5.80
20.35
100.00
8.97
D.2
Combination of TEEN with existing methods
Due to the flexibility of proposed TEEN can be seen as a plug-ond-play module to calibrate the biased prototypes of new classes. For more in-depth understanding of the TEEN we combine TEEN with FACT (58) and show the performance on different performance measures in Figure 6 and Figure We empirically find that combining the TEEN with FACT has little effect the average accuracy across all classes (i.e., the left figures in Figure 6 Figure 7 and Figure 8). However, the TEEN can improve the average accuracy of new classes and the harmonic mean accuracy. In other words, the existing prototype-bused methods (i.e., FACT) benefit the well-calibration effect of TEEN and achieve better-balanced performance between the base and new classes. We believe more methods (e.g., (22 can benefit from the flexibility of TEEN.
E Broader impact
Our proposed method can significantly improve the accuracy new classes in FSCIL and FSL scenarios, leading a better understanding of the low performance of new classes. Furthermore, the analyses this study are first proposed in the FSCIL and FSL fields which point out that we should pay more attention the performance of new classes. In addition this, our method may also have potential heuristic effects on the expectation estimation of the distribution.
15

---

FACT
60
Accuncy (%))
Accuracy (9%
Accuracy (%)
40
20
0.0
2,5
5.0
7.5
10.0
40
2
Incremental Session
Incremental Session
Incremental Scasion
(a) Accuracy on classes
(b) Harmonic mean accuracy
(c) Accuracy on new classes
Figure 6: We compare FACT without (i.. FACT in figures) and with TEEN (i.e., FACT w/ TEEN on CUB200 dataset. The FACT benefits from the well-calibration effect of TEEN and achieves better-balanced performance between the base and classes
TEEN
FACT
FACT W'TEEN
FACT T'TEEN
20
Accuracy (96)
Accuracy (%)
Accuracy
60-
10
50
40-
Incremental Session
Incremental Session
Incremental Session
(a) Accuracy on all classes
(h) Harmonic mean accuracy
(c) Accuracy on new classes
Figure 7: We compare FACT without (i.e. FACT in figures) and with TEEN (i... FACT w/ TEEN on miniImageNet dataset. The FACT benefits from the well-calibration effeet of TEEN and achieves better-balanced performance between the base and new classes
80
40
FACT
EACT TEEN
FACT
FACT wi W/TEEN
PACT
FACT WITEN
75
Accuracy (%6
Accuracy (%)
45
Accuracy (%)
3.5
70
65
60
10
60
55
50
Incremeettal Session
Incremental Session
Incremental Session
(a) Accuracy all classes
(b) Harmonic mean accuracy
(c) Accuracy new classes
Figure 8: We compare FACT without (i.e., FACT in figures) and with TEEN (i.e., FACT w/ TEEN on CIFAR100 dataset. The FACT benefits from the well-calibration effect of TEEN and achieves better-balanced performance between the base and new classes
Table 10: Detailed prediction results of False Negative Rate/False Positive Rate (%) on minilmageNet dataset. The analysis results are from session because new classes do not exist in session 0. Exceedingly high FPR and relatively low FNR show the instances of new classes are easily misclassified into base classes and the instances of base classes also easily misclassified into base classes. TEEN can achieve relatively lower FPR than baseline methods, which demonstrates the validity of the proposed calibration strategy.
Session
3
6
FNR/FRR
|NN FPR
FNR
FPR
FNR
FPR
FNR
FPR
|NN
FPR
|NR
FPR
FNR
FPR
PNR
FPR
ProtoNet 40
2.55
71.60
3.98 67.30
4.83
66.07
5.83 62.05
6.48 $9.16
7.18
58.17
7.93 54.83
8.85 52.23
CEC 557
3.45
68.40
5.60 65.50
61.93
7.73
58.30
8.47
56.68
9,58
54.50
10.22
52.54
10.93 50.05
FACT [58
2.07 72.40
3.42 70.60
4.22
70.13
4.55
69.40
4.97
68.28
5.12 68.17
5.33
66.51
5.58 66.55
TEEN
88.02 46.40
11.35 38.60
13. 37.53
15.32
35.20
16.47
32.48
17.
3.57
18.63
28.03
19.97 26.35
16

---

Table 11:
Detailed prediction results TNRTBB (%) on minilmageNee dataset. analysis results are from session because new classes do not exist session For new classes, we only consider the 10 most similar base classes out of 60 base classes. For base classes, we suppose C2 new classes exist in the current incremental session i. We only consider the most similar 220% C, new classes. Class similarity adopts cosine similarity between different class prototypes. TEEN can achieve relatively lower TBR than baseline methods, which demonstrates the validity the proposed calibration strategy.
Session
TNR/IBR
[NR TBR TNR TBR TNR TBR TNR TBR TNR TBR
TNN TBR TNN TBR TNR TBR
ProtoNet
3.47 73.01
5.02 61.79 7.06 55.05
8.08 $2.95 7.14 53.27
8.56 51.96
8.53 49.18
9.21 50.91
CEC 57
2.87 71.70
4.53 61.08 5.66 58.26
6.35 55.16
6.07 54.12
6.92 52.53
8.24 49.96
8.20 51.64
FACT 58
2.33 70.00
3.32 61.01
5.50
55.17
6.88 50.55
6.57
50.52
7.72 49.30
8.88
46.99
9.31 48.48
TEEN
2.16 66.77
2.53
55.14
3.55
44.87|
3.85
37.68
3.57
39.47
4.02
37.07
4.76 33.83
4.85 35.04
17


Reviews:
[{"review_id": "MrblQg3Or5", "reviewer": "Reviewer_RFbX", "summary": "The paper closely looks at problems in prototypical networks and methods in the context of few shot class incremental learning. The authors observe semantic similarity between new and base prototypes along with new classes being misclassified regularly as semantically similar base classes. Towards this they use a training free calibration strategy to guide the new ill calibrated prototypes using the base prototypes. Essentially bridging the lack of training data and suppressing bias.", "questions": "- See weaknesses.   \n- Discussions and comparisons to the most recent sota are necessary\n", "limitations": "The authors have adequately mentioned the limitations. And motivate the setting where the pretrain base classes being from different data distribution as the incremental new classes is actually a more realistic setting.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The major strength of the paper lies in the simplicity of the method. Instead of compromising the base class performance they approach the problem from a new class orientation. Doing so relieves the method from the convoluted approaches of feature space reservation[1], meta learning schemes[2] or self supervision[3]. Instead the methods strength lies in utilising the uncompromised power of the base prototypes.    \n- The fact that almost all evaluation metrics one way or the other focus on the novel class performance and not the less interpretable average accuracy and performance decay rate makes the paper even more attractive. This shows commitment towards pushing the frontier in aspects previously underexplored and facilitating a fair and thorough comparison.  \n- The work is constructed clearly and simply. It seems obvious that the authors followed a train of thought. They end up convincing the reader on why each module is essential. It generally reads very smoothly, hopping from hypothesis to observations to results and repeats. For example section 3 which tries to “Understand the reason for poor performance in new classes” leads smoothly into the observations for that section which motivates the section on calibration (section 4) and this section takes a similar hypothesis-observation-result route.   \n- The fact that almost all evaluation metrics one way or the other focus on the novel class performance and not the less interpretable average accuracy and performance decay rate makes the paper even more attractive. This shows commitment towards pushing the frontier in aspects previously underexplored and facilitating a fair and thorough comparison. \n", "weaknesses": "- Session 4 uses a variety of terminology. Using a consistent term for each prototype could potentially read better. (For example ill-calibrated new prototypes, new prototypes, biased prototypes)  \n- In relation to prior work it seems only Section 1 is truly dedicated to the literature. And even then it seems the prior works that this paper branches out from are not discussed in appropriate detail. Prior methods from the 3 other papers from Table 1 and Table 2 are not detailed in any preceding or subsequent section. Their would be reason to do so as the method directly criticises (line 86-87 or line 190-191) prior works. Prefacing in better detail how previous methods mitigate biases could be essential in understanding the state of the literature and motivate TEEN.  \n- [4] show in “Simpleshot” that L2 normalisation of prototypes improves performance. This is also confirmed by [5] in their nearest neighbour method for few shot learning. The authors overlooked this crucial feature transformation which is commonly conducted in previous studies on Non Parametric methods for FSCIL for example the NoNPC method by Oh et al. [6] This omission somewhat limits the comprehensiveness of their findings despite the improved results given the prevalnece of the method.   \n- [7], [8] are missing comparisons. Therefore, the tables that compare the method to the current sota would should less prominent results.  \n\nMinor Remarks and Typos:  \n- 213: “rely” should be “relies”  \n- 2: “scarce”  \n- The use of tau parameter in equation (5) might be better understood if shifted to equation 6 as S_{b,n} is a scalar and in any case it seems more common that a temperature scaling parameter appears in the softmax. Would recommend to make that alteration.  \n- 73: “neglect” should be “negligence”  \n\n\n[1] Zhou, Da-Wei, et al. \"Forward compatible few-shot class- incremental learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n \t\t\t\t\t\t\t\t\n[2] Zhang, Chi, et al. \"Few-shot incremental learning with continually evolved classifiers.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\n \t\t\t\t\t\t\t\t\n[3] Ahmad, Touqeer, et al. \"Few-shot class incremental learning leveraging self-supervised features.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n \t\t\t\t\t\t\t\t\n[4] Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Laurens van der Maaten. Simpleshot: Revisiting nearest-neighbor classification for few- shot learning. arXiv preprint arXiv:1911.04623, 2019.\n \t\t\t\t\t\t\t\t\n[5] Wang, Guangpeng, and Yongxiong Wang. \"Self-attention network for few-shot learning based on nearest-neighbor algorithm.\" Machine Vision and Applications 34.2 (2023): 28.\n \t\t\t\t\t\t\t\t\n[6] Oh, J., & Yun, S.-Y. (2022). Demystifying the Base and Novel Performances for Few-shot Class-incremental Learning. http://arxiv.org /abs/2206.10596\n\n[7]  Peng, Can et al. Few-Shot Class-Incremental Learning from an Open-Set Perspective, ECCV 2022\n\n[8] Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse inspired feature-classifier alignment for few-shot class-incremental learning. In International Conference on Learning Representations 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1** Session 4 uses a variety of terminology. Using a consistent term for each prototype could potentially read better. (For example ill-calibrated new prototypes, new prototypes, biased prototypes)\n\n**A1** We thank the reviewer RFbX for the kind and meaningful writing suggestions. We will fix the terminology and typos in the final version.\n\n---\n\n**Q2** In relation to prior work it seems only Section 1 is truly dedicated to the literature. And even then it seems the prior works that this paper branches out from are not discussed in appropriate detail. Prior methods from the 3 other papers from Table 1 and Table 2 are not detailed in any preceding or subsequent section. Their would be reason to do so as the method directly criticises (line 86-87 or line 190-191) prior works. Prefacing in better detail how previous methods mitigate biases could be essential in understanding the state of the literature and motivate TEEN.\n\n**A2** We thank the reviewer RFbX's constructive suggestions. We summarize the ProtoNet, CEC and FACT in the response and a more detailed introduction will be included in the final version. These three baselines freeze the feature extractor to mitigate the forgetting problem in incremental learning scenarios and adopt the prototype-based classifier to mitigate the overfitting problem in the few-shot scenario. We will summarize their differences separately.\n\n- The ProtoNet is a popular baseline method in the FSCIL field. It only pre-trains the feature extractor on base classes with a vanilla cross-entropy loss and plugs the prototype of novel classes to jointly recognize the base and novel classes. \n- The CEC follows a meta-learning paradigm and construct some pseudo few-shot task in the pre-training stage. Based on these pseudo few-shot tasks, CEC trains a classiﬁer adaptation module to adapt new classes in the incremental sessions.\n-  The FACT follows a forward-compatible paradigm to preserve some feature space for the incoming novel classes. \n\n**Summary**: Although these FSCIL methods achieve a competitive average performance of base and new classes, we observe that these methods still perform poorly on new classes (See section 3). Our method TEEN aims to *explicitly* mitigate the problem of poor performance on new classes and bridge the gap between base and novel classes and finally achieve better average performance.\n\n---\n\n**Q3** [4] show in “Simpleshot” that L2 normalisation of prototypes improves performance. This is also confirmed by [5] in their nearest neighbour method for few shot learning. The authors overlooked this crucial feature transformation which is commonly conducted in previous studies on Non Parametric methods for FSCIL for example the NoNPC method by Oh et al. [6]\n\n**A3** We thank the reviewer RFbX for providing the related works. We agree that the widely-used $L_2$ normalization of the prototypes is beneficial in few-shot related tasks. Therefore, we do not overlook yet adopt this normalization in Line219-Line220 (Eq6). Besides, although [6] also adopts the $L_2$ normalization. **There still exists main differences between our paper and NoNPC [6]**. For example, we explicitly *observe and explore the reason for the low performance of new classes* in the FSCIL scenario, which is absent in [6]. Based on our **unique** empirical analysis, we propose *explicitly* calibrating the novel prototypes through the implicit semantic similarity overlooked by existing works with a *training-free* method. A more detailed discussion with NoNPC will be included in the final version.\n\n---\n\n**Q4** [7], [8] are missing comparisons. Therefore, the tables that compare the method to the current sota would should less prominent results.\n\n**A4** We appreciate the valuable suggestions from the reviewers regarding the strong baseline methods. We acknowledge that TEEN may not directly outperform the missing baseline methods [8]. However, in our supplementary material, we demonstrate the potential of TEEN as a *plug-and-play* module in Figure 1 and Figure 2. We have tried to integrate TEEN with the mentioned method NC-FSCIL and fine-tune TEEN's two hyper-parameters in the evaluation stage of NC-FSCIL [8]. The plug-and-play nature of TEEN also allows it to achieve competitive performance.\n\n|                  |   0   |   1   |   2   |   3   |   4   |   5   |   6   |   7   |   8   |\n| :--------------: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n|    ALICE [7]     | 80.60 | 70.60 | 67.40 | 64.50 | 62.50 | 60.00 | 57.80 | 56.80 | 55.70 |\n|   NC-FSCIL [8]   | 84.02 | 76.80 | 72.00 | 67.83 | 66.35 | 64.04 | 61.46 | 59.54 | 58.31 |\n| NC-FSCIL w/ TEEN | 84.06 | 76.82 | 72.12 | 67.86 | 66.41 | 64.21 | 61.63 | 59.55 | 58.43 |\n\n"}, {"review_id": "ZhttCnrXcL", "reviewer": "Reviewer_wHh2", "summary": "This paper presents a novel training-free calibration approach for few-shot class incremental learning. The authors make an observation that one main problem with FSCIL is that data of new classes can be easily mis-classified as base session classes. By utilizing the well-calibrated embeddings of base session classes to help embeedings of the new classes, the author improve the performance of FSCIL.", "questions": "1. I suggest the author provide more ablation study on robustness of hyper-parameter on more dataset.\n2. I suggest the author provide more justification on the designing of the method (i.e., why this simple linear interpolation form). Such justification can be empirical visualizations, theoretical analysis, etc.", "limitations": "The authors have adequately addressed the limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The observation of new classes being confused as old classes is somewhat novel.\n2. The proposed classifier-calibration method is interesting and novel.\n3. The proposed method is simple yet effective.\n4. Detailed ablation studies have been performed on the introduced hyper-parameters.", "weaknesses": "1. One of the reviewer's concern is in terms of the robustness of the method to hyper-parameter \\alpha and \\tau. Are the optimal hyper-parameters the same for different datasets or different incrementing procedures (i.e., how many classes per incremental session)?\n2. In addition, the author should provide more detailed justification for the design of the method. Why use this simple linear interpolation for calibration? Are there any empirical observations or theory to support this design?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1** One of the reviewer's concern is in terms of the robustness of the method to hyper-parameter $\\alpha$ and $\\tau$. Are the optimal hyper-parameters the same for different datasets or different incrementing procedures (i.e., how many classes per incremental session)?\n\n**A1** We are sorry for your confusion. In fact, we clearly show the the value of $\\alpha$ and $\\tau$ on different benchmarks in line 266. The optimal value of $\\tau$ is stable for all benchmark datasets and the $\\alpha$ is different. A more detailed ablation studies on the $\\alpha$ and $\\tau$  are shown in Figure 5. Regarding the number of classes per incremental session, we introduce the base experiment setting in lines 254 to 256 and provide detailed settings in Section 4.1 of the supplementary material. Please refer to the corresponding section mentioned above for the relevant experimental settings.\n\n---\n\n**Q2** In addition, the author should provide more detailed justification for the design of the method. Why use this simple linear interpolation for calibration? Are there any empirical observations or theory to support this design?\n\n**A2** We are sorry for your confusion. The format of linear interpolation is simple and widely-used in different machine learning or deep learning fields, such as mixup [1] and LVQ algorithm [2]. A simple and intuitive understanding is that the form of linear interpolation *pulls a vector closer towards another vector in terms of direction*. To visualize this more, we have used a toy example from 2d gaussian distribution for a simple demonstration. Please refer to **the Figure 1 in the Rebuttal PDF** for a more detailed introduction. In addition to this empirical interpretation, the empirical observation in Section 3 also strongly supports our design. Specifically, we empirically find the reason for the low performance of the new class is that it is heavily misclassified into the most similar base classes. Therefore, we can improve the performance of the new class by pulling the biased new prototypes towards the prototypes of the most similar base class. \n\n\n\n[1] Zhang, Hongyi, et al. mixup: Beyond Empirical Risk Minimization. https://arxiv.org/abs/1710.09412\n\n[2] Learning vector quantization. https://en.wikipedia.org/wiki/Learning_vector_quantization\n\n---\n\n**Q3** I suggest the author provide more ablation study on robustness of hyper-parameter on more dataset.\n\n**A3** We thank the reviewer for the suggestion. Please refer to **the Figure 2 in the Rebuttal PDF** for ablation studis of hyper-parameters (i.e., $\\alpha$ and $\\tau$) on more datasets. To better illustrate the trends, we have included the replicated trend figure5 (a)) in this rebuttal pdf.  We observed that the trend of the hyperparameter $\\alpha$ remains consistent across all benchmark datasets. Although the trend of the hyperparameter $\\tau$ may vary slightly between the CUB200, CIFAR100, and miniImageNet datasets, maintaining uniform settings still allows TEEN to maintain competitive performance. Notably, the trend of $\\tau$ in miniImageNet and CIFAR100 datasets is perfectly identical. We hypothesize that the slight difference observed in CUB200 may be attributed to the fact that it is a *fine-grained* dataset."}, {"review_id": "CHT7iwyREQ", "reviewer": "Reviewer_aEVg", "summary": "The authors work on the Few-Shot Class-Incremental Learning (FSCIL) scenario and propose the Training-frEE calibratioN (TEEN) strategy. This strategy enhances the discriminability of new classes by fusing the new prototypes with base prototypes. This approach is different from previous methods, which either introduce extra learnable components or rely on a frozen feature extractor.", "questions": "I can understand that the prototype of a novel class is biased due to the lack of samples, and it can be easily misclassified as a base class. Intuitively, if the novel class is prone to be classified as the most similar base class, interpolation among such features could exacerbate this issue. But why is the use of weighted base prototypes to calibrate novel ones beneficial in enhancing the discriminability of such a prototype while reducing the discriminability of base prototypes (lines 217-218, 238-239)?. And could you please clarify the definition of discriminability?", "limitations": "As mentioned in the weaknesses section, the proposed idea is not novel at all. It unfortunately is just a tiny incremental contribution which is rather insufficient for a publication in top tier conference such as NeurIPS.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The authors observed that the feature extractor, although only trained on base classes, can surprisingly represent the semantic similarity between the base and unseen new classes. Accordingly, they proposed a solution.\n2. The paper is easy to follow, with clear experimental details that aid reproducibility. The motivations are also presented clearly.", "weaknesses": "1. The issue of misclassification has already been observed and studied in previous few-shot learning works. Therefore, the poor performance of new classes is not surprising in the task of Few-Shot Class-Incremental Learning (FSCIL).\n2. Forming connections across samples from base and novel classes is not a new concept. For instance, FADI[1] discovered that a novel class may implicitly leverage the knowledge of multiple base classes to construct its feature space. It then builds a discriminative feature space for each novel class via association and discrimination steps.\n[1] Few-Shot Object Detection via Association and Discrimination, Yuhang Cao et al., NeurIPS 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1** The issue of misclassification has already been observed and studied in previous few-shot learning works. Therefore, the poor performance of new classes is not surprising in the task of Few-Shot Class-Incremental Learning (FSCIL).\n\n**A1** We thank the reviewer for the suggestion.  **We must emphasize that our paper not only illustrates the phenomenon of low performance on new classes but also investigates the underlying causes behind this phenomenon**. Furthermore, based on our analysis of this phenomenon, we propose our  approach (i.e., TEEN). These analyses are unique in the FSCIL field and we believe that these analysis can provide new insights for the FSCIL field. Besides, if there are some papers that closely resemble our experimental analysis and methods, we would appreciate it if you could directly point out those relevant papers. We will include a detailed discussion of these papers in the final version of our paper.\n\n---\n\n**Q2** Forming connections across samples from base and novel classes is not a new concept. For instance, FADI[1] discovered that a novel class may implicitly leverage the knowledge of multiple base classes to construct its feature space. It then builds a discriminative feature space for each novel class via association and discrimination steps. [1] Few-Shot Object Detection via Association and Discrimination, Yuhang Cao et al., NeurIPS 2021.\n\n**A2**  Thank you for providing this helpful citation. We differentiate our method from [1] based on the following perspectives:\n\n- **Setting and Motivation**: [1] focus on the Few-Shot Object Detection (**FSOD**) task and argue that directly fine-tune the model pre-trained on all base classes with abundant samples will lead a inferior performance of new classes. However, it donot analyze the advantages of connecting new and base classes in the FSOD scenario. In other words, the motivation behind connecting base and novel classes in the FSOD scenario is unclea, and there is a lack of experimental insights in [1]. In contrast, our paper focuses on the **FSCIL** task and first explore the reasons for the low performance of new classes in FSCIL. We find that new classes are generally easily misclassified into the most similar base classes, and we also empirically find that feature extractors are still able to characterize the semantic similarity between new and old classes even if they have not been trained on new classes. Based on these unique analyses, we achieve the goal of calibrating the new class prototype with the old class by utilizing feature extractors that have only been trained on the base class and its implied semantic similarity\n- **Methodology**: [1] adopt a two-stage (i.e., association and discrimination) learning method to learn a more discriminative novel classiﬁer. However, it adopt the  WordNet [2] as an auxiliary to describe the semantic similarity of each classes and use the Lin Similarity. Besides, [1] align the novel classes to the most similar base class. After that, [1] adopt a *Set-Specialized Margin Loss* to explicitly improve the discriminability. In contrast,  our method TEEN propose to leverage the feature extractor trained *only on base classes* to characterize the semantic similarity between the base and novel classes and donot extra semantic characterization model, which is more simple. Besides, TEEN directly used the cosine similarity to calibrate the new prototypes based on the weighted base prototypes and donot involve any training module or stage, which is more efficient.\n\n[2] George A Miller. Wordnet: a lexical database for english. \n\n---\n\n**Q3** why is the use of weighted base prototypes to calibrate novel ones beneficial in enhancing the discriminability of such a prototype while reducing the discriminability of base prototypes (lines 217-218, 238-239)?. And could you please clarify the definition of discriminability?\n\n**A3**  Thanks for your question. We will address your questions separately in two ways.\n\n-  **Firstly**, we clarify the definition of discriminability in the FSCIL context. Here, when we talk about *discriminability*, we mainly refer to **prototypes**, which are the class centers of each class. From the results, if the classification accuracy of a class improves, we can infer that the discriminability of that category's prototype has also improved. \n- **Secondly**, we need to emphasize that in FSCIL, the model is required to recognize both new and base classes *simultaneously*. In our approach, we utilize semantic similarity to calibrate the new class prototypes based on a weighted combination of prototypes from semantically related base classes. Intuitively, this process brings the new class prototypes closer to the base class prototypes to a certain extent. Consequently, some samples from the base classes may be mistakenly classified as the calibrated new classes. We comprehensively analyze the potential negative (in Figure 3 and section 4.2) and positive effect of our calibration method (section 5), which show the efficiency of TEEN. Please refer to section 5.3 for the effectiness of semantic-based weight.\n\n---\n\n**Q4** The proposed idea is not novel at all.\n\n**A4** We recapitulate our contributions. First, we empirically find that new class performance was\nmuch lower than the base class in previous methods. Then, we explore the causes of this phenomenon. These analyses are first proposed in the FSCIL field, which points out that we should pay more attention to the performance of new classes. Besides, we observe the feature extractor trained on base classes can also depict the semantic similarity between the base and new classes. Based on these analyses, we propose TEEN, which not only achieved a higher average accuracy but also improved the accuracy of new classes (**10.02% ∼ 18.40% better than the runner-up method**). Finally, we also validate TEEN on the Few-Shot Learning (FSL) task, which can also show competitive performance."}, {"review_id": "HU14KlQorh", "reviewer": "Reviewer_3Zyg", "summary": "This paper tackles the problem of few-shot class incremental learning based on prototypical network. Motivated by the observation that novel classes are easily misclassified as base classes, the authors propose a prototype calibration strategy. The calibrated prototype is a weighted sum of prototype computed from novel class support set and base classes semantically similar with the novel class. Experiments on miniImageNet, CUB and CIFAR demonstrate the superiority of the proposed method over previous ones.", "questions": "The semantic similarity based calibration is a straightforward and commonly used method used in the classification area. There may lack technical contribution to the community.", "limitations": "The limitation of cross-dataset evaluation is discussed.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well motivated that the inferior performance of prototypical network results from the phenomenon that novel classes are easily misclassified as base classes. The phenomenon is well-studied via experiments.\n2. The paper is well written and easy to follow.\n3. The authors propose a simple but effective calibration strategy to improve the performance.", "weaknesses": "1. The paper introduces two hyper parameters which need to be exhaustively searched for every dataset.\n2. There lacks cross-dataset experiments to show the effectiveness of the proposed method when transferring knowledge from one dataset to another.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1** The paper introduces two hyper parameters which need to be exhaustively searched for every dataset.\n\n**A1** We thank the reviewer for the constructive question. We show the performance trend with respect to $\\alpha$ and $\\tau$ on each benchmark dataset in   **Figure 2 in the Rebuttal PDF**. To better illustrate the trends, we have included the replicated trend in figure 5 (a)) in this rebuttal pdf.  We observed that the trend of the hyperparameter $\\alpha$ remains consistent across all benchmark datasets. Although the trend of the hyperparameter $\\tau$ may vary slightly between the CUB200, CIFAR100, and miniImageNet datasets, maintaining uniform settings still allows TEEN to maintain competitive performance. Notably, the trend of $\\tau$ in miniImageNet and CIFAR100 datasets is identical. We hypothesize that the slight difference observed in CUB200 may be attributed to the fact that it is a *fine-grained* dataset.\n\n---\n\n**Q2** There lacks cross-dataset experiments to show the effectiveness of the proposed method when transferring knowledge from one dataset to another. \n\n**A2**  In fact, we introduce this more realistic FSCIL scenario (e.g., pre-training and fine-tuning on *independent* datasets) in our **limitation** part. We believe that this is a limitation currently present in mainstream FSCIL settings and could be a potential research direction. This limitation can be viewed as an assumed constraint in all existing FSCIL methods, which can be further explored in future research. Nevertheless, we construct a more strict FSCIL scenario to evaluate the effectiveness of TEEN. Please refer to the global response (**Cross-dataset  few-shot class-incremental learning scenario**) for detailed results for this part. Specifically, we pre-train the model on the base classes of the CUB200 dataset and incrementally learn the few-shot tasks in the miniImageNet dataset. The comparison results show that TEEN can also perform competitively. We speculate that existing methods often overly rely on specialized learning processes on the pre-training dataset. For example, methods like CEC require learning a classiﬁer adaptation module on the base classes. These modules, which are excessively trained on the pre-training dataset, may result in poor performance of existing methods in incremental learning scenarios with dataset shift. In contrast, *TEEN does not introduce additional modules for adaptation*. It simply leverages the potential semantic similarity between classes to calibrate the prototypes. This training-free characteristics allow TEEN to maintain competitive performance in scenarios with dataset shifts.\n\n---\n\n**A3** The semantic similarity based calibration is a straightforward and commonly used method used in the classification area. There may lack technical contribution to the community.\n\n**Q3** Different from existing semantic-based calibration methods, we propose a specific *training-free* calibration method (TEEN) to improve the discriminability of novel classes. Notably, the feature extractor used in TEEN is *only* trained on base classes and *does not involve any extra module or data to characterize the semantic similarity*. Besides, the empirical analysis and observations in our paper are also meaningful in the scenario of FSCIL. In our global response, we reiterate the contributions and novelty of our paper. Please refer to the global response for a detailed conclusion."}, {"review_id": "t2OVqFILJH", "reviewer": "Reviewer_TNew", "summary": "The paper proposes a strategy called Training-frEE calibratioN (TEEN) for Few-Shot Class-Incremental Learning (FSCIL) scenario to enhance the discriminability of new classes by fusing the new prototypes with weighted base prototypes. TEEN demonstrates remarkable performance and consistent improvements over baseline methods in the few-shot learning scenario.", "questions": "See the weakness section for more details", "limitations": "\nSee the weakness section for more details", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper addresses the Few-Shot Class-Incremental Learning (FSCIL) scenario, which is a challenging and important problem in real-world scenarios.\n\nThe proposed strategy, TEEN, is simple yet effective and demonstrates remarkable performance and consistent improvements over baseline methods in the few-shot learning scenarios.\n", "weaknesses": "The paper demonstrates significant improvement on new classes. However, based on the experimental results, it appears that the False Negative ratio, which involves classifying base instances into incorrect classes, may increase. It would be beneficial to address how this problem is handled and how the performance can be balanced between base classes and new classes. Additional analysis on this topic would be appreciated.\n\nAs stated by the authors, pre-training on a dataset that is independent of the subsequent data distribution would be advantageous.\n\nThe paper lacks a comparison of the proposed method with state-of-the-art methods in other few-shot learning scenarios, such as few-shot domain adaptation or few-shot semi-supervised learning. It would be beneficial to include more analysis and comparisons with these methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**  The paper demonstrates significant improvement on new classes. However, based on the experimental results, it appears that the False Negative ratio, which involves classifying base instances into incorrect classes, may increase. It would be beneficial to address how this problem is handled and how the performance can be balanced between base classes and new classes. Additional analysis on this topic would be appreciated\n\n**A1** We thank the reviewer for the constructive question. Firstly, we comprehensively analyze the potential negative effect of TEEN in section 4.2. From the final benchmark results presented in Section 5, it can be observed that **the potential negative effect is overshadowed by the more substantial positive effect of the improvement in the new prototypes**. Furthermore, we include a hyper-parameter $\\alpha$ to adjust the strength of the calibration. We give a detailed ablation study of $\\alpha$ in Figure 5. The hyper-parameter $\\alpha$ temporarily provides a simple solution to ensure performance and more complicated methods to mitigate the increasing False Negative ratio may be designed in the future.\n\n---\n\n**Q2** As stated by the authors, pre-training on a dataset that is independent of the subsequent data distribution would be advantageous. The paper lacks a comparison of the proposed method with state-of-the-art methods in other few-shot learning scenarios, such as few-shot domain adaptation or few-shot semi-supervised learning. It would be beneficial to include more analysis and comparisons with these methods. \n\n**A2** We thank the reviewer for the suggestion and the potentially related field. To the best of our knowledge, conventional methods in the few-shot scenarios do not consider the process of *incremental learning.* These methods were not designed with the continuous arrival of new data in mind, **making them unsuitable for FSCIL tasks**. If there are any papers closely related to the cross-dataset few-shot class-incremental learning scenario, we would appreciate it if you could directly point out those relevant papers and we will discuss these papers in the final version. Additionally, there is currently no strict definition for cross-dataset few-shot class-incremental learning (e.g., the selection of datasets or choice of evaluation metrics). In fact, we consider the cross-dataset setup as a general assumption and limitation in all FSCIL problems. Hence, in the **limitations** section, we mention this as a potential future research direction. However, to better demonstrate the performance of TEEN, we attempted a simple cross-dataset experiment by constructing a **CUB200->miniImageNet** dataset comparison to address your potential concerns. The results are shown in the global response (**Cross-dataset  few-shot class-incremental learning scenario**). Notably, TEEN still shows competitive performance. We speculate that existing methods (e.g., CEC and FACT) often employ complex learning algorithms that lead to excessive adaptation to the pre-training dataset. In contrast, TEEN relies on the calibration of semantic similarity, which may alleviate the potential negative impact of dataset shift.\n\n"}]

Author rebuttal:
We express our profound gratitude to the reviewers for their insightful and valuable comments. We are pleased that the reviewers find the simplicity and efficiency of the proposed method TEEN (RFbX, wHh2, 3Zyg, TNew) and our work clear and easy to follow (RFbX, 3Zyg, aEVg).  They also consider our method and empirical analysis interesting, novel and well-motivated (wHh2, aEVg, 3Zyg).  The reviewer wHh2 acknowledges our detailed ablation studies. In particular, reviewer RFbX acknowledges that our approach **takes a new class orientation, pushing the frontier in previously underexplored aspects and facilitating a fair and comprehensive comparison** and praises our work as **attractive**.

Before our rebuttal, we recapitulate our *contribution, novelty and some easily overlooked shining points*:

- We empirically find that new class performance was much lower than base class in previous methods. Then, we explore the causes of this phenomenon. *These analyses are first proposed in the FSCIL field, which points out that we should pay more attention to the performance of new classes*.

- We also observe that the feature extractor, solely trained on base classes, can capture the semantic similarity between the base and new classes.  Based on these analyses, we propose TEEN, which not only achieved a higher average accuracy but also improved the accuracy of new classes.

- In addition to the FSCIL task, we also validate TEEN on the Few-Shot Learning (FSL) task, which can also show competitive performance. In contrast, existing methods lack effectiveness for **both** FSCIL and FSL tasks.

  

To better address the reviewer's concerns, we conducted experiments primarily focusing on three aspects: cross-dataset evaluation, hyper-parameter robustness and the intuitive explanation of TEEN.

- **Cross-dataset few-shot class-incremental learning scenario**.  We *reuse* the feature extractor pretrained on the base session of **CUB200** and use this feature extractor for the incremental learning on **miniImageNet** datasets. Specifically, Session 0 consists of 100 classes from the CUB200 dataset, while the incremental sessions (Session 1-6) include classes from the miniImageNet dataset in a 5-way 5-shot format. In the cross-dataset incremental learning scenario, the performance of CEC and FACT  show a significant drop, while **the TEEN continues to perform competitively**. We speculate that these methods (CEC and FACT) may have designed overly complex modules (e.g., classiﬁer adaptation module in CEC) or mechanisms (e.g., the forward-compatible paradigm in FACT), which led to excessive adaptation to the pre-training dataset. In contrast, TEEN relies solely on the underlying semantic similarity relationships to calibrate the new class prototypes, which may make TEEN more resilient to dataset shifts.

|      | 0 (CUB200) |   1   |   2   |   3   |   4   |   5   |   6   |
| :--: | :--------: | :---: | :---: | :---: | :---: | :---: | :---: |
| CEC  |   75.85    | 65.34 | 63.21 | 61.30 | 59.24 | 57.29 | 55.21 |
| FACT |   75.90    | 63.02 | 60.12 | 58.42 | 55.32 | 53.54 | 51.48 |
| TEEN |   77.26    | 68.42 | 66.32 | 65.19 | 62.14 | 59.43 | 58.56 |

- **The robustness of the hyper-parameters $\alpha$ and $\tau$**. We show the performance trend with respect to $\alpha$ and $\tau$ on each benchmark dataset **in the Rebuttal PDF (Figure 2)**. To better illustrate the trends, we have included the replicated trend figure 5 (a)) in this rebuttal pdf.  We observed that the trend of the hyperparameter $\alpha$ remains consistent across all benchmark datasets. Although the trend of the hyperparameter $\tau$ may vary slightly between the CUB200, CIFAR100, and miniImageNet datasets, maintaining uniform settings still allows TEEN to maintain competitive performance. Notably, the trend of $\tau$ in miniImageNet and CIFAR100 datasets is perfectly identical. We hypothesize that the slight difference observed in CUB200 may be attributed to the fact that it is a *fine-grained* dataset.
- **An example that provides an intuitive explanation of TEEN using a toy example**. To better help reviewers understand the design of our calibration method, we show an intuitive visualization using a toy dataset sampled from a 2D Gaussian distribution (**Figure 1 in Rebuttal PDF**). Specifically, we constructed four classes, each corresponding to a Gaussian distribution. Three of these classes served as the base classes (with *a large number of* samples), while one class represented the new class (with *a small number of* samples). By applying TEEN to the new class, we can visually observe that the calibrated prototypes of the new class are closer to the desired class prototypes than their initial uncalibrated counterparts.

**We have included the requested comparison and discussions in the corresponding answers and the rebuttal PDF**. The relevant papers mentioned in the response or any further detailed discussions will be included in the final version. Please check the answers to specific comments and Rebuttal PDF for details.

Meta-review: This paper presents an approach for class-incremental learning in a few-shot framework. The core of the approach relies on similarity among classes. While I agree with some of the reviewer comments that this is not the most novel paper in terms of ideas: using similarity among classes for the incremental learning problem, it takes this simple idea and executes it well. It also has detailed experimental evaluations to support the proposed approach.

The rebuttal and the discussions that ensued have also convinced a couple of reviewers to increase their scores, while the others maintained their positive evaluations. Overall, the paper clearly shows something that works and how it works. Based on this, the AC recommends the paper for acceptance.
Decision: Accept (poster)