Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima
Authors: Guangyuan SHI
Jiaxin Chen
Wenlong Zhang
Li-Ming Zhan
Xiao-Ming Wu
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: incremental few-shot learning
catastrophic forgetting
flat minima

Abstract: This paper considers incremental few-shot learning, which requires a model to continually recognize new categories with only a few examples provided. Our study shows that existing methods severely suffer from catastrophic forgetting, a well-known problem in incremental learning, which is aggravated due to data scarcity and imbalance in the few-shot setting. Our analysis further suggests that to prevent catastrophic forgetting, actions need to be taken in the primitive stage -- the training of base classes instead of later few-shot learning sessions. Therefore, we propose to search for flat local minima of the base training objective function and then fine-tune the model parameters within the flat region on new tasks. In this way, the model can efficiently learn new classes while preserving the old ones. Comprehensive experimental results demonstrate that our approach outperforms all prior state-of-the-art methods and is very close to the approximate upper bound. The source code is available at https://github.com/moukamisama/F2M.

Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima
Guangyuan Shi," Jiaxin Chen; Wenlong Zhang, Li-Ming Zhan, Xiao-Ming Wu
Department of Computing
The Hong Kong Polytechnic University
(guang-yun yuan. shi, jiax. chen, wenlong zhang, lmzhan Amz zhan (achann@connen &connect. pociit...oo....... hk
xiao-ming. wtt-mmiig.uuuplyuu.....uumm wu@polyu. edu.
Abstract
This paper considers incremental few-shot learning, which requires model continually recognize new categories with only few examples provided. Our study shows that existing methods severely suffer from catastrophic forgetting, well-known problem in incremental learning, which is aggravated due to data scareity and imbalance in the few-shot setting. Our analysis further suggests that prevent catastrophic forgetting, actions need to be taken in the primitive stage the training of base classes instead of later few-shot learning sessions. Therefore, proposc to search for flat local minima of the base training objective function and then fine-tune the model parameters within the flat region on new tasks. this way, the model can efficiently learn new classes while preserving the old ones. Comprehensive experimental results demonstrate that our approach outperforms all prior state-of-the-ard methods and is very close the approximate upper bound. The source code is available at wtttt:://// // grithub.com com/ http///ooumiisssm sati
Introduction
Why study incremental few-shot learning? Incremental learning enables model continually learn new concepts from new data without forgetting previously learned knowledge. Rooted from real-world applications, this topic has attracted significant amount of interest in recent years (5) 30
31 40 26]. Incremental learning assumes sufficient training data is provided for new classes, which impractical in many application scenarios, especially when the classes which are costly or difficult to collect. This motivates the study of incremental few-shot learning, more difficult paradigm that aims continually learn new tasks with only few examples.
categories
Challenges. The major challenge for incremental learning is catastrophic forgetting 14 which refers the drastic performance drop previous tasks after learning new tasks. This phenomenon caused by the inaccessibility to previous data while learning on new data. Catastrophic forgetting presents bigger challenge for incremental few-shot learning. Due to the small amount of training data in new tasks, the model tends to severely overfit on new classes while quickly forgetting old classes, resulting in catastrophic performance.
28 35
Current research. The study of incremental few-shot learning has just started
9 S
34 [59]. Current works mainly borrow ideas from research in incremental learning to overcome the forgetting problem, by enforcing strong constraints on model parameters to penalize the changes of parameters 33 28 [66, by saving small amount of exemplars from old classes and adding
constraints on the exemplars to avoid forgetting 440 20 4]. However, in our empirical study, we find
"quual contribution Corresponding author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

that an intransigent model that only trains on base classes and does not tune on new tasks consistently outperforms state-of-the-e the-ttt methods, including a joint-training method 447 that uses all encountered data for training and hence suffers from severe data imbalance. This observation motivates us to address this harsh problem from different angle.
Our solution. Unlike existing solutions that try to overcome the catastrophic forgetting problem during the process of learning new tasks, we adopt a different approach by considering this issue during the training of base classes. Specifically, we propose to search for flat local minima of the base training objective function. For any parameter vector in the flat region around the minima, the loss is small, and the base classes are supposed to be well separated. The flat local minima can be found by adding random noise the model parameters for multiple times and jointly optimizing multiple loss functions. During the following incremental few-shot learning stage, we fine-tune the model parameters within the flat region, which can be achieved by clamping the parameters after updating them on few-shot tasks. In this way, the model can efficiently learn new classes while preserving the old ones. Our key contributions are summarized as follows:
We conduct comprehensive empirical study on existing incremental few-shot learning methods and discover that simple baseline model that only trains on base classes outper- forms state-of-the-a methods, which demonstrates the severity of catastrophic forgetting.
We propose novel approach for incremental few-shot learning by addressing the catas- trophic forgetting problem in the primitive stage. Through finding the flat minima region during training on base classes and fine-tume tne-ttnnig within the region while learning on new tasks, our model can overcome catastrophic forgetting and avoid overfitting.
Comprehensive experimental results on CIFAR-100, minimageNeet, and CUB-200-244 show that our approach outperforms all state-of-the-ar methods and achieves performance that is very close to the approximate upper bound.
2 Related Work
Few-shot learning aims to learn to generalize to new categories with a few labeled samples in each
class. Current few-shot methods mainly include optimization-basee based methods [12 23 (2 39 45
46, [55] and metric-based methods [13
methods can achieve fast adaptation to new tasks with limited samples by learning specific optimization algorithm. Metric based approaches exploit different distance metrics such as L2 distance [44), cosine similarity [49], and DeepEMD (58) in the learned metric/embedding space measure the similarity between samples. Recently, Tian er al. [48] find that standard supervised training can learn good metric space for unseen classes, which echoes with our observation on the proposed baseline model in Sec.
19
44
49
52
58
57 53]. Optimization-ban
Incremental learning focuses the challenging problem of continually learning to recognize new classes in new coming data without forgetting old classes 66
10 51]. Previous research 51 and multi-task incremental learning [21 31 42). To overcome the catastrophic forgetting problem, some attempts propose to impose strong constraints on model parameters by penalizing the changes of parameters [28] [.. Other attempts try to enforce constraints on the exemplars of old classes by restricting the output logits [40] or penalizing the changes of embedding angles [20]. In this work, our empirical study shows that imposing strong constraints on the arriving new classes may not be a promising way to tackle incrrenntal few-shot learning, due to the scarcity of training data for new classes.
mainly includes multi-class incremental learning 4 38 33
54
Incremental few-shot learning [4 60 8 aims to incrementally learn from very few samples. TOPCI (47) proposes neural gas network to learn and preserve the topology of the feature manifold formed by different classes. FSLL 34 only selects few model parameters for incremental learning and ensures the parameters are close to the optimal ones. To overcome catastrophic forgetting, IDLVQC 8 imposes constraints on the saved exemplars of each class by restricting the embedding drift, and Zhang et al. [59] propose to fix the embedding network for incremental learning. Similar to the finding of Zhang et al., we also discover that an intransigent model that simply does not adapt to new tasks can outperform prior -ate--of-the-art methods.
Robust optimization. It has been found that flat local minima leads to better generalization capabili- ties than sharp minima in the sense that a flat minimizer is more robust when the test loss is shifted

---

have been proposed to optimize neural networks towards flat local minima. In this paper, we show that for incremental few-shot learning, finding flat minima in the base session and tuning the model within the flat region on new tasks can significantly mitigate catastrophic forgetting.
due to random perturbations 17 24]. A substantial body of methods 22 37 II [5]
3 Severity of Catastrophic Forgetting in Incremental Few-Shot Learning
3.1 Problem Statement
Incremental few-shot learning (IFL) aims to continually learn recognize new classes with only few examples. Similar to incremental learning (IL), an IFL model is trained by a sequence of training sessions {D) D',, where D' { = (x/, vf)}; is the training data of session and æf is an example of class y E C (the class set of session (). In IFL, the base session DD' usually contains large number of classes with sufficient training data for each class, while the following sessions (t 2 2) only have small number of classes with few training samples per class, e.g., D' is often presented as an N N-way K-shot task with small N and K. The key difference between IL and IFL is, for IL, sufficient training data is provided in each session. Similar to IL, in each training session t of IFL,, the model has only access to the training data DD' and possibly a small amount of saved exemplars from previous sessions. When the training of session t is completed, the model is evaluated on test samples from all encountered classes C = U'=1 C², where it is assumed that there is no overlap between the classes of different sessions, i.e., J and i + j. C² nc = 01.
Catastrophic forgetting. IFL is undoubtedly more challenging problem than IL due to the data scarcity setting. IL. suffers from catastrophic forgetting, a well-known phenomenon and long-standing issue, which refers to the drastic drop test performance on previous (old) classes, caused by the inaccessibility of old data in the current training session. Unfortunately, catastrophic forgetting is an even bigger issue for IFL, because data scarcity makes it difficult adapt well to new tasks and learn new concepts, while the adaptation process could easily lead to the forgetting of base classes. In the following, we illustrate this point by evaluating a simple baseline model for IFL.
3.2 A Simple Baseline Model for IFL
We consider an intransigent model that simply does not adapt to new tasks. Particularly, the model only needs to be trained in the base session D and is directly used for inference in all sessions.
Training (t 1). We train a feature extractor f parameterized by é with a cully-connected layer as classifier by minimizing the standard cross-entropy loss using the training examples of DD' The feature extractor f is fixed for the following sessions 2 2) without any fine-tuning on new classes.
Inference (test). In each session, the inference is conducted by simple nearest class mean (NCM) classification algorithm (36). Specifically, all the training and test samples mapped to the embedding space of the feature extractor f, and Euclidean distance d(.,-) is used to measure the similarity between them. The classifier given by
1 11n==c) N.
ch argmin d({(x;))) (), where Pe cEC
(1)
where C denotes all the encountered classes, Pe refers to the prototype of class e (the mean vector of all the training samples of class in the embedding space), and Ne denotes the number of the training images of class c. Note that save the prototypes of all classes in C for later evaluation.
The baseline model outperforms state-of-the-art IFL and IL methods. We compare the above baseline model against -ftte-af- IFL methods including FSLL [34], IDLVQC [8 and TOPIC [47], IL methods including Rebalance (0) and iCarl 440, and a joint-training method that uses all previously seen data including the base and the following few-shot tasks for training, for IFL. The performance is evaluated on minilmageNNt, CIFAR- 100, and CUB-200. We tune the methods re-implemented by to the best performance. For the other methods, we use the results reported in the original papers. The experimental details are provided in Sec. 5 shown Fig. T the baseline model consistently outperforms all the compared methods including the joint-training
3

---

CIFAR-100
minilmageNet
CUB-200-2011
70
810
Accurasy
60
(y))) (56)
Accurasy
50
40
30
2
3
Sessions
Juini-training
ICaRL
Rehalance
ESLL
IDLVQ-C
Baseline
Figure I: Comparison of the proposed baseline model with state-of-the-an IFL and IL methods and the joint-training method.The baseline model outperforms all the other methods.
method (which suffers from severe data imbalance) every datase The fact that intransigent model performs best suggests that
For IFL, preserving the old (base classes) may be more critical than adapting to the new. Due to data scarcity, the performance gain on new classes is limited and cannot make up for the significant performance drop on base classes.
40 that enforce strong constraints on model parameters or exem- plars during fine-tuning on new classes cannot effectively prevent catastrophic forgetting in IFL, indicating that actions may need be taken in the base training stage.
Prior works
4 Overcoming Catastrophic Forgetting in IFL by Finding Flat Minima
The goal of IFL is to preserve the old while adapting to the new efficiently. The results and analysis in Sec. [3 suggest that might be "a bit late" try to prevent catastrophic forgetting in the few-shot learning sessions (1 2 2), which motivates us to consider this problem in the base training session.
Overview of our approach. To overcome catastrophic forgetting in IFL, propose to find b-flat (b > 0) local minima 8* of the base training objective function and then fine-tune the model within the flat region in later few-shot learning sessions. Specifically, for any parameter vector in the flat region, i.e., 8" b I z 8* + b, the risk (loss) of the base classes is minimized such that the classes well separated in the embedding space of fo. In the later incremental few-shot learning sessions 2 2), we fine-tune the model parameters within this region to learn new classes, i.e., to find
arg min
Σ L(z;0), s.t. 0" - ZED:
+
As such, the fine-tuned model 6' can adapt to classes while preserving the old ones. Also, due to the nature of few-shot learning, to avoid excessive training and overfitting, it suffices to tune the model in relatively small region. graphical illustration of approach and prior arts, as well as the notions of sharp minima and flat minima, are presented in Fig.
4.1 Searching for Flat Local Minima in the Base Training Stage
formal definition of b-flat local minima given as follows.
Definition (b-Flat Local Minima). Given a real-valued objective function L(z; for any b > O is a b-flat local minima of L(z; 0), if the following conditions are satisfied.
Condition 1: L(z; 0*) = L(z; + where -b S and b; = b.
Condition there exist Cj x
and C2 0* + s.t. f(z; 0) > L(2;8'), where
x 0 < 0* - and L(z; 0*) < L(z; 0), where 0 + b 4 C2.
We notice that similar observation is made in newly released paper
59

---

Loss of base classes
Loss of new classes
Sharp local minima
Flat local minima
Learned model
Direct Tuning
Adding Parameter Constraints
Finding Flat Minima
Clamping
Ra
Flat region
Rg
R1
01
0
0:
81
(a)
(b)
(c)
Figure 2: Illustration of our approach and existing solutions. - indicates the incremental learning steps on new classes. R and R2 respectively denote the loss of base classes before and after minimizing the loss of new classes. (a) SGD finds sharp minima in the base training. Directly tuming the model on new classes will result in severe performance drop on base classes. (b) Enforcing strong constraints on parameters by penalizing parameter changes [ 28 34 may still lead significant performance drop on base classes. (c) Finding flat local minima of base classes and clamping the parameters after trained on new classes make them fall within the flat region can effectively mitigate catastrophic forgetting.
In practice, it is hard to find the flat local minima that strictly satisfies the above definition, which may not even exist. Hence, our goal is to find an approximately flat local minima of the base training objective function. To this end, we propose to add some small random noise to the model parameters. The noise can be added for multiple times obtain similar but different loss functions, which will be optimized together to locate the flat minima region. The intuition is clear the parameter vectors around the flat local minima also have small function values.
To formally state the idea, we assume that the model is parameterized by = (cb w) where ø denotes the parameters of the embedding network and denotes the parameters of the classifier. z denotes a labelled training sample. Denote the loss function by L: TRd - IR. Our target is to minimize the expected loss function R: Rd R w.r.t. the joint distribution of data z and noise e, ie..
- -
S J Rdc JR. (1= dP())dP(e) = E[ +c,
R(O))
(2)
where P(z) is the data distribution and P(e) is the noise distribution, and æ and independent. Since it is impossible to minimize the expected loss, we minimize its estimation, the empirical loss, which is given by
M
1
[(0)
Σ
Crase(2 di tjj v), where
(3)
M
Lows (2:
Loelz; c,, + +>|c² SEDI
Σ
Σ llpe 11.1-..1.. CEC1
(4)
D1
where ej is a noise vector sampled from P(e). is the sampling times, Loe(z; o + ej v) refers to the cross-entropy loss of training sample and p. and p. are the class prototypes before and after injecting noise respectively. The first term of Loase is designed to find the flat region where the parameters á of the embedding network enforces the class prototypes fixed within such region, which is designed to solve the prototype drift problem [54 X (the class prototypes change after updating the network) in later incremental learning sessions such that the saved base class prototypes can be directly used for evaluation in later sessions.
well separate the base classes. The second term
4.2 Incremental Few-shot Learning within the Flat Region
In the incremental few-shot learning sessions (t 2 2), we fine-tune the parameters & of the embedding network within the flat region to learn new classes. It is worth noting that while the flat region might be relatively small, it is enough for incremental few-shot learning. Because only few training samples are provided for each new class, to prevent overfitting in few-shot learning, excessive training should be avoided and only small number of update iterations should be applied.
5

---

Algorithm 1: F2M
Input:
the flat region bound randomly initialized { w. the step sizes and B.
// Training over base classes t t=
for epoch = 1,2.... do
jo 1,2..... M do
Sample noise vector ; ~ P(c), s.t. -b bb
Add the noise to the parameters of the embedding network, i.e. 0 = { + ej. 163;
Compute the base loss Ctase with Eq.
Reset the parameters, i.e., 0 = {0, v)
end
Update 0 VV) with the loss L defined Eq. end
Normalize and save the protolype of each base class;
// Incremental learning 2 2
Combine the training data D and the exemplars saved in previous few-shot sessions 2 l. for epoch & = 1,2.... do
Compute the metric-based classification loss fm by Eq. 5
Update Φ Øb BV (2; é);
Clamp the parameters @ to ensure they fall in the flat minima region;
end
Randomly select and save few exemplars from the training data Dt:
Normalize and save the prototype of each new class; Output: Model parameters {Φ, })
We employ a metric-based classification algorithm with Euclidean distance to fine-tune the parametens. The loss function defined as
c-d(per((rvd))
(m o)
ΣΣ c) log( ZED
1
Easc e d(pey (x:zo)) ).z
(5)
where -(::-) denotes Euclidean distance, Pe is the prototype of class c, C C
Cs refers to all encountered classes, and D = D' UP denotes the union of the current training data DD and the exemplar set {12,... .... P--1): J. where Pt, (2 S t. t) is the of saved exemplars in session f.- Note that the prototypes of new classes are computed by Eq. [ and those of base classes are saved in the base session. After updating the embedding network parameters, we clamp them to ensure that they fall within the flat region, i.e. o² b 3 Φ 3 o" b, where ø² denotes the optimal parameter vector learned in the base session. After tuningn we evaluate the model using the nearest class mean classifier as in Eq. D with previously saved prototypes and newly computed ones. The whole training process described in Algorithm T Note that to calibrate the estimates of the classifier, we normalize all prototypes make those of base classes and those of new classes have the same norm.
4.3 Convergence Analysis
Our aim is to find flat region within which all parameter vectors work well. We then minimize the expected loss w.r.t. the joint distribution of noise and data z. To approximate this expected loss, sample from P(c) for multiple times in each iteration and optimize the objective function using stochastic gradient descent (SGD). Here, we provide theoretical guarantees for our method. Given the non-convex loss function in Eq. 3 prove the convergence of our proposed method. The proof
idea is inspired by the convergence analysis of SGD 3 27
Formally, in each batch k. let 2 denote the batch data, { be the sampled noises, and og be the step size. the base training session, we update the model parameters as follows:
a 02
OAk M
V Lrase (ZR; th + EGg ()) = 0g-
OCk
Σ g(zss dh + Ej vk))
(6)
where g(24) the + Ej, v/a) = Lause (zu: oh + Ej thx) the gradient. To formally analyze the convergence of our algorithm, we define the following assumptions.

---

Assumption 4.1 (L-smooth risk function). The expected loss function R I2 - R (Eq. is continuously differentiable and L-smooth with constant such that
|VR((0) VR(6)) L| 6%
This assumption is significant for the convergence analysis gradient- based optimization algorithms, since it limits how fast the gradient of the loss function can change w.r.t. the parameter vector.
Assumption 4.2. The expected loss function satisfies the following conditions:
Condition 1: R is bounded below by scalar R", given the sequence of parameters {0k}.
Condition 2: For all l E N and j E M),
Bz,,s (=Ks th + ej vh)) = (.(O)).
(8)
Condition 3: There exist scalars mi and m2 2 for all N and [1, M).
Ma (z<s Øx + Eg. vx.) - m, + m2 VR(0.)1
Exn.s denotes the expectation w.r.t. the joint distribution of random variables z and ej, and Ver,e, [ denotes the variance. Condition ensures that the expected loss R. is bounded by minimum value Hº during the updates, which is a natural and practical assumption. Condition assumes that the gradient g(zni Φb + εj,t%) is an unbiased estimate of RRO2... This is a striet assumption made to simplify the proof, but it can be easily relaxed to a general and easily-met condition that there exist (11 2 H2 > 0 satisfying ||Ezz.e, [g(=:: @x + cj. th))lll S f1 V (rl))((()))) R(#x)) ll and V (R(O))"Es..+ (z4; tE. vhx)] 2 μell vellv((O,) I12. Therefore, the convergence can proved in similar way using the techniques presented in the Appendix. Condition assumes the variance of the gradient g( on + Ej. (x) cannot be arbitrarily large, which is also reasonable in practice. To facilitate later analysis, similar to [43, we restrict the step sizes as follows.
Assumption 4.3. The learning rates satisfy:
Σ ak 0c,
2 ag
00.
(10)
This assumption can be easily met, since in practice the learning rate dr is usually far less than 1 and decreases w.r.t. k. Based on the above assumptions, we can derive the following theorem.
Theorem 4.1. Under assumptions 4.1 4.2 and 4.3 we further assume that the risk function R is twice differentiable, and that | R
is sg-smooth with constant L2 >> O, then we have
lim B ((0+)) 0.
(11)
k
This theorem establishes the convergence of our algorithm. The proof is provided in Appendix A.1.
Experiments
In this section, we empirically evaluate our proposed method for incremental few-shot learning and demonstrate its effectiveness by comparison with state-of-the-art methods.
5.1 Experimental Setup
Datasets. For CIFAR-100 and minilmageNet, randomly select 60 classes as the base classes and the remaining 40 classes as the new classes. In each incremental learning session, we construct 5-way 5-shot tasks by randomly picking 5 classes and sampling 5 examples for each class. For CUB-200- 20--200-001 with 200 classes, we select 100 classes as the base classes and 100 classes as the new ones. We test 10-way 5-shot tasks this dataset.
Baselines. We compare our method F2M with methods: the Baseline proposed in See. 3 joint- training method that uses all previously seen data including the base and the following few-shot
7

---

Table I: Classification accuracy CIFAR-100 100 for 5-way 5-shot incremental learning. indicates
our re-immy implemeenttiion.
sessions
The gap with cRT
Method
2
3
4
5
6
7
cRT (25
65.18 18
63.89
60.20
57.23
53.71
50.39
48.77
47.29
45.28
Joint-training
65.18
61.45
57.36
53.68
50.84
47.33
44.79
42.62
40.08
-5,20
Baseline
65.18
61.67
58.61
55.11
51.86
49.43
47.60
45.64
43.83
-1.45
iCaRL 40
66.52
57.26
54.27
50.62
47.33
44.99
43.14
41.16
39.49
-5.79
Rebalance
20
66,66
61.42
57.29
53.02
48.85
45.68
43.06
40.56
38.35
-6.93
FSLL 84
65.18
56.24
54.55
51.61
49.11
47.27
45.35
43,95
42.22
-3.08
iCaRL. 40
64.10
53.28
41.69
34.13
27.93
25.06
20.41
15.48
13.73
-31.55
Rebalance
64.10
$3.05
43.96
36.97
31.61
26.73
21.23
16.78
13.54
-31.74
TOPIC +7
64.10
55.88
47.07
45.16
40.11
36.38
33.96
31.55
29.37
-1599
FSL. 30
60
55.85
51.71
48.59
45.34
43.25
41.52
39.81
38.16
-7.12
FSLL+SS
34
66.76
55.52
52.20
49.
46.23
44.64
43.07
41.20
39.57
-5.71
F2M
64.71
62.05
59.01
55.58
52.55
49.96
48.08
46,28
44.67
-0.61
Table 2: Classification accuracy minilmageNett for 5-way 5-shot incremental learning. indicates our re-implementation.
sessions
The gap with cRT
Method
2
3
4
5
6
7
cRT 225
67.30
64.15
60.59
57.32
54,22
51.43
48.92
46.78
44.85
Joint-training"
67.30
62.34
57.79
54.08
50.93
47.65
44.64
42.61
40.29
-456
Baseline
67.30
63.18
59.62
56.33
53.28
50.50
47.96
45.85
43.88
-0.97
iCaRL 40
67.35
59.91
55.64
52.60
49.43
46.73
44.13
42.17
40.29
-4.56
Rehalance
20
67.91
6
58.75
54.83
50.68
47.11
43.88
41.19
38.72
-6.13
FSLL 34
67.30
59.81
57.26
54.57
52.05
49.42
46.95
44.94
42.87
-1.11
iCaRL 40
61.31
46.32
42.94
37.63
30.49
24.00
20.89
18.80
17.21
-27.64
Rehalance
61.31
47.80
39.31
25.68
21.35
18.67
17.24
14.17
-30.68
TOPIC (47
61.31
50.09
45.17
41.16
37.48
35.52
32.19
29.46
24.42
-20.43
FSLL 34
66.48
61.75
58.16
54.16
51.10
48.53
46.54
44.20
42.28
-2.57
FSLL+SS 34
68.85
63.14
59.24
55.23
52.24
49.65
47.74
45.23
43.92
-0.93
IDLVQ-C
64.77
59.87
55.93
52.62
49.88
47.55
44.83
43.14
41.84
-3.01
F2M
67.28
63.80
60.38
57.06
54.08
51.39
48.82
46,58
44.65
-0.20
tasks for training, the classifier re-training method (cRT) 25 for long-tailed classification trained
with all encountered data, iCaRL [40), Rebalance 20 TOPIC 47). FSLL [34], and IDLVQ-C [8]
For fair comparison, we re-implement cRT (25). iCaRL 40) Rebalance 20) FSLL
34 and the
joint-training method and tune them to their best performance. We also provide the results reported in the original papers for comparison. The results of TOPIC and IDLVQ-C g are copied from the original papers. Note that for IL. foint-training is naturally the upper bound of incremental learning algorithms, however, for IFL, coint-training not a good approximation of the upper bound because data imbalance makes the model perform significantly poorer on new classes (long-tailed classes). To address the data imbalance issue, we re-implement the cRT method as approximale upper bound.
Experimental details. The experiments are conducted with NVIDIA GPU RTX3090 on CUDA 11.0. We randomly split each dataset multiple tasks (sessions). For each dataset (with fixed split), we run each algorithm for 10 times and report the mean accuracy. We adopt ResNet18 [16) as the backbone network. For data augmentation, we use standard random crop and horizontal flip. In the base training stage, we select the last 4 or 8 convolution layers to inject noise, because these layers output higher-level feature representations. The flat region bound b set as 0.01. We the number of times for noise sampling as M ~ 4, since a larger M will increase the training time. In each incremental few-shot learning session, the total number training epochs is 6, and learning rate is 0.02. To verify the correctness of our implementation, we conduct experiments on incremental learning and compare our results to those reported on CIFAR- in Appendix A.3. More experiment details are provided in Appendix A.2.
8

---

Table 3: Classification accuracy CUB-200-200 1 for 10-way 5-shot incremental learning. indicates
our re- implementation.
sessions
The gap with cRT
Method
2
3
5
7
8
10
11
cRT [25]
80.83
78.51
76.12
73.93
71.46
68.96
67.73
66.75
64.22
62.53
61.08
Joint-training"
80.83
77.57
74.11
70.75
68.52
65.97
64.58
62.22
60.18
58.49
56.78
-4.30
Baseline
80.87
77.15
74.46
72.26
69.47
67.18
65.62
63.68
61.30
59.72
58.12
-2.96
iCaRL
40
79.58
67.63
64.17
61.80
58.10
55.51
53.34
50.89
48.62
47.34
45.60
-15.48
Rebalance
20
80,94
70.32
62.96
57.19
51.06
46,70
44,03
40.15
36.75
34,88
32.09
-28.99
FSLL 14
80.83
77.38
72.37
71.84
67.51
65.30
63.75
61.16
59.05
58.03
55.82
-5.26
iCaRL 40
52.65
48.61
44.16
36.62
29.52
27.83
26.26
24.01
23.89
21.16
-39,92
Rehalance
20
68868
57.12
44.21
28.78
26.71
25.66
24.62
21.52
20.12
20.06
19.87
.41.21
TOPIC [47
68.68
62.49
54.81
49.99
45.25
41,40
38.35
35.36
32.22
28.31
26,28
-34,80
FSLL [4
72.77
69.33
65.51
62.66
61.10
58.65
57.78
57.26
55.59
55.39
54.21
-6.87
ESLLASS
75.63
71.81
66.16
64.32
62.61
58.70
56.45
56.41
IDLVQ-C
8
77.37
74.72
70.28
67.13
65.34
63.52
62.10
61.54
59.04
58.68
57.81
-3,27
F2M
81.07
78.16
75.57
72.89
70.86
68.17
67.01
65.26
63.36
61.76
60.26
-0.82
Table 4: Comparison of the flatness of the local minima found by the Baseline and our F2M.
Indicator I
Variance o²
Method
Training Sct
Testing Sct
Training Sct
Testing Sct
0.2993 F2M 0.0506
Baseline
0.4582 0.0800
0.1451 0.0296
0,2395
0.0334
5.2 Comparison with the State-of-the-Art
F2M outperforms the state-of-the-art methods. The main results on CIFAR-100, minilmageNet and CUB-200-201 I are presented in Table Table 2 and
Based on the experi- ment results, have the following observations: 1) The Baseline introduced in Sec. outperforms the state-of-the-arr art approaches on all incremental sessions. 2) As expected, cRT consistently outper- forms the Baseline up 1% to 3% by considering the data imbalance problem and applying proper techniques to tackle the long-tailed classification problem to improve performance. Hence, it is reasonable to cRT the approximate upper bound of IFL.. 3) Our F2M outperforms the state-of- the-art methods and the Baseline. Moreover, the performance of F2M is very close to the approximate upper bound, i.e., the gap with cRT is only 0.2% in the last session on minilmageNet. The results show that even with strong constraints [20] 40 (34 and saved examplars of base classes [20] 40 8] current methods cannot effectively address the catastrophic forgetting problem. In contrast, finding flat minima seems promising approach to overcome this harsh problem.
5.3 Ablation Study and Analysis
Analysis on the flatness of local minima. Here, we verify that our method can find more flat local minima than the Baseline. For a found local minima 6", we measure its flatness follows. We sample the noise for 1000 times. For each time, we inject the sampled noise to 6* and calculate the loss
L.. 1000 1000 (L: Z)
Then, we adopt the indicator / = 1000 Σ i=1 (L, -² and variance x² 1
10000
to measure the flatness. L denotes the loss of 0*, and L denotes the average loss of { [=1 values of the indicator and variance of F2M and the Baseline are presented in Table which clearly demonstrate that our method can find a more flat local minima.
1000 The
Ablation study on the designs of our method. Here, we study the effectiveness of each design of our method, including adding noise to the model parameters for finding b-flat local minima (FM) during the base training session, the prototype fixing term (PF) used in the base training objective (Eq.4), parameter clamping (PC) during incremental learning, and prototype normalization (PN). We
conduct an ablation study by removing each component in turn and report the experimental results in T
Finding b-flat local minima. Standard supervised training with SGD as the optimizer tends to converge to sharp local minima. It leads to significant drop in performance because the loss changes quickly

---

Table 5: Ablation study of our F2M on CIFAR- 100. PD refers the performance dropping rate.
FM
PF
PC
PN
sessions
PD !
65.18
60.83
53.13
43.57
23.75
10.76
08.26
07.24
06.45
58.73
65.18
59.48
56.77
52.99
50.09
47.80
45.92
44.20
42.55
22.63
64.71
59.54
53.03
45.09
41.68
39.04
38.64
37.19
36.01
28.70
64.55
61.27
58.33
54.82
51.60
49.22
47.48
45.78
44.08
20.47
64.71
61.75
58.80
55.33
52.27
49.75
47.72
46.01
44.43
20.28
V
64.71
61.99
58.99
55.58
52.55
49.96
48.08
46.28
44.67
20.04
Table 6: Study of the flat region bound b for 5-way 5-shot incremental learning on CIFAR-100. The
top 3 results in each row are boldface.
The hyperparameter
Session
0.0025
0.005
0.01
0.02
0.04
0.08
Session (60 bases classes)
64.85
64.67
64.81
64.71
63.30
62.25
Session 9 (All 100 classes)
44.16
44.54
44.58
44.67
43.75
43.04
Scssion (60 base classes)
59.58
59.69
59.73
59.44
58.38
57.21
Scssion (40 new classes)
21.03
21.81
21.86
22.52
21.80
21.77
in the neighborhood of the sharp local minima. As shown in Table even with parameter clamping during incremental learning, the performance still drops significantly. contrast, restricting the parameters in small flat region can mitigate the forgetting problem.
Prototype fixing. Without fixing the prototypes after injecting noise to selected layers during the process of finding local minima, i.c. removing the second term of Eq. [] it is still possible to tune the model within the flat region well separate base classes. However, the saved prototypes of base classes will become less accurate because the embeddings of the base samples suffer from semantic drift [54 As shown in Table it results in a performance drop of nearly 0.6%.
Parameter clamping. Parameter clamping restricts the model parameters to the b-flat region after incremental few-shot learning. Outside the b-flat region, the performance drops quickly. It can be seen from Table that removing parameter clamping leads to a significant drop in performance.
Prototype normalization. As mentioned in Sec. 4.2 we normalize the class prototypes to calibrate the estimates of the class mean classifier. The results in Table (ableB]show the effectiveness of normalization, which helps to further improve the performance.
Study of the flat region bound b. We study the effect of the flat region bound b for 5-way 5-shot incremental learning on 100. We report the test accuracy in session (base session) and session (last session) w.r.t. different b in Table can be seen that the best results are achieved for b E [0.005, 0.02]. A larger b (e.g., 0.04 0.08) leads to significant performance drop on base classes, even for those in session 1, indicating that there may not exist a large flat region around good local minima. Meanwhile, a smaller b (e.g., 0.0025) results in a performance decline new classes, due to the overly small capacity of the flat region. This illustrates the trade-off effect of b.
Conclusion
We have proposed novel approach overcome catastrophic forgetting in incremental few-shot learning by finding flat local minima of the objective function in the base training stage and then fine-tuning the model within the flat region on new tasks. Extensive experiments benchmark datasets show that our model can effectively mitigate catastrophic forgetting and adapt to new classes. limitation of our method is that may be suitable for medium- high-shot tasks, since the flat region is relatively small, which limits the model capacity. However, it is still possible to adapt our core idea for incremental learning. For example, one can search for a less flat but wider local minima region in the base training stage and tune the model within this region during incremental learning sessions, where previous techniques such as elastic weight consolidation (EWC) (2 can be used to constraint the model parameters. This could be an interesting direction for future research.
10

---

Acknowledgments and Disclosure of Funding
We would like to thank the anonymous reviewers for their insightful and helpful comments. This research was supported by the grant of DaSAIL project P0030935 funded by PolyU/UGC.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference Computer Vision (ECCV), pages 139-154, 2018.
[2] Carlo Baldassi, Fabrizio Pittorino, and Riccardo Zecchina. Shaping the learning landscape in neural networks around wide flat minima. Proc. Natl. Acad. Sci. USA, 117(1):111-77,, 2020.
[3] Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. Siam Review, 60(2):223-31) 211, 2018.
[4] Francisco M Castro, Manuel J Alahari. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV), pages 233-248, 2018.
Nicolás Guil, Cordelia Schmid, and Karteek
[5] Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pages 532-547, 2018.
[6] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. International Conference on Learning Representations, 2018.
[7] Hung-Jen Chen, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, and Min Sun. Mitigating for- getting in online continual learning via instance-aware parameterization, Advances in Neural Information Processing Systems, 33, 2020.
[8] Kuilin Chen and Chi-Guhn Lee. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations, 2021.
[9] Ali Cheraghian, Shafin Rahman, Pengfei Fang. Soumava Kumar Roy, Lars Petersson, and Mchrtash Harandi. Semantic-aware knowledge distillation for few-shot class- incremenall learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2543, 2021.
[10] Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning without memorizing. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[11] Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Gal Elidan, Kristian Kersting, and Alexander Ihler, editors, Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017. AUAI Press, 2017.
[12] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In International Conference on Machine Learning. pages 1126 133.. PMLR, 2017.
[13] Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4367-4375, 2018.
[14] Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint axXiv:1312.6211, 2013.

---

[15] Haowei He, Gao Huang, and Yang Yuan. Asymmetric valleys: Beyond sharp and flat local minima. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché- Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 2549-2560, 2019.
[16) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and paltern recognition, pages 770-778, 2016.
[17] Geoffrey E. Hinton and Drew van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Lenny Pitt, editor, Proceedings of the Sixth Annual ACM Conference Computational Learning Theory, COLT 1993, Santa Cruz, CA, USA, July 26-28, 1993, pages 5-13. ACM, 1993.
[18] Sepp Hochreiter and Jürgen Schmidhuber. Simplifying neural nets by discovering flat minima. In Gerald Tesauro, David S. Touretzky, and Todd K. Leen, editors, Advances in Neural Information Processing Systems 7, [NIPS Conference. Denver, Colorado, USA, 1994], pages 529-536. MIT Press, 1994.
[19] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 4003-4014 2019.
[20] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 831-839 2019.
[21] Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In International Conference on Learning Representations, 2018
[22] Xinting Hu, Kaihua Tang. Chunyan Miao, Xian-Sheng Hua, and Hanwang Zhang. Distilling causal effect of data in class-incremental learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3957-3966, 2021.
[23] Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta-learning for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition, pages 17719-11727. 2019.
[24] Yiding Jiang*. Behnam Neyshabur*, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In International Conference on Learning Representations, 2020.
[25] Bingyi Kang, Saining Xie, Mareus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations, 2019.
[26] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. In International Conference on Learning Representations, 2018.
[27] J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. Annals of Mathematical Statistics, 23:462-4666 1952.
[28] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Ctrabska-Barwinee et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3222 2017.
[29] Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
[30] Ilja Kuzborskij, Francesco Orabona, and Barbara Caputo. From to n+ 1: Multiclass transfer incremental learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2013.
12

---

[31] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):935--2947, 2017.
[32] Yaoyao Liu, Bernt Schiele, and Qianru Sun. An ensemble of epoch-wise empirical bayes for few-shot learning. European Conference on Computer Vision, pages 404-421. Springer, 2020.
[33] Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi- class incremental learning without forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 12245-12254 2020.
[34] Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2337-2345, 2021.
[35] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
[36] Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance- based image classification: Generalizing new classes at near-zero cost. IEEE transactions pattern analysis and machine intelligence, 25(11):2624-2637 2013.
[37] Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi, Elizaveta Demyanenko, and Riccardo Zecchina. Entropic gradient descent algorithms and wide flat minima. In International Conference Learning Representations, 2021.
[38] Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13588 11588,11397, 2020.
[39] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
[40] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[41] Mengye Ren, Renjie Liao, Ethan Fetaya, and Richard $ Zemel. Incremental few-shot learning with attention attractor networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 5275-5285, 2019.
[42] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Ger- ald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations, 2018.
[43] H. Robbins. A stochastic approximation method. Annals of Mathematical Statistics, 22:400-407, 2007.
[44] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Proceedings of the s1 International Conference on Neural Information Processing Systems, pages 4080 4090, 2017.
[45] Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning through hard tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020.
[46] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Reta-transser learning for few- shot learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 403-412, 2019.
[47] Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few-shot fass-incrrmental learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12183-12192, 2020.
13

---

[48] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: good embedding is all you need? In Computer ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pages 266-282. Springer, 2020.
[49] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 3637-3645, 2016.
[50] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010.
[51] Qianru Sun Yaoyao Liu, Bernt Schiele. Adaptive aggregation networks for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition (CVPR), 2021.
[52] Han-Jia Ye, Hexiang Hu, and De-Chuan Zhan. Learning adaptive classifiers synthesis for generalized few-shot learning. International Journal of Computer Vision, pages 1-24, 2021.
[53] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8808-8817, 2020.
[54] Lu Yu, Bartlomiej Twardowski, X. Liu, Luis Herranz, Kai Wang, Yong mei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 6980-6988 2020.
[55] Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua. Interventional few-shot learning. Advances in Neural Information Processing Systems, 33, 2020.
[56] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987-3995. PMLR, 2017.
[57] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Differentiable earth mover's distance for few-shot learning. arXiv e-prints, pages arXiv-2003, 2020.
[58] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classifi- cation with differentiable earth mover's 's distance and structured classifiers. In Proceedings of the IEEE/VVF Conference on Computer Vision and Pattern Recognition, pages 12203-12213, 2020.
[59] Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning with continually evolved classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12455-12464, 2021.
[60] Hanbin Zhao, Yongjian Fu, Mintong Kang, Q Tian, F Wu, and X Li. Mgsvf: Multi-grained slow vs. fast framework for sho- class-incremental learning. arXiv preprint NEt::2006.155244
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
14

---

(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 4.3
(b) Did you include complete proof's of all theoretical results? [Yes] See Appendix A.I.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL.)? [Yes] The link the source code is provided in the abstract.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 5.1 and Appendix A.2.
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix A.3.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 5.1
4. If you are using existing assets (e.g., code, data, models) or curating/releusing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] We have cited 29
49 50
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "ktYzKJQYPRj", "reviewer": "Reviewer_oBeE", "summary": "This paper proposes an incremental few-shot learning method focusing on training the base model that can easily learn new classes, preserving the classification capability for the old classes. When training the base model, the proposed method, Finding Flat Minima (F2M) finds the flat minima where the loss function does not change a lot even if a random noise is injected. When the novel classes are given, F2M updates the feature extractor and clamp the parameters to make the feature extractor resides in the flat minima region. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Strength]\n1.\tThe idea of finding flat minima to overcome the catastrophic forgetting and ensure the base performance is interesting. The proposed F2M outperforms the other incremental few-shot learning methods for all sessions except for the base classes training phase (t=1).\n2.\tThe proposed F2M does not saves the data from prior sessions except for prototypes for inference. \n3.\tThe paper is well written and easy to follow. Also, the detailed ablation experiment results are provided to help understanding. \n\n[Weakness]\n1.\tAlthough the paper argue that proposed method finds the flat minima, the analysis about flatness is missing. The loss used for training base model is the averaged loss for the noise injected models, and the authors provided convergence analysis on this loss. However, minimizing the averaged loss across the noise injected models does not ensure the flatness of the minima. So, to claim that the minima found by minimizing the loss in Eq (3), the analysis on the losses of the noise-injected models after training is required.\n2.\tIn Eq (4), the class prototypes before and after injecting noise are utilized for prototype fixing regularization. However, this means that F2M have to compute the prototypes of the base class every time the noise is injected: M+1 times for each update. Considering the fact that there are many classes and many samples for the base classes, this prototype fixing is computationally inefficient. If I miss some details about the prototype fixing, please fix my misunderstanding in rebuttal.\n3.\tAnalysis on the sampling times $M$ and noise bound value $b$ is missing. These values decide the flat area around the flat minima, and the performance would be affected by theses value. However, there is no analysis on $M$ and $b$ in the main paper nor the appendix. Moreover, the exact value $M$ used for the experiments is not reported. \n4.\tComparison with single session incremental few-shot learning is missing. Like [42] in the main paper, there are some meta-learning based single session incremental FSL methods are being studied. Although this paper targets on multi-session incremental FSL with different setting and different dataset split, it would be more informative to compare the proposed F2M with that kind of methods, considering that the idea of finding flat minima seems valuable for the single session incremental few-shot learning task too.\n\n* There is a typo in Table 2 – the miniImageNet task is 5-way, but it is written as 10-way.\n\n-------------------------------------------------------------------------------------------------------------------------------------------\nPost Rebuttal\n- Reviewer clarified the confusing parts of the paper, and added useful analysis during rebuttal. Therefore, I raise my score to 6.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors did not addressed the limitations of  the paper. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "MwL5UNIFel", "reviewer": "Reviewer_aQXy", "summary": "The paper introduces a novel approach for few-shot incremental learning, which searches for the flat local minima of the base training objective function and then fine-tune the model parameters within the flat region on new tasks. This way, the model could learn the new classes without forgetting the old ones. The approach is well-motivated and is scientifically sound.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-documented and clearly written. The related work section covers the most relevant papers in the literature.  Extensive experiments have been performed and the approach was conveniently compared with the current state-of-the-art, demonstrating its superiority.\nHere are my concerns:\n1. Section 4.1: Definition 1, Condition 1: 'b_i' is not introduced. Who is 'epsilon'?\n2. In Section 4.3, you make a series of assumptions. How realistic are they in practice? What measures do you take in order to guarantee that they are met?\n3. How do you find the optimal parameter 'b'? What value do you use in your experiments? A study of the effect of the parameter 'b' on the performance would be interesting.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper does not have a negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "Not applicable.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours reading + 1 hour filling the questionnaire"}, "extra_text": {}}, {"review_id": "HOdTyhLStCT", "reviewer": "Reviewer_P52B", "summary": "This paper describes an approach to incremental few-shot learning based on\nfinding a good starting configuration in parameter space that leads to more\nstable incremental few-shot learning with less forgetting. The authors propose\nto learn an approximately flat region on the initial training set and to then\nconstrain few-shot incremental learning to remain within this flat region and\nthus minimize forgetting. Experimental results are given on a wide range of\nfew-shot incremental benchmarks, including CIFAR-100, miniImageNet, and CUB-200.\n", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is very well-written with excellent motivations, empirical and\ntheoretical follow-through, and convincing experimental evaluation. I have\npicked and poked and prodded at the work, but can find very little to improve. I\nespecially appreciated the empirical motivations of the work in section 3, as\nwell as the convergence analysis. A few that I think should be addressed:\n\n1. The motivation of starting from a flat region and regularizing incremental\n   learning based on it is reminiscent of Elastic Weight Consolidation, but\n   motivated from a different starting point -- pun intended since EWC does not\n   constrain the initial starting point, but rather only regularizes based on an\n   approximation of the loss surface at an initial solution. It would be\n   interesting to see how and why regularization using the Laplace approximation\n   of the loss landscape (a la EWC) around a flat minima works when compared to\n   clamping to remain in the flat region.\n2. I do not find any discussion of the flat region bound $b$ used to determine\n   the flat region. I expect a smaller $b$ leads to a smaller flat region and\n   thus less plasticity, and a larger $b$ to a larger flat region and more risk\n   of forgetting. It would be nice to see an ablation on this parameter to\n   understand better this tradeoff.\n\n**To summarize**: I really like this paper and I think it is novel approach that\nadds something very interesting to the discussion on incremental learning.\n\n**Post-rebuttal Summary**: I appreciate the author efforts to clarify and provide additional ablations. Other reviewers point to somewhat more critical blemishes and/or questions about the proposed approach, but I am still very positive about the work. The manuscript is very well-reasoned and well-written, and I think it adds something interesting to the discussion on few-shot IL. I do suggest that the authors consider carefully the comments from reviewer dLsD on carefully articulating the contribution of the work and the relationship between flat minima, few-shot learning, and incremental learning.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Discussion of limitations and societal impact is entirely missing.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "4rPHnOBHunF", "reviewer": "Reviewer_dLsD", "summary": "The proposed method is a nice trial for tackling the issues in a challenging data setting --- class incremental learning with few-shot samples in every new class. The paper writing is clear to show that the converge of the proposed method is theoretically guaranteed and the performance shows better than some baselines --- incremental learning methods (but missing incremental few-shot learning methods introduced in Related Works [48, 42, 62, 9, 34, 61]). ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I have two concerns about this paper. One is the performance should be highly related to the discrepancy gap between base classes and the coming (new) classes in incremental learning phases but there is no analysis on this. The other one is the motivation is not very strong to use in the only setting of few-shot samples in new classes. Please check the detailed comments.\n\n1. The performance looks good but this paper only shows its comparisons with some incremental learning methods (baselines methods implemented in a new incremenral few-shot settings). The comparisons to closely related works such as [48, 42, 62, 9, 34, 61] are missing totally. \n\n2. The performance of the models are related to the pattern differences between base classes and the coming (new) classes, which is not analyzed in the paper (as it used only one fixed setting of class splits). Given a larger difference between base and new classes, does the found flat minima (using base classes) still work for learning new classes? In addition, the experiments are a bit limited to small datasets. \n\n3. Flat minima regions guarantee a good generalization of the trained model to new classes. Is this especially helpful for incremental learning in few-shot settings? It is interesting to see its empirical performance on the standard settings of class incremental learning, e.g., using ImageNet which is large-scale or at least using Sub-ImageNet.\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: After reading the paper and other reviews, and interaction with the authors via the rebuttal, all reviewers recommend to accept the paper. It's well written and motivated, and the proposed method, which is relatively simple, is reasonably evaluated in the context of incremental few-shot learning.
Decision: Accept (Spotlight)