Prototype Recalls for Continual Learning
Authors: Mengmi Zhang
Tao Wang
Joo Hwee Lim
Jiashi Feng
Venue:  (2020)
Keywords: continual learning
catastrophic forgetting
prototypes
image classification
few-shot continual learning

Abstract: Continual learning is a critical ability of continually acquiring and transferring knowledge without catastrophically forgetting previously learned knowledge. However, enabling continual learning for AI remains a long-standing challenge. In this work, we propose a novel method, Prototype Recalls, that efficiently embeds and recalls previously learnt knowledge to tackle catastrophic forgetting issue. In particular, we consider continual learning in classification tasks. For each classification task, our method learns a metric space containing a set of prototypes where embedding of the samples from the same class cluster around prototypes and class-representative prototypes are separated apart. To alleviate catastrophic forgetting, our method preserves the embedding function from the samples to the previous metric space, through our proposed prototype recalls from previous tasks. Specifically, the recalling process is implemented by replaying a small number of samples from previous tasks and correspondingly matching their embedding to their nearest class-representative prototypes. Compared with recent continual learning methods, our contributions are fourfold: first, our method achieves the best memory retention capability while adapting quickly to new tasks. Second, our method uses metric learning for classification and does not require adding in new neurons given new object classes. Third, our method is more memory efficient since only class-representative prototypes need to be recalled. Fourth, our method suggests a promising solution for few-shot continual learning. Without tampering with the performance on initial tasks, our method learns novel concepts given a few training examples of each class in new tasks. 

Under review as a conference paper at ICLR 2020
PROTOTYPE RECALLS FOR CONTINUAL LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Continual learning is a critical ability of continually acquiring and transferring knowledge without catastrophically forgetting previously learned knowledge. However, enabling continual learning for Al remains loog-sttadding challenge. In this work, we propose a novel method, Prototype Recalls, that efficiently embeds and recalls previously learnt knowledge to tackle catastrophic forgetting issue. In particular, we consider continual learning classification tasks. For each classification task, our method learns metric space containing a set of prototypes where embedding of the samples from the same class cluster around prototypes and dlassreepreeentative prototypes
separated apart. To alleviate catastrophic forgetting, our method preserves the embedding function from the samples to the previous metric space, through our proposed prototype recalls from previous tasks. Specifically, the recalling process is implemented by replaying a small number of samples from previous tasks and correspondingly matching their embedding to their nearest class-represe resressttseet prototypes. Compared with recent continual learning methods, our contributions are fourfold: first, our method achieves the best memory retention capability while adapting quickly to new tasks. Second, our method uses metric learning for classification, and does not require adding in new neurons given new object classes. Third, our method is more memory efficient since only lass-reppesentattiv prototypes need to be recalled. Fourth, our method suggests promising solution for few-shot continual learning. Without tampering with the performance on initial tasks, our method learns novel concepts given a few training examples of each class in new tasks.
INTRODUCTION
Continual learning, also known as lifelong learning, the crucial ability for humans to continually acquire and transfer new knowledge across their lifespans while retaining previously learnt experiences Hassabis et (2117). This ability also critical for artificial intelligence (Al) systems to interact with the real world and process continuous streams of information Thrun & Mitchell (1995). However, the continual acquisition of incrementally available data from non-stationary data distributions generally leads to catastrophic forgetting in the system McCloskey & Cohen (198);; Ratcliff (1990); French] (1999). Continual learning remains a long-standing challenge for deep neural network models since these models typically learn representations from stationary batches of training data and tend to fail to retain good performances in previous tasks when data become incrementally available over tasks Kemker et al. (2018): Maltoni & Lomonaco (2019).
Numerous methods for alleviating catastrophic forgetting have been currently proposed. The most pragmatical way is jointly train deep neural network models on both old and new tasks, which however demands a large amount of resources to store previous training data and hinders the learning of novel data in real time. Another option is to complement the training data for each new task with "pseudo-data" of the previous tasks [Shin et al.] (2017): Robins (1995). Besides the main model for task performance, separate generative model trained to generate fake historical data used for pseudo-rehearsal. Deep Generative Replay (DGR) [Shin et al.] (2017) replaces the storage of the previous training data with a Generative Adversarial Network to synthesize training data on all previously learnt tasks. These generative approaches have succeeded over very simple and artificial inputs but they cannot tackle more complicated inputs Atkinson et al. (2018). Moreover, to synthesize the historical data reasonably well, the size of the generative model is usually huge that costs much memory Wen et al. (2018). An alternative method is to store the weights of the model trained previous tasks, and impose constraints of weight updates new tasks He & Jaeger

---

Under review as a conference paper at ICLR 2020
2018); Kirkpatrick et al. (0177); ) Zenke al. (2017): Lee al. (2017) Lopez-Paz Paz al. (2017)
For example, Elastic Weight Consolidation (EWC) Kirkpatrick et al. (2117) 7) and Learning Without Forgetting (LwF) Li & Hotem (2018) store all the model parameters on previously learnt tasks, estimate their importance on previous tasks and penalize future changes to the weights on new tasks. However, selecting the "important" parameters for previous tasks complicates the implementation by exhaustive hyper-parameter tuning. In addition, state-of-the-art neural network models often involve millions of parameters and storing all network parameters from previous tasks does not necessarily reduce the memory cost Wen et al. (2018). In contrast with these methods, storing a small subset of examples from previous tasks and replaying the "exact subset" substantially boost performance Kemker & Kanan (2017); Rebuffi et al. (2017); Nguyen et al. (2017). To achieve the desired network behavior on previous tasks, incremental Classifier and Representation Learner (iCARL) Rebuffi et al.] (2017) and Few-shot Self- (FSR) [Wen et al. (2018) follow the idea of logit matching or knowledge distillation in model compression Ba & Caruana (2014); Bucilua cet al. (2006); Hinton et al.] (2015). However, such approaches ignore the topological relations among clusters in the embedding space and rely too much on a small amount of individual data, which may result in overfitting as shown in experiments (Section 4.2). In contrast with them, without tampering the performance in memory retention, our method learns embedding functions and compares the feature similarities represented by class prototypes in the embedding space which improves generalization, especially in the few-shot settings, as also been verified in works Hoffer & Ailon (2015): Snell al. (2017).
In this paper, we propose the method, Prototype Recalls, for continual learning in classification tasks. Similar as et al. (2017), we use a neural network learn class-representative prototypes in an embedding space and classify embedded test data by finding their nearest class prototype. To tackle the problem of catastrophic forgetting, we impose additional constraints on the network by classifying the embedded test data based on prototypes from previous tasks, which promotes the preservation of initial embedding function. For example (Figure []. in the first task (Subfigure [Ta), the network learns color prototypes to classify blue and yellow circles and in the second task (Subfigure [bb), the network learns shape prototypes to classify green circles and triangles. With chtastrophically forgetting color features, the network extracts circle features the first task and fails to classify blue and yellow circles. To alleviate catastrophic forgetting, our method replays the embeded previous samples (blue and yellow circles) and match them with previous color prototypes (blue and yellow) which reminds the network of extracting both color and shape features in both classification tasks.
We evaluate our method under two typical experimental protocols, incremental domain and incremental class, for continual learning across three benchmark datasets, MNIST Dengi (2012), CIFAR 10 Krizhevsky & Hinton (2009) and minilmageNNet Deng et al.] (2009). Compared with the state-of-thh- our method significantly boosts the performance of continual learning in terms of memory retention capability while being able to adapt to new tasks. Unlike parameter regularization methods or iCARL or FSR, our approach further reduces the memory storage by replacing logits of each data or network parameters with one prototype of each class in the episodic memory. Moreover, in contrast to these methods where the last layer in traditional classification networks often structurally depends on the number of classes, our method leverages metric learning, maintains the same network architecture and does not require adding new neurons or layers for new object classes. Additionally, without sacrificing classification accuracy on initial tasks, our method can generalize to learn new concepts given a few training examples in new tasks due to the advantage of metric learning, commonly used in few-shot settings (nell et al. (0017) Hoffer & (0155)
2 PROPOSED METHOD
We propose the method, Prototype Recalls, for continual learning. For a sequence of datasets D1, D2...., Dt.... given D. in any task t where t E (1,,,..,?): the goal for the model fr is to retain the good classification performance on all T datasets after being sequentially trained over T tasks. The value of T is not pre-determined. The model fr with learnable parameters ó is only allowed to carry over a limited amount of information from the previous T - 1 tasks. This constraint eliminates the naive solution of combining all previous datasets form one big training set for fine- tuning the model fr at task T. Each dataset D, consists of N, labeled examples D, (X,, Yi)- = {(r1e, yir), (N yNt) where each Fat E R is the D-dimensional feature
2

---

Under review as a conference paper at ICLR 2020
1
D
Dr
1 1
3,
Ct,t
f
ft-1
ft
?
C2,t
C2
22t
C,,t
Task = t-1
Task
Task
(a) Classification
(h) Eliminating Catastrophic Forgetting
Figure I: Illustration of classification and catastrophic forgetting alleviation in our proposed method. In (a), dataset D,-1 contains two classes. Prototypes C1,--1 and 02 1 obtained by averaging example embeddings of the same class. White square with question mark (embedded test data) be classified by finding its nearest prototype. In (b), dataset D, containing two new classes is introduced in task t. The prototypes Ci,,, i 1,....
task are computed from D, U D.-1 where Dt-1 randomly sampled subset from D--1- To eliminate catastrophic forgetting, our method constantly recalls metric space at task - 1 by making example embeddings on D as denoted by dashed color shapes, close prototypes Ct-1.
vector of an example and yir E {I,..,, Kr} is the corresponding class label. Ske denotes the set of examples labeled with class kG-
At task if we simply train a model by only minimizing the classification loss Lelossi, Drr on dataset Dr. the model will forget how perform classification previous datasets Dz, t T which is described as catastrophic forgetting problem McCloskey & Cohen (1989); Ratchiff 1990); French (1999). Here show how the model trained in method retains the good performance all previous tasks while adaptively learning new tasks. The loss for all the previous datasets is denoted by Lr(f)= Et= ED, (Lf(x)) Y.). Our objective is to learn fr defined as follows:
fr arg min min
(f) arg min
Lelassi. D. (ft) + Letassi, Dr (f) + Σ (D.(ff)))
(I)
t=1
t=1
where Letossi, Dr (f) defines the classification loss of f on dataset DT and 8p, (f ft) measures the differences the network behaviors in the embedding space learnt by and fe D introduced later in Equ 7 Given fi.....fze fr-1 that are learnt from the previous tasks, at task T, learning fr requires minimizing both terms Lelassi, D, (f) and E 6p. ft). In the subsections below and we describe how to optimize these two terms.
Figure I
2.1 CLASSIFICATION
To perform classification on datasct Dr, our method learns an embedding space in which points cluster around single prototype representation for each class and classification is performed by finding the nearest class prototype [Snell al. (2017) (Figure Lap. Compared classification networks with specific classification layer attached in the end, such as iCARL and FSR, our method keeps the network architecture unchanged while finding the nearest neighbour in the embedding space, which would lead to more efficient memory usage. For example, in one of the continual learning protocols Snell al. (2017) where the models are asked to classify incremental classes (also see Section 1.I). traditional classification networks have to expand their architectures by accommodating more output units in the last classification layer based on the number of incremental classes and consequently, additional network parameters have to be added into the memory.
traditional
Without loss of generality, here we show how our method performs classification on Dr. First, the model learns embedding function RD R and computes an M -imimensional prototype OAT E AM which is the mean of the embeddings from examples SeT:
CKT
|krTl
Σ
((TT).
(2)
(NeT.Ber S
3

---

Under review as a conference paper at ICLR 2020
The pairwise distance of one embedding and one prototype within the same class should be smaller
than the intra-class ones. Our method introduces a distance function : RM x M - [0, oo). For each example Ir, it estimates distance distribution based softmax over distances to the prototypes of Ky classes in the embedding space:
exp(-d(f(er))
Po(UT = kT(æT) = Σkr Kr exp(-d(f (xT), (xT)). ceT))
(3)
The objective function Lelussi, Dr (J) is to minimize the negative log-probability log (v kr ( of the ground truth class label kr via Stochastic Gradient Descent Bottou (2010):
D, (f) = -og (v = kr (mr)
(4)
In practice, when Ny is large, computing CKT costly and memory inefficient during training. Thus, at each training iteration, we randomly sample two complement subsets from SeT over all KT classes: one for computing prototypes and the other for estimating distance distribution. Our primary choice of the distance function d(-) is squared Euclidean distance which has been verified to be effective in Snell et al. (2017). In addition, we include temperature hyperparameter T in d(-) as introduced in network distillation literature [Hintnn et al. (2015) and set its value empirically based on the validation sets. A higher value for T produces a softer probability distribution over classes.
2.2 PROTOTYPE RECALL
Regardless of the changes of the network parameters from de कr at task 1 and T respectively, the primary goal of fr is to learn the embedding function which results in the similar metric space as fe on dataset D; in task (Figure lb). Given a limited amount of memory, a direct approach is to randomly sample a small subset D. = {xx (0(") = =1,,..... L..., m) from D, and replay these examples on task T. There have been some attempts Chen et (2012) Koh & Liang (2017); |Brahma & Othon (2018) selecting representative examples for Dj based on different scoring functions. However, the recent work Wen et al. (2018) has shown that random sampling uniformly across classes has already yielded outstanding performance in continual learning tasks. Hence, adopt the same random sampling strategy to form 1),
Intuitively, if the number of data samples in D, is very large, the network could re-produce the metric space at task by replaying Dr, which is our desired goal. However, this does not hold in practice given limited memory capacity. With the simple inductive bias that the metric space at task / can be underlined by class-representative prototypes, we introduce another loss that embedded data sample in Dj should still be closest to their corresponding class prototype among all prototypes at task t. This ensures the metric space represented by a set of prototypes learnt from D) by fr provides good approximation to the one in task l.
Formally, for any after task f, we formulate the regularization of network behaviors dD, (f, f() in the metric space of task t by satisfying two criteria: first, f learns a metric space to classify D. by minimizing the classification loss Letassi, (J), as introduced in Sec. 2. above; second, to preserve the similar topological structure D.. (f, ft) among clusters on dataset D., the embeddings fāt) predicted by f based on Dù should produce the similar distance distribution based on softmax over the distance to prototypes ekt computed using fo on dataset D,: ((a)
(u = ke)()t) Σk exp( (ap(-d[jr (( (11(
Ckt
|Skll
Σ
felxa))
(5)
Concretely, L D.., fi) to minimize the negative log- aog-rrbbbblityy po(βe ke) of the ground truth class label k; conditioned prototypes Cki, which is pre-computed using f. in Eg 5 at task t and stored in the episodic memory until task T::
L De, (f, log (o = ky (.).E.)
(6)
Overall, define 6p, (f. f.) in Eq 1as below:
pp. (J. ft) LaaassiiD, (f) reguu. (S, St)-
4

---

Under review as a conference paper at ICLR 2020
Algorithm 1: Prototype recall algorithm in continual learning for training episode
Input A sequence of datasets D1, Da, Dr. per task l. leed-forware neural network learning embedding function f. Episodic memory with capacity Sampled m examples per dataset. Output: Update the network parameters di
for each task t do then
if
Classity D1: Compute prototypes (CL1, .. CKI
// Equ.
Store prototypes (ci1,..,, CKI] and m sampled examples Di in episodic memory;
else
Classify Di; Compute prototypes {cie, .., Cic): for all previous tasks from 1
// Equ.
3
do
minimize 8D, (f, ft) using following:
Equ
Classify D:
Equ. 2
Prototype recalls using D. and prototypes (C, CKi) } Update m based C:; Store prototypes {Ci CKt} and D. episodic memory: // Egu.
episodic memory // Equ. 5
Training
Training
Task (permutation 1)
Tack (2 classes)
2
5
7
g ?
(permutation 2]
Task (3 classes, classes from previous tasks)
Task 20 (permutation 20)
(10 classes, previous tasks)
Evaluation Testing
Testing
With permutation unknown, which digit the given image (from 9)?
With task unknown, which class is the
given image (incremental across tasks)?
(a) Permuted MNIST in Incremental Domain (b) Split CIFAR10 10 in Incremental Class
Figure Schematics of two task protocols in our continual learning experiments: (a) learning with incremental domain on permuted MNIST; and (b) learning with incremental classes on split CIFAR10.
2.3 DYNAMIC EPISODIC MEMORY ALLOCATION
Given limited amount of memory with capacity C, our proposed method has to store small subset D, with examples randomly sampled from D, and prototypes Ckt, k {L,..., Ke} computed using embedding function f: R > M on D, where < T. The following constraint has to be satisfied:
T-1 C C= K.KM mDD
(8)
When the number of tasks is small, m can be large and the episodic memory stores more examples D. Dynamic memory allocation of enabling more example replays in earlier tasks puts more emphasis reviewing carlier tasks which casier to forget, and introduces more varieties in data distributions when matching with prototypes. Pscudocode to our proposed algorithm in continual learning for training episode is provided in Algorithm
3 EXPERIMENTAL DETAILS
We introduce two task protocols for evaluating continual learning algorithms with different memory usage three benchmark datasets. Source codes will be public available upon acceptance.

---

Under review as a conference paper at ICLR 2020
3.1
TASK PROTOCOLS
Permuted MNIST in incremental domain task is benchmark task protocol in continual learning Lee et al. (2017); Lopez-Paz et al. (2017); Zenke al. (2017) (Figure 2a). In each task, fixed permutation sequence is randomly generated and is applied to input images in MNIST Deng (2012). Though the input distribution always changes across tasks, models are trained to classify 10 digits in each task and the model structure is always the same. There 20 tasks in total. During testing, the task identity is not available to models. The models have to classify input images into out of 10 digits.
Split CIFARIO and split MinilmageNett incremental class task is a more challenging task protocol where models need to infer the task identity and meanwhile solve each task. The input data is also more complex which includes classification on natural images in CIFARIO Krizhevsky & Hinton 2009) and minilmageNet Deng et al. (2009). The former contains 10 classes and the latter consists of TO0 classes. In CIFARTO, the model is first trained with 2 classes and later with 1 more class in each subsequent task. There are 9 tasks in total and 5,000 images per class in the training set. In minilmageNet, models are trained with 10 classes in each task. There are 10 tasks in total and 480 images per class in the training set.
Few-shot Continual Learning Humans can learn novel concepts given a few examples without sacrificing classification accuracy on initial tasks Gidaris & Komodakis (2018). However, typical continual learning schemes assume that a large amount of training data over all tasks is always available for fine-tuning networks to adapt to new data distributions, which does not always hold in practice. We revise task protocols to more challenging ones: networks are trained with a few examples per class in sequential tasks except for the first task. For example, on CIFAR ImminilmageNee,, we train the models with 5,000/480 example images per class in the first task and 50/100 images per class in subsequent tasks.
3.2 BASELINES
We include the following categories of continual learning methods for comparing with our method. To eliminate the effect of network structures in performance, introduce control conditions with the same architecture complexity for all the methods in the same task across all the experiments.
Parameter Regularization Methods: Elastic Weight Consolidation (EWC) Kirkpatrick al. (017). Synaptic Intelligence (SI) Zenke et al. (2017) and Memory Aware Synapses (MAS) al. (201)) where regularization terms are added in the loss function; online EWC[/ccwar et al. (0188) which is extension of EWC in scalability to a large number of tasks; L2 distance indicating parameter changes between tasks added in the loss Kirkpatrick al. 2):17); SGD, which is a naive baseline without any regularization terms, is optimized with Stochastic Gradient Descent Bottou 2010) sequentially over all tasks.
Memory Distillation and Replay Methods: incremental Classifier and Representation Learner (iCARL) Rebuffi et al.] (2017) and Few-shot Self-Reminder (FSR) Wen et al. (2018) propose to regularize network behaviors
exact pseudo replay. Specifically, in FSR, there are two variants: KSR-KLD for logits matching via KullbackLeibler Divergence loss and FSR-MSE for logits distillation via L2 distance loss.
Performance is reported in terms of both mean and standard deviation after 10 runs per protocol. Since generative model- approaches yaa de Ven & Tolias (2018); [Shin al. (2017) greatly alter architecture of the classification networks, we do not compare with them.
3.3 MEMORY COMPARISON
For fair comparison, we use the same feed-forward architecture for all the methods and allocate comparable amount of memory as EWCKKirpatriick et al. (2017) and other parameter regularization methods, for storing example images per class and their prototypes. In EWC, the model often allocates a memory size twice as the number of network parameters for computing Fisher information matrix which can be used for regularizing changes of network parameters Kirkpatrick et al. (2017). In more challenging classification tasks, the network size tends to be larger and hence, these methods require much more memory. In Table 1 show example of memory allocation

---

Under review as a conference paper at ICLR 2020
300
Average Task Task Classificattin Accuracy (%
100
First Taa Clas Classification Accuracy (%G
93
92
83
82
85
OUTS
75
EWConline
EWConline
EWC
72
EWC
12
MAS
51
65
ICARL
FSR
14
16
18
20
10
12
14
16
18
Task Number
Task Number
(a) First task accuracy on Permuted MNIST (ID)
(b) Average accuracy on Permuted MNIST (ID)
Average Task Clussification Accuracy (%)
102
Average Task Task Classification Accuracy (%)
60-
ours
ours
EWConline
EWConline
EWC
EWC
L2
L2
SGD
SGD
FSR
20
FSR
52
30
15
Task Number
Task Number
(c) Average accuracy on Split CIFARIO (IC)
(d) Average accuracy on Split MinilmageNett (IC)
Figure 3: First and average task classification accuracies from 10 random runs on permuted MNIST,
split CIFAR10 and split minilmagetNet in incremental domain (ID) and incremental class (IC) tasks.
on split CIFARIO in incremental class tasks with full memory and little memory respectively. The foed-forward classification network contains around 16.3 x 10 parameters. Weight regularization that, which takes about 32.63 x 10 parameters. The input RGB images are of size 3 x 32 x 32. Via Equ. D, our method can allocate episodic storing 16.3 x 105/(3 x 32 32) = 530 example images per class. In experiments with little training data as described Section 3.1 we reduce m 10 example images per class.
methods require memory allocation twice
memory with full capacity C 16.3 10a and calculate m which is equivalent
4 EXPERIMENTAL RESULTS
4.1 ALLEVIATING FORGETTING
Figure reports the results of continual learning methods with full memory under the two task protocols. All compared continual learning methods outperform SGD (cyan) which baseline without preventing catastrophic forgetting. Our method (red) achieves the highest average classification accuracy among all the compared methods, including both parameter regularization methods and memory- based methods, with minimum forgetting.
A good continual learning method should not only show good memory retention but also be able to adapt to new tasks. In Figure 3a although our method (red) performs par with EWC (brown) and FSR (date) in retaining the classification accuracy on dataset D2 in the first task along with 20 sequential tasks, the average classification accuracy of method is far higher than EWC (brown) and FSR (date) as shown in Figure 36 indicating both of these methods are able to retain good memory but fail to learn new tasks. After the 13th task, the average classification performance of

---

Under review as a conference paper at ICLR 2020
Average Task Classification Accuracy (%6
100
Average Task T Classification Accuracy (%)
60-
FursFew
oursFewShot
EWConline
EWConline
50-
20-
MAS
MAS
FSR-KLD
FSR-KLD
ICARL
ICARL
FSR-MSE
FSR-MSE
60
SGD
30
SGD
20-
10-
10
Task Number
Task Number
(a) Split CIFAR10
(b) Split minilmageNet
Figure 4: Few-shot average task classification accuracies on split CIFARIO 10 and minilmagetNet in incremental class task with few training data in the second tasks and onwards.
EWC is even worse than SGD. Across total 20 tasks, method leads FSR (date) by 3% more accurate on average. Similar reasoning can be applied to comparison with SI (green): although our method performs comparably well as SI in terms of average classification accuracy, SI fails to retain the classification accuracy on D1, which is 6% lower than ours in the 20th task.
Figure 3c and 3d show the average task classification accuracy over sequential tasks in incremental class protocol. Incremental class protocol is more challenging than incremental domain protocol, since the models have to infer both the task identity and class labels in the task. Our method (red) performs slightly better than iCARL (date) and has the hightest average classification accuracy in continual learning. Compared with third best method, FSR (green), our method yields constantly around 5% higher average across all tasks CIFARIO and minilmageNNt respectively. Note that most weight regularization methods, such as EWC (brown), perform as badly as SGD. It is possible that EWC computes Fisher matrix to maintain local information and does not consider the scenarios when data distributions across tasks are too far apart. On the contrary, our method maintains remarkably better performance than EWC, because ours focuses primarily the behaviors of network outputs, which indirectly relaxes the constraint about the change of network parameters.
4.2 FEW-SHOT CONTINUAL LEARNING
We evaluate continual learning methods with little memory under two task protocols with few training data in the second tasks and onwards except for the first tasks. Figure + reports their performance. Our method (red) has the highest average classification accuracy over all sequential tasks among state-of-the-art methods with 27% and 1 1% vs. 19% and 4% of FSR-KLD (yellow), which the second best, at the 9th and 10th tasks on CIFARIO and minilmageNet respectively. Weight regularization methods, such as EWConline (blue) and MAS (brown), perform as badly as SGD (cyan), worse than logits matching methods, such as FSR (green and yellow) or iCARL (purple). Similar observations have been made as Figure with full training data.
Compared with logits matching methods, our method has the highest average task classification accuracy. It reveals that our method performs classification via metric learning in an effective few-shot manner. It is also because our network architecture not dependent on the number of output classes and the knowledge in previous tasks can be well preserved and transferred to new tasks. superior to traditional networks with new parameters added in the last classification layer, which easily leads to overfitting. As side benefit, given the same number of example inputs in the episodic memory, our method is more efficient in memory usage since it stores one prototype per class instead of the logits for each example input as verified in Table T
4.3 NETWORK ANALYSIS
We also study the effects of the following three factors upon performance improvement. Figure 5 reports the average classification accuracy of these ablated methods. (1) Intuitively, limited memory capacity restricts number of example inputs to re-play and leads to performance drop. On permuted MNIST in incremental domain, with full memory capacity reduced by 2.5 times (from 5,000
8

---

Under review as a conference paper at ICLR 2020
ourst alliumn
purs iillet am
eplay
UniformMen
Full Irall Memory Magnitudes 10
EWC
online
MAS
L22
our
32.03
3263
52.33
52.33
3263
32,60
Litle Training und Linle Memory Sioe in Magnitudes of 10"
MAS
EWC
FSR-KLD
iCARL
FSR-MSE
32.63
32.63
19:40
19:40
19:40
19.39
16
Figure 5: Average task classification accuracies for the ablated models on permuted MNIST in incremental domain task.
Table I: Memory allocation for continual learning methods on CIFARIO in incremental class task.
Task
=3
Task
plane
29-
car
10
13
28
cat
deer
dog
frog
horse
15
ship
truck
-42 -20
30
-00
-90
30
100
Figure 6: 2D visualization of embedding clusters learnt by our method using t-sne Van Der Maaten (2014) on split CIFARIO in incremental class task. The first task (Task 1) a binary classification problem. In subsequent tasks, one new class is added.
example inputs to 2,000), our method shows a moderate decrease of average classification accuracy by 1% in the 20th task. (2) We also compare method with memory replay optimized by cross-entropy loss at full memory conditions. A performance drop around 1.5% is observed which validates classifying example inputs based on initial prototypes results in better performance in memory retention. (3) Given fixed C, our method adopts the strategy of decreasing m numbers of example inputs memory, with the increasing number of tasks. The performance drop of 1.5% using uniform memory allocation demonstrates the usefulness of dynamic memory allocation which enforces more examples to be replayed in earlier tasks, and therefore promotes memory retention.
In Figure 6 we provide visualizations of class embeddings by projecting these latent representations of classes into 2D space. can be seen that our method is capable of clustering latent representations belonging to the same class and meanwhile accommodating new class embeddings across sequential tasks. Interestingly, the clusters are topologically organized based on feature similarities among classes and the topological structure from the same classes is preserved across tasks. For example, the cluster of "bird" (black) is close to that of "plane" (orange) in Task and the same two clusters still close in Task 9. This again validates that classifying example inputs from previous tasks based on initial prototypes promotes preservation of topological structure in the initial metric space.
5 CONCLUSION
We address the problem of catastrophic forgetting by proposing prototype recalls in classification tasks. In addition to significantly alleviating catastrophic forgetting benchmark datasets, our method is superior to others in terms of making the memory usage efficient, and being generalizable learning novel concepts given only a few training examples in new tasks.
However, given a finite memory capacity and a high number of tasks, we recognize that our method, just like other memory-based continual learning algorithms, have limitations in number of prototypes stored. The memory requirement of our method increases linearly with the number of continuous tasks. In practice, there is always a trade-off between memory usage and retention. We believe that our method is one of the most efficient continual learning methods in eliminating catastrophic forgetting with decent amount of memory usage. Moreover, we restrict ourselves in classification tasks with discrete prototypes. In the future work, to apply algorithm in more complex and challenging problems, such as regression and reinforcement learning (RL), one possible solution is to quantize the continuous space in regression formulate RL in discrete state-action pairs.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 159-154, 2018.
Craig Atkinson, Brendan McCane, Lech Szymanski, and Anthony Robins. Pseudo-recursal: Solving the catastrophic forgetting problem in deep neural networks. arXiv preprint arXiv: 1702::1802.0385, 2018.
Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In Advances in neural information processing systems, pp. 2654-2662, 2014.
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT pp. 177-186. Springer, 2010.
Pratik Prabhanjan Brahma and Adrienne Othon. Subset replay based continual learning for scalable improvement of autonomous systems. In 2018 IEEE/CVF Conference on Computer Vision and Patlern Recognition Workshops (CVPRW), pp. 1179-11798, IEEE, 2018.
Cristian Bucilua, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 535-541. ACM, 2006.
Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. arXiv preprint arXiv:1203.3422 2012.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 141-)144 2012.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135, 1999.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp- 4367-4375, 2018
Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick.
Neuroscience-inspiin artificial intelligence. Neuron, 95(2):245-25) 258. 2017.
Xu He and Herbert Jaeger. Overcoming catastrophic interference using conceptor-aided backpropagation. 2018.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.022 2015.
Flad Hoffer and Nir Ailon. Deep metric learning using triplet network. In International Workshop on Similarity- Based Pattern Recognition, pp. 94.92. Springer, 2015.
Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning.
arXiv preprint arXiv: /7/ 11..//111111....1111 2017.
Ronald Kemker, Mare McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Thirry-econd AAAI conference on artificial intelligence, 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Bariikka. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 11((13::521-3526 2017.
10

---

Under review as a conference paper at ICLR 2020
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions.
In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp- 1885-1894. JMLR. org, 2017.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang, Overcoming catastrophic forgetting by incremental moment matching. In Advances in neural information processing systems, pp. 4652-4662, 2017.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, (0(12e229552)))22099000 2018.
David Lopez-Paz et al. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
Davide Maltoni and Vincenzo Lomonaco. Continuous learning single-inerementll-askk scenarios. Neural Networks, 2019.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard Turner. Variational continual learning. arXiv preprint 1rXiv:1710.1062 10628, 2017.
Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
Sylvestre-A Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorchearsal. Connection Science, 7(2): 123-146, 1995.
Jonathan Schwarz, Jelena Luketina, Wojeiech Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. arXiv preprint arXiv: 1805 06370, 2018
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-29999 2017.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pp. 4077 40,7,
Sebastian Thrun and Tom M Mitchell. Lifelong robot learning. Robotics and autonomous systems, 15(1-2):25-46, 1995.
Gido M van de Ven and Andreas Tolias. Generative replay with feedback connections as general strategy for continual learning. arXiv preprint arXiv: /809. /0635, 2018.
Laurens Van Der Maaten. Accelerating t-sne using tree-based algorithms. The Journal of Machine Learning Research, 15(1):3221-3245) 2014.
Junfeng Wen, Yanshuai Cao, and Ruitong Huang. Few-shot self reminder to overcome catastrophic forgetting. arXiv preprint arXiv: 1112.0054.. 2018.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference Machine Learming- Volume 70, pp- 3987-3995. JMLR. org, 2017.
11


Reviews:
[{"review_id": "ryxg6XY2FS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The proposed method addresses continual learning, by learning a mapping from the input space to an embedding space, and employing a loss that encourages clustering the embeddings by class (and task?) around some centroids called prototypes. Catastrophic forgetting is mitigated by adding a penalty term that is proportional to the distance of the embeddings under the current network of some samples from the past tasks, and the centroids previously associated to each of them.\n\nThe idea might have some merit, but the paper in its current state fails to convey it clearly and is too vague on the experimental settings for the results to be considered in any way. The notation is at times unnecessary heavy (see specific comments below), and many details on the method and experimental setup are missing to the point that it’s difficult to judge it and it would be impossible to reproduce it. In addition, it’s unclear whether the baselines have been properly tuned since the reported performance is incompatible with that of the original paper.\n\nThe submitted paper is clearly not ready for publication. I can consider modifying my vote if the paper undergoes a major review, fixing the notation, explaining the method and the experimental settings extensively and clearly, with proper comparison in terms of capacity/performance tradeoff w.r.t. the state of the art.\n\n\nDetailed feedback:\n\n1) Notation\n- I believe the exposition would be more fluent and easier to follow if you described the method under the assumption that all tasks have the same number of classes and a shared dataset, and mentioning that the method can be easily extended to work in the more general setting. Under this assumption, e.g., c_{kT} would simplify to c_{k}.\n\n- When you refer to a generic task, please use a lower case letter (e.g., t) instead of T (e.g., first paragraph of page 3, Eqn. 1, Eqn. 2, ..). Use capital letters for constants only (e.g., use T for the total number of tasks). When you need to refer to two tasks that are in a particular relative ordering. For instance, Eqn. 1 should be the argmin of “\\sum_{s=1}^{t} [...], with s < t”. Similarly, Sec 2.2 should refer to tasks s and t, rather than t and T (unless T is specifically referring to the last task alone, which doesn’t seem the case here as from what I gather the same should hold for all tasks, i.e., for a specific task t and any previous task s, with s < t.).\n\n- If I am not mistaken, L_T is defined just before Eqn. 1 and only used there. It is not very informative and it’s based on some undefined loss L. I would suggest to drop it entirely, and simplify Eqn. 1 accordingly.\n\n- Please avoid subscript with commas as much as possible, as they are difficult to parse. E.g., L_{classi, Dt} can become L_{classi_t} (or even L_{c_t} or L_{c}^{(t)}), remaining equally informative. All other “D_t” subscripts can be similarly replaced by “t” throughout the text. Generally speaking, I would recommend the ^{(t)} notation to denote task t (which you already adopted when defining \\tilde{D}_t\n\n- I believe Eqn. 1 can be simplified by merging the two summations into a single expression. Also, I think it would be clearer to define the argmin to be over “f \\in \\mathcal{F}”, since it is a search over the space of all possible functions. Finally, I also find a bit confusing the use of f and f_t to denote an evolving (currently evaluated) function and a fixed (best function from previous search) one respectively, but I am not sure I have a suggestion to improve this notation.\n\n- Please avoid unnecessary repetitions. In a number of cases a big part of a formula is defined inline in the text just before the equation. This doesn’t add to the conversation and makes it unnecessarily hard to parse the text (e.g., inline formulas just before Eqns. 4 and 6. Also prefer a textual description instead of the variable when introducing a definition (e.g., before Eqn. 7, “we define the distance as” or “we define the distance between the functions f and f_t embeddings as”)\n\n- Please fix all citations according to the ICLR style guidelines: When the authors or the publication are included in the sentence, the citation should not be in parenthesis (as in ``See \\citet{Hinton06} for more information.''). Otherwise, the citation should be in parenthesis (as in ``Deep learning shows promise to make progress towards AI~\\citep{Bengio+chapter2007}.''). For instance, all citations in Sec 2.2 should use \\citep.\n\n2) To my best understanding classification in the embedding space is performed with some variation of k-means, with k being the number of “prototypes” for the task at hand (i.e., the number of classes for the specific task). If this is correct, referring to k-means in the paper and highlighting the similarities and differences to it, would have been a more effective way to describe your method.\n\n3) The authors claim not to use a final classification, as opposed to competing methods, hence not being subject to memory usage increases when the number of classes increase (sec 2.1). Surprisingly though, they then employ a softmax over the prototypes (Eqn. 3) which to the best of my knowledge would require an incoming layer whose dimensionality would need to grow with the increase of the number of prototypes, (i.e., classes). Please update the text to either correct the claim or explain exactly how your method can drive the softmax in the case of an increasing number of classes without increasing the dimensionality of the layer before the softmax.\n\n4) Similarly, the authors claim that “Unlike parameter regularization methods or iCARL or FSR, our approach further reduces the memory storage by replacing logits of each data or network parameters with one prototype of each class in the episodic memory”. This is not entirely true, as they also need to store a (unspecified) number of samples from all tasks to prevent forgetting. It is unclear that replacing a snapshot of the network parameters is better (in terms of memory consumption) than storing some data from each task, and a proper discussion on this crucial point of the paper is missing.\n\n\n5) Apart from the losses, all the details on the architecture (type and number of layers, activation functions, ..) and on the hyperparameters are missing. The paper only mentions that the same feedforward architecture as EWC is used, with similar memory usage, but also mentions that unlike all other methods (including EWC I assume), theirs doesn’t need to vary the architecture if the number of tasks is increased. It is then completely unclear to me how this is possible if the architecture is the same. If it’s not, which parts of the network are shared and which are task specific. I assume most of the network to be shared among tasks and only the last layer (that drives the softmax that determines the probabilities of each class (prototype) for the current task) to be task-specific. This details are crucial to the understanding of the method and to be able to reproduce the results.\n\n6) Crucially, the reported Permuted MNIST performance on EWC is much worse than that on the original paper. This suggests that the baselines have not been properly tuned or there are issues in the implementation.\n\n7) Unless I am mistaken, in Sec 3.3 there seems to be a flaw in the calculation of the number of examples that can be stored by the proposed method in order to use approximately the same memory as the baselines. Indeed, the memory allocation for weight regularization is independent of the number of classes and so should be the equivalent capacity allocated for samples retention, i.e., in order to be comparable with EWC (and similar methods), the proposed model should be allowed to store 530 examples in total, rather than per class.\n\n8) I suggest to drop the definitions of the obvious functions, such as the softmax and the NLL (that is even defined twice!), and use that space to improve the description of the model and algorithm instead.\n\n9) Can you clarify what you mean by “The pairwise distance of one embedding and one prototype within the same class should be smaller than the intra-class ones.”? Should it be inter-class?\n\n10) In my opinion the narrative would be clearer backwards: rather than first defining the loss as a sum of three sub-losses, I’d suggest to define the sub-losses first and finally aggregate them. In its current state, Eqn. 1 is not informative until \\delta_{D_t} is defined, a full page later.\n\n11) Eqns. 3 and 5 are not “distance distributions” but rather “class probabilities distributions”, if I understood correctly. Can you confirm and amend the text?\n\n12) Eqn. 5 is more confusing than informative, I recommend to remove it and replace it with a better textual description. There is no need to redefine c_{kt}, as it is already defined in Eqn. 2 exactly in the same way. Secondly, there is no need to define the softmax over the distances a second time, as it’s already defined in Eqn. 3 and only the arguments change. \n\n13) Re Sec 2.3, to my best understanding there is no “emphasis on reviewing earlier tasks”, as the number of rehearsal samples is the same for all tasks. In other words, \\tilde{D}_t is sampled uniformly across tasks I believe. Is this right?\n\n14) Sec 3.1, “typical continual learning schemes assume that a large amount of training data over all tasks is always available for fine-tuning”. This is not true, and it’s actually usually not the case  (see e.g., generative replay approaches, or those that rely on pruning or other forms of architecture modification.)\n\n15) The introduction fails to mention the category of approaches that try to combat forgetting by being smart about the architecture, e.g., Progressive nets, Progress & compress, etc, .. Please amend this.\n\n\n\nMinor:\n\n- Add more space after Figure 1’s caption, so that it’s clear where the caption ends and the text begins.\n- Eqn. 2, no need to define yiT since it’s not used in the equation.\n- The sentence at the end of Sec 2.1 (“In practice … estimating the distance distribution” ..) should go after Eqn. 2. Similarly, the sentence “Our primary choice of  the distance function” would be better placed right after the definition of the distance function IMO.\n- The MNIST citation is wrong. Please correct it: LeCun, Yann and Cortes, Corinna. \"MNIST handwritten digit database.\" (1998)\n- The introduction is a bit long and a “related work” section is missing. You might want to split the introduction into two chapters.\n- Do not use D to denote the datasets and the dimensionality of the datasets.\n- The sentence after Eqn. 1 “learning f_T requires minimizing both terms ..” is incorrect, as it goes against Eqn. 1 in which there are 3 terms rather than 2.\n- End of page 3: “without loss of generality”...w.r.t. what? The phrase doesn’t seem to fit the context, consider changing it.\n- End of page 7, the detailed description of Figure 3 would be better placed in the caption of the figure itself.\n\nTypos: \n- Page 2: state-of-the-arts -> state-of-the-art\n- “There have been some attempts [citations] selecting”, “of” is missing\n- End of page 4: which is pre-computed -> are\n- End of page 5: public -> publicly", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}, {"review_id": "r1ebFgw3FB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper proposes a method for continual learning. The method is based on the learning of a metric space where classes are represented by prototypes in this space. To prevent forgetting the method proposes to perform prototype recall, aiming to keep prototypes in the same location in embedding space (Fig 1b). The method is compared with several recent methods and is shown to outperform them on two small datasets (MNIST permuted and CIFAR10). The idea of using prototypes for continual learning is interesting, as the authors point out, this does not require adding new neurons to the network for new tasks.\n\nI found the paper hard to follow (presentation needs improvement) and the experimental results are lacking. I, therefore, recommend weak reject at the moment. \n\nClarity method:\nI did an effort to understand the details of the paper but found the chosen notation hard to follow. Some remarks with respect to that. \n\n1. The delta loss is based on previous models (last term Eq. 1), which you do not want to store. It is important to immediately point out that you are not storing them (if I understand correctly) and that you are actually only storing the prototypes (Eq. 5). \n2. Could the loss in Eq 6 be seen as a distillation loss for an embedding space. You want to stay close to previous network ?\n3. It was unclear for me how often prototypes where updated, in every minibatch ? \n4.The algorithm is not very clear (what is a training episode?). Sampled m examples per dataset (should be per class per dataset). From the algorithm it seems that the loss in Eq 4 (first line after else) is first minimized separately from the loss in Eq. 7 . Should this not been done jointly ? \n5.What is the contribution of dynamic episodic memory allocation ? I am not sure if the Equation is very helpful. Is in practice M<<D ?\n6. Exemplars can only be chosen ones (when considering the task). I would like the authors to confirm  that they do that, because I was not sure from the text (they can be deleted later but not replaced with other exemplars from the same task). \n\nExperimental results:\n1. The authors should provide more experimental details. What network is used? Is it a pretrained network. \n2. Colors in Fig 3 preferably stay the same for same methods. The legend should only show results which are present in graph. (Fig 3c has 5 curves and 10 methods in legend). Why are there 21 tasks in Fig 3a and b, should it not be 20 ? Why are there 9 tasks in 3c,  should it not be 10 ? SGD results missing in Fig 3a and b, but mentioned in text. What happens with EWC, it has excellent results in Fig 3a but does bad in 3b, is there an explanation ?\n3. Ablation study should include ablation of the proposed losses, do they actually help in improving the results. The ablation is on memory usage which is not the main contribution of the paper. It would  be better to use CIFAR 10 then MNIST premuted for ablation (see for example ‘A COMPREHENSIVE, APPLICATION-ORIENTED STUDY OF CATASTROPHIC FORGETTING IN DNNS’ on the limitations of MNIST permuted). \n4. Figure 6 did not help me understnad the method. It would be nice if you could plot the prototypes, than we can see if they keep in the same location. \n\nMinor:\nI am not sure the authors are aware of difference \\cite and \\citep which would improve readability. \n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "BkeEGgTkqr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper summary:\nThis paper proposes a method for continual learning/few-shot learning which involves (1) putting class prototypes in a replay buffer, at test time computing the distance to these prototypes in order to classify the example (2) regularizing the embedding space used to compute prototypes by ensuring classification accuracy on a replay buffer of data examples does not degrade. \n\nPaper contributions:\n - The idea of putting prototypes in a replay buffer, and of the regularization for ensuring classification accuracy on earlier examples does not degrade (I'm not up-to-date enough in this area to know if these ideas are novel)\n - Experiments in continual learning and few shot learning which compare the proposed method to alternatives\n - Ablation study with different settings of the proposed method - exploration of the embedding clusters learned by the method\n\nReview summary & decision: \nThis paper introduces two ideas that I think seem quite good and could be valuable to the community. However, I found the presentation of material very confusing and poorly organized, some claims misleading, and some experiments missing. I recommend rejection of the paper in its current state.\n\nReasons for decision:\n1. The intro, \"proposed method\" section, and \"prototype recall\" section are repetitive of each other (especially proposed method and prototype recall). The latter two sections also have some information that should go in the related work (aka introduction). I actually like the merging of related work and introduction though; I think it could be nicely space-efficient if things weren't so repetitive later.\n2. Overall I found the paper is very disorganized and hard to follow; the description of what is done (in particular the fact that you evaluate on three different tasks) is very unclear. It's mentioned in the last paragraph of the intro that you do two experimental protocols, but these are not referred to (at least not by the same name) in all of the experiments. Also there are experiments on few shot learning which are only ambiguously referred to in the abstract. The relationship of \"classification\" and \"eliminating catastrophic forgetting\" to these tasks is not discussed, even though there is a whole section called \"classification\" which seems like a misleading title for this section - it seems more like this is further description of your proposed method, along with some related work. The terms \"single head\" and \"multi head\" evaluation are in common use in continual learning, but are not referred to here at all. The pseudocode algorithm with references to equations makes it reasonably clear what is going on, but it would be helpful if it appeared much earlier and the discussion followed this outline (e.g. have this be the first thing in the section, and then have sections called \"Prototype computation\", \"Episodic memory\" and \"Prototype recall\"), and if it were made more clear where the actual task comes in. Also there's a lot of related work in the Experiments section. It seems like the few-shot experiments are kind of tacked on as an afterthought; the intro doesn't mention these and there are methods (e.g. logit matching) used here that are not mentioned in the intro/related work.\n3. What is the motivation for averaging embeddings? As far as I can tell this is mentioned in the caption of Figure 1; it should be discussed elsewhere.\n4. Part of the motivation for your approach as given in the abstract and intro is not to have to save examples, but your method does require this for computing the regularization term. I would like to see an ablation study of the proposed method with varying amounts of data examples, including 0 (i.e. without this regularization); as far as I understand this is only done for 10 and 530 examples. It seems odd to refer to this as \"memory\"; this only makes sense if we know you're referring to episodic memory, which is not clear (or even mentioned) in the caption for Figure 5. Based on the fact that the method performs poorly with only 10 examples as the number of tasks goes up, I would say it's misleading to claim you use only a small number of samples, as well as to claim that you only need to store class-representative prototypes (you also need to store these examples).\n\nFeedback/suggestions/nits (not necessarily part of decision assessment):\n - \"continual learning is a critical ability\" this kind of subjective assessment doesn't add to our understanding and is generally not recommended in scientific writing. This first sentence does not do a good job of explaining what is continual learning for someone who does not know.\n - \"enabling CL\" strange word to use here.\n  - Inconsistent capitalization of prototype recalls\n - In the abstract, the 4 contributions seem repetitive (1 and 3 seem the same to me) and the 4th contribution makes it sound like an avenue for future work than an actual contribution.\n - Intro is repetitive of the abstract\n - citations are frequently incorrectly formatted (should use \\citep, not \\citet unless you are referring to the authors by name)\n - unclear what \"stationary batches\" means; do you mean in continual learning the data distribution is non-stationary?\n - verb tense: \"have been currently proposed\" does not make sense\n - I don't understand the criticism of \"Learning without forgetting\" (that it requires exhaustive hyperparameter tuning); your method also has hyperparameters to tune. Do they have more, and is their method more sensitive to hyperparameters? What's the evidence for this?\n - If your criticism of replay-based methods is that they ignore topology among clusters in embedding space, you should explain why/how your method does not do this. Likewise for relying on \"small amounts of individual data\"; if seems to me that your method has the same structure, and for both your and these methods the size of the replay buffer is just a hyperparameter, unless I've misunderstood something.\n - \"our method learns embedding functions... has also been verfied in works\" what exactly has been verified? What do those works do? This sentence is unclear.\n - Hoffer & Ailon is incorrectly cited as Ailon (2015); in general the references are poorly formatted\n - Snell et al is barely mentioned in the related work, but the two methods seem very similar to me; I think it merits more discussion (in particular, highlighting how what you do is different / an extension). The \"classification\" section seems to imply you compute prototypes exactly as in Snell et al., but elsewhere it sounds like you do something different. \n -  Figure 1 is not clear; one must have already read the full explanation of the two settings (which is spread over the intro and other sections) to understand it. Just mentioning that these two are different evaluation settings would be helpful. Also,  I find the relationship between the tasks unclear.\n  - \"yielded outstanding performance\" what is outstanding in this context?\n - not sure it's accurate to call \"the metric space at task t can be underlined by class-representative prototypes\" an \"inductive bias\". Also what does \"underline\" mean here?\n - \"without loss of generality\" I don't think this phrase is used appropriately here\n - Algorithm 1's for loop is quite unclear to me, in particular it's unclear where you actually do the task you're supposed to be doing.\n - EWC and yours are the same colour in Figure 3\n - Figure captions should mention what the abbreviations mean (or mention where in the text these explanations can be found).\n - What are we supposed to tak away from the TSNE plots? This explanation should be in the caption, and the tsne hyperparameters should be in appendix.\n - \"we restrict ourselves to... discrete prototypes\" not obvious what discrete means here; you're using a continuous embedding space. I assume you mean discrete output tasks (i.e. classification), but this is unclear.\n - It would be nice to see results about computation time; it seems like computing the prototypes might be very intensive.\n\nQuestions:\n1. Can you contrast your method with prototypical networks? Are prototypes computed in the same way, and then put in a replay buffer along with examples? \n2. Why averaging the embeddings? Did you try anything else?\n3. How do your task protocols correspond to \"single head\" and \"multi head\" evaluation protocols? Why weren't these mentioned? Why did you choose these protocols?\n4. The approach of using the same architecture and same memory allowance for all the methods tested seems reasonable to me, but can you think of alternatives, or reasons that this could be unfair to any of the methods?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers have provided extensive comments, we encourage the authors to take them into account seriously in further iterations of this work.
Decision: Reject