Looking Back on Learned Experiences  For Class/task Incremental Learning
Authors: Mozhgan PourKeshavarzi
Guoying Zhao
Mohammad Sabokrou
Venue: ICLR 2022 Spotlight (2022)
Keywords: Deepl Learning
Class Incremental learning
Continual learning
Experiences

Abstract: Classical deep neural networks are limited in their ability to learn from emerging streams of training data. When trained sequentially on new or evolving tasks, their performance degrades sharply, making them inappropriate in real-world use cases. Existing methods tackle it by either storing old data samples or only updating a parameter set of deep neural networks, which, however, demands a large memory budget or spoils the flexibility of models to learn the incremented task distribution. In this paper, we shed light on an on-call transfer set to provide past experiences whenever a new task arises in the data stream. In particular, we propose a Cost-Free Incremental Learning (CF-IL) not only to replay past experiences the model has learned but also to perform this in a cost free manner. Towards this end, we introduced a memory recovery paradigm in which we query the network to synthesize past exemplars whenever a new task emerges. Thus, our method needs no extra memory for data buffering or network growing, besides calls the proposed memory recovery paradigm to provide past exemplars, named a transfer set in order to mitigate catastrophically forgetting the former tasks in the Incremental Learning (IL) setup. Moreover, in contrast with recently proposed methods, the suggested paradigm does not desire a parallel architecture since it only relies on the learner network. Compared to the state-of-the-art data techniques without buffering past data samples, CF-IL demonstrates significantly better performance on the well-known datasets whether a task oracle is available in test time (Task-IL) or not (Class-IL).

Published as a conference paper at ICLR 2022
LOOKING BACK ON LEARNED EXPERIENCES FOR CLASS/TASK INCREMENTAL LEARNING
Mozhgan PourKeshavarz! Guoying Zhao
Mohammad Sabokrou'
School of Computer Science, Institute for Research in Fundamental Sciences (IPM)
"Center for Machine Vision and Signal Analysis, University of Oulu, Finland
[pourkeshavarz, Sabokroj@ipm ir guoying. zhao@oulu.
ABSTRACT
Classical deep neural networks are limited in their ability to learn from emerging streams of training data. When trained sequentially on new or evolving tasks, their performance degrades sharply, making them inappropriate in real-world use cases. Existing methods tackle it by either storing old data samples or only updating a parameter set of deep neural networks, which, however, demands large memory budget or spoils the flexibility of models to learn the incremented task distribution. this paper, we shed light on an on-call transfer set to provide past experiences whenever a new task arises in the data stream. In particular, we propose a Cost- Free Incremental Learning (CF-IL) not only to replay past experiences the model has learned but also to perform this in a cost free manner. Towards this end, we introduced a memory recovery paradigm in which we query the network to synthesize past exemplars whenever a new task emerges. Thus, our method needs no extra memory for data buffering or network growing, besides calls the proposed memory recovery paradigm to provide past exemplars, named transfer set in order to mitigate catastrophically forgetting the former tasks in the Incremental Learning (IL) setup. Moreover, contrast with recently proposed methods, the suggested paradigm does not desire a parallel architecture since it only relies on the learner network. Compared to the state-of-the-art data techniques without buffering past data samples, CF-II demonstrates significantly better performance on the not (Class-IL
datasets whether a task oracle is available in test time (Task-IL)
INTRODUCTION
Conventional Deep Neural Networks (DNNs) are typically trained in offline batch setting where all data are available at once. However, in real-world settings, models may incrementally encounter new classes when trained online. In such scenarios, deep learning models suffer from catastrophic forgetting McCloskey & Cohen (1989). meaning they forget the previously obtained knowledge when adapting to the new information from the incoming observations. This is mainly because, models overwrite the decisive parameters for earlier tasks while learning the new one.
Incremental Learning (IL) methods aim at training single DNN from an unlimited stream of data, mitigating catastrophic forgetting conditioned on the limited computational overhead and memory budget. In the IL literature, the typical setting model learn many tasks sequentially where each task contains one or more disjoint classification problems. The majority of the existing works assume that task identities are provided at the test time so that one can select the relevant part of the network for each example. This setup has been named task-IL, where a more general cireumstance is that task labels are available only during training, has been named class-IL.
Recently, various approaches have been put forward to solve catastrophic forgetting. Some works investigate on parameter isolation approaches using either expanding network architecture (Aljundi et al. 2017 Xu & Zhu 2018 Yoon et al.] 2017 Rosenfeld & Tsotsos 2018) or considering
fixed-sized set of model parameters (Masse et al. 2018), pruning (Mallya & Lazebnik 2018 to
'The code is available http:/ghhhheeoiiigggghiiiiiieeee

---

Published as a conference paper at ICLR 2022
learn new task. Thus, they are only applicable to the task incremental scenario. Besides, several studies suggest a regularization term added to the loss function (Kirkpatrick et al. 2017 Aljundi et al. 2018: Zenke et al.) 2017 Nguyen et al. 2017) to preserve the knowledge of previous classes without a need for knowing the task label during inference. Despite the increasing number of these methodologies, they can not reach reasonable performance. Another followed terminology is experience replay-based methods which store actual data samples from the past (Buzzega et al.) 2020 Riemer et al., 20188 Chaudhry et al. 2018) to alleviate catastrophic forgetting problem, where more of the recent state-of-the-art (SOTA) works fall into this category. Despite the success of this approach, they suffer from main limitation that is the fixed memory budget when increasing the number of classes, leading to severe performance degradation. To overcome this constraint, several works propose to generate past data samples (Zhai et al., 2019), as pseudo-data examples, conditioned on the model underlying distribution. Although their method are considered data-free approach by discarding the need of storing real data, they achieve better performance than previous data-free works. However, one main drawback that these approaches encounter is need for a high- capacity auxiliary network to generate data which increase the complexity of the system in terms of craiiin-ddiffiull and computation overhead.
In this paper, we propose a novel Cost-Free Incremental Learning (CF-IL) paradigm, which allows the model to remember previous knowledge by itself, thereby eliminating the need for external memory. Also, in the proposed method, there is no need for a parallel network to generate past exemplars. Besides, we ask the learner network to remember what it has learned before learning new classes. Inspired by (Fredrikson et al. 2015 He et al. 2019 Nayak et al.) 2019), we propose solution reply the previous experiences by recovering the learner network memory. The main idea is having only the learned network to not only remember what it has learned but also stay learned while encountering newly received knowledge, much like how humans continually learn and remember them by themselves with no external help. The most likely method to learn in this way is the regularization-base method. However, they still could not compromise between human- like learning and performance. Besides, our method can achieve better result by a significant margin with no additional cost.
In summary, the main contributions of this paper are: (1)To the best of our knowledge, we introduce, for the first time, the idea of CF-IL to make experience replay without a need to a parallel auxiliary network, or any other additional requirement, (2)Since we only rely on the learner network, there is no need for buffering past data samples, extracted prior knowledge, and a parallel high-capacity network, resulting in significantly reduced computational cost and memory footprint, and (3) method compromises between decreasing memory budget and maintaining performance comparing to the current SOTA task on both class and task incremental learning methods.
2 RELATED WORK
Several studies have been carried out to tackle catastrophic forgetting which falls into four categories as follows.
This approach is the first investigations into incremental learning in which new set of network parameters is dedicated for each task in dynamic architecture networks (Aljundi) cet al. 2017; Xu & Zhu 2018 Yoon al., 2017, Rosenfeld & Tsotsos 2018). Therefore, they require the task label to be known at test time to trigger the correct state of the network, which can not be employed in the class-IL setting where task labels are not provided. Moreover, maintaining class incremental performance can be met at the cost of limited scalability.
Model Growing:
Memory Replay: These methods allow access to a fixed memory buffer to hold actual samples (Rebuffi et al. 2017: Iselc & Cosgun| 2018 Chaudhry et al., 2019 Rolnick et al.) 2018). When learning new tasks, samples from the former tasks are either reused as model inputs for rehearsal or to constrain optimization (Lopez-Paz & Ranzato| 2017 Chaudhry et al. 2018) of the recent task loss to prevent preceding task interference. Although this approach is pioneering, its main limitation is a fixed budget, which caused the bias problem (Hou et al. 2019 Wu et al.) 2019) by the imbalanced number of old and new class training samples (Belouadah & Popescu| 2019). In the meantime, some works explore a aaeudo-memory replay (Robins) 1995; Atkinson al.) approach in which generative model is adopted to synthesize past samples when facing a new task resulting increasing training complexity.
2

---

Published as a conference paper at ICLR 2022
{1}
R(C)
201
D
L
0,003
1000
1903
Cost-Freee Incremental Learning (CF-IL)
Memory Recovery Paradigm R(C,
Figure 1: Overview of the proposed approach. When new incremented task data I emerge, the memory recovery paradigm R(C) is called to generate transfer set S, including synthesized images for the classes that the learner network L. has learned before. In the proposed recovery engine, we first generate preliminary model outputs (V) for each learned class and then form refined set (V}" by applying constraint F((V),)) using a dynamic recommender vector 7 adjust due to underlying distribution. Then, a random noise input is initialized to be optimized conditioned on the refined output [V}*. This procedure is performed for each learned class several times to form the transfer set S. Finally, the learnrr network is retrained on the combined dataset D, including transfer set and new incremented task data I using a two-part cost function composing classification (CE) and knowledge distillation (KD) terms, respectively.
Regularization-bused These works penalize the model for updating important parameters for ear- lier tasks by adding regularization term the loss function. The importance score of cach parame- ter can be estimated through the Fisher Information Matrix (Kirkpatrick et al. (017), gradient extent (Aljundi al. 2018), and uncertainty estimation using Bayesian Neural Network (Nguyen al., 2017). Although the vast amount of works falls into this family, soft penalties might not be sufficient to restrict the optimization process to stay in the feasible region of previous tasks, particularly with long sequences (Farquhar & Gal 2018), resulting in increased forgetting of earlier tasks.
Parameter Isolation: These methods rely estimating binary masks that allocate fixed number of parameters for each task where the network architecture remains static. Such masks can be estimated either by random assignment (Masse al. 2018), pruning (Mallya & Lazebnik 2018), or gradient descent (Mallya et al., 2018 Serra et al. 2018). However, this family is limited to the task incremental setting and better agreed for learning a long sequence of tasks when model capacity is not a concern, and optimal performance is the priority.
3 PROBLEM SETTING
Without loss of generality, a classical IL scenario can be assumed as set T = (Ti..... Tn) of N different tasks. Each task T; takes the form of classification problem, T, [x ) where Vy E (1,...,4). When new task arrives we ask the network learn to classify the new images, being simultaneously able to solve the former tasks. Our method only relies on single DNN network, named the learner network L, in all steps with no external memory and no parallel network. We formalize the setting conditioned on the absence of a task oracle at test time as follows: Task- IL and Class-II. In the Task-IL setting, the learner network L is composed of a feature extractor followed by multiple heads where each head is composed of linear classifier with softmax activation which computes task-specific classification probabilities. On the contrary, in the Class-IL setting, the learner network has single head while is shared over all the tasks that make the Class-IL more challenging. In this paper, we consider both settings and report the results regarding them.
4 PROPOSED METHOD
Availability of the actual data samples can enable solving for straightforward retraining, thus learn- ing incrementally overcome catastrophic forgetting problem. However, preserving earlier data
3

---

Published as a conference paper at ICLR 2022
is not straightforward in real-world scenarios due computational overhead and privacy concern.. In this paper, the main idea is to recall (instead of preserving) past exemplars, known as network experiences, without using either parallel network or memory buffer. To this end, we propose CF- II which, whenever a new task arises, we trigger memory recovery paradigm R(C) to obtain a transfer set S of past experiences as surrogates for the earlier tasks observations. Then, we form combined training dataset DD from the recovered transfer set S and the current incremented class(es) I data. Thus, the learner network L, which supposed learn the task sequences ' incrementally, is retrained on D in each time step. In the suggested Memory Recovery Paradigm (MRP), we first model the learner network output L regarding the case when real sample from the past is fed to the model. These outputs are provided by generating initial output vectors (V) and refining them by applying the F(V,7) constraint, as a supervision, to perform adjustment due to underlying dis- tribution obtained from dynamic recommender 7, resulting in reaching (V}*. Fig. shows clear overview of the proposed method. In the rest of this section, we first describe the proposed CF-IL and then outline the suggested MRP in detail.
4.1 COST-FREE INCREMENTAL LEARNING
In this work, the cost-free incremental learning term expresses that there is no need for a memory buffer learn incrementally, while the DNN model complexity fixed. Besides, we synthesize a transfer set S as past experiences of the network in a cost-free manner. To put this idea into practice,
run the following steps, which are listed in Alg. []] Whenever CF-IL receives a new incremented task data I = {2 yi) S i S 3, E [1. T. where L. is the total number of samples of I classes belonging to the new incremented task T initiates the memory recovery procedure R(C)
(Alg. (2 to create the transfer set S = {(ā,, ūū) , 1 S j S K, Új E [1..... k]). where K is the total number synthesized samples of l classes that have been learned by the learner network L. Next, the samples from the transfer set S is fed to the learner network, and the resulting network's logits O = (z). oK (I)] for all samples are stored. Then, we form a combined dataset D per each task 7,, [1, ...., NJ in the task sequence T. Finally, the learner network L parameters to minimize a cost function such a way that each data sample from the new incremented class will be classified correctly, classification loss, and for the samples in S, current network's 's logits will be reproduced as close as those have been stored in the previous step, as knowledge distillation loss by KD term
updated
We employ the softmax cross entropy as the classification loss, which is computed using Equ.
(CE(C(x;0o).u)) =
Σ
log (x))
(Z. i=1
where sy=s is the indicator function and (z( is the output probability (i.e, softmax of logits) of the 7th class in / new incremented class(es).
For the distillation purpose, we adopt knowledge distillation from network output (logits), known as dark knowledge Hinton et al. (2015); Wang & Yoon (2020); Buzzega et al. (2020), and our objective is to minimize the Euclidean distance between the stored logits in the transfer set O =
[or ox (x) and those generated by the learner network Ô = (x)......, (x)) as follows:
(xD(L(x;0c),0)
Σ Σ
(o) o (x)]l2
(a.))SS j=1
Having look at both cost functions, we define the total loss function by linear combination as below:
& = ECE + XLRD
where X is predefined parameter to control the degree of distillation.
4.2 MEMORY RECOVERY PARADIGM
The proposed MRP is devised to synthesize the previously learned experiences (knowledge) by L. In particular, we synthesis past samples in a way that the learner network strongly believes them to be actual samples that belong to categories in the underlying data distribution. Therefore, they
4

---

Published as a conference paper at ICLR 2022
Figure 2: Visualization of the sampled recovered data from the learner network using the proposed memory recovery paradigm, while training on CIFAR1O dataset.
the network acquaintance that might not be natural-looking data. is worth noting that we only use the learned parameters of the learner model L since they can be interpreted as the memory of the model in which the essence of training has been saved and encoded.
In the suggested paradigm, we first model the network output space using two-stepped approach and then generate synthesized samples by back-propagating the modeled output through the net- work, where each is described in the rest. The whole procedure is shown in Alg. 2
Modeling the Network Output Space the first phase, we model the output space of the learner network. Suppose ydd Liz, (T,,C) is the actual model output when jth class of 1th task observation was fed to the network. To reproduce such output in the absence of actual previous one, (V*)'3 ~ y²² we consider two-stepped approach as explained bellow.
In (Nayak et al. 2019), it is investigated that sampling the output vector from Dirichlet distri- bution in which ingredients are the range [0. 1] and whose sum is one is straightforward way for modelling the output space of the learner network. Despite interesting results, this method lacks preserving extra class similarity, hence generating outlier vectors that do not follow the networks underlying distribution. Thus, false memory happens when we query the memory for specific target. In this case, the result of memory recovery is a mixture of some targets that do not really exist in the past experience, resulting in confusing the learner in the retraining phase. In our novel memory recovery process, we make supervision as the second step, to detect such outliers to boost the retrieved memory in terms of generating single-target- based transfer set. To do this, we make supervision by applying a constraint on the generated vector F(V,7) 7), in which an arbitrary gen- erated vector from the previous step is good candidate only if has distance lower than predefined threshold to a dynamic recommender 7y This recommendation vector supposed to be declared at a low-cost and represent the class similarity as well. Considering these concerns, we im- ply a dynamic confusion matrix CM that is constructed incrementally when training each task. An arbitrary confusion matrix a table that typically used to describe the performance of a classifi- cation model. Intuitively, when a model wrongly classifies some samples to class it means class e is highly correlated with the target class. On the other side, when a model always correctly classifies class from the target class, it means class is highly distinguishable from the target class. This is the exact expected result from a DNN model output. Mathematically, we check if the generated vector V for arbitrary class i, has distance smaller than a predefined threshold 1) the 7h row of the constructed dynamic confusion matrix CM', as obtain the checked output vector V
Back propagate Through the Network In the second phase, conditioned on the network's gener- ated outputs (V*) ). we synthesize transfer set S. For an arbitrary generated vector V*, we start with a random noisy image x sampled from a uniform distribution in the range [0, 1] and update till the cross-entropy (CE) loss between the generated model output V* and the model output predicted by the learner network L (r, 0r, is minimized.
x = argmin&ce (z, (2, T),
(4)
where 7 is the temperature value used in the softmax layer (Hinton et al. (0055) for the distillation purpose. This procedure is renewed for each of the k classes has been learned K/k: times, where K is the whole number of samples to be formed the transfer set S.
The proposed paradigm is remedy for memory bottlenecks so that the model alone alleviate catastrophic forgetting, resulting in remarkable performance among the data-free works on both task-Il. and class-II., as will be seen in the next section.
5

---

Published as a conference paper at ICLR 2022
Algorithm 1: Cost-Free Incremental Learning
Algorithm 2: Mem. Recovery Paradigm
Input Task sequence set Transfer size
Input Learner model parameters Cc.
K, Learner model parameters 0r
Transfer set size K, Confusion
Require: Confusion Matrix CM from L.
Matrix CM
Incremented class(es) I
Output Transfer set S
// memory recovery paradigm
Require: Dirichlet distribution
S + K, CMM)
parameters a, B, Thresh. 7). Temperature for distillation T
number of samples i
// store network outputs and Ground Truth for S with pre-update parameters for (z, in S do
k number of classes from L // Nunber of samples
p K/k
O (()
per class
S S+0
// form combined training set
D (' E Ii} U 0 (( Oi) xt E St}
Constraint F(X,,) - (1 -212 < n)
for = 1 k do
? + CM*
1/ define pwottdd loss function
for = 1=::pmm do do
Total Σ IN (c + (x,,,)ED
Sample Vj +
Dir (d. x a²) V" Vj
while Constraint F(V) 7)
11 optimize the network for the total loss
passed
Initialize 2² to random noise
for (xr, y) in range(K L.) do
z) t optimize 2² with EC (V*) L (a), 0c,
sample a batch b, form D, where
(z, O) S
sample a batch b2 form DD where (x, v) E I update 0c by taking SGD step on bi + by
S S SUJ
loss Totaa (0c)
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We evaluate our method on two commonly-used datasets for incremental image classification tasks: CIFAR-10 (Krizhevsky et al. 2009) and Tiny-ImageNet (Le & Yang 2015). To form incremented tasks, we randomly choose 2, 20 classes for each task resulting in a sequence of 5, 20 sequential tasks for CIFAR-10 and Tiny- Tiny-Imaged ImtgeNet, respectively, which is typical configuration in literature. Following iCaRL (Rebuffi et al. 2017), a ResNet18 (He et al. (016) is used in the learner network without pre-training. To perform fair comparison with other IL methods, using the Stochastic Gradient Descent (SGD) optimizer with a learning rate of 0.1 with other param- eters set to their default values. We consider the number of epochs per task concerning the dataset complexity; thus, we set it to 50 for Sequential CIFAR-10 and 100 for Sequential Tiny-ImageNet, respectively. In the training phase, we select batch of data from the incremented task and mini- batch of data from the transfer set S, where depending on the hardware restriction, we set both to 32. We show all the classes in the same fixed order across different runs, and results are averaged across ten runs, each involving a different initialization. For visual simplicity, we only report mean values. Following the convention of the MI. community, hyperparams are selected by performing a grid-search validation set, obtained by sampling 10% of the training set. In the MRP, we con- sider the temperature value 7 to 20 for the distillation purpose. In the Dirichlet distribution, we set in 0.1] for each dataset, where half the transfer set S is synthesized by 1 and the rest with 0.1 as in (Nayak et al. 2019). The value in the constraint step is empirically valued at 0.7. To optimize the random noisy image. employ the Adam optimizer with learning rate of 0.01, while the maximum number of iterations is set to 1500. For all experiments, we consider the average accuracy across tasks as the evaluation criteria.
train the networks

---

Published as a conference paper at ICLR 2022
(8)
(6)
(4)
100
100
Average Acuraay
60
80
80
640
11 0.4
0.45
40
20
40
40
20
50
500
5000
6000
7000
2
8
Number classes
Number of classes
Figure 3: Experimental results on CIFAR-10 dataset. (a) classification accuracy curves for CF-IL with various transfer set size. Performance comparison of the CF-IL with various hyperparameters A (c) and 77 (d).
[0,015, 6A11, 8215, 8012, D.O5, 0012, 0856, w.k.,0.00 aniss, 2.100,
Figure 4: Visualization of happened False Memory when retrieving data from the memory recovery step (Right) and correct sample which passed the second step in our introduced network output modeling procedure (Left).
5.2 ABLATION STUDIES
Synthesized Transfer Set: We follow a two-step strategy to generate the network output in MRP. Firstly, we follow the approach adopted in (Nayak
2019) in which Dirichlet distribution Dir(d, (xx) is utilized to generate the vectors in sotmax space, where we need to set two parameters in this distribution: a, and 3 (d is the dimension of the expected vector). is the concentration parameter to control the probability mass of sample from the distribution. Decreasing its value caused a vector that hugely concentrated only few ingredients and vice versa. a interpreted as a class similarity vector obtained from the network's final layer and can be scaled with the B parameter. This method encounters false memory since some generated vectors are mix-up of several targets. To tackle this problem, we suggest applying constraint refine these vectors. An examples of false memory and a correct one for the target class dog are shown in Fig. 4 Moreover, present several samples of retrieved images belonging to several classes, when querying the learner network L using our novel MRP, as shown in Fig 2 We can see how the network retains its learned knowledge in its memory which is specific pattern representing the target classes.
1 and S: À controls how much the transfer set S should be attended as Eq. 3. and 7 is related how much the generated model output should be close to the recommendation vector. To select the best one for the two mentioned parameter, we perform grid search, where a brief of examination We also examine the impact of transfer set size on the performance of the incrementally learning classes. To this end, we set up the proposed CF-IL on CIFAR-10 10 dataset with different sizes of transfer sets, including [50, 500, 5000, 6000, 7000), wherein in the one before the latest case, the retrieved data from the past is equal to the incremented class. Fig. (a) shows the performance. Obviously, increasing the number of synthesized samples in the transfer set has significant impact on the performance. It is worth noting that an optimal size for the transfer set depends on the task complexity in terms of the number of classes and variations of the actual images. Thus, increasing the set size higher than reasonable one might increase the risk overfitting.
reported in Fig. 3 (c) and (d), where a take and 7) 0.3 and 0.45, respectively.
We conduct set of experiments MRP as a buffer booster which means it can be used as an extension in any buffer-basee works. In this case, with having fixed-sized buffer store actual data from the past, add the transfer set
CF-IL in mased works:
CIFAR-100 by adopting

---

Published as a conference paper at ICLR 2022
resulting in more balanced data when training in which no actual data is buffered, and a few samples (10 samples per class) are buffered from the past, specified by buffer size. The evaluation results reported in the table below, where N means the incremental phases equal the number of tasks. As shown in the table, while having the same II. method, our method is powerful extension while it has more consistent results when increasing the number of incremental phases means our MRP can synthesise past samples effectively regardless of how many classes have been learned by the learner network.
arrival task. We also consider two cases
Table Classification results (average accuracy) on CIFAR- --0 datasct for suffer-based methods and ablative models with adopting MRP as a buffer booster.
Method N
5
10
25
buffer size
iCaRL
57.12
52.66
48.22
2K
iCaRL + mnemonics
59.88
57.53
54.30
2K
LUCIR
63.17
60.14
57.54
2K
LUCIR + mnemonics
64.95
63.25
63.70
2K
iCaRL MRP
61.03
59.79
58.01
2K
LUCIR + MRP
65.32
65.21
65.80
2K
CF-IL
62.37
62.01
61.79
0
CF-IL
63.64
63.18
63.12
IK
5.3 COMPARED METHOSS
To have fair comparison, we first should determine that work falls into which categories mentioned in Sec 2 Secondly, discuss two main aspects that should be considered when it comes to comparing methods: Performance, complexity. Obviously,
method is neither model-growe growing PNN (Rusu et al.] 2016) nor sarameter-isolatiinn since our learner network has a fixed architecture while the whole parameters
share among all tasks. In terms of replaying past samples, CF-II
may be counted as a member of ceanoor-rreplay-aseee group MER (Riemer et al. 2018). iCaRL. (Rebuffi et al.) (017). DER (Buzzega et al. 2020). GEM (Lopez-Paz
Ranzato 2017 a-GEM
[Caaudhry et al. 2018), GSS (Aljundi et al. 2019). TPCIL (Tao et al. (020). Mnemonics (Liu
al.] 2020), BIC (Wu al. 2019) that more recent SOTAS fall into this category. However, keeping past examples in fixed-size buffer is not applicable when growing the number of tasks and has memory overhead resulting in increasing complexity of the system. Keeping these issues in mind, we compare our method with prominent works in this category while allocating the same buffer size and omit the buffer for our approach. Our work somehow can regarded as a data-free work, so it falls into the regularization-base ALASSO (Park et | (211)). UCB (Ebrahimi et al.) 2019). Although this category of works performs incremental learning without any equipment, they can't 't gain outstanding performance, while CF-IL achieves remarkable results. The more comparable group of works is pseudo-memory-ree memory-rehearsa DGM (Ostapenko et al., 2019), DGR (Shin et al., 2017), MeRGAN (Wu et al., 2018) which try to synthesis past knowledge. Thanks to the generative adversarial networks, this group of papers meet comparable result against memory-rehearsa works without buffering strategy. However, CF-IL has the same terminology while reducing the complexity by eliminating the need for an auxiliary network along with improving performance.
group oEWC (Schwarz et al., (2018). SI (Zenke al. (017)),
5.4 RESULTS
Table 2 shows the performance of CF-IL in comparison the other mentioned methods on the two commonly used considered datasets. Our proposed method CF-IL has achieved the SOTA perfor- mance in almost all settings. PNN (Rusu al. 2016), one of the dynamic architecture works, pro- duces the most substantial results in the task-Il. setting, specifically 95.13% and 67. 84% in CIFAR- 10 and tiny-ImageNNt, respectively. However, it suffers from an exponential memory increasing issue, making it impossible in class-IL and challenging datasets with more classes. Besides, our method won second place in the Task setting at the cost of keeping the network architecture fixed with margins 2.01% % on CIFAR- 10 and 0.42% on Tiny-ImageNet than the top rank. Compared with regularization-ban methods SI (Zenke et al. 2017), oEWC (Schwarz et al. 2018), and ALASSO (Park al. 2019) the gap does not seem to be bridged, indicating that the old parameter set's regu-
8

---

Published as a conference paper at ICLR 2022
Table 2: Classification results (average accuracy) benchmarks, averaged across runs. 7." denotes experiments we were unable to run, because of compatibility issues. Best performance marked in bold in each section. We use public code (Buzzega al. (2200) to reproduce results. Sections show joint training, model growing, parameter isolation, memory replay, regularization, and pseudo memory replay from the top.
R-F0
Tiny-ImageNe
Method
Class-IL
Task-IL
Class-IL
Task-IL
JOINT
92.02
98.29
59.45
82.07
PNN
95.131
67.84
HAL
41.79
84.54
MER
57.74
93.61
9.99
48.64
GEM
26.20
92.16
a-GEM
22.67
89.48
8.06
25.33
iCaRL
47.55
88.22
9.38
31.55
DER
70.51
93.40
17.75
51.78
BiC
49.18
89.91
11.09
32.94
WA
53.76
94.21
15.42
37.12
Mnemonics
57.26
95.91
23.47
42.36
TPCIL
58.72
96.67
25.03
48.41
oEWC
19.49
68.29
7.58
19.20
SI
19.48
68.05
6.58
36.32
ALA
25.19
73.79
17.02
48.07
UCB
56.23
78.56
23.43
49.01
DGM
71.94
89.91
28.45
61.32
DGR
49.69
79.86
17.38
38.41
MeRGAN
66.76
84.76
23.86
58.32
CF-IL (Ours)
75.34
93.12
33.06
67.42
larization is sufficient to mitigate forgetting since the wights' importance, calculated in an earlier task, might be unreliable in later ones. Even though CF-IL works without allocating memory buffer, our method can achieve excellent results in both class-IL and task-IL on both datasets. In particular, our proposed method obtains 75.43% 93. 12% in the class-IL, Task-IL settings the CIFAR-10. class-IL, Task-IL settings on the Tiny- ImageNet, respectively, making the relative improvement over the recent SOTA TPCIL (Tao et al. 2020) by 16.62%, 8.03% in the class-ll. settings on the CIFAR-10 and Tiny-ImageNet, and 19.01% the task-II. the more chal- lenging ImageNm respectively. We further investigate the impact of buffering samples against synthesizing samples in the ablation study. Compared to pseudo-rchcarsal- have recently gained much attention due the power of simulating past experiences rather than storing them, the proposed method shows a significant outperformance in both accuracy and pro- cessing time since there no auxiliary network needed. In particular, our method achieves relative improvement of 40%.3.22. 3.21 4.666 % 6. % over DGM (Ostapenko et al. 2019).
and 733.06%, 67.42%
methods, which
CONCLUSION
We have proposed a novel strategy for IL to address the memory issue, which is crucial when the number classes becomes large. In particular, we perform in both class-IL and task-IL settings in a cost-free manner. This strategy is implemented through memory recovery paradigm with no additional equipment. only relies on the single DNN, known the learner, to retrieve the network's past knowledge as a transfer set to look back on learned experiences. Our method has outstanding results, compared with recent prominent works.
ACKNOWLEDGEMENT
The authors would like to thank Part Research Center (Partdp.ai) for contributing to the hardware infrastructure used for experiments.

---

Published as a conference paper at ICLR 2022
REFERENCES
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with network of experts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- on Computer Vision (ECCV), pp. 139-154, 2018.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. arXiv preprint arXiv:1903.08671 20119,
C Atkinson, B McCane, L Szymanski, and A Robins. Pseudo-recursal: Solving the catastrophic forgetting problem in deep neural networks. arxiv 2018. arXiv preprint arXiv:1802.03877
Eden Belouadah and Adrian Popescu. 112m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 583-592, 2019.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark expe- rience for general continual learning: strong, simple baseline. arXiv preprint arXiv:2004.072/1, 2020.
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:/8/2.004200 2018.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and M Ranzato. Continual learning with tiny episodic memories. 2019.
Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Mareus Rohrbach. Unsertainty-guider continual learning with baycsian neural networks. arXiv preprint 1906..1900.022 2019.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733 2018
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confi- dence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Confer- ence on Computer and Communications Security, pp. 1322-1333 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Zecheng He, Tianwei Zhang, and Ruby B Lee. Model inversion attacks against collaborative in- ference. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 148-162, 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.022 02.533 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 831-839, 2019.
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barnskkskaaia et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13)3351-3557 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of 2009.
features from tiny images.
10

---

Published as a conference paper at ICLR 2022
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 2015.
Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi- class incremental learning without forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12245-12254, 2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning.
arXiv preprint arXiv: arXiv:/706.0000 770.00800008 2017.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 7765-7773, 2018.
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network multi- ple tasks by learning to mask weights. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 67-82, 2018.
Nicolas Y Masse, Gregory Grant, and David Freedman. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proceedings of the National Academy of Sciences, 115(44))
2018.
Michael McCloskey and Neal J Cohen. Catastrophic interference in comnectionist networks: The sequential learning problem. In Psychology of learning and molivation, volume 24, pp. 109-165. Elsevier, 1989.
Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty. Zero-shot knowledge distillation in deep networks. In International Con- ference on Machine Learning, pp. 4743-4751. PMLR, 2019.
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. 2017.
arXiv preprint
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11321-11329, 2019.
Dongmin Park, Scokil Hong, Bohyung Han, and Kyoung Mu Lee. Continual learning by asym- metric loss approximation with single-side overestimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3335-3344, 2019.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arXiv: 1810. 1910, 2018
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2): 123-146, 1995.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P Lillicrap, and Greg Wayne. Experience replay for continual learning. arXiv preprint 1rXiv:1811.11663 2018.
Amir Rosenfeld and John Tsotsos. Incremental learning through deep adaptation. IEEE transac- tions on pattern analysis and machine intelligence, 42(3):651-663, 2018.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.000 046776 2016.
Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barne Barwinska. Yee Whye Teh, Razvan Pascamu, and Raia Hadsell. Progress compress: A scalable framework for contin- ual learning. In International Conference Machine Learning, pp. 4528-4537 PMLR, 2018.
11

---

Published as a conference paper at ICLR 2022
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pp- 4548 4557. PMLR, 2018.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. arXiv preprint arXiv: 170.:0000,0,6690,, 2017.
Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong. Xing Wei, and Yihong Gong. Topology-preserving clas-incrrmmnnal learning. In European Conference on Computer Vision, pp. Springer, 2020.
Lin Wang and Kuk-Jin Yoon. Knowledge distillation and student-teacher learning for visual intelli- gence: A review and new outlooks. arXiv preprint arXiv:2004.05937, 2020.
Chenshen Wu, Luis Herranz, Xialci Liu, Yaxing Wang. Joost Van de Weijer, and Bogdan Raducanu. Memory replay gans: learning to generate images from new categories without forgetting. arXiv preprint arXiv:1809.008 02058, 2018.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Ju Xu and Zhanxing Zhu. Reinforced continual learning. arXiv preprint crXiv:1805.123360 2018.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint vrXXiv1708.000 01547, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
Mengyao Zhai, Lei Chen, Frederick Tung. Jiawei He, Megha Nawhal, and Greg Mori. Lifelong gan: Continual learning for conditional image generation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 2759-2768, 2019.
12


Reviews:
[{"review_id": "YI5mRodxZPO", "reviewer": "Reviewer_Q41Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- It is a cool idea to build a continual learning method that doesn't grow the model size over time or rely on external memory.  The proposed method distills information from the past networks to form a synthetic set of past concepts. Although the example synthesis method is largely based on the Zero-Shot Knowledge Distillation (Nayak et al. 2019), embedding this technique into a continual learning framework is very reasonable and novel. \n- The paper is well written and the structure is clear.  The clarity of `Modeling the Network Output Space` can be improved though as it's unclear how exactly to synthesize images from the past weights. \n\nCons\n- \"our method needs no fixed-size memory\" - This might be inaccurate as the model still needs to store the network from previous tasks (or the teacher network). It doesn't need external memory to store past examples or meta data. \n- Missing ablation study on example synthesis.  Is image synthesis necessary? Can you only synthesize features?  How fast is the synthesis procedure? It seems the computation is still an issue since this approach requires synthesizing images first and then training on the new tasks.  It might make sense to compare with existing approaches on computation, training time, etc. \n- Missing ablation study on the hyperparameters. How are the temperature, number of synthetic samples,  lambda is chosen? \n- Missing baselines in FS-IL and ZS-IL.  Table 1 reports the results in FS and ZS settings but didn't refer to previous methods for comparison. This Github link maintains recent works in this line. https://github.com/zhoudw-zdw/Awesome-Few-Shot-Class-Incremental-Learning\n- Missing comparison with the synthesis approaches with metadata.  How well does the method perform in terms of image realism? For example, ACE uses the mean and std of the past data to synthesize images. (https://openaccess.thecvf.com/content_ICCV_2019/html/Wu_ACE_Adapting_to_Changing_Environments_for_Semantic_Segmentation_ICCV_2019_paper.html)\n- It might be better to try the approach on more benchmarks like CIFAR-100,  CUB, iNaturalist, etc.\n\n\n\n\n\n", "summary_of_the_paper": "This paper proposes a zero-shot incremental learning approach that doesn't store past examples or metadata for experience reply. It synthesizes past experiences from the network parameters through a memory recovery paradigm.  The proposed method doesn't rely on external memory or parallel networks and achieves very competitive results on Task-IL and Class-IL tasks. ", "summary_of_the_review": "This paper provides a very interesting and novel idea for continual learning. However, the empirical evaluation and ablation study seem to be weak. \n\n=== Post-rebuttal Comments===\nThe authors addressed most of my concerns in the feedback.  I kept my score and leaned towards acceptance. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "I2XFWctXwI", "reviewer": "Reviewer_WhxT", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n&nbsp;\n\n- This paper is well-organized and easy to follow.\n- The proposed method is technically sound. \n- Extensive experiment results are provided. \n\n&nbsp;\n\n### Weaknesses\n\n&nbsp;\n\n- ***Replaying synthesised samples for old classes is not a novel idea.*** It has been proposed in [A]. In my view, the proposed memory recovery paradigm is very similar to the “data-free generative replay” strategy in [A].\n\n&nbsp;\n\n- ***The authors only provide experiment results on small-scale datasets, e.g., CIFAR-10 and Tiny-ImageNet.*** However, we usually use larger datasets to evaluate incremental learning methods. For example, iCaRL (Rebuffi et al., 2017), BiC (Wu et al., 2019), Mnemonics (Liu et al., 2020) run experiments on CIFAR-100 and ImageNet-1k in their original papers. I don’t see why the authors don’t follow the previous work and choose small-scale datasets.\n\n&nbsp;\n\n- ***Updating synthesised samples requires additional computational costs.*** The authors should compare the training time of their method with related work. If their method requires much additional training time, it is not practical. \n\n&nbsp;\n\n- ***The authors reduced the margin, which might practically violate the length limit.*** For example, the margin in Section 6 is significantly different from the ICLR template. \n\n&nbsp;\n\n- ***There are some minor formatting issues in this paper,*** for example, in the last paragraph on Page 4. The authors should carefully check the paper and make sure there is no such issue in the revision. \n\n&nbsp;\n\n- ***It would be better to provide a reproducibility statement in the paper.*** The authors provide a GitHub link, but I cannot see the code using that link. \n\n&nbsp;\n\n### Reference\n*[A] Choi, Yoojin, Mostafa El-Khamy, and Jungwon Lee. \"Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay.\" CVPR 2021 Workshops.*", "summary_of_the_paper": "In order to achieve exemplar-free incremental learning, the authors introduce a memory recovery paradigm to synthesize past exemplars. They also propose to evaluate incremental learning based on the zero-shot learning setting. Extensive experiment results are provided to show the effectiveness of the proposed method. ", "summary_of_the_review": "Overall, I think this paper proposes an interesting method and provides extensive experimental results. So I tend to accept this paper. However, similar ideas have appeared in the existing work, such as [A]. Many details of this paper are not polished. So the overall score is “borderline accept.”\n\n[A] Choi, Yoojin, Mostafa El-Khamy, and Jungwon Lee. \"Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay.\" CVPR 2021 Workshops.\n\n&nbsp;\n\n### === Post-rebuttal Comments===\nThe authors addressed most of my concerns in the feedback. They also provided the additional experimental results I asked for. I recommend acceptance. \n\nThe authors should include the additional results and corresponding analyses in the next revision.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8NiJp8sCUL5", "reviewer": "Reviewer_ruLe", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The proposed method takes an advantage of generative model-based approach without an explicit generative model.\n\n- Experimental results looks strong.\n\n- Discussion on memory replay methods is incomplete or misleading. Recent works (roughly, those published in 2019 or later) proposed to solve the bias problem, while the statement in this paper sounds the solution is not investigated. There are some early works addressing the bias problem but missing in this work: [Castro et al.], [Javed et al.], [Lee et al.].\n\n- The proposed method is not really zero-shot \"learning\"; In zero-shot learning, your model has never seen zero-shot classes but inferred them from auxiliary information.  It seems they argue so because their method has no extra memory or generative model, but I believe zero-shot learning is not the term for this case, thus misleading. The experimenal setting of LwF (though later works added extra memory to keep previously seen data in their experiments for fair comparison) or the usage of unlabeled data in [Lee et al.] (again, though they also added extra memory for previously seen data for fair comparison) also do not require an extra memory or generative model, but they have never claimed that their method is zero-shot (incremental) learning. I found that the zero-shot knowledge distillation work is cited, but their work is about zero-shot \"knowledge distillation,\" not zero-shot learning. I found that this paper did not cite any zero-shot learning works; I recommend [Xian et al.] to read.\n\n- More ablation study is required to convince the experimenal results: for example, the choice of the prior distribution for the recommender vector (even one-hot + Gaussian noise + normalization could be a simple baseline), the learning schedule for the memory recovery step (by the way, you should also report how long this step takes, otherwise your method has an unfair advantage on the training time than other compared methods), the size of the synthesized dataset, and so on.\n\n- In Table 2, I recommend to put the name of each types, either in the caption or by adding a column; otherwise it is hard to figure out how they are grouped, unless you fully memorize the related work section and the abbreviation of all prior works.\n\n[Castro et al.] End-to-end incremental learning. In ECCV, 2018.\\\n[Javed et al.] Revisiting Distillation and Incremental Classifier Learning. In ACCV, 2018.\\\n[Lee et al.] Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild. In ICCV, 2019.\\\n[Xian et al.] Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly. TPAMI, 2018.\n\n## Post rebuttal\n\nAs the authors addressed most of my concerns, I change my recommendation to 6: marginally above the acceptance threshold.\n", "summary_of_the_paper": "This paper proposes to generate pseudo data using the gradient-based method for class- and task-incremental learning. Experimental results show the effectiveness of the proposed method.", "summary_of_the_review": "The method is simple yet effective, which is good. However, \"zero-shot\" sounds misleading, and ablation study is not enough.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a zero-shot incremental learning approach that does not store past samples for experience replay. The idea is novel and well motivated, and the paper is well written. Reviewers' comments were mainly about missing baselines, missing ablation studies, and  clarifications about the proposed method. In the revised paper, the authors provided more justifications and added new experimental results on large benchmark datasets as well as ablation studies. After discussion, all the reviewers are positive about this submission. 

Thus, I recommend to accept this paper. I encourage the authors to take the review feedback into account in the final version.
Decision: Accept (Spotlight)