Improving Continual Learning by Accurate Gradient Reconstructions of the Past
Authors: Erik Daxberger
Siddharth Swaroop
Kazuki Osawa
Rio Yokota
Richard E Turner
José Miguel Hernández-Lobato
Mohammad Emtiyaz Khan
Venue: Submitted to ICLR 2023 (2023)

Abstract: Knowledge reuse is essential for continual learning, and current methods attempt to realize it through regularization or experience replay. These two strategies have complementary strengths, e.g., regularization methods are compact, but replay methods can mimic batch training more accurately. At present, little has been done to find principled ways to combine the two methods and current heuristics can give suboptimal performance. Here, we provide a principled approach to combine and improve them by using a recently proposed principle of adaptation, where the goal is to reconstruct the “gradients of the past”, i.e., to mimic batch training by estimating gradients from past data. Using this principle, we design a prior that provably gives better gradient reconstructions by utilizing two types of replay and a quadratic weight-regularizer. This improves performance on standard benchmarks such as Split CIFAR, Split TinyImageNet, and ImageNet-1000. Our work shows that a good combination of replay and regularizer-based methods can be very effective in reducing forgetting, and can sometimes even completely eliminate it.

Under review as a conference paper at ICLR 2023
IMPROVING CONTINUAL LEARNING BY ACCURATE GRADIENT RECONSTRUCTIONS OF THE PAST
Anonymous authors Paper under double-blind review
ABSTRACT
Knowledge reuse is essential for continual learning, and current methods attempt to realize it through regularization or experience replay. These two strategies have complementary strengths, e.g., regularization methods are compact, but replay methods can mimic batch training more accurately. At present, little has been done to find principled ways to combine the two methods and current heuris- tics can give suboptimal performance. Here, we provide a principled approach to combine and improve them by using a recently proposed principle of adaptation, where the goal is to reconstruct the "gradients of the past", i.e., to mimic batch training by estimating gradients from past data. Using this principle, we design prior that provably gives better gradient reconstructions by utilizing two types of replay and quadratic weight-regularizer. This improves performance on stan- dard benchmarks such as Split CIFAR, Split Tiny Our work shows that a good combination of replay and regularizer- methods can be very effective in reducing forgetting, and can sometimes even completely eliminate it.
and ImageNet-1000 1000.
INTRODUCTION
Continual learning Paaiisi 2019) aims for accurate incremental training over large number of individual tasks/examples. This can potentially reduce the frequency of retraining in deep learning, making algorithms easier to use and deploy, while also reducing their environmental impact (Diethe) et al. 2019; Paleyes et al. 2020). The main challenge in continual learning is to remember past knowledge and reuse to continue to adapt to new data. This can be difficult because the future unknown and can interfere with past knowledge (Sutton) [9866] Mermillod al. [013 Kirkpatrick ct al. 2017). Performance, therefore, heavily depends on the strategies used to represent and reuse past knowledge.
Two popular strategies of knowledge reuse are based on regularization and experience replay and have complementary strengths. For example, the well-known Elastic- Elastic-Weight Consolidation (EWC) (Kirkpatrick et al. (017). which regularizes the new weight-vector to keep it close to the old one, compact and requires storing only vectors, importance (often the empirical Fisher). A variety of other such regularizers have been proposed (Schwarz al.) 2018 Zenke et al.) 2017b Li Hoiem 2018 Nguyen et al. 2018 This very different from experience replay (Robins) 1995 Shhin et al. 2017), where past examples are simply added during future training. Memory cost here can substantial, but it can boost accuracy if the memory represents the past well. Clearly, combining the two approaches can strike good balance between performance and memory size.
containing the weights and the other their
At present, little has been done to find principled ways combine the two strategies. Some works have used knowledge distillation (Rebuffi et a 2017 Buzzega et al. 2020) or functional regulariza- tion (Titsias et al. 2020 Pan et al. 2020), where predictions evaluated at the examples in memory regularized. Such approaches are promising, but it is not clear why the specific choices of regular- izers and memory work well and whether there better choices that lead to further improvements. Here, we aim to fix this issue.
In this paper, we provide principled approach to combine and improve the two strategies. Our approach is based on recently proposed principle of adaptation (Khan & Swaroop 202I). where prior called Knowledge-adaptationn prior (K-prior) is used to reconstruct the gradients of the past

---

Under review as a conference paper at ICLR 2023
Previous Methods
Our Method
Function Regularizer
Σ (x;)) Sanaia (a;;) CMM
Kprior+EWC++eplay
M
M
EWC
[(f((x)), Swas (w)))
Woia Gola (wol(ww-ur) (aia
ieM:
+ Wor-woold (w
Rlald
Gold
Experience Replay
Wold
Goldy.
iEMICMi
fw SEM
M2
M
Figure 1: Using the principle of gradient reconstructions, we design a new prior (right) to combine different types of regularization and memory- memory-based methods. EWC uses quadratic regularizer based on the old weight-vector and its importance, while experience replay uses a memory of past examples along with their labels. Function regularization often does require the labels, but also lacks a weight regularizer. Our method combines these different types of methods. The notable differences are that our method's importance vector excludes the examples in the memory set, and that experience replay is applied only to the second memory set. The rest of the examples in the memory set do not require labels, and can be compressed to only keep a small set of representative inputs that may not be part of the old training sets.
training objective. In the continual learning setting. this requires an accurate reconstruction of the gradients over all the past tasks, and can be used as guideline design better priors. Khaa & Swa- rroo (2021) considered only one-task adaptation, which does not consider the errors accumulated multiple tasks in problems like continual learning. They also used simple quadratic regular- izer, and did not employ experience replay. Our goal here is to extend their method to multiple tasks and use it to combine different regularization and replay methods.
Using the principle of gradient reconstruction of the past, we design a prior that combines a weight regularizer, functional regularizer, and experience replay (Fig. D. Each piece contributes the reduction of different type of eтror. The reduction in the reconstruction eтror leads to consis- tent improvements on standard benchmarks for multi-tass image classification in iask-incremental continual learning setting, such as Split CIFAR, Split TinylmageNet, and ImageNet- 1000, across various memory budgets from small to large sizes. The results demonstrate the effectiveness of approach. The approach is principled and can yield provably better strategies than the current heuristics used the literature.
CONTINUAL LEARNING METHODS
We focus on a continual learning (CL) problem to incrementally learn from sequence of data sets Di,..., Dr, corresponding to total of 7 tasks. This is different from the commonly used batch- training deep learning, where data from all the tasks is assumed to be available all times during training. CL is challenging because the model needs to repeatedly adapt to new tasks, while not forgetting previous- previous-gathered knowledge. Our goal to get the performance as close as possible to the model that is trained on data from all tasks.
Formally, consider a supervised learning problem with D, containing N input-output pairs (æ,, 9:), with 2; R and ye E V, and wish train model with output fwr (æ;) (also denoted f()) and a P-length parameter w in a space W C R.. Then, at any given task t, the best possible model parameters wi can be obtained by training all the data examples D _ D., for example,
2

---

Under review as a conference paper at ICLR 2023
by solving optimization problem shown below,
: = arg min f
ghatch (w), where 4 chatch (w)
((ush((in)) + R(w). iEDi.
Σ
(1)
WEW
Here, we assume the loss ((ys,((ft) be defined through the log- log-likelihood of an exponential family distribution (e.g.. cross-entropy), with h(f) being transformation of the model outputs softmax) defined using the link-function the distribution. We denote regularizer by R(w), which could be either be implicit or explicitly defined, but in what follows, we will use an L2 regularizer R(w)= 2 with 2 O.
The main challenge in CL is remember the useful past knowledge extracted from previous tasks Dia-1, and reuse it during training over the new task Dj to get as close as possible to w We would like compact summary of the past knowledge, because data. We will now briefly describe two strategies used the literature, based on regularization and experience reply, and discuss the challenges in combining them.
not allowed to store all past
Regularization-basee approaches try to keep the new weight-vector close to the old one, with the hope that this will avoid forgetting and facilitate knowledge reuse. For example, the most common is weight-regularization technique known as Elastic Weiiht Consolidation (EWC) (Kirkpatrick al. 2017), where, at a task t, we minimize the following objective using the previous weight 11
( enoight (1)) = Σ (vi, + (w w:-1) FF-1(1 - w0:1),, iDD
(2)
where the second term is quadratic regularizer with weight-importann matrix F and trade-off hyperparameter. The simplest choice of F, to use the diagonal a generalized Gauss-Newton (GGN) matrix (Martens) 2014), where the GGN is defined as,
G(Dia) = Σ [V (( [V
(3)
iED
where V fff, denotes the derivative of (x;) with respect to w at we (the Jacobian), and N (f()) denotes the derivative of h(f) with respect to f and evaluated at a function value f³. Often, the regularizer 8 is added to the GGN matrix to reduce ill-conditioning. This method uses a compact representation of the past knowledge as need to only store w;-1 and the diagonal of F.-1- which takes O(P) memory size. Computation of F,-1 can be done in an online fashion as the training proceeds over tasks (Schwarz al.) 2018). There exist other choices for the importance vector (Zenke et al.) 2017a Alluunii et al. 2018: Benzing 2022). This method is also simple to implement within deep-learning codebases, and requires relatively little implementation overhead.
Instead of directly regularizing the weights, experience-replay based methods (Robins) 1995 Shin
al. 2017) simply store subset of past data in memory Mr. and add the new data during
training:
(w)
Σ
(ly,, h(f)))
iED,UM,-1
This method can be accurate when the memory represents the data well, but this often requires large memory that grows with the number of tasks.
A popular approach to combine these two previous approaches is to functional regularization (Tissias et al., 2020. Pan et al.) 2020) where, instead of regularizing the weights, we regularize the function outputs a few past input locations stored in memory:
gunn (wn)
Σ ((n. VED,
Σ JEMM-1
(4)
Some methods implement this via knowledge wwwlledgeddistilatiion (Rebuthi et al. 2017 Buzzega et al. 2020
and some also simply use the squared loss instead of using the original loss (Benjamin al. 2019
An advantage of functional regularization is that it does not require the labels associated with the inputs in My, which enables the use of arbitrary input which is not restricted to be from D D
3

---

Under review as a conference paper at ICLR 2023
For example, use deep generative model generate pseudo-inputs (Shin al. 2017). or learn them as in sparse Gaussian processes (Titsias et al. 2020).
Overall, we see that each method has its own complementary strengths. Weight-regullrizaten is compact, experience replay can be accurate, and functional regularization can use arbitrary inputs in memory sets. Combining these approaches can strike good balance between performance and memory size, but at present, little has been done to find principled ways to combine them. One could simply add them together, but there are many choices one need make. For example, how should we choose the importance vector, the memory set, and the specific forms of the regularizers? Our goal in this paper to provide a principled approach answer such questions.
PRINCIPLE OF ADAPTATION: GRADIENT RECONSTRUCTION OF THE PAST
We will use the recently proposed principle of adaptation by Khan & Swaroop (021) to combine and improve the CL strategies discussed in the previous section. The principle suggests to reconstruct the gradient of the past objective by using a combination of weight and function-space regularizers. Specifically, at task t, we consider minimizing (for some T > 0)
K (w) = Σ ((,,h(fW) + K(w; wt Me-1
(5)
iED;
with K(w; Mc being regularizer that combines a weight-space regularizer using w; and a function-space regularizer over the memory set M₁. The regularizer called the Knowledge- adaptation prior (or K-prior), and is designed with the goal to minimize the gradient reconstruction error of the past training objectives. We will use r = 1 unless noted otherwise.
Specifically, task t, the past training objective is ebach(()) and we want to design the prior to minimize the magnitude of the gradient error for all w:
ex(w)= == (w) VK(w; MI-1)
The loss dbathh depends all the past data D1.t-1, and goal is reconstruct its gradient by using the weight vector w and a memory Mr-1-
Khan & Swaroop 2221)) showed that many existing adaptive strategies in machine learning for one-step adaptation tasks can be recovered from this principle. For example, for generalized-linear models fú a² w, the error (ww is zero when we the following K-prior with M Di:::
5 [(w wir1) (w wi11). K(w; Men)) = Σ e(hifee M. + 6EMi-1
(6)
This is surprising because the K-prior does not use the labels, just like the functional regularization discussed earlier, yet the gradient be reconstructed by using the predictions at the inputs locations æ; E D1:--1- Many other similar results are shown by Khan & Swaroop (2021) for other models, such support vector machines, Gaussian processes, variational inference, knowledge distillation, and the memory-based methods used in continual learning.
which will fix in this paper. First, they did not design any priors for multi-task setup such as continual learning. Second, the error incurred by their K-prior, of form Eq. 46). is non-zero for neural networks: see Khan & Swaroop 2021 Sec. 4.2). For the continual learning problem, this can be disastrous because errors can accumulate quickly over tasks, deteriorating performance. Third, the weight-regularizer in Eq. (6) ignores the weight importance, which is commonly used in other works (Kirkpatrick et al. 2017 Schwarz et al.) 2018 Ritter et al. 2018) and can lead to suboptimal performance.
There are multiple issues with the work of Khan & Swaroop
In this paper, we will fix these issues and use the principle to combine and improve regularization and memory-based methods. We will start with functional-regulaaizaiioo and then add a weight regularizer and experience replay to decrease its error. This will give us a prior that provably gives better error than each individual method.
4

---

Under review as a conference paper at ICLR 2023
4 A NEW IMPROVED K-PRIOR
Using the principle described in the previous section, we will now design a prior that combines weight regularizer, functional regularizer, and experience replay (Fig. T The functional regu- larizer is based on knowledge distillation, and combined with an EWC-style quadratic regularizer which uses a specific importance vector to minimize the emror in the K-prior of Eq. ( An ad- ditional experience-reple memory set. We will see that each piece contributes to the reduction of a different type of error, and the combination overall gives lower error than each individual component.
term further reduces the error by storing the labels for subset of the
4.1 THE ERROR IN THE K-PRIOR EQ. (6) WHIEN USING A LIMITED MEMORY
We start by analyzing the error in the K-prior of Eq. (6) when it is defined with limited memory Mr-1. 1. instead of the full data D1s-1- As shown Appendix A the error is given as follows,
e, exww)=
Σ
V h(Sw)) -(Jrr...)
Lio w + ow:--;
IED 1
iED1e-1
: e (anren) Dun PIm-)\M-1))
(w:
where ri., -hfw.)) y4 is the residual left after prediction. The first eπror term e arises due to the use of limited memory M,-1, and can be reduced to zero by increasing the memory size to include all the past input examples. In contrast, the second error term e arises due the of neural networks, and reduces only when the network gets better in predicting the past data D Da--1, that is, when the residuals go zero. We will now show that em can be reduced by adding an EWC-style weight regularizer, while eN can be reduced by adding an experience replay term with a specific memory.
4.2 REDUCING USING AN EWC-STYLE REGULARIZER
The error e mem can be reduced by using a first-order Taylor approximation of ((())) at wi-1,
M(fin) x h(f(..,) + h (f (VV. f'....) ((w wi-1). Plugging this the definition of the error, we get
(8)
Vin N((Fmr..) [ ) (w w,,
mem
Σ
t
GGDia 1/M4 1)
where we use the definition of the GGN matrix given in Eq. gradient of the EWC-style regularizer,
3
The right hand side equal to the
w;-1), which uses the GGN over the past data but excludes the memory M,- from it. We can now define a new K-prior by adding the correction term as follows KG, ef which gives,
chee (w- wi-1) G(Dia-I\Me-))e
(9)
Kmmem (w; w W1-1, M--1)
Σ iEMi-1
eMMlie (,((()) { w,
FF-1(w (T..)
w,-1).
(10)
where we define Fi-1 G(D duced in the K-prior of Eq. (6) due to a limited memory. Similarly to Online EWC, we can use diagonal approximation to the GGN, and update it online.
8I. This K-prior provably reduces the error intro-
The new K-prior not only reduces the gradient error, but also is more general because other regular- izers are obtained as special cases by changing the memory. For an empty memory M,- = 0. reduces to the EWC regularizer of Eq. (2) because then the first term in Eq. (10) disappears and the importance F, is the GGN defined over all the past data, plus SI. On the other hand, when the mem- ory includes all past data, it reduces to the original K K-prior in Eq. (6). When using limited memory, the new K- X-prior combines the functional and weight regularizer in a way to reduce the error both EWC and the original K-prior.
5

---

Under review as a conference paper at ICLR 2023
4.3
REDUCING e NN USING EXPERIENCE REPLAY
The eNN term depends on the mistakes made on the past data, and can be corrected by including an additional memory 1 of the past data where mistakes are significant. We first note that the emor is equivalent to the gradient of the following,
furw. + fow T wt-1: iE.22,1-1
Σ
(11)
when M22,-1 is set to all the past data. Therefore, by adding A Kr + C,N, we reduce the error simply to EiED
V fiu w
4.4 K-PRIOR WITH EWC C-SYLE REGULARIZER AND EXPERIENCE REPLAY
Our new improved K- K-prior is obtained by simply correcting the K-prior from Eq. (6) by adding the correction terms, that is, Kerow K. + Ct CN We make a specific choice of the memory: choose M2,2-1 to be a subset of MM, This, as we show now, simplifies the computation and brings experience replay into our new K-prior.
Consider the following two terms taken from cor
from Eq.
10
and K cor
NN from Eq.
11
Σ e (N(
Σ fwr
BEM.-
iEM2,t-1
The gradient of these two can be simplified, when the second memory is subset of the first one,
fiu
(h(f())
h(fūr
Σ
Viuri 10111
iEM2.11
Σ
V fir
h(fin)
h(fine))
Vin h(f(in) M(fm 1)] Vfm
iEMi-i
iEM2,
Σ
V fir h(fin)
(ffre))
Σ
V (b(fi) vt]l
SEMI-I
iEM2,.
where in the last line used the definition of the residual to simplify. This gradient is equal the gradient of a sum of a functional-regularizee term and an experience replay term,
Σ
Σ iEM2.
E(r, h(Ti)
(12)
Therefore, rewrite the new K-prior as the following combination,
neew (w; 11,1,
Σ (h
Σ
e h(fa))
iEM1.-1
M
Functional regulurization
Experience replay
(w W:-1
F F. w,-1) Jow T
w,-1
(13)
riiggh-regulaiizatiion
where M1,111 denotes the M,-1 without the M2,1-1. The new prior combines a weight regularizer, a functional regularizer, and experience replay. The functional regularizer is based on knowledge distillation, and combined with an EWC-style quadratic regularizer which uses an importance vector obtained over all the past data but excluding the memory M Using this combination, the new prior gives provably lower gradient reconstruction error than each method alone.
In our experiments, we select the memories as follows. We first randomly (uniformly) sample M from all data D1.t-1. and then sample M2 as a subset of MI. We generally set [M = MM for simplicity. We found this to be superior using all soft labels (i.e. where Mi M2 0, as in vanilla K K-priors) or all hard labels (i.e. where M1 My, as in Replay (Robins) 1995). which
is known perform poorly (Khan & Swaroop 2021 ). We found this simple selection strategy to
work surprisingly well, and leave the study of more sophisticated approaches (as well as settings
where MM * () for future work.

---

Under review as a conference paper at ICLR 2023
0.825
Forgetting (aeg, over taskss))s)
0.800
0.775
-0.02
Test (avg.
0.750
0.725
Xprior=EWC+Reee CoRpoplay
Batch Joint Batch Separ.
-0.04
Batch Joint
Xprior-EWC
Kprior-EWC-RReplay Kprior
0.700
Kprior+ Replay
Ouline EWC
-0.06
Kprior
Online FWC
0.675
200
1000
2000
5000
Number of memorable points (per task)
Number of tasks
Number tasks
Figure 2: Results on Split-CIFAR, Kprior+EWC+Replay is superior across memory sizes, closely approaching Batch Joint for memory size 5,000 (left; x-axis log-scaled). It also forgets less (relative to Batch Joint) with growing number tasks, for memory sizes 100 (middle) and 5,000 (right).
5 EMPIRICAL EVALUATION
We first describe the general experimental setup used throughout our evaluation. We then present empirical results corroborating the practical efficacy of our proposed EpCeely method. We focus multi-class classification in the task-incremental learning setting. In particular, we evaluate on three continual learning benchmarks with increasing size and thus difficulty: 1) Split- peediuu-tt-llage-schhee and 3) ImageNet-100V
CIFAR (medium-scale), 2) (large-scale). See Appendix for more details on the experiments (e.g. hyperparameters used).
5.1 EXPERIMENTAL SETUP
CL setup. We mostly follow previous works on CL. We consider the common multi-head task- incremental (Van de Ven & Tolias 2019) setting with known task identities: each method is trained sequentially on all tasks with separate classification head per task, and told which task an input belongs both at train and test time. We report test accuracy of the final model trained on the entire task sequence (averaged over the test sets of all observed tasks). We also compute average forgetting (aka Backward-transker difference accuracy between when a task is first trained and after the final task. We plot mean + standard error over three seeds, and assess performance across wide range of memory sizes.
as defined in Lopez-Paz & Ranzato (207)). which captures the (average)
Methods. In addition to our proposed KprioryEWC+Replay method, we evaluate four relevant baselines for comparison. As method combines Kprior with both EWC and Replay, also do ablations to assess the benefit of either term alone. In summary, we consider the following methods:
1. Kprior+EWC+Replay Our proposed regularizer which combines the K-prior function reg- ularizer over M₁ with the EWC-style weight regularizer and the Replay term over M2.
2. Kprior. The original K-prior regularizer over M1 as proposed by Khan & Swaroop 2021 without the EWC-style weight regularization term and without the Replay term over M2.
3. Kprior+EWC (ablation). regularizer combining the K-prior function regularizer over Mi with only the EWC-style weight regularization, i.e. without the Replay term over M2.
4. Kprior+Replay (ablation). regularizer combining the K-prior function regularizer Mi with only the Replay term over M2, i.e. without the EWC-style weight regularization.
5. Batch Joint. Joint batch training of single multi-head model across the data of all tasks, i.e. the optimal CL solution which serves as an upper bound we wish approach.
6. Batch Separate. Independent batch training of a separate model for each task.
7. Online EWC (Schwarz et al. 2018), which has the same weight regularizer as Kprior+ EWC+Replay, but withou the function regularizer Mi the Replay term over M2.

---

Under review as a conference paper at ICLR 2023
0.55
Test (avg. tasks)
0.50
PackNet
iCaRL
0.50
MAS
0.45
EBLL
0.45
HAT
A
R FM
0.40
Kprior
Kprior+FWC .... Batch Separ.
Online EWC
0.35
LWF
GEM
EWC
Batch Joint
R-PM
50
100
200
500
1000
2000
4000
500
1000
mode-IMM
Number of memorable points (per task)
memorable points (per task)
Figure 3: Results on Split-TinlImgee Kprior+EWC performs well across memory sizes (left; x-axis log-scaled) and compared to further strong baselines from Delange et al. (2021) (right).
5.2 RESULTS ON SPLIT-CIFAR
Setup. Split-CIFAR Zenke et al. 2017b) has 6 tasks with 10 classes each. The first task is CIFAR- 10 (Krizhevsky et al. 2009) with 50,000 training and 10,000 test data points across classes. The subsequent 5 tasks are taken sequentially from CIFAR-100 (Krizhevsky et al. 22009) each with 5,000 training and 1,000 test data points across 10 classes, In total, we thus have 90,000 data points. We use the same CifarNet model as Zenke et al. (2017b); Pan et al. (2020): multi-head CNN with convolutional layers, followed by 2 dense layers with dropout, with ~ 1.2MM model parameters in total. On each task, we train for 80 epochs using Adam with learning rate 10-3 and batch size 256.
Results. Fig. 2 shows our results on SIFAR-CIFAR. We consider memory sizes between 100 and 5,000 per task; 5,000, thus store 10% of the data for task and all data for tasks 2-5. We that Kpri or performs poorly at small memory sizes and is much worse than Onli inline EWC. While performance improves noticeably with growing memory size, Kprior remains far below Bat Joint even at memory size 5,000. This confirms that when using NN instead of GLM, the the- ory behind Kpricr (see Section 3 indeed ceases hold. However, when we add the Replay term to correct for that error (Kpr i or+Replay), performance is substantially boosted at large memory sizes, actually enabling us to reach Batch Joint performance for memory size 5,000. However, the Replay correction term does not help at small memory sizes. In contrast, adding the EWC-style weight regularizer ExpriortEWe substantially improves performance at small memory sizes, em- pirically confirming the property that Kpriort EWC converges Online EWC for small mem- ories (see Section 4.2).. However, we also confirm that for large memories, Kprior+ con- verges to Kpr i for, resulting in performance drop. Finally, we see that Kprior+EWC+Rep) combines the complementary benefits of both error correction terms, i.e. of both EWC-style weight- regularization and Replay, to significantly improve upon vanilla Xprior in both the small and large memory regime. Fig. (mid & right) further shows that our method can leverage prior knowledge more effectively than other methods, thereby suffering less from forgetting with growing number of tasks. This confirms that the two error correction terms in Kpr poriorto +WW+RRplla are particu- larly important for mitigating error accumulation across longer task sequences.
5.3 RESULTS ON SPLIT-TINYMMGGNNE
Setup. Following Delange et al. (021), we construct Salit-Tinylmageeee by dividing Tinylma- geNet (Le & Yang 2013) into a sequence of 10 tasks with 20 classes each (using the same random division as in Delange et al. (2021). l. Each class has 500 data points split into training (80%) and val- idation (20%). and 50 test points (totalling to 11,000 points). We use the VGG-like BASE model from Delange et al. (202I) with 6 convolutional layers, max pooling layers, and 2 dense layers, with total of 3.55 parameters. On each task. train for 70 epochs (with carly stopping and exponential learning rate decay, without regularization) using SGD with momentum 0.9 and batch size 200. This replicates the setup of Delange al. (2021) to make results directly comparable
'The only difference lies in the hyperparameter tuning procedure: while Delange al. 2021 use their proposed online tuning algorithm, we resort standard grid search for simplicity; see Appendix for details.
8

---

Under review as a conference paper at ICLR 2023
Test (avg. tasks)
Forgetting (avg. over taskss
0.8
-0.05
-0.10
0.4
Cpeeelayy
Kprior
Kprior+EWC Repior+Replay
Online EWC
0.15
200
500
1000
2000
5000
10000
8
10
Number memorable points (per task)
Number of tasks
Figure 4: Results on ImageNet-1000. Kprior+EWC+Replay performs favorably across a range of memory sizes (left; x-axis log-scaled). It also suffers less from forgetting (relative to Batch Joint) with an increasing number tasks, here exemplary shown at the largest memory size of 10K (right).
Results. Fig. 3 shows our results on Pelit-TnylmaaeNer We found that the Replay error correction term does not help on this benchmark, so we representatively plot just Kprior and Kprior ior-+ +EWC can substantially improve over Kprior (especially at small mem- ory sizes) and Onl ine EWC (Fig. left). It also compares favourably against diverse range of other strong CL methods across all three CL paradigms: 1) memorylrehearsal
We again see that Kprior+
iCaRL (Rebuffi
et al. 2017), 7), GEM Lopez-Pazz & Ranzato 2017). R-FM & R-PM (Delange al. 2021), 2) weight- regularization - LwF (Li & Hoiem 2017), EBLL (Rannen al. 2017). EWC (Kirkpatrick et al,
2017). SI (Zenke et al. (017a). MAS (Aljundi al. 2018). mode- IMM (Lee et al. 2017) and 3)
architectural PackNet (Mallya & Lazebnik
2018). HAT Serra et al.
2018 (Fig.) right)
5.4 RESULTS ON IMAGENET- 1000
Setup. We consider the ImageNet-1000 benchmark proposed by Rebuffi et al. (017). which ran- domly (uniformly) splits the full ImageNet dataset [Deng al. (2009) of 11.2M data points into sequence of 10 tasks with 100 classes and ~~20 data points each. Following Rebuffi et al. (0177).
use ResNet-18 with MIM model parameters. For training on each task, we use the ImageNet reference training pipeline (with 40 epoch configuration) of the FFCV library (Leclere et al. 2022
Results. Fig. 4 shows our results on ImageNet- 1000. We consider memory sizes between 200 and 10K per task, where the latter amounts to just under 10% of the data. The observed trends qual- itatively match those from previous experiments. memory sizes, and while it improves with increasing memory, it peaks at 5K memory and then even starts declining. We hypothesize that this is again due to accumulation of the NN error, which might become more severe with larger memory more data points contribute the error. This evidenced by the fact that correcting for the NN error SepriorrRReleee substantially boosts performance at 10K memory (but it remains poor small memories). In contrast, Xprior+EWC combines the
particular, Kprior orr underperforms for small
again improves accuracy only for small memories. Finally, benefit of both error correction terms perform well across all memory sizes. It also again forgets less along the task sequence, demonstrating that it better mitigates error accumulation.
CONCLUSION
We proposed to address the CL problem in theoretically- groounddd way by explicitly approxi- mating the optimal model obtained via batch-training on all tasks jointly. To this end, we devel- oped Kprior+EWC++Replay which efficiently re-uses prior knowledge by combining principles from function-regularizatioo beeight-reuularizatin and experience replay. Empirically, we demonstrated the effectiveness and scalability of our method across memory sizes, compared to various basclines.
Thhis is likely because almost perfect train accuracy is attained on all tasks (see e.g. Table 14 in Delange al. 2)21)). Thus, NN in Eq. close zero, such that NN emrr correction cannot boost performance. Results are from Delange et al. (2021): their total memory sizes [4500, 9000] equal (500, 1000) per task.
For all details of the training procedure, see see/htssss:///// //g1 thub. com/ ibf Fev/ F Ffov- i magenet/

---

Under review as a conference paper at ICLR 2023
REFERENCES
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Ari S Benjamin, David Rolnick, and Konrad Kording. Measuring and regularizing networks in function space. International Conference on Learning Representation, 2019.
Frederik Benzing. Unifying importance based regularisation methods for continual learning. In International Conference on Artificial Intelligence and Statistics, pp. 2372-2396. PMLR, 2022.
Pictro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark ex-
perience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 33:15991 11520-159300 2020.
Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. leee, 2009.
Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, and Neil Lawrence. Continual learning in practice. arXiv preprint 0rXiv:1903.05202 2019.
Mohammad Emtiyaz Khan and Siddharth Swaroop. Knowledge-adaptation priors. Advances in neural information processing systems, 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwingg et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):35213352) 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Guillaume Leclere, Andrew Ilyas, Logan Engstrom, Sung Min Park. Hadi Salman, and Aleksander Madry. ffev. bttps://github. com www//ibffew//feee.// 2022. commit (25386557621371 1cc8601 833ddd36f996666002.
Sang-Woo Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information pro- cessing systems, 30, 2017.
Li and D. Hoiem. Learning without forgetting. Machine Intelligence, 40(12):2935-290 2018.
IEEE Transactions on Pattern Analysis and
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis 2017.
and machine intelligence,
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks single network by iterative pruning. In Proceedingy of the IEEE conference on Computer Vision and Pallern Recognition, pp. 7765-7773, 2018.
James Martens. New insights and perspectives on the natural gradient method. arXiv preprint arXiv: /412 2014.
10

---

Under review as a conference paper at ICLR 2023
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The tigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in psychology, 4:504, 2013.
dilemma: Inves-
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard Turner. Variational continual learning. International Conference on Learning Representation, 2018.
Andrei Paleyes, Raoul-Gabriel Urma, and Neil D Lawrence. Challenges in deploying machine learning: a survey of case studies. arXiv preprint arXiv:20/1.09926, 2020.
Pingbo Pan, Siddharth Swaroop, Alexander Immer, Runa Eschenhagen, Richard E. Turner, and Mohammad Emtiyaz Khan. Continual deep learning by functional regularisation of memorable past. Advances in neural information processing systems, 2020.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
Amal Rannen, Rahaf Aljundi, Matthew Blaschko, and Tinne Tuytelaars. Encoder based lifelong learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1320- 1328, 2017.
Sylvestre- Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampper. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp- 2001-2010, 2017.
Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, pp- 3738-3748, 2018.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2): 123-146, 1995.
Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for contin- ual learning. In Proceedings of Machine Learning Research. PMLR, 2018.
Yee Whye
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task, In International Conference on Machine Learning, pp- 4548 4557. PMLR, 2018
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in neural information processing systems, 2017.
Richard Sutton. Two problems with back propagation and other steepest descent learning procedures for networks. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society, 1986, pp. 823-832, 1986.
Michalis K Titsias, Jonathan Schwarz, Alexander G de G Matthews, Razvan Pascanu, and Yee Whye Teh. Functional regularisation for continual learning using Gaussian processes. International Conference on Learning Representation, 2020.
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995. JMLR. org, 2017a
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995 PMLR, 2017b.
11

---

Under review as a conference paper at ICLR 2023
A DERIVATION OF ERROR FOR K-PRIORS WITH LIMITED MEMORY EQ. (7)
Recall that füv = (2;)) is a shorthand for the model outputs. For ease-of-notation, we will also use the shorthand hew h(f(w) h( (2:)) ) for the model predictions. Consider the following expression for the gradient of the eexponentialffamilyy loss (Khan & Swaroop. 2021).
Vely,, h(fw)) = V [h(f(] - -y:]
(14)
We then have,
haachh
(w) Vet px prior (w)
hatch
(w0) - Ex-rrior (20)
Σ
(yee mlw))) + (ome))
- Σ
e (Mi K(w; ((ww;
EEDi
REDD
( (2)))( K(w;; (10;;)))
Σ
(yis Ee
li B Mow) + f 11?
iED
iEM
Σ
Ve (yi, Mow)
Σ ve
(hee, n:o + :rw.-1
SED
iEM
[
yi]
V fiin
hig nim
+ 81
iEDLa
iEM
E
f h Ys
+
hin
hi
h
him
Σ ieM
füv hi hmm .] 8r
iED
Vi
+
fi
V f him Mmne
+ 8w
iED
NED
iEDia-1
iEM
Σ
Ti Mum Vi
+ +800
w
iEDL-IIM
iEDia-1
Σ
M
Viur wi-
+ 5
w
VEDLa-IM
iEDii-1
where rw.,, = Mu..,
Yi is the residual of the i'th input using the past model parameters w;-11
B EXPERIMENT DETAILS
For all methods in all experiments, we tuned hyperparameters in the common way by conducting a aypponentially-spacedd grid search and evaluating performance on a held-out validation set. In particular, we tune the following hyperparameters: temperature parameter T to scale the logits in knowledge distillation, see Khan & Swaroop (202T) for discussion), trade-off parameter in front of the K- K-prior-style function regularization term, and trade-off parameter À in front of the EWC-style weight regularization term
the Replay term (as commonly-done
We used the following tuned values for those hyperparameters: for Split-CIFAR we have T = 2.0 for all methods, 7 = 0.1 for Kprior and Kprior+Replay, т = 0.25 and À = 2.0 for Kprior+WWC and Kprior+EWC+Replae
and X 10.0 for Online EWC. For Split-Tinylmagee Tmny.mageNet, and 7 = 16.0 for all methods. We found that a fixed À across tasks does not work well for this bench- mark, so we tuned separate X per task, resulting in the sequence [330, 85, 45, 30, 20, 15. 10, 10) for all methods. In contrast to our grid search procedure, Delange et al. 2021 use a dedicated itera- tive hyperparameter tuning strategy that trades-off plasticity vs. stability. For ImageNet, used T' = 1.0, À = 1.0 and T = 0.16 for all methods.
have T 1.0
12


Reviews:
[{"review_id": "rmq5_7KJiL", "reviewer": "Reviewer_6uVZ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to address the CL problem by approximating the optimal model obtained via batch-training on all tasks jointly. To achieve this, the Kprior+EWC+Replay is developed to efficiently re-use prior knowledge. Experimental results demonstrate the effectiveness and scalability of the proposed method.", "summary_of_the_review": "The paper provides a general principle to combine the commonly used CL strategies, but the novelty and the experimental settings can be further improved. I tend to reject it.", "strength_and_weaknesses": "Strength\n\n1.The proposed method provides a principled way to combine and improve the regularization and replay methods.\n\n2.The theoretical analysis of the proposed method is detailed and sufficient.\n\n\nWeaknesses\n\n1.The comparison with other CL methods for task incremental learning is not enough.\n\n2.The experimental design is repetitive and simple, and the amount of information shown by experimental results is not enough.\n\n3.As a combined methods, only one set of components are selected(Kprior、EWC、Replay).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_7lQqdgrgt", "reviewer": "Reviewer_goJh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper combines data replay and EWC into a single objective to tackle continual learning of tasks.\n\nResults confirm the benefit of combining an experience replay buffer to replay data from old tasks, and functional priors such as EWC.\n\nAuthors perform several ablations on split CIFAR100, split mini imagenet and imagenet 1000.", "summary_of_the_review": "The work presents an empirical study of combining different methods proposed to deal with catastrophic forgetting in continual learning. Although well presented, the results are not surprising and don't bring new insights about continual learning.", "strength_and_weaknesses": "Strenghts:\n - clear presentation of the method\n - the empirical results show improvement over the baselines\n - baselines\n\nWeaknesses:\n - limited novelty\n - small batch of experiments (no RL; baselines were also evaluated in RL setups)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1T91JaLANV4", "reviewer": "Reviewer_ozeD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a method combining two approaches: regularization and experience replay. Using K-prior which tries to reconstruct the gradient information of old tasks, the authors smoothly derive the regularization and experience replay through minimizing the error $e^{mem}$ and $e^{NN}$. In the experiment, the proposed method achieves comparable results to the baselines.", "summary_of_the_review": "I vote to reject this paper. The motivations are weak, and proposed methods are not be a novel approach. Furthermore, some statements are not justified and not easy to understand.", "strength_and_weaknesses": "**Cons:**\n\nC1. The comment \"Weight-regularization is compact, experience replay can be accurate, and functional regularization can use arbitrary inputs in memory sets\" is not persuasive and ambiguous to be a strength. Using multiple approaches at once can rather take much more memory and computational costs. It would be better to specify the details on how those approaches can be compatible each other.\n\nC2. The motivations for combining the regularization and experience replay are somewhat weak. Just combining those two approaches are not closely related to solving the fundamental problems in CL. It is not clear what kind of problems the authors try to solve in this paper. \n\nC3. The performance of proposed method is much lower than baselines. Though it uses the memory exemplar, the accuracy is slightly lower than PackNet which does not use any memory exemplars during training. \n\nC4. Is it correct to saying \"This K-prior provably reduces the error introduced in the K-prior of Eq. (6) due to a limited memory.\" which is below (10)? I am not sure adding just Fisher matrix to (6) provable reduces the error", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "04XB8ki1mB", "reviewer": "Reviewer_jV4b", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on the problem of combining replay models and regularization techniques for task-incremental continual learning (TIL). In the words of the authors, \"we design a prior that provably gives better gradient reconstructions by utilizing two types of replay and a quadratic weight-regularizer\". The concept of prior is taken from (Khan & Swaroop, 2021), where the focus was on model retraining to, e.g., unlearn data. Intuitively, a prior is a term that is added to the loss (Eq. (5)), whose gradients approximate the past loss function (equation below (5)). \n\nIn the TIL case, the correction is the loss computed over a small memory of the past task. The authors show that a series of simplifications bring back a number of known techniques:\n\n1) A functional regularization over the memory (Eq. (6)).\n2) An EWC-like term over the past data minus the memory (Eq. (9)).\n3) An experience replay term over a subset of the memory (Eq. (12)).\n\nIn practice, the algorithm proposed in the paper is a combination of these three (known) terms, with the requirement that the memory for (3) is a subset of the memory for (1), and that (2) is computed over the past data minus the memory itself.", "summary_of_the_review": "The paper provides an interesting derivation of known ideas from the CL literature, eventually leading to a combination of three known techniques from the literature.", "strength_and_weaknesses": "I have found the derivation of the three techniques starting from a single principle to be interesting and, as far as I know, novel.\n\nThe derivation itself is not always easy to follow. For example, on page 4 they state \"For example, for generalized-linear models\" and they provide Eq. (6). Later on, they use this equation as the basis for their extensions, which was not clear immediately.\n\nThe experiments are comprehensive and show the usefulness of the method, although they do not provide a lot of information about computational time and FLOPS.\n\nMy main concern is that the combination of methods is not particularly novel, especially since the proposed derivation does not offer a lot of insights except for a careful choice of the memory itself. In addition, the authors are focusing only on the simple TIL scenario, it is unclear whether this can be extended to more complex (e.g., class-incremental) scenarios.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject