Loss Decoupling for Task-Agnostic Continual Learning
Authors: Yan-Shuo Liang
Wu-Jun Li
Venue: NeurIPS 2023 poster (2023)
Keywords: Continual Learning
stability
plasticity

Abstract: Continual learning requires the model to learn multiple tasks in a sequential order. To perform continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Task-agnostic problem in continual learning is a challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. In task-agnostic problem, the model needs to learn two new objectives for learning a new task, including distinguishing new classes from old classes and distinguishing between different new classes. For task-agnostic problem, replay-based methods are commonly used. These methods update the model with both saved old samples and new samples for continual learning. Most existing replay-based methods mix the two objectives in task-agnostic problem together, inhibiting the models from achieving a good trade-off between stability and plasticity. In this paper, we propose a simple yet effective method, called loss decoupling (LODE), for task-agnostic continual learning. LODE separates the two objectives for the new task by decoupling the loss of the new task. As a result, LODE can assign different weights for different objectives, which provides a way to obtain a better trade-off between stability and plasticity than those methods with coupled loss. Experiments show that LODE can outperform existing state-of-the-art replay-based methods on multiple continual learning datasets.

Loss Decoupling for Task-Agnostic Continual Learning
Yan-Shuo Liang and Wu-Jun Li"
National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, P. China liangys@smail. .nju. edu. en, liwujundnju.ede minnnju.edu.cm
Abstract
Continual learning requires the model to learn multiple tasks in a sequential order. To perform continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Task-agnostic problem in continual learning is challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. In task-agnostic problem, the model needs to learn two new objectives for learning a new task, including distinguishing new classes from old classes and distinguishing between different new classes. For task-agnostic problem, replay-based methods are commonly used. These methods update the model with both saved old samples and new samples for continual learning. Most existing replay-based methods mix the two objectives in task-agnostic problem together, inhibiting the models from achieving good trade-off between stability and plasticity. In this paper, we propose a simple yet effective method, called loss decoupling (LODE), for task-agnostic continual learning. LODE separates the two objectives for the new task by decoupling the loss of the new task. As result, LODE can assign different weights for different objectives, which provides way obtain a better trade-off between stability and plasticity than those methods with coupled loss. Experiments show that LODE can outperform existing state-of-the-e oftt---rttt-ttttt replay-based methods multiple continual learning datasets.
1 Introduction
Continual learning requires the model learn multiple tasks sequential order 31 However, neural network models suffer from phenomenon called catastrophic forgetting (CF) [12). in which the performance of the network on the old tasks degrades significantly after learns new task. To
address this challenge and enable continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Nevertheless, an excess of stability plasticity can interfere with the other [39], and hence the model needs to make trade-off between stability and plasticity [39 31).
There exist two different kinds of problems, task-agnostic problem and task-aware problem, in continual learning. fask-agnostic problem (39 continual learning challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. contrast, task-aware problem 28 in continual learning enables the model to get task identities in the inference stage. Therefore, the model for task-aware problem only needs distinguish the classes belonging to the same task. The difference between task-agnostic problem and task-aware problem 28 29 shows that the model in task-agnostic problem
"Wu-Jun Li is the corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

O Old classes
Scadaty
Ralovodd 2rna
Plasbuty for clas dkdintion
Pasiciy rewide clace (iilintion
Banibally
Newlold class distinction
New cless distinction
Exsting methods
Ours
(a) Different learning objectives
Stabilityplasttiit trade-off in different methods
Figure 1: (a) shows two different learning objectives for learning the new task. The left side of (b) shows the trade-off between stability and plasticity in most existing methods, which mix different learning objectives. The right side of (b) shows that method separates the learning of new task into two objectives and hence separately considers the trade-off between stability and plasticity for these two learning objectives. y-axis in (b) represents the model's abilities, including plasticity and stability.
needs to learn two objectives for learning a new task, including distinguishing new classes from old classes (called new/old class distinction) and distinguishing between different new classes (called (a) illustrates these two learning objectives.
new class distinction). Figure !
Many methods have been proposed for continual learning, including regularization bbasde methods 455
20 2], expansion-based based methods [34 18 23 and replay-based methods 3 TO
For task-agnostic
problem, replay-based based methods are commonly used. These methods use a memory buffer to maintain small portion of samples from the old classes. When learning
task, the model retrieves old samples from memory and updates the parameters with both new and old samples. As illustrated in Figure (b) from the methodological perspective, most existing continual learning methods [9] 7 43
mix the two objectives in task-agnostic problem together (discussed later in Section (.T]. But these two learning objectives may cause different degrees of forgetting in continual learning and thus different trade-off strategies between stability and plasticity are required for these two learning objectives. More specifically, if a new learning objective leads to more forgetting, good continual learner should pay more attention to the model's 's stability. On the contrary, if new learning objective leads to less forgetting, a good continual learner should pay more attention to the model's 's plasticity for this objective. However, when the model mixes different learning objectives together, adjusting one of the learning objectives may influence others, inhibiting the model from achieving a good trade-off between stability and plasticity.
In this paper, we propose simple yet effective method, called loss decoupling (LODE), for task- agnostic continual learning. The main contributions of this paper are listed as follows:
By deeply analyzing the impacts of new/old class distinction and new class distinction, we find that these two learning objectives cause different degrees of forgetting. Therefore, mixing these two objectives together is detrimental for the model to make good trade-off between stability and plasticity.
LODE separates the two objectives for the new task by decoupling the loss of the new task. As a result, LODE can assign different weights for different objectives, which provides a way obtain better trade-off between stability and plasticity than those methods with coupled loss.
Experiments show that LODE can outperform existing state-of-the replay-based methods multiple continual learning datasets.
2
Related Work
Continual learning can be offline or online. In offline continual learning setting 17
32
24
the model receives the entire dataset of new task and updates its parameters multiple times over this dataset. In online continual learning setting II TO the data from each task is sequentially concatenated non-stationary data stream and each data of each task can only appear once in the

---

data stream. Hence, the model can only receive a mini-batch of samples from the data stream at time and update its parameters based on this mini-batch.
Different types of methods have been proposed for continual learning, including regularization-bane
methods 45 20 2). expansion-based methods 64 18 23 and replay-based methods [3] 10 4 For
task-agnostic problem, replay-based methods are commonly used. Some replay-based methods 40 42] achieve replay by learning a generative model for generating old samples. However, learning a good generative model is challenging in some settings, like online continual learning [3 Some replay-basee methods like experience replay (ER) 9 maintain a memory buffer to save old samples and replay them with new samples. Some methods combine experience replay with knowledge distillation [15]. Specifically, these methods store either a old model [43] or the outputs of the old models 77. Some methods [25] like experience replay with asymmetric cross entropy (ER-ACE) 8 improve ER by changing the loss of the new task so that the model can avoid large representation drift. Some methods like error-sensitive modulation experience replay (ERMER) (66 and complementary learning system experience replay (CL.SEE)
use an extra set of model's parameters to aggregate the knowledge of different tasks, Other methods try to search for valuable samples [3] or optimize the distribution of the memory [99 41 to overcome forgetting. However, all these methods mix two new learning objectives in Figure (a), inhibiting the models from achieving good trade-off between stability and plasticity.
3 Methodology
In this section, we first formulate the problem in continual learning. Then, we deeply analyze the problem of mixing different learning objectives together in existing methods. After that, we propose our method called loss decoupling (LOOE)), which can be used in both offline and online continual learning settings. Finally, we discuss the relation between LODE and existing methods.
3.1 Problem Formulation
Continual learning requires the model to learn from multiple tasks in a sequential order. We use D, {a+, vt J.vi to denote the dataset of the I-th task, where of denotes an input sample and y! denotes its label. N₂ denotes the number of samples for task 1. The total number of tasks is denoted by T. When the model learns new task, the model can obtain limited or no data from the old tasks, potentially leading to catastrophic forgetting. In this work, we consider a challenging continual learning problem called task-agnostic problem, in which task identities are not available in the inference stage. In the task-agnostic problem, when the model learns on a new task t, some new classes are presented to the model. The model must possess the abilities to maintain performance on old classes (stability) and adapt itself to learn new classes (plasticity).
Geplay-based methods 7 maintain a memory buffer M with limited size to store a small portion of old samples. When receiving a mini-batch of new samples B, from a new task I, the model retrieves a mini-batch of samples BM from M and replays them with the new samples B, to achieve a trade-off between stability and plasticity. The losses used most existing replay-based methods can be written as follows:
B2
Bml Σ
C -=
1 B
Σ
Onew Lnew (Se(
+
1
(e)
BM
Here, fnew is the loss for the new task and is mainly for the plasticity of the model. In contrast, Crep is the replay loss and is mainly for the stability of the model. We follow most existing works 77 and assume that Lnew is a cross-entropy loss. Grep usually varies for different methods. For example, Crep can be a cross-entropy loss B [9] or a combination of cross-entropy and regularization losses 7 5 36). Based on Figure (a), we can find that Lnew is not only related to new/old class distinction, but also related to new class distinction in the task-agnostic problem. Hence, existing methods with the loss in (I) mix the two different learning objectives (new/old class distinction and new class distinction) together.
Note that there exists type of replay-based methods 43 data from D, U M, which makes their losses can not be written the form shown in (L. In some challenging continual learning settings like online continual learning setting, the model cannot access
14
38
directly sampling training
3

---

Accuracy of Fiest Task
1.0
Accuracy of the Fiest Task
0.75
Ranove Gd
100
0.8
0.50
Remove G6
Froquency
649
Proquency
100
0.6
0.25
50
50-
Ranove (a(s(a)
0.4
1,69
0.00
10
10
Number Tasks (a) Scq-
Number Tasks (b) Soq-CIFARI00
Loss Value (c) ScQ-CIFARI0
Loss Value (d) Soq-CIFARI00
Figure 2: (a) and (b) show the variation of the first task's s accuracy on different datasets. (c) and (d) show the distribution of different losses over different datasets before learning the second task. Here, the frequency corresponding to each loss value represents the number of samples with this loss value.
the entire dataset of a new task. At this time, is impossible for the model to sample from Dr U. On the contrary, loss in (T) can be adapted online continual learning by sampling B, from the data stream S and sampling BM from M separately. Hence, in this work, we focus on the methods whose loss function has the form in (T). In Section and Appendix, we also compare our method with the methods that sample training data from D, U.M
3.2 Analyzing Learning Objectives by Decoupling Loss
When learning the t-th task, we assume there are m old classes, which are denoted as Co {1, 2, ..., mJ. We also assume there are n new classes, which are denoted as C, = {m + 1,m + 2,... m n). Then, for a given new sample (x, y) where y Cn, we can compute its loss Lnew as
exp(oy)
(new(fe(z). lce (
-log
Gi=l exp(o:)
Here, (+) denotes the cross-entropy loss. (0t, 02, O2,..., Own Om+ -.: Omt+n] denotes the logits out- puted by fe (z)). We decouple the loss Lnew (fe())), v) according to the two learning objectives in Figure (a):
exp(o,)
Σ Dim+t exp(o:) Enta exp(o:)
Lnew (fe(()))))))))) log
log
(3)
Enth 2i=m+1 exp(o:)
=lco(Se()) C. (n())())).
Here, we use len(;; Cn) to denote the cross-entropy loss restricted to new classes C., Obviously, lox (f))())),)))))))))))))) y; C.,) is related new class distinction. (ar)) is related to new/old class distinction. Note that both Lee (fe yx. C,) and (z)) are for the plasticity of the model and may cause catastrophic forgetting. Furthermore, loss tce (f)( y; C,) and loss (f)( have the same weight in ① due to the coupling property.
We use experience replay (ER) which is one of the most popular replay- based methods and can be expressed in the form of (1. to evaluate the impact of (-; C.) and e, (-). Specifically, we first let the model learn on the first task through valina stochastic gradient descent (2I) Then, before learning the subsequent tasks, we remove one of the two losses in (3) from ER and analyze the forgetting of the first task. The experiments are conducted on two datasets Seq-CIFAR 10 and Seq- 100, which will be introduced in Section 4 The experimental settings also follow the descriptions in
Section 4.1 Figure (a) and Figure 2 (b) show the accuracy of the first task when the model learns subsequent tasks. Here, 'Remove fee (=: Cn)" means that we remove lee C,) from I after learning the first task. 'Remove (.,())' means that we remove (-)-) from (I) after learning the first task. The results show that removing (-) results in less forgetting of the first task than removing tae (- C.)C.
In other words, (.)()) leads to more forgetting than (:.
It is intuitively reasonable that (.). leads to more forgetting than (:: First, since based methods usually keep limited samples in memory, when the model learns a new task, it has access to much fewer samples from the old classes than from the new classes. Therefore, utilizing loss (.) to learn to distinguish between new classes and old classes introduces a risk of biasing the model towards the new classes, potentially leading to serious catastrophic forgetting. In contrast, loss

---

Combining LODE with ESMER A recent method called error sensitive modulation experience replay (ESMER) (36] suggests that the model should learn more from smaller losses to avoid large feature drift. The loss in ESMER is slightly different from the loss in [ Specifically, ESMER assigns different weights to different new samples in B, according to their loss values. Although (+e(::Cr is independent of the old classes, thereby avoiding introducing a risk of biasing the model towards the new classes. Second, in (c) and (dg, we also show the value of the losses (:: Cn) and (.) before using ER to learn task 2. We can find that the value of e, (.) is larger than the value of (=e(::C, on both Seq-CIFARTI 10 and Seq-CIFAR 100. This finding consistent with existing works 36]. which suggest that larger losses for the new task may result in larger feature drift, leading to more forgetting. In Appendix, we also provide the TSNE visualizations when either (-) or Lee (:; Cn) is removed, further confirming that (-) leads to more forgetting than bee C.).
Since (-) leads to more forgetting than (+ Cx), treating these two losses equally is not reasonable. In particular, based on the above analysis, we can find that good continual learner should assign larger weight to (=e(:;Cn) and smaller weight to e (-). However, loss in (I) fails to achieve this goal due to the coupling property.
3.3 Loss Decoupling for Continual Learning
Section 3.2 has demonstrated the impact of different learning objectives on the model's 's forgetting and the issue of the coupling property in existing replay- based methods. To address this issue, propose a new method called loss decoupling (LODE), which removes the coupling property present in existing methods.
Specifically, our LODE uses the following loss to perform continual learning:
Bxd
1
C
(Rlcal + B26, (2en (Se(()))))))))))))))))))))))(( (Fo())))
Crep(Se (2(") y.
B.
1BM
(4)
Here, 31 and B2 are two coefficients that control the weight of the two different learning objectives. The finding in Section shows that LODE should set f32 to be smaller than 31, to make the model achieve better trade-off between stability and plasticity than existing methods usually with coupled [eal
loss. Furthermore, since (.) is for new/old class distinction, LODE sets 82 proportional to the
make the model not bias toward old or new classes. In contrast, since (= is only related to the new classes, LODE sets 31 to constant value. More specifically, LODE sets B1 and B2 as
ratio [Col
|cnl B1 c, 82 P2= Ic.
(5)
Here, C and p are two hyperparameters. Note that when the number of tasks increases, the number of old classes also increases. particular, when the number of old tasks large, the number of old classes |C.| is usually much larger than the number of new classes [C,]] At this time, 32 is much smaller than 3r. Setting 32 to be as large as 31. or setting 31 to be as small as 32 fails to make the model achieve good trade-off between stability and plasticity, which will be verified in the experiment.
Note that we do not specify the form of Crep in (4) Therefore, our method can be combined with many replay-based methods with form (I) and improve these methods. Here, we give some examples which combine LODE with different state-of-the-ae continual learning methods.
Combining LODE with ER and DER++ Both experience replay (ER) 9 and dark experience replay++ (DER++) 77 can be written in the form of (1). Therefore, the combinations of our LODE with these two methods are direct. Specifically, given a new batch of samples B,, the model compute lon (:: C,)) and (.) according to (3). Then. the model computes Lrep with the old samples B BM through the specific formulation in ER or DER++ Finally, the model can get the loss (4).
the loss in ESMER is slightly different from the loss in T.. still mixes the two different learning objectives together. Therefore, we can combine ESMER with our LODE through similar form
5

---

Algorithm Loss Decoupling (LODE) for Continual Learning
1: Input: sequence of tasks with datasets (Di,..., Dr J. neural network model fe
2: Output: learned neural network model (-).
3: while Get a mini-bate mini-batch of samples B, from task do
4: Sample a mini-batch By from memory M:
5: Specify the weights for the two different learning objectives by
6: Get the losses for learning objective through (3)
7: Compute the final loss through
8: Perform backward propagation and update the model (e(+) through SGD;
9: Update memory MM with B, through some memory update methods;
10: end while
The decoupled loss for ESMER can be written as follows:
BMl Σ f=1
my (fll (
31
B
BM
Here, my is the weight assigned to sample i in ESMER. We find that the loss in (6) not only modulates weights for different new samples but also modulates weights for the two different learning objectives.
Algorithm gives the whole learning process of LODE.
3.4 Relation with Existing Methods
The loss in many existing methods has the coupling property like the loss in T However, some methods, from the perspective of this work, use the losses that show a certain degree of decoupling be written as
property. For example, the loss ER-ACE
B
BMl Σ
1
C
Loo (ela))))
(fe(x;)) M)
Br
[BML
can be seen from In particular, loss in by setting 31 = and 82 = in 4 SSIL [ has a similar form to that 7 but uses a different frep compared to ER-ACE. Due to the lack of (.), the loss in (7) can only leverage the new classes' samples kept in memory to learn the objective of new/old class distinction. In experiments, we will show that setting 32 in (4) performs worse than setting 33 + 0 in 44.
that this method removes the term (-) from and only retains lae (:; C.)) is special case of the loss in (4). More specifically, we can get the loss in (
Some existing methods also incorporate the idea of separating objectives continual learning. However, these methods primarily designed for task-aware problem. For instance, bilevel memory system with knowledge projection (BMKP) [37] requires the task identities to choose the corresponding knowledge representations during testing, making it unsuitable in the task-agnostic problem. Space decoupling (SD) [46) does not explicitly mention that it only considers task-aware problem, but its experiments completely follow some task-aware methods [35 26), indicating its focus on task-aware problem.
4 Experiments
4.1 Experimental Settings
Datasets We use three popular datasets for evaluation. including Beq-CIFAR1O B], Seq- CIFAR100 [10), and Seq-TinylmegeNet (22). Seq-CIFAR10 consists of 5 disjoint tasks with each task having classes and 10k training samples. Seq-CIFAR 100 consists of 5 disjoint tasks with each task having 20 classes and 10k training samples. Seq-TinylmageNe consists of 10 disjoint tasks with each task having 20 classes and 10k training samples. The statistics of different datasets are given in Appendix. All the experiments are for the task-agnostic problem.

---

Table 1: Classification results which are averaged across runs.
Keeping
Seq-CIFARIO
CTF-COO
Seq-TinyImageNVe
Extra Model
joint finetune
91 86
70.10 1020.0
59.82
no
19.65
17.41 10.00
8.132004
Buffer Size
500
5120
500
5120
500
5120
SCR 27
1.595
82,47=0.44
23.06=am2
45.022067
8.37:
18.20
PCR 25
34.24932
82.58=0.42
28.38+4046
52.51+161
11.88+101
26.39+1.44
MIR
63.93
63.73=0097
27.80+0.z
53.73
11.22++0.
30.60+0.00
ER-ACE
68.45
83.49=00.00
40.67
58.564491
17.73#
37.99+a.17
ER 9
61.78
83.64=0095
27.69+0..8
53.86440.7
10.36+ta11
27.544030
LODE (ER)
68, 17.877:5 10.71
83, 7..40.888
41.52:11.22
48.59
17.77:1008
38.341001
DER+ 7
19.99
85, 66..1610.14
12.08
62.73.1058
19.281001
39.7210.7
LODE (DER++)
75.45 1..90
85.78
46.31
64.00
21.15
40.31=0.03
CLS-ER [5]
70.73=054
85.73
51.21
60.17
29.41+166
45.66+047
TAMiL
74.25
84.82
50.62.023
63.7740.3
27.83
43 43.00
iCaRL (33
61.60
72.01
4995
54.23
20.01
30.34
yes
BIC 43
52.63+2246
79.98
37.064000
60.43
29.82
37 37.604023
SSIL
64.31
71.72:1.47
41.61
57.531052
16.801a71
40.0610.58
ESMER 36
71.48200.8
79.19=0.6
52.37:087
63.994011
30.97:111
44.07:0052
LODE (ESMER)
74.53
85.34=0.41
55.06
65.69
32.15
46.40_a.46
Baselines We compare our method with many state-of-the-art replay-based continual learning methods, including incremental classifier and representation learning (iCaRL) [33], bias correc- tion (BIC) [43) separated softmax for incremental learning (SSIL) [1], experience replay (ER) [9],
maximally interfere retrieval (MIR) R dark experience replay++ (DER++) 77 supervised con- trastive replay (SCR) (27). proxy-based contrastive replay (PCR) (25). experience replay with asymmetric cross entropy (ER-ACE) 88, error sensitive modulating experience replay (ESMER) (36),
complementary learning system experience replay (CLS-ER) [5]. and task lask-specific attention modules in lifelong learning (TAMiL) 6 For CLS-ER. we follow the existing method [6] and implement with single exponential moving average model. We also include two methods without continual learning, joint and finetune, in the comparison. joint joint denotes the method which learns all the tasks jointly while finetune denotes the method which learns all the tasks sequentially without any memory. The accuracy of joint can be treated as the accuracy upper-bound and the accuracy of finetune can be treated as the accuracy lower-bound. Among the methods we mentioned above, some methods only maintain a single learning model to perform continual learning, while others require an extra model in memory for knowledge integration or distillation. Since keeping more models requires more memory, and memory cost is an important metric in continual learning [30], we group different methods by whether keeping extra model (refer Table I to make more fair comparison.
Architecture and Training Details We follow existing continual learning works 7 6 and use standard & as the neural network architecture in all the experiments unless otherwise stated. The experiments are built on top of the mammoth 7 continual learning repository in PyTorch like many existing works 7 (. We stochastic gradient descent (SGD) optimize the parameters. The batch size and replay size are set to 32 to follow the existing continual learning works [7] []. We also follow existing methods 7 55) set memory 500 and 5120 for all the datasets. The hyperparameters are selected through a small validation set. For the experiments of all the methods on all the datasets, we apply random crops and horizontal flips to both newly coming samples and buffered (saved) samples like existing works 7 5). For each of our experiments, we report the average and standard deviation of the mean test accuracy of all the tasks across 5 runs with different seeds. More details of training and hyperparameters for different methods are given in Appendix.
4.2
Experimental Results
4.2.1 Accuracy
Table shows the results of different methods on different datasets. Here, 'Kepping Extra Model' represents whether method needs to keep an extra model for continual learning, described
7

---

100
100
ER. ACI
ER. ACE ESMER
80
DER
80
DER
Accuracy (%)
Accumey (%)
Accuracy (%)
Accuracy (%)
90
FSMER
80
LO0K (RAMER)
LO0K
LOOE (ESMER))
USSMER
60
60
80
60-
40
70
40
7
10
Number Tasks
Number Tasks (b) Soq-CIFARI00
Number of Tasks (c) 4 Tasks in Total
Number of Tasks
(a) Scq-
(d) 10 Tasks in Total
Figure 3: (a) and (b) show the variation of the accuracy for different methods on Seq- CIFARIO and S 100. (c) and (d) show the variation of accuracy on Seq-CIFARIO0 100 with different number of tasks.
76
55
55
74
50-
50
Accurucy
Accurccy
Accurucy
72
Accuricy
72
45
LODE (NFR+)))
45
LODE (NFR+)))
20
LODE (DER)1)
LODE (NFR)))
40
LODE (LSMLL)
LODE (USMER)
LODE (LSMUR)
LODE (USMER)
40
0.0
(a) Seq- CIFARIO
(b) CIF-CIF
(c) Seq. CIFARIO
(d) Seq. CIFARIOO
Figure 4: (a) and (b) show the variation of the accuracy for different p. (c) and (d) show the variation of the accuracy for different C.
in Section 4.1 We combine our LODE with ER, DER++ and ESMER, which can be written in the form of D. We use LODE (ER), LODE (DER++) and LODE (ESMER) to denote them, respectively. Our experimental results confirm that the integration of our LODE method improves the performance of each combined method. For example, when compared to DER++ with buffer size 500, LODE (DER++) exhibits a 2. 16% improvement on Seq-CIFAR1O, 4. 4.23% improvement on Seq-CIFAR 100 and a 1.87% improvement on SoqTTimIImageNett Notably, LODE (DER++) achieves the best performance on Seq-CIFARIO. LODE (ESMER) achieves the best performance on Seq-CIFARI00 100 and Soy-TinylmageNet.
Figure 3 (a) and Figure 3 (b) show the variation of accuracy on Req-CIFAR1O and Seq-CIFARIO0
after the learning of each task. As we can see, LODE improves DER++ and ESMER at the end of each task. 3igure and Figure show the results on Aoq-CIFFRIOO with different numbers of tasks. When there are tasks Seq- Seq-CIFAR100 100, each task consists of 25 exclusive classes. Similarly, when there are 10 tasks in Seq-CIFAR 100, each task consists of 10 exclusive classes. As we can see, when the number of tasks varies, LODE still gives improvements on different methods. Appendix, we show the results of more methods Seq- CAn-CIFAROOO 100 with different numbers of tasks.
4.2.2 Hyperparameter Analysis
We analyze the hyperparameters in LODE. We choose LODE (DER++) and LODE (ESMER) to analyze hyperparameters, as they represent the best performance of methods that retain or do not retain an extra model for continual learning, respectively.
We first vary the value of p in 5 to show its impact on the performance of the model. Figure 4 (a) and Figure 4 (b) give the analysis 32 = 0 and the weight of (.) in (4) is always zero. At this time, the loss in (4) degenerates to the loss in 7 and the model fails to get satisfactory performance. When the value of increases, 33 also increases and the performance of the model first increases and then decreases. This phenomenon is intuitively reasonable since larger weight for (-)( leads more forgetting and thus influences the overall model performance.
CIFACIO and C00 Note that when p p= 0.
We also vary the value of C' in 5 to show its impact on the performance of the model. Figure 4 (c) and Figure (d) show the results varying on Seq-CIFAR10 and Seq-CIFAR 100, respectively. We can find that the performance of the model decreases significantly when C' is close to zero. In
8

---

Table 2: Ablation study on Seq-CIFARI0 and BIq-CIIAR
LODE (DER++)
LODE (ESMER)
Beq-CIPARRIG
Seq-CIFAR100
Seq-CIFAR10
Soq-CIFAR
81 C, f22
Cal (Ours)
75,45
46.31
74.53+0u5
55.06
= P Cn.
71.18:am
37.49=1.79
73.41
45.642087
col
81 = 32 C
73.80
12.08
73.08
52.37=007
B1 P T.
Co
82 = C
73. 03.19
40.79
72.38
51.86:10.35
Table 3: Classification results which are averaged across 5 runs in the online continual learning setting.
Keeping Extra Model
Seq-CIFAR10
Seq-CIFARI00
Seq-IinyImageNet
SCR 27
69.4913302
36.091082
20.04:1.24
PCR
13.28
34.89
23.84100.0
ER-ACE 8
69. 19.1711644
35.24
23.42
MIR 3
71.10
35.08
20.64+1.17
no
ER 9
67.93
34.40
21.144a.
LODE (ER)
69.63
36.91
24.31
DER+
72.30
34.72+151
20.40+1.0
LODE (DER++)
74.00=0.0
37.82+116
25,30 -180 1,80
particular, the model gets the best performance when the value C' is between 1 and 5. Since and C' are highly different when the model achieves the best performance, decoupling the loss through 3 is necessary for continual learning.
4.2.3 Ablation Study
We change the value of 33 and 32 to show the effectiveness of setting 31 and Be through (5). We first set the value of
82 in (4) to remove the decoupling property. There are two possibilities to set 3 = 32. The first possibility is to set 31 B2 = C' and the second possibility is to set possibilities, which significantly inferior our method. This means that separating the different objectives by decoupling the loss of the new task is necessary for the model to achieve good performance. In Table we also show the result of variant by exchanging the value of 31 and 82g which means $1 p |cs. and B3 = C. We can find that the performance of this variant still significantly inferior our method.
81 B2
C.. c. T shows the results of these
4.2.4 Online Continual Learning
We also perform experiment for the online continual learning setting [ where the datasets of different tasks are concatenated to non-stationary data stream. Since online continual learning is more challenging than offline continual learning, existing methods [I] usually use larger memory in online continual learning, especially for challenging datasets such Seq-CIFAR 100 and Seq- TinyImageNet. Hence, we set the memory size to 5120 for the datasets. To follow existing online continual learning methods 3 [0], the experimental settings we use here are different from those introduced in Section 4.1 These settings are introduced in Appendix.
Table 3 shows the results of different methods. We exclude those methods that have been imple- mented only in offline continual learning or those that have demonstrated lower performance in online continual learning in previous works g 23 Similar to that in the offline continual learning setting, we can find that the integration of our LODE method also significantly improves the overall accuracy of each combined method. For example, when compared with DER++, LODE (DER++) exhibits 1.70% improvement on Seq-CIFAR10, 3.1 1% improvement on Seq-CIFAR 100 and 4.9% improvement Seq-TTiy TinyImageNet.

---

Conclusion
In this work, we propose new method called loss decoupling (LODE) for continual learning. Different from most existing replay-based methods which mixes the two different learning objectives together to learn the new task, LODE separates the two learning objectives for the new task by decoupling the loss of the new task. Experiments show that LODE can achieve a better trade-off between stability and plasticity than other baselines, and thus outperform other state-of- the-art replay- based methods across multiple datasets. Future work will extend LODE to other continual learning problem like aasre problem and study the effectiveness LODE for other types of tasks.
Acknowledgment
This work is supported by NSFC Project (No.62192783) National Key R&D Program of China (No. 200007719 and Fundamental Research Funds for the Central Universi- ties (No.020214380109).
References
[1] Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. SS- IL: separated softmax for incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 824-833, 2021.
[2] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, pages 139-154, 2018.
[3] Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, and Tinne Tuytelaars. Online continual learning with maximally interfered retrieval. arXiv preprint arXiv: 119900804744 2019.
[4] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems, pages 11816-11823 2019.
[5] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general con- tinual learning method based on complementary learning system. In International Conference on Learning Representations, 2022.
[6] Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. Task-aware information routing from common representation space in lifelong learning. In International Conference on Learning Representations, 2023.
[7] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in Neural Information Processing Systems, pages 15920-15930, 2020.
[8] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference Learning Representations, 2022.
[9] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint 1rXIv:1902.10486 2019.
[10] Aristotelis Chrysakis and Marie- Francine Moens. Online continual learning from imbalanced data. In Proceedings of the International Conference on Machine Learning, pages 1952-1961, 2020.
[11] Aristotelis Chrysakis and Marie-Francin Moens. Online bias correction for task-free continual learning, In International Conference Learning Representations, 2023.
10

---

[12] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Puttern Analysis and Machine Intelligence, pages 33366-385, 2021.
[13] Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, and Pheng-Ann Heng. Flattening sharpness for dynamic gradient projection memory benefits continual learning. Advances in Neural Information Processing Systems, pages 18710-18721 2021.
[14] Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Proceedings of the European Conference on Computer Vision, pages 86-102, 2020.
[15] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey, International Journal of Computer Vision, pages 1789-1819, 2021.
[16] Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems, pages 11588-1 1598,, 2020.
[17] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 831-839, 2019.
[18] Steven C. Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien Hhnn Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems, pages 13647-13657, 2019.
[19] Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. caadient-based editing of memory examples for online task-free continual learning. Advances in Neural Information Processing Systems, pages 29 29193-29205, 2021.
[20] Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with node-importance based adaptive group sparse regularization. Advances in Neural Information Processing Systems, pages 3647-3658, 2020.
[21] Nikhil Ketkar and Nikhil Ketkar. Stochastic gradient descent. Deep learning with Python: A Hands-on Introduction, pages 113-132, 2017.
[22] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, page 2015.
[23] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In Proceedings of the International Conference on Machine Learning, pages 9925-3934 2019.
[24] Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7816-7825, 2023.
[25] Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, and Yunming Ye. Per: Proxy-based con- trastive replay for online class-ineremental continual learning. arXiv preprint 300::2304044408. 2023.
[26] Sen Lin, Yang. Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual learning. In International Conference on Learning Representations, 2022.
[27] Zheda Mai, Ruiwen Li. Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings of the JEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 3599, 2021.
[28] Marc Masana, Joost Van Weijer, Bartlomiej Twardowski, al. On the importance of cross-task features for class-incrementaà learning. arXiv preprint arXiv:2106.11930 2021.
11

---

[29] Sudhanshu Mittal, Silvio Galesso, and Thomas Brox. Essentials for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 3513-3522, 2021.
[30] Martin Mundt, Steven Lang, Quentin Delfosse, and Kristian Kersting, Cleva-compass: A contin- ual learning evaluation assessment compass promote research transparency and comparability. In International Conference on Learning Representations, 2022.
[31] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, pages 54-71, 2019.
[32] Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 33588-135999 2020.
[33] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5533-5542, 2017.
[34] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 0667,0467, 2016.
[35] Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference on Learning Representations, 2021.
[36] Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. Error sensitivity modulation based experience replay: Mitigating abrupt representation drift in continual learning. In International Conference on Learning Representations, 2023.
[37] Wenju Sun, Qingyong Li, Jing Zhang, Wen Wang, and Yangli-ao Geng. Decoupling learning and remembering: A bilevel memory framework with knowledge projection for maskeincremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20186-20195, 2023.
[38] Fu- Yun Wang, De- Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for incrementlltttlll learning. In Proceedings of the European Conference on Computer Vision, pages 398-414, 2022.
[39] Liyun Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00489 2023.
[40] Zhen Wang, Liu Liu, Yiqun Duan, and Dacheng Tao. Continual learning through retrieval and imagination. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2022.
[41] Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tiehang Duan, and Mingchen Gao. Improving frsk-free continual learning by distributionally robust memory evolution. In Proceedings of the International Conference on Machine Learning, pages 22985-22998 2022.
[42] Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay GANs: learning to generate images from new categories without forgetting. In Advances in Neural Information Processing Systems, pages 5966-5976, 2018.
[43] Yue WU, Yimpeng Chen, Lijuan Wang. Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 374-382, 2019.
[44] Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3014-3022 3023, 2021.
[45] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the International Conference on Machine Learning, pages 3987-3995, 2017.
12

---

[46] Zhen Zhao. Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1718-3727, 2023.
[47] Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng- Lin Liu. Prototype augmentation and self-supervision for incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5871-5880, 2021.
13

---

A Statistics of Different Continual Learning Datasets
We give detailed statistics on datasets in this section. Specifically, Table shows the detailed infor- mation of three datasets, including Seq-CIFAR I0, Seq-CIFAR 100, and Beq-Tinylmaaell These datasets are constructed by one of the following datasets: CIFARIO, 10, CIFARIO0 or TinylmageNet. For each task, 5% of the training samples divided into validation set.
Table 4: Statistics three datasets
Seq. CIFARIO Seq-CIFARI00
Seq- TmayemaeeNet
Task Number
5
5
10
Input Size
3
x 32 x 32
3 x 32 32
3 x 64 x 64
Classes per Task
20
20
Training Samples
10000
10000
10000
Testing Samples
1000
1000
1000
B More Experimental Details
B.1 Hyperparameters
Table provides the hyperparameters for different methods, which will be used for comparison. p is selected among four different values [0.01, 0.05, 0.1, 0.2] through the validation sets we split from the training sets. We do not carefully select hyperparameter but set as for all the experiments to reduce the reliance of the model hyperparameters.
is important to note that both DER+ and ESMER have their own specific hyperparameters that may affect the performance of the model. To ensure fair comparison, we maintain the specific hyperparameters of each method consistent when combined with our LODE, except for exponential decay hyperparameter in ESMER. This means that we use the same hyperparameters settings as the original implementation when evaluating our combined models. For the exponential decay hyperparameter in ESMER, find that setting it the same value ESMER (0.999) makes the model too conservative to learn the new task in LODE (ESMER). Therefore, we set to 0.998 in order to make the model not conservative. Furthermore, follow the mammoth continual learning repository (7] to set the specific hyperparameters in each method or get them by the validation sets split from the training sets.
Table 5: List of hyperparameters for different methods
Split-CIFARI0 Split-CIFAR100
Split- Oplit-Tiny ImageNee
Methods
Buffer
C
P
C
LODE (ER)
500
0.1
0.01
0.05
5120
0.1
0.05
1
0.05
LODE (DER++)
500
0.05
0.1
5120
0.1
0.2
1
0.1
LODE (ESMER)
500
0.05
0.05
5120
0.1
0.2
0.1
The learning rate is determined by grid-search. Specifically, the open-source code of the existing work DER 7 has conducted grid-search determine hyperparameters, including the learning rate, for wide range of baselines. Since our experimental setting follows this work, the hyperparameters, including the learning rate, are kept consistent with the code of this work. For the baselines that are not included in the code of DER but follow the experimental settings of DER, we follow their open- source code to set the hyperparameters, such as learning rate and epoch, which are also determined by grid-search. For the methods not following the experimental setting of DER, we conduct grid-search on validation datasets determine the hyperparameters, including the learning rate. When combining our LODE with other methods, we keep the learning rate consistent with the original methods.
14

---

Table 6: Variation of the first task's accuracy SeQ-CIFA
After Task
After Task 2 After Task 3 After Task 4
After Task 5
Remove (:e(: (:)C)
27.40 87.40=0.2
54.284341
56. 48
50.77 54.1820.8
43.922087 48.50
Remove (-)
82.38
63.27
Table 7: Results of combining with other methods.
Seq- CIFARIOROO
PASS 47
47 37.45
LODE (PASS)
20.49
B.2 Experimental Setting in Online Continual Learning
Datasets We use three popular datasets for evaluation, including Seq-CIFARI0 (3], Seq- CIFARI00 10,, and Seq- Beq-Tinylmmageel [22]. We follow existing online continual learning methods to partition tasks across different datasets for the purpose of evaluation. Specifically, Seq-CIFAR 10 con- sists of 5 disjoint tasks with each task having classes and 10k training samples. Beq-CIFARR1O 100 con- sists of 20 disjoint tasks with each task having classes and 2.5k training samples. TinyTmagggee consists of 20 disjoint tasks with each task having 10 classes and 5k training samples. All the experi- ments are for the task-agnostic problem.
Reduced-RRssNetIB our model architecture, which has been commonly used in prior online continual learning methods 8 41. Reduced--ReNNet1 88 is variant of the original ResNet18 that has fewer channel sizes, and fewer filters overall. We use stochastic gradient descent (SGD) to update the model for all the methods. We set memory sizes M=5120 for all the datasets. Like the existing online continual learning works [8] [] the replay size [M is the same as the mini-batch size [Bel (fixed to 10). Data augmentation is the same as that in offline continual learning. Specifically, we use data augmentation such as random cropping and random rotation for both new mini-batch B, and old mini-batch ByM
Training Details For all the datasets,
C More Experimental Results
C.1 Combine with More Methods
LODE can be applied in exemplar-free setting. We apply our LODE to a popular exemplar-free continual learning method called prototype augmentation and self-supervision (PASS) [47], which is designed for task-agnostic problems and utilizes cross-entropy loss for learning new tasks. Table 6 shows the variation of the accuracy for the first task during the learning of subsequent tasks. We can find that removing (-) makes the model suffer from less forgetting than removing toe (=;C,). Table shows the final results of PASS and LODE (PASS). We can find that LODE (PASS) outperforms PASS on Seq-CIFARE CIFAR100.
C.2 TSNE Visualization
We present the TSNE visualization in Figure 5 which shows the learned representations of different models on Beq-CIFAR1O As we can see, removing loss (.(()) makes the representation of the old classes overlap less than removing loss (-: C.) Therefore, removing loss (a(-) makes the model suffer from less forgetting than removing loss lex C,). We will add this figure the final version of our paper. Thanks for the suggestion.
C.3 Plasticity of the Model
We also add graph in Figure 6 which shows the accuracy each new task during the training of Seq-CIFAR10 10. As we can see, removing either loss (: or loss (-) decreases the model's plasticity and leads to lower performance on the new task. Hence, both of these two losses are necessary for the learning of new task.
15

---

(a) TSNE visualization of removing (: (e((;Cn) C.) after learning task
(b) TSNE visualization of removing (x(-) after learning task
(c) TSNE visualization of removing (=r(:; after learning task
(d) TSNE visualization of removing (.(-) after learning task
Figure 5: TSNE visualization
0.75
Accuracy
Accurazy
0.8
0.50
Jamon fall S0
Ramove (al
Remon Ief)
0:25
Remoo fal?
Keep
Kepp (4b
0.6
3 Number of Tasks (b) Seq- CIFARIOOOOOO
Number of Tacles (a) Seq CIFARIIFROOOOOOOO
Figure 6: (a) and (b) show the variation of the new task's 's accuracy on different datasets.
C.4 More Comparison
Table presents the results of different methods on Seq-CIFARI00 with varying numbers of tasks. Here, the memory size is set as 500 for all the methods. As we can see, when the number of tasks varies, LODE still gives improvements on different methods. Furthermore, LODE (ESMER) still gets the best performance among all the methods.
9 gives the results of additional methods that directly sample samples from D, U.M. Some methods like BIC and iCaRL, which directly sample samples from D, U, have been compared with our methods in the main text. Since all these methods keep an extra model in memory for distillation, compare them with LODE (ESMER). As see, method still gets the best performance.
the First Tak
Accunac of Fiest Tase
150
((CC)
Frequeccy
Frequeiey
0.6-
100
04
Romova (e(eC))
Remake Coi
Hemovee 6.23
Q2
Kemove (.f))
0-
Number of Tocks (a) Seq-CIFAR10
Number of Tasks (b) Seq-CIFARR10
Lass Value (c) CIF-CIO
Loss Value (d) SSQ-CIFAR100
Figure 7: (a) and (b) show the variation of the first task's accuracy on different datasets. (c) and (d) show the distribution of different losses over different datasets before learning the second task. Here, the frequency corresponding each loss value represents the number of samples with this loss value.
C.5 More Analysis by Decoupling Loss
We also use dark experience replay++ (DER++) 7 evaluate the impact of toe Cx)) and (-).). Figure ( the results. The results are similar to that in ER. Specifically, removing (.(.) results in less forgetting of the first task than removing Loe C:.). In other words, (.(() leads more forgetting than (:: ). Furthermore, the value of (.) is larger than the value of Lee (:; C.,) on both Beq-CIFAR10 and Seq- S.F-CIFAR000.
16

---

Table 8: Results CIFAR100 with different numbers of tasks. The memory size is set 500.
Keeping
10
Extra Model
PCR
251
30.9
29.8
MIR 3
30.44+085
22.01+0a.
ER-ACE 8
40.71
36.95=088
no
ER 9
30.72
20.45
LODE (ER)
45.60+
37. =7774
DER++ 7
43. 1:068
36.30
LODE (DER++)
48.95
38.61
CLS-ER
54.82
43. $3.05....66
TAMiL 6
42.52 23.42
iCaRL 3
50.97+1.04
42.25=0022
yes
BIC 43
40.75++0.7
20.71
SSIL
39.39:10.6
33.06-008
ESMER 36
55.27
45.54
LODE (ESMER)
57.92
46.37+
Table Results of more methods on Seq-CIFAR100 and Teg-Tinnlmageeet with memory size 500.
Seq-CIFAR100 Seq-TinylmageNet
iCaRL 33
49.59
20.01
BIC
4R
37.064060
29.82=0.8
SSIL
41.61
16.80=a.1
PODNet
45.954044
26.10=171
UCIR 17
47.56400.3
26.22
26,
LODE (ESMER)
55.06
32.15
17


Reviews:
[{"review_id": "GvToMnteRZ", "reviewer": "Reviewer_Y5HQ", "summary": "This paper investigated class incremental continual learning (CL). By showing that two objectives, i.e., classifying among new classes and classifying between new classes and old classes, can have different impacts on the performance of CL, this paper proposed to decouple the new task loss and assign different weights to these two objectives. Experiments are conducted on multiple datasets to verify the effectiveness of the proposed method LODE in comparison with related baseline methods.", "questions": "1. How did you select the values of $C$ and $\\rho$ in (5)?\n\n2. Conceptually, the loss decoupling can be used in CL without using data replay. In this way, wouldn't it be more clear to understand the impact of these two parts on the performance of CL?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "It is interesting to investigate the different impacts inter-task and intra-task classification on the performance of class incremental CL.", "weaknesses": "1. The results in Figure 2(a) seems very counter-intuitive. When removing $l_n$, the training of the current task only focuses on the classification within the task and ignores the differentiation from old classes in the memory. In this case, the forgetting should be larger compared to the case of removing $l_{ce}$, while in Figure 2(a) the forgetting was smaller. The authors didn't give a convincing explanation of the underlying reason for the phenomenon.\n\n 2. The weight selection (5) in LODE is entirely based on heuristics and manually determined. It is hard to justify the selection of the weights, but the performance of LODE is highly sensitive to the weight selection as shown in Table 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The results in Figure 2(a) seem very counter-intuitive. The authors didn't give a convincing explanation of the underlying reason for the phenomenon.**\n\n**A1**: The results in Figure 2 (a) of the paper are intuitively reasonable. First, since replay-based methods usually keep limited samples in memory, when the model learns a new task, it has access to much fewer samples from the old classes than from the new classes. Therefore, utilizing loss $l_n(\\cdot)$ to learn to distinguish between new classes and old classes introduces a risk of biasing the model towards the new classes, potentially leading to serious catastrophic forgetting [1]. In contrast, loss $l_{ce}(\\cdot;C_n)$ is independent of the old classes, thereby avoiding introducing a risk of biasing the model towards the new classes. Second, as shown in Figure 2 (c) of the paper, the value of loss $l_n(\\cdot)$ is typically larger than the value of $l_{ce}(\\cdot;C_n)$ during training. This finding is consistent with existing work [3], which suggests that larger loss for the new task may result in larger feature drift, leading to more forgetting. Based on these two reasons, it is intuitively reasonable that loss $l_n(\\cdot)$ causes larger forgetting for the old classes than loss $l_{ce}(\\cdot;C_n)$, implying that removing loss $l_n(\\cdot)$ results in less forgetting than removing loss $l_{ce}(\\cdot;C_n)$. Thanks for the information. We will add this explanation in the final version of our paper.\n\n\n**Q2: The weight selection (5) in LODE is entirely based on heuristics and manually determined. It is hard to justify the selection of the weights, but the performance of LODE is highly sensitive to the weight selection as shown in Table 2.**\n\n**A2**: Actually, the weight selection (5) in LODE is not based on heuristics and manually determined. We give the explanation for Equation (5) in Section 3.3. First, loss $l_n(\\cdot)$ aims to distinguish new classes from the old classes and the replay loss $l_{rep}(\\cdot)$ in Equation (4) keeps the performance on the old classes. Minimizing losses $l_n(\\cdot)$ and $l_{rep}(\\cdot)$ together makes the model learn to distinguish whether a given sample belongs to the new classes or the old classes, which is a binary classification task. If we set both the weights of $l_n(\\cdot)$ and $l_{rep}(\\cdot)$ as constants during the whole learning process, the model may suffer from large forgetting for the old classes. More specifically, when the number of tasks is large and each new task only increases a small number of new classes, the number of old classes is much larger than the number of new classes. At this time, setting the weight of $l_n(\\cdot)$ as a constant may introduce a risk of biasing the model towards the new classes. On the contrary, setting the weight of $l_n(\\cdot)$ proportional to the ratio $\\frac{|C_n|}{|C_o|}$ removes this bias since the weight of $l_n(\\cdot)$ will be small when the number of old classes is much larger than the number of new classes. Different from $l_n(\\cdot)$, loss $l_{ce}(\\cdot;C_n)$ is independent of the old classes and thus it does not introduce a risk of biasing the model towards the new classes. Therefore, we set the weight of $l_{ce}(\\cdot;C_n)$ as a constant C to maintain the model’s plasticity. Table 2 verifies the effectiveness of this weight selection strategy. \n\n**Q3: How did you select the values of C and ρ in (5)?**\n\n**A3**: We keep the value of C as 1 for all the experiments in Table 1 and Table 3, which we find is enough for the model to perform well. We follow existing continual learning method DER [4] to select the value of ρ. Specifically, we select the value of ρ through grid-search on a validation set obtained by sampling 5% of the training set. We investigate the influence of these two hyperparameters in Figure 4.\n\n**Q4: Conceptually, the loss decoupling can be used in CL without using data replay. In this way, wouldn't it be more clear to understand the impact of these two parts on the performance of CL?**\n\n**A4**: Thanks a lot for this informative comment. Yes, the loss decoupling can be used in some CL methods without using data replay as long as these methods use a standard cross-entropy loss to learn new tasks, which is the common choice for many CL methods. We focus on replay-based methods since they are more effective for task-agnostic problem due to the replaying of old data, particularly in the online continual learning setting. Here, we choose a CL method without using data replay, called prototype augmentation and self-supervision (PASS) [2] to evaluate our methods. The following two tables show the results of PASS on Seq-CIFAR100. The results are consistent with CL methods using data replay. Specifically, the first table shows the variation of the accuracy for the first task during the learning of subsequent tasks. We can find that removing $l_n(\\cdot)$ makes the model suffer from less forgetting than removing $l_{ce}(\\cdot;C_n)$. Table 2 shows that LODE (PASS) outperforms PASS. We will add these results in the final version of our paper.\n\n||After Task 1|After Task 2|After Task 3|After Task 4|After Task 5\n|:-|:-:|:-:|:-:|:-:|:-:\n|Remove $l_{ce}(\\cdot;C_n)$|**87.40±0.22**|54.28±3.41|56.18±4.08|50.77±2.57|43.92±0.87\n|Remove $l_n(\\cdot)$|**87.40±0.22**|**82.38±0.89**|**63.27±0.75**|**54.18±0.68**|**48.50±0.95**\n\n||ACC (%)\n|:-|:-:\n|PASS|47.45±0.37\n|LODE (PASS)|**50.49±0.22**\n\n[1] Ahn H, Kwak J, Lim S, et al. Ss-il: Separated softmax for incremental learning. ICCV, 2021.\n\n[2] Zhu F, Zhang X Y, Wang C, et al. Prototype augmentation and self-supervision for incremental learning. CVPR, 2021.\n\n[3] Sarfraz F, Arani E, Zonooz B. Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning. ICLR, 2023.\n\n[4] Buzzega P, Boschini M, Porrello A, et al. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 2020."}, {"review_id": "sXKyeNVI9B", "reviewer": "Reviewer_odRp", "summary": "Several replay based continual learning methods decouple their loss into a loss based on samples from the replay to maintain performance on old tasks, and a loss based on data from the current task to learn the new task. This paper proposes to further decouple the latter loss into a component that helps the model distinguish whether a sample is in the new task or from previous tasks, and a component helps the model distinguish between just the new classes in the current task. The paper shows that decoupling this loss and giving different weight to the two components helps in maintaining performance and results in better final accuracy for Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImagenet.", "questions": "- For section 4.2.3, did you do a hyperparameter search for the values of $\\beta_1$ and $\\beta_2$ for the methods that weren’t yours?\n- The $l_n$ loss in the denominator is essentially a binary classification loss. Since you are only using positive examples to compute this loss, it increases the value of the logits of the current task and decreases the value of the logits of previous tasks for your current data. Did you try also using some data from the replay buffer for this loss?\n- For Figure 2, it might be helpful to also see on the graph a line where neither component is removed (ie both components of the decoupled loss without reweighting), and a line with the coupled loss used in traditional ER (ie Equation 2). This could tell us how much the decoupling formulation that is proposed actually corresponds to the usual coupled formulation. Another graph to add could be the accuracy on the new task being trained, allowing us to get a sense of how these losses affect plasticity.", "limitations": "The work presents a basic discussion of limitations and societal impacts. The limitations section could be further developed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes an intuitive decoupling of the loss used in many continual learning works. It shows that doing this decoupling is useful, and can improve performance by weighting the different parts of the decoupled loss differently.\n- The paper’s experiments and presentation of experiments are well done, although I do ask the authors to address some of the questions I present below. The hyperparameter sensitivity analysis is useful in showing that the method is at least somewhat robust to hyperparameter changes.", "weaknesses": "The main weaknesses are mostly related to the additional questions and experiments I ask for in the next section. There are a few experiments I’d like to see that could make the paper stronger.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: For section 4.2.3, did you do a hyperparameter search for the values of $β_1$ and $β_2$ for the methods that weren’t yours?**\n\n**A1**: In Section 4.2.3, we did not do a hyperparameter search for the values of $β_1$ and $β_2$ for the methods that weren’t ours and kept their value consistent with our method. To be more rigorous, we add the hyperparameter search for the values of $β_1$ and $β_2$ for the methods that weren’t ours and present them in the following table. The results lead to the same conclusion as that in Section 4.2.3. We will replace Table 2 with this table in the final version of our paper. Thanks for the suggestion.\n\n|  LODE (DER++)|Seq-CIFAR10|Seq-CIFAR100\n|:-|:-:|:-:\n| $β_1=C,β_2=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert}$  | **75.45±0.90**   | **46.31±1.01**  \n| $β_1=β_2=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert}$    | 71.18±0.80   | 37.49±1.79  \n| $β_1=β_2=C$                   | 73.80±0.20   | 42.08±1.71  \n| $β_1=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert},β_2=C$  | 73.19±0.15   | 40.79±0.12  \n|  **LODE (ESMER)**   | **Seq-CIFAR10** | **Seq-CIFAR100**| \n| $β_1=C,β_2=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert}$  | **74.53±0.95**   | **55.06±0.35**  |\n| $β_1=β_2=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert}$    | 73.41±0.40   |  45.64±0.87 |\n| $β_1=β_2=C$                   | 73.08±0.81   | 52.37±0.87  |\n| $β_1=ρ\\frac{\\vert C_n\\vert}{\\vert C_o\\vert},β_2=C$  | 72.38±0.24   | 51.86±0.35  |\n\n**Q2: The loss $l_n$ in the denominator is essentially a binary classification loss. Since you are only using positive examples to compute this loss, it increases the value of the logits of the current task and decreases the value of the logits of previous tasks for your current data. Did you try also using some data from the replay buffer for this loss?**\n\n**A2**: We did not try using any data from replay buffer for this loss since the replay loss $l_{rep}(\\cdot)$ has already included the cross-entropy loss $l_{ce}(\\cdot)$ computed by the samples of old classes in most replay-based methods. At this time, we can decouple the loss $l_{ce}(\\cdot)$ included in $l_{rep}(\\cdot)$ in a similar way as described in Equation (3) of the paper:\n$$l_{ce}(f_Θ(x),y)=-log⁡(\\frac{\\exp⁡(o_y)}{∑_{i=1}^{m+n}\\exp⁡(o_i)})-log⁡(\\frac{∑_{i=1}^m \\exp⁡(o_i)}{∑_{i=1}^{m+n}\\exp⁡(o_i)}).$$\nHere, $m$ and $n$ denote the number of old and new classes, respectively. The second term of this equation is the loss $l_n(\\cdot)$ computed using the negative examples. Since the replay loss $l_{rep}(\\cdot)$ has already included the loss $l_n(\\cdot)$ computed using the negative examples, we did not try using some data from the replay buffer for this loss.\n\n**Q3: For Figure 2, it might be helpful to also see on the graph a line where neither component is removed (ie both components of the decoupled loss without reweighting), and a line with the coupled loss used in traditional ER (ie Equation 2). This could tell us how much the decoupling formulation that is proposed actually corresponds to the usual coupled formulation. Another graph to add could be the accuracy on the new task being trained, allowing us to get a sense of how these losses affect plasticity.**\n\n**A3**: We add the line where neither component is removed in Figure I of the uploaded PDF. As we can see, when neither component is removed from Equation (2), the model forgets either more than or similarly to removing loss $l_{ce}(\\cdot;C_n)$. We also add a graph in Figure II of the uploaded PDF, which shows the accuracy of each new task during the training. As we can see, removing either loss $l_{ce}(\\cdot;C_n)$ or loss $l_n(\\cdot)$ decreases the model’s plasticity and leads to lower performance on the new task. Hence, both of these two losses are necessary for the learning of a new task. We will add these graphs in the final version of our paper. Thanks for the suggestion.\n\n[1] Buzzega P, Boschini M, Porrello A, et al. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 2020."}, {"review_id": "efgIyTLxl6", "reviewer": "Reviewer_n7mC", "summary": "The paper titled \"Loss Decoupling for Continual Learning\" addresses the challenge of catastrophic forgetting in continual learning, where a model needs to learn multiple tasks sequentially. The paper focuses on the task-agnostic problem, where task identities are not available during inference. The main contribution of the paper is the proposal of a method called loss decoupling (LODE) for continual learning. LODE separates the learning objectives for a new task by decoupling the loss, allowing for different weights to be assigned to different objectives. Experimental evaluations on multiple datasets demonstrate that LODE outperforms existing state-of-the-art replay-based methods, achieving a better trade-off between stability and plasticity.", "questions": "1. Could you elaborate on how the proposed LODE method can be applied to other continual learning problems beyond the task-agnostic setting? What are the potential challenges and considerations in extending LODE to different problem formulations?\n2. Can you provide further insights or examples regarding the generalizability of LODE to other types of tasks, apart from image classification? \n3. Regarding the limitation of focusing on the task-agnostic problem, have you identified any potential strategies or directions for separating learning objectives in other types of continual learning problems? If not, what are the challenges or considerations in extending loss decoupling to task-aware problems?\n4. How do you get this Figure1 (b)? Is this based on some specific data set or an ideal description? What does the y-axis represent?", "limitations": "/", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper introduces a novel method, LODE, which decouples the learning objectives for a new task in continual learning. This approach offers a fresh perspective on addressing the trade-off between stability and plasticity. The idea of assigning different weights to different objectives based on their impact on forgetting is unique and contributes to the advancement of continual learning methods.\n2. The paper demonstrates a high level of quality in terms of experimental design, clear presentation of results, and thorough analysis. The use of popular datasets, the comparison with state-of-the-art methods, and the inclusion of multiple runs with different seeds enhance the reliability and validity of the experimental findings. The clarity of the paper's structure and explanations contributes to its overall quality.\n3. The experimental results demonstrate the effectiveness of LODE in outperforming existing state-of-the-art methods, which has practical implications for enhancing continual learning algorithms and facilitating real-world applications.", "weaknesses": "1. The paper focuses solely on the task-agnostic problem and does not explore the separation of learning objectives in other continual learning problems. It would be valuable for future work to investigate the application of loss decoupling to different problem settings, thereby broadening the scope and applicability of the proposed method.\n2. Lack of comparison with other loss decoupling methods [1][2]: While the paper proposes LODE as a novel method, it does not compare LODE with other loss decoupling techniques, if any exist. Including such a comparison would strengthen the paper's novelty and showcase the advantages of LODE over alternative approaches.\n3. The experiments primarily consider image classification tasks, and the paper does not thoroughly explore the effectiveness of LODE for other types of tasks. Conducting experiments and providing insights on the generalizability of LODE to different domains would enhance the paper's impact and practical relevance.\n4. The application of the proposed method in continual learning has strong limitations: 1) It must be a replay-based method and satisfy the loss Equation 1; 2) It must be a task-agnostic setting. However, the paper uses a very broad title, covering the entire field of continual learning.\n\n[1] Sun, W., Li, Q., Zhang, J., Wang, W., & Geng, Y. A. (2023). Decoupling Learning and Remembering: A Bilevel Memory Framework With Knowledge Projection for Task-Incremental Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20186-20195).\n[2] Zhao, Z., Zhang, Z., Tan, X., Liu, J., Qu, Y., Xie, Y., & Ma, L. (2023). Rethinking Gradient Projection Continual Learning: Stability/Plasticity Feature Space Decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3718-3727).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The paper focuses solely on the task-agnostic problem. ... investigate the application of loss decoupling to different problem settings.**\n\n**A1**: Task-agnostic problem we consider in this work is a challenging problem where task identities are not available during testing. In contrast, for another popular problem in continual learning (CL), called task-aware problem, the model can obtain task identities during testing. Most CL methods, including the work that the reviewer mentioned, consider only one of them. Specifically, the work mentioned by the reviewer, called bilevel memory system with knowledge projection (BMKP) [1], considers only task-aware problem, as shown in Section 3.2 of their paper and their code. In contrast, our method considers task-agnostic problem, which is more challenging than task-aware problem. In **A6**, we give the insight about broadening the applicability of our proposed method to different problem settings, which will be our future work. Thanks for the suggestion.\n\n**Q2: Lack of comparison with other loss decoupling methods [1][2].**\n\n**A2**: We do not compare with the methods that the reviewer mentioned since these methods consider task-aware problem while our method considers task-agnostic problem. Specifically, BMKP [1] requires the task identities to choose the corresponding knowledge representations during testing, making it unsuitable in the task-agnostic problem. Space decoupling (SD) [2] does not explicitly mention that it only considers task-aware problem, but its experiments completely follow a task-aware CL method called TRGP [3], indicating its focus on task-aware problem. Furthermore, both these methods are published at CVPR 2023. Since the conference data of CVPR 2023 was after NeurIPS 2023 submission deadline, we were not aware of these two methods when we submitted our paper. We will cite these two methods in the final version of our paper and discuss the difference between our method and them.\n\n**Q3: The experiments primarily consider image classification tasks, ... not explore other types of tasks.**\n\n**A3**: We primarily consider image classification tasks since many existing methods, including the works the reviewer mentioned [1][2], consider primarily image classification tasks. We have introduced this limitation in Section 6 of our paper, which will be further studied in future work.\n\n**Q4: limitations: 1 It must be a replay-based method and satisfy Equation 1; 2 It must be a task-agnostic setting. the paper uses a very broad title.**\n\n**A4**: First, replay-based methods are among the most representative CL methods, and most existing state-of-the-art replay-based methods satisfy Equation (1). Second, task-agnostic problem is challenging and many existing replay-based methods focus on this problem. We will change the title to “loss decoupling for task-agnostic continual learning” in the final version of our paper. Thanks for the suggestion.\n\n**Q5: provide further insights regarding the generalizability of LODE to other types of tasks?**\n\n**A5**: First, we should emphasize that many CL methods, including BMKP and SD mentioned by the reviewer, focus solely on image classification. Second, from the methodology perspective, our methods can be applied to other tasks like NLP and speech data analysis in the task-agnostic problem, where the learning of a new task can be decoupled into learning a new task itself and learning to distinguish between the new task and the old tasks. This is because the main contribution of our method lies in decoupling the loss function which is a general one for modeling different types of input data.\n\n**Q6: elaborate how LODE method can be applied to other continual learning problems? identify potential strategies for separating learning objectives in other continual learning problems?**\n\n**A6**: We can leverage the semantic information of the classes to separate learning objectives in other types of CL problems including task-aware problems. Taking the task-aware problem as an example, assume the model is going to learn to distinguish three classes, including tiger, wolf and bee. Assume the model has learned to distinguish between cat and dog. Since tiger is similar to cat and wolf is similar to dog semantically, the model may change very little when learning to distinguish between tiger and wolf. Note that changing less is conducive to overcome catastrophic forgetting, we can divide the new task into two learning objectives, where the first objective makes the model distinguish between tiger and wolf, and the second objective makes the model distinguish between other category pairs. We will add this discussion in the final version of our paper. Thanks for the suggestion.\n\n**Q7: How do you get Figure 1 (b)? based on some specific data set or an ideal description? What does the y-axis represent?**\n\n**A7**: Figure 1 (b) is an idea description. This figure describes the coupling property of existing replay-based methods. Specifically, existing replay-based methods either sample from $\\mathcal{D}_t$ and $\\mathcal{M}$ separately, or directly sample from $\\mathcal{D}_t⋃\\mathcal{M}$. Then, they compute the loss for the new task through cross-entropy, which couples different new learning objectives. Figure 1 (b) is used to introduce this phenomenon figuratively. The y-axis represents the model’s abilities, including its ability to adapt itself to the new task (plasticity) and its ability to overcome forgetting (stability). This explanation will be added in the final version. Thanks for the comment.\n\n[1] Sun W, Li Q, Zhang J, et al. Decoupling Learning and Remembering: A Bilevel Memory Framework with Knowledge Projection for Task-Incremental Learning. CVPR, 2023.\n\n[2] Kim S, Noci L, Orvieto A, et al. Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning. CVPR, 2023.\n\n[3] Lin S, Yang L, Fan D, et al. TRGP: Trust Region Gradient Projection for Continual Learning. ICLR, 2022."}, {"review_id": "DIEwcrPVng", "reviewer": "Reviewer_Tdag", "summary": "This paper investigates the stability-plasticity tradeoff problem in a task-agnostic continual learning (CL) setting by decoupling the loss of the new task (LODE). LODE introduces two separate objectives for the new task: new/old class distinction and new class distinction. The authors analyze the impact of these objectives on forgetting and propose a strategy that assigns different weights to achieve a better stability-plasticity tradeoff. The proposed method is extensively evaluated through experiments and ablation studies on CIFAR10, CIFAR100, and TinyImageNet, demonstrating its effectiveness. ", "questions": "- Can LODE be applied in an exemplar-free setting? It would be helpful to have a discussion or a simple experiment addressing this. \n\n- The proposed method is only combined with similar replay methods (ER, DER, and ESMER). More combinations or a discussion of this limitation would be valuable. ", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed idea is simple and interesting, offering a promising approach to improve the stability-plasticity tradeoff. It is likely to gain attention within the CL community. \n\n- The submission is well-written and easy to follow. \n\n- The experimental settings and comparisons are detailed, considering both offline and online settings. ", "weaknesses": "- The experiments only employ ResNet18; it would be beneficial to include a larger model for evaluation. \n\n- The paper lacks a description of the learning rate. Is it the same for all compared methods? Was a grid-search conducted to determine it? \n\n- The use of TSNE visualization could provide a better way to validate the assumptions made. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The experiments only employ ResNet18; it would be beneficial to include a larger model for evaluation.**\n\n**A1**: Our experiments are built on top of the mammoth [3] continual learning repository in PyTorch like many existing works [2,3]. This repository consists of only ResNet18 and a small MLP for all the experiments. To make our experiments convincing, we do not change the model architecture. CL with larger models is also very interesting, but this is not the focus of this paper. We leave it for future study. Thanks for the suggestion.\n\n**Q2: The paper lacks a description of the learning rate. Is it the same for all compared methods? Was a grid-search conducted to determine it?**\n\n**A2**: Yes, the learning rate is determined by a grid-search. Specifically, the open-source code of the existing work DER has conducted a grid-search to determine hyperparameters, including the learning rate, for a wide range of baselines. Since our experimental setting follows this work, the hyperparameters, including the learning rate, are kept consistent with the code of this work. For the baselines that are not included in the code of DER but follow the experimental settings of DER, we follow their open-source code to set the hyperparameters, including the learning rate, which is also determined by a grid-search. For the methods not following the experimental setting of DER, we conduct a grid-search on validation datasets to determine the hyperparameters, including the learning rate. When combining our LODE with other methods, we keep the learning rate consistent with the original methods. We will add the description of the learning rate in the final version of our paper. Thanks for the suggestion.\n\n**Q3: The use of TSNE visualization could provide a better way to validate the assumptions made.**\n\n**A3**: We present the TSNE visualization in Figure III of the uploaded PDF. As we can see, removing loss $l_n(\\cdot)$ makes the representation of the old classes overlap less than removing loss $l_{ce}(\\cdot;C_n)$. Therefore, removing loss $l_n(\\cdot)$ makes the model suffer from less forgetting than removing loss $l_{ce}(\\cdot;C_n)$. We will add this figure in the final version of our paper. Thanks for the suggestion.\n\n**Q4: Can LODE be applied in an exemplar-free setting? It would be helpful to have a discussion or a simple experiment addressing this.**\n\n**A4**: YYes, LODE can be applied in an exemplar-free setting. We apply our LODE to a popular exemplar-free continual learning method called prototype augmentation and self-supervision (PASS) [1], which is designed for task-agnostic problems and utilizes a cross-entropy loss for learning new tasks. Specifically, we decouple the loss in PASS in the same way as Equation (3) and apply our LODE to it. The following table shows the results of PASS and LODE (PASS). We can find that LODE (PASS) outperforms PASS on Seq-CIFAR100. More results of combining our LODE with exemplar-free methods will be added in the final version of our paper. Thanks for the suggestion.\n\n| Methods       |  ACC (%) \n|:-|:-:\n| PASS          | 47.45±0.37 \n| LODE (PASS)  | **50.49±0.22** \n\n**Q5: The proposed method is only combined with similar replay methods (ER, DER, and ESMER). More combinations or a discussion of this limitation would be valuable.**\n\n**A5**: From the methodology perspective, any methods with the loss in the form of Equation (1) can be combined with our methods. We combined with ER, DER, and ESMER since these methods are either state-of-the-art methods or competitive in a wide range of settings, including both offline and online continual learning setting. In **A4**, we present the combination of our method with other continual learning method. We will add the results and further discussion of limitations of the experiments in the final version of our paper. Thanks for the suggestion.\n\n[1] Zhu F, Zhang X Y, Wang C, et al. Prototype augmentation and self-supervision for incremental learning. CVPR, 2021.\n\n[2] Sarfraz F, Arani E, Zonooz B. Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning. ICLR, 2023.\n\n[3] Buzzega P, Boschini M, Porrello A, et al. Dark experience for general continual learning: a strong, simple baseline. NeurIPS, 2020."}]

Author rebuttal:
We appreciate the reviewers' positive comments:
1. The problem is interesting to investigate the different impacts inter-task and intra-task classification on the performance of class incremental CL. (reviewer Y5HQ)
2. The proposed method is intuitive and can improve performance in many continual learning works. / The proposed method contributes to the advancement of continual learning methods. (reviewer odRp, reviewer n7mC)
3. This paper offers a promising approach to improve the stability-plasticity tradeoff. / The paper introduces a novel method and offers a fresh perspective on addressing the trade-off between stability and plasticity. (reviewer Tdag, reviewer n7mC)
4. The submission is well-written and easy to follow. / The clarity of the paper's structure and explanations contributes to its overall quality. (reviewer Tdag, reviewer n7mC)
5. The experimental design demonstrates a high level of quality. / The paper’s experiments and presentation of experiments are well done. / The experimental settings and comparisons are detailed. (reviewer n7mC, reviewer odRp, reviewer Tdag)

At the same time, we are grateful for the valuable questions and concerns of all the reviewers and have provided individual responses accordingly.

Meta-review: The rebuttal and discussion convinced three reviewers to raise their score leading to an overall positive recommendation from the reviewers (5567). The AC follows the recommendation of the reviewers and recommends acceptance. Given the importance of the rebuttal/discussion, the authors should carefully incorporate these into their final version, including the improved explanations, improved analysis hyperparameters, and inclusion of missing related work.
Decision: Accept (poster)