Title: Loss Decoupling for Task-Agnostic Continual Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Continual Learning

stability

plasticity
Abstract: Continual learning requires the model to learn multiple tasks in a sequential order. To perform continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Task-agnostic problem in continual learning is a challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. In task-agnostic problem, the model needs to learn two new objectives for learning a new task, including distinguishing new classes from old classes and distinguishing between different new classes. For task-agnostic problem, replay-based methods are commonly used. These methods update the model with both saved old samples and new samples for continual learning. Most existing replay-based methods mix the two objectives in task-agnostic problem together, inhibiting the models from achieving a good trade-off between stability and plasticity. In this paper, we propose a simple yet effective method, called loss decoupling (LODE), for task-agnostic continual learning. LODE separates the two objectives for the new task by decoupling the loss of the new task. As a result, LODE can assign different weights for different objectives, which provides a way to obtain a better trade-off between stability and plasticity than those methods with coupled loss. Experiments show that LODE can outperform existing state-of-the-art replay-based methods on multiple continual learning datasets.

Loss Decoupling for Task-Agnostic Continual Learning

Abstract

Continual learning requires the model to learn multiple tasks in a sequential order. To perform continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Task-agnostic problem in continual learning is a challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. In task-agnostic problem, the model needs to learn two new objectives for learning a new task, including distinguishing new classes from old classes and distinguishing between different new classes. For task-agnostic problem, replay-based methods are commonly used. These methods update the model with both saved old samples and new samples for continual learning. Most existing replay-based methods mix the two objectives in task-agnostic problem together, inhibiting the models from achieving a good trade-off between stability and plasticity. In this paper, we propose a simple yet effective method, called loss decoupling (LODE), for task-agnostic continual learning. LODE separates the two objectives for the new task by decoupling the loss of the new task. As a result, LODE can assign different weights for different objectives, which provides a way to obtain a better trade-off between stability and plasticity than those methods with coupled loss. Experiments show that LODE can outperform existing state-of-the-art replay-based methods on multiple continual learning datasets.

1 Introduction

Continual learning requires the model to learn multiple tasks in a sequential order [31]. However, neural network models suffer from a phenomenon called catastrophic forgetting (CF) [12], in which the performance of the network on the old tasks degrades significantly after it learns a new task. To address this challenge and enable continual learning, the model must possess the abilities to maintain performance on old tasks (stability) and adapt itself to learn new tasks (plasticity). Nevertheless, an excess of stability or plasticity can interfere with the other [39], and hence the model needs to make a trade-off between stability and plasticity [39, 31].

There exist two different kinds of problems, task-agnostic problem and task-aware problem, in continual learning. Task-agnostic problem [39] in continual learning is a challenging problem, in which task identities are not available in the inference stage and hence the model must learn to distinguish all the classes in all the tasks. In contrast, task-aware problem [28] in continual learning enables the model to get task identities in the inference stage. Therefore, the model for task-aware problem only needs to distinguish the classes belonging to the same task. The difference between task-agnostic problem and task-aware problem [28, 29] shows that the model in task-agnostic problem

New/old class distinction

New class distinction

: Old classes

: New classes

Exsting methods

Ours

:

Stability

:

Overal plasticity

:

Balanced area

:

Plasticity for new/old class distinction

:

Plasticity for new class distinction

Separating the learning of new task

More Stability

More Plasticity

More Stability

More Plasticity

(a) Different learning objectives

(b) Stability-plasticity trade-off in different methods

needs to learn two objectives for learning a new task, including distinguishing new classes from old classes (called new/old class distinction) and distinguishing between different new classes (called new class distinction). Figure 1 (a) illustrates these two learning objectives.

Many methods have been proposed for continual learning, including regularization-based methods [45, 20, 2], expansion-based methods [34, 18, 23] and replay-based methods [3, 10, 4]. For task-agnostic problem, replay-based methods are commonly used. These methods use a memory buffer to maintain a small portion of samples from the old classes. When learning a new task, the model retrieves old samples from memory and updates the parameters with both new and old samples. As illustrated in Figure 1 (b) from the methodological perspective, most existing continual learning methods [9, 7, 43] mix the two objectives in task-agnostic problem together (discussed later in Section 3.1). But these two learning objectives may cause different degrees of forgetting in continual learning and thus different trade-off strategies between stability and plasticity are required for these two learning objectives. More specifically, if a new learning objective leads to more forgetting, a good continual learner should pay more attention to the model's stability. On the contrary, if a new learning objective leads to less forgetting, a good continual learner should pay more attention to the model's plasticity for this objective. However, when the model mixes different learning objectives together, adjusting one of the learning objectives may influence others, inhibiting the model from achieving a good trade-off between stability and plasticity.

In this paper, we propose a simple yet effective method, called loss decoupling (LODE), for taskagnostic continual learning. The main contributions of this paper are listed as follows:

By deeply analyzing the impacts of new/old class distinction and new class distinction, we find that these two learning objectives cause different degrees of forgetting. Therefore, mixing these two objectives together is detrimental for the model to make a good trade-off between stability and plasticity.

LODE separates the two objectives for the new task by decoupling the loss of the new task. As a result, LODE can assign different weights for different objectives, which provides a way to obtain a better trade-off between stability and plasticity than those methods with coupled loss.

Experiments show that LODE can outperform existing state-of-the-art replay-based methods on multiple continual learning datasets.

2 Related Work

Continual learning can be offline or online. In offline continual learning setting [17, 32, 44, 24], the model receives the entire dataset of a new task and updates its parameters multiple times over this dataset. In online continual learning setting [11, 4, 10], the data from each task is sequentially concatenated as a non-stationary data stream and each data of each task can only appear once in the

data stream. Hence, the model can only receive a mini-batch of samples from the data stream at a time and update its parameters based on this mini-batch.

Different types of methods have been proposed for continual learning, including regularization-based methods [45, 20, 2], expansion-based methods [34, 18, 23] and replay-based methods [3, 10, 4]. For task-agnostic problem, replay-based methods are commonly used. Some replay-based methods [40, 42] achieve replay by learning a generative model for generating old samples. However, learning a good generative model is challenging in some settings, like online continual learning [13]. Some replay-based methods like experience replay (ER) [9] maintain a memory buffer to save old samples and replay them with new samples. Some methods combine experience replay with knowledge distillation [15]. Specifically, these methods store either a old model [43] or the outputs of the old models [7]. Some methods [25] like experience replay with asymmetric cross entropy (ER-ACE) [8] improve ER by changing the loss of the new task so that the model can avoid large representation drift. Some methods like error-sensitive modulation experience replay (ERMER) [36] and complementary learning system experience replay (CLS-ER) [5] use an extra set of model's parameters to aggregate the knowledge of different tasks. Other methods try to search for valuable samples [3] or optimize the distribution of the memory [19, 41] to overcome forgetting. However, all these methods mix two new learning objectives in Figure 1 (a), inhibiting the models from achieving a good trade-off between stability and plasticity.

3 Methodology

In this section, we first formulate the problem in continual learning. Then, we deeply analyze the problem of mixing different learning objectives together in existing methods. After that, we propose our method called loss decoupling (LODE), which can be used in both offline and online continual learning settings. Finally, we discuss the relation between LODE and existing methods.

3.1 Problem Formulation

Continual learning requires the model to learn from multiple tasks in a sequential order. We use D t = { x t i , y t i } N t to denote the dataset of the t -th task, where x t i denotes an input sample and y t i denotes its label. N t denotes the number of samples for task t . The total number of tasks is denoted by T . When the model learns a new task, the model can obtain limited or no data from the old tasks, potentially leading to catastrophic forgetting. In this work, we consider a challenging continual learning problem called task-agnostic problem, in which task identities are not available in the inference stage. In the task-agnostic problem, when the model learns on a new task t , some new classes are presented to the model. The model must possess the abilities to maintain performance on old classes (stability) and adapt itself to learn new classes (plasticity).

Replay-based methods [16, 7] maintain a memory buffer M with limited size to store a small portion of old samples. When receiving a mini-batch of new samples B t from a new task t , the model retrieves a mini-batch of samples B M from M and replays them with the new samples B t to achieve a trade-off between stability and plasticity. The losses used in most existing replay-based methods can be written as follows:

Here, L new is the loss for the new task and is mainly for the plasticity of the model. In contrast, L rep is the replay loss and is mainly for the stability of the model. We follow most existing works [7, 43] and assume that L new is a cross-entropy loss. L rep usually varies for different methods. For example, L rep can be a cross-entropy loss [3, 9] or a combination of cross-entropy and regularization losses [7, 5, 36]. Based on Figure 1 (a), we can find that L new is not only related to new/old class distinction, but also related to new class distinction in the task-agnostic problem. Hence, existing methods with the loss in (1) mix the two different learning objectives (new/old class distinction and new class distinction) together.

Note that there exists a type of replay-based methods [43, 17, 14, 38] directly sampling training data from D t ∪ M , which makes their losses can not be written in the form shown in (1). In some challenging continual learning settings like online continual learning setting, the model cannot access

1

3

5

Number of Tasks

0.4

0.6

0.8

1.0

Accuracy of the First Task

(a) Seq-CIFAR10

Remove

/lscript

ce

(

·

;

C

n

)

Remove

/lscript

n

(

·

)

1

3

5

Number of Tasks

0.00

0.25

0.50

0.75

Accuracy of the First Task

(b) Seq-CIFAR100

Remove

/lscript

ce

(

·

;

C

n

)

Remove

/lscript

n

(

·

)

5

10

Loss Value

0

50

100

Frequency

(c) Seq-CIFAR10

/lscript

ce

(

·

;

C

n

)

/lscript

n

(

·

)

5

10

Loss Value

0

50

100

Frequency

(d) Seq-CIFAR100

/lscript

ce

(

·

;

C

n

)

/lscript

n

(

·

)

the entire dataset of a new task. At this time, it is impossible for the model to sample from D t ∪ M . On the contrary, loss in (1) can be adapted to online continual learning by sampling B t from the data stream S and sampling B M from M separately. Hence, in this work, we focus on the methods whose loss function has the form in (1). In Section 4 and Appendix, we also compare our method with the methods that sample training data from D t ∪ M .

3.2 Analyzing Learning Objectives by Decoupling Loss

When learning the t -th task, we assume there are m old classes, which are denoted as C o = { 1 , 2 , ..., m } . We also assume there are n new classes, which are denoted as C n = { m + 1 , m + 2 , ..., m + n } . Then, for a given new sample ( x , y ) where y ∈ C n , we can compute its loss L new as

Here, glyph[lscript] ce ( · ) denotes the cross-entropy loss. [ o 1 , o 2 , ..., o m , o m +1 , ..., o m + n ] denotes the logits outputed by f Θ ( x ) . We decouple the loss L new ( f Θ ( x ) , y ) according to the two learning objectives in Figure 1 (a):

Here, we use glyph[lscript] ce ( · ; C n ) to denote the cross-entropy loss restricted to new classes C n . Obviously, glyph[lscript] ce ( f Θ ( x ) , y ; C n ) is related to new class distinction. glyph[lscript] n ( f Θ ( x )) is related to new/old class distinction. Note that both glyph[lscript] ce ( f Θ ( x ) , y ; C n ) and glyph[lscript] n ( f Θ ( x )) are for the plasticity of the model and may cause catastrophic forgetting. Furthermore, loss glyph[lscript] ce ( f Θ ( x ) , y ; C n ) and loss glyph[lscript] n ( f Θ ( x )) have the same weight in (1) due to the coupling property.

We use experience replay (ER) [9], which is one of the most popular replay-based methods and can be expressed in the form of (1), to evaluate the impact of glyph[lscript] ce ( · ; C n ) and glyph[lscript] n ( · ) . Specifically, we first let the model learn on the first task through valina stochastic gradient descent [21]. Then, before learning the subsequent tasks, we remove one of the two losses in (3) from ER and analyze the forgetting of the first task. The experiments are conducted on two datasets Seq-CIFAR10 and Seq-CIFAR100, which will be introduced in Section 4. The experimental settings also follow the descriptions in Section 4.1. Figure 2 (a) and Figure 2 (b) show the accuracy of the first task when the model learns subsequent tasks. Here, 'Remove glyph[lscript] ce ( · ; C n ) ' means that we remove glyph[lscript] ce ( · ; C n ) from (1) after learning the first task. 'Remove glyph[lscript] n ( · ) ' means that we remove glyph[lscript] n ( · ) from (1) after learning the first task. The results show that removing glyph[lscript] n ( · ) results in less forgetting of the first task than removing glyph[lscript] ce ( · ; C n ) . In other words, glyph[lscript] n ( · ) leads to more forgetting than glyph[lscript] ce ( · ; C n ) .

It is intuitively reasonable that glyph[lscript] n ( · ) leads to more forgetting than glyph[lscript] ce ( · ; C n ) . First, since replay-based methods usually keep limited samples in memory, when the model learns a new task, it has access to much fewer samples from the old classes than from the new classes. Therefore, utilizing loss glyph[lscript] n ( · ) to learn to distinguish between new classes and old classes introduces a risk of biasing the model towards the new classes, potentially leading to serious catastrophic forgetting. In contrast, loss

glyph[lscript] ce ( · ; C n ) is independent of the old classes, thereby avoiding introducing a risk of biasing the model towards the new classes. Second, in Figure 2 (c) and Figure 2 (d), we also show the value of the losses glyph[lscript] ce ( · ; C n ) and glyph[lscript] n ( · ) before using ER to learn task 2. We can find that the value of glyph[lscript] n ( · ) is larger than the value of glyph[lscript] ce ( · ; C n ) on both Seq-CIFAR10 and Seq-CIFAR100. This finding is consistent with existing works [8, 36], which suggest that larger losses for the new task may result in larger feature drift, leading to more forgetting. In Appendix, we also provide the TSNE visualizations when either glyph[lscript] n ( · ) or glyph[lscript] ce ( · ; C n ) is removed, further confirming that glyph[lscript] n ( · ) leads to more forgetting than glyph[lscript] ce ( · ; C n ) .

Since glyph[lscript] n ( · ) leads to more forgetting than glyph[lscript] ce ( · ; C n ) , treating these two losses equally is not reasonable. In particular, based on the above analysis, we can find that a good continual learner should assign a larger weight to glyph[lscript] ce ( · ; C n ) and a smaller weight to glyph[lscript] n ( · ) . However, loss in (1) fails to achieve this goal due to the coupling property.

3.3 Loss Decoupling for Continual Learning

Section 3.2 has demonstrated the impact of different learning objectives on the model's forgetting and the issue of the coupling property in existing replay-based methods. To address this issue, we propose a new method called loss decoupling (LODE), which removes the coupling property present in existing methods.

Specifically, our LODE uses the following loss to perform continual learning:

Here, β 1 and β 2 are two coefficients that control the weight of the two different learning objectives. The finding in Section 3.2 shows that LODE should set β 2 to be smaller than β 1 , to make the model achieve a better trade-off between stability and plasticity than existing methods usually with coupled loss. Furthermore, since glyph[lscript] n ( · ) is for new/old class distinction, LODE sets β 2 proportional to the ratio |C n | |C o | to make the model not bias toward old or new classes. In contrast, since glyph[lscript] ce ( · ; C n ) is only related to the new classes, LODE sets β 1 to be a constant value. More specifically, LODE sets β 1 and β 2 as

Here, C and ρ are two hyperparameters. Note that when the number of tasks increases, the number of old classes also increases. In particular, when the number of old tasks is large, the number of old classes |C o | is usually much larger than the number of new classes |C n | . At this time, β 2 is much smaller than β 1 . Setting β 2 to be as large as β 1 , or setting β 1 to be as small as β 2 fails to make the model achieve a good trade-off between stability and plasticity, which will be verified in the experiment.

Note that we do not specify the form of L rep in (4). Therefore, our method can be combined with many replay-based methods with form (1) and improve these methods. Here, we give some examples which combine LODE with different state-of-the-art continual learning methods.

Combining LODE with ER and DER++ Both experience replay (ER) [9] and dark experience replay++ (DER++) [7] can be written in the form of (1). Therefore, the combinations of our LODE with these two methods are direct. Specifically, given a new batch of samples B t , the model compute glyph[lscript] ce ( · ; C n ) and glyph[lscript] n ( · ) according to (3). Then, the model computes L rep with the old samples B M through the specific formulation in ER or DER++. Finally, the model can get the loss (4).

Combining LODE with ESMER A recent method called error sensitive modulation experience replay (ESMER) [36] suggests that the model should learn more from smaller losses to avoid large feature drift. The loss in ESMER is slightly different from the loss in (1). Specifically, ESMER assigns different weights to different new samples in B t according to their loss values. Although the loss in ESMER is slightly different from the loss in (1), it still mixes the two different learning objectives together. Therefore, we can combine ESMER with our LODE through a similar form

Algorithm 1 Loss Decoupling (LODE) for Continual Learning

1: Input: a sequence of tasks with datasets {D 1 , ..., D T } , a neural network model f Θ ( · ) .

2: Output: a learned neural network model f Θ ( · ) .

3: while Get a mini-batch of samples B t from a task t do

4: Sample a mini-batch B M from memory M ;

5: Specify the weights for the two different learning objectives by (5);

6: Get the losses for learning objective through (3);

7: Compute the final loss through (4).

8: Perform backward propagation and update the model f Θ ( · ) through SGD;

9: Update memory M with B t through some memory update methods;

10: end while

to (4). The decoupled loss for ESMER can be written as follows:

Here, w i is the weight assigned to new sample i in ESMER. We can find that the loss in (6) not only modulates weights for different new samples but also modulates weights for the two different learning objectives.

Algorithm 1 gives the whole learning process of LODE.

3.4 Relation with Existing Methods

The loss in many existing methods has the coupling property like the loss in (1). However, some methods, from the perspective of this work, use the losses that show a certain degree of decoupling property. For example, the loss in ER-ACE [8] can be written as

It can be seen from (7) that this method removes the term glyph[lscript] n ( · ) from (3) and only retains glyph[lscript] ce ( · ; C n ) . In particular, loss in (7) is a special case of the loss in (4). More specifically, we can get the loss in (7) by setting β 1 = 1 and β 2 = 0 in (4). SSIL [1] has a similar form to that in (7) but uses a different L rep compared to ER-ACE. Due to the lack of glyph[lscript] n ( · ) , the loss in (7) can only leverage the new classes' samples kept in memory to learn the objective of new/old class distinction. In experiments, we will show that setting β 2 = 0 in (4) performs worse than setting β 2 = 0 in (4).

glyph[negationslash]

Some existing methods also incorporate the idea of separating objectives in continual learning. However, these methods are primarily designed for task-aware problem. For instance, bilevel memory system with knowledge projection (BMKP) [37] requires the task identities to choose the corresponding knowledge representations during testing, making it unsuitable in the task-agnostic problem. Space decoupling (SD) [46] does not explicitly mention that it only considers task-aware problem, but its experiments completely follow some task-aware methods [35, 26], indicating its focus on task-aware problem.

4 Experiments

4.1 Experimental Settings

Datasets We use three popular datasets for evaluation, including Seq-CIFAR10 [3], SeqCIFAR100 [10], and Seq-TinyImageNet [22]. Seq-CIFAR10 consists of 5 disjoint tasks with each task having 2 classes and 10k training samples. Seq-CIFAR100 consists of 5 disjoint tasks with each task having 20 classes and 10k training samples. Seq-TinyImageNet consists of 10 disjoint tasks with each task having 20 classes and 10k training samples. The statistics of different datasets are given in Appendix. All the experiments are for the task-agnostic problem.

Baselines We compare our method with many state-of-the-art replay-based continual learning methods, including incremental classifier and representation learning (iCaRL) [33], bias correction (BIC) [43], separated softmax for incremental learning (SSIL) [1], experience replay (ER) [9], maximally interfere retrieval (MIR) [3], dark experience replay++ (DER++) [7], supervised contrastive replay (SCR) [27], proxy-based contrastive replay (PCR) [25], experience replay with asymmetric cross entropy (ER-ACE) [8], error sensitive modulating experience replay (ESMER) [36], complementary learning system experience replay (CLS-ER) [5], and task-specific attention modules in lifelong learning (TAMiL) [6]. For CLS-ER, we follow the existing method [6] and implement it with a single exponential moving average model. We also include two methods without continual learning, joint and finetune , in the comparison. Here, joint denotes the method which learns all the tasks jointly while finetune denotes the method which learns all the tasks sequentially without any memory. The accuracy of joint can be treated as the accuracy upper-bound and the accuracy of finetune can be treated as the accuracy lower-bound. Among the methods we mentioned above, some methods only maintain a single learning model to perform continual learning, while others require an extra model in memory for knowledge integration or distillation. Since keeping more models requires more memory, and memory cost is an important metric in continual learning [30], we group different methods by whether keeping extra model (refer to Table 1) to make a more fair comparison.

Architecture and Training Details We follow existing continual learning works [7, 6] and use standard ResNet18 as the neural network architecture in all the experiments unless otherwise stated. The experiments are built on top of the mammoth [7] continual learning repository in PyTorch like many existing works [7, 6]. We use stochastic gradient descent (SGD) to optimize the parameters. The batch size and replay size are set to 32 to follow the existing continual learning works [7, 6]. We also follow existing methods [7, 5] to set memory as 500 and 5120 for all the datasets. The hyperparameters are selected through a small validation set. For the experiments of all the methods on all the datasets, we apply random crops and horizontal flips to both newly coming samples and buffered (saved) samples like existing works [7, 5]. For each of our experiments, we report the average and standard deviation of the mean test accuracy of all the tasks across 5 runs with different seeds. More details of training and hyperparameters for different methods are given in Appendix.

4.2 Experimental Results

4.2.1 Accuracy

Table 1 shows the results of different methods on different datasets. Here, 'Keeping Extra Model' represents whether a method needs to keep an extra model for continual learning, as described

1

3

5

Number of Tasks

70

80

90

100

Accuracy (%)

(a) Seq-CIFAR10

ER_ACE

DER++

LODE (DER++)

ESMER

LODE (ESMER)

1

3

5

Number of Tasks

40

60

80

100

Accuracy (%)

(b) Seq-CIFAR100

ER_ACE

DER++

LODE (DER++)

ESMER

LODE (ESMER)

1

2

3

4

Number of Tasks

60

80

Accuracy (%)

(c) 4 Tasks in Total

DER++

LODE (DER++)

ESMER

LODE (ESMER)

1

4

7

10

Number of Tasks

40

60

80

Accuracy (%)

(d) 10 Tasks in Total

DER++

LODE (DER++)

ESMER

LODE (ESMER)

0.0

0.5

1.0

ρ

70

72

74

76

Accuracy (%)

(a) Seq-CIFAR10

LODE (DER++)

LODE (ESMER)

0.0

0.2

0.4

ρ

40

45

50

55

Accuracy (%)

(b) Seq-CIFAR100

LODE (DER++)

LODE (ESMER)

0

5

10

C

68

70

72

75

Accuracy (%)

(c) Seq-CIFAR10

LODE (DER++)

LODE (ESMER)

0

5

10

C

40

45

50

55

Accuracy (%)

(d) Seq-CIFAR100

LODE (DER++)

LODE (ESMER)

in Section 4.1. We combine our LODE with ER, DER++ and ESMER, which can be written in the form of (1). We use LODE (ER), LODE (DER++) and LODE (ESMER) to denote them, respectively. Our experimental results confirm that the integration of our LODE method improves the performance of each combined method. For example, when compared to DER++ with buffer size 500, LODE (DER++) exhibits a 2 . 16% improvement on Seq-CIFAR10, a 4 . 23% improvement on Seq-CIFAR100 and a 1 . 87% improvement on Seq-TinyImageNet. Notably, LODE (DER++) achieves the best performance on Seq-CIFAR10. LODE (ESMER) achieves the best performance on Seq-CIFAR100 and Seq-TinyImageNet.

Figure 3 (a) and Figure 3 (b) show the variation of accuracy on Seq-CIFAR10 and Seq-CIFAR100 after the learning of each task. As we can see, LODE improves DER++ and ESMER at the end of each task. Figure 3 (c) and Figure 3 (d) show the results on Seq-CIFAR100 with different numbers of tasks. When there are 4 tasks in Seq-CIFAR100, each task consists of 25 exclusive classes. Similarly, when there are 10 tasks in Seq-CIFAR100, each task consists of 10 exclusive classes. As we can see, when the number of tasks varies, LODE still gives improvements on different methods. In Appendix, we show the results of more methods on Seq-CIFAR100 with different numbers of tasks.

4.2.2 Hyperparameter Analysis

We analyze the hyperparameters in LODE. We choose LODE (DER++) and LODE (ESMER) to analyze hyperparameters, as they represent the best performance of methods that retain or do not retain an extra model for continual learning, respectively.

We first vary the value of ρ in (5) to show its impact on the performance of the model. Figure 4 (a) and Figure 4 (b) give the analysis on Seq-CIFAR10 and Seq-CIFAR100. Note that when ρ = 0 , β 2 = 0 and the weight of glyph[lscript] n ( · ) in (4) is always zero. At this time, the loss in (4) degenerates to the loss in (7) and the model fails to get satisfactory performance. When the value of ρ increases, β 2 also increases and the performance of the model first increases and then decreases. This phenomenon is intuitively reasonable since a larger weight for glyph[lscript] n ( · ) leads to more forgetting and thus influences the overall model performance.

We also vary the value of C in (5) to show its impact on the performance of the model. Figure 4 (c) and Figure 4 (d) show the results of varying C on Seq-CIFAR10 and Seq-CIFAR100, respectively. We can find that the performance of the model decreases significantly when C is close to zero. In

particular, the model gets the best performance when the value of C is between 1 and 5. Since ρ and C are highly different when the model achieves the best performance, decoupling the loss through (3) is necessary for continual learning.

4.2.3 Ablation Study

We change the value of β 1 and β 2 to show the effectiveness of setting β 1 and β 2 through (5). We first set the value of β 1 = β 2 in (4) to remove the decoupling property. There are two possibilities to set β 1 = β 2 . The first possibility is to set β 1 = β 2 = C and the second possibility is to set β 1 = β 2 = ρ |C n | |C o | . Table 2 shows the results of these two possibilities, which are significantly inferior to our method. This means that separating the two different objectives by decoupling the loss of the new task is necessary for the model to achieve good performance. In Table 2, we also show the result of a variant by exchanging the value of β 1 and β 2 , which means β 1 = ρ |C n | |C o | and β 2 = C . We can find that the performance of this variant is still significantly inferior to our method.

4.2.4 Online Continual Learning

We also perform experiment for the online continual learning setting [3, 10] where the datasets of different tasks are concatenated to a non-stationary data stream. Since online continual learning is more challenging than offline continual learning, existing methods [11] usually use larger memory in online continual learning, especially for challenging datasets such as Seq-CIFAR100 and SeqTinyImageNet. Hence, we set the memory size to 5120 for all the datasets. To follow existing online continual learning methods [3, 10], the experimental settings we use here are different from those introduced in Section 4.1. These settings are introduced in Appendix.

Table 3 shows the results of different methods. We exclude those methods that have been implemented only in offline continual learning or those that have demonstrated lower performance in online continual learning in previous works [8, 25]. Similar to that in the offline continual learning setting, we can find that the integration of our LODE method also significantly improves the overall accuracy of each combined method. For example, when compared with DER++, LODE (DER++) exhibits a 1 . 70% improvement on Seq-CIFAR10, a 3 . 1% improvement on Seq-CIFAR100 and a 4 . 9% improvement on Seq-TinyImageNet.

5 Conclusion

In this work, we propose a new method called loss decoupling (LODE) for continual learning. Different from most existing replay-based methods which mixes the two different learning objectives together to learn the new task, LODE separates the two learning objectives for the new task by decoupling the loss of the new task. Experiments show that LODE can achieve a better trade-off between stability and plasticity than other baselines, and thus outperform other state-of-the-art replaybased methods across multiple datasets. Future work will extend LODE to other continual learning problem like task-aware problem and study the effectiveness of LODE for other types of tasks.

Acknowledgment

This work is supported by NSFC Project (No.62192783), National Key R&D Program of China (No.2020YFA0713901), and Fundamental Research Funds for the Central Universities (No.020214380108).

References

Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. SSIL: separated softmax for incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 824-833, 2021.

Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision , pages 139-154, 2018.

Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, and Tinne Tuytelaars. Online continual learning with maximally interfered retrieval. arXiv preprint arXiv:1908.04742 , 2019.

Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems , pages 11816-11825, 2019.

Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. In International Conference on Learning Representations , 2022.

Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. Task-aware information routing from common representation space in lifelong learning. In International Conference on Learning Representations , 2023.

Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in Neural Information Processing Systems , pages 15920-15930, 2020.

Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations , 2022.

Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc'Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486 , 2019.

Aristotelis Chrysakis and Marie-Francine Moens. Online continual learning from imbalanced data. In Proceedings of the International Conference on Machine Learning , pages 1952-1961, 2020.

Aristotelis Chrysakis and Marie-Francine Moens. Online bias correction for task-free continual learning. In International Conference on Learning Representations , 2023.

Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence , pages 3366-3385, 2021.

Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, and Pheng-Ann Heng. Flattening sharpness for dynamic gradient projection memory benefits continual learning. Advances in Neural Information Processing Systems , pages 18710-18721, 2021.

Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Proceedings of the European Conference on Computer Vision , pages 86-102, 2020.

Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision , pages 1789-1819, 2021.

Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems , pages 11588-11598, 2020.

Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 831-839, 2019.

Steven C. Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems , pages 13647-13657, 2019.

Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. Gradient-based editing of memory examples for online task-free continual learning. Advances in Neural Information Processing Systems , pages 29193-29205, 2021.

Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with node-importance based adaptive group sparse regularization. Advances in Neural Information Processing Systems , pages 3647-3658, 2020.

Nikhil Ketkar and Nikhil Ketkar. Stochastic gradient descent. Deep learning with Python: A Hands-on Introduction , pages 113-132, 2017.

Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N , page 3, 2015.

Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In Proceedings of the International Conference on Machine Learning , pages 3925-3934, 2019.

Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7816-7825, 2023.

Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, and Yunming Ye. Pcr: Proxy-based contrastive replay for online class-incremental continual learning. arXiv preprint arXiv:2304.04408 , 2023.

Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual learning. In International Conference on Learning Representations , 2022.

Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3589-3599, 2021.

Marc Masana, Joost Van de Weijer, Bartłomiej Twardowski, et al. On the importance of cross-task features for class-incremental learning. arXiv preprint arXiv:2106.11930 , 2021.

Sudhanshu Mittal, Silvio Galesso, and Thomas Brox. Essentials for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3513-3522, 2021.

Martin Mundt, Steven Lang, Quentin Delfosse, and Kristian Kersting. Cleva-compass: A continual learning evaluation assessment compass to promote research transparency and comparability. In International Conference on Learning Representations , 2022.

German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks , pages 54-71, 2019.

Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13588-13597, 2020.

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5533-5542, 2017.

Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671 , 2016.

Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference on Learning Representations , 2021.

Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. Error sensitivity modulation based experience replay: Mitigating abrupt representation drift in continual learning. In International Conference on Learning Representations , 2023.

Wenju Sun, Qingyong Li, Jing Zhang, Wen Wang, and Yangli-ao Geng. Decoupling learning and remembering: A bilevel memory framework with knowledge projection for task-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20186-20195, 2023.

Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class-incremental learning. In Proceedings of the European Conference on Computer Vision , pages 398-414, 2022.

Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487 , 2023.

Zhen Wang, Liu Liu, Yiqun Duan, and Dacheng Tao. Continual learning through retrieval and imagination. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 8594-8602, 2022.

Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tiehang Duan, and Mingchen Gao. Improving task-free continual learning by distributionally robust memory evolution. In Proceedings of the International Conference on Machine Learning , pages 22985-22998, 2022.

Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay GANs: learning to generate images from new categories without forgetting. In Advances in Neural Information Processing Systems , pages 5966-5976, 2018.

Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 374-382, 2019.

Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3014-3023, 2021.

Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the International Conference on Machine Learning , pages 3987-3995, 2017.

Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3718-3727, 2023.

Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. Prototype augmentation and self-supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5871-5880, 2021.

A Statistics of Different Continual Learning Datasets

We give detailed statistics on datasets in this section. Specifically, Table 4 shows the detailed information of three datasets, including Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImageNet. These datasets are constructed by one of the following datasets: CIFAR10, CIFAR100 or TinyImageNet. For each task, 5% of the training samples are divided into a validation set.

B More Experimental Details

B.1 Hyperparameters

Table 5 provides the hyperparameters for different methods, which will be used for comparison. ρ is selected among four different values [0 . 01 , 0 . 05 , 0 . 1 , 0 . 2] through the validation sets we split from the training sets. We do not carefully select hyperparameter C but set it as 1 for all the experiments to reduce the reliance of the model on hyperparameters.

It is important to note that both DER++ and ESMER have their own specific hyperparameters that may affect the performance of the model. To ensure a fair comparison, we maintain the specific hyperparameters of each method consistent when combined with our LODE, except for the exponential decay hyperparameter α in ESMER. This means that we use the same hyperparameters settings as the original implementation when evaluating our combined models. For the exponential decay hyperparameter α in ESMER, we find that setting it to the same value as ESMER (0.999) makes the model too conservative to learn the new task in LODE (ESMER). Therefore, we set it to 0.998 in order to make the model not so conservative. Furthermore, we follow the mammoth continual learning repository [7] to set the specific hyperparameters in each method or get them by the validation sets we split from the training sets.

The learning rate is determined by a grid-search. Specifically, the open-source code of the existing work DER [7] has conducted a grid-search to determine hyperparameters, including the learning rate, for a wide range of baselines. Since our experimental setting follows this work, the hyperparameters, including the learning rate, are kept consistent with the code of this work. For the baselines that are not included in the code of DER but follow the experimental settings of DER, we follow their opensource code to set the hyperparameters, such as learning rate and epoch, which are also determined by grid-search. For the methods not following the experimental setting of DER, we conduct a grid-search on validation datasets to determine the hyperparameters, including the learning rate. When combining our LODE with other methods, we keep the learning rate consistent with the original methods.

B.2 Experimental Setting in Online Continual Learning

Datasets We use three popular datasets for evaluation, including Seq-CIFAR10 [3], SeqCIFAR100 [10], and Seq-TinyImageNet [22]. We follow existing online continual learning methods to partition tasks across different datasets for the purpose of evaluation. Specifically, Seq-CIFAR10 consists of 5 disjoint tasks with each task having 2 classes and 10k training samples. Seq-CIFAR100 consists of 20 disjoint tasks with each task having 5 classes and 2.5k training samples. Seq-TinyImageNet consists of 20 disjoint tasks with each task having 10 classes and 5k training samples. All the experiments are for the task-agnostic problem.

Training Details For all the datasets, we use reduced-ResNet18 as our model architecture, which has been commonly used in prior online continual learning methods [8, 41]. Reduced-ResNet18 is a variant of the original ResNet18 that has fewer channel sizes, and fewer filters overall. We use stochastic gradient descent (SGD) to update the model for all the methods. We set memory sizes M=5120 for all the datasets. Like the existing online continual learning works [8, 3], the replay size |B M | is the same as the mini-batch size |B t | (fixed to 10). Data augmentation is the same as that in offline continual learning. Specifically, we use data augmentation such as random cropping and random rotation for both new mini-batch B t and old mini-batch B M .

C More Experimental Results

C.1 Combine with More Methods

LODE can be applied in an exemplar-free setting. We apply our LODE to a popular exemplar-free continual learning method called prototype augmentation and self-supervision (PASS) [47], which is designed for task-agnostic problems and utilizes a cross-entropy loss for learning new tasks. Table 6 shows the variation of the accuracy for the first task during the learning of subsequent tasks. We can find that removing glyph[lscript] n ( · ) makes the model suffer from less forgetting than removing glyph[lscript] ce ( · ; C n ) . Table 7 shows the final results of PASS and LODE (PASS). We can find that LODE (PASS) outperforms PASS on Seq-CIFAR100.

C.2 TSNE Visualization

We present the TSNE visualization in Figure 5, which shows the learned representations of different models on Seq-CIFAR10. As we can see, removing loss glyph[lscript] n ( · ) makes the representation of the old classes overlap less than removing loss glyph[lscript] ce ( · ; C n ) . Therefore, removing loss glyph[lscript] n ( · ) makes the model suffer from less forgetting than removing loss glyph[lscript] ce ( · ; C n ) . We will add this figure in the final version of our paper. Thanks for the suggestion.

C.3 Plasticity of the Model

We also add a graph in Figure 6, which shows the accuracy of each new task during the training of Seq-CIFAR10. As we can see, removing either loss glyph[lscript] ce ( · ; C n ) or loss glyph[lscript] n ( · ) decreases the model's plasticity and leads to lower performance on the new task. Hence, both of these two losses are necessary for the learning of a new task.

80

60

20

-75

05-

52-

50

75

100

20

D

=20

-75

-50

52-

6

75

40

20

20

-40

-60

-75

05-

-25

25

50

75

100

1

3

5

Number of Tasks

0.6

0.8

1.0

(a) Seq-CIFAR10

Remove

ce

( ;

n

)

Remove

n

( )

Keep Both (ER)

1

3

5

Number of Tasks

0.25

0.50

0.75

(b) Seq-CIFAR100

Remove

ce

( ;

n

)

Remove

n

( )

Keep Both (ER)

C.4 More Comparison

Table 8 presents the results of different methods on Seq-CIFAR100 with varying numbers of tasks. Here, the memory size is set as 500 for all the methods. As we can see, when the number of tasks varies, LODE still gives improvements on different methods. Furthermore, LODE (ESMER) still gets the best performance among all the methods.

Table 9 gives the results of additional methods that directly sample samples from D t ∪ M . Some methods like BIC and iCaRL, which directly sample samples from D t ∪M , have been compared with our methods in the main text. Since all these methods keep an extra model in memory for distillation, we compare them with LODE (ESMER). As we can see, our method still gets the best performance.

1

3

5

Number of Tasks

0.4

0.6

0.8

1.0

Accuracy of the First Task

(a) Seq-CIFAR10

Remove

/lscript

ce

(

·

;

C

n

)

Remove

/lscript

n

(

·

)

1

3

5

Number of Tasks

0.2

0.4

0.6

0.8

Accuracy of the First Task

(b) Seq-CIFAR100

Remove

/lscript

ce

(

·

;

C

n

)

Remove

/lscript

n

(

·

)

5

10

Loss Value

0

50

100

150

Frequency

(c) Seq-CIFAR10

/lscript

ce

(

·

;

C

n

)

/lscript

n

(

·

)

5

10

Loss Value

0

50

100

150

Frequency

(d) Seq-CIFAR100

/lscript

ce

(

·

;

C

n

)

/lscript

n

(

·

)

C.5 More Analysis by Decoupling Loss

We also use dark experience replay++ (DER++) [7] to evaluate the impact of glyph[lscript] ce ( · ; C n ) and glyph[lscript] n ( · ) . Figure 7 gives the results. The results are similar to that in ER. Specifically, removing glyph[lscript] n ( · ) results in less forgetting of the first task than removing glyph[lscript] ce ( · ; C n ) . In other words, glyph[lscript] n ( · ) leads to more forgetting than glyph[lscript] ce ( · ; C n ) . Furthermore, the value of glyph[lscript] n ( · ) is larger than the value of glyph[lscript] ce ( · ; C n ) on both Seq-CIFAR10 and Seq-CIFAR100.

60

40

20

-20

4D

60

-75

-50

-25

50

75

Peer reviews:
comments: SUMMARY:
This paper investigates the stability-plasticity tradeoff problem in a task-agnostic continual learning (CL) setting by decoupling the loss of the new task (LODE). LODE introduces two separate objectives for the new task: new/old class distinction and new class distinction. The authors analyze the impact of these objectives on forgetting and propose a strategy that assigns different weights to achieve a better stability-plasticity tradeoff. The proposed method is extensively evaluated through experiments and ablation studies on CIFAR10, CIFAR100, and TinyImageNet, demonstrating its effectiveness.

STRENGTHS:
- The proposed idea is simple and interesting, offering a promising approach to improve the stability-plasticity tradeoff. It is likely to gain attention within the CL community. 

- The submission is well-written and easy to follow. 

- The experimental settings and comparisons are detailed, considering both offline and online settings.

WEAKNESSES:
- The experiments only employ ResNet18; it would be beneficial to include a larger model for evaluation. 

- The paper lacks a description of the learning rate. Is it the same for all compared methods? Was a grid-search conducted to determine it? 

- The use of TSNE visualization could provide a better way to validate the assumptions made.

QUESTIONS:
- Can LODE be applied in an exemplar-free setting? It would be helpful to have a discussion or a simple experiment addressing this. 

- The proposed method is only combined with similar replay methods (ER, DER, and ESMER). More combinations or a discussion of this limitation would be valuable.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper titled \"Loss Decoupling for Continual Learning\" addresses the challenge of catastrophic forgetting in continual learning, where a model needs to learn multiple tasks sequentially. The paper focuses on the task-agnostic problem, where task identities are not available during inference. The main contribution of the paper is the proposal of a method called loss decoupling (LODE) for continual learning. LODE separates the learning objectives for a new task by decoupling the loss, allowing for different weights to be assigned to different objectives. Experimental evaluations on multiple datasets demonstrate that LODE outperforms existing state-of-the-art replay-based methods, achieving a better trade-off between stability and plasticity.\n\nSTRENGTHS:\n1. The paper introduces a novel method, LODE, which decouples the learning objectives for a new task in continual learning. This approach offers a fresh perspective on addressing the trade-off between stability and plasticity. The idea of assigning different weights to different objectives based on their impact on forgetting is unique and contributes to the advancement of continual learning methods.\n2. The paper demonstrates a high level of quality in terms of experimental design, clear presentation of results, and thorough analysis. The use of popular datasets, the comparison with state-of-the-art methods, and the inclusion of multiple runs with different seeds enhance the reliability and validity of the experimental findings. The clarity of the paper's structure and explanations contributes to its overall quality.\n3. The experimental results demonstrate the effectiveness of LODE in outperforming existing state-of-the-art methods, which has practical implications for enhancing continual learning algorithms and facilitating real-world applications.\n\nWEAKNESSES:\n1. The paper focuses solely on the task-agnostic problem and does not explore the separation of learning objectives in other continual learning problems. It would be valuable for future work to investigate the application of loss decoupling to different problem settings, thereby broadening the scope and applicability of the proposed method.\n2. Lack of comparison with other loss decoupling methods [1][2]: While the paper proposes LODE as a novel method, it does not compare LODE with other loss decoupling techniques, if any exist. Including such a comparison would strengthen the paper's novelty and showcase the advantages of LODE over alternative approaches.\n3. The experiments primarily consider image classification tasks, and the paper does not thoroughly explore the effectiveness of LODE for other types of tasks. Conducting experiments and providing insights on the generalizability of LODE to different domains would enhance the paper's impact and practical relevance.\n4. The application of the proposed method in continual learning has strong limitations: 1) It must be a replay-based method and satisfy the loss Equation 1; 2) It must be a task-agnostic setting. However, the paper uses a very broad title, covering the entire field of continual learning.\n\n[1] Sun, W., Li, Q., Zhang, J., Wang, W., & Geng, Y. A. (2023). Decoupling Learning and Remembering: A Bilevel Memory Framework With Knowledge Projection for Task-Incremental Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20186-20195).\n[2] Zhao, Z., Zhang, Z., Tan, X., Liu, J., Qu, Y., Xie, Y., & Ma, L. (2023). Rethinking Gradient Projection Continual Learning: Stability/Plasticity Feature Space Decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3718-3727).\n\nQUESTIONS:\n1. Could you elaborate on how the proposed LODE method can be applied to other continual learning problems beyond the task-agnostic setting? What are the potential challenges and considerations in extending LODE to different problem formulations?\n2. Can you provide further insights or examples regarding the generalizability of LODE to other types of tasks, apart from image classification? \n3. Regarding the limitation of focusing on the task-agnostic problem, have you identified any potential strategies or directions for separating learning objectives in other types of continual learning problems? If not, what are the challenges or considerations in extending loss decoupling to task-aware problems?\n4. How do you get this Figure1 (b)? Is this based on some specific data set or an ideal description? What does the y-axis represent?\n\nLIMITATIONS:\n/", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSeveral replay based continual learning methods decouple their loss into a loss based on samples from the replay to maintain performance on old tasks, and a loss based on data from the current task to learn the new task. This paper proposes to further decouple the latter loss into a component that helps the model distinguish whether a sample is in the new task or from previous tasks, and a component helps the model distinguish between just the new classes in the current task. The paper shows that decoupling this loss and giving different weight to the two components helps in maintaining performance and results in better final accuracy for Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImagenet.\n\nSTRENGTHS:\n- The paper proposes an intuitive decoupling of the loss used in many continual learning works. It shows that doing this decoupling is useful, and can improve performance by weighting the different parts of the decoupled loss differently.\n- The paper’s experiments and presentation of experiments are well done, although I do ask the authors to address some of the questions I present below. The hyperparameter sensitivity analysis is useful in showing that the method is at least somewhat robust to hyperparameter changes.\n\nWEAKNESSES:\nThe main weaknesses are mostly related to the additional questions and experiments I ask for in the next section. There are a few experiments I’d like to see that could make the paper stronger.\n\nQUESTIONS:\n- For section 4.2.3, did you do a hyperparameter search for the values of $\\beta_1$ and $\\beta_2$ for the methods that weren’t yours?\n- The $l_n$ loss in the denominator is essentially a binary classification loss. Since you are only using positive examples to compute this loss, it increases the value of the logits of the current task and decreases the value of the logits of previous tasks for your current data. Did you try also using some data from the replay buffer for this loss?\n- For Figure 2, it might be helpful to also see on the graph a line where neither component is removed (ie both components of the decoupled loss without reweighting), and a line with the coupled loss used in traditional ER (ie Equation 2). This could tell us how much the decoupling formulation that is proposed actually corresponds to the usual coupled formulation. Another graph to add could be the accuracy on the new task being trained, allowing us to get a sense of how these losses affect plasticity.\n\nLIMITATIONS:\nThe work presents a basic discussion of limitations and societal impacts. The limitations section could be further developed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper investigated class incremental continual learning (CL). By showing that two objectives, i.e., classifying among new classes and classifying between new classes and old classes, can have different impacts on the performance of CL, this paper proposed to decouple the new task loss and assign different weights to these two objectives. Experiments are conducted on multiple datasets to verify the effectiveness of the proposed method LODE in comparison with related baseline methods.

STRENGTHS:
It is interesting to investigate the different impacts inter-task and intra-task classification on the performance of class incremental CL.

WEAKNESSES:
1. The results in Figure 2(a) seems very counter-intuitive. When removing $l_n$, the training of the current task only focuses on the classification within the task and ignores the differentiation from old classes in the memory. In this case, the forgetting should be larger compared to the case of removing $l_{ce}$, while in Figure 2(a) the forgetting was smaller. The authors didn't give a convincing explanation of the underlying reason for the phenomenon.

 2. The weight selection (5) in LODE is entirely based on heuristics and manually determined. It is hard to justify the selection of the weights, but the performance of LODE is highly sensitive to the weight selection as shown in Table 2.

QUESTIONS:
1. How did you select the values of $C$ and $\rho$ in (5)?

2. Conceptually, the loss decoupling can be used in CL without using data replay. In this way, wouldn't it be more clear to understand the impact of these two parts on the performance of CL?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07