Continual Learning with Soft-Masking of Parameter-Level Gradient Flow
Authors: Tatsuya Konishi
Mori Kurokawa
Chihiro Ono
Zixuan Ke
Gyuhak Kim
Bing Liu
Venue: Submitted to ICLR 2023 (2023)
Keywords: continual learning
catastrophic forgetting
knowledge transfer

Abstract: Existing research on task incremental learning in continual learning has primarily focused on preventing catastrophic forgetting (CF). Several techniques have achieved learning with no CF. However, they attain it by letting each task monopolize a sub-network in a shared network, which seriously limits knowledge transfer (KT) and causes over-consumption of the network capacity, i.e., as more tasks are learned, the performance deteriorates. The goal of this paper is threefold: (1) overcoming CF, (2) encouraging KT, and (3) tackling the capacity problem. A novel and simple technique (called SPG) is proposed that soft-masks (partially blocks) parameter updating in training based on the importance of each parameter to old tasks. Each task still uses the full network, i.e., no monopoly of any part of the network by any task, which enables maximum KT and reduction of capacity usage. Extensive experiments demonstrate the effectiveness of SPG in achieving all three objectives. More notably, it attains significant transfer of knowledge not only among similar tasks (with shared knowledge) but also among dissimilar tasks (with little shared knowledge) while preventing CF.

CONTINUAL LEARNING WITH SOFT-MASKING OF PARAMETER-LEVEL GRADIENT FLOW
Anonymous authors Paper under double-blind review
ABSTRACT
Existing research on task incremental learning in continual learning has primar- ily focused on preventing catastrophic forgelling (CF). Several techniques have achieved learning with no CF. However, they attain it by letting each task mo- nopolize a sub-network in shared network, which seriously limits knowledge transfer (KT) and causes over-consumption of the network capacity, i.c., as more tasks are learned, the performance deteriorates. The goal of this paper threefold: (1) overcoming CF, (2) encouraging KT, and (3) tackling the capacity problem. A novel and simple technique (called SPG) is proposed that soft-masks (partially blocks) parameter updating in training based on the importance of each parameter to old tasks. Each task still uses the full network, i.e., no monopoly of any part of the network by any task, which enables maximum KT and reduction of capacity usage. Extensive experiments demonstrate the effectiveness of SPG in achieving all three objectives. More notably, it attains significant transfer of knowledge not only among similar tasks (with shared knowledge) but also among dissimilar tasks (with little shared knowledge) while preventing CF. [
INTRODUCTION
Catastrophic forgetting (CF) and knowledge transfer (KT) are two key challenges of continual learn- ing (CL), which learns a sequence of tasks incrementally. CF refers to the phenomenon where model loses some of its performance on previous tasks once it learns a new task. KT means that tasks may help each other learn by sharing knowledge. This work further investigates these prob- lems in the popular CIL paradigm of CL, called task incremental learning (TIL). In TIL, each task consists of several classes of objects be learned. Once task is learned, its data discarded and will not be available for later use. During testing, the task id is provided for each test sample so that the corresponding classification head of the task can be used for prediction.
Several effective approaches have been proposed for TIL that can achieve learning with little or no CF. Parameter isolation is perhaps the most successful one which the system learns to mask a sub-network for each task in a shared network. HAT (Serra et al. 2018) and SupSup (Wortsman jet al., 2020) are two representative systems. HAT learns neurons (not parameters) that are important for each task and in learning new task, these important neurons to previous tasks are maskedasked or blocked to prevent updating in the backward pass. Only those frese (unmasked) neurons and their parameters trainable. Thus, as more tasks are learned, the number of free neurons left becomes fewer, making later tasks harder to learn. Further, a neuron is masked, all the parameters feeding it are also masked, which consumes great deal of network capacity. While in HAT, as more and more tasks are learned, it has less and less capacity to learn new tasks, which results in gradual performance deterioration (see Section 4.3). the proposed method only soft-masks parameters and thus consumes much less network capacity. SupSup uses a different approach to learn and to fix a sub-network for each task. Since it does not learn parameters but only learn separate masks per task, it can largely avoid the reduction in capacity. However, that limits knowledge transfer. As the sub-networks for old tasks cannot be updated, these approaches can have two major shortcomings: (1) limited knowledge transfer, and/or (2) over-consumption of network capacity. CAT (Ke et al.) 2020) tries to improve knowledge transfer of HAT by detecting task similarities. If the new task is found similar to some previous tasks, these tasks' masks are removed so that the new task training
'The code is contained in the Supplementary Materials.

---

can update the parameters of these tasks for backward pass. However, this is risky because if dissimilar task is detected as similar, serious CF occurs, and if similar tasks are detected as dissimilar, its knowledge transfer will be limited. G
To tackle these problems, we propose a simple and very different approach, named "Soft-masking of Parameter-level Gradient flow" (SPG). It surprisingly effective and contributes in following ways:
(1). Instead of learning hard/binary attentions on neurons for each task and masking/blocking these task and in testing like HAT, SPG computes importance score for each
neurons in training network parameter (not neuron) to old tasks using gradients. The reason that gradients can be used in the importance computation is because gradients directly tell how a change to a specific param- eter will affect the output classification and may cause CF. SPG uses the importance score of each parameter as a soft-mast to constrain the gradient flow in backpropagation to ensure those important parameters to old tasks have minimum changes in learning the new task to prevent forgetting of previous knowledge. To our knowledge, the soft-masking of parameters has not been done before.
(2). SPG has some resemblance to the popular regularization-bant based approach, e.g., EWC (Kirk- patrick et al.] 2017), in that both use importance of parameters to constrain changes to important parameters of old tasks. But there is a major difference. SPG directly controls each parameter (fine- grained), but FWC controls all parameters together using a regularization term in the loss to penalize the sum of changes to all parameters in the network (rather coarse-grained) Section 4.2 shows that our soft- noft-masking is significantly better than the regularization. We believe this is an important result.
(3). In the forward pass, masks are applied, which encourages knowledge transfer among tasks. This is better than CAT as SPG does not need CAT's extra mechanism for task similarity comparison. Knowledge sharing and transfer in SPG are automatic. SupSup cannot do knowledge transfer.
(4). As SPG soft-masks parameters. it does not monopolize any parameters or sub-network like HAT for each task and SPG's forward pass does not use any masks. This reduces the capacity problem.
Section 2 shows that SPG is also very different from other TIL approaches. Experiments have been conducted with (1) similar tasks to demonstrate SPG's better ability to transfer knowledge across tasks, and (2) dissimilar tasks to show SPG's ability to overcome CF and to deal with the capacity problem. SPG is superior in both, which none of the baselines is able achieve.
2 RELATED WORK
Approaches in continual learning can be grouped into three main categories. We review them below.
Regularization-basse This approach computes importance values of either parameters or their gradients on previous tasks, and adds a regularization in the loss to restrict changes to those impor- tant parameters to mitigate CF. EWC (Kirkpatrick et al. 2017) uses the Fisher information matrix to represent the importance of parameters and a regularization
penalize the sum of changes to all parameters. SI (Zenke et al.] (2117]) extends EWC to reduce the complexity in computing the penalty.
Many other approaches (Li & Hoiem| 2016 Zhang al.) 2020 Ahn et al. 2019) in this category have also been proposed, but they still have difficulty to prevent CF. As discussed in the introduction section, the proposed approach SPG has some resemblance to regularization based method EWC. But the coarse-grained approach of using regularization is significant poorer than the fine-grained soft-masking in SPG for overcoming CF as we will see in Section
This approach introduces a small memory buffer to store data from previous tasks and replay them in learning the new task to prevent CF (Lopez-Paz Ranzato 2017 Chaudhry et al. (219)). Some approaches (Shin et al. [2017] Deja et 2021) also prepare data generators for previous tasks, and the generated pseudo-samples are used instead of real samples. Although several other approaches (Rebuffi et al. 2017]: Riemer et al. 2019 Aljundi et al., (019) have been proposed, they still suffer from CF. SPG does not save any replay data or generate pseudo-replay data.
Memory-based:
Parameter iilation-based This approach is most similar to ours SPG. It tries to learn a sub- network for each task (tasks may share parameters and neurons), which limits knowledge transfer. We have discussed HAT, SupSup, and CAT in Section Many others also take similar approaches,
For example, we have conducted experiment on sequence of 10 similar tasks of the dataset FF- 10 (see Section for the data deseription) and only 2 out of 10 tasks were detected as similar by CAT.
2

---

Algorithm Continual Learning in SPG.
1: for = 1,.... .T' do
2: # 'Training phase task (. Mr is the model for task
3: repeat
4: Compute gradients }g:} with Me using (X,, Yt).
5: for all parameters of i-th layer do
6:
g' + Eq. (5)
7:
Update Me with the modified gradients {g(}-
8:
until MM, converges.
9:
# Computing the normalized importance of parameters after training task t.
10: for - 1,................ ,L do
Compute loss L Eq. 2 for all parameters of i-th layer do
12:
13:
Y;
1,7 f Eq. 11)
for all parameters of i-th layer do
Y + Eq. 3), 7. + Eq. 44)
16:
Store only {y; <t } for future tasks.
e.g., Progressive Networks (PGN) (Rusu et al. 2016) and PathNet (Fernando et al. 2017). In par- ticular, PGN allocates sub-network for each task in advance, and progressively concatenates pre- vious sub-networks while freezing parameters that are allocated to previous tasks. PathNet splits each layer into multiple sub-modee aub-modules, and finds the best pathway designated to each task. In sum- mary, parameter isolation-basee methods suffer from vver-oonsumption of network capacity and have limited knowledge transfer ability, which the proposed method tries address.
PROPOSED SPG
As discussed in Section T the current parameter isolation approaches like HAT (Serra et al. 2018) and SupSup (Wortsman et 2020) are very effective for overcoming CF, but they hinder know ledge transfer and/or consume too much learning capacity of the network. For such model to improve knowledge transfer, it needs to decide which parameters can be shared and updated for new task. That the approach taken in CAT (Ke et al. 2020). CAT finds similar tasks and removes their masks for updating, but may find wrong similar tasks, which causes CF. Further, parameters the atomic information units, not neurons, which HAT masks. a neuron is masked, all parameters feeding into are masked, which costs huge amount of learning capacity. SPG directly soft-masks parameters based on their importance to previous tasks, which is a more flexible and uses much less learning space. Soft-masking clearly enables automatic knowledge transfer.
In SPG, the importance of a parameter to a task is computed based on its gradient. We do so because gradients of parameters directly and quantitatively reflect how much changing parameter affects the final loss. Additionally, we normalize the gradients of the parameters within each layer as we found that gradients in different layers can have very different magnitudes, which makes difficult to compute the relative importance reliably. The normalized importance scores are accumulated by which the corresponding gradients are reduced the optimization step to avoid forgetting the knowledge learned from the previous tasks. The whole algorithm is described in Algorithm
3.1 COMPUTING THE NORMALIZED IMPORTANCE OF PARAMETERS
The importance of each parameter to task t is computed right after completing the training of task t following these steps. Task i's training data, (X,, Yt), is given again to the trained model of task t, and the gradient of each parameter in i-th layer (i.e., each weigh or bias of each layer) is then computed and used for computing the importance
the parameter. Note that we use 0, (a vector) to represent all parameters of the i-th layer. This process does not update the model parameters. The reason that the importance is computed after training of the current task has converged as follows. Even after model converges, some parameters that changing those parameters may take the model out the (local) minimum leading to forget- ting. On the contrary, if all parameters have similar gradients (i.e, balanced directions of gradients),
have larger gradients, which indicate
3

---

MY
My
Task r's head
1
Task head
:
Mr
Task t's head
(a) After learning task
(b) After learning task t (t 7).
Figure/ 1: Cross-head importance (CHI). In the above figures, 'hţ and w/y represents the output of the i-th neuron in the 1-th layer just after training task / and the parameter in I-th layer connecting between the neurons 'h, to '1[+1, respectively. (a) The importance of w/y to task computed based on its gradient, OL /8wt.. normalized, and then accumulated. (b) After learning task t r), the state of related parameters might have been changed. To reflect importance to task again with the current neurons' output (c.g., 'h, rather than old hh(), an additional loss, L is computed at task r's head using task E's data as unlabeled data for task
changing the parameters will not likely change the model much to cause forgetting. Based on this assumption, we utilize the normalized gradients after training as signal to indicate such dangerous parameter updates. In our ablation experiments, we show that taking gradients before model con- vergence has similar effects. The proposed mechanism in SPG has the merit that it keeps the model flexible as does not fully block parameters using an importance threshold or binary masks. While HAT completely blocks important neurons, which results in the loss of trainable parameters over time, SPG allows most parameters remain "alive" even when most of them do not change much.
Additionally, computing the gradients based only the model (M₃) of the current task t does not deal with another issue. We use an example illustrated Fig. T For example, just after learning task T, the gradient of parameter is computed and normalized among the same layer's parameters to be accumulated. Even though during learning task t (t > 7) the parameter is not much changed considering its accumulated importance, the end of learning task 4, the state of related parameters might have been changed, by which the normalized importance may become less useful. To reflect the parameter's importance task 7 again in the current network state, we introduce cross-head importance (CHI) mechanism. In particular, an additional loss, Sum (M, (X,)). is computed with each previous task's head by substituting task t's data unlabeled data for the previous tasks. By taking this loss, parameters affecting the logits more for previous tasks are regarded more impor- tant. Finally, both the normalized importance computed for the current task's head and the ones for previous tasks' heads in CHI are considered by taking element-wis maximum, as shown in Eq. (3)
To put things together, the proposed method computes the normalized importance, 2/s of the set of parameters of the i-th layer, 0,, using each task r's model (1 < T S (), Mr, as follows:
OC )
%.t tanh Norm
(1)
00,
hh [MMM, (x, Yi)
(r=
x Mcan(x)
Norm(x)
(2)
Sum (Mr (X.))
(T<
VVr((z) +
* max (7t ,,,,"
(3)
where max (.) means element-wise maximum, and e, L.. a small value for stability and the loss function, respectively. Eq. ① normalizes the gradients over the same layer to avoid the discrepancies caused by large differences of gradient magnitudes in different layers. For the current task's head (when t), normal loss function (e.g., cross entropy) is used as L in Eq. (2). However, for each previous task's 's head (when T < !), since the current task data do not belong to any previous classes, the loss L is defined by Sum(M- (X)) over previous classes' logits in the proposed CHI mechanism. Essentially, this operation computes the importance of parameters based on the data of task t's impact on all tasks learned so far. Finally, to prevent forgetting as much as possible, we take the accumulated importance, 7i= follows:
St
7/ max (7). 7i (="))
where an all- zelozero vector is used as 7; This 7s depicts how important each parameter is to the learned tasks, i.e., from task 1 to task l.
4

---

Table 1: Statistics of the datasets. can take 10 and 20. Validation sets are used for early stopping.
Dataset
#Tasks
#Classes per task
#Train
#Validation
#Test
C-n
100/n /n
45, 000
5, 5,000
10, 000
T-n
200/n
90, 000
10, 000
10, 000
I-100
100
10
1, 000, 000
100, 000
50, 000
FC-n
2
400n
40n
80n
FE-n
7
62
3100n
310n
620n
Memory needed to parameter importance: Regardless of the number of tasks, at any time only the accumulated importance y is saved after the learning of each task that can be used again in the next task for Eq. (4) 7, has the same size as the number of parameters, [O:].
3.2 TRAINING PHASE
To suppress the update of important parameters in backward pass in learning task the gradients of all parameters are modified based on the accumulated importance as follow:
g - - (1-1- 7:5-)) 9:.
(5)
where gi and gf represent the original gradients of the parameters (0,) of the i-th layer and the mod- ified ones, which will be used in the actual optimization, respectively. Also, SPG does not employ anything special in the forward pass except it needs the task id to locate the correct classification head, which follows the standard TIL scenario. In testing, SPG behaves in the same way.
4 EXPERIMENTS
The proposed SPG is evaluated using 5 datasets. Their statistics are given in Table D Below, we use "-n" to depict that n tasks are created from each dataset (n takes 10 20). The first three datasets are used to create dissimilar tasks to evaluate the ability of SPG in overcoming CF.
Datasets:
The last two consist of similar tasks, which are used evaluate SPG's knowledge transfer ability.
(1) CIFAR100-n (C-n): CIFAR100 (Krizhevsky & Hinton 2009) is dataset that has images of 100 classes. We split it into n tasks so that each task has 100/ n classes. (2) TmnemmageNet-n (T-n): TinylmageNet (Wu et al. 2017) is modified subset of the original ImageNet (Russakovsky et al.) 2015) dataset, and has 200 classes. Each task contains 200/n n classes. (3) ImageNet-100 (I-100): ImageNet (Russakovsky et al. 2015) contains 1000 classes of objects. We split it to 100 tasks, each of which has 1O classes, to stress-test systems using large number of tasks and classes. (4) F- CelebA-n (FC-n): Federated CelebA (Liu et al. 2013)) is dataset of celebrities' face images with several attributes. We use it with binary labels indicating whether he/she in the image is smiling not. Each task consists of images of one celebrity. (5) F-EMNIST-n (FE-n): Federated EM- NIST (Liu et al. 2015) is a dataset that has 62 classes of hand-written symbols written by different persons. Each task consists of hand-written symbols of one person.
Baselines: We use 15 baselines. 10 of them are existing classical and most recent task incremental learning (TIL) systems, EWC KKirkparrick al.) 2017), (-177,A--EEA. A-GEM (Chaudhry et al. 2019). SI (Zenke
et al. 2017), UCL (Ahn et al. 2019). TAG (Malviya et al. 2022). PGN (Rusu et al. 2016)) Path- Net (ernando et al. 2017). HAT (Serra et al. 2018), CAT (ee etal. 2020). and SupSup (Worsmman ct al. (222)). Additionally. three simple methods are used for references: multi-task learning (MTL) that trains all the tasks together, one task learning (ONE) that learns a separate model/network for each task and thus has no CF or KT, and naive continual learning (NCL) that learns each new task without taking any care of previous tasks, i.e., no mechanism to deal with CF. Since HAT, our main baseline and perhaps the most effective TIL system, adopts AlexNet (Krizhevsky et al. 2012) as its backbone, all our experiments are conducted with AlexNet. For other baselines, their original codes used with switching their backbones AlexNet for fair comparison. Furthermore, to com- pare our soft-masking with the regularization-bat approach and our gradient- graddeen-bbased importance with Fisher information matrix (FI) based importance in EWC, more baselines EWC-GI and
5

---

SPG-FI are created. EWC-GI is EWC with its FI based importance replaced by gradient- based importance (GI) in Section 3.1 i... the same penalty/regularizationn in EWC is applied on our accu- mulated importance, 7,5h ",
in Eq. 4 when learning task t (no soft-masking). SPG-FI is SPG with our gradient-based importance replaced by FI based importance in EWC. We also examine some other baselines (e.g., hard-masking in SPG and method replaying all data) in Appendix 6-8.
Evaluation Metries: The following three metrics are used. Let a be the test accuracy of task i task just after model completes task j.
(1) Accuracy: The average of accuracy for all tasks of a dataset after learning the final task. It is computed by 1/T Et of where T is the total number of tasks in an experiment.
(2) Forward transfer: This measures how much the learning of previous tasks contributes to the learning of the current task. It is computed by 1/TE: (aj - -)), where 3r represents the test accuracy of task L in the ONE method, which learns each task separately.
(3) Backward transfer: This measures how the learning of the current task affects the performance
of the previous tasks. Negative values indicate forgetting. It is computed by 1/T Σ (a? o!).
4.1 TRAINING DETAILS
The networks are trained with SGD by minimizing the cross-entropy loss except for TAG, which uses the RMSProp optimizer as SGD-based TAG has not been provided by the authors. The mini- batch size is 64 except MTL that uses 640 for its stability to learn more tasks and classes together. flype-parameters. such as the dropout rate or each method's specific hyper-paramters, are searched based on Trre-structured Parzen Estimator (Bergstra et al. 2011). With the found best hyper- parameters, the experiments are conducted times with different seeds for each pair of dataset and a method, and the average accuracy and standard deviation are reported.
4.2 RESULTS: ACCURACY, FORWARD TRANSFER AND BACKWARD TRANSFER
Tables 2 and 4 report the accuracy, forward and backward transfer results, respectively. Each result in the tables is the average over 5 different seeds. Since CAT takes too much time to train, proportionally to the square of the number of tasks, we are unable get its results for I-100 (Ima- geNet with 100 tasks) due to our limited computational resources.
Dissimilar tasks (C-n, T-n. 1-100): MTL performs the best in all cases, but NCL performs poorly due to serious CF (negative backward transfer) as expected. While PGN, PathNet, HAT, CAT, and SupSup can achieve training with no forgetting (0 backward transfer), on average SPG clearly out- performs all of them. Although SPG slightly underperforms PathNet and SupSup in C-10 and C-20, their accuracy are markedly lower in the other settings due to PathNet's ss capacity problem (see Sec- tion 4.3) and SupSup's lack of knowledge sharing. The backward transfer results in Table 0 show that SPG has slight forgetting. However, its positive forward transfer results in Table more than make up for the forgetting and give SPG the best final accuracy in most cases. As explained in Section [] the regularization based approach is closely related to our work. However, the represen- tative methods, FWC, SI and UCL., perform poorly (see Table due to serious CF (see Table 4 that cannot be compensated by their positive forward transfer (see Table (3). Although TAG has almost no CF, its forward transfer is limited, resulting poorer final accuracy. Comparing our gradient based importance (GI) and Fisher information (FI) matrix based importance, we observe that EWC- GI outperforms EWC except for C-20 (EWC is only 1% better), and SPG outperforms SPG-FI. Comparing soft- masking and regularization using the same importance measure, we can see SPG markedly better than EWC-GI except for T-10 (EWC-GI only 1 1% better), and SPG-FI is also better than EWC except for C-10 (EWC is only 1 1% better).
Similar tasks (FC-n, FE-n): Table 2 shows that SPG almost as strong as MTL., and achieves the best performance among all baselines due to its positive forward (see Table3) and positive backward knowledge transfer ability (see Table NCL, A-GEM, UCL, and SI also perform well with positive or very little negative backward transfer since the tasks are similar and CF hardly happens. TAG underperforms them due to it's limited transfer. PGN, PathNet, and HAT have lower accuracy as their forward transfer is limited (i.e., they just reuse learned parameters in the forward pass) and no positive backward transfer. SupSup, which does not have any mechanism for knowledge transfer, results in much lower performance. As expected, it has backward transfer (see Table 4. Its

---

Table 2: Accuracy results. Best methods in each dataset are emphasized in bold, and second best methods are underlined (same in the following tables).
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FF-10
FR-20
Avg.
(MTL)
0.76
a.78
0.53
0. m0.0
0.65
0. 8888mm
0.88am
0.862mm
a.87
0.76
(ONE)
0.67
0.76=am
0.44-aa4
0.54-00)
4.490mm
0. 75
0. 0.79
0.81
0.80
0.67
NCC
0.5
"0.54-500
0.37=ka7
0. 10.00
=0
0.862
0.862am
0.002
A-GEM
0.51
0.57
0.36
a.32
0. 1883nn
0.83a
0.87amm
a.87
0.62
PGN
0.61=am
0.74=a00
0,40=
0.54-6.)
0. 0.33=
0.
0e
0.82+amt
+.81
0.64
PathNet
0.68
0.72 aoo3
0.48 auo
0.50
0.47 anno
0.77
0.80
0.85mmm
1.84
0.68
HAT
0.63-oo
0.72 oo.2
0.46-ea1 46--
0.52 o..se
0.45-1om
4.79
0.82
0.84.am
0.85
0.67
CAT
0.64:omm
0.74
0. 0.
0.51=
N/A
0 0.83amm
0.832mm
0.83amm
0.84
N/A
SupSup
0.66:000
0.76
0.44
0.54
0. 0.49
0. 0.71
0.72
0.81=am
0.80sam
0.66
UCL
0.65
0. 0.74
0.45
0.55
0.37
0.86
0.87
0.852mm
0.85
0.69
0.63
0. 0.70-
0.46--001
0.53-
0.44
0.86-10.0
0.87:000
1.88.am
0.88
0.69
TAG
0.61
0.68
0.43
0.50
0.45
0.74
0.77
0.84mm
0.84
0.65
EWC
0.62.oom
0.61
0.37 ...0
0.41 .o.i
m.25
0.81 100
0.86
1.87
0.87
0.63
BWC-GI
0.63
0.60 om
0.48 out
0.49 o..0
0.53 1n.3
1.83
0.85
1.86
0.87.om
0.68
SPG-FI
0.61aom
0.68
0.44
0. 19
0.86
0,87
0.86amm
2.88
0.69
SPG
0.66=a01
0. 7.000
0.47=000
0.57=500
0.55-0.0
0.86
0,87
0.87
0.87
0.72
Table 3: Forward transfer results. MTL has no results here as it learns all tasks together.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FE-20
Avg.
NCL
0.07
-0.03
-0.03
-0.04
-0.01
+0.08
+0.06
+0.04
+0.06
+0.01
A A-EEM
0.03
-0.01
-0.04
-0.04
0.01
+0.08
+0005
+0.04
+0.06
+0.01
PGN
0.06
-0.03
-0.44
-0.01
-016
0.00
-0.04
+0.01
+0.01
-0.04
PathNet
+0.01
-0.05
+0.05
-0.04
-.02
+0.02
+0.02
+0.04
+0.05
+0.01
HAT
-0.04
-0.05
+0.02
-0.03
00.44
+0.04
+0.03
+0.03
+0.05
0.00
CAT
-0.03
-0.03
0.00
-0.04
N/A
+00.88
+0.04
+0.02
+0.04
N/A
SupSup
0.01
0.01
0.00
0.00
0.00
-..4
-0.07
0.00
0.00
-0.01
UCL
+0.04
+0.06
+0.05
+0.08
+0.03
+0008
+0.06
+0.03
+0.05
+0.05
SI
+0.07
+0.04
+0.13
+0.09
+0.13
+0.07
+0.06
+0.06
+0.08
+0.08
TAG
-0.06
-0.07
0.00
-0.04
-0.05
-0.01
-0.02
+0.03
+0.04
-0.02
EWC
0.00
-0.03
-0.04
-0.06
-0.01
+0.08
+0.06
+0.05
+0.07
+0.01
FWC-GI
-0.02
+0.02
+0.06
+0.07
+0.12
+0.09
+0.06
+0.04
+0.06
+0.06
SPG-FI
+0.01
0..1
+0.05
+0.03
+0.10
+0.09
+0.06
+0.04
+0.08
+0.05
SPG
+0.04
+0.03
+0.03
+0.06
+ +0.08
+0.09
+0,06
+0.05
+0.07
+0006
negative forward transfer for FC-10 and FC-20 is because it does not learn parameters usual but masks, which is weaker as we observe. CAT is slightly better due to its stronger positive forward transfer and no negative backward transfer. We also observe that SPG-FI and EWC-GI perform similarly to SPG and EWC as suppressing updates of important parameters becomes less critical in similar tasks and thus the choice of GI FI is not important. EWC-GI and EWC are worse than SPG and SPG-FI in FC-10 and FC-20, indicating regularization may harm the learning of new tasks.
Summary: SPG markedly outperforms all the baselines. When tasks are dissimilar, its positive forward transfer capability overcomes its slight forgetting (negative backward transfer) and achieves the best or competitive accuracy results. has the strong positive forward transfer even with dissim- ilar tasks, which has not been realized by the other parameter isolation-barl based basclines. When tasks are similar, it has both positive forward and backward transfer to achieve the best accuracy results by not completely blocking parameters parameter isolation- based methods do but only partially blocking them. Moreover, soft- most-na (SPG and SPG-FI) is better than regularization (EWC-GI and EWC), and our gradient-based importance (SPG and EWC-GI) is better than Fl (SPG-FI and EWC). We report the amount of compute and the network size in Tables 9 and 10 in Appendix.
4.3 CAPACITY CONSUMPTION
The reason that PGN, PathNet, HAT, and CAT have lower accuracy on average than SPG despite the fact that they can learn with no forgetting (see backward transfer in Table 4) is mainly because
7

---

Table 4: Backward transfer results. MTL has no results here as learns all tasks together.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FF-20
Avg.
NCL
-0.09
-0.20
-0.03
-0.09
-0.18
+-0.2
-0.01
+0.01
+0.01
-0.06
A-GEM
-0.13
-0.21
-0.04
-0.09
-0.16
0.00
-0.01
+0.01
+0.01
-0.07
PGN
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
PathNet
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
HAT
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
CAT
0.00
0.00
0.00
0.00
N/A
0.00
0.00
0.00
0.00
N/A
SupSup
0.00
0.00
0,00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
UCL
0.06
- 0.099
- 0.03
0.07
0.14
+0.03
+0,02
+0001
+0001
-0.03 -..04
SI
0.12
-0.05
0.11
00100
-0.17
+0,05
+0,02
+0.01
+0.01
-0.05
TAG
0.01
0..02
0.00
0.01
+0.01
0.00
0.00
0.00
0.00
0.00
EWC
0.06
0.13
0.04
0.07
0.22
0..01
+0.02
+0.01
0.00
0.06
EWC-GI
0,01
0.13
0.00
0.
0,08
"0.00
0.00
+0.02
70,02
7.00
SPG-FI
-007
-0.07
0.04
0007
-..10
+0.03
+0.02
+0.02
+0.01
-0.03
SPG
0.04
-0.04
0.00
0.03
00.02
+0.02
+0.02
+0.01
+0.01
0.01
Table 5: Each cell reports how many percentage of parameters are completely blocked (i.e., impor- tance 1) just after learning task is the total number of tasks (e.g.. = 10 for C-10).
Dissimilar tasks
Similar tasks
t
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FE-20
HAT
1.11%
15.91%
2.88%
0.16%
23.50%
0.28%
6.29%
23.94%
19.03%
1
SPG
0.09%
0.14%
0.09%
0.05%
0.11%
0.07%
0.16%
0.13%
0.13%
HAT
22.14%
'822%%
36.68%
22 06.66%
99.82%
3.00%
65.13%
36. 81%
"3.55%
T/2
SPG
0.74%
1.78%
0.50%
0.83%
2.86%
0.42%
1.23%
0.96%
1.52%
HAT
41.76%
'9.66%
37.46%
338. 68.64%%
90.88%
9.76%
70.50%
86%8
"9.55%%%
T
SPG
1.53%
2.76%
0.84%
1.43%
4.18%
0.71%
2.03%
1.58%
2.27%
they suffer from the capacity problem, which is also indirectly reflected in lower forward transfer in Table 3 Although SupSup does not suffer from this problem, its architecture prevents transfer and gives markedly poorer performances especially in similar tasks (see Table 2 As discussed in aaattion-nbsddd methods freeze sub-network for each task, as more
S since these parameter tasks are learned, the capacity of the network left for learning new knowledge becomes less and less except for SupSup, which leads to poorer performance for later tasks. Table 5 shows the percentage of parameters in the whole network that are completely blocked by HAT and SPG (layer-wise results are given in Table 7 in Appendix). can be seen that SPG blocks much fewer parameters than HAT does in all cases. This advantage allows SPG to have more flexibility and capacity to learn while mitigating forgetting, which leads to better performances.
Fig. 2 plots the forward transfer of each task. A positive value in the figures means that a method's forward test result (the test result of the task obtained right after the task is learned) better than ONE, benefited from the forward knowledge transfer. We can clearly observe downward trend of these basclines for dissimilar tasks ((a)-c)), especially (a), (b), (d), and (c). We believe that this is due to the capacity problem, i.e., as more tasks are learned, they gradually lose their learning capacity. The reason that PGN increases its performance at the beginning is because PGN splits the model into sub-networks for all tasks in advance, and when new task comes, all previous sub- networks are combined to produce the model, which thus gradually improves the learning capability and then stabilizes. However, its performance significantly lower than others overall. On the other hand, SPG shows a upward trend in all cases, even for dissimilar tasks. due to its positive forward transfer, which indicates SPG has much higher capacity to learn. For Figure (f), the difference is not obvious as the tasks are similar, which can share many parameters and has less capacity issue. However, their accuracy in Table 2 are inferior SPG due to their weaker forward transfer.
4.4 ABLATION STUDIES
SPG has two mechanisms that can vary. The first one is with or without cross-head importance (CHI), as introduced in Eq. (3). and the other is when compute the importance of parameters. The
8

---

Forward traeder C-10
Forward transter on C-20
Forward transfer on T:16
0.00
-0.
-030
-0:5
0.15
10
(a)
Forward Stanafer 120
transferr
Furward transfer FE-20
0.2
-03
-21
#lides
(d)
(1)
Figure 2: We plot the forward transfer of each task along with the number of tasks learned. (a) to (e) are the plots with dissimilar tasks, while (f) is the one with similar tasks. Three additional figures given in Fig. in Appendix.
Table 6:
The results for the ablation studies. The averaged accuracy presented. The averaged convergence epoch per dataset also shown as "Cvg." for reference.
Dissimilar tasks
Similar tasks
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-
FE-20
Cvg.
53
53
39
45
40
66
53
52
50
SPG
0.66-cm
0.75:0m
0.47-am
0.37.000
0.55.cm
0.87-im
"..77am
"0.87-6x0
@cl
0.65.om
0.74.um
0.47.am:
0.56mmm
0.58.mm
0.86
0.87.mm
0.88
0.88
@e10
0.66.cm
0.75
0.47.com o.0
0.56
0,55
m.86
0.87
0.87
0.88
@e20
0.65
0.74:om
0.47-am
0.56:am
0.55.am
0.86
0.86
0.87.0m
0.88-..m
-CHI
0.64
0.73
0.47
0.56mmm
0.55.am
0,87 amus
0.87:nmm
0.88 .
0.88
reported results so far are based on the importance computed using gradients after model conver- gence. also possible to compute the importance in earlier epochs before model convergence.
The ablation results presented in Table 6 where "@en" means SPG employing gradients ob- tained after epoch n (e.g., 1, 10, 20), instead of after convergence (SPG itself), for parameter impor- tance computation. "-CHI" means SPG without cross-head importance (CHI) where 7 is replaced with y in Eq. 3) without using the previous tasks' heads. We can observe from the results that the timing when the gradients are taken in importance computation has little impact on the final accuracy. implies that the gradient direction does change much from the beginning of training. This is perhaps not so surprising because each task does not train from scratch but based on exist- ing network trained from old tasks and what parameters need significant changes are roughly fixed. About CHI, it improves the performance of dissimilar tasks, but may sacrifice little in knowledge transfer in similar tasks because this mechanism blocks more gradient flow of parameters. However, even with CHI, SPG performs the best for similar tasks as shown in Table Given these results, we believe that the advantage CHI outweighs its disadvantage. More analysis of how CHI contributes to suppressing parameter updates is given in Table g in Appendix.
5 CONCLUSION
To overcome the difficulty of balancing forgetting prevention and knowledge transfer in continual learning, we proposed novel and simple method, called SPG, that blocks/masks parameters not completely but partially to give the model more flexibility and capacity to learn. The proposed soft- masking mechanism not only overcomes CF but also performs knowledge transfer automatically. Conceptually, it is related to the regularization approach, but we have argued and evaluated, markedly outperforms the regularization approach. Extensive experiments have demonstrated that SPG markedly outperforms strong baselines.

---

REFERENCES
Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Learning with Adaptive Regularization. In Proc. of NeurIPS, 2019.
Uncertainty-based Continual
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caue CaceiCa Online Continual Learning with Maximal Interfered Retrieval. In Proc. of NeurlPS, 2019.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for Hyper-Parameter Parameter Optimization. In Proc. of NeurIPS, 2011.
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient Lifelong Learning with A-GEM. In Proc. of ICLR, 2019.
Kamil Deja, Pawel Wawrzyúski, Daniel Marczak, Wojeiech Masarezyk, and Tomasz Trzciński. BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learning. In Proc. of
IJCNN, 2021.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. PathNet: Evolution Channels Gradient Descent in Super Neural Networks, 2017.
Zixuan Ke, Bing Liu, and Xingchang Huang. Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks. In Proc. of NeurlPS, 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. In Proc. of NAS, 2017.
Alex Krizhevsky and Geoffrey Hinton. 2009.
Learning Multiple Layers of Features from Tiny Images,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. ImageNet Classification with Deep Con- volutional Neural Networks. In Proc. of NeurlPS, 2012.
Zhizhong Li and Derek Hoiem. Learning without Forgetting. In Proc. of ECCV, 2016.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. In Proc. of ICCV, 2015.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient Episodic Memory for Continual Learning. In Proc. of NeurlPS, 2017.
Pranshu Malviya, Balaraman Ravindran, and Sarath Chandar. TAG: Task-based Accumulated Gra- dients for Lifelong Learning. Proc. of CoLLAs, 2022.
Sylvestre-Alviis Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incremental Classifier and Representation Learning. In Proc. of CVPR, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interfer- ence. In Proc. of ICLR, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukeuoglu, Razvan Pascanu, and Raia Hadsell. Progressive Neural Networks, 2016.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming Catastrophic Forgetting with Hard Attention to the Task. In Proc. of ICML., 2018.
10

---

Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual Learning with Deep Gener- ative Replay. In Proc. of NeurIPS, 2017.
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in Superposition. In Proc. of NeurIPS, 2020.
Jiayu Wu, Qixiang Zhang, and Guoxi Xu. Tiny ImageNet Challenge, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual Learning Through Synaptic Intelli- gence. In Proc. of ICML. 2017.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C. C. Jay Kuo. Class-incremental Learning via Deep Model Consolidation. In Proc. of WACV, 2020.

---

A APPENDIX
A.1 FORWARD TRANSFER
Fig. in the main text of the paper shows the forward transfer plots for some datasets but not all due space limitations. Fig. 3 here presents the forward transfer results for all datasets. It can be clearly seen that SPG has the best positive forward transfer and keeps it constantly in almost all cases. On the other hand, the other parameter isolation-basee based methods, PGN, PathNet, HAT, and CAT lose their ability for the forward transfer especially in later tasks for the dissimilar task experiments (i.e., (a) to (e)).
on
0.02
terente
0.00
0.07
-0.1-
0.00
-0.12-
-02
OnE
MA
CAT
Falinee
FUN
-0.33
3
10
10
15
5
(a)
(h)
Forward transfer 1-20
Forward transter on 100
Forward transfer on FC
tcreed Coserie
0.2
0.2
-0.1
-0.1
-02
-0.3
13
15
20
30
40
25
(d)
on FC-20
Forward transter on FE 20
Forward
0.
tewwert www.ctrnnee
0.1
Torwed
0.1
0.2
-0.3
-0.1
13
10 W'addo
(g)
(h)
Figure 3: Forward transfer results. (a) to (e) are for dissimilar tasks, and (f) to (i) for similar tasks.
A.2 CAPACITY CONSUMPTION AT EACH LAYER
We show the percentage of parameters in the whole network that are fully blocked in Table the main text of the paper. Here Table presents the same result for each layer. We use AlexNet as the backbone, and it has three convolution layers followed by two fully-connected layers.
As we described in Section 4.3 SPG blocks much fewer parameters than what HAT does in all cases. Additionally, we can see from Table 7 the significant difference between HAT and SPG in their layerlayrrwwie wise tendency. HAT blocks more parameters earlier layers (e.g., after learning task 5 of C-10, 77% of parameters in the 1st convolution layer are completely blocked while 52% of the ones in the 2nd convolution layer are), which is reasonable given that the earlier layers are supposed to extract basic features and thus changing their parameters without being blocked could easily cause more forgetting than later layers. On the other hand, SPG contrarily tends to completely block more parameters in later layers (e.g., after learning task 5 of C-10, 0% of parameters in the Ist convolution layer are completely blocked while 0.47% of ones in 2nd convolution layer are). Since SPG computes parameters' importance based normalization, this result implics that later layers are likely to include more parameters on which some of the tasks highly depend. It can be said that SPG keeps earlier layers alive with less blocking
their gradients with regard to the loss through
12

---

Table 7: The percentage of parameters that are completely blocked for each layer.
(a) Results for the 1st convolution layer.
Dissimilar tasks
Similar tasks
1
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FB-10
FF-20
HAT
25.00%
24.38%
31.56%
14.38%
60.00%
17.81%
40.00%
75.00%
67.19%
SPG
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
0.00%
HAT
77.81%
98.14%
83.44%
75.06%
100.00%
73.44%
100.00%
98.75%
9.33%
T/2
SPG
0.00%
0.00%
0.00%
0.00%
0.11%
0.00%
0.00%
0.00%
0.00%
HAT
95.62%
100.00%
97.19%
95.31%
100.00%
98.75%
100.00%
100.00%
100.00%
SPG
0.00%
0.00%
0.00%
0.02%
0.22%
0.00%
0.00%
0.00%
0.00%
(b) Results for the 2nd convolution layer.
Dissimilar tasks
Similar tasks
t.
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FR-10
FF-20
HAT
5.63%
7.26%
8.93%
1.54%
33.94%
2.23%
11.98%
55.69%
45.66 45.66%
1
SPG
0.07%
0.17%
0.02%
0.06%
0.03%
0.01%
0.02%
0.12%
0.09%
HAT
32.42%
99.98%
63.74%
36. 18%
106.00%
41.25%
08.28%
98.4%%
90.2%%
T/2
SPG
0.47%
1.92%
0.11%
0.32%
0.52%
0.03%
0.17%
0.59%
0.61%
HAT
80.96%
99.88%
90.21%
86, 48.40%
106.00%
86.87%
99.38%
100.00%
100.00%
T
SPG
0.93%
2.54%
0.14%
0.37%
0.73%
0.05%
0.25%
0.85%
(c) Results for the convolution layer.
Dissimilar tasks
Similar tasks
t
Model
C-10
C-20
T-10
T-20
1-100
FC-10
FC-20
FE-10
FE-20
HAT
4.79%
8.73%
7.45%
0.73%
30.57%
0.99%
7.97%
43.67%
36.43%
1
SPG
0.08%
0.19%
0.06%
0.04%
0.12%
0.04%
0.15%
0.09%
0.06%
HAT
42.62%
98.68%
60.20%
47.14%
100.00%
14.67%
88.72%
99.97%
99.14%
T/2
SPG
0.61%
2.24%
0.27%
0.55%
0.54%
0.27%
1.12%
0.43%
0.43%
HAT
66.78%
99.69%
87.09%
70.10%
100.00%
43.1%%
95.27%
79..77%
99.34%
T
SPG
1.14%
2.87%
0.36%
0.77%
0.66%
0.41%
1.62%
0.59%
0.59%
(d) Results for the 4th fully-connected layer.
Dissimilar tasks
Similar tasks
1
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FE-20
HAT
2.70%
12.59%
3.95%
0.21%
25.68%
0.34%
6.48%
27. 27.72%
22.65%
1
SPG
0.07%
0.09%
0.09%
0.03%
0.11%
0.07%
0.17%
0.13%
0.12%
HAT
29.71%
98. 98.27%
47.66%
29.43%
99.88%
3.77%
70.90%
90. 18%
95.64%
T/2
SPG
0.58%
1.52%
0.38%
0.57%
1.06%
0.17%
1.30%
0.75%
1.10%
HAT 52.36%
99.67%
72.28%
50.64%
99.91%
12.11%
84.04%
98.55%
98.20%
T
SPG
1.16%
2.20%
0.51%
0.82%
1.24%
0.81%
2.29%
1.17%
1.58%
(e) Results for the 5th fully-comnected layer.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FE-20
HAT
1.34%
17.95%
2.07%
0.08%
21.97%
0.18%
6.02%
20.83%
16.16%
SPG
0.10%
0.16%
0.09%
0.06%
0.11%
0.08%
0.16%
0.16%
0.14%
HÄT
17.60%
98.2%%
29.91%
13.30%
59.15%
2.15%
61.82%
3I.56%
02.35%
T/2
SPG
0.82%
1.90%
0.57%
0.97%
3.88%
0.40%
1.22%
1.08%
1.76%
HAT
37.95%
9937%%
78.51%
30.65%
90.86%
6.11%
76.38%
97.11%
97.00%
T
SPG
1.73%
3.05%
1.03%
1.77%
5.82%
0.68%
1.95%
1.82%
2.67%
for better basic feature learning (i.e., leading to positive knowledge transfer) while blocks some specific parameters in later layers that supposed to be more important for previous tasks, which different from what HAT does. Again, we see that SPG fully blocks only a tiny percentage of parameters.
13

---

Table 8: Quantitative contribution of CHI in learning task i' the total number of tasks (e.g.,
T = 10 for C-10). The average and standard deviation over different seeds are presented.
C-10
C0
T-10
G-each
F-total
2
0.34axee
0. 0.10mmn
0.08
0.30
0.00
0.20
0.00
0.47
0.10
0.30
0.10
0.61 -eoe
0.22-m0x
0.20-
0.04-1aa
0.70-1as
0.02-6o8
0.69-o0e
0.23-mm
0.23-va
0.04-1.0
0.75
0.28anni
0.13
0.01
0.85
0.34
0.064cma
0.00aaee
0.79ums
0.29
0.14
0.01
T-20
1-100
PC-10
F-each
G-each
F-toral
G-toral
F-each
G-each
F-total
G-total
F-cach
G-each
F-toral
G-toral
2
0.43axu
0.11aon
0.27
a.09
0.56.sas
0.12.
a,37
0.12
0.31
0.08.vm
0. 0.200us:
0.07
0.78amon
0.30smnn
0.14
0.01
+.80
a.37
0.02-emm
0.00seox
0.49
1.aonn
0. 0.16m
0.02
7
0.87-
0.35
0.07 -.07
0.00~1.0
0.94
0.43-ra
0.02+cm
0.00xeoe
0.51 -nox
0. 18
0.06-aar
0.00-
FC-20
FE- 10
FE-20
Freach
G-each
F-total
G-total
Feeach
Greach
F-total
G-total
Freach
Greach
F-total
G-toral
0.26-ors
0.08-omt
0. 0.20
0.07-3m
0.28-1a7
0.08-r
0.20-cao
0.07-ao
0.37-mor
0.12-om
0.25-am
0.00-am
T/2
0.52seor
0.19
0.10
0.01
0.47+1a
0.174cm
0.174ca:
0.02+exx
0.60anmi
0.24.0m
0.13+sm
0.01-1m
T
0.60aamu
0.23amo
0.07
0.00
0.63.184
0,284ra2
0,14amx:
0.01ave
0.74
0.37amm
0.10aam
0.00aam
A.3 QUANTITATIVE ANALYSIS ON CROSS-HEAD IMPORTANCE (CHI)
We analyze in detail how CHI quantitatively contributes to suppressing parameter updates with the following four metrics.
(1) Overwrite Frequency at each task (F-each): How often does the importance from CHI have larger value than the one from the current task (1 means that it always happens)? corresponds to cases where 7, frr > 7, for any r(1 S r < t) in Eq. (3).
(2) Overwrite Cap at each task (G-each): When the cases of F-each happen, how much is the difference of overwriting average? It is defined by the average of max (r %7 1 Y
(3) Overwrite Frequency total (F-total): How often does the importance from CHI actually overwrite the accumulated importance through the maximum operation? It corresponds to cases where Y 7
<t-1 for any r(1 S 5 < in Eq. (4).
(4) Overwrite Gap total (G-total): When the cases of F-total happen, how much is the difference of overwriting on average? It is defined by the average of max (7, 1.1 ,7, 1 7, St-1
The result presented in Table We can clearly observe that CHI adds more importance some parameters (e.g., in C-10, about 13-23% of parameters constantly update their accumulated im- portance by the ones from CHI), which is denoted by F-total. Since introduce CHI further mitigate forgetting by accumulating more importance, this expectation is consistent with the ob- served results. Although CHI also overwrites the accumulated importance in similar tasks (c.g., FC-10), it overwrites less frequently (see F-each and F-total) with smaller gap overall (see G-each and G-total), which reasonable given that the tasks are similar.
A.4 COMPUTATION TIME
The computation time averaged over 5 different seeds is reported in Table For I-100, SPG requires a significantly shorter time to learn than other parameter isolation- based approaches (i.e., PGN to SupSup). This is mainly because these approaches suffer from the capacity problem and thus need a long time to learn with limited number of free parameters, which becomes remarkable with lot of tasks to learn (i.e., 100 tasks).
A.5 NETWORK SIZE
The number of learnable parameters of each system is presented in Table Note that all approaches adopt AlexNet their backbone, and the number of parameters vary depending their additional structures such as attention mechanisms or mub-modules. It also depends on datasets because each dataset has different number of tasks and in TII, each task has different classification head and
14

---

Table 9: Computation time for training in minutes.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FB-20
(MTL)
714
410
911
8_0
352as2
1
321
2
711
(ONE)
510
410
1211
11.0
13612
200
4
420
611
NCL
4+0
4+0
7+0
7-0
79-6
1-0
T+0
341
7+0
A-GEM
10+0
16+0
23+3
25+3
256+7
1-0
2+0
8+1
19=1
PGN
19+1
25+1
38+3
42+1
2918+76
2=0
5=0
15+1
45=2
PathNet
24
22+3
44+2
40s2
773+21
1=0
1=0
620
10=1
UCL
175:08
169
197:05
251
1650
6
11:1
79
94
HAT
14+1
14+4
22+1
29
1124+224
3=0
5:1
12tH
16:0
CAT
119:08
323=18
362+18
871
N/A
17+3
113-17
67+14
412:96
SupSup
29+6
29+6
75+0
58+1
507+3
1-0
1+0
20+0
44+1
SI
17+1
13+1
43+17
38+7
171+1
2-0
3+1
9+1
14+0
TAG
33+3
59+5
91+5
112+28
5821+24
2-0
4+1
11+1
26-4
EWC
15+1
16+1
19+0
27+0
375+1
1-0
3+0
540
16-1
EWC-GI
2812
2213
3914
37.5
4I91is
510
2315
39
SPG-FI
10.1
710
11.0
1315
239
1.0
110
710
121a
SPG
19+4
13#2
27++
28+1
386+30
1=0
2=0
13s0
20
Table 10: The number of learnable parameters of each model.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
1-100
FC-10
FC-20
FE-10
FE-20
(MTL)
6. 6,708.77 772
77708.72
6.913.672 672
6, 0.913. 672
4,552,877 872
6. 5.444, 852
6,585, 832
7. 730, 796
0.001. 176
(ONE)
708,
708, 772
913113
512.
665444,
6, 6.685, 8322
7. 730,
0.000..11
176
NCC
6. 708, 772
6. 708, 772
0,013. 072
6. 91.3, 672
K. 532, 8772"
0,51C 852
66585, R52
7. 799.7766
0.7001.
176
A-GEM
6, 708,
6, 708. 772
013, 672
6, 913, 672
552, 872
6, 6.444 852
(1.555,88 832
7. 730. 796
U01. 176
PGI
3,367, 187
717.711
100
5813
PathNNl
19, 503,
170770 840
22, 086, 648
807, 392
18, 497, 4.9
775,374
5,888, 586
4. 1.9.22.
270. 940
HAT
6, 754, 2112
6, 799 662
6. 969. 112
7. 0044 552
0000 277
6, 500) 2502
6, 676666 7122
7. 776. 236
9. 0822 056
CAT
29, 523, 228
30, 653, 972
40, 801. 912
40, 915, 272
N/A
38, 531, 3888
38, 906, 702
46, 226, 660
55, 131, 480
SupSup
UCL
6, 316
6, 713, 316
216
918,
6, 5..499 3066
5.0000 376
7. 735.
005, 720
SI
6. 71,7,8 772
6, 708, 772
6. 672
6. 9131
8, 5.552, 872
6. 2-44 852
6, 585, 832
7. 730, 796
9. 000.001.. 176
TAG
6, 708.
6, 708. 772
913. 672
6.913, 672
552, 872
6, 6,544, 852
58558882
7. 730. 796
081. 176
EWC
772
7.
EWC-OT
6, 707..7777 7/22"
708
0.013.002
66 91.5 872
5:2.372
6,51C 852
67585, B392
7:907796
SPG-FI
6, 708, 772
6, 708, 772
6, 913
6, 913, 672
8, 552, 872
6, 5.544, 852
6, 585, 8332
7. 730, 796
9. 001, 176
SPG
778,
6,708, 772
613.
6, 679...672
532,
6, 54444, 852
6, 832
7. 73700,700
0..001, 176
the number of units in each classification head depends on the number classes each task. It can be seen that PathNet, CAT, and SupSup need more parameters than SPG and other approaches.
A.6 COMPARISON WITH NCL
The difference between each method and NCL. can be seen in Tables and each table gives the result of every method, including NCL. Here we only show the improvement of each method compared to NCL in terms of average accuracy, which indicates the forgetting of each system com- pared with NCL (Table Let ag and ç/ be the test accuracy of task just after learning task by each method (as described in Section and by NCL,, respectively. The average accuracy difference in Table computed by 1/T E (af (F where T is the total number of tasks.
We can observe that SPG is the best overall (positive values indicate improvements over NCL). The best baseline is SI. Its performance similar SPG for similar tasks experiments as for these experiments there is a lot of knowledge sharing among tasks and thus little forgetting. However, for the dissimilar tasks experiments, SI is much worse than SPG due to much more serious forgetting.
15

---

Table 11: Accuracy difference compared to NCL.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE-10
FE-20
Avg.
A-GEM
0.00
0.03
-0.01
0.01
0.02
-0.01
-0.01
0.00
0.00
0.00
PGN
0.10
0.20
0.02
0.12
0.02
0.10
-0.10
-0.04
-0.06
0.02
PathNet
0.17
0.17
0.11
0.09
0.17
0.07
-0.04
-0.01
0.02
0.06
HAT
0.12
0.18
0.08
0.11
0.15
0.05
-0.02
0.02
- 0.02
0.06
CAT
0.13
0.20
0.06
0.10
N/A
0.01
0.01
0.03
0.02
N/A
SupSup
0.15
0.21
0.07
0.13
0.18
0.13
0.12
0.06
0.07
0.04
UCL
0.14
0.20
0.08
0.14
0.07
0.02
0.02
0.01
0.01
0.07
SI
0.12
0.16
0.09
0.11
0.14
0.02
0.03
0.01
0.01
0.08
TAG
0.10
0.14
0.06
0.08
0.14
0.10
0.07
0..0
-0.03
0.03
EWC
0.11
0.06
0.01
0.00
-0.0
0.03
0.02
0.01
0.00
0.01
EWC-Gi
0.12
0.06
0.11
0.07
0.22
0.01
0.01
0.00
0.01
0.07
SPG-FI
0.10
0.14
0.07
0.10
0.18
0.02
0.03
0.00
0.02
0.07
SPG
0.15
0.21
0.10
0.16
0.25
0.01
0.03
0.01
0.01
0.10
Table 12: Accuracy results with additional baselines, B1 and B2.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
1-20
1-100
PC-10
FC-20
FE-10
FE-20
Avg.
(MTL)
0.76amm
0.78
0.53.mm
0.60aoea
0.65aam
0.88anm
0.88
0.86.mm
0.87
0.76
(ONE)
0.67
0.76
0.44s0ne
0.54g
0. 0.492am
0. 0.75me
0.7 0.79ga0
0.81
0.80=am
0.67
"NC.."
0.5T Tooe
0.547emm
0.37 immi
T.T
0..T mmm
0.847am
0.87 Tam
0.86mmm
70.72
B1(L0)
0.60aam
0.78
0.47
0.55
N/A
0.86
0.87
0.87
0.85
N/A
B1(0.5)
0.65
0.74
0.43
0.50amm
N/A
0.87=0.00
0. 0.85
0. 0.85=000
N/A
B1(0.))
0.62 coo
0.74.ccm
0.40
1.51
N/A
0.85
0.87
0.85
0.85
N/A
0.70
0.79
2.47
2.58
48
0.85
0.862no
0.56-t00
SPG
0.66
0.75
0.47 ammm
0.57 soco
0.55
0.86
0.87
0.87 amm
0.87 20.00
0.72
A.7 COMPARISON WITH ADDITIONAL BASELINES
We have compared our SPG with existing baselines that follow the conventional continual learn- ing (CL) setting in Section in the main text. Here additionally compare SPG with two conventional CL baselines suggested by an anonymous reviewer.
The two baselines as follows: B1(r): method assuming that all data of the previous tasks available. means the ratio of previous (replay) data used in learning a new task (e.g., when r 1.0, all the previous data used). We use
1..00.5/0.1. B2: method that copies the network at
the end of each task, and does only fine-tuning of the copy to learn the new task. The trained model saved for the task. The results are presented in Table 12 We could not do B1 for I-100 due to our limited computational resources as it has too many tasks (100) and takes too much time.
For B1(r), it can be seen that the larger the r (1.0/0.5/0.1) is, the better the results (except for T-20), which is obviously because it can utilize more data. What is interesting is that even B (1(0.)), which uses all the previous data, can worse than MTL in some cases (e.g.. C-10, T- T-10, and T-20). We believe the reason is that sequentially establishing different decision boundaries for multiple tasks may sacrifice the effectiveness of learning. Although SPG weaker than (1(1.)) in dissimilar tasks, it outperforms B (0.5), which indicates SPG is more effecttvee at least than storing one half of the previous data. Note that SPG does not store any previous data. For similar tasks, B (O(5))) and B(0.1) weaker than SPG. B1(())) has the same results SPG.
For B2, it enjoys forward knowledge transfer, but since B2 is not a typical continual learning method, it has catastrophic forgetting (CF) issue. It is thus not surprising that B2 outperforms SPG on the first two dissimilar tasks (C-10 and C-20). But SPG shows competitive performance with B2 under CF of continual learning for T-10 and T-20, and significantly outperforms B2 on I-100.
16

---

Table 13: Accuracy results with additional baselines, B1 and B2.
Dissimilar tasks
Model
T-50
T-100
I-50
1-100
I-200
(ONE)
0.71+ +0.00
0.85-o00
0.40+0.00
0.49+a.00
0.60++.00
B2
0.72.000
1.84
0.3810.00
0.48an.11
0.62.1.00
SPG
0.71
0.841001
0.4510.0
0.55
0.66...00
Table 14: Accuracy results with HPG.
Dissimilar tasks
Similar tasks
Model
C-10
C-20
T-10
T-20
I-100
FC-10
FC-20
FE- 10
FE-20
Avg.
(MTL)
0.76
0.78
0.53
0.60.ma
0.65
m.88.um
0.88 .0.0
0.86.um
0.87
0.76
(ONE)
U. aa
0.76
U. U.44amm
0.54ano
0.49
0.752mm
0.79
0.81
0.80g
0.67
NCL
0.51am02
0.54
u.37
0.41ana
0.31
0.S4
0.84=
Ü. 1..66am
0.S6
0.62
HAT
0.63
0.72
0.46sam
0.52
0.45
0.79
0.82
0. 0t
O. 0.85=mo
0.67
SupSup
0.66.um
0.76.
0.44.um
0.54.m00
0.49
0.71
0.72...o
0.80 1.8
0.66
HPG
1.63
0.73 ooe
0.45..00
0.53 moo
0.49.om
0.87 mm.
0.86 .uos
0.87 10m
0.87 c.00
0.70
SPG
0.662um
0.75
0.47 amm
0.57 anco
0.55
0.86
0.87
0.87
0.87 20.00
0.72
To have better understanding of this interesting phenomenon, we conduct additional experiments with more tasks, T-50, T-100, I-50, and I-200. We show the results in Table 13
We can see that B2 is not better than SPG and in fact for the last three, it is actually significantly poorer than SPG. We believe the reason is as follows. The initialization of parameters affects the performance of learning of new task, and model that has learned task does not always provide good initialized parameters for the next task. For the ImageNet data, which is used for 1-50, I- I-100, and 1-200, it is more difficult dataset. In this case, the model parameters get optimized more specifically for each task, and the learning of the next task in B2 may start with parameters that are far from good initialization, which leads to poorer results than SPG. On the other hand, since SPG preserves knowledge from all previous tasks, the accumulated knowledge in the network enables the model utilize good set of parameters to learn the new tasks (see below for similar tasks as well) to produce better results, which is indicated by its strong transfer compared to ONE and also as described in Table 3 in the paper.
SPG is also consistently better than B2 for similar tasks. This is because of the limited knowledge transfer B2. Let the current task learned by B2 be t. B2 can only transfer knowledge from the t- 1 task model to task t. Although some knowledge of carlier tasks before t é-1 1 may still be in the model for - 1, it is largely forgotten because B2 has no mechanism to handle forgetting. Our SPG can preserve the task specific knowledge of each task and thus is able to leverage the knowledge from more and suitable tasks to learn task better.
A.8 COMPARISON BETWEEN SOFT-MASKING AND HARD-MASKING
HAT uses hard-masking (on neurons) and we have shown that our SPG (soft-masking parameters) outperforms HAT significantly. SupSup hard-masks parameters. SPG also outperforms SupSup. Although SPG directly uses importance score (0 to 1) mask parameters and does not have a threshold for hard-masking (0 or 1), we create hard mask version of SPG, called "HPG" (i.e., Hard-masking, not soft-masking) by applying a threshold convert an importance value (0 to 1) to binary value (0 or 1). For example, if the threshold is 0.5, HPG treats importance larger than 0.5 as I (blocking), otherwise 0 (not blocking). We search for the best threshold for each dataset from 0.1/0.3/0.5/0.7/00 7/0.9, and report the best accuracy results in Table 14
The results still show SPG is superior. For dissimilar tasks, HPG that does hard-masking is sig- nificantly worse than SPG, which means that it is difficult to manually control the threshold and it is more effective to use the importance scores as soft-masks (as in SPG). On the other hand, both HPG and SPG work similarly for similar tasks, which is reasonable for similar tasks, completely
17

---

blocking parameters is not as harmful as for dissimilar tasks because similar tasks share lot of knowledge or parameters.
18


Reviews:
[{"review_id": "vYu6cmOz2X", "reviewer": "Reviewer_5rH3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The goal of proposed method is to overcome the catastrophic forgetting in continual setting, while promoting the forward knowledge transfer, and it proposes a simple soft-masking approach applied to the gradients during back-propagation.", "summary_of_the_review": "- The method is simpler than other methods and the overall performance seems meaningful, but it is not well explained why the proposed method works better than others.", "strength_and_weaknesses": "(+) it simply uses gradient masks that can be easily computed by using gradient values.\n\n(+) the importance values can also be used in EWC framework, and this is computationally cheaper than using the Fisher information.\n\n(+) masking is only applied during the learning phase, and it is not required during the inference time.\n\n(-) the proposed method shows competitive performance compered to other methods, but it is not well-studied why it works better than others.\n\n(-) it is unclear why the loss $\\text{Sum}(\\mathcal{M}_\\tau(X_t))$ for the previous task $\\tau < t$ is useful to compute the importance value. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "pg_wnTcMYv", "reviewer": "Reviewer_s4uK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a new method (SPG) for task-incremental learning. SPG uses soft-masks to condition the full model for each task. Experiment results on benchmark datasets demonstrate the effectiveness of the proposed method.", "summary_of_the_review": "The overall quality of the work does not meet the standard of ICLR, given the limited novelty and simplicity of the topic. Substantial amount of work is required to improve the paper. Therefore, I would recommend rejection.", "strength_and_weaknesses": "The main strength of the paper is that it is relatively easy to follow and understand the high-level idea.\n\nWeaknesses:\n- The topic of task-incremental learning is not particular interesting, given its proven simplicity in prior work.\n- The core idea of the work is incremental: using importance mask to condition the model is common in architecture-based methods.\n- The empirical evaluation is questionable. For example, in Table 2, all methods in the table has very small standard deviation. I do not think it corresponds with reported results in prior work.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "C5RXOWhl_A", "reviewer": "Reviewer_7vyQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a novel task-incremental approach that diminishes the gradients based on their importance (SPG). \nSpecifically, the importance is calculated as the normalized gradients on the current and past tasks.\nFor the current task, the gradients are taken w.r.t. the log-likelihood of the data.\nFor past tasks, the logits' gradients are used instead.\n\nAn extensive empirical study shows that SPG is on average superior than task-incremental baselines.", "summary_of_the_review": "Simple and sensible approach.\n\nBecause of the weaknesses of the studied scenarios, the authors need some particular new baselines (see **weaknesses**)\n\n---- POST REBUTTAL ------\n\nI updated my score from a 5 to 6 in light of the added baselines.", "strength_and_weaknesses": "**Strengths**\n\nThe method is simple and sensible.\nThe empirical study is extensive and appropriate.\nMy intuition is that the reported results would generalize to most task-incremental scenarios.\n\n**Weaknesses**\n\nMy main concern is about the studied scenario.\nMy opinion is that task-incremental learning isn't aligned with practical use-cases and over-studied.\nI understand that it's a useful setting to study catastrophic forgetting.\nHowever, we are seeing increasing evidence that catastrophic forgetting isn't as catastrophic as previously thought [1], especially in task-incremental scenarios where one does not have to deal with the problem of calibrating a single-head on all tasks.\n\nFurthermore, most literature in this space often get away with omitting the replay baseline, which is challenging to outperform, at least when computational resources are correctly fixed throughout the methods.\n\nNow, I do think that in the realm of task-incremental papers, SPG is a good one.\nI'm willing to increase my score if the SPG survives the introduction of the following baselines:\n- replay where all past data is allowed (memory is cheap, so there's no need for artificially constraining to small buffers).\n    - try different current vs replay task balances\n- this one is really important: copy the network at the end of each task, and only finetune the copy\n    - this method is simple, doesn't incur forgetting, should allow for some forward transfer, and has the same computational requirements as other baselines.\n\n\n[1] Pretrained Language Model in Continual Learning: A Comparative Study\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject