Continual Learning Based on Sub-Networks and Task Similarity
Authors: Zixuan Ke
Bing Liu
Wenhan Xiong
Asli Celikyilmaz
Haoran Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Continual learning
NLP tasks
Task Similarity
Sub-network

Abstract: Continual learning (CL) has two main objectives: preventing catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. The existing literature mainly tries to overcome CF. Although some papers have focused on both CF and KT, they may still suffer from CF because of their ineffective handling of previous tasks and/or poor task similarity detection mechanisms to achieve KT. This work presents a new CL method that addresses the above issues. First, it overcomes CF by isolating the knowledge of each task via a learned mask that indicates a sub-network. Second, it proposes a novel technique to compute how important each mask is to the new task, which indicates how the new task is similar to an underlying old task. Similar tasks can share the same mask/subnetwork for KT, while dissimilar tasks use different masks/sub-networks for CF prevention. Comprehensive experiments have been conducted using a range of NLP problems, including classification, generation, and extraction to show that the proposed method consistently outperforms prior state-of-the-art baselines.

Under review as a conference paper at ICLR 2023
CONTINUAL LEARNING BASED ON SUB-NETWORKS AND TASK SIMILARITY
Anonymous authors Paper under double-blind review
ABSTRACT
Continual learning (CL) has two main objectives: preventing (CF) and encouraging knowledge transfer (KT) across tasks. The existing literature mainly tries to overcome CF. Although some papers have focused on both CF and KT, they may still suffer from CF because of their ineffective handling of previous tasks and/or poor task similarity detection mechanisms achieve KT. This work presents a new CL method that addresses the above issues. First, it overcomes CF by isolating the knowledge of each task via learned mask that indicates sub-network. Second, it proposes novel technique to compute how important each mask to the new task, which indicates how the new task is similar underlying old task. Similar tasks can share the same mask/sub- network for KT, while dissimilar tasks use different prevention. Comprehensive experiments have been conducted using range of NLP problems, including classification, generation, and extraction to show that the proposed method consistently outperforms prior state-of state-of-the-att baselines !
forgeo
for CF
INTRODUCTION
This paper studies continual learning (CL) of a sequence of natural language processing (NLP) tasks in the task continual learning (Task-CL) setting. It deals with both catastrophic forgetting (CF) (McCloskey & Cohen 1989) and knowledge transfer (KT) across tasks. In Task-CL, the task ID is provided for each test case testing. In learning, after a task is learned, its data is no longer accessible. Another CL setting is class continual learning (Class-CL), which provides no task ID in testing and it solves different type of problems.
Existing research in CL. has almost exclusively focused on overcoming CF (Kirkpatrick et al. 2016 Serrà al. 2018; Wortsman et al.) 20200. Limited work has being done on KT except (Ke et al. 2020: 2021| Wang et al., 2022). But KT is particularly important for NLP because many tasks in NLP share similar knowledge that can be leveraged to achieve better accuracy. We humans are also particularly good at leveraging prior knowledge to help learn new skills. To achieve KT in learning a new task, CAT (Ke et al.] 2020) first detects previous tasks that are similar to the current task so that the current task learning can leverage the knowledge learned from the similar past tasks. CAT uses the hard-attention mechanism in HAT (Serrà et al., 2018) to deal with CF, which masks out those important neurons for each task that the training of new tasks cannot change them in back-propii poo-ppropagation. However, different tasks can share neurons. This approach has major problem for KT that is very hard, if not impossible, to solve. After the similar previous tasks are detected, CAT opens the masks of these tasks so that the new task learning can modify their parameters to achieve both forward and backward KT. This clearly helps KT. But this can cause CF for dissimilar tasks that share parameters with those similar tasks. CAT's task similarity comparison method based on the transfer learning performance can be quite inaccurate too. The KT methods in (Ke et al.. 202T Wang et al. 2022) are based on instance-level feature similarity comparison using dot product or cosine, which can be inaccurate as well (see experiment results in Sec. 4.2)
To deal with these issues, we would like to have (I) learning method that can isolate the knowledge of each task without parameter overlapping among tasks to deal with harmful interference in KT and (2) task similarity detection method that is directly related to the loss of previous tasks (even though we do not have their data) for more accurate similar task detection.
'The code has been uploaded supplementary materials.

---

Under review as a conference paper at ICLR 2023
For (1), we draw inspiration from the sub-network masking idea in (Wortsman al. 2020), where the underlying backbone network is fixed but a binary mask is learned to find a sub-network for each task, which encodes the model for the task. The mask is basically set of binary gates that indicates which parameters in the backbone network should be used for a task model. Thus, different task models have no interference cause CF although sub-networks of multiple tasks can share neurons and parameters because the underlying backbone network is fixed and shared by all tasks, Although this helps learn a sub-network to achieve no interference (CF) in transfer, the original (Wortsman et al., 2020), by design, cannot do KT. It is still very challenging to detect task similarity and to know what level of similarity similar enough to ensure positive transfer. If wrong similarity threshold is used, CF will be serious. To this end, we propose novel method to detect similarity in (2).
For (2), we propose to determine whether a previous task is similar to the current task by assessing the importance of the mask (which represents the model or sub-network of a task) for the previous task & to the current task t. To compute the importance score, we make use of an effective idea from the network pruning community (Michel et al. 2019). In Michel et al. (2019), the gradient on each parameter is serving as the importance of the parameter. The less important parameters (determined by a threshold) are regarded unimportant and removed to reduce the network size. However, it is not obvious how we can compute the importance of each mask with its sub-network to the current task and how determine whether the current task is similar enough task based on the importance so that they can perform KT. This paper proposes a novel method to perform the above two functions. set of virtual/dummy gate variables are introduced represent the mask/sub-netwwod so that we can compute the gradient of each mask/sub-network. The gradient, based on the current task data and directly related current task loss, serves as the importance of the mask/sub- networkrrrr to the current task. The more important a mask/sub-network is, it is more likely that the previous task that has used the mask is similar the current task. To mitigate the possible forgetting, novel importance comparison mechanism is also proposed to take the previous task gradient into account.
previous
Based on the proposed idea, a new method, called TST (Task-CL based on Sub-networks and Task similarity), is proposed. TST is evaluated using datasets for classification, generation, and extraction with similar tasks and dissimilar tasks. The results demonstrate the high effectiveness of TST.
In summary, this paper makes two key contributions.
1. It proposes new Task-CL method TST based on sub-networks and task similarity. TST not only overcomes CF but also enables effective KT. For KT, it learns the current task in the sub-network of previous task without interference with any other tasks and thus will not cause any CF for the other tasks. This cannot be achieved by other existing methods.
2. It proposes novel task similarity detection method based on gradients computed on masks. This method is simple and yet highly effective. It is instrumental for effective KT.
2 RELATED WORK
Continual learning, Existing CL work mainly focused on overcoming CF: (1) Regularization based
approaches (Kirkpatrick et al. 2016: Lee al.) SSef al. 2017 Zenke et al. 2017 Rusu al. 2016) add a regularization in the loss to penalize changes to parameters that are important to previous lasks. (2) Gradient projection (Zeng et al. 2019) ensures the gradient updates occur in the orthogonal
direction to the input of old tasks. (3) Parameter isolation (Serrà et al. 2018 Ke et al. 2020 Mallya
Lazebnik 2018: Fernando al. 2017 |Worsman et al. 2020) learns dedicated sub-network for each task and masks it out in learning new tasks. (3) Replay- approaches (Rebuffi et al.) 2017
Lopez-Paz & Ranzato 2017, Chaudhry et al. 2019 Wang et al., 2020). retain some training data of old tasks and use them in learning new task. The methods in (Shin et al.. 2017 Kamra et al. 2017
Rostami et al. 2019: He & Jaeger 2018) learn data generators and generate old task data for learning
nnww task. These are clearly very different from our method as we don't 't use any replay data.
Continual learning in NLP. Above existing approaches usually do not use a pre-trained model. However, in NLP, almost all recent CL/non-CL techniques use pre-trained language models (LMs).
We categorize them into 3 types based on which part in the pre-trained LM is trainable. The first one
below belongs replay regularization families and last two types belong parameter-isolation. (1).
cecisformeruupdaatin based- this family updates the Transformer directly. IDBR (Huang et al. 2021) disentangles task-shared and task-specific knowledge in BERT via regularizations. LAMOL. Sun
2

---

Under review as a conference paper at ICLR 2023
et al. 2020) is replay method based on GPT-2. It generates previous task samples before learning the new task. (2). Prompt-bass based a prompt (Lester et al., (021) is an added structure to the input
layer. C-PT (Zhu et al. 2022) learns separate prompt for each task. L2P (Wang et al. 2022) learns a prompt pool for all tasks and prevents CF by choosing the matched prompt and allowing transfer by sharing the same prompt pool. (3). Adapter-based an adapter (Houlsby et al. 2019) is an added
structure in each layer of the Transformer. AdapterCL (Madotto et (020) trains separate adapter for each task. It thus has no CF or KT. CTR (Ke al. 202I) trains shared adapter for all tasks. prevents CF via task masks (Serrà et al., 2018) and achieves KT via capsule networks (Sabour et al., 2017p. Our method TST also belongs to this family but TST updates the mask instead of the adapter. It needs no replay samples or regularization and can effectively detect the task similarity based on the mask importance. We have discussed the advantages of TST in Sec.
Network pruning as importance computation. It is known that many parameters in neural network are redundant and can be pruned (Li et al. 2021 Lai et al. 202T). This has also been shown for pre-trained Transformer (Chen et al. 2020; Lin et al. 2020; Gao et al. 202 Michel et al. 2019 Voita et al.] (019). One method is to discard the parameters with small absolute values (Han et al. 2013: Guo et al.) 2016). Other methods prune at higher levels. In Transformer-basee
model, these
include pruning the attention head (Michel et al.) 2019; Voita et al. 2019 McCarley et al. 2019) and
pruning sub-layers in standard Transformer layer (an et al. 2020: Saijad et al.) (020). However, the above methods are not directly applicable to us as we need to compute the importance of mask to detect task similarity, while the above approaches are all for the importance of network parameters.
PROPOSED TST TECHNIQUE
TST is designed for both CF prevention and KT (knowledge transfer). Most of existing methods focus on CF prevention. The few methods that also do KT are based on HAT, which, as we discussed in Sec. 1. camnot completely isolates the parameters for different tasks and the shared parameters still cause forgetting when updating the sub-networks for similar tasks. This work borrows the idea of sub-network masking from (Wortsman al. 2020), which trains mask isolate sub-network for a task with the network parameters fixed. Thus, different tasks have no interference to cause CF.
The proposed method maintains a shared mask pool. In learning new task, the system first computes the importance of each mask in the pool to the new task based on the gradient of virtual gate variable representing the mask (Sec. [3.2p. Based on the mask importance, if the mask of a previous task is likely to improve both the previous task and the current task, the mask and its underlying sub-network will be used to train the current new task for KT (Sec. (.33. If such a mask is not found, the unused mask in the pool will be trained to find a new sub-network for the task so that it will not cause CF for any previous task. The whole process illustrated in Figure T which is based on the adapter in pre-trainee rre-traind Transformer framework (Houlsby et al. 2019 Both the adapter (randomly initialized) and the pre-trained Transformer are fixed. Only the mask, which is applied only the adapter, is trained to find sub-network in the adapter for each task.
3.1 SUB-NETWORE MASKING AND MASK POOL
Let the mask pool with K masks be ( (K is hyper-parameter). To learn task, a mask is selected from the pool and trained. Each mask the inserted adapter. The detailed mask training is given in Appendix B A mask is always binary and indicates sub-network by element wise multiplying with the adapter parameters,
(k) is of the same size as the parameter set (W) of
(k)) = f(x, W & (*).
where @ is element-wise multiplication, f refers the adapter. W is fixed and shared by all tasks. Only M (k) is trainable. W @ (*) represents sub-network and can be indicated by mask M
3.2 COMPUTING MASK IMPORTANCE
Since the network weights are always fixed, a simple way to perform CL is to use a different mask for each task so that the resulting sub-networks for different tasks will not affect each other. This
Details of adapter is given in Appendix A
3

---

Under review as a conference paper at ICLR 2023
(A) Mask
(B) Mask Selection
(C) Mask Training
Mask Pool
Gate Variables
(k
(g(K)
Importance Computation
1=1
CB
(1)
g(i
impt
0a
M(2)
a, a,4 (a)
(())
Mimpt
W
W
(()
K
u, (a)
& 1
Adapter
Adapter
{(a)
(D)
[(x.. (o)lyy)) N
Forward
Forward
K
Max
Saved Importance
00
Adapter
ΣN 10
Adapter
Backward
Backward
Figure 1: Illustration of TST (from left to right). There are K masks in the mask pool. where each mask is of the same size the adapter weight and always binary (Appendix B before learning. When a new task 1 comes, TST first multiplies each mask with an virtual gate variable (with value of 1). The resulting gated masks are then combined via summation to produce the combined mask for importance computation (Eq. 2). (A) shows the mask importance computation (Sec. 3.2). We mask the adapter using the combined mask Mmmpi and compute the gradient of each virtual gate variable via cross-entropy loss (F) The normalized magnitude of the resulting gradient serves as the importance of each mask to the current task t (Eqs. aand (5). (B) shows the mask selection (.3).. We take the mask with the highest importance as candidate mask a and select a mask using the saved importance of the previous task that used the mask. Only the mask of higher importance for the current task data is selected. Otherwise, we select an unused mask (Eqs. 6 and 7 (C) shows the mask training (Sec. 3.4). Using the selected mask q. we can mask the adapter and train the mask with cross-entropy. As result, we share the same mask for similar tasks and isolate the mask from dissimilar tasks, achieving both forgetting prevention and knowledge transfer (KT).
gives us chance to safely update the masks of sub-networts of similar tasks without affecting any other tasks. However, this cannot achieve KT (knowledge transfer), which is a main goal of this paper. Thus, before learning a new task, we propose to select mask for KT based on task similarity, where each task may use the same mask The intuition is that if two tasks are similar, they can share the same mask from the pool to encourage knowledge transfer; otherwise, the new task should select unused mask to avoid forgetting. We present how to compute the task similarity next.
previous task (for KT) or unused mask in the pool.
Virtual gate variables. Intuitively, if mask is important for two different tasks, the two tasks similar. This motivates us to compute the importance of each mask in the pool to the current task based on its data. To this end, we introduce a set of gate variables, (A) J== where each variable corresponds to mask in the pool. The variable serves mask, which is used to compute the importance of the mask to the current task. We call these virtual gate variables since each (X) is initialized 1, and is not changed. This is because we only need its gradient to compute the importance of a mask and will not use the gradient to update any parameter. We first define the virtual network of masks for our importance computation (see Figure [(A)).
proxy for computing the gradient on each
K
Mimpt
(t) (I)),,
A=1
where g is scalar- scalar-valued valued gate variable and M is a mask. Eq. basically combines all masks (A))) * (K) so that we can compute the gradient on each gate variable based on the current task data. The resulting combined mask, Minp is then used mask the adapter, similar to Eq. | imps
fxx W & Mimpi)
4

---

Under review as a conference paper at ICLR 2023
omm!' can be used to compute the loss based on the current task data and the gradient each gate variable ((k) gives us the importance of the corresponding mask (K).
Mask importance based on the gradients of gate variables. To compute the mask importance, we borrow the idea from neural network punning (Michel et al. (2019). Given the data from the current task t, Dr = [ y ) the importance of a mask is estimated with a gradient-based score.
N
V
giss
N
==1
where g(v is the gradient on the gate variable g (*))
acimpt (27 vE
g(4)
0,00
(5)
where Chmpt (impt stands for importance) is a problem-specife loss function Again, the gradient in Eq. is only for importance computation and is not used update the parameters. Only the final selected mask is trained in Sec. 3.4
Cimpt for mask importance. Eq. offers way to compute the mask importance w.r.t. a given loss
Cimpr A nature idea is to use the cross-entropy los L as Limpi By doing so, we can easily get the importance scores of all masks in the mask pool to the current task Note that the resulting importance score (()) is saved for each task.
3.3 SELECTING A MASK TO LEARN TASK t BASED ON MASK IMPORTANCE
Recall that the goal of computing mask importance is to find the task similarity and select a mask for the current task based on it. To achieve this, we propose a 2-step mechanism:
(1) Find the mask with highest importance as the candidate for knowledge transfer. We leverage the importance scores computed using the data of the current task t, [ Intuitively, the highest score indicates the most useful sub-network for task L. If this sub-network has been used by a previous task, it indicates the previous task's 's sub-network is helpful to task / and the two tasks are very likely to be similar. Formally, in learning task t, we first get the highest importance to task t,
k==
((a) max max((17))..1/"00)
(6)
where is the ID of the most usefuul/importan mask (sub-netword to the current task t. The previous task that has used (a)), denoted task [
is then the candidate similar previous task to task t.
(2) Select a mask for task L. We cannot simply say that the previous task a similar to current task 1. We want to ensure that the update of the mask for by L will not cause forgetting for 8, and can result in improvement for both. Eq. 6 is only about whether a previous used sub-network (a) can help task 6, it does not consider whether the updating of the (4) by 1 will cause forgetting for Therefore, need one more step to be confident that forgetting is unlikely to occur due the updating of (a) by i's data. Since task s's data is no longer accessible, we propose use the saved importance score (I(")) computed using 8's data for mask a to achieve the goaf? This is because T is computed from the previous task s's data when we were in task and is directly related to the
"This paper focuses on supervised tasks. Future work will extent TST unsupervised tasks like topic modeling (Gupta et al. (020).
'Different problems in our experiments (generation. classification and extraction) all use cross-entropy loss. "To facilitate the importance comparison, the importance scores for each mask & is normalized so that the importance scores for all masks have a mean of 0 and standard deviation of 1. To simplify the notation, we use the same (*)) to represent the normalized importance of mask (k) computed from current task t.
'If multiple tasks have selected mask we the task with the highest importance as task We realize the mask has been updated if is shared by multiple tasks and the importance needs to be updated well, which we do not do. Our current simple approach works well our experiments. We leave the issue to our future work.
? (1(0) is computed and saved after training task s (to ease notation, we still use the same I!")), This to ensure that r(()) is computed based on the trained mask and is comparable 7(oo)
5

---

Under review as a conference paper at ICLR 2023
previous task loss. For mask a, now have two importance scores; r( computed from the current task t and % computed from the previous task which learned the mask a. We then compare their values and select the mask via,
T(a)
(7)
u, otherwise,
where u refers to any mask in the pool that has not been selected and used by a previous task. The above inequality can tell us whether the updating of the mask a by the current task is likely to help the previous task s. The rationale for Eq. is that _!()) refers to the normalized magnitude of the gradient that can decrease the previous task loss most. Based on the concept of gradient-based importance in (Michel et af. 2019), since mask a is more important to the current task than to the previous task the current task can potentially improve task and vice versa to achieve KT. It is thus reasonable to use the mask a (and its associated sub-network) to also learn the current task t. If mask a is not as important to the current task & as to the previous task 8, the current task data is unlikely to help the previous task s and vice versa, and a unused mask in the pool should be used to learn f to avoid CF on any previous task. Our empirical results show that Eq. works well.
3.4 TRAINING THE SELECTED SUB-NETWORK
Using the selected mask M (4) for the current task, we can train the task by plugging ( in Eq. with the cross-entropy loss CCE.
4 EXPERIMENTS
We now evaluate the proposed system TST. We first learn all tasks sequentially. After that, their task models are tested using their respective test sets. TST does not use any replay data.
4.1 DATASETS AND BASELINES
Datasets: We use the five datasets covering a wide range of NLP problems, including classification, generation, and extraction. We introduce each of them bellow. For detailed datasets statistics, please see Appendix C (1), ASC (Aspect Sentiment Classification). This dataset is from (Ke et al. 2021 and has 19 tasks. Each task classifies the opinion (positive, negative, or neutral) in a review sentence at the aspect-level of product or service. For example, "The picture is good but the sound is poor" about a TV expresses a positive opinion about the aspect "picture" and a negative opinion about the aspect "sound." (2). CCD (Continual Classification Dataset). This is text classification dataset (de Masson d' Autumme et al. 2019) that is popular in continual learning for NLP. It contains 5 tasks include AGNews (news classification), Yelp (sentiment analysis), Amazon (sentiment analysis). DBpedia (Wikipedia article classification) and Yahoo (questions and answers categorization) 8 (3). SUM (ConvoSum). This is conversational abstractive summarization dataset with 6 tasks domains Fabbrr et al.) 2021). Given conversation from a domain. the system generates its summary. (4). DRG (Dialogue Response Generation). This popular task-oriented dialogue response dataset Multi-WoZ2.0) (Ramadan et al., 2018) with 5 tasks/domains. Given the intent and dialogue state (slot-value pairs containing messages to express), the system is expected generate a response. (5).
NER (Named Entity Recognition). This data consists of 5 tasks, including conl103 (Sang & Meulder 2003). wikigold [Balasuriya et al., 2009), btc (Derczynski et al. 2016)), re3d (Laboratory (2017). and gum Zelles 2017). Each task needs to classify mentions into pre-defined entity types.
Among the 5 datasets, two of them (ASC and NER) have similar tasks and our goal achieve both CF prevention and KT. Three of them (SUM, CCD, DRG) consist of dissimilar tasks that have little shared knowledge to transfer. Then the main goal is to ensure there is little or no CF.
Baselines. We setup 12 baselines, including both non-continual and continual learning (CL) methods.
$ *Since each of these datasets is quite large, we randomly sampled 500 samples from each class for each task due to our resource limitations.
Due resource limitations, we randomly sampled 200 samples for each task.

---

Under review as a conference paper at ICLR 2023
NLn-CL baselines: Comb/MTL and Comb/MTL (Adapter) train tasks in multi-task or data combined setting, where the former trains the whole LM and the latter trains only the adapter. These two widely accepted as upper bounds of continual learning. ONE builds separate model for each task by fine-tuning the LM, which clearly has no knowledge transfer (KT) or CF. ONE (Adapter) (Madotto et al.) (2020) trains adapter for each task separately (called AdapterCL in its original paper), ONE (Prompt) (Zhu et al. 2022) trains a prompt for each task (called C-PT in its original paper).
CL baselines. The CL setting includes an naive continual learning (NCL) method where the system learns the tasks one by one with no mechanism deal with CF or to encourage transfer, and 7 state-of-the art task continual learning (Task- CL) methods.
The 7 CL baselines include: 4 caaspter-based methods CTR et al. 2021). HAT (Serrà et al. 2018 SupSup (Wortsman et al. (020) and CAT (Ke et al. 2020). They all train shared adapter while SupSup trains a sub-network mask on the fixed adapter. HAT is one of the most effective Task-CL. methods with little forgetting. CTR (Ke et al. [02T) encourages transfer via capsule networks and transfer routing. SupSup uses the similar sub-network masking method as TST but without any consideration on knowledge transfer. CAT and CTR are two systems that deal with both CF and KT. prompt- baantt-bssddded method L2P (Wang et al., 2022). which trains a prompt pool to transfer task knowledge and key-value pair prompt selection strategy to select the task-specific prompt (it thus deals with both KT and CF); 2 baselines that modify the Transformer: LAMOL (Sun et al., 2020 is pseudo-replay method using GPT-2. EWC (Kirkpatrick et al. 2016 is regularization method.
4.2 EVALUATION RESULTS AND ANALYSTS
Since we need backbone LM that can do both classification and generation, we adopt BARTLARGE (Lewis et al. 2020) as our LM. Fine-tuning of BART follows the standard practice Due to space limits, detailed hyperparameters are given in Appendix D Since the order of the tasks in a sequence may impact the final results, we ran 5 randomly sampled task sequences. We compute different metrics for different types of tasks using their standard metrics [22] Table I gives the average result of each system on all tasks of each dataset over the random task sequences. Note that LAMOL for NER is not included as it is not obvious how to adapt LAMOL for token-level classification.
Superiority of TST. Table D shows that TST clearly outperforms all baselines for tasks with shared knowledge (ASC and NER), and has no forgetting for tasks with little shared knowledge (SUM, CCD and DRG). Below, we discuss additional observations.
(1). TST achieves both CF prevention and knowledge transfer (KT). Using ONE control, we can see TST outperforms ONE in two datasets (ASC and NER) with similar tasks, indicating effective KT. TST achieves similar results to ONE in other datasets, indicating effective CF prevention. We can also see TST is very similar to MTL/Comb. This again shows the effectiveness of TST.
(2). TST is more effective than the baseline CL systems (EWC, HAT, SupSup, LAMOL) that only deal with CF. This is because regularization-base EWC sacrifices accuracy for overcoming CF and parameter- issoaatin based SupSup prevents any possible knowledge transfer (KT) and is thus poorer in ASC and NER as these two datasets contain similar tasks. The other 3 datasets all consists of very dissimilar tasks with little shared knowledge to transfer, so TST is similar to the baselines that only deal with CF like SupSup. HAT has little KT in classification tasks, which makes ASC poorer. It has forgetting in generation tasks as it cannot isolate parameters in the shared LM head. LAMOL has only weak CF prevention and KT it highly relies on the replay data, which could be of low quality.
(3), TST is also more effective than the baseline CL systems that try to deal with both CF and KT (CAT, CTR and L2P). Among these systems, L2P performs the worst due to the poor prompt selection
For classification datasets (ASC, CCD and NER). we conduct multi-task learning (MTL. experiment. For generation datasets (SUM and DRG), it is not possible to use MTL the language modeling head on top of BART a linear layer with weights tied to the input embeddings. We follow the standard practice (e.g.. Qin Joty 2022): y: Madotto et al. (2200) and pool all data together to train a single shared head (called "Comb").
For the ASC tasks, we adopt the ASC formulation in (Xu al. 2019). where the aspect term and sentence concatenated via </ s>. The opinion is predicted using the average over all tokens.
"SSecifficalyy, we use Macro-Fi Fl and accuracy for the sequence- classification tasks (ASC and CCD), where Macro-F1 (MF1) is the primary metric because highly imbalanced classes in ASC introduce biases in accuracy. We use Rouge score (R1, R2 and RI. for SUM, BLEU score for DRG and Fl for NER.
7

---

Under review as a conference paper at ICLR 2023
Data
SUM
ASC
CCD
DRG
NER
Scenario
Model
R1
R2
RL
MFI
Acc
MFI
Acc
BLEU
F1
Comb/MTT
39,3861
10.3258
35.0513
0.9228
0.9482
0.9157
0.9055
0.2529
0.6333
Coml/MTL (Adapter)
38.8423
11.3795
34,8061
0.9217
0.9465
0,9199
0.9114
0.2450
0.6061
Non-CL
ONE
39.0738
10.7076
35.2501
0.8555
0.9150
0.9107
0.9109
0.2414
0.5933
ONE (Adupter)
38.9000
11 11,5449
35.2323
0.8395
0.9090
0.9078
0.9081
0.2342
0.5669
ONE (Prompt)
30.6709
7.2285
27.5339
0.7646
0.8554
0.8623
0.8628
0.1267
0.4590
NCL.
32.6824
6.8418
29.1898
0.8925
0.9304
0,8508
0.8510
0.2231
0.4919
EWC
32.6362
7.1160
29.0004
0.8836
0.9260
0.8727
0.8737
0.1830
0,5176
HAT
37.1127
10:4009
33.5331
0.8933
0.9328
0.9021
0.9023
0.2147
0.5231
CL
SupSup
38,3660
11.6289
34,7497
0.8898
0.9335
0.9098
0.9100
0.2471
0.5893
LAMOL
10.8817
1.3885
6.8711
0,8462
0.9017
0.5444
0.6704
0.1996
CAT
37,2350
10,5304
33.7733
0.8431
0.8898
0,9082
0,9090
0.2172
0.5073
CTR
37.3360
10.7305
33,6853
0.8886
0.9294
0,9054
0.9058
0.2139
0,5185
L,2
26.6538
4.8220
23.8983
0.7481
0,8464
0.8535
0.8554
0.0852
0,4422
TST (forward)
38.7075
11.3096
34,8344
0.9118
0.9457
0,9123
0.9125
0.2447
0.6159
TST
38.7758
11.3652
34,9935
0.9161
0.9453
0.9104
0.9103
0.2425
0.6169
Table I: Performance for different type of tasks, averaged over 5 random sequences (the standard deviation is reported in Appendix [ The results individual sequences are reported in Appendix F means not applicable. TST (forward) is the average test performance of each task when it was We bold the best performance within CL baselines.
first learned (refer to the text in Sec. 4.2)
Data Model
SUM
ASC
CCD
DRG BLEU
NER
Scenario
R1
R2
RL
MF1
Acc
MF1
Acc
F1
NCL
8,3709
5.5409
5.9023
-0.0044
0,0031
0,0567
0.0564
0.0301
0.1549
EWC
2.5835
1.6784
2.0420
0,0781
0.0278
0.0894
0.0817
0.0078
0.0753
HAT
1.0993
1.2171
0.7840
-0.0088
-0.0036
-0.0007
-0.0008
0.0148
-0.0344
SupSup
0,0000
0.0000
0,0000
0.0000
0,0000
0.0000
0,0000
0.0000
0.0000
CL
LAMOL
3,9017
1.4158
2.3189
-0.0188
-0.0152
0.3226
0.2730
0.0229
CAT
1.8510
1.4146
1.3684
0.0205
0,0081
-0.0013
-0.0021
0.0226
0.0081
CTR
1.4349
0,9002
0.8590
-0.0038
-0.0008
-0,0027
-0.0027
0.0191
0.0058
1.2P
7.2695
3.5268
4.6789
0.0030
0,0054
0.0094
0.0082
0.0342
0.0096
TST
0.3456
0.3051
0.2469
-0,0097
-0,0026
0,0075
0,0078
0.0133
-0.0062
Table 2: Forgetting rate averages over 5 random sequences. Positive forgetting rate indicates forgetting while negative forgetting rate indicates knowledge transfer.
(we can see it is even poorer than ONE (prompt), indicating that its selection causes forgetting). CAT performs better, but still has a large gap compared to TST, due to its inaccurate similarity detection and ineffectiveness in handling previous tasks. CTR is the best one among the three, but it still worse than TST because of the inaccurate instance-level similarity detection.
(4). Sub-network masking on adapters helps TST achieve good results. TST uses adapters and can see ONE (Adapter) is much better than ONE (Prompt) and is similar to ONE. The performance of prompt-based based CL baseline L2P much worse than other baselines. This is because prompts do not have sufficient trainable parameters, which are also randomly initialized and can be difficult to train.
Knowledge transfer and forgetting prevention. To validate TST's effectiveness in dealing with forgetting with a sequence of dissimilar tasks, we compute the Forgetting Rate (Liu et al.) (020). FR - A., A.. where A,, is the forward performance of task i and A is the performance of task i after training the last task f. We average over all tasks except the last one because the last task obviously has no forgetting. We report the forward results (TST (forward)) and forgetting rate FR (averaged over 5 random task sequences) for all datasets in Table 2 Regarding forward performance, TST (forward) is clearly better than ONE, indicating effective forward transfer (learned tasks help the new task). Regarding backward transfer, We can see negative forgetting rates in ASC and NER, indicating some positive backward transfer (the training of a new task helps some old tasks). For the other 3 dissimilar datsets, we can see that TST has only slight forgetting. We can also see that the baselines have much larger forgetting comparing to TST. One exception is SupSup, which has no forgetting because it trains different maakss/uu-netwool certainly good for forgetting prevention but makes knowledge transfer impossible.
for different tasks. This is
Effectiveness of task similarity detection. Comparing the forgetting rate in Table 2 we can see that TST has only a very slight forgetting compared to baselines, indicating the effectiveness of its
Mchta et al.
2021 defined different forgetting rate. Appendix G will argue that ours more effective.
8

---

Under review as a conference paper at ICLR 2023
Data
SUM
ASC
CCD
DRG
NER
Scenario
Model
R1
R2
RL
MFI
Acc
MF1
Ace
BLEU
F1
ONE ONE (Adapter)
39,0738
10.7076
35.2501
0.8555
0.9150
0.9107
0.9109
0.2414
0.5933
Non- CL
38,9000
11.5449
35.2323
0.8395
0.9090
0,9078
0.9081
0.2342
0.5669
ONE (Prompt)
30.6709
7.2285
27.5339
0.7646
0.8554
0.8623
0.8628
0.1267
0.4590
IST (wlo pool)
29.9746
7.1159
27.2446
0,9088
0,9427
0.8760
0.8778
0.1959
0.4836
CL
TST (wlo similarity)
38.3660
11.6289
34.7497
0.8898
0.9335
0.9098
0.9100
0.2471
0.5893
TST (wlo comparison)
34,8660
9.6079
31.4729
0,9084
0.9421
0.8931
0.8923
0.2180
0.5443
TST
38.7758
11.3652
34,9935
0.9161
0.9453
0.9104
0,9103
0.2425
0.6169
Table 3: Ablation experiment results- averages over random sequences (the standard deviation is reported in Appendix Eb. We bold the best performance within CL baselines.
Data
SUM
ASC
CCD
DRG BLEU
NER
Scenario
Model
R1
R2
MF1
Acc
MF1
Acc
Fi
TST (wlo pool)
8,9037
4.7496
5.7680
-0.0055
-0.0008
0.0413
0,0400
0.0494
0.1603
CL
TST (wlo similarity)
0.0000
0,0000
0.0000
0.0000
0.0000
0.0000
0.0000
0,0000
0.0000
TST (w/o comparison)
2,7010
1.4196
1.9688
.0.0079
40.0039
0.0202
0.0214
0.0299
0.0850
TST
0.3456
0.3051
0.2469
-0.0097
-0.0026
0.0075
0.0078
0.0133
-0.0062
Table 4: Forgetting rate for ablation experiment results.
similarity detection. CAT, CTR and L2P all have much more forgetting than TST except on the CCD data, indicating they have poorer similarity detection. To provide more evidences, we show the task similarity detection results for TST in Appendix HH For NER, we see that tasks "conll2003", "wikigold" and "btc" use the same mask (indicating they are similar). This makes sense as they all share the same named entity classes while the other tasks have different named entities. In contrast, CAT, which the only baseline that does task-level similarity detection, finds all NER tasks dissimilar. In SUM, TST finds "icsi" and "ami" to be similar (sharing mask). This is also reasonable because both tasks are summarization on meetings while the others are conversations of diverse forms. Similarly, TST finds "Yelp" and "Amazon" to be similar in CCD because they are all sentiment classification tasks. CAT finds no similarity in both CCD and SUM tasks and produces worse results. Regarding ASC and DRG, it is hard say which system is better as ASC tasks all somehow similar and all tasks of DRG are dissimilar, but we see from the results in Table Ithat TST better. Similarity detection results of different orders and random seeds reported in Appendix
Ablation study. We want to know whether (1) the proposed mask pool, (2) the similarity detection and (3) the importance comparison are helpful. To answer (1), we conduct the ablation experiment TST (w/ pool), where we use only one single mask/sub-network in the pool for all tasks. To answer (2), we conduct the experiment TST (w/o similarity), where we remove the similarity detection and use different masks for different tasks (thus no CF or KT). To answer (3), we conduct experiment TST (w/o comparison), where we remove the importance comparison and simply use the mask with the highest importance based on the current task data as the mask for learning the current task.
show the ablation results and the corresponding forgetting rates. We can see that the full TST gives the best average result, indicating that every component helps. Additional observations are: (1) TST's gain is partially from the mask pool as TST (w/o pool) is poorer on average, particularly for those datasets having little shared knowledge; (2) Similarity detection helps as TST (w/o similarity) gives worse performance. (3) Our importance comparison is effective. We see that TST (w/o comparison) is poorer, indicating the importance comparison learning in TST helps mitigate CF.
5 CONCLUSION
This paper studied task continual learning (Task-CL) on wide range of NLP problems using pre- trained BART model as backbone to achieve both CF prevention and knowledge transfer (KT), The key novelty of the proposed technique TST is the novel task similarity detection method based on mask importance. The importance is computed to find a similar task so that effective KT and CF prevention be achieved. CF is prevented by using a different mask while KT is achieved by sharing the same mask. Experimental results showed that TST markedly improves the performance of both the new task and the old tasks via KT and is also effective at overcoming CF.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Dominic Balasuriya, Nicky Ringland, Joel Nothman, Tara Murphy, and James R. Curran. Named entity recognition in wikipedia. In Iryna Gurevych and Torsten Zesch (eds.), Proceedings of the 1st 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Seman- tic Resources @ Caruurees@IJNLPPP. 2009, Suntec, Singapore, August 7, 2009, pp. 10-18. Association for Computational Linguistics, 2009.
Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville. Estimating propagating gradients through stochastic neurons for conditional computation. CoRR, abs/1308.3433 0333,3332, 2013. URL
http://aaxxv..rrg/ ab// /308 3432
Arslan Chaudhry, Mare' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR, 2019.
Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, and Michael Carbin. The lottery ticket hypothesis for pre-trained bert networks. Advances in neural information processing systems, 14115583418866, 2020.
Cyprien de Masson d' Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' d'Alché-Boo Buc, Emily B. Fox, and Roman Garnett (eds.), Advances Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019.
Lcon Derezynski, Kalina Bontcheva, and Ian Roberts. Broad twitter corpus: A diverse named entity recognition resource. In Nicoletta Calzolari, Yuji Matsumoto, and Rashmi Prasad (eds.), COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan, pp. 1169-1179, ACL, 2016
Alexander R. Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang. Haoran Li, Yashar Mehdad, and Dragomir R. Radev. Convosumm: Conversation summarization benchmark and improved ab- stractive summarization with argument mining. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/JJCNP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 6866-6880. Association for Computational Linguistics, 2021.
Angela Fan, Edouard Grave, and Armand Joulin. Reducing transformer depth on demand with structured dropout. In International Conference on Learning Representations, 2020. URL
ltttps://openeviier net htte//fooomndd1y122yssss..
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR, 2017.
Yang Gao, Nicolo Colombo, and Wei Wang. Adapting by pruning: case study on bert. arXiv preprint arXiv:2105.03343. 2021.
Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic network surgery for efficient dans. NIPS, 29, 2016.
Pankaj Gupta, Yatin Chaudhary, Thomas A. Runkler, and Hinrich Schütze. Neural topic model- ing with continual lifelong learning. In Proceedings of the 37th International Conference on Machine Learning, ICML. 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 3907-3917. PMLR, 2020. URL http::/procedaaggs
mlr. www.ss/vi9//.//
ptepss//11////gptt22aaaaa. quw.a h
Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
Xu He and Herbert Jaeger. Overcoming catastrophic interference using conceptor-aided backpropa- gation. In ICLR, 2018
10

---

Under review as a conference paper at ICLR 2023
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), JCML, 2019.
Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang. and Diyi Yang. Continual learning for text classification with information disentanglement based regularization. arXiv preprint arXiv:2104.054:0 2021.
Nitin Kamra, Umang Gupta, and Yan Liu. Deep generative dual memory network for continual learning. CoRR, 2017.
Zixuan Ke, Bing Liu, and Xingchang Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. In Newr/PS, 2020.
Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. Advances in Neural Information Processing Systems, 34, 2021.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- Demis
drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. CoRR, 2016.
Defence Science Technology Laboratory. Relationship and entity extraction evaluation dataset. 2017.
URL https://github httpe://ggi.hbb..ommdd/////33
Cheng-I Jeff Lai, Yang Zhang, Alexander H Liu, Shiyu Chang, Yi-Lun Liao, Yung-Sung Chuang, Kaizhi Qian, Sameer Khurana, David Cox, and Jim Glass, Parp: Prune, adjust and re-prune for self-supervised speech recognition. NeurlPS, 34, 2021.
Sang-Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NIPS
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efffcienn prompt tuning. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), EMNLP, 2021.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising eequence-to-sqquene pre-training for natural language generation, translation, and comprehension. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 7871-7880. Association for Computational Linguistics, 2020. doi: (0.18653/v1/20220 main.. 703. URL
orgg10 wt..11665/1//00 wttp:r//0..863311122200000 http://100220...11mmmoaa
acl ael-mmann mw...ann....mm 703
Jiaoda Li, Ryan Cotterell, and Mrinmaya Sachan. Differentiable subset pruning of transformer heads.
Transactions of the Association for Computational Linguistics, 9:1 9:1442-1459, 2021.
Zi Lin, Jeremiah Zhe Liu, Zi Yang, Nan Hua, and Dan Roth. Pruning redundant mappings in transformer models via spectral-norrmalieed identity prior. arXiv preprint urXiv:2010.01791, 2020.
Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi-class incremental learning without forgetting. CVPR, 2020.
David Lopez-Paz and Mare" Aurelio Ranzato. Gradient episodic memory for continual learning. In NIPS, 2017.
Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu. Zhou Yu. Eunjoon Cho, and Zhiguang Wang. Continual learning in task-oriented dialogue systems. arXiv preprint arXiv:20/2. 2.000/2./5504 /5504, 2020.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In CVPR, 2018
11

---

Under review as a conference paper at ICLR 2023
JS McCarley, Rishav Chakravarti, and Avirup Sil. Structured pruning of bert-based question answering model. arXiv preprint (Et1:09100003600 2019.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. 1989.
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. CoRR, abs/2112.09153,, 2021. URL https!
09153
Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advances in neural information processing systems, 32, 2019.
Chengwei Qin and Shafiq R. Joty. LFPT5: unified framework for lifelong few-shot language learning based on prompt tuning of T5, In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https://cpenret
net. www.oooorrmmmnH/..... 1PMM)
Osman Ramadan, Pawet Budzianowski, and Milica Gasic. Large-scale multi-domale belief tracking with knowledge sharing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, volume 2, pp. 432-437, 437, 2018.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In CVPR, 2017.
Mohammad Rostami, Soheil Kolouri, and Praveen K. Pilly. Complementary learning for overcoming catastrophic forgetting using experience replay. In IJCAI, 2019.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. CoRR, 2016.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton, Dynamic routing between capsules. In NIPS, 2017.
Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. On the effect of dropping layers of pre-trained transformer models. arXiv preprint XXXv:20004008444 2020.
Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the conll-2003 shared task: Language- independent named entity recognition. In Walter Daelemans and Miles Osborne (eds.), Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 June 1, 2003, pp. 142-147, ACL, 2003.
Ari Seff, Alex Beatson, Daniel Suo, and Han Liu, Continual learning in generative adversarial nets. CoRR, 777959. 2017.
Joan Serrà, Didac Suris. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, 2018.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS, 2017.
Fan-Keng Sun, Cheng-Hao Ho, and Hung- Yi Lec. Lamol: Language modeling is all you need for lifelong language learning. In ICLR, 2020. URL wtt
Skgxen4YDS
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418, 2019.
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, 2022.
Zirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos, and Jaime Carbonell. Efficient meta lifelong- learning with limited memory. In EMNLP, 2020.
12

---

Under review as a conference paper at ICLR 2023
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NeurIPS, 2020.
Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. BERT post-training for review reading comprehension and aspect-based sentiment analysis. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), NAACL-HLT, 2019.
Amir Zeldes. The GUM corpus: creating multilayer resources in the classroom. Lang. Resour. Evaluation, 51(3):581 2017. doi: 10.10077/100799-166-44333 URL https://doi 9343-3
oxr/100100079105599-00
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continuous learning of context-dependent processing in neural networks. Nature Machine Intelligence, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, and Minlie Huang. Continual prompt tuning for dialog state tracking. arXiv preprint arXiv:2203. 2022.
13

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL DETAILS ABOUT ADAPTER
TST leverages adapter to do masking. An adapter is trainable parameters inserted to each Transformer layers, which adapts the output distribution of a pre-traines trairraiee LM without modifying its original weights (the original LM is fixed), An adapter block is simply 2-layer fully connected network with layer normalization and residual connections. Figure illustrate the LM with adapter.
Output
Transformer Layer
Feed-forward layer
Layer Norm
Layer Norm
Feed-forward
layer
Adapter
Multi-headed Attention
Adapter
Feed-forward
Feed-forward layers
layer
Transformer with adapter
Adapter
Figure 2: Architecture of Transformer with adapters. An adapter (blue component) is inserted in each layer. Only the blue and green boxes are trainable while the LM fixed during training.
B ADDITIONAL DETAILS ABOUT MASK TRAINING
In Sec. 3.1 we mentioned that we train mask to identify sub-network for a task without training the adapter's weights/parameters (W). We follow the idea in Wortsman al. (2020). Specifically, to
train the mask, we learn randomly initialized score 8ig trained, these scores are thresholded to obtain the mask that can indicate sub-network. Therefore, the M (R) can be seen as,
(k) for each entry/parameter Way W. Once
M h(s())) (k)
(k)
(8)
where h is function which outputs for top-n% of the scores in the layer with n as a pre-defined mask density (we use 80 following the Wortsman et al. (2020). To make the scores differentiable, we leverage the "straight-through" trick (Bengio et al. 2013) to update the scores as follows,
(k)
(k)
$..
8.
V
(4)
HC
OL
01,
ac O
(M)
08
(k)
OI, 08
(k)
aI,
where I, and O; refer the input and output the units and j respectively, We note this is efficient because the adapter size is small. On the disk, we need to store the additional Boolean mask which takes only 1-bit for each parameter. During continual training,
possible that we select a trained
(k) mask (Sec. 3.3 In this case, the stored binary mask can be used to initialize the score $..
C ADDITIONAL DETAILS ABOUT THE DATASETS
Recall that TST uses five datasets. Here give their detailed statistics.
(1) ASC. ASC is more than a traditional classification problem because of the additional input of the aspect and the fact that in the same sentence different aspects can have different opinions. Here show more detailed on these datasets in Table 5
(2) CCD, SUM, DRG and NER. We give their detailed statistic in Table
14

---

Under review as a conference paper at ICLR 2023
Tasks/Domains
#Training
#Validating
#Tosting
Speaker
233 S7352 A.287 P/63 N.O Nc.
3,4 AJSS P/9 NJU Ne.
38 S./4 A,40 P/4 N.O No.
200 $/245
N.
22/.311
N.O
Computer
187 S/283 A/2118 P/65 N.O Ne.
25 S.35 A./23 P/12 NO No.
29 S./36 A./2 P/7 N.O No.
Nokia66610
2934 A.30
S./3
Nikon4300
131 S/162 3./135 P/27 N. Ne.
15 A, /18 P/2 N Ne.
155.21 A./1 P/2 N,0 Ne.
Creative
582 AJ422
Ne.
P/43
S./5
CanonG3
S./228 A/180 P/48 N. Ne.
S, A/2 P/8 N, Ne. P/27
24 5./2 A,224 P/S N. Ne.
ApexAD
281 S7543 A./
Ne.
A./I P/12
CanonD500
103 S./188 A.96 P./2 N.O No.
S1 A/14 P/1 N.O No.
13/15 A//1 P/A N.O No. 16 S./2 A./2 P/I N.O Ne.
Canon100
5./175 A./123 P/52 N.JO Ne.
1/2 A P/2 NJJU Ne.
Diaper
166 S./191 A/144 P/48 N, No.
2./2/ A/I8 P/6 N.O Ne.
24 S./24 A./2 P/2 N.O No.
Hitachi
152 S,/212 A./153 P/59 N.JU Ne.
236 A P/7 NJJU Ne.
S.27 A./14 P/13 N.JU Ne,
Ipod
124 S./153 A./10 P/52 N.JO
S./1 A/14 P/S N.O Ne.
5,220 A./1 P/S N.J Ne. 20 S.23 A./1 P/7 N.O Ne.
Linksys
152 S/176 A./288 P/48 NO Ne.
1./2 A AJ13 P/9 N. No.
MieroMP3
384 $/484 A./340 P//14 JO Ne.
4//6 A./48 P/13 NJO Ne.
S./1 A./3 P./2 N.O Ne. 39 S.46 A.33 P/16 N./ No. 24 S.22 A.S P/20 N.O No.
Nokia6600
298 /362 A./244 P/1 18 N Ne.
26 5./4 A./32 P/13 N.0 Ne.
Norton
S/194 A/54 P/140 N.JO
17 S.24 A/IS P/9 N. No.
Restaurant
1893 $/3452 A.20994 P/719 N./579 Ne.
84 $/150 7.700 P/26 N/54 Ne.
600 S/1120 AJ728 P/196 N/196 Ne. S,/68 A./41 P/128 N./19
Laptop
S/2163 A./90 N NJ433
S/150 P/66 N/27 Ne.
Table 5: Statistics of the ASC tasks. S.: number of sentences; A: number of aspects; P., N., and Ne.: number aspects with positive, negative and neutral opinions, respectively. Note that the "Restaurant" and "Laptop" have 3 classes of opinion polarities (positive, negative and neutral) while the others have only classes (positive and negative).
Dataset
Tasks/Domains
#Training
#Validating
#Testing
#Classes
Yahoo
4500
500
4840
10
AGnews
1785
199
1756
2
CCD
Amazon
898
100
998
2
Dbpedia
6237
693
6748
14
Yelp
900
100
984
iesi
43
10
6
ami
97
20
20
reddit
201
50
250
SUM
stack
205
50
250
nyt
200
50
250
emails
215
50
250
taxi
406
71
56
hotel
3366
143
177
DRG
attraction
298
27
28
train
1954
196
262
restaurant
569
63
59
con112003
200
3250
3453
9
wikigold
200
170
170
9
NER
btc
200
934
934
9
re3d
200
77
200
21
gum
200
250
1000
23
Table 6: Statistics of the CCD. SUM, DRE and NER datasets. Number of classes is not applicable to SUM and DRG because they are generation datasets.
D HYPERPARAMETERS
Unless otherwise stated, the same hyper-parameters are used in all experiments. The maximum input length set 128 for all datasets except for SUM which uses 1024 due its longer sequences. AdamM optimizer is used. The learming rate is set to 5e-5 for Transformer (search within (5e-2.5e- adapter and classifier. The and adapter bottleneck size is 64,
3,5e-4,5e-5)), 3e-2 for prompt (search within [3e-1, 1,,,0000,,00,1111
prompt length is set to 20 (search within following the original papers (Houlsby et al., 2019). The batch size set to 32 and the number of training epochs is set to 50 with early stopping. The number of mask K in the mask pool is set to the same as the number task. For classification tasks, a separate classification head is used for each task in the sequence. For generation tasks, shared LM head is used for all tasks in sequence. further set the number of beams to for beam search and constrain the target length between 30 to 200. For image-based (EWC, HAT, SupSup, CAT, I.2P) and GoBERTa-basse (CTR) systems,
15

---

Under review as a conference paper at ICLR 2023
adapt them for text classification and generation by replacing their feature extractors with BART. For LAMOL, we directly run the author provided code. Except for the aforementioned hyper-parameters, all bascline-specifie Hyper-parameteen follow those in their original papers.
E STANDARD DEVIATIONS
Table 7 reports the standard deviations of the corresponding results in Table the main paper) of TST and the considered baselines over 5 runs with random sequences. We only report the CL baselines since they related to the task order. We can see the results of TST is stable. SupSup trains different mask/sub-networks for different tasks, so it not related to the task order and not reported in the table.
Table repport the standard deviations of the corresponding results in Table (in the main paper) of TST and the considered baselines over 5 with random sequences. We the results of TST and its variants are stable. TST (w/o similarity) trains different mask/sub-netwwrrk for different tasks, so it is not related task order and reported in the table.
Scenario
Data Model
SUM
ASC
CCD
DRG
NER
NCL
=0,6028
20,5458
10.6623
10,0040
10.0027
10.0233
=0.0230
=0,0060
20.0767
EWC
+1.3205
20.5331
+1.1258
10.0093
+0.0063
10.0227
=000266
20.0217
+0.0707
HAT
+1 +1.333
+0.4912
+1.1464
+1.0130
+0.0042
+0.0028
+0.0028
+0.0094
+000063
CL
LAMOL
1122519
10.4218
10.5231
10.0085
+0.0039
40.0317
=0.0258
-0.0088
CAT
+10024
#0.4027
+0,9029
+0.0096
+0.0021
+0.0019
=0.021
#0.0051
+00010
CTR
111995
10.5189
10.9646
10.0080
10.0030
+0,0009
200010
=0.0127
10.0049
L2P
22.1496
20.4747
+2.0267
+0.0650
+0.0361
10.0181
=00176
1000216
+0.0022
TST
+0.5287
+0.3585
+0.5175
+0.0027
+0.0003
+000049
+0.0048
+0.0079
+0,0031
Table 7: Standard deviations of the corresponding metrics of the proposed TST model and the baselines on the five different datasets.
Scenario
SUM R2
ASC
CCD
DRG BLEU
NER
Model
RL
MFI
Acc
MF1
Acc
TST (wro pool)
-0.3504
+0.5482
#0.0499
+0.0034
+0.0287
CL.
TST (wlo comparison)
+1.0655
005163
10.95994
+0.0073
+0.0028
+0.0098
10.0089
000057
10.0374
TST
+0.5287
0033585
=0.0027
+0.0032
#0.0049
+0,0048
-0,0779
+0,0011
Table 8: Standard deviations of the corresponding metrics of the proposed TST model and the six ablation experiments.
F RESULTS FOR INDIVIDUAL SEQUENCES
we report the results averaged over 5 random sequences (different task orders). In this section, we give the results of each sequence Table We can see that the order indeed affects the results but not by much. In summary, we believe the average over random sequences in Table [ can show the effectiveness of TST.
In Table !
SUM R2
ASC
CCD
DRG BLEU
NER
Order
RI
RL
MFI
Acc
MF1
Acc
Fl
39.0532
11.7343
35.4234
0.9144
0.9497
0.9102
0.9104
0.2422
0.6137
38.4992
11.0726
34.7320
0.9171
0.9435
0.9081
0.9084
0.2429
0.6137
37.9608
10.7857
34.0924
0.9200
0.9466
0.9036
0.9038
0.2376
0.6216
37.6063
10.7691
34.0197
0.9130
0.9412
0.9100
0.9093
0.2472
0.6163
38.7750
11.2888
34.8252
0.900S
0.9454
0.9109
0.9112
0.2396
0.6192
Table 9: Results for individual domains.
G DIFFERENCE BETWEEN OUR FORGETTING RATE AND THE ONE IN (MEHTA
AL.
202T
Unlike our forgetting rate in Sec. 4 the forgetting rate in Mehta et al. 2021 is defined as F?
D / (el... 1- (Sr.r S.r).. Since both metrics measure everything from the standing
16

---

Under review as a conference paper at ICLR 2023
point of the end of continual learning, i.e., after all tasks are learned, we believe our measure is more reasonable. Let us use some examples to illustrate,
(I). task 1 archives the accuracy 0.5 right after its training, it achieves 0.4 after task 2, and it achieves 0.3 after task 3 (final task). In this case, both measures give the same forgetting 0.2.
(2). if task 1 archives the accuracy 0.5 right after its training, it achieves 0.8 after task 2, and it achieves 0.82 after task 3 (final task). If we take max (E)), then the forgetting is -0.02, but our method will give -0.32. In this case, our method is more reasonable because precisely shows how much backward transfer (negative value here means backward transfer) has been achieved for task after task 2 and task 3 are learned since both measures evaluate from the same reference point, i.e., after all tasks are learned (the last task is in both measures, i.e., task 3 in this example).
(3). If task archives the accuracy 0.5 right after its training, 0.8 after task and 0.4 after task 3 (final task), our metric will give the forgetting 0.1. F will give 0.4. This case is more debatable because F' catches the worst forgetting. But since we evaluate after all tasks are learned (the reference point is when the last task is learned), again we believe that our method more reasonable.
(4), If task archives the accuracy 0.5 right after its training, 0.1 after task and 0.4 after task 3 (final task), both metrics will give the forgetting 0.1. In this case, Fy does not catch the worst forgetting of 0.4 (0.5-0.1) in the process. Again, if we agree that we evaluate from the reference point of when the last task is learned, then both measures are fine in this case.
In summary, while we believe ours is more reasonable, a better metric may be designed in the future to characterize forgetting and knowledge transfer in the continual learning process.
H COMPARING TASK SIMILARITY DETECTION RESULTS BETWEEN CAT AND TST
In Sec. 4 we discussed the effectiveness of task similarity comparison and compare the similarity detection results of TST and CAT. Figure Show the task similarity results on the ASC data for CAT. CAT did not find any similar tasks for the other datasets. The analysis already in Sec. 4
show the task similarity results for TST. Figure shows
DETAILED SIMILARITY DETECTION RESULTS FOR TST IN DIFFERENT SEQUENCES AND RANDOM SEEDS
In this section, we are interested in how the random seeds and the task orders affect the detected similarity.
We give the detected similar and dissimilar tasks for all datasets in Table 11 The detailed sequences for these datasets are given in Table TO We can observe the followings:
(1) The similarity detection results in different task orders and different seeds have some differences but not much. It is reasonable to have differences as the learning of tasks is quite dynamic.
(2) The effect of random seeds and orders is different across datasets. For example, in SUM, the similarity detection results in different seeds have differences, which is probably because each task in SUM is long document and has more information and easier to detect the similarity. In other datasets, We can see different orders and seeds made some differences, indicating that it more difficult to detect similarity for them, which is probably because the tasks in these datasets are sentence-bart based text classification (extraction or generation) and the information contained in a sentence is limited. Thus, the detection has larger variance.
In summary, as we discussed in Sec. 4p. the task similarity is fuzzy and it is hard to say which task similarity is actually correct wrong as there are ground- similarity labels. However, the average results over random sequences in Table show us the effectiveness TST.
17

---

Under review as a conference paper at ICLR 2023
laptop
CanonS100
Speaker
CanonlowerShtt55500
Nokia6600
pod
ApexAD2600e
LinksysRouter
MicroMP3
Hitach/oouter
NkonCoolpix4300
Norton
Nokia6610
Creativelabs
Router
DlaperChamp
Computer
Canon63
CanonG3
Camputer
DiaperChamp
fourer
Creativeatte abs
Nokiati510
Norton
restaurant
Wlloncoolpx4300
HRtachitouter
MccoMP3
LitssysRauee
ApexAO2600e
Cononoowesattsson
Bodd
Nokiat8000
Speaker
Canon5110
laptop
Figure 3: Similarity detection results for ASC of CAT. The y-axis gives the sequence of tasks that are learned, and the x-axis lists the same set of task names. The yellow cells indicate that the corresponding tasks are detected by CAT as similar (one task can be similar multiple tasks but only the previously learned tasks are considered for the current task) while the purple cells indicate the tasks are dissimilar or unknown (because they have not been learned). For example row 3 means when we are learning task "DiaperChamp" CAT finds it to be dissimilar to any of learned tasks ("CanonG3" and "Computer"); Row means when we are learning task "Router". CAT finds be similar to task "DiaperChamp".
18

---

Under review as a conference paper at ICLR 2023
CanonG3
Computer
DiaperChamp
Router
CreativeLabs
Nokia6610
Norton
restaurant
NikonCoolpix4300
HitachiRouter
MicroMP3
LinksysRouter
ApexAD2600e
ipod
Nokia6600
monnowershoot55000
Speaker
CanonS100
laptop
24
15
16
17
18
(a) ASC
E31
371
90
modd
(b) SUM
(c) DRG
0X82000
magole
+
Oppd
0.11
(d) CCD
(e) NER
Figure 4: Similarity detection results of TST (after the continual training of all tasks). X-axis shows mask IDs and y-axis shows the task names. The yellow cells indicate that the masks are used by some tasks while the purple cells indicate the masks are not used by their corresponding tasks. If multiple tasks use the same mask, they regarded similar by system TST. Note that not all masks are used because different tasks may share the same mask.
19

---

Under review as a conference paper at ICLR 2023
Dataset
Order
Sequences
ies reddit
nyt
2
nyt - stack - reddit + emails - ami i0si
SUM
emails
reddit nyt
stack - nyt reddit emails - ami iosi
reddit
nyt
stack emails
CanoaG3 Computer DiaperChamp-- Router CreativeLabs
Nokiu6610 - Norton
Nikond300 HitachiiRooeer
MieroMP3 LinksysRouter ApexAD2660 ipod Nokia6600
CanonDS00 + Speaker CanonS
laptop
Taptop rest HitachiRonter CanonS 100
ipod
ApexAA2260 MicroMP3 - Speaker - LiinksysRouter - Nokia6660 Nikon4300
CamenD500 CreativeLLas Norton
2
Compuler + Router DiaperChump
ASC
CanomD3iX = LinksysRouter Nikom430X - Norton - Computer
MicroMP3 ADxxAD2000
ipod CanonG3
laptop -a Nokia666 - HitachiRouter - Speaker DiaperChamp
CanonS Rouser Nokia6600 CrcativeLabs
CanonGa - Compater Canon 5500 Nokia6600 Nikon4300 +
LinksysRouter ADpxAAD0000 Roater Speaker laplop laptop
Camon8100
Norion - CreativeLLass ipod
Nokia6610 MicaoMP3 DhaperChamp HitachiRouter
yahoo -2 agngggg amazio dhperii yelp
yahco - yelp amazon dbpediia agnows
CCD
3
amazon yelp yahoo agnews dhpedin
yahoo chpedia - yelp agnews
5
yahoo - yelp - dbpedia amazon + agnews
taxi - - attraction
train - restaurant
-
restaurant
DRG
attraction - hotel - restauraat - taxi train
attraction hotel
min
restaurant + attmccion Imin - taxi hotel
conli2003 wikgold b0c re3d gum
2
bte conl12003 - wikigold
re3d
NER
wikigold re3d con112003 + gum
bte con112006 wikigold gum re3d
wikigold re3d gum coal12003
Table 10: Random sequence orders of the 5 datasets.
20

---

Under review as a conference paper at ICLR 2023
Simalar
Disasmilar tucks
[amts [c91]
ny,, stock,reddis, smails reack,reddis,, emails
[ami xcsi]
Order
nys, stsckiii,,,d,, amails nyt, stack,.rdddis, anails
[amti ics1]
SUM
ami, losi, nyt, roclir email
[ami 3051
ny,, stcc.rcdais, emants myl. stackreddiss, emails
[ami, tesi)
Seed
[Amti (C91]
stack,reddiss, smails
x051]
stack,reddis,, emails
stnck.eeddiss amails
(CanunG3, 3, Computer. DiaperChurpp Router,
Nodiafii 10, Norion, Nikon4300,, Speaker)
[Creativetab,, CamonS 100)
ApexAD2600, ipod, C50
(MicroMP5, LisksysSouurr laptop),
[HitcciiKottss, Nokizt6001
[HizchiRaneer CanonS 1D0,.
CanonD500, CreativelLabs.
Nakiseel, DiaperChamp ).
Tipod, Nokiat600, Narton, LinksysRouter,
laptop, rest, ApexAD2600) MicroMP3,
Nikom4300, Computer).
|Router. Canon(a3)
[LinsyysRoueer Ni0on
Norion, ApexAT77600)
[MicrMMP3. LiinchiRouteer
CanonD500, CreativeLabs,
DisperChamp, CanonS (00).
Nokis6600, Router, Computer
frest, ipod, 10, Speaker).
(CanonG3, laplop)
[omputer Cimon NotuakiN, Nato043001
ipod, Nokis6610, HiuchhiRouee).. (1 [iinkyyRonts]]]
ConoeC33, Router, Spenker
laptop, CanonšI 100, MirroMP3).
[rest, DisgerChumpl, [Murion. ApcxAD2600)
[ (Rouicr, Computer, rest),
[Linksy-Router, DisperChamp, CononDS00).
[laptap, Nokisc600, HitcchiRoute,
Nonon, ApexAD2600, MicroMP3, Speaker
Canon'S 100, Nokia6610).
[Nikon4300, ipod, Crestivel .as. Conoe(3)
ASC
[ThischiRouter, CanonS 100, D50
CrouliveLabs, Nokkaar 10, DiaperChumpl:
(ipod. Nokiat600, Nocton.
laptop, rest, AD260 MicroMP3,
LinksysRouter, Nikon43000, Computer). [Ronter, Canon(i3)
[red, HiachhRRoterr
Nikvo4300, CanonG3, MicroMP5).
[CanonS100. 100, SS00 Speaker.
laptop, Router, LinksyRouier, ApexAD22600, Nokiab6U0, CreativeLabs
Computer. DiaperChamp). [Norton, Nokiab6 10)
[rest, ipod, ApexAD2800).
[HincchiRoteer, CanenS 100, Nokin/6000)
[CanonD500, Nokia6610, DispeChaapp.
(Creativel als, Norton).
laptap, Router, Spenker
(MicroMP3, Linksy,Router, CanonG3).
[Nikoe4300. Computer)
(ipod, ApcxAD2660).
(CaminD3000, DiaperChamp)
laptop, rest, HosachiRouter CunonS10,, Nokia5600.
[CrsativelLalx MicroMPS, Spenker, LinksysRoner) (CanonG3, Rouler)
Norto, Nokin6610, Nikoo4300,, Computer
[rest, Hiarchioonterr Canon (TBl, 1poal, Narton).
(ApexADD2600. CanenD500, Speaker).
[MisuMPP3, Linksy-Ronteri
ligtop, Nokia6600, Araaaeoo,,Cewiveealt CrewiveLabs, Computer, DiaperChamp
[Nikoo44300, Canentaa).
[Nokia6610, Ronter]
[yclp, amazon
yaboo, uhpodia, agnews yshon, dhpedia, agnows abpediia
[yelp. amazon]
Order
(yelp, amazon), [yahoo, agnews)
(yahoo, dbpedia, yelp)
(yelp. amazon]
yahoo, dhpodia, agnews
(yelp. Jmazon
yalooo alpplli, agnews yshooo
[yelp, dhyedia). [amazon, agnews)
Seed
[yelp, dipediaa]
yaloo, ammzon, agnews
yelp. dlipddin, yaloo, amazon, agnews yelp. Jbpedia, yahoo, amazon, agnews
Intraaction, train]
12X1, hotel, restaerunt taxi, hotel, restsomett
[sttraction, triin]
Crder
(nttraction, hotel, restammet) [bilraasion, urain)
tasi, train
taxi. botel, restburmt
DRG
Altraction, tmin, taxi, hotel, restaurant
Intractioon
antractioon
hotel,
[nttraction, train]
t2xi, hotel, restaornar
attriction, train, taxi, hotel, restaurant actraction, rain, taxi, hotel, restaurant
(000 wikigold, [cc (btc, re3d)
re
conti2003, wakigold, bac, re3d, gum conli22015, wikigold, gum conti2003, wikigold, bec, read.gum gum 2001 wikigokl, bus, re3d, garn ne garn
Order
NER
[o01 wiligokt, [ic
con112003, wikigold. bic, re3d, gumn wikigeld, gum, bec conli2003, wiligold, gum c0m113003, wikigold, bec, re3d, gues
Seed
[ve3d, c0ol12003) (bbc, rc3d)
Table 11: Similarity detection results for all 5 datasets in different orders and different seeds (based on order 1). Tasks in "()" means they are detected as similar. "_" indicates there is no similar tasks found.
21


Reviews:
[{"review_id": "yyuLNUfCvV", "reviewer": "Reviewer_ktyy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the problem of task continual learning wherein new tasks are continuously learned in an incremental fashion. Specifically, the paper aims to achieve two key objectives for a successful continual learning system: preventing catastrophic forgetting of the previous tasks and enabling knowledge transfer to new tasks. To achieve this, the paper proposes the TST approach, Task-continual learning based on Sub-networks and Task similarity. The main idea is to start with a mask pool, then select the relevant mask for the given task with gradient-based importance measure, and fine-tune it with the current task data. With the proper selection of the relevant mask, one can alleviate the interference from unrelated tasks and simultaneously enable backward/ forward transfer to related/ current tasks. The paper conducts an extensive set of experiments on a wide range of NLP tasks - sentence-level classification, token-level classification, and generation and compares the proposed TST approach with relevant baselines. In summary, the paper shows that TST improves the overall accuracy of the system after sequentially training on different tasks, reduces forgetting, and in some cases (ASC and NER) even enables positive backward transfer.  ", "summary_of_the_review": "Overall, I rate this paper marginally below the acceptance threshold. It is interesting to see the efficacy of the proposed approach for task continual learning. However, with the currently reported numbers, it is unclear how significant the improvements are over considered baselines. The addition of the statistical significance test results would be more convincing and I would be happy to increase my score during the discussion phase.", "strength_and_weaknesses": "Strengths:\n\n1. The paper attempts to solve an important problem of task continual learning with a specific focus on reducing forgetting and enabling forward transfer objectives. These objectives constitute important desiderata for realistic continual learning systems.\n\n2. The proposed TST approach employs parameter-efficient adapter modules to learn sub-networks for different tasks, enabling their approach to scale to a large number of tasks. Also, the proposed task similarity detection method based on gradients is easy to compute and seems effective.\n\n3. The paper conducts a rigorous set of experiments spanning different NLP tasks and compares their performance with several relevant baselines.\n\nWeakness:\n\n1. Although the paper conducts several experiments, there are some open questions regarding the statistical significance of the results provided in Table 1. Specifically, comparing the results with SupSup, it is unclear whether TST is the clear winner (see SUM, CCD, DRG). As the paper reports, the average results over 5 random task orderings, please consider reporting significant test results. \n\n2. Apart from alleviating forgetting of the previous task, the paper claims that the TST approach also encourages knowledge transfer. Results presented in Table 1 do not provide any convincing evidence for the same. The paper should consider reporting the learning accuracy as mentioned in [1]. For example, to understand the knowledge transfer from Tasks 1 and 2  while learning Task 3, one should consider reporting average performance for tasks after learning on their datasets. Also, if the paper argues that there is positive backward transfer in that case to compute the forgetting, the drop in the performance should be from the max performance for previous tasks and not just right after Task i. Please see the definitions of forgetting and learning accuracy in Eq(1) in [1] for reference.\n\nReferences: \n\n[1] Mehta, Sanket Vaibhav, Darshan Patil, Sarath Chandar, and Emma Strubell. \"An empirical investigation of the role of pre-training in lifelong learning.\" arXiv preprint arXiv:2112.09153 (2021).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "oUFL7ESut2", "reviewer": "Reviewer_Umdv", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Prior work in continual learning has focused on addressing the catastrophic forgetting (CF) problem. However, this paper notes that \"knowledge transfer\" (KT) between tasks is also important, and proposes a new method which \"addresses both issues.\" \n\nTo address CF, each task is associated with a learned mask identifying a task-specific sub-network. The mechanism by which this is accomplished is “borrowed from (Wortsman et al., 2020)” and consists of boolean gates indicating which parameters in the network should be employed for which tasks.    \n\nTo address KT, the idea is to measure the importance of the mask of the previous task relative to the current task. This uses an idea from network pruning (Michel et al., 2019) and uses gradients as measure of parameter importance. A contribution of this work is to extend this idea to entire task-specific masks.\n\nThe experiments consist of five benchmarks: (1) 19 tasks for aspect sentiment classification, (2) 5 tasks for continual classification dataset, (3) 6 tasks from ConvoSum, (4) 5 tasks for dialogue response generation and (5) 5 named-entity recognition tasks. On these benchmarks, 11 baselines are compared, including continual learning and non continual learning methods.\n", "summary_of_the_review": "Overall, this paper provides a sensible framework for continual learning, providing mechanisms to address both CF and KT. My main concern is the lack of novelty: the main technical contribution is how task similarity is estimated but even this is largely based on prior work. Furthermore, while experiments are fairly comprehensive, in some cases the improvement over baselines is inconsistent or relatively small (e.g., SupSup).", "strength_and_weaknesses": "The experiments are fairly systematic in terms of considering a large number of baselines and datasets. However, it should be noted that the improvements over certain CL baselines like SupSup (Wortsman et al., 2020) are very similar in performance (in one case, SupSup achieves 11.63, while the proposed method gets 11.36 and is incorrectly labelled in bold as the best performance).\n\nI would have liked to see more motivation for the proposed masking approach. Specifically, looking at the gradients does not strike me as the most obvious or computationally expedient mechanism to compute task similarity. For example, task similarity could be measured instead on the basis of simple statistics of the feature representations (e.g., multivariate Gaussian fit to each task’s activations).  In general, additional discussion of the limitations of the proposed approach and discussion of alternatives would benefit the paper.\n\nI would have liked to see a multi-task “oracle” that has access to all tasks at once during training. This would help characterise the potential performance benefit of knowledge transfer separate from the impact of catastrophic forgetting.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YWeV4LQwYZx", "reviewer": "Reviewer_iK2Y", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work introduces a new continual learning method TST.\nTST considers both preventing catastrophic forgetting (CF) and encouraging knowledge transfer (KT).\nTo prevent CF, TST directly use one existing CL method called SupSup.\nTo encourage KT, TST borrows the idea of gradient-based importance score and applies this score to assign sub-network masks for sequential tasks.\nExperimental results show that TST meets its motivation.\nConcretely, it improves the CL performance on similar tasks and keeps low forgetting rates.\n", "summary_of_the_review": "**Post-revision update:**\nThe response make me further worried about the relation between the 'performance improvement' and the story-telling of 'task similarity detection'.\nThe insight about 'task similarity detection' is not well-supported.\nThus, I decrease the overall score.\n\n\nI have some concerns for the effectiveness of the proposed method and therefore give a borderline score.\nI list several questions and will change my score according to the response.", "strength_and_weaknesses": "**Strength**\n\n* The motivation behind the design of TST (i.e., assigning sub-network masks based on task similarity) is promising.\n* The design of importance score (Equ. 2-5) and the choosing strategy (Equ. 7) is interesting. Moreover, the analysis results in Table 3 and 4 show the effectiveness of these designs.\n\n**Weakness**\n\n* The impt mask defined in Equ. 2 and used in Equ. 3 is a little weird. It seems that the elements in impt mask are not 0/1 and could be extremely larger than 1. Therefore, multiplying this mask to the original weight matrix may strongly affect the functionality of the original modular (i.e., the adapter), and the gradients (which will be used to calculate the importance score) can also be influenced.\n* Lack of further analysis of the mechanism behind TST.  I suppose the most important and interesting part in this work is the designs in Equ. 2-7. I agree that these designs sounds good, but I feel it is too amazing for a neural model to perform in a so interpretable way.\nThis is my major concerns to this work. I list several concrete questions as follows and will change my score according to your response.\n\n**Questions**\n\n* Q1: **Did you cherry pick the visualization in Figure 3?** Your analysis in 'Effectiveness of task similarity detection' tried to demonstrate that the actions (or called the choices of masks) in TST are in line with the intuition that the similar tasks share the mask while dissimilar tasks choose different masks.\nTherefore, I'm wondering if these analysis are from the cherry picked model or general ones.\nFor instance, for the two tasks 'icsi' and 'ami' in SUM, when the random seed and task sequence order are changed, can TST still find them similar?\n* Q2: **Does Equ. 7 work correctly at the beginning of the task sequences?** Since masks are randomly initialized, the importance scores calculated in the first several tasks could be hugely affected by this randomness, and the comparison in Equ. 7 might also be influenced. Moreover, I notice that for (a) (b) (e) in Figure 3, TST found most similar tasks at the beginning of the task sequences. Therefore, I'm wondering whether these 'similar tasks' are superficial correlations according to their order in task sequences.\n\n**Minor Questions**\n\n* How are the adapter used in TST trained? Since the paper claims that only the mask is trained, I wonder if the adapter is just randomly initialized without further training?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MuUkRTY47MA", "reviewer": "Reviewer_k1vn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- This works address two majors challenges in continual learning: catastrophic forgetting  and knowledge transfer (KT) across tasks applied to NLP tasks.\n- In doing so, the approach is based on sub-networks, and computes task similaity via gradients and mask importance where catastrophic forgetting is controlled using a different mask while knowledge transfer across tasks is encourage using the same mask across tasks.\n- The proposed approach is applied to several NLP tasks using 5 datasets, and evalued against exhaustive baselines. The paper show improved performance in terms of controlling catastrophic forgetting  and performing knowledge transfer (KT) across tasks\n- The paper is clearly written and well motivated \n", "summary_of_the_review": "Please see above.", "strength_and_weaknesses": "Strengths:\n- paper address key questions in continual learning and applied to NLP tasks\n- the approach is incremental yet effective\n- novel ideas in computing task similarity using mask importance applied to presenting CF and encouraging knowledge transfer, however inspired from existing works\n- paper is well motivated and clearly written \n- evaluates the proposed approach on 5 different NLP data sets \n- paper employs a good set of related baselines to empirical comparisons \n- sound quantitate analysis and ablation study \n\nWeaknesses:\n- incremental novel steps (limited novelty) yet effective and simple \n- unclear / missing experimental setup: how is the data split over time across tasks in Cl setup? Does a different sequence of tasks leads to different performances?\n- unclear how the approach address the question: how new entities are learned over time, not seen in the historical tasks? How does the SoftMax adjust and the parameters shared over a sequence of NRE tasks as new entities evolve?\n\nQuestion:\n- See Weaknesses section \n- How does the approach scale to unsupervised tasks - such as topic modeling in continual learning paradigm [1]? How does the task similarity be applied on document/text latent representations: topics?\n- Include additional existing works such as [2] in continual topic modeling: Unsupervised Continual learning for NLP that overcomes CF via both regularization approach and data replay and computes task similarity using topics - that represents data documents [1] and perform KT.\nIt is interesting to see how the tasks similarity approaches scales to unsupervised continual learning (close to real world). \n\n\nReferences:\n[1] Topic modeling using topics from many domains, lifelong learning and big data. ICML 2014.\n[2] Neural Topic Modeling with Continual Lifelong Learning. ICML 2020.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject