Venue: ACL 2023

# Rehearsal-free Continual Language Learning via Efficient Parameter Isolation

Zhicheng Wang1, Yufang Liu1, Tao Ji1, Xiaoling Wang1, Yuanbin Wu1

**Congcong Jiang2, Ye Chao2, Zhencong Han2, Ling Wang2, Xu Shao2, Wenqiu Zeng2 \({}^{1}\)School of Computer Science and Technology, East China Normal University \({}^{2}\)Information Technology Department, Huatai Securities zcwang@stu.ecnu.edu.cn ybwu@cs.ecnu.edu.cn**

###### Abstract

We study the problem of defying catastrophic forgetting when learning a series of language processing tasks. Compared with previous methods, we emphasize the importance of not caching history tasks' data, which makes the problem more challenging. Our proposed method applies the parameter isolation strategy. For each task, it allocates a small portion of private parameters and learns them with a shared pre-trained model. To load correct parameters at testing time, we introduce a simple yet effective non-parametric method. Experiments on continual language learning benchmarks show that our method is significantly better than all existing no-data-cache methods, and is comparable (or even better) than those using historical data1.

Footnote 1: Code is available at [https://github.com/Dicer-Zz/EPI](https://github.com/Dicer-Zz/EPI)

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

+
Footnote †: _Volume 1: Long Papers_, pages 10933–10946

## 1 Introduction

Deployment of NLP models could be dynamic in real-world scenarios: models need to evolve continually when coming in new tasks or new domains (updating an event detection model to handle new event types, for example). Recent studies on continual (language) learning [8; 5] show that, compared with sticking with one single unchanging task, incrementally updating a model for a series of tasks is challenging: learning a new task will make our model perform poorly on previously learned tasks. The so-called _catastrophic forgetting_ phenomenon is the central research topic in continual learning [7].

One method for mitigating forgetting is through data replay (_rehearsal_). The model caches some previous tasks' training data along the sequential learning process. When learning a new task, all previous tasks are re-learned with the current task. The vanilla rehearsal-based approach performs well on many continual learning problems [10; 11; 12], but access to previous tasks' data could conflict with the original intention of continual learning. In fact, if previous tasks' data are given, it approaches multi-task learning where all tasks are given at once and forgetting is not a problem. Meanwhile, in some situations, previous datasets are not available due to regulation or privacy considerations. Therefore, it is essential to look into rehearsal-free methods.

In this work, we study parameter isolation strategies for rehearsal-free continual language learning. Basically, each time a new task comes in, such a strategy allocates a new set of model parameters for that task, which aims to prevent potential interference with already learned knowledge, so that forgetting could be alleviated [14; 15]. Three cruxes of making the idea work in practice are,

* for a possibly large number of tasks, we need to control the storage budget for each task.
* given a set of learned models, we need to identify the right model to query for a test sample.
* besides alleviating forgetting, we also would like to facilitate information sharing among tasks.

We give a solution for the above cruxes under the framework of parameter efficient tuning (PET) of pre-trained language models (PLMs). For each task in the continual learning sequence, we build a model for it using two sets of parameters: one is from a frozen PLM, another is an additional small set of new parameters (namely, _delta parameters_[16]). PET methods [11; 12; 13] show that by only fine-tuning delta parameters, performances of downstream language learning tasks could be competitive with (or even better than) the full-scale fine-tuning of the PLM. Therefore, we can share

the PLM among all tasks in continual learning, while keeping a private delta parameter for each task, which is usually negligible compared with the PLM (less than \(0.3\%\) of the PLM's parameters in our experiments).

Second, to determine which model to query in testing time, we propose using a nonparametric task identification method. Specifically, for each task, we record the first and the second moment of its training samples, and approximate the task's input distribution with a Gaussian. Given a sample, we test whether it belongs to a task using its Mahalanobis distance to the task's Gaussian Lee et al. (2018); Ren et al. (2021). We show that, compared with state-of-the-art parametric task identifiers Wang et al. (2022b, a), the nonparametric method significantly boosts the accuracy of getting the correct model (and performances of continual learning) despite its simplicity.

Third, to enhance knowledge transfer among continually learned tasks, we investigate information sharing methods among delta parameters of each task. We show that, by simply initializing the current task's delta parameters with those of previous tasks (either by directly copying from them or soft selecting via attention mechanisms), the transfer of knowledge from learned tasks could be improved, especially in few-shot situations.

We conduct extensive evaluations on standard continual language learning benchmarks. The results show that our algorithm not only outperforms all existing rehearsal-free methods by a large margin (\(40\%\) F1), but also is competitive with (or even better than) state-of-the-art rehearsal-based methods with the standard setting of data cache size.

## 2 Problem Definition

Continual learning aims to solve problems in streams: a model no longer faces a single unchanging task but a series of tasks arrived sequentially. Following previous works, we focus on task streams containing text classification problems, while the algorithm could be extended to other language learning problems.

Denote \({\cal T}=\{T_{1},T_{2},\cdots,T_{|{\cal T}|}\}\) as a sequence of tasks, and the training set of the \(t\)-th task \(T_{t}\) is \(\{(x^{t}_{i},y^{t}_{i})\}\), where \(x^{t}_{i}\in{\cal X}_{t}\) is an input text, \(y^{t}_{i}\in{\cal Y}_{t}\) is its class label. At timestamp \(t\), a continual learning model learns a mapping \(f_{\theta}:\cup_{\tau=1}^{t}{\cal X}_{\tau}\mapsto\cup_{\tau=1}^{t}{\cal Y}_{\tau}\). The key property is that the model can only query the training data of \(T_{t}\), and aims to update the model parameter \(\theta\) to not only predict the \(t\)-th task, but also keep performances on all previous tasks (even without using their data). In the testing phase, the model predicts a sample's label from all seen labels \(\cup_{\tau=1}^{t}{\cal Y}_{\tau}\) without knowing which task it belongs to (_class incremental_).

In this work, we will focus on the above _rehearsal-free_ setting of continual learning. Several relaxations of this setting could be applied to upper-bound its performances,

* _rehearsal-based_ continual learning, where some previous tasks' data can be used when training the current task. If all previous data are given, the problem is reduced to multi-task learning.
* _task incremental_ continual learning, where task labels are given in testing time. It provides additional information about samples and makes prediction easier than the class incremental setting.

## 3 Approaches

Our method is summarized in Figure 1. In the training stage, for a new task, we assign (and save) a new private delta parameter for it and train the parameter (jointly with a shared frozen PLM) on the task's dataset. We also save some statistics about the training set to model the task as a Gaussian distribution (e.g., the averaged vector representations of samples). In the testing stage, for a given sample, we fetch a proper learned delta parameter for predicting its label by consulting the Mahalanobis distance between the sample and the distributions of all previously encountered tasks. In the following, we start by describing the parameter isolation strategy with PET methods (Section 3.1). Then, we illustrate the task identification method used in the testing stage (Section 3.2) and knowledge sharing methods (Section 3.3).

### Parameter Isolation with PET

One reason that causes catastrophic forgetting in continual learning is the interference among tasks: different tasks may guide the model to move towards different directions. Therefore, a simple way to alleviate forgetting is to use different parameters for different tasks. On the other side, a sufficient model capacity for each task is crucial to get high-performance models. Directly separating models could make large storage costs. Here, we adopt the parameter-efficient tuning framework which is able 

to take advantage of a fixed powerful pre-trained model while keeping individual tasks moderately separated. In the following, we briefly review prefix tuning Li and Liang (2021) (the main PET method used in our experiments). 2

Footnote 2: The same idea can be applied with other PET methods (e.g., prompt-tuning Hu et al. (2022), LoRA Liu et al. (2022a)).

Denote a Transformer-based pre-trained model with multi-head attention blocks to be \({\rm MHA}(\mathbf{X})\). It maps an input sentence \(\mathbf{X}\in\mathbb{R}^{n\times d}\) to a hidden semantic space \(\mathbf{H}\in\mathbb{R}^{n\times d}\), where \(n\) is the sentence length, \(d\) is the dimension of token embeddings and hidden vectors. For the \(t\)-th task in continual learning, we prepare a set of task-specific parameters by prepending \(p\) soft virtual tokens to the input of multi-head attention blocks at each layer. Denote parameters in the prefix as \(\mathbf{P}\in\mathbb{R}^{p\times d\times L}\), where \(L\) is the number of MHA layers in the pre-trained model. 3

Footnote 3: In the implementation, virtual tokens are added by concatenating two \(\mathbb{R}^{p\times d}\) matrices to key and value matrices.

Besides prefixes \(\mathbf{P}\), another set of task-specific parameters are classification heads (we apply softmax activation for the multi-class classification). 4 During the training process, parameters of the PLM are frozen (thus, shared among all tasks), and only private prefixes and classification heads are trained.

Private parameters are saved after training. One key property of PET methods is that one can train well enough models for tasks with a small size of \(\mathbf{P}\) and a powerful PLM. It facilitates reducing the storage cost for parameter isolation methods. while retaining high-performance models for individual tasks (e.g., decrease by \(1.34\%\) in a benchmark dataset). In our experiments, the size of \(\mathbf{P}\) is \(0.27\%\) of the PLM. Hence, the same storage budget for two tasks with full PLM fine-tuning can support \(370\) tasks with prefix tuning.

Footnote 4: For simplicity, we assume tasks have disjoint label spaces, thus their classification heads are different. In practice, those heads could be shared.

### Task Identification

As we use different prefixes for different tasks, when a test sample comes, it is necessary to determine which prefix should be applied. One approach is to try all learned prefixes, and choose one with the largest prediction probability on the sample. In practice, this method suffers from the overconfidence problem of softmax predictors: even for those unrelated tasks, a prefix might give a prediction with high probability.

Another approach is to compare the test sample with training samples and choose its nearest neighbor's prefix Mensink et al. (2013). It works around the over-confidence problem, but still depends on

Figure 1: The process of our method. The left panel of the figure shows the training process. We counted the class prototypes and covariance matrix of each dataset while training. The right panel shows the inference process, which consists of two forward propagations: (A) obtain a PET module suitable for the input sample by PLM. (B) predict the label of it by PLM with the selected module.

 

the robustness of sample representation and distance metrics. Here, inspired by recent studies on out-of-distribution detection (Lee et al., 2018; Ren et al., 2021), we improve this method by comparing the test sample with full distributions of tasks' training samples. In the following, we represent an input \(x\) with the average of the last Transformer block's outputs of the PLM, denoted by \(h(x)\) (i.e., the "pre-logit" vector).

First, for the \(t\)-th task, we perform a Guassian discriminant analysis as suggested by the softmax classifier: given \(p(y=c|h(x))\) in the form of softmax (\(c\in{\cal Y}_{t}\) is a class label), we can assume \(p(h(x)|y=c)\) is a Gaussian with estimated mean \(\mu^{c}_{t}\) and a shared covariance \(\mathbf{\Sigma}_{t}\) among classes,

\[\mu^{c}_{t}=\frac{1}{N_{c}}\sum_{y_{i}=c}h(x_{i}), \tag{1}\] \[\mathbf{\Sigma}_{t}=\frac{1}{N}\sum_{c}\sum_{y_{i}=c}\left(h(x_{i})- \mu^{c}_{t}\right)\left(h(x_{i})-\mu^{c}_{t}\right)\overline{\phantom{\mathbf{ \Sigma}_{t}}} \tag{2}\]

where \(N_{c}\) is the number of training samples with label \(c\), and the values are obtained by maximum likelihood estimation.

Next, for a test sample \(x\), we compare it with all tasks' Gaussian and choose the nearest task's prefix as the prefix for predicting \(x\)'s label. Unlike computing distance between samples, we need to apply metrics measuring distance between samples and distributions. Here, we use Mahalanobis distance. Specifically, the distance between \(x\) and class \(c\) is

\[-\left(h(x)-\mu^{c}_{t}\right)^{\top}\left(\mathbf{\Sigma}_{t}\right)^{-1}\left( h(x)-\mu^{c}_{t}\right). \tag{3}\]

In practice, we find that directly ranking tasks with \(\mathbf{\Sigma}_{t}\) makes the distances have large numeric deviation. To make the computation more stable, we further share the covariance among all tasks \(\mathbf{\Sigma}=\sum_{t}\mathbf{\Sigma}_{t}\) and change the computation of Mahalanobis distance accordingly.

To implement above non-parametric task identification method, we need additional storage for store class means \(\{\mu^{c}_{t}\}\) and a shared covariance \(\mathbf{\Sigma}\). Furthermore, though they are moments of distributions, there may be a chance to get information about individual samples (Dwork and Roth, 2014). Here, inspired by querying with randomized response in differential privacy, we propose to add randomly masking on sample representations \(h(x)\). Specifically, we assign a random mask (with \(q\%\) entries \(0\), other entries \(1\)) for all tasks. For each \(h(x)\), the masked dimensions are dropped during the computation of \(\mu^{c}_{t}\) and \(\mathbf{\Sigma}^{c}_{t}\). The model saves the masked vectors and the mask itself (or the random seed generating the mask) for testing time distance computation. Besides encrypting moments, the simple masking strategy also helps to reduce storage cost.

### Knowledge Transfer

Separating parameters is effective in mitigating catastrophic forgetting, however, it also blocks knowledge transfer among tasks. Given a sequence of learned prefixes \(\mathbf{P}=\{\mathbf{P}_{1},...,\mathbf{P}_{t-1}\}\), we try several ways to utilize the knowledge acquired from preceding tasks in the hope that they could improve and accelerate the current task learning,

* **Prefix Fusion.** A natural way to combine knowledge from previous tasks is through the use of the attention mechanism. This allows the model to automatically extract useful information from previous tasks and integrate it into the current learning process. To achieve this, we prepend the learned prefixes of previous tasks to the prefix of the current task. Knowledge transfer is automatically facilitated through the multi-head attention mechanism of the Transformer. During new prefix learning, we fix the prefixes of previous tasks to avoid parameter drifting and catastrophic forgetting.
* **Prefix Initialization.** Training starting from well-trained parameters is another way to promote knowledge transfer, thus we can initialize a prefix with previously learned prefixes instead of random initialization. A good initial point can also help to speed up the convergence of the training process of PET methods. We try two ways, namely initialized with the _last prefix_\(\mathbf{P}_{t}\leftarrow\mathbf{P}_{t-1}\) and initialized with the _mean prefix_\(\mathbf{P}_{t}\leftarrow\frac{1}{t-1}\sum_{\tau=1}^{t-1}\mathbf{P}_{\tau}\).

## 4 Experiments

DatasetsTo demonstrate the generalizability of our approach, we use two kinds of datasets, differentiated according to the domain relevance between tasks. _Far-domain_, where the domain boundary among tasks are clear. Following MbPA++ (de Masson d'Autume et al., 2019) and IDBR (Huang et al., 2021a), we use 5-datasets collected by Zhang et al. (2015) to evaluate our method. It consists AG News (news), Yelp (business reviews), Amazon (product reviews), Yahoo!Answer (Q&A), and DBPedia (encyclopedic articles). These datasets are categorized into two 

text classification tasks: topic classification (AG News, Yahoo!Answers, and DBPedia) and sentiment classification [21] where the tasks are more closely related. We use Web of Science (WOS) from kowsari-etal-2017-semevaluating and 20 Newsgroups from lang-1995-kang-etal-2015-semevaluating to assess our method for datasets with high inter-task relevance. WOS contains seven parent classes and five sub-classes under each parent class which have close relations. We organize continual learning tasks according to parent classes. The 20 Newsgroups consists six topics of news. We rearranged it into four tasks based on the principle of maximizing inter-task correlation. The details of the two datasets are in Appendix B.

MetricsLet \(a_{i,j}\) be the testing accuracy on the \(i\)-th task after training on \(j\)-th task, the metrics for evaluating are,

* **Performance of Continual Learning (CL)**. The average accuracy of all tasks after training on the last task, \(\frac{1}{|\mathcal{T}|}\sum_{i=1}^{|\mathcal{T}|}a_{i,|\mathcal{T}|}\)
* **Forgetting (Fgt)**. The degree of forgetting of previous tasks after training on the last task, \(\frac{1}{|\mathcal{T}|-1}\sum_{i=1}^{|\mathcal{T}|-1}(\max_{k=i}^{|\mathcal{ T}|-1}a_{i,k}-a_{i,|\mathcal{T}|})\)
* **Accuracy of Task Identification (TI)**. The accuracy of getting the correct prefix for testing samples after training on all tasks.

BaselinesWe use the following continual learning techniques as baselines:

* **FT**, fine-tuning a model for each task sequentially while catastrophic forgetting occurs. This method is the lower bound of continual learning.
* **MTL**, training a model on all tasks as multi-task learning. This method is the upper bound of continual learning.
* **Replay**, saving part of the previous tasks as memory, train a model one step on the memory after every \(\beta\) ( 10 for our experiments) steps of training on the new task.
* **LwF**Li and Hoiem (2017), a typical regularization-based approach. We also combine LwF with the replay method as an enhancement.
* **IDBR**Huang et al. (2021), disentangling information by two simple auxiliary tasks (next sentence prediction and task-id prediction) for learning better generic and specific representation spaces. This approach applies both episodic memory replay and regularization techniques.
* **L2P**Wang et al. (2022), it first introduces a prompt-based framework to continual learning. The main difference between L2P and our method is that L2P uses a parametric task identifier, while our identifier is non-parametric. We also adopt L2P-R, which is L2P equipped with a rehearsal buffer. To ensure a fair comparison, **FT, MTL, Replay** and **LwF** are all prefix-based. We re-implement **L2P** to make it support prefixes (instead of only prompts in the original code). DetailsWe use BERT Devlin et al. (2019) from HuggingFace Transformers Wolf et al. (2020) as the PLM of our model. We set the default prefixes length to \(16\). Because our methods progressively assign a prefix to new tasks, we assign the same number of parameters to all baseline methods to ensure a fair comparison. See Appendix C for other configurations.

### Main Results

In Table 1, we present the results of our method and baselines on the sampled 5-datasets and WOS. On the sampled 5-datasets, the rehearsal-based approach stores 50 samples per class, equivalent to 2.5% of the entire training dataset (except for IDBR, which is 20 per class). From the results, we can find that,

* On sampled 5-datasets, our approach surpasses all rehearsal-free methods by a large margin. It is even better than rehearsal-based methods: compared with the previous SOTA method IDBR, our method improves the accuracy from 73.19% to 74.43%, reducing the gap to the upper bound (75.40%) by 56%.
* When changing task orders of the sampled 5-datasets, the standard deviation of our method is small, which indicates that it is insensitive to task orders, which is rare in approaches without replaying.
* On the more challenging WOS, our approach is still comparable to replay methods that use approximately 10% training data (20 samples per class) 

Table 2 presents the performances on the full 5-datasets. The conclusions drawn from this table are generally consistent with Table 1, but it also showcases some new findings:

* Our rehearsal-free method, even without task ID during the testing phase (class incremental), can outperform previous SOTA methods such as LAMOL Sun et al. (2020), which relies on rehearsal and has knowledge of the task ID during testing (task incremental).
* When provided with the task ID ("oracle"), our method's performance can be further improved by \(1.7\%\). This suggests that there is still potential for enhancing our approach.

### Discussions

To further inspect the proposed methods, we investigate the following research questions.

How the task identifier performs?To show the validity of the Mahalanobis distance (MD), we compare it with two methods: (i) Maximum over softmax probability (MSP): using all prefixes for inference (task by task) and then choosing the label with the highest probability; (ii) Euclidean distance (ED) which ignores the covariance in Equation 3. The results are in Figure 2. We find that,

* MSP performs poorly (especially on 5-datasets). It shows that, at least for the vanilla MSP metric, the over-confidence problem is still essential. Another drawback of MSP is that it has to perform forward passes with all prefixes while our method only needs to compare vectors.
* Mahalanobis distance is always better than Euclidean distance for task identification. Therefore, different dimensions of sample representations should have different importance for detecting tasks, and modeling tasks' distribution with anisotropic Gaussians provides a better approximation of the actual distribution.

\begin{table}
\begin{tabular}{l||c|c c c c c||c} \hline \hline \multirow{2}{*}{**Method**} & **Buffer** & \multicolumn{4}{c}{**5-datasets**} & \multirow{2}{*}{**Average**} & **Buffer** & \multirow{2}{*}{**WOS**} \\  & **Size** & **Order 1** & **Order 2** & **Order 3** & & & **77.86\(\pm\)**0.48 \\ \hline Replay & \multirow{2}{*}{50/class} & 67.87\(\pm\)0.18 & 68.27\(\pm\)0.46 & 69.01\(\pm\)0.10 & 68.38\(\pm\)0.47 & \multirow{2}{*}{20/class} & **77.86\(\pm\)**0.48 \\   +LwF & \multirow{2}{*}{50/class} & 70.08\(\pm\)0.15 & 70.12\(\pm\)0.09 & 69.65\(\pm\)0.36 & 69.95\(\pm\)0.21 & \multirow{2}{*}{20/class} & 76.78\(\pm\)0.66 \\ L2P-R & & 68.02\(\pm\)0.27 & 67.90\(\pm\)0.16 & 68.49\(\pm\)0.27 & 68.14\(\pm\)0.25 & & 77.10\(\pm\)0.44 \\ IDR\({}^{\dagger}\) & (20/class) & 72.63 & 73.72 & 73.23 & 73.19\(\pm\)0.44 & & – \\ \hline FT & \multirow{2}{*}{0} & 29.54\(\pm\)0.63 & 36.06\(\pm\)0.88 & 24.65\(\pm\)1.16 & 30.08\(\pm\)4.67 & \multirow{2}{*}{53.86\(\pm\)1.64} \\ LwF & & 27.40\(\pm\)0.98 & 31.27\(\pm\)0.91 & 23.34\(\pm\)2.01 & 27.33\(\pm\)3.24 & \multirow{2}{*}{0} & 30.96\(\pm\)0.48 \\ L2P & & 30.34\(\pm\)0.32 & 35.78\(\pm\)0.12 & 23.45\(\pm\)0.47 & 29.86\(\pm\)5.05 & & 54.62\(\pm\)0.85 \\
**Ours** & & **74.38\(\pm\)0.04** & **74.53\(\pm\)0.07** & **74.38\(\pm\)0.01** & **74.43\(\pm\)0.07** & & 77.83\(\pm\)**0.14** \\ \hline MTL & – & 75.40 & 75.40 & 75.40 & 75.40 & – & 85.25 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summarization of results on sampled setting of 5-datasets and WOS. We report the average accuracy and standard deviation over 3 runs. \(\dagger\) denotes the results come from Huang et al. (2021b). Dash line indicates data is not available or not meaningful. The best results of all methods are bolded, and the best results of rehearsal-free methods are underlined. We also bold the smallest standard deviation of all methods, where a smaller one indicates a more robust method.

\begin{table}
\begin{tabular}{l l l l l l l l l l} \hline \hline
**Method** & **MR** & **TT** & **TI** & **LA** & \multicolumn{4}{c}{**5-datasets**} \\ \hline
**Order** & & & & **1** & **2** & **3** & **4** & **Average** \\ MbPA++\({}^{\dagger}\) & ✓ & & & ✓ & 74.9 & 73.1 & 74.9 & 74.1 & 74.3 \\ LAMOL\({}^{\dagger}\) & ✓ & ✓ & ✓ & 76.1 & 76.1 & 77.2 & 76.7 & 76.5 \\ IDBR\({}^{\dagger}\) & ✓ & ✓ & & 75.9 & 76.2 & 76.4 & 76.7 & 76.3 \\ \hline Ours & & ✓ & & **77.4** & **77.3** & **77.2** & **77.4** & **77.3** \\ Ours (oracle) & & ✓ & ✓ & 79.1 & 79.0 & 78.9 & 78.9 & 79.0 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results on the full setting of 5-datasets. All results are averaged over 3 runs. \(\dagger\) denotes the results come from Huang et al. (2021b). To compare different methods, four method features have been defined, with a checkmark indicating the presence of the corresponding feature. Specifically, **MR** indicates whether the method requires replaying stored memory (rehearsal-based or rehearsal-free). **TT** indicates whether task-id is available during training. **TI** indicates whether task-id is available during inference. **LA** indicates whether performing local adaptation during inference.

 



**Does prefix tuning help continual language learning?** We can directly use the non-parametric task identifier to infer class labels of samples. Hence, one question is whether the additional prefix parameters provide performance gains. We build two non-parametric classifiers which use Euclidean and Mahalanobis distance. The samples are encoded only with representations from the PLM (without prefixes). As shown in Table 3, we can find that,

* Regarding the performance of continual learning (CL), the two non-parametric classifiers perform quite well: the classifier with Euclidean distance performs much better (\(62.8\%\)) than all rehearsal-free method in Table 1 (\(<30\%\)) on 5-datasets, and is competitive on WOS. The classifier with Mahalanobis distance is more effective (\(69.8\%\)): it is even competitive with the primary reply method (**Replay**). The success of non-parametric classifiers implies that the powerful representation ability from PLM is crucial to perform continual learning.
* With the help of parametric prefix tuning, the performances of the two methods are largely boosted. It proves that task-specific information is also important. Regarding the performances of direct fine-tuning in Table 1, using separate prefixes could alleviate catastrophic forgetting.

How do different PLMs influence performances?The above analyses on non-parametric classifiers suggest us to explore the influence of different PLMs (Figure 3 and Table 4). We find that,

* Owing to the presence of a larger number of parameters, the sentence representations generated by larger models enhance task identification. Moreover, previous studies also suggest that the larger the PLM, the more effective the PET Lester et al. (2021); Liu et al. (2022b). Figure 3 presents the performance across model scales, which are consistent with the previous findings in continual language learning.
* The PLM pre-trained on the same domain of continual learning tasks is able to extract a more valuable representation. For WOS, we replace BERT with SciBERT Beltagy et al. (2019), which is a BERT-like model pre-trained on the scientific (WOS domain) corpus. As shown in Table 4, the closer SciBERT performs even better than bert-large though it is smaller (with the same size as bert-base). Therefore, if some unsupervised texts from the same domain of continual learning are given, one could fine-tune the PLM for a better performance.

Figure 3: The results of bert-base (~110M) and bert-large (~340M) on three datasets. \(\uparrow\) means larger is better, and \(\downarrow\) means smaller is better.

Figure 2: Comparisons of prompt selection strategies on three datasets. We report the average result over 3 runs. MSP: Maximum Softmax Probability. ED: Euclidean Distance. MD: Mahalanobis Distance.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method** & **Parametric** & **5-datasets** & **WOS** & **20 Newsgroups** \\ \hline Euclidean & no & 62.82 & 53.61 & 53.43 \\ + Prefix & yes & 71.44 & 66.99 & 55.78 \\ Mahalanobis & no & 69.78 & 72.36 & 63.08 \\ + Prefix & yes & **74.38** & **77.84** & **64.05** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparing non-parametric and parametric classifiers. The best results is highlighted in bold.



How random masking influence the task identifier?To validate the effect of random masking, we use five masking ratios equally spaced from 0% to 80% and two high masking ratios (95% and 99%) The results are in Figure 4. Our method shows a slight performance degradation in the range of masking ratio from 0% to 80%. Compared with no mask (i.e., masking ratio equal to 0), a masking ratio of 80% still gives more than 90% of the performance, and it only needs about 4% of the storage space for all three datasets. It suggests that pre-logit vectors (\(h(x)\)) is highly redundant for distinguishing tasks. However, as the masking ratio increases further, the task identification performances decrease rapidly, which causes the end continual learning performances to decrease as well.

Does knowledge transferability of prefixes matter?Finally, we evaluate knowledge transfer methods (Section 3.3) in few-shot settings. As shown in Table 5, we observe knowledge transfer in all settings on WOS for all transfer methods but only in the 10-shot setting on 5-datasets for **Mean**. This is because the tasks in the WOS dataset are related, and thus knowledge can be shared among them. Moreover, we do not observe forward transfer in the full-shot setting on any benchmark, because knowledge transfer is unnecessary when the data are sufficient.

## 5 Related work

Continual LearningWe discuss three main categories of continual learning methods: _rehearsal-based_, _regularization-based_, and _parameter isolation_ methods.

_Rehearsal-based methods_ alleviate catastrophic forgetting by replaying stored examples [26, 7, 1] or pseudo-generative examples [27, 28, 29] of previous tasks. Unfortunately, all of them carry the risk of privacy leakage and need nontrivial storage space.

_Regularization-based methods_ restrict the updating of the model weights by knowledge distillation [18, 19] or parameter importance [17, 21, 22] to preserve the knowledge of previous tasks. However, these methods emphasize the model's stability to previous tasks while weakening its plasticity to the new task [17].

_Parameter isolation methods_ assign a task-specific parameter to a new task by splitting [26, 27, 28] or extending [26, 27] the current model to prevent interference between tasks. Because these methods require task-id to choose the proper model at testing time, they only apply to task incremental

Figure 4: Accuracy of task identification and continual learning vs. random masking ratio. The results are averaged over 3 runs. Storage Ratio: The ratio of storage space required for mask to that without mask.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Size** & **TI** & **CL** & **Fgt** \\ \hline
**bert-base** & 110 M & 85.78 & 77.84 & 6.14 \\
**bert-large** & 340 M & 87.48 & 79.90 & 5.63 \\
**sci-bert-base** & 110 M & **87.98** & **79.92** & **5.01** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparisons between models of varying sizes (base vs. large) or trained on different datasets (bert vs. sci-bert) on the WOS dataset. The average results over 3 runs are reported, with the best results highlighted in bold.

 

continual learning. Although our method also belongs to this category, our method applies to class incremental by introducing task identification.

## 6 Conclusion

In this work, we propose a new rehearsal-free parameter isolation continual learning method that leverages the capabilities of a pre-trained language model (PLM). Extensive experiments show that our method surpasses all rehearsal-free methods by a significant margin and is comparable (or even better) than previous start-of-the-art rehearsal-based methods on two benchmarks, whether the tasks are near or far. Meanwhile, we introduce random static masking to reduce the storage required by our method to adapt it to more demanding scenarios.

### Limitations

Although our proposed knowledge transfer methods work well on WOS in the few-shot setting, it is less effective on 5-datasets. Moreover, all methods fail in the full-shot setting. Based on our approach, a more general approach to knowledge transfer is expected in future works. In addition, our approach requires a well-trained language model for task identification and a Transformer-based model (well-trained also) for parameter efficient tuning. Therefore, it is challenging to cooperate our approach with a language model with random initialization or non-transformer architecture.

## Acknowledgement

The authors wish to thank all reviewers for their helpful comments and suggestions. The corresponding authors are Zhicheng Wang, Tao Ji and Yuanbin Wu. This research was (partially) supported by NSFC(62076097), National Key R&D Program of China (2021YFC3340700), and East China Normal University International Conference Grant Programme.

## References

* ECCV 2018
- 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part III, Lecture Notes in Computer Science, pp. 144-161. Cited by: SS1.
* I. Beltagy, K. Lo, and A. Cohan (2019)SciBERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 3615-3620. External Links: Link, Document Cited by: SS1.
* M. Biesialska, K. Biegisalska, and M. R. Costa-jussa (2020)Continual lifelong learning in natural language processing: A survey. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 6523-6541. Cited by: SS1.
* M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars (2021)A continual learning survey: defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence44 (7), pp. 3366-3385. Cited by: SS1.
* C. de Masson d'Autume, S. Ruder, L. Kong, and D. Yogatama (2019)Episodic memory in lifelong language learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 13122-13131. Cited by: SS1.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171-4186. External Links: ISBN 978-3-319-3 

Yufan Huang, Yanzhe Zhang, Jiaoa Chen, Xuezhi Wang, and Diyi Yang. 2021a. Continual learning for text classification with information disentanglement based regularization. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021_, pages 2736-2746. Association for Computational Linguistics.
* Huang et al. (2021b) Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. 2021b. Continual learning for text classification with information disentanglement based regularization. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2736-2746, Online. Association for Computational Linguistics.
* Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. _Proceedings of the national academy of sciences_, 114(13):3521-3526.
* Kirkpatrick et al. (2016) James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kiearan Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2016. Overcoming catastrophic forgetting in neural networks. _CoRR_, abs/1612.00796.
* Kowsari et al. (2017) Kamran Kowsari, Donald E Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S Gerber, and Laura E Barnes. 2017. Holttex: Hierarchical deep learning for text classification. In _2017 16th IEEE international conference on machine learning and applications (ICMLA)_, pages 364-371. IEEE.
* Lang (1995) Ken Lang. 1995. Newsweeder: Learning to filter netnews. In _Proceedings of the Twelfth International Conference on Machine Learning_, pages 331-339.
* Lee et al. (2018) Kimin Lee, Ribok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montreal, Canada_, pages 7167-7177.
* Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 3045-3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Lhoest et al. (2017) Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Sasko, Gunjan Chhablami, Bravitya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry, Angelina McMillan-Major, Philipp Schmid, Sylvian Gugger, Clement Delangue, Theo Matsuirex, Lysandre Debut, Stas Bekman, Pierric Cisatec, Thibault Goehringer, Victor Mustar, Francois Lagunas, Alexander Rush, and Thomas Wolf. 2021. Datasets: A community library for natural language processing. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_, pages 175-184, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 4582-4597.
* Li and Hoiem (2017) Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. _IEEE transactions on pattern analysis and machine intelligence_, 40(12):2935-2947.
* Liu et al. (2022a) Xiao Liu, Kaikuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022a. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 61-68.
* Liu et al. (2022b) Xiao Liu, Kaikuan Ji, Yicheng Fu, Weng Tam, Zhenxiao Du, Zhilin Yang, and Jie Tang. 2022b. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 61-68, Dublin, Ireland. Association for Computational Linguistics.
* Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. _arXiv preprint arXiv:1711.05101_.
* Mallya and Lazebnik (2018) Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In _2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018_, pages 7765-7773. Computer Vision Foundation / IEEE Computer Society.
* Mensink et al. (2013) Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. 2013. Distance-based image classification: Generalizing to new classes at near-zero cost. _IEEE Trans. Pattern Anal. Mach. Intell._, 35(11):2624-2637.
* Parisi et al. (2019) German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. _Neural Networks_, 113:54-71 

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarI: Incremental classifier and representation learning. In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_, pages 2001-2010.
* Ren et al. (2021) Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. 2021. A simple fix to mahalanobis distance for improving near-ood detection. _CoRR_, abs/2106.09022.
* Rolnick et al. (2019) David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. _Advances in Neural Information Processing Systems_, 32.
* Rusu et al. (2016) Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. _CoRR_, abs/1606.04671.
* Serra et al. (2018) Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018. Overcoming catastrophic forgetting with hard attention to the task. In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmassan, Stockholm, Sweden, July 10-15, 2018_, volume 80 of _Proceedings of Machine Learning Research_, pages 4555-4564. PMLR.
* Shin et al. (2017) Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. _CoRR_, abs/1705.08690.
* Su et al. (2020) Xin Su, Shangqi Guo, Tian Tan, and Feng Chen. 2020. Generative memory for lifelong learning. _IEEE Trans. Neural Networks Learn. Syst._, 31(6):1884-1898.
* Sun et al. (2020) Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2020. LAMOL: language modeling for lifelong language learning. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Triki et al. (2017) Amal Rannen Triki, Rahaf Aljundi, Matthew B. Blaschko, and Tinne Tuytelaars. 2017. Encoder based lifelong learning. In _IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017_, pages 1329-1337. IEEE Computer Society.
* ECCV 2022
- 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXVI_, volume 13686 of _Lecture Notes in Computer Science_, pages 631-648. Springer.
* Wang et al. (2022b) Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. 2022b. Learning to prompt for continual learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 139-149.
* Demos, Online, November 16-20, 2020_, pages 38-45. Association for Computational Linguistics.
* Xu and Zhu (2018) Ju Xu and Zhanxing Zhu. 2018. Reinforced continual learning. In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montreal, Canada_, pages 907-916.
* Zenke et al. (2017) Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_, volume 70 of _Proceedings of Machine Learning Research_, pages 3987-3995. PMLR.
* Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. _Advances in neural information processing systems_, 28.

 



## Appendix A Potential Risks

Pre-trained language models (PLMs) may inherit biases from the training corpus, resulting in offensive behaviors. Combining our approach with these toxic language models and deploying them in realistic scenarios might cause negative social impacts.

## Appendix B Datasets Details

For 5-datasets, the sampled setting samples 2,000 training examples and 2,000 validation examples from the original training set. The full setting is the same as the dataset used by de Masson d'Autume et al. (2019). We show task orders of 5-datasets we used in Table 6. We did not merge the label space of Yelp and Amazon as MbPA++ and IDBR did to create a more challenging setup. Our label space can be mapped to the label space used by MbPA++ and IDBR and not vice versa.

We download WOS-11967 from Huggingface Datasets Lhoest et al. (2021), and we show the statistics of it in Table 7. We split the whole dataset to train/val/test set in the ratio of 0.6:0.2:0.2.

We access 20 Newsgroups from HuggingFace Datasets corresponding to the 20news-bydata version on the official site. We show the task separation in Table 8. We take one-sixth of the training set as a validation set for a train/val/test splitting ratio 0.5:0.1:0.4.

## Appendix C Experimental Details

We train all models using AdamW Loshchilov and Hutter (2017) with \(\beta_{1}=0.9\) and \(\beta_{2}=0.99\) coupled with a linear scheduler with a warm-up ratio of 0.1. For sampled 5-datasets, WOS, and 20 Newsgroups, we set the identical learning rate \(\lambda=0.03\). For the full 5-datasets, we do grid searching for the learning rate for each task respectively, and the final learning rates are \(0.003,0.009,0.005,0.007,0.003\) for AG News, Amazon, Yelp, Yahoo!Answer and DBpedia, respectively. All experiments are conducted on NVIDIA RTX 3090 with 24GB video memory with a batch size of 32 and the maximum length of a sentence is 256.

\begin{table}
\begin{tabular}{l|c c} \hline \hline
**Order** & **Task Sequence** \\ \hline
1 & ag →yel → amazon → yahoo → dbpedia \\
2 & yelo →yahoo → amazon → dbpedia → ag \\
3 & dbpedia →yahoo → ag → amazon → yeplý \\
4 & ye lap →ag → dbpedia → amazon → yahoo \\ \hline \hline \end{tabular}
\end{table}
Table 6: Four task orders of the 5-datasets.

\begin{table}
\begin{tabular}{l l r r r r r r} \hline \hline Task ID & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ \# of examples & 1499 & 1132 & 1959 & 1925 & 2107 & 1617 & 1728 \\ \hline \hline \end{tabular}
\end{table}
Table 7: The statistic of original WOS-11967

\begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Class Name** & **Train** & **Test** \\ \hline Task 1 & & & \\ comp.graphics & 584 & 389 \\ rec.autos & 594 & 396 \\ sci.crypt & 595 & 396 \\ misc.forsale & 585 & 390 \\ talk.polities.misc & 465 & 310 \\ talk.religion.misc & 377 & 251 \\ \hline \hline Task 2 & & & \\ comp.os.ms-windows.misc & 591 & 394 \\ rec.motorcycles & 598 & 398 \\ sci.electronics & 591 & 393 \\ talk.pollicts.guns & 546 & 364 \\ alt.atheism & 480 & 319 \\ \hline \hline Task 3 & & & \\ comp.sys.ibm.pc.hardware & 590 & 392 \\ rec.sport.baseball & 597 & 397 \\ sci.med & 594 & 396 \\ talk.pollicits.mideast & 564 & 376 \\ soc.religion.christian & 599 & 398 \\ \hline \hline Task 4 & & & \\ comp.sys.mac.hardware & 578 & 385 \\ rec.sport.hockey & 600 & 399 \\ sci.space & 593 & 394 \\ comp.windows.x & 593 & 395 \\ \hline \hline \end{tabular}
\end{table}
Table 8: The task separation and statistic of original 20 Newsgroups.





**ACL 2023 Responsible NLP Checklist**

A For every submission: A1. Did you describe the limitations of your work? _Section Limitations (Line 558-570) A2. Did you discuss any potential risks of your work? _Appendix A (Line 838-844) A3. Do the abstract and introduction summarize the paper's main claims? Abstract and section 1 (Line 1-112) A4. Have you used AI writing assistants when working on this paper? Left blank. B Did you use or create scientific artifacts? Section 4 (Line 297-323) B1. Did you cite the creators of artifacts you used? Section 4 (Line 297-323) B2. Did you discuss the license or terms for use and / or distribution of any artifacts? We do not find any license in original papers. B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? _Section 4 (Line 297-323)_ B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? _All of datasets comes from publicly available news or published articles, so we don't think there are such problems. B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? That information can be found in original papers. B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. _Appendix B (Line 845-865)_ C Did you run computational experiments? _Section 4_ C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? _Section 4 and append C (Line 876-879)

C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? _Section 4 and appendix C (Line 867-876)_
* C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? _Section 4_
* C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? _Section 4 (Line 368-370)_
* D **Did you use human annotators (e.g., crowdworkers) or research with human participants?** _Left blank._
* D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? _No response._
* D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants' demographic (e.g., country of residence)? _No response._
* D3. Did you discuss whether and how consent was obtained from people whose data you're using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? _No response._
* D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? _No response._
* D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? _No response._