Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023)

Abstract: We study the problem of defying catastrophic forgetting when learning a series of language processing tasks. Compared with previous methods, we emphasize the importance of not caching history tasks’ data, which makes the problem more challenging. Our proposed method applies the parameter isolation strategy. For each task, it allocates a small portion of private parameters and learns them with a shared pre-trained model. To load correct parameters at testing time, we introduce a simple yet effective non-parametric method. Experiments on continual language learning benchmarks show that our method is significantly better than all existing no-data-cache methods, and is comparable (or even better) than those using historical data.

## introduction deployment of nlp models could be dynamic in real-world scenarios: models need to evolve continually when coming in new tasks or new domains (updating an event detection model to handle new event types, for example). recent studies on continual (language) learning @xcite show that, compared with sticking with one single unchanging task, incrementally updating a model for a series of tasks is challenging: learning a new task will make our model perform poorly on previously learned tasks. the so-called catastrophic forgetting phenomenon is the central research topic in continual learning @xcite . one method for mitigating forgetting is through data replay (rehearsal). the model caches some previous tasks' training data along the sequential learning process. when learning a new task, all previous tasks are re-learned with the current task. the vanilla rehearsal-based approach performs well on many continual learning problems @xcite @xcite , but access to previous tasks' data could conflict with the original intention of continual learning. in fact, if previous tasks' data are given, it approaches multi-task learning where all tasks are given at once and forgetting is not a problem. meanwhile, in some situations, previous datasets are not available due to regulation or privacy considerations. therefore, it is essential to look into rehearsal-free methods. in this work, we study parameter isolation strategies for rehearsal-free continual language learning. basically, each time a new task comes in, such a strategy allocates a new set of model parameters for that task, which aims to prevent potential interference with already learned knowledge, so that forgetting could be alleviated @xcite . three cruxes of making the idea work in practice are, we give a solution for the above cruxes under the framework of parameter efficient tuning (pet) of pre-trained language models (plms). for each task in the continual learning sequence, we build a model for it using two sets of parameters: one is from a frozen plm, another is an additional small set of new parameters (namely, delta parameters @xcite ). pet methods @xcite @xcite show that by only fine-tuning delta parameters, performances of downstream language learning tasks could be competitive with (or even better than) the full-scale fine-tuning of the plm. therefore, we can share the plm among all tasks in continual learning, while keeping a private delta parameter for each task, which is usually negligible compared with the plm (less than 0.3% of the plm's parameters in our experiments). second, to determine which model to query in testing time, we propose using a nonparametric task identification method. specifically, for each task, we record the first and the second moment of its training samples, and approximate the task's input distribution with a gaussian. given a sample, we test whether it belongs to a task using its mahalanobis distance to the task's gaussian @xcite . we show that, compared with state-of-the-art parametric task identifiers @xcite , the nonparametric method significantly boosts the accuracy of getting the correct model (and performances of continual learning) despite its simplicity. third, to enhance knowledge transfer among continually learned tasks, we investigate information sharing methods among delta parameters of each task. we show that, by simply initializing the current task's delta parameters with those of previous tasks (either by directly copying from them or soft selecting via attention mechanisms), the transfer of knowledge from learned tasks could be improved, especially in few-shot situations. we conduct extensive evaluations on standard continual language learning benchmarks. the results show that our algorithm not only outperforms all existing rehearsal-free methods by a large margin (40% f1), but also is competitive with (or even better than) state-of-the-art rehearsal-based methods with the standard setting of data cache size. ## problem definition continual learning aims to solve problems in streams: a model no longer faces a single unchanging task but a series of tasks arrived sequentially. following previous works, we focus on task streams containing text classification problems, while the algorithm could be extended to other language learning problems. denote @xmath0, t 2 , • • • , t |t | } as a sequence of tasks, and the training set of the t-th task t t is {(x t i , y t i )}, where x t @xmath1, y t @xmath2. at timestamp t, a continual learning model learns a mapping f θ : the key property is that the model can only query the training data of t t , and aims to update the model parameter θ to not only predict the t-th task, but also keep performances on all previous tasks (even without using their data). in the testing phase, the model predicts a sample's label from all seen labels ∪ t τ =1 y τ without knowing which task it belongs to (class incremental). in this work, we will focus on the above rehearsal-free setting of continual learning. several relaxations of this setting could be applied to upper-bound its performances, ## approaches our method is summarized in figure 1 . in the training stage, for a new task, we assign (and save) a new private delta parameter for it and train the parameter (jointly with a shared frozen plm) on the task's dataset. we also save some statistics about the training set to model the task as a gaussian distribution (e.g., the averaged vector representations of samples). in the testing stage, for a given sample, we fetch a proper learned delta parameter for predicting its label by consulting the mahalanobis distance between the sample and the distributions of all previously encountered tasks. in the following, we start by describing the parameter isolation strategy with pet methods (section 3.1). then, we illustrate the task identification method used in the testing stage (section 3.2) and knowledge sharing methods (section 3.3). ## experiments datasets to demonstrate the generalizability of our approach, we use two kinds of datasets, differentiated according to the domain relevance between tasks. far-domain, where the domain boundary among tasks are clear. following mbpa++ (de masson d'autume et al., 2019) and idbr @xcite , we use 5-datasets collected by @xcite to evaluate our method. it consists ag news (news), yelp (business reviews), amazon (product reviews), yahoo!answer (q&a), and dbpedia (encyclopedic articles). these datasets are categorized into two text classification tasks: topic classification (ag news, yahoo!answers, and dbpedia) and sentiment classification (yelp and amazon). neardomain where the tasks are more closely related. we use web of science (wos) from @xcite and 20 newsgroups from @xcite to assess our method for datasets with high inter-task relevance. wos contains seven parent classes and five sub-classes under each parent class which have close relations. we organize continual learning tasks according to parent classes. the 20 newsgroups consists six topics of news. we rearranged it into four tasks based on the principle of maximizing inter-task correlation. the details of the two datasets are in appendix b. metrics let a i,j be the testing accuracy on the i-th task after training on j-th task, the metrics for evaluating are, baselines we use the following continual learning techniques as baselines: to ensure a fair comparison, ft, mtl, replay and lwf are all prefix-based. we re-implement l2p to make it support prefixes (instead of only prompts in the original code). details we use bert @xcite from huggingface transformers @xcite as the plm of our model. we set the default prefixes length to 16. because our methods progressively assign a prefix to new tasks, we assign the same number of parameters to all baseline methods to ensure a fair comparison. see appendix c for other configurations. ## conclusion in this work, we propose a new rehearsal-free parameter isolation continual learning method that leverages the capabilities of a pre-trained language model (plm). extensive experiments show that our method surpasses all rehearsal-free methods by a significant margin and is comparable (or even better) than previous start-of-the-art rehearsal-based methods on two benchmarks, whether the tasks are near or far. meanwhile, we introduce random static masking to reduce the storage required by our method to adapt it to more demanding scenarios.