Semi-Supervised Lifelong Language Learning (2022)

Abstract: Lifelong learning aims to accumulate knowledge and alleviate catastrophic forgetting when learning tasks sequentially. However, existing lifelong language learning methods only focus on the supervised learning setting. Unlabeled data, which can be easily accessed in real-world scenarios, are underexplored. In this paper, we explore a novel setting, semi-supervised lifelong language learning (SSLL), where a model learns sequentially arriving language tasks with both labeled and unlabeled data. We propose an unlabeled data enhanced lifelong learner to explore SSLL. Specially, we dedicate task-specific modules to alleviate catastrophic forgetting and design two modules to exploit unlabeled data: (1) a virtual supervision enhanced task solver is constructed on a teacher-student framework to mine the underlying knowledge from unlabeled data; and (2) a backward augmented learner is built to encourage knowledge transfer from newly arrived unlabeled data to previous tasks. Experimental results on various language tasks demonstrate our model’s effectiveness and superiority over competitive baselines under the new setting SSLL.

## introduction a remarkable ability of humans is to learn and accumulate knowledge continuously throughout their lifetime. such lifelong learning ability is crucial for computational systems interacting with the real world and processing continuous streams of information @xcite . however, most deep neural networks studies assume data distributions are stationary, which is not applicable in the real-world environments that dynamically evolve. in such real scenarios, models often suffer from catastrophic forgetting @xcite : a phenomenon where models forget previously learned knowledge when learning new tasks sequentially. various approaches have been proposed to alleviate catastrophic forgetting in lifelong scenarios. attempts include constraining the variants of important weights with regularization @xcite , storing real samples or using pseudo samples for previous tasks to maintain the learned knowledge @xcite @xcite , or dedicating taskspecific modules to avoid the interference among tasks @xcite @xcite . despite their reported effectiveness, these approaches are mostly designed to handle supervised learning tasks, where only labeled data are available. in real-world scenarios, labeled data are generally expensive and time-consuming to obtain, whereas unlabeled data are much easier to collect. these unlabeled data often carry rich information and have been successfully utilized to improve model performance in semi-supervised learning @xcite . in this paper, we investigate a novel setting: semi-supervised lifelong language learning (ssll), where a model learns sequentially arriving language tasks with limited labeled data and adequate unlabeled data (see the training process in fig. 1 ). the abundant information in the unlabeled data can not only facilitate learning the current task but also benefit learned tasks with similar data distributions. for example, sentiment analysis and topic classification tasks may only differ in their la-bel spaces while sharing the same set of unlabeled data. we can transfer knowledge among the two kinds of tasks using these unlabeled data @xcite . this phenomenon naturally leads to two challenges to be faced in the ssll scenario: (1) how to fully exploit unlabeled data to facilitate each arrived language task? and (2) how to leverage newly arrived unlabeled data to encourage knowledge transfer to previous tasks? with this in mind, we propose an unlabeled data enhanced lifelong learner (uell) framework to explore the new setting ssll. specifically, we dedicate task-specific parameters to alleviate catastrophic forgetting in uell. we construct two modules to tackle the challenges mentioned above. the first module is a virtual supervision enhanced solver that exploits unlabeled data using a teacher-student framework. the teacher generates pseudo labels for unlabeled data as virtual supervision and guides the student according to its learning progress. the student also learns from pseudo labels through self-study. the second module is a backward augmented learner that encourages knowledge transfer from the current task to previously learned tasks. the generated pseudo data for each learned task are augmented by retrieving semantically similar unlabeled samples from the current task, where the latter are leveraged to transfer knowledge backward to previously learned tasks. we conduct extensive experiments and analyses on both language understanding and generation tasks and demonstrate that uell can effectively address the challenges of ssll. our main contributions are as follows: semi-supervised learning aims to learn from both labeled and unlabeled data @xcite @xcite . as an efficient approach for semisupervised learning, pseudo labeling @xcite tries to utilize unlabeled data by predicting their labels. some studies @xcite train multiple learners and exploit disagreements among different learners. other studies utilize self-training to generate pseudo labels for unlabeled data @xcite @xcite . consistency regularization @xcite @xcite @xcite is another popular scheme. it regularizes the model to be invariant to small perturbations on the input, hidden states, or model parameters. mean teacher @xcite is an efficient method to implement consistency regularization, where a student model and a teacher model are maintained to enforce the predictions' consistency. this paper combines pseudo labeling with a teacher-student framework to handle semi-supervised learning in the lifelong scenarios. lifelong learning aims to learn a sequence of tasks without forgetting previously learned knowledge. three categories of approaches are generally used in lifelong learning: regularization-based methods either impose constraints on the variation of important weights when learning new tasks @xcite , or introduce knowledge distillation to preserve the previously learned knowledge @xcite ; replay-based methods store real samples @xcite or generate pseudo samples @xcite for learned tasks to consolidate previous knowledge; architecture-based methods construct task- student teacher augment teacher guidance self-study ema virtual supervision enhanced solver backward augmented learner retrain labeled data pseudo labeled data solvers for previous tasks unlabeled data grade good generate labels current task previous @xmath0. some studies @xcite use static architectures and apply task-specific routes through the architectures to prevent forgetting, while other studies @xcite @xcite dynamically expand the model with task-specific parameters. recently, some studies have tried to investigate semi-supervised lifelong learning for image classification tasks by modeling data distributions with generative adversarial networks @xcite , or relying on a super-class structure of image datasets to exploit unlabeled data @xcite @xcite . however, it is non-trivial to extend these works into language tasks since they either require continuous input spaces or utilize extra super-class structures, and all these works do not consider backward knowledge transfer with the help of unlabeled data. ## task definition and formulation in ssll, we sequentially learn a stream of semisupervised language tasks t 1 , t 2 , . . . , t n . the data of each task t t contain limited labeled data d s @xmath1, y s i )} and abundant unlabeled data d u @xmath2, where x s i and x u i are input samples and y s i is the output label of x s i . moreover, the data for task t t arrive after task t t-1 is learned @xcite , and we have no access to previous tasks' data. the final goal is to optimize the model's average performance on all tasks after training the whole sequence @xcite . inspired by @xcite , we frame different types of nlp tasks into a unified text-to-text format. specifically, for each (x s , y s ) ∈ d s , we format x s as a concatenation of a context and a question, and serialize y s as an answer sequence representing the label of x s (fig. 2 ). for instance, a sample in a sentiment classification task contains the input x s : "i enjoyed the movie. (context) what's the sentiment? (question)" and the output y s : "positive". for each sample x @xmath3, the input x u is only a context.