Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021)

Abstract: Continual learning has become increasingly important as it enables NLP models to constantly learn and gain knowledge over time. Previous continual learning methods are mainly designed to preserve knowledge from previous tasks, without much emphasis on how to well generalize models to new tasks. In this work, we propose an information disentanglement based regularization method for continual learning on text classification. Our proposed method first disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task, and further regularizes these representations differently to better constrain the knowledge required to generalize. We also introduce two simple auxiliary tasks: next sentence prediction and task-id prediction, for learning better generic and specific representation spaces. Experiments conducted on large-scale benchmarks demonstrate the effectiveness of our method in continual text classification tasks with various sequences and lengths over state-of-the-art baselines. We have publicly released our code at https://github.com/GT-SALT/IDBR .

## introduction computational systems in real world scenarios face changing environment frequently, and thus are often required to learn continually from dynamic streams of data building on what was learnt before @xcite . for example, a tweeter classifier needs to deal with trending topics which are constantly emerging. while being an intrinsic nature of human to continually acquire and transfer knowledge throughout lifespans, most machine learning models often suffer from catastrophic forgetting: when learning on new tasks, models dramatically and rapidly forget knowledge from previous tasks @xcite . as a result, continual learning (cl) @xcite has received more attention recently as it can enable models to perform positive transfer @xcite as well as remember previously seen tasks. a growing body of research has been conducted to equip neural networks with the ability of continual learning abilities @xcite @xcite . existing continual learning methods on nlp tasks can be broadly categorized into two classes: purely replay based methods @xcite where examples from previous tasks are stored and re-trained during the learning of the new task to retain old information, and regularization based methods @xcite where constraints are added on model parameters to prevent them from changing too much while learning new tasks. the former usually stores an extensive amount of data from old tasks (de masson d' @xcite or trains language models based on task identifiers to generate sufficient examples @xcite , which significantly increases memory costs and training time. while the latter utilizes previous examples efficiently via the constraints added on text hidden space or model parameters, it generally views them as equally important and regularize them to the same extent @xcite , making it hard for models to differentiate informative representation that needs to be retained from ones that need a large degree of updates. however, we argue that when learning new tasks, task generic information and task specific information should be treated differently, as these generic representation might function consistently while task specific representations might need to be changed significantly. to this end, we propose an information disentanglement based regularization method for continual learning on text classification. specifically, we first disentangle the text hidden representation space (e.g., the output representation of bert @xcite ) into a task generic space and a task specific space using two auxiliary tasks: next sentence prediction for learning task generic information and task identifier prediction for learning task specific representations. when training on new tasks, we constrain the task generic representation to be relatively stable and representations of task specific aspects to be more flexible. to further alleviate catastrophic forgetting without much increases of memory and training time, we propose to augment our regularization-based methods by storing and replaying only a small amount of representative examples (e.g., 1% samples selected by memory selection rules like k-means @xcite ). to sum up, our contributions are threefold: continual learning existing continual learning research can be broadly divided into four categories: (i) replay-based method, which remind models of information from seen tasks via experience replay (de masson d'autume et al., 2019), distillation (rebuffi et al., 2017), representation alignment @xcite or optimization constraints @xcite using examples sampled from previous tasks @xcite or synthesized with generative models @xcite ; textual information disentanglement our work is related to information disentanglement for text data, which has been extensively explored in generation tasks like style transfer @xcite @xcite , where text hidden representations are often disentangled into sentiment @xcite , content @xcite and syntax @xcite information through supervised learning from pre-defined labels @xcite or unsupervised learning with adversarial training @xcite . building on these prior works, we differentiate task generic space from task specific space via supervision from two simple yet effective auxiliary tasks: next sentence prediction and task identifier prediction. related learning paradigms there exists some other learning paradigms also dealing with multiple tasks, such as multi-task learning @xcite and transfer learning @xcite . however, neither can fit in the scenario of learning multiple tasks sequentially. the former could be adapted to dynamic environments by storing all seen training data and retraining the model after the arrival of new tasks, which highly decreases efficiency and is impractical in deployment. the latter only focuses on the target tasks and ignores catastrophic forgetting on the source tasks. a more thorough discussion can be found in @xcite . ## problem formulation in this work, we focus on continual learning for a sequence of text classification tasks {t 1 , ...t n }, where we learn a model f θ (.), θ is a set of parameters shared by all tasks and each task t i contains a different set of sentence-label training pairs, (x i 1:m , y i 1:m ). after learning all tasks in the sequence, we seek to minimize the generalization error on all tasks @xcite we use two commonly-used techniques for this problem setting in our proposed model: ## method in continual learning, the model needs to adapt to new tasks quickly while maintaining the ability to recover information from previous tasks, hence not all information stored in the hidden representation space should be treated equally. in previous work like style transfer @xcite and controlled text generation @xcite , certain information (such as content and syntax) is extracted and shared among different categories and other information (such as style and polarity) is manipulated for each specific category. similarly, in our continual learning scenario, there is shared knowledge among different tasks as well while the model needs to learn and maintain specific knowledge for each individual task in the learning process. this key observation motivates us to propose an information-disentanglement based regularization for continual text classification to retain shared knowledge while adapting specific knowledge to streams of tasks (section 4.1). we also incorporate a small set of representative replay samples to alleviate catastrophic forgetting (section 4.3). our model architecture is shown in figure 1 . ## results and discussion we evaluate models after training on all tasks and report their average accuracies on all test sets as our metric. ## conclusion in this work, we introduce an information disentanglement based regularization (idbr) method for continual text classification, where we disentangle the hidden space into task generic space and task specific space and further regularize them differently. we also leverage k-means as the memory selection rule to help the model benefit from the augmented episodic memory module. experiments conducted on five benchmark datasets demonstrate that idbr achieves better performances compared to previous state-of-the-art baselines on sequences of text classification tasks with various orders and lengths. we believe the proposed approach can be extended to continual learning for other nlp tasks such as sequence generation and sequence labeling as well, and plan to explore them in the future.