Continual Lifelong Learning in Natural Language Processing: A Survey (2020)

Abstract: Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. However, it is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. Furthermore, CL is particularly challenging for language learning, as natural language is ambiguous: it is discrete, compositional, and its meaning is context-dependent. In this work, we look at the problem of CL through the lens of various NLP tasks. Our survey discusses major challenges in CL and current methods applied in neural network models. We also provide a critical review of the existing CL evaluation methods and datasets in NLP. Finally, we present our outlook on future research directions.

## introduction human beings learn by building on their memories and applying past knowledge to understand new concepts. unlike humans, existing neural networks (nns) mostly learn in isolation and can be used effectively only for a limited time. models become less accurate over time, for instance, due to the changing distribution of data -the phenomenon known as concept drift @xcite . with the advent of deep learning, the problem of continual learning (cl) in natural language processing (nlp) is becoming even more pressing, as current approaches are not able to effectively retain previously learned knowledge and adapt to new information at the same time. throughout the years, numerous methods have been proposed to address the challenge known as catastrophic forgetting (cf) or catastrophic interference @xcite . naïve approaches to mitigate the problem, such as retraining the model from scratch to adapt to a new task (or a new data distribution), are costly and time-consuming. this is reinforced by the problems of capacity saturation and model expansion. concretely, a parametric model, while learning data samples with different distributions or progressing through a sequence of tasks, eventually reaches a point at which no more knowledge can be stored -i.e. its representational capacity approaches the limit @xcite . at this point, either model's capacity is expanded, or a selective forgetting -which likely incurs performance degradation -is applied. the latter choice may result either in a deterioration of prediction accuracy on new tasks (or data distributions) or forgetting the knowledge acquired before. this constraint is underpinned by a defining characteristic of cl, known as the stability-plasticity dilemma. specifically, the phenomenon considers the model's attempt to strike a balance between its stability (the ability to retain prior knowledge) and its plasticity (the ability to adapt to new knowledge). cl in the nlp domain, as opposed to computer vision or robotics, is still nascent @xcite . the differences are reflected in the small number of proposed methods aiming to alleviate the aforementioned issues and the evaluation benchmarks. to the best of our knowledge, apart from the work of @xcite , our paper is the only study summarizing the research progress related to continual, lifelong learning in nlp. ## learning paradigms in this section, we discuss principles of cl and related machine learning (ml) paradigms, as well as contemporary approaches to mitigate cf. ## evaluation even though cl is now experiencing a surge in the number of proposed new methods, there is no unified approach when it comes to their evaluation using benchmark datasets and metrics @xcite . and as we will show in this section, this is especially true in the nlp domain. there is a scarcity of datasets and benchmark evaluation schemes available specifically for cl in nlp. ## continual learning in nlp tasks natural language processing covers a diverse assortment of tasks. despite the variety of nlp tasks and methods, there are some common themes. on a syntax level, sentences in any domain or task follow the same syntax rules. furthermore, regardless of task or domain, there are words and phrases that have almost the same meaning. therefore, sharing of syntax and semantic knowledge should be feasible across nlp tasks. in this section, we explore how cl methods are used in most popular nlp tasks. ## research gaps and future directions although there is a growing number of task-specific approaches to cl in nlp, nevertheless, the body of research work remains rather scant @xcite . while the majority of current nlp methods is task-specific, we believe task-agnostic approaches will become much more prevalent. contemporary methods are limited along three dimensions: data, model architectures, and hardware. in the real world, we often deal with partial information data. moreover, data is drawn from noni.i.d. distributions, and is subject to agents' interventions or environmental changes. although attempts exist, where a model learns from a stream of examples without knowing from which dataset and distribution they originate from (e.g. d'autume et al., 2019), such approaches are rare. furthermore, learning on a very few examples (e.g. via few-shot transfer learning) @xcite is a major challenge for current models, even more so performing out-of-distribution generalization @xcite . in particular, widely used in nlp sequence-to-sequence models still struggle with systematic generalization @xcite , being unable to learn general rules and reason about highlevel language concepts. for instance, recent work on counterfactual language representations by @xcite is a promising step in that direction. the non-stationary learning problem can be alleviated by understanding and inferring causal relations from data (e.g. @xcite -which is an outstanding challenge @xcite -and coming up with combinations that are unlikely to be present in training distributions @xcite . namely, language is compositional; hence, the model can dynamically manipulate the semantic concepts which can be recombined in novel situations @xcite and later supported by language-based abductive reasoning (e.g. @xcite . on a model level, a combination of cl with bayesian principles should allow to identify better the importance of each parameter of an nn and aid parameter pruning and quantization (e.g. @xcite . we believe that not only the parameter informativeness should be uncertaintyguided, but also the periodic replay of previous memories should be informed by causality. furthermore, it is important to focus on reducing model capacity and computing requirements. even though the overparametrization of nns is pervasive @xcite , many current cl approaches promote the expansion of parameter space. we envision further research efforts focused on compression methods, such as knowledge distillation, low-rank factorization and model pruning. importantly, while cl allows for continuous adaptation, we believe that integrating cl with meta-learning has the potential to further unlock generalization capabilities in nlp. as meta-learning is able to efficiently learn with limited samples, hence such a cl model would adapt quicker in dynamic environments (e.g. @xcite . this would be especially beneficial for nlp systems operating in low-resource language and domain settings. finally, further research aiming at developing comprehensive benchmarks for cl in nlp would be an important addition to the existing studies. on the one hand, we observe a proliferation of multi-task benchmarks (e.g. @xcite @xcite . on the other hand, the cl paradigm and evaluation of cl systems call for more robust approaches than traditional performance metrics (e.g. accuracy, f1 measure) and multi-task evaluation schemes with clearly defined data and task boundaries. ## conclusion in this work, we provided a comprehensive overview of existing research on cl in nlp. we presented a classification of ml paradigms and methods for alleviating cf, as well as discussed how they are applied to various nlp tasks. also, we summarized available benchmark datasets and evaluation approaches. finally, we identified research gaps and outlined directions for future research endeavors. we hope this survey sparks interest in cl in nlp and inspires to view linguistic intelligence in a more holistic way.