Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021)

Abstract: The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a lesser degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.

## introduction the ability to recall acquired knowledge for learning new tasks quickly and efficiently over time has been seen as a crucial metric of general linguistic intelligence @xcite . progress on this research problem has led to remarkable improvements in recent works on few-shot learning @xcite . however, these methods have primarily focused on learning from a static set of tasks (datasets) in an offline manner, without dynamically expanding the acquired task 1 task 2 task 3 … predicting on seen tasks adapting to new tasks training evaluation cola sst-2 mrpc e.g., glue tasks without adaptation with few-shot adaption knowledge over time. this training scheme is in contrast with the way humans process natural language @xcite : humans are able to process novel meanings by retaining past knowledge, combining/decomposing chunks of language into prior learned language components, and avoid learning from scratch. motivated by this observation, we study whether nlp models could accumulate generalizable knowledge continuously over a sequence of tasks and learn to generalize to new tasks rapidly (i.e., with few examples). this problem has not been investigated in the existing works -a related line of efforts that look to learn from sequentially arriving tasks, known as continual learning (cl) or lifelong learning @xcite @xcite , mainly focus on retaining the performance on seen tasks when the model is continuously updated on new tasks (i.e., to overcome the catastrophic forgetting issue). to study this ability, we propose the continual learning of few-shot learners (clif) setup (illustrated in figure 1 ) to simulate the challenge: in clif, the model learns over a sequence of nlp tasks (arriving one by one; without revisiting), and then evaluated in terms of (i) generalization to new (few-shot learning) tasks; and (ii) preserving its performance on solving seen tasks. we train and evaluate over a diverse set of nlp tasks, spanning over entity typing, sentiment analysis, natural language inference, and other classification tasks. with the clif setup, we conduct a series of experiments on existing models, in order to understand the relationship between continuous knowledge accumulation and few-shot generalization. our first analysis is to understand how the generalization ability evolves during continual training, and whether catastrophic forgetting affects the acquisition of generalization ability. we find a negative effect of catastrophic forgetting on the generalization ability, and a stronger negative effect on the performance over the seen tasks. in a follow-up analysis, we find most existing cl methods hardly benefit models' generalization ability, even they are shown to alleviate catastrophic forgetting. this implies some nontrivial challenges for accumulating knowledge that can help model generalization. inspired by recent research on hypernetworks for few-shot learning @xcite and continual learning approach using hypernetworks (von oswald et al., 2020), we propose bi-level hypernetworks for adapters with regularization to address challenges of the clif. we evaluate these approaches extensively by varying the number of training examples and the orders of tasks at training. to summarize, the main contribution of this work is threefold (1) we propose clif setup, its data streams and protocols to comprehensively evaluate lifelong knowledge accumulation in nlp, and (2) we compare existing algorithms to demonstrate weaknesses of these algorithms (3) and propose bi-level hypernetworks for adapters with regularization as a solution to inspire future works. ## method this section presents baseline methods to set up the lower bounds for the clif problem, and approaches to improve the performance. we view an approach by its base model and the learning algorithm. we first introduce the base models in our study (sec. 3.1); then, we introduce a few existing methods for continual learning and continual metalearning (sec. 3.2). finally, we present a novel regularized bi-level adapter generation framework to better address the clif problem (sec. 3.3). ## results and analysis we address our two major research questions in this section: (1) how models accumulate generalizable knowledge over time in a cl setup compared to offline setups given potential catastrophic forgetting, and (2) whether continual learning approaches reduce catastrophic forgetting of both seen-task performance and generalizable knowledge. we experiment with various combinations of model architectures in 3.1 and learning algorithms 3.2. we note a method by its model architecture and cl algorithm applied, e.g., bart-vanilla, bihnet-ewc. we include details of implementation in appendix a. ## conclusion we present the continual learning of few-shot learners (clif) challenge to simulate the scenario where a learner continually accumulate (generalizable) knowledge over a sequence of nlp tasks, while retaining its performance on the seen tasks. we propose evaluation protocols to study the performance of existing continual learning algorithm, and present our method bihnet-reg. we demonstrate the potentials of building a nlp system that, through continual training, can perform more tasks and also become more efficient in mastering new tasks. future works include extending our work to task agnostic scenarios where the distribution of data may shift continuously and studying algorithms for continual refinement of large-scale pre-trained models with emerging unlabeled data. task order tasks clif-26 default cola, sst2, mrpc, qqp, stsb, mnli, qnli, wnli, rte relevance ↓ mnli, sst2, qqp, qnli, stsb, mrpc, cola, rte, wnli relevance ↑ wnli, rte, cola, mrpc, stsb, qnli, qqp, sst2, mnli clif-55 default ai2_arc, aqua_rat, boolq, codah, commonsense_qa, cosmos_qa, dream, eli5-askh, eli5-asks, eli5-eli5, freebase_qa, hellaswag, jeopardy, kilt_hotpotqa, kilt_nq, kilt_trex, kilt_zsre, lama-conceptnet, lama-google_re, lama-squad, lama-trex, math_qa, mc_taco, numer_sense, openbookqa, qasc, quail, quarel, quartz-no_knowledge, quartz-with_knowledge, race-high, race-middle, sciq, search_qa, social_i_qa, squad-no_context, superglue-copa, superglue-multirc, swag, web_questions, wino_grande, wiqa