Continual Few-Shot Learning for Text Classification (2021)

Abstract: Natural Language Processing (NLP) is increasingly relying on general end-to-end systems that need to handle many different linguistic phenomena and nuances. For example, a Natural Language Inference (NLI) system has to recognize sentiment, handle numbers, perform coreference, etc. Our solutions to complex problems are still far from perfect, so it is important to create systems that can learn to correct mistakes quickly, incrementally, and with little training data. In this work, we propose a continual few-shot learning (CFL) task, in which a system is challenged with a difficult phenomenon and asked to learn to correct mistakes with only a few (10 to 15) training examples. To this end, we first create benchmarks based on previously annotated data: two NLI (ANLI and SNLI) and one sentiment analysis (IMDB) datasets. Next, we present various baselines from diverse paradigms (e.g., memory-aware synapses and Prototypical networks) and compare them on few-shot learning and continual few-shot learning setups. Our contributions are in creating a benchmark suite and evaluation protocol for continual few-shot learning on the text classification tasks, and making several interesting observations on the behavior of similarity-based methods. We hope that our work serves as a useful starting point for future work on this important topic.

## introduction large end-to-end neural models are becoming more pervasive in computer vision (cv) and natural language processing (nlp). in nlp in particular, large language models such as bert @xcite fine-tuned end-to-end for a task, have advanced the state-of-the-art for many problems such as classification, natural language inference (nli), and question answering (qa) (devlin 1 https://github.com/ ramakanth-pasunuru/cfl-benchmark et al., 2019; @xcite . end-to-end models are conceptually simpler than the previously-popular pipelined models, making them easier to deploy and maintain. however, because large end-to-end models are black-boxes, it is difficult to correct the mistakes that they make. practical, real-world applications of nlp require such mistakes to be corrected on the fly as the system operates. for example, when a translation system makes a harmful mistake (e.g., translates "emnlp" to "icml"), a phrase-based system can be corrected by finding and modifying the responsible entries in the phrase table @xcite , whereas there is no equivalent way to correct that in an end-to-end neural mt system. similarly, systems have been shown to exhibit bias (e.g., gender or racial stereotypes) toward certain inputs of text, which we want to correct via few examples on the fly. further, the examples that provide supervision to correct mistakes or learn a phenomenon are often hard or impossible to acquire (e.g., due to privacy or ethics issues) @xcite . hence, it is important to effectively learn to correct mistakes using few extra training examples. recent work has shown the generalization capability of large pre-trained models to handle multiple tasks with zero to few training examples @xcite @xcite . for example, @xcite has shown that system trained for nli can be used to perform new tasks zero-shot, i.e., without any task-specific training data. we believe that similar models can be used to rapidly learn to correct a phenomenon within the same task from a few (e.g., 10 or 15) training examples. from a practical point of view, we need our trained systems to rapidly adapt to new phenomena (or correct its mistakes) using very few extra training examples, and do it continually as new phenomena (or errors) are discovered over time. tackling this important setting, we take a fresh look at continual learning in nlp and formulate a new setting that bears similarity to both continual and few-shot learning, but also differs from both in important ways. we dub the new setting "continual few-shot learning" (cfl) and formulate the following two requirements: to this end, we propose a benchmark suite and evaluation protocol for continual few-shot learning (cfl) on text classification tasks. our benchmark suite consists of both existing and newly created datasets. more precisely, we use the dataset with several linguistic categories annotated by @xcite from anli round-3 @xcite ; and also provide two new datasets with linguistic categories that we annotated using the counterfactual augmented data provided by @xcite on snli natural language inference dataset @xcite and imdb sentiment analysis dataset @xcite . we discuss several methods as important promising baselines for cfl, borrowing from the literature of few-shot learning and continual learning. we classify these baselines into parameter correction methods (e.g., mas @xcite ) and non-parametric feature matching methods (e.g., prototypical networks (pn) @xcite ). we compare these methods on our benchmark suite in a traditional few-shot setup and observe that non-parametric feature matching methods perform surprisingly better than other methods. next, we test the same methods in a continual few-shot setup and observe that a simple fine-tuning method performs better than other parameter correction methods like mas. the non-parametric feature matching based pn performs well on the examples that are being corrected (few-shot categories), but at the expense of the original performance. further, we also observe a large performance improvement on the few-shot categories in this setup. additionally, we provide interesting ablations to understand the usefulness and generalization capabilities of pn for few-shot linguistic categories. we compare models trained with cross-entropy loss versus prototypical loss via empirical studies and t-sne plots, and discuss their major differences in detail. we hope that our cfl benchmark suite and evaluation protocol will serve as a useful starting baseline point and encourage substantial progress and future work by the community on this important practical setting. ## related work cfl bears similarity to few-shot learning, continuous learning, and online learning. below, we discuss these three paradigms and highlight the similarity and differences from our approach. few-shot learning. the goal in few-shot learning is to learn a new task from only a few labeled examples. few-shot learning problems are studied in the image domain @xcite @xcite @xcite , focusing mainly on two kinds of approaches: metric-based approaches and optimization-based approaches. metric-based approaches learn generalizable metrics and corresponding matching functions from multiple training tasks with limited labels @xcite . for example, @xcite proposed to build representations for each class using supporting examples and then comparing the test instances by euclidean distances. optimization approaches aim to learn to optimize model parameters based on the gradients computed from limited labeled examples @xcite @xcite . in the language domain, @xcite proposed to use a weighted combination of multiple metrics obtained from meta-training tasks for inferring on a newly-seen few-shot task. on the dataset side, @xcite introduce a few-shot relation classification dataset. recently, large-scale pretrained language models have been used for fewshot learning of downstream tasks @xcite . @xcite used pre-trained entailment system for generalizing across more domains or new tasks when there are only a handful of labeled examples. all of the above-mentioned approaches focus on few-shot learning for new tasks. in contrast, we consider the same original task, but target examples that can be considered new because they require solving a linguistic phenomenon, an error category, or a new domain. unlike few-shot learning, we also require models that can maintain or improve performance on the existing data. ## datasets in this section, we describe all the english datasets that we curated and borrowed from previous works for creating a benchmark suite for continual fewshot learning (cfl). ## results in this section, we report the performances of various baselines discussed in sec. 4 on our benchmark suite. we refer to appendix for training details. ## ablations and analyses robustness of prototypical networks. to ablate on how prototypical networks (pn) performs on the original data (e.g., mnli or snli or imdb), we use the model trained with the cross-entropy loss and test it using pn with the original training data as the support set. surprisingly, we observe that pn performs equal to that of general softmaxbased prediction on all three datasets (see cross-entropy vs. prototypical loss. we train a model with prototypical network (pn) loss (minimize the distance between training examples and the approximated class representations) and compare it with cross-entropy (ce) loss. we also tried combining both the original dataset and the few-shot categories as the support set, 9 and observe a performance drop in the anli r3 few-shot categories, but still better than just using original dataset (mnli) as support set ( ## conclusion we presented a benchmark suite and evaluation protocol for continual few-shot learning (cfl) on the text classification tasks. we presented several methods as important baselines for our cfl setup. further, we provided several interesting ablations to understand the use of non-parametric feature matching methods for cfl. we hope that our work will serve as a useful starting point to encourage future work on this important practical setting.