Meta-training with Demonstration Retrieval for Efficient Few-shot Learning (2023)

Abstract: Large language models show impressive results on few-shot NLP tasks. However, these models are memory and computation-intensive. Meta-training allows one to leverage smaller models for few-shot generalization in a domain-general and task-agnostic manner; however, these methods alone results in models that may not have sufficient parameterization or knowledge to adapt quickly to a large variety of tasks. To overcome this issue, we propose meta-training with demonstration retrieval, where we use a dense passage retriever to retrieve semantically similar labeled demonstrations to each example for more varied supervision. By separating external knowledge from model parameters, we can use meta-training to train parameter-efficient models that generalize well on a larger variety of tasks. We construct a meta-training set from UnifiedQA and CrossFit, and propose a demonstration bank based on UnifiedQA tasks. To our knowledge, our work is the first to combine retrieval with meta-training, to use DPR models to retrieve demonstrations, and to leverage demonstrations from many tasks simultaneously, rather than randomly sampling demonstrations from the training set of the target task. Our approach outperforms a variety of targeted parameter-efficient and retrieval-augmented few-shot methods on QA, NLI, and text classification tasks (including SQuAD, QNLI, and TREC). Our approach can be meta-trained and fine-tuned quickly on a single GPU.

## introduction large language models (llms) have become increasingly popular due to their impressive fewshot performance on many nlp tasks and domains @xcite . this has resulted in many few-shot learning methods based on llms that require ever-larger gpus and figure 1 : our approach. given an input x from one of many possible qa tasks, we use a dense passage retriever to retrieve k semantically similar demonstrations @xmath0,...,k from a memory bank z composed of labeled examples. we meta-train bart, supervising it to generate the (question and) answer y given x and z across a diverse collection of qa tasks. increasing computation. methods requiring no parameter updates such as in-context learning @xcite and parameter-efficient methods like adapters @xcite partially mitigate these downsides, but ultimately, larger computation budgets are increasingly necessary to achieve stateof-the-art few-shot performance-even to simply load models and perform inference. meta-learning @xcite and meta-training @xcite are methods that make smaller language models capable of quicker and more robust few-shot performance across multiple tasks and domains. however, smaller models may not be able to store enough knowledge for effective generalization in many domains and tasks simultaneously. retrieval is one way to overcome this: by separating parametric knowledge in the language model from external knowledge (stored as retrievable text), one can leverage much more information than could be stored in the parameters of a language model. for example, retrieval-augmented generation (rag; @xcite and retrieval-enhanced transformers (retro; @xcite retrieve natural language passages to improve performance on knowledge-intensive nlp tasks, although they do not perform meta-learning or meta-training and only evaluate on high-resource knowledgeintensive tasks. we thus propose meta-training with demonstration retrieval as a more parameter-efficient way to leverage demonstrations for few-shot learning. we retrieve semantically similar labeled demonstrations for each training and test example during meta-training and fine-tuning. on a relatively small sequence-to-sequence model (bart large , 440m parameters), we show our proposed approach is capable of generalizing quickly and well on a variety of downstream tasks ( our experiments show that our method ( §3) outperforms tailored efficient few-shot baselines and other retrieval-augmented models on various tasks, including natural language inference (nli), paraphrase detection, and extractive question answering ( §5). to our knowledge, our work is the first to combine retrieval with meta-training (or multitask training more broadly), to use dpr models to retrieve demonstrations, and to leverage demonstrations from many tasks simultaneously, rather than retrieving random or k-nearest demonstrations from the training set of the target task. our code is available on github. foot_0 2 related work meta-learning @xcite is a class of methods that supervise a model on how to learn; the goal is to leverage a collection of meta-training tasks to learn a better learning algorithm that generalizes to held-out tasks. inspired by meta-learning, some recent stud-ies have attempted to induce specific abilities in language models in a task-and domain-agnostic manner via meta-training; this entails directly supervising a model on labeled examples from various tasks (sometimes using some controlled format or template @xcite ) to directly induce specific abilities or better inductive biases that improve generalization. metatraining is typically accomplished via some form of controlled multi-task learning, as in @xcite . many studies have explored multi-task and multi-domain learning @xcite @xcite @xcite , but these studies often leverage tasks that improve a model's abilities for some specific (set of) downstream tasks. in meta-training, we aim to directly improve the learning algorithm via controlled supervision, which should improve out-of-distribution generalization by teaching a model some helpful ability-such as in-context learning-that can result in gains on various downstream tasks @xcite . we focus on meta-training with examples from qa datasets. few-shot learning is a common setting in which a model is supervised on only a few labeled examples. many methods for improving fewshot performance are based on scaling model and data size @xcite . our goal is to improve few-shot performance across tasks in a computation-and memoryefficient manner, so we focus on smaller models that can be trained efficiently on a single gpu. some parameter-efficient few-shot methods have been proposed, including cloze-style prompting (schick and schütze, 2021b), fine-tuning with manually tuned (schick and schütze, 2021a) and automatically tuned prompts and demonstrations @xcite , and meta-learning @xcite @xcite . one advantage of our approach is that it does not require significant prompt tuning: rather, we standardize all of our tasks into a single format, similar to @xcite . this saves human time and computational resources. crucially, these approaches compare probabilities of single tokens or small pre-selected label sets; thus, they cannot be used for open-domain tasks like question answering. some work has proposed generative few-shot methods for opendomain tasks: this includes reformatting the input data to match a model's pre-training format @xcite , pre-training models to select relevant spans from context passages @xcite , and running a secondary pre-training step on labeled classification data @xcite . our model should be effective on many tasks, even when the label space is large and differs across examples; thus, our method is based on a generative sequence-to-sequence model. in-context learning (icl; @xcite is increasingly used in few-shot methods; here, labeled demonstrations are concatenated to the same context as a test example to teach a model how to perform a task without additional gradient updates. studies have analyzed what kinds of demonstrations are most effective @xcite , as well as what makes demonstrations effective @xcite . our demonstration retrieval approach is most similar to @xcite , who encode demonstrations and test examples into a sentence embedding space and retrieve the knearest demonstrations. our method differs in multiple ways: we use dense passage retrievers instead of sentence embeddings; we use demonstrations from many training sets instead of the training set of the target task; and we perform gradient updates with demonstrations, which is more feasible on our relatively small bart large -based model. @xcite find that very large lms (>68b parameters) are required for icl to be effective, but @xcite find that meta-training can be used to make a much smaller model (gpt2 large , 774m parameters) capable of leveraging demonstrations. here, we make bart large (440m parameters) better at leveraging demonstrations through meta-training with demonstrations, like @xcite ; however, their method is designed for zero-shot generalization, and it selects from a constrained set of pre-defined labels. our method is designed for few-shot settings and can be applied to open-domain tasks. retrieval-augmented generation models consist of two components: generators and retrievers. the generator is typically a decoder-only lm @xcite or sequence-to-sequence (seq2seq) model @xcite ; we use seq2seq models. the retriever is most often a dense passage retrieval (dpr; @xcite ) model based on bert base . rag models are typically evaluated on knowledge-intensive tasks like abstractive qa and fact verification. thus, the memory bank typically consists of wikipedia passages, which augments the model with additional factual knowledge separate from the generator's parameters. @xcite adapts this architecture for few-shot knowledge-intensive tasks using a very large generator (t5 x(x)l ) and a contriever-based @xcite retriever. however, we are interested in more general-purpose methods, as well as more parameter-and memory-efficient methods that train or fine-tune quickly on a single gpu. thus, we propose a task-agnostic and domain-general method to improve smaller generative models for few-shot settings: specifically, a retrieval-augmented metatraining step and a memory bank of labeled qa demonstrations instead of wikipedia passages. ## experimental setup we evaluate on a variety of qa and classification tasks ( our few-shot classification task splits are from @xcite . we evaluate on sentence pair classification tasks which are not contained in our meta-training or demonstration tasks; sentence pair classification tasks like natural language inference (nli) and paraphrase classification can be easily reformatted to our question/answer/context format. we also evaluate on trec, which is a singlesentence text classification task where the model must guess the category of the answer to a question (e.g., human, location, number), rather than the answer itself. for each task and few-shot split size, we average scores across 5 random few-shot samples. ## results our model's f 1 scores for extractive question answering (figure 2 ) are higher than models of similar parameterizations, including similar models that have been meta-trained using the same training data. our model also outperforms strong clas- with meta-training, rag-m obtains performance much closer to our model. this tells us that meta-training is responsible for much of the performance gains we observe, though the demonstration memory bank also improves performance to a lesser extent. on mrpc, rag-m outperforms our model, indicating that there exist some nonknowledge-intensive tasks where wikipedia passages are more helpful than qa demonstrations. ## conclusions we have proposed a meta-training method ( §3.2) that retrieves ( §3.1) semantically similar demonstrations from a diverse demonstration bank ( §3.3). our method achieves higher performance on average across many tasks than other strong parameterefficient few-shot baselines ( §5). in future work, one could explore a mixture of demonstration retrieval and passage retrieval for improved performance on a wider variety of tasks-including knowledge-intensive tasks.