M eta ICL : Learning to Learn In Context (2022)

Abstract: We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks. This meta-training enables the model to more effectively learn a new task in context at test time, by simply conditioning on a few training examples with no parameter updates or task-specific templates. We experiment on a large, diverse collection of tasks consisting of 142 NLP datasets including classification, question answering, natural language inference, paraphrase detection and more, across seven different meta-training/target splits. MetaICL outperforms a range of baselines including in-context learning without meta-training and multi-task learning followed by zero-shot transfer. We find that the gains are particularly significant for target tasks that have domain shifts from the meta-training tasks, and that using a diverse set of the meta-training tasks is key to improvements. We also show that MetaICL approaches (and sometimes beats) the performance of models fully finetuned on the target task training data, and outperforms much bigger models with nearly 8x parameters.

## introduction large language models (lms) have recently been shown to be able to do in-context learning @xcite , where they learn a new task simply by conditioning on a few training examples and predicting which tokens best complete a test input. this type of learning is attractive because the model learns a new task through inference alone, without any parameter updates. however, performance significantly lags behind supervised finetuning, results are often high variance @xcite , and it can be difficult to engineer the templates that convert existing tasks to this format. in this paper, we address these challenges by introducing metaicl: meta-training for in-context learning. metaicl tunes a pretrained language model on a large set of tasks to learn how to incontext learn, and is evaluated on strictly new unseen tasks. each meta-training example matches the test setup-it includes k + 1 training examples from one task that will be presented together as a single sequence to the language model, and the output of the final example is used to calculate the cross-entropy training loss. simply finetuning the model in this data setup directly leads to better incontext learning-the model learns to recover the semantics of the task from the given examples, as must be done for in-context learning of a new task at test time. this approach is related to recent work that uses multi-task learning for better zero-shot performance at test time @xcite @xcite @xcite . however, metaicl is distinct as it allows learning new tasks from k examples alone, without relying on a task reformatting (e.g., reducing everything to question answering) or task-specific templates (e.g., converting different tasks to a language modeling problem). we experiment on a large, diverse collection of tasks taken from @xcite and @xcite , including 142 text classification, question answering, natural language inference and paraphrase detection datasets. we report seven different settings, all with no overlap between meta-training and target tasks. this leads to 52 unique target tasks in total, which is the largest among all recent related work to the best of our knowledge. experimental results show that metaicl consistently outperforms baselines including (1) a variety of lm in-context learning baselines without metatraining @xcite @xcite , and (2) multi-task learning followed by zero-shot transfer @xcite @xcite . gains over multi-task zero-shot transfer are particularly significant when meta-training tasks and target tasks are dissimilar, e.g. there are large differences in task formats, domains, or required skills. this demonstrates that metaicl enables the model to recover the semantics of the task in context during inference even when the target does not share similarities with meta-training tasks. metaicl often gets close to (and sometimes beats) the performance of models trained with supervised finetuning on the target datasets, and perform as well as models with 8x parameters. we also perform extensive ablations to identify key ingredients for success of metaicl such as the number and diversity of meta-training tasks. finally, we demonstrate metaicl without any templates is better than recent work using human-written natural instructions, while the best performance is achieved by combining both approaches. code and data are publicly released at github.com/ facebookresearch/metaicl . ## related work in-context learning @xcite propose to use a language model (lm) conditioned on a concatenation of training examples for few-shot learning with no parameter updates. it has been further improved by later work @xcite @xcite , showing promising results on a variety of tasks. however, in-context learning with an lm achieves poor performance when the target task is very different from language modeling in nature or the lm is not large enough. moreover, it can have high variance and poor worst-case accuracy @xcite . our paper is based on the core idea of in-context learning by conditioning on training examples. we show that, by explicitly training on an in-context learning objective, metaicl achieves substantial improvements even with smaller lms. meta-training via multi-task learning our work is broadly inspired by a large body of work in meta-learning @xcite and multi-task learning @xcite . prior work has shown that multi-task learning on a large collection of tasks leads to better performance on a new task, either when tested zero-shot @xcite @xcite or when further finetuned @xcite . in particular, the former is closely related to our work, as it eliminates the need for parameter updates on a target task. however, these zero-shot models are either limited to tasks sharing the same format as training tasks (e.g., a question answering format) @xcite , or rely heavily on taskspecific templates @xcite @xcite which are difficult to engineer due to high variance in performance from very small changes @xcite . in this paper, we propose a meta-training method for better in-context learning that improves fewshot performance. we show that it effectively learns semantics of a new task with no manual effort, significantly outperforming zero-shot transfer methods. foot_0 furthermore, while @xcite show that meta-training helps only when the model has 68b or more parameters, our experiments demonstrate improvements with a much smaller model (770m). chen et al. ( 2022 ), concurrently to our work, propose meta-training for in-context learning. our approach differs in a number of ways: we remove requirements of human-written templates or instructions, and include more diverse tasks, stronger baselines, and extensive experiments in much larger scale with many meta-training/target splits. ## metaicl we introduce metaicl: meta-training for in-context learning. meta-training inference task c meta-training tasks an unseen target task data given training examples @xmath1, y ## conclusion in this paper, we introduced metaicl, a new fewshot learning method where an lm is meta-trained to learn to in-context learn, i.e. condition on training examples to recover the task and make predictions. we experiment with a large, diverse collection of tasks, consisting of 142 unique tasks in total and 52 unique target tasks, using seven different settings. metaicl outperforms a range of strong baselines including in-context learning without meta-training and multi-task learning followed by zero-shot transfer, and outperforms or matches 8x bigger models. we identify ingredients for success of metaicl such as the number and diversity of meta-training tasks. we also demonstrate that, while metaicl is better than recent work using natural instructions, they are complementary and the best performance is achieved by integrating metaicl with instructions. limitation & future work our work is limited in multiple dimensions. first, in-context learning approaches in general requires much longer context at both meta-training and inference due to feeding the concatenation of the training data, thus being less efficient compared to baselines that do not use in-context learning. second, our work experiment with a casual language model with modest size (gpt-2 large, 770m parameters). future work may investigate extending our approach to a masked language model and a larger model. third, our experiments focus on classification and multichoice tasks where a set of candidate options is given. future work may study applying our ap-proach for a wider range of tasks including freeform generation. other avenues for future work include further improving metaicl to outperform supervised models with meta-training, identification of which meta-training tasks are helpful on target tasks, and how to better combine humanwritten instructions and metaicl.