Improving In-Context Few-Shot Learning via Self-Supervised Training (2022)

Abstract: Self-supervised pretraining has made few-shot learning possible for many NLP tasks. But the pretraining objectives are not typically adapted specifically for in-context few-shot learning. In this paper, we propose to use self-supervision in an intermediate training stage between pretraining and downstream few-shot usage with the goal to teach the model to perform in-context few shot learning. We propose and evaluate four self-supervised objectives on two benchmarks. We find that the intermediate self-supervision stage produces models that outperform strong baselines. Ablation study shows that several factors affect the downstream performance, such as the amount of training data and the diversity of the self-supervised objectives. Human-annotated cross-task supervision and self-supervision are complementary. Qualitative analysis suggests that the self-supervised-trained models are better at following task requirements.

## introduction in-context few-shot learning seeks to solve unseen tasks at inference time by conditioning on a few training examples. in particular, in this case we are interested in methods that forgo any weight updates @xcite . prior work has been focused on improving inference time algorithms (e.g., rescoring generated outputs @xcite , selecting @xcite and ordering @xcite the given few-shot examples) and incorporating extra resources (e.g., fine-tuning models on human-annotated datasets @xcite @xcite ). we hypothesise that a different way to improve in-context few-shot learning is through designing self-supervised objectives that more closely resemble the format of tasks that the model will be asked to perform. to do so, we cast the self-supervised training as an intermediate training stage between language model pretraining and downstream fewshot evaluation. in particular, we construct training datasets based on the self-supervised objectives following similar formats used in the downstream tasks, fine-tune pretrained language model checkpoints on the training datasets, and then evaluate the models on benchmarks. in experiments, we consider four self-supervised objectives, including masked word prediction and classification tasks related to next sentence prediction @xcite . we evaluate models on two benchmarks (13 tasks in total): super-glue @xcite and natural-instructions @xcite . superglue focuses on discriminative tasks, and natural-instructions is a set of generative tasks. empirically, we experiment with pretrained language models of two sizes: 125 million parameters and 1.3 billion parameters. we show that in our best setting, the 1.3 billion parameters model trained by the self-supervision performs better than the initial pretrained language models and two strong baselines on average. further analysis reveals that (1) the effectiveness of the self-supervision depends on the amount of training data, but the benefit of adding more data is diminishing; (2) the improvements brought by the self-supervision are in part due to the semantic similarity between the training and evaluation tasks; ## related work in-context few-shot learning. @xcite discover that large pretrained language models can solve unseen tasks at inference time. recent work has improved the in-context few-shot performance by rescoring generated outputs @xcite , selecting @xcite and ordering @xcite the given few-shot examples. other work studies pretrained language models' crosstask generalization abilities for in-context fewshot or zero-shot learning using human-annotated datasets @xcite @xcite @xcite via instructions @xcite @xcite and retrieved examples @xcite . our work differs in that we focus on selfsupervised training. fine-tuning for few-shot learning. pretrained language models for few-shot learning typically follows the "pretrain then fine-tune" paradigm @xcite @xcite , where recent work has focused on designing templates for few-shot fine-tuning @xcite @xcite @xcite , and optimizing soft prompts @xcite @xcite @xcite . other work focuses on unifying task formats to maximize the benefits of human annotations, including question answering @xcite , textual entailment @xcite @xcite , and many other tasks @xcite @xcite . in contrast, our focus is on in-context few-shot learning, without fine-tuning models on downstream task examples. pretraining for few-shot learning. several papers have adapted various resources for pretraining models to enhance their performances on fewshot learning, such as pretraining on hypertext @xcite , question-infused pretraining @xcite , and self-training @xcite @xcite . pretraining approaches have targeted specific tasks, such as task-oriented dialog @xcite , intent detection @xcite , and data-to-text generation @xcite . our work differs as we use plain text as opposed to (naturally-occurring) human-annotated resources. relatedly, @xcite used self-supervised meta-learning for fewshot text classification rather than in-context fewshot learning. intermediate fine-tuning. since our approach involves an extra training stage between pretraining and downstream evaluation, it is also related to prior work that uses multi-stage fine-tuning on human-annotated datasets for generic tasks @xcite @xcite @xcite ) and text classification @xcite . relevant work also studies intermediate fine-tuning using crosslingual supervision @xcite . @xcite use an intermediate self-supervised training stage for machine translation quality estimation. ## method we describe four self-supervised training objectives that will be used to train models before downstream evaluations. we begin by defining the example and the instance used during our self-supervised training. an example is an input-output pair. to differentiate the input and the output, we append special tokens "input:" and "output:" to the beginning of input text and output text respectively where the two texts are also separated by the newline token (see figure 1 for examples). 1 an instance is a linearized string formed by several examples from the same task (e.g., see figure 2 ). as we encode the text using causal attention, the examples closer to the beginning of input sequences can be seen as task demonstrations, resulting in efficient computation. when constructing the training examples, we pick three or more consecutive sentences (depending on the minimum sequence length we enforce on the sentences) and then apply task-specific rules to automatically create training data. to form a training instance, we randomly select examples from the same task until reaching the maximum sequence length (i.e., 2048). during training, we compute a cross-entropy loss on tokens in the output texts. natural language processing is a subfield of computer science concerned with the interactions between computers and human language. the goal is a computer capable of "understanding" the contents of documents. ## conclusion we evaluated four self-supervised objectives on two few-shot benchmarks by casting the selfsupervised training as an intermediate training stage between language model pretraining and downstream few-shot evaluation. empirically, we have shown that the models trained by the selfsupervised objectives show the best performances compared to strong baselines on average. analysis showed that (1) the amount of self-supervised training data and the diversity of the self-supervised tasks can affect the downstream performances.;