Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (2022)

Abstract: Large language models (LMs) are able to in-context learn—perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding of how the model learns and which aspects of the demonstrations contribute to end task performance. In this paper, we show that ground truth demonstrations are in fact not required—randomly replacing labels in the demonstrations barely hurts performance on a range of classification and multi-choce tasks, consistently over 12 different models including GPT-3. Instead, we find that other aspects of the demonstrations are the key drivers of endtask performance, including the fact that they provide a few examples of (1) the label space, (2) the distribution of the input text, and (3) the overall format of the sequence. Together, our analysis provides a new way of understanding how and why in-context learning works, while opening up new questions about how much can be learned from large language models through inference alone.

## introduction large language models (lms) have shown impressive performance on downstream tasks by simply conditioning on a few input-label pairs (demonstrations); this type of inference has been referred to as in-context learning @xcite . despite incontext learning consistently outperforming zeroshot inference on a wide range of tasks @xcite , there is little understanding of how it works and which aspects of the demonstrations contribute to end task performance. in this paper, we show that ground truth demonstrations are in fact not required for effective incontext learning (section 4). specifically, replacing the labels in demonstrations with random labels barely hurts performance in a range of classification and multi-choice tasks (figure 1 ). the result ## related work large language models have been key to strong performance in a wide range of downstream tasks @xcite @xcite @xcite . while finetuning has been a popular approach to transfer to new tasks @xcite , it is often impractical to finetune a very large model (e.g. ≥10b parameters). @xcite propose in-context learning as an alternative way to learn a new task. as depicted in figure 2 , the lm learns a new task via inference alone by conditioning on a concatenation of the training data as demonstrations, without any gradient updates. in-context learning has been the focus of significant study since its introduction. prior work proposes better ways of formulating the problem @xcite @xcite , better ways of choosing labeled examples for the demonstrations @xcite @xcite , meta-training with an explicit in-context learning objective @xcite , and learning to follow instructions as a variant of in-context learning @xcite @xcite . at the same time, some work reports brittleness and oversensitivity for in-context learning @xcite @xcite . relatively less work has been done to understand why in-context learning works. @xcite provide theoretical analysis that in-context learning can be formalized as bayesian inference that model # params public meta-trained ## experimental setup we describe the experimental setup used in our analysis (section 4 and 5). ## why does in-context learning work? section 4 shows that the ground truth input-label mapping in the demonstrations has little impact to performance gains from in-context learning. this section further examines what other aspects of the demonstrations lead to good performance of incontext learning. we identify four aspects of the demonstrations (x 1 , y 1 )...(x k , y k ) that potentially provide learning signal (depicted in figure 7 ). ## discussion & conclusion in this paper, we study the role of the demonstrations with respect to the success of in-context learning. we find that the ground truth input-label mapping in the demonstrations matters significantly less than one might think-replacing gold labels with random labels in the demonstrations only marginally lowers the performance. we then identify a series of aspects in the demonstrations and examine which aspect actually contributes to performance gains. results reveal that (1) gains are mainly coming from independent specification of the input space and the label space, (2) the models can still retain up to 95% of performance gains by using either the inputs only or the label set only if the right format is used, and (3) meta-training with an in-context learning objective magnifies these trends. together, our findings lead to a set of broader indications about in-context learning, as well as avenues for future work. does the model learn at test time? if we take a strict definition of learning: capturing the inputlabel correspondence given in the training data, then our findings suggest that lms do not learn new tasks at test time. our analysis shows that the model may ignore the task defined by the demonstrations and instead use prior from pretraining. however, learning a new task can be interpreted more broadly: it may include adapting to specific input and label distributions and the format suggested by the demonstrations, and ultimately getting to make a prediction more accurately. with this definition of learning, the model does learn the task from the demonstrations. our experiments indicate that the model does make use of aspects of the demonstrations and achieve performance gains.