Can language models learn from explanations in context? (2022)

Abstract: Language Models (LMs) can perform new tasks by adapting to a few in-context examples. For humans, explanations that connect examples to task principles can improve learning. We therefore investigate whether explanations of few-shot examples can help LMs. We annotate questions from 40 challenging tasks with answer explanations, and various matched control explanations. We evaluate how different types of explanations, instructions, and controls affect zero- and few-shot performance. We analyze these results using statistical multilevel modeling techniques that account for the nested dependencies among conditions, tasks, prompts, and models. We find that explanations can improve performance—even without tuning. Furthermore, explanations hand-tuned for performance on a small validation set offer substantially larger benefits, and building a prompt by selecting examples and explanations together substantially improves performance over selecting examples alone. Finally, even untuned explanations outperform carefully matched controls, suggesting that the benefits are due to the link between an example and its explanation, rather than lower-level features. However, only large models benefit. In summary, explanations can support the in-context learning of large LMs on challenging tasks.

## introduction a new paradigm has emerged in natural language processing: few-shot prompting of language @xcite . large lms appear to exhibit some incontext learning abilities, such that they can infer how to perform a new language task few-shot @xcite -from a few examples of input and output pairs within the model's context window, but without training. for instance, these models can answer trivia questions or perform arithmetic more accurately with a few examples of relevant questions and correct answers in context. although it is not always clear what is being learned or inferred from these prompts (e.g. @xcite , prompting is a growing subfield @xcite . this ability to adapt to a new task from a fewshot prompt bears some resemblance to the flexibility with which humans can adapt with instructions or examples. however, explanations also play a central role in human learning @xcite )explanations highlight task principles that allow us to generalize broadly @xcite . for example, an explanation can elaborate on a terse answer (e.g. "false") by connecting it to the broader reasoning process necessary to solve the problem (e.g. "these statements are contradictory; a number cannot be both prime and divisible by 6, which is composite."). thus, explanations can clarify the intended task by illustrating the principles that link questions to answers. we therefore investigate whether lms can also benefit from explanations when learning from examples in-context. can explanations of answers improve few-shot task performance? note that this question does not focus on explainability as a means to help users understand a model (e.g. @xcite ; instead, we focus on whether few-shot explanations can help the model itself to "understand" the task, as assessed through performance (cf. @xcite . this question is interesting for multiple reasons. practically, it has the potential to improve few-shot performance. but more fundamentally, the answer sheds light on the scientific question of what kind of in-context learning abilities lms exhibit, which is a topic of ongoing debate (e.g. @xcite . several converging findings suggest that explanations might improve few-shot performance. first, prompting with explicit instructions or task descriptions can be an effective way to adapt lms to a new task (e.g. @xcite . furthermore, breaking down the steps of a reasoning process for lms can improve few-shot performance @xcite @xcite . explanations of the examples in a few-shot prompt might likewise support inferring the correct task, and thereby improve downstream task performance. lms could benefit from explanations without necessarily "understanding" explanations in a human-like way (cf. @xcite . explanations are prevalent in human discourse, and therefore lms will have encountered explanations in training that are intended (by the humans who wrote them) to clarify understanding and improve future reasoning. for example, there are many exam answer keys online which contain questions, answers, and explanations of those answers that could help predict the answers to subsequent questions on the exam. these data patterns may be sufficient for learning to predict the answers to subsequent questions better after seeing explanations. we are not the first to explore explanations; given the increasing interest in prompt tuning, there has been other work on how in-context auxiliary information can affect model performance (e.g. @xcite , as well as a huge variety of broader work beyond prompting on topics like training or tuning with explanations (e.g. @xcite @xcite @xcite . compared to prior work we include a particular focus on the effect of post-answer explanations-whereas e.g. @xcite provide chains of reasoning before the answers. pre-and post-answer explanations have different effects on model reasoning and evaluation, and offer distinct scientific insights. furthermore, we evaluate on a distinctly challenging and diverse task set, and include careful control comparisons and analyses. see section 4 for further discussion. we highlight the following contributions: task instruction few-shot example #1 answer explanation 4 more examples + explanations target question answer these questions by identifying whether the second sentence is an appropriate paraphrase of the first, metaphorical sentence. q: david's eyes were like daggers at paul when paul invited his new girlfriend to dance. <--> david had two daggers when paul invited his new girlfriend to dance. choice: true choice: false a: false explanation: david's eyes were not literally daggers, it is a metaphor used to imply that david was glaring fiercely at paul. figure 1 : example prompt including a task instruction, few-shot examples with explanations, and the target question. task instructions outline the task to be performed, before the examples (or alone, in a zero-shot prompt). answer explanations can be added to each of the examples in a few shot prompt. performance is always evaluated on a target question. because explanations are added to examples on a separate line after answers, the same evaluation metric can be used for the target question regardless of whether explanations are provided. (example from metaphor_boolean.) ## results we first present the estimates from hierarchical logistic regressions predicting the performance of the largest model (280 billion parameters). in fig. 2 we show these estimates of the effect-size improvement in performance produced by adding different types of content to the prompt. each point estimates the effect of that prompt component while controlling for and aggregating across the effects of other components of the prompt. adding few-shot examples to the prompt substantially improves perfomance relative to a zero-shot prompt. untuned in fig. 3 we present distributions of benefits from different explanation types-improvement in log-score relative to matched conditions without explanation. untuned explanations have variable effects, but are beneficial on average. selecting examples and explanations together more consistently improves over selecting examples alone. handtuning explanations offers more substantial benefits, at least on the five tasks where we evaluated it. we also find that the effect of untuned explanations does not vary substantially in prompts with or without task instructions or non-instructions (app. d.2). individual task results can be found in app. d.7. only large models benefit from untuned explanations: fig. 4 shows raw summaries of the average effect of untuned explanations across model scales. untuned explanations offer a modest increase in average performance for the largest model. smaller models do not benefit. a hierarchical regression confirms that larger models benefit signifi- figure 4 : the effects of explanations on average accuracy, across model sizes. untuned explanations lead to modest increases in accuracy from the largest model relative to few-shot prompts without explanations. selecting the few-shot examples with explanations using a small validation set improves substantially over selecting only the few-shot examples without explanations, or with control explanations. note that these results aggregate across all 40 tasks, which have different difficulties and numbers of possible answers. see app. d.7 for pertask plots. (error bars are bootstrap 95%-cis. points are offset horizontally to avoid overlap.) cantly more from explanations (app. d.1.2). untuned explanations outperform matched control conditions: control explanations and noninstructions are neutral or harmful for the largest model. that is, there appears to be a unique benefit to real explanations-even untuned-relative to the control conditions. we depict this average effect of explanations and controls in appx. d.4. real explanations outperform even the same prompt with the explanations applied to other examples, showing that the benefits depend on the relationship between the example and explanation rather than lower-level features. a hierarchical regression confirms that untuned explanations significantly outperform all control conditions for the largest model (app. d.1.3). are explanations uniquely beneficial for particular task types? as noted above, the tasks we used are annotated with content keywords. we created 8 keyword clusters ranging from common sense to mathematics, and explored the effect of explanations within each (appx. d.5). the benefits of explanations appear fairly consistent across clusters. however, each cluster is relatively small, so the results should not be interpreted too strongly. ## related work recent observations that lms can perform tasks from a few examples @xcite have stimulated discussion about what underlies that learning-e.g., @xcite suggest that in-context learning is a form of implicit bayesian inference. other researchers have questioned whether what occurs is really "learning," or simply recovering previously-experienced tasks @xcite ). foot_1 for example, @xcite observe that prompts with randomized labels only mildly impair performance in common nlp tasks such as entailment. in the discussion, we describe the contribution of our results to understanding lm's in-context learning. task instructions: a variety of prior work has explored task instructions as zero-shot prompts @xcite , or as part of a few-shot prompt (e.g. @xcite . models can be sensitive to the framing of the tasks in these prompts, and can benefit from explicitly decomposing problems into multiple steps @xcite . le scao and rush (2021) estimated that task prompts can be worth many examples, at least when training a classifier. in-context explanations and reasoning decomposition: mishra et al. (2021b) explore prompts that contain task descriptions and examples with explanations (as well as other features such as guidelines). however, they do not estimate the independent effect of explanations, nor do they compare to control conditions. @xcite and @xcite use few-shot prompting to encourage lms to explain their answers, but to produce explanations for user interpretation, rather than for improved task performance. @xcite showed that "self-talk"-sampling answers to supporting questions from the model and adding them in context-improves question answering. other recent work has shown that models can benefit from examples that decompose the reasoning process leading to an answer @xcite , especially when they are augmented with external scratchpads to store intermediate computations @xcite . these decompositions can be seen as a particular kind of explanation. however, in contrast to closelyrelated works like @xcite , we provided explanations after the answer in the prompt, rather than providing reasoning chains before the answer. we nevertheless observed benefits from these postanswer explanations (even on some tasks like arithmetic where computing intermediate steps is use-ful). this is a critical distinction that is relevant to the distinct benefits explanations can provide to the model; see discussion. in addition, we evaluate on a broader set of challenging tasks than many prior works, include a broader set of matched controls, and provide more in depth statistical analyses. training language processing models with instructions or explanations: various prior works have explored training or tuning of language processing models with task instructions (e.g. @xcite @xcite , and with explanations @xcite @xcite @xcite . many works that focus on tuning with explanations used methods such as span or word highlighting rather than natural language explanations, see @xcite for a review. while this broader prior work is relevant to the general idea that explanations can be useful in natural language processing, it does not immediately bear on the question of how in-context explanations affect models that are not trained with a focus on task instructions or explanations. however, models that are explicitly trained with explanations or instructions would likely show more benefit from explanations in context (cf. @xcite . explanations beyond nlp: training with explanations (language or otherwise) has also shown benefits in domains beyond language. for example, explanations have been used for training or tuning models for computer vision @xcite @xcite , for mathematical problem solving by program induction @xcite , or relational reasoning and reinforcement learning @xcite . scaling and emergence: while lm loss can exhibit predictable scaling with model size (kaplan et al., 2020), these smooth changes can lead to qualitative differences in specific behaviours (ganguli et al., 2022)-"more is different" @xcite . for a few examples: brown et al. (2020) observe sharp transitions in performance on arithmetic tasks with increasing scale; and @xcite find that only large lms can generalize from instruction fine-tuning to perform novel tasks zero-shot. our finding that the benefits of explanations emerge with model scale is consistent with these prior findings. future larger (or otherwise improved) models may exhibit yet larger benefits. ## discussion can explanations improve few-shot learning? yes, but the benefits depend on model scale and explanation quality. for a large model, even untuned explanations resulted in a modest but significant improvement in performance-a benefit about one third as large as initially adding few-shot examples, but twice as large as adding task instructions. thus, even without tuning, explanations can improve the few-shot performance of large lms. furthermore, tuning the explanations for the task (using a small validation set) can substantially increase their benefits. first, selecting examples with explanations for a prompt offers larger performance improvements than selecting examples alone. second, hand-tuning explanations on a fixed set of examples can offer larger benefits, even on challenging tasks. while performance is still far from perfect, these findings-especially in the context of recent related work @xcite @xcite -suggest that explanations could improve few-shot prompting. why are post-answer explanations interesting? in contrast to some related works @xcite , we focused on providing explanations after answers, rather than chains of reasoning before the answer. while @xcite did not observe a benefit of post-answer explanations, our results were more positive, likely due to the fact that their tasks required more iterative reasoning. in addition to this difference, pre-and postanswer modes of explanation also have different implications at test-time, and different scientific implications about the model's in-context inferences. at test, post-answer explanations do not affect the evaluation pipeline, because the model will produce an answer before it produces an explanation. by contrast, pre-answer chains-of-reasoning require the evaluation function to parse the answer from the model output. for optimal performance, pre-answer reasoning and post-answer explanations might have complementary, combinable benefits. indeed, @xcite used postanswer-hint "rationalizations" as part of their bootstrapped training procedure. it is often useful to break problems into a series of steps, but some explanations are holistic and only make sense once the answer is known. from a scientific perspective, these two approaches represent distinct mechanisms by which a model could benefit from explanations. pre-answer chains of reasoning can allow the model to output a reasoning process and process the problem step-by-step before answering. by contrast, postanswer explanations can only shape the reasoning processes abstractly, by changing task inference. because the model has equal numbers of processing steps between the question and answer across all of our prompt conditions, the effect of explanations is due to changes in how the model is processing the question and possible answers. the benefits of explanations therefore result from fairly sophisticated higher-order inferences during in-context learning, as we discuss next. what do our results imply about lms' abilities for in-context learning? the process by which lms adapt to a prompt is debated. for example, @xcite make the interesting observation that gpt-3 can perform nearly as well on common tasks like entailment even with random answers in the prompt, and therefore question whether the models are really "learning" in context. comparisons between our explanations and matched controls allow us to test whether superficial features of the explanations drive their benefits. in particular, the other item explanation prompt contains the same sentences as the real explanation prompt; the explanations are simply paired with different examples. the largest model shows a significant advantage for real explanations even relative to this closely-matched control-in fact, the control appears no better than no explanations. this suggests that the largest model is using the relationship between the examples and their corresponding explanations when processing answers for a target question, rather than simply relying on the distribution of words or sentences in the prompt. thus, the largest models do appear to exhibit some fairly sophisticated higher-order inferences in context. while these observations do not rule out the possibility that the model is simply using the explanations to recall a task seen in training, the fact that the big-bench collaboration (2021) tasks we use are adversarially sampled to be unique and challenging makes this somewhat less likely. one possible reconciliation with past findings is that the models are "lazy" and rely on low-level features when they can, and only use higher-level relationships in challenging settings that require them (cf. @xcite . however, further investigation will be needed to fully resolve these issues. ## conclusions including explanations with examples in a few-shot prompt can improve in-context task inference for language models. explanations that are tuned using a validation set are especially effective, but even untuned explanations have modest positive effects and outperform carefully matched control conditions. however, in our experiments this capability only emerges in the largest models (although it is possible smaller models might benefit from more explanations, or on simpler tasks). these results have implications both for prompt engineering, and for scientific understanding of the in-context learning abilities of large language models.