Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning (2023)

Abstract: Finding the best way of adapting pre-trained language models to a task is a big challenge in current NLP. Just like the previous generation of task-tuned models (TT), models that are adapted to tasks via in-context-learning (ICL) or instruction tuning (IT) are robust in some setups, but not in others. Here, we present a detailed analysis of which design choices cause instabilities and inconsistencies in LLM predictions. First, we show how spurious correlations between input distributions and labels – a known issue in TT models – form only a minor problem for prompted models. Then we engage in a systematic, holistic evaluation of different factors that have been found to influence predictions in a prompting setup. We test all possible combinations of a range of factors on both vanilla and instruction-tuned LLMs of different scale, and statistically analyse the results to show which factors are the most influential, the most interactive or the most stable. From our results, we deduce which factors can be used without precautions, should be avoided or handled with care in most settings.

## introduction transfer learning from large-scale pre-trained language models is nowadays the standard approach to a wide range of nlp tasks. one of its great challenges is to optimally interface information that pre-trained language models accumulate in their parameters and adapt it to the task of interest @xcite . the standard approach for task adaptation has recently shifted from updating model parameters for a specific task (from here on task tuning or tt) to using promptingbased methods based on in-context learning (from here on icl). icl can be subdivided into few-shot @xcite or zero-shot inference (primarily using instruction-tuned models @xcite . both approaches offer certain benefits over tt: it eliminates costly, task-specific finetuning and provides greater flexibility, as a single model can be applied to many tasks. however, icl also currently yields overall weaker performance compared to task-tuning and is less stable and reliable on many benchmarks (see, e.g. @xcite @xcite @xcite . while for tt, much research has been conducted to understand weaknesses in the paradigm (for an overview, see @xcite , the sources of instabilities in icl remain nebulous. since icl is more constrained (less data and no parameter updates), out-of-distribution generalisation has been suggested to be less of a problem @xcite . on the other hand, new frontiers emerge. for example, the format, order, or semantics of provided in-context examples can greatly influence learning outcomes, as does the proportion of labels in the context and the exact labels used @xcite . little is known, however, about how these factors interact (work from @xcite , suggests that they cannot be isolated); it is unclear which aspects are consistently beneficial, which vary across setups, and which are sensible to combine or decouple. the volatility of the paradigm warrants more research into the reliability of different design choices. in this paper, we conduct a detailed exploration of vanilla and instruction-tuned llms across various shifts and setups to understand their robustness. we start with one of the prominent themes in robustness studies for tt models: robustness to spurious correlations between input and label distributions @xcite @xcite and find that in icl, spurious correlations do not have a significant impact on learning outcomes. we go on to investigate icl's sensitivity to other features of adaptation context, as well as the consistency of predictions across different design choices. to do so, we conduct a large-scale grid search across various combinations of factors and statistically analyse the results to shed light on the inter-dependencies of different design choices. we find that the exact in-context setup (the number of in-context examples, the distribution of in-context labels, or the type of instructions given in the context) has a surprisingly small but reliable impact on prediction outcomes. on the other hand, the type of instructions used to query the target has, by far, the most significant impact on model behaviour. it is also the most volatile across settings, making it the most pivotal factor. ## background and related work in the following, we first briefly define tt and icl and then cover known problems with model robustness. ## experiment i: robustness to spurious correlations we clarify open questions about robustness of incontext learners by elucidating their sensitivity to factors to which they should be invariant (from here on invariance factors). first, we focus on one of the most prominent forms of non-robustness in tt models: susceptibility to spurious correlations between inputs and labels (see section 2.1). in the first set of experiments, we test how different models behave when spurious correlations are contained in their adaptation data. ## experiment ii: consistency evaluation in icl in the previous section, we saw that the robustness of in-context learners is likely influenced more by other factors than by spurious correlations in the in-context data. although previous studies have reported the susceptibilities of llms to various factors, the impact of different design decisions and their interactions in the context of icl robust1 an illustrative example of the base data being easier: adversarial qa contains only a single answer alternative while squad contains three. ## discussion we will first summarise the findings of this paper and then discuss their implications. findings we saw in section 3 how spurious correlations do not influence predictions in icl in a relevant manner as they did previously in tt. this, however, does not resolve the problem of robustness: depending on the setup, icl accuracy in our experiments differs up to 40%, as other factors in the setup become pivotally important. we here conducted a comprehensive analysis of the influence of different setups on the consistency of predictions in icl models. considering different setups, wellchosen instructions promise the largest performance gains across many setups. at the same time, they are among the most volatile factors of all and highly sensitive to the setting in which they are used. on the other hand, factors that relate to the exact organisation of the in-context examples, such as the label distribution or in-context instructions (cross-instructions), have surprisingly small impacts. other factors like n-shots -among others -are not interactive, which makes them much easier to handle: their expected gain or loss should, in most cases, correspond to our observed main effects. across all of our experiments, we also find the general tendency that larger numbers of model parameters and instruction tuning are beneficial for model consistency across many settings. implications and future research what do these findings imply? as we have seen, inconsistency is a severe concern in icl, and we here contribute to narrowing down its sources. unlike previously in tt, concentrating on spurious correlations is not vital for icl robustness and investigating design choices concerned with in-context examples (i.e. the exact few-shot setting) promises to be less impactful or mostly dependent on other setup factors. instead, our findings suggest that the exact phrasing of instruction templates is pivotally important. to get hold of inconsistent predictions in icl, finding the exact properties of instructions that so strongly influence model predictions is a sensible next step (potentially with a similar methodology as it is presented here). insights into the impact of instruction properties can help us to find the source of inconsistencies and avoid them in production, while they can also contribute to the theoretical understanding of in-context learning which is currently still under investigation. while our analysis focused on the few-shot setting, it also significantly impacts the increasingly popular zero-shot learning, as instructions are central in that setting. for model deployment, our findings demand caution as minor changes to certain parts of prompts (e.g. the instructions) can change the performance of the general setup. this is especially true for employing smaller, untuned models. a consistent finding across all our experiments is that instruction tuning improves consistency and robustness to irrelevant factors across all setups. therefore, we advocate for the use of tuned models to improve robustness. finally, recent research has suggested that dynamics in icl are, to a certain degree, chaotic @xcite . it might be advised to use more diverse evaluation setups and a rigorous statistical analysis of the results to guarantee the generality of results and avoid type-i errors in publications @xcite . ## conclusion we here analysed robustness and variability in the recent learning paradigm of icl, showing that they are generally different from in task-tuning. by using a methodology that covers a wide range of potential prompt design decisions, we show which factors actually matter in prompt design and how these factors influence each other.