Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity (2022)

Abstract: When primed with only a handful of training samples, very large, pretrained language models such as GPT-3 have shown competitive results when compared to fully-supervised, fine-tuned, large, pretrained language models. We demonstrate that the order in which the samples are provided can make the difference between near state-of-the-art and random guess performance: essentially some permutations are “fantastic” and some not. We analyse this phenomenon in detail, establishing that: it is present across model sizes (even for the largest current models), it is not related to a specific subset of samples, and that a given good permutation for one model is not transferable to another. While one could use a development set to determine which permutations are performant, this would deviate from the true few-shot setting as it requires additional annotated data. Instead, we use the generative nature of language models to construct an artificial development set and based on entropy statistics of the candidate permutations on this set, we identify performant prompts. Our method yields a 13% relative improvement for GPT-family models across eleven different established text classification tasks.

## introduction large pretrained language models (plms, @xcite @xcite @xcite have shown remarkable performance when conditioned with an appropriate textual context @xcite @xcite @xcite . for example, when conditioned on a long document and a "tl;dr:" token, they can generate a summary of said document, and when provided a partial question ("the theory of relativity was developed by __"), they can generate the correct answer. perhaps most strikingly, when primed with a context consisting of very few training examples, they produce text classification results that can match those of fully supervised models. this type of few shot setting, is commonly referred to as "in-context learning" @xcite . a core component of in-context learning is the text-based prompt that serves as the context. composing a prompt requires: (i) text linearisation using a template; and (ii) training sample concatenation (see perhaps counter-intuitively, we find that the right sample order can make as much of a difference as example training set (the greatest musicians, 1) (redundant concept, 0) linearization review: the greatest musicians. sentiment: positive review: redundant concept. sentiment: negative concatenation review: the greatest musicians. sentiment: positive. review: redundant concept. sentiment: negative or review: redundant concept. sentiment: negative. review: the greatest musicians. sentiment: positive the right template. as can be seen in figure 1 , some permutations have comparable performance (over 85% accuracy) to supervised training for sentiment classification, while others perform close to random (around 50%). this order sensitivity is universal across models, and although increasing the model size somewhat addresses it, the problem is still present for some text classification tasks (subj in figure 1 ) for models with billions of parameters. in our analysis, we find no common denominator between performant sample orders and that they are not transferable across different model sizes and tasks. in a fully-supervised setting, we could rely on a development set to select among sample orders. however, this is not desirable in a few-shot setting where the size of the development set is very limited, even unavailable @xcite . instead, we use the generative nature of language models to construct an unlabelled artificial development set and refer to it as a probing set. as the probing set is unlabelled, we use the predicted label distribution statistics and propose entropy-based metrics to measure the quality of candidate prompts.experimental results show that we can achieve on average 13% relative improvement across eleven different established text classification tasks across all different sizes (four orders of magnitude) of plms. to summarise, our contributions are as follows: ## order sensitivity and prompt design in this section, we study the relationship between permutation performance and various factors. for the ease of visualisation, we use a fixed random subset of four samples with a balanced label distribution from the sst-2 dataset and consider all 24 possible sample order permutations. this setup is illustrated in figure 2 . we also test five randomlyselected sets of examples and summarised variance statistics in the experiment section (section 5). although beneficial, increasing model size does not guarantee low variance we evaluate the order permutations for four different sizes of gpt-2 (0.1b-1.5b) foot_0 and gpt-3 (2.7b-175b). as we can observe in figure 1 , models can obtain remarkable few-shot performance. we see that the gpt2-xl (1.5b) model can even surpass 90% accuracy given just four samples. this result is comparable to those of supervised models trained on more than 60,000 samples. however, the performance variation of different permutations remain a big issue, especially for "smaller" models. 2 the same model can exhibit nearly perfect behaviour given one sample order, but then fall back to be on par with a random baseline for another. while increasing the model size (by a few order of magnitudes) can sometimes alleviate the issue, it still cannot resolve it entirely (especially if we consider tasks other than sst-2). in contrast, different initialisations of supervised fine-tuning approaches typically result in less than 1% standard deviation for their test set performance @xcite . ## methodology the previous section demonstrates that prompt order can have a substantial effect on performance, with some orderings of the same prompts for the same model providing random performance, and other "better" orderings providing performance competitive with supervised approaches. this suggests that there could be various ways of selecting prompt orders to achieve better performance, but the challenge is to do so automatically and without the need for additional labels (e.g., a development set). hence, in this section, we explore the question of: "how can we automatically generate a 'probing set' to find performant prompt orderings"? we approach this by: (i) for a randomly-selected set of training samples, we use every possible ordering permutation of this set as candidates; (ii) constructing a probing set by querying the language model using all candidate prompts as context; and (iii) use this probing set to identify the best ordering by ranking them using a probing metric. ## experimental setup we use four different sizes of gpt-2 @xcite (with 0.1b, 0.3b, 0.8b, and 1.5b parameteers) and two sizes of gpt-3 (brown et al., 2020) (with 2.7b, and 175b parameters). due to limited context window size (up to 1024 word-pieces for the gpt-2 series of models), we use a 4-shot setting for all datasets except agnews and dbpedia. our experiments are based on the open-source checkpoints of gpt-2 models and access to the openai gpt-3 api. 5 for probing set generation, we restrict the maximum generation length to 128. we also use sampling with a temperature, t, of 2, and we also make use of block n-gram repetitions @xcite to encourage diverse generation. we use 24 different permutations for each set of randomly selected training samples and use 5 different sets (except for gpt-3 with 175b parameters, where we only do two sets with 12 different permutation due to the high monetary cost) for each experiment, giving a total of 120 runs. we report the mean and standard deviation of the corresponding evaluation metric over 5 different sets. for performant prompt selection, we rank candidate prompts using the locale and globale probing metrics over the automatically generated probing set. we then select top k samples ranked by highest entropy values, where @xmath0, of the available 24 permutations as performant prompts. finally, we use these performant prompts to evaluate performance on various datasets and demonstrate both better performance and reduced variance. we also provide results for a majority baseline, which always predicts the majority label in the dataset, as a lower-bound of performance. we also provide an oracle to show the upper-bound of performance by selecting the top four performant orderings based on prompt performance on the validation set. ## results we report experimental results in entropy-based probing is effective for performant prompt selection regardless of model size we find that globale achieves, on average, a 13% relative improvement across the eleven different sentence classification tasks in comparison to prompts that do not make use of probing. locale provides results slightly inferior to globale, with an average 9.6% relative improvement over the baseline model. our selected performant prompts also demonstrate considerably lower variance than using all candidate prompts. ## conclusion we have shown that few-shot prompts suffer from order sensitivity, in that for the same prompt the order in which samples are provided can make the difference between state-of-the-art and random performance. in our analysis of the problem, we established that it is present across tasks, model sizes, prompt templates, samples, and number of training samples. to alleviate this problem, we introduced a novel probing method that exploits the generative nature of language models to construct an artificial development set. we were able to identity performant permutations using entropy-based statistics over this set, leading to an on average 13% improvement across eleven text classification tasks.