Demystifying Prompts in Language Models via Perplexity Estimation (2023)

Abstract: Language models can be prompted to perform a wide variety of tasks with zero- and few-shot in-context learning. However, performance varies significantly with the choice of prompt, and we do not yet understand why this happens. In this paper, we analyze the factors that contribute to this variance and establish a new empirical hypothesis: the performance of a prompt is predicted by the extent to which the model is familiar with the language it contains. Over a wide range of tasks, we show that the lower the perplexity of the prompt, the better it is able to perform the task, when considering reasonable prompts that are related to it. As part of our analysis, we also devise a method to automatically extend a small seed set of manually written prompts by paraphrasing with GPT3 and backtranslation. This larger set allows us to verify that perplexity is a strong predictor of the success of a prompt and we show that the lowest perplexity prompts are consistently effective.

## introduction language models can be prompted to perform a wide range of zero-and few-shot learning tasks @xcite . however, there is significant variance in the performance of seemingly similar prompts @xcite : for ag news @xcite , we find an over 30 point accuracy gap between different manually curated prompts (see we are interested in understanding what makes some prompts better than others, and using this understanding to create better prompts for given tasks and models. we hypothesize that the lower the perplexity of a prompt is, the better its performance on the task will be, when considering reasonable prompts that are related to the task. this is based on the intuition that the more frequently the prompt (or very similar phrases) appears in the training data, the more the model is familiar with it and is able to perform the described task. we refrain from using the training data directly as it is often unavailable, expensive to search due to its size, and hard to use for approximate matching of similar prompts. instead, we focus on the perplexity of the prompt as a proxy for its occurrences in the data. to enable more complete analysis, we automatically expand the set of manually created prompts for the task by paraphrasing, resulting in a much larger and diverse set of prompts. we focus on prompts in english that reasonably describe the task for two reasons: (a) our main motivation is to understand what lies under the variance of performance in this type of prompt; (b) we aim to devise a useful method for creating prompts that are consistently effective, that could be easily adopted and interpreted by future, potentially non-expert users. we show empirically that our hypothesis holds across a diverse set of tasks (including classification and word prediction), models, and model sizes, providing us some insights about the underlying mechanism of prompting (see figure 1 ). as a result, we devise a method, spell (selecting prompts by estimating lm likelihood), for creating prompts in an informed manner. we show that using spell to choose prompts results in less variability in performance as well as in accuracy gains (1.8 accuracy points with opt and 2.3 accuracy points with bloom on average). importantly, our method does not require labels at all, only a small sample of inputs for the task. our contributions can be summarized as follows: (a) we formalize the notion that better familiarity of the model with the prompt correlates with better performance (section 2); (b) we automatically elaborate a given set of seed prompts using paraphrasing (section 3); (c) we establish experimentally the hypothesis that lower perplexity of the prompt correlates well with better performance (section 5); (d) we devise a method to create a more consistent set of prompts, that also improve results even with no labels for the task (section 7). despite the popularity of prompting as a method for using language models @xcite @xcite , the cause for the different behavior of various prompts remains unclear so far. ## automatic expansion of seed prompts we are interested in expanding our pool of prompts in order to: (a) have a more diverse set of prompts, making it more likely to find a better prompt for our task, and (b) support better analysis to validate our prompt quality hypothesis. in this section, we describe our method for automatically expanding a seed set of manually created prompts using paraphrasing. step 0: creating a seed set of manually-written prompts we first write/collect a small set of human written prompts that describe the task. for classification tasks we assume that the input appears before the prompt, with no choices appearing as part of the prompt (to help in smooth paraphrasing of the prompt itself). ## analysis next, we further explore the observed relationship between model perplexity and prompt performance. despite the consistently high correlation between these two factors, the structure of this relationship varies across tasks (section 6.1). additionally, we find that the automatically added prompts are highquality and not a significant source of noise (section 6.2), and that the best prompts selected by our approach vary across models (section 6.3). ## spell: selecting prompts by estimating lm likelihood the primary contribution of this work is the analysis of the relationship between prompt perplexity and downstream task performance (section 5). as one potential application of our findings, we also present a new method, spell, for generating and selecting consistently effective prompts. assuming a fixed computational budget for finding effective prompts for a given task, and that the search space might be quite large, we devise the following straightforward procedure: using this algorithm, we show empirically that it is best to prioritize experimenting with the lowest perplexity prompts, as they are more stable (exhibit less variation in performance) and perform better than manual prompts on average. this method also does not require any labels for the task, and is applicable to any task, also by non-experts, given example inputs only. ## related work relation between performance and training data previous work looking directly into the relation between the training data and the performance is limited. @xcite study numeric deduction tasks, and examine the correlations between the model performance on specific test instances and the frequency of terms from those instances in the pretraining data. they find that the models are more accurate on instances whose terms are more prevalent in the training data. additionally, @xcite propose a method to effectively identify a very small subset of pretraining data that directly supports the model in performing a specific task. @xcite use causal inference to measure the effect of pretraining data statistics on factual knowledge performance, and @xcite show correlational and causal relationships between accuracy and relevant document count (from training data) for qa datasets. prompt tuning and analysis there is a very rich line of work trying to find prompts automatically. @xcite present an automated method to create discrete prompts for a diverse set of tasks, based on a gradient-guided search, and they demonstrate their method on masked lms. other work also focuses on discrete prompts, aiming to improve zero-shot performance @xcite @xcite , or trains continuous prompts @xcite @xcite . on top of works that suggest a variety of methods for creating better prompts, some work also analyzes those prompts to try and get some insights about them: @xcite find that model performance is highly sensitive to small changes in wordings and @xcite point to a surprising disconnect between continuous and discrete prompts. ## conclusion we investigate the phenomenon where some prompts perform better than others despite appearing similar to the human users of lms. specifically, we hypothesize that the perplexity of a prompt under a given lm is closely tied to its task performance. we test this theory on a large number of tasks and autoregressive lms, and the resulting correlation study validates our hypothesis. further analysis of this relationship demonstrates that the best prompts differ across models, highlighting the importance of model-specific analysis, and that the underlying structure of the relationship between perplexity and performance varies across tasks. in light of these findings, we then propose a method, spell, to help users find wellperforming prompts for new tasks. empirical validation of the proposed procedure shows that spell generates effective prompts with low variability in performance, and produces small gains of 1.8 (2.3) accuracy points with opt (bloom) over manual prompts. we therefore conclude that spell provides a general and interpretable approach for applying lms to new tasks while requiring minimal human effort, and no labels.