POSIX : A Prompt Sensitivity Index For Large Language Models (2024)

Abstract: Despite their remarkable capabilities, Large Language Models (LLMs) are found to be surprisingly sensitive to minor variations in prompts, often generating significantly divergent outputs in response to minor variations in the prompts, such as spelling errors, alteration of wording or the prompt template. However, while assessing the quality of an LLM, the focus often tends to be solely on its performance on downstream tasks, while very little to no attention is paid to prompt sensitivity. To fill this gap, we propose POSIX – a novel PrOmpt Sensitivity IndeX as a reliable measure of prompt sensitivity, thereby offering a more comprehensive evaluation of LLM performance. The key idea behind POSIX is to capture the relative change in loglikelihood of a given response upon replacing the corresponding prompt with a different intent-preserving prompt. We provide thorough empirical evidence demonstrating the efficacy of POSIX in capturing prompt sensitivity and subsequently use it to measure and thereby compare prompt sensitivity of various open source LLMs. We find that merely increasing the parameter count or instruction tuning does not necessarily reduce prompt sensitivity whereas adding some few-shot exemplars, even just one, almost always leads to significant decrease in prompt sensitivity. We also find that alterations to prompt template lead to the highest sensitivity in the case of MCQ type tasks, whereas paraphrasing results in the highest sensitivity in open-ended generation tasks. The code for reproducing our results is open-sourced at https://github.com/kowndinya-renduchintala/POSIX.

## introduction large language models (llms) are pre-trained on enormous amounts of text data using the next-token-prediction objective and they can perform a variety of nlp tasks via "prompting" @xcite @xcite @xcite . however, llms have been found to be surprisingly sensitive even to the smallest of variations in prompts that do not significantly alter its meaning -such as wording, prompt template or even minor spelling errors -so much so that prompt engineering, which is a process of iteratively tuning prompts to elicit desired responses, has become a widespread practice @xcite . despite prompt sensitivity being a crucial aspect for assessing the usability of an llm, standard evaluation benchmarks such as mmlu @xcite or bbh @xcite focus predominantly on performance metrics like exact match, leaving prompt sensitivity sidelined. similarly, the model cards and blog posts announcing the most commonly used llms often do not contain prompt sensitivity analysis at all (ai@meta, 2024). however, from a user-centric perspective, models with low prompt sensitivity are generally preferred over highly prompt-sensitive ones, even if both perform similarly on standard benchmarks. this is largely because a real-world user may not always be able to formulate the "optimal" prompt everytime. moreover, a universally applicable optimal prompt that can work across different model architectures may not even exist. therefore, it becomes essential to develop a dedicated procedure to systematically evaluate and quantify the sensitivity of llms towards intent-preserving (or, intentaligned) variations in prompts. while the exploration of the topic of quantifying prompt sensitivity is limited, there exist a few works which study prompt sensitivity and attempt to quantify it. for instance, the helm benchmark @xcite implements various kinds of perturbations to the prompts, such as typos and misspellings, and reports the exact match score of responses obtained using the perturbed prompts. @xcite studied the sensitivity to variations in prompt templates. they introduced the concept of performance spread -defined as the difference between the highest and lowest average performances observed across a wide range of prompt templates -and used this as a surrogate for prompt sensitivity. likewise, @xcite first generated responses using variations of the same prompt and then used variation-ratio @xcite as the surrogate for sensitivity. despite being in the right direction, the existing efforts to quantify prompt sensitivity lack nuance. relying on variations in model accuracy as a proxy for sensitivity ignores the model's behavior in case of incorrect responses, failing to distinguish between a model that consistently generates the same incorrect response every time (low sensitivity) and one that generates a different incorrect response with each prompt variation (high sensitivity). additionally, response distribution matters: a model producing a consistent response in all but one case is less sensitive than one generating two different responses -each equally frequent. the exact match score is also brittle, heavily penalizing minor wording differences in correct responses. furthermore, models might show significant variance in the loglikelihoods of responses for different prompt variations even when the responses are identical. finally, the existing studies are also limited to deterministic responses, like multiple-choice questions or short answers @xcite and do not consider the sensitivity in the case of open-ended generative tasks. with this context, in this work, we focus on the following research question: given a prompt along with its intent-preserving variations and the corresponding set of responses generated by a language model, how do we measure the sensitivity of the llm on the given set of prompts such that the measure incorporates the following four key factors? as a first step towards addressing the research question, we identify the key property that should hold for an ideal llm that is not sensitive to variations in prompts, provided the underlying intent is unchanged. for this llm, the probability of generating a certain response should remain almost the same even if minor changes are introduced in the prompt. for instance, let x 1 and x 2 be two prompts such that one is an intent-preserving variant of the other and let y 1 and y 2 be the corresponding responses generated by the llm under consideration. then the assumption, which turns out to be the cornerstone of our index , is that for an llm that is not sensitive, p(y 1 |x 1 ) should not change much if x 1 is replaced by its intent-preserving variant, x 2 , i.e., p(y 1 |x 1 ) ≈ p(y 1 |x 2 ). similarly the other way round -p(y 2 |x 2 ) ≈ p(y 2 |x 1 ). we emphasize that this notion of sensitivity does not consider the ground-truth response for quantifying sensitivity; instead, it compares the likelihood of generating various responses with different intent-preserving variants of the same prompt. in that sense, it is completely orthogonal to metrics like exact match (which are purely performancebased) and adds a new dimension altogether for evaluating llms. the main contributions of our work can be summarized as follows: in this work, we also advocate for benchmarks with multiple variations of the same prompt. however, instead of relying on measures based on performance alone for measuring sensitivity or robustness, we argue the need for a comprehensive measure that can capture the prompt sensitivity of llms effectively. prompt engineering. due to such extensive variation in the performance of llms on slight modifications in input prompt, it is crucial to query llms with the optimal prompt to get the desired output. prompt engineering is the practice of crafting tailored prompts for input to the llms to guide them towards the intended responses. though for real-world use cases, users often perform prompt engineering manually, prior studies have also proposed ways to automate this process @xcite . another method for obtaining a better input prompt is meta-prompting @xcite @xcite , which aims to improve a prompt iteratively with the help of an llm itself through further prompting. furthermore, for tasks involving reasoning, chain-of-thought prompting @xcite has been observed to be very effective. for designing a few-shot prompt, the choice of in-context examples plays a crucial role in the performance of llms. while existing studies @xcite show that examples which are semantically similar to the input work the best, in some cases selecting diverse examples can be beneficial @xcite . to maximize the potential of llms, it is therefore crucial to be aware of the best practices for prompting. ## posix: prompt sensitivity index based on the notion of sensitivity introduced in section 1, we will first define which intent-preserving variations we consider in this work and describe posix in detail, including a brief description of the design choices involved. ## experimental setup to analyse the effectiveness of posix in capturing various facets of sensitivity as described in section 3 and to quantify and compare the prompt sensitivity of various llms using it, we experiment on massive multitask language understanding benchmark, or mmlu @xcite , for classification tasks (posed as mcq questions), and on alpaca @xcite for open-ended generation tasks. we also include big bench hard, or bbh @xcite families: llama, mistral, and olmo. these models include llama-2 7b (base and chat variants) @xcite , llama-3 8b (base and instruct variants), mistral 7b (base and instruct variants) @xcite and olmo 7b (base and instruct variants) @xcite . all our experiments were run on 8 nvidia a100-sxm4-80gb gpus. for each prompt, we generate 60 variants such that they are intent-aligned with each other and also with the original prompt. these 60 variants are composed of equal proportion of three types of variations -introduction of minor spelling errors, alteration of the prompt template and paraphrasing/re-wording of the prompt. to study the effect of variation type on sensitivity, each prompt variant is generated using one of these three possible alterations. the variation types are discussed in detail below: spelling errors: to introduce spelling errors, we randomly select one, two, four or eight tokens from the question in the prompt and introduce one of four possible spelling errors: (i) insertion, in which a random letter is added within the token; (ii) omission, in which a letter at a randomly chosen position is deleted; (iii) transposition, in which two adjacent letters are swapped; or (iv) substitution, in which a letter at a randomly chosen position is replaced with one of its adjacent letters on the keyboard. the specific error to be applied to each token is chosen randomly. the 20 variations for spelling errors are derived from the combinations of the number of tokens with errors (one, two, four, eight) and five different seeds to control randomness. these types of spelling errors are based on the study by @xcite . prompt templates: for altering the prompt template, we design 20 different templates based on the grammar defined by @xcite . these templates are designed to maintain the core meaning of the prompt while altering its structure. the prompt templates used in our study are listed in appendix b. paraphrases: to create paraphrased variations of the prompts, we use gpt-3.5-turbo to generate 20 paraphrases for each original prompt (the full dataset of generated paraphrases is open-sourced however a few examples of generated paraphrases are also present in appendix f for reference). these paraphrases are such that they rephrase the question while preserving its original intent and meaning. for each prompt, we compute posix for each type of variation, based on definition 3.3, using an intent-aligned prompt set that includes the original prompt and its 20 corresponding variants. additionally, for the mixture category, we calculate posix using a set of 21 prompt variants sampled uniformly from the three variation categories. for experiments on mmlu, we generate five tokens as output, as the tasks are of multiple-choice question-answering format; whereas for the openended questions in alpaca, 30 tokens are generated. posix is, however, comparable for arbitrary response lengths as it is normalized by the number of tokens in the response. ## conclusions we introduced posix -a novel prompt sensitivity index, as a reliable measure of sensitivity of llms towards intent-preserving variations in prompts such as spelling errors, prompt templates, and alterations in the wording. we presented thorough empirical analysis for the efficacy of posix in capturing prompt sensitivity and subsequently used it to measure and compare multiple open-source llms, revealing some interesting observations such as prompt template is the most sensitive variant type for mcq tasks and paraphrasing is the most sensitive variant type for open-ended generation tasks, and also that parameter count or instruction tuning do not necessarily decrease prompt sensitivity of the models. these findings highlight the nuanced behaviour of llms towards prompt variations, underscoring the importance of considering prompt sensitivity index for their holistic evaluation.