The language of prompting: What linguistic properties make a prompt successful? (2023)

Abstract: The latest generation of LLMs can be prompted to achieve impressive zero-shot or few-shot performance in many NLP tasks. However, since performance is highly sensitive to the choice of prompts, considerable effort has been devoted to crowd-sourcing prompts or designing methods for prompt optimisation. Yet, we still lack a systematic understanding of how linguistic properties of prompts correlate with the task performance. In this work, we investigate how LLMs of different sizes, pre-trained and instruction-tuned, perform on prompts that are semantically equivalent, but vary in linguistic structure. We investigate both grammatical properties such as mood, tense, aspect and modality, as well as lexico-semantic variation through the use of synonyms. Our findings contradict the common assumption that LLMs achieve optimal performance on prompts which reflect language use in pretraining or instruction-tuning data. Prompts transfer poorly between datasets or models, and performance cannot generally be explained by perplexity, word frequency, word sense ambiguity or prompt length. Based on our results, we put forward a proposal for a more robust and comprehensive evaluation standard for prompting research.

## introduction nlp has witnessed a rapid succession of large pretrained language models (llms) being released accompanied by reports of impressive performance on a multitude of tasks @xcite @xcite . many works show that increasing model scale decreases pre-training loss and improves downstream task performance on average @xcite ) paired with an ongoing discussion on the superiority (or inferiority) of expertwritten prompts over generated ones @xcite . fundamentally, such variability in performance across prompting strategies raises the question of how llms process prompts based on language seen during training. do they conform to linguistic intuition and respond better to lower perplexity prompts featuring straight-forward sentence structures, frequent and less ambiguous words, or language that has been seen during instruction-tuning? to test this hypothesis, we examine prompting performance variability in llms through the lens of linguistics. to the best of our knowledge, we conduct the first controlled study of llm performance variability across semantically equivalent prompts that differ in linguistic structure. specifically, we manually construct parallel sets of prompts that vary systematically in grammatical mood, tense, aspect and modality (550 prompts in total 3 ). we study the influence of word frequency and ambiguity by exchanging content words for alternative synonyms. we evaluate five llms of different sizes, both instruction-tuned and not instruction-tuned: llama 30b @xcite , opt 1.3b, 30b @xcite and opt-iml 1.3b and 30b @xcite . we focus on understanding of instructions, and therefore, evaluate our models in a zero-shot fashion. we experiment with six datasets for three different tasks: sentiment classification, question answering and natural language inference (nli). for the instruction-tuned models, our choice of tasks cov-ers the fully supervised, cross-dataset and crosstask setting. overall, we observe large performance variation due to changes in linguistic structure of the prompt ( §5); and this holds even for instructiontuned models on seen tasks. furthermore, contrary to previous findings @xcite , model performance does not appear to correlate with perplexity of the prompts ( §6). further, we find no correlation between performance and prompt length, word sense ambiguity or word frequency. in many cases more complex sentence structures and rare synonyms outperform simpler formulations. our findings contradict the universal assumptions that llms perform best given low perplexity prompts featuring words which we assume (or know) to be frequent in pre-training and instruction-tuning data. this stresses the need for further research into the link between statistical distribution of language at different training stages and model behaviour. with regards to evaluation practices in the field, our work highlights the limitations of benchmarking multiple models on single, fixed prompts and reporting best-case performance. prompts generally transfer poorly between datasets even for the same model, let alone across models ( §5.2). instructiontuning ( §5.3) or increasing model size ( §5.4) do not preclude the possibility of considerable performance variation, even on seen tasks. these findings, coupled with the fact that many works do not release their prompts, make the results of existing evaluations less reliable and difficult to reproduce. in section 7, we put forward a proposal for a more robust and comprehensive evaluation framework for prompting research. ## related work instability in prompting papers accompanying newly released models, rarely report prompts used for their evaluation and, typically, do not evaluate on multiple prompts @xcite to date, few works have investigated robustness to different prompt formulations. @xcite show that machine-generated prompts are not robust to token deletion or reordering. @xcite evaluate pre-trained and instruction-tuned models on nli in the fewshot setting. they find that while instruction-tuning helps robustness against prompt variation, instruction-tuned models respond favourably even to misleading instructions. shaikh et al. @xcite show that gpt-3 scores drastically worse on bias and toxicity challenge sets with the addition of 'let's think step by step.' to a given prompt for chain-of-thought reasoning @xcite . @xcite find that performance on arithmetic tasks correlates with frequency of integers in the training data. perhaps closest to our work, gonen et al. @xcite find that lower perplexity of the prompt correlates with higher performance for opt @xcite and bloom (scao et al., 2022) on a variety of different tasks. in priming or in-context learning, lms profit from being shown the required input-output format, the distribution of inputs and the label space, while ground truth labels don't seem to be required @xcite . performance is also sensitive to the ordering of demonstration examples @xcite @xcite . chinchilla performs better on abstract reasoning tasks when test samples cater to prior knowledge acquired through pretraining @xcite . prompting evaluation practices. @xcite point out that evaluating models on the same prompt does not make for a direct comparison, since models' exposure to different pretraining data results in different responses to individual prompts. @xcite find that machinegenerated prompts do not achieve equal performance gains across datasets for the same task. @xcite posit that subpar performance of lms is due to different viable answers outside the answer choices competing for probability mass ("surface form competition"), reducing the score for the correct answer among answer choices. they mediate this using domain conditional pmi. @xcite observe that models overpredict label words that occur more frequently in a prompt, at its end, or are frequent in the pretraining data. they propose fitting an affine function to the lm scores, so that answer options are equally likely for 'content-free' dummy examples. gonen et al., 2022; sorensen et al., 2022; liao et al., 2022) we do not aim at proposing prompt selection methods or calibrate predictions. while many (semi-)automatic approaches to generating prompts have been proposed (liu et al., 2023; shin et al., 2020; jiang et al., 2020; gao et al., 2021, i.a.), we resort to crafting prompts manually so as to maintain fine-grained control over sentence structures in our prompts. logan iv et al. (2022) provide evidence that manually written prompts (schick and schütze, 2021) can yield better result than automatically sourced prompts. further, ishibashi et al. ( 2023 ) point out that automatically generated prompts contain atypical language use, punctuation or spelling mistakes and generalise poorly across datasets. to isolate the effect of individual instructions we restrict ourselves to the zero-shot setting and do not include any demonstration examples in-context. indeed, lms have been shown to perform reasonably well in the few-shot setting given unrelated or misleading instructions @xcite . similarly, we abstain from prompt-tuning on demonstration examples, so as to not introduce additional sources of variance @xcite . ## tasks and datasets to draw robust conclusions across tasks, we conduct experiments on multiple datasets, namely @xcite our guiding principle in choosing datasets was to have a diverse set of tasks on which pre-trained models such as llama or opt achieve abovechance performance in the zero-shot setting @xcite . for opt-iml, our choice covers the supervised, cross-dataset, and cross-task setting. the opt-iml models have been instruction-tuned on imdb and sst using prompts from prompt-source (sanh et al., 2022) and flan @xcite . they have been tuned on qa datasets such as sciq, but not on boolq or arc-e (iyer et al., 2022) which we use. all textual entailment tasks (including rte and cb) are fully-held out during training of opt-iml (iyer et al., 2022). ## analysis since accuracy varies considerably, we now analyse whether higher accuracy can be explained by lower prompt perplexity, prompt length, or the use of less ambiguous or more frequent words in the prompt. we present correlation results in prompt perplexity following gonen et al. ## conclusion we evaluated five llms, both pretrained and instruction-tuned, on parallel sets of prompts that systematically differ in grammatical mood, aspect, tense, modality and use of synonyms. we find that there is no favoured sentence structure that performs best across models and tasks. prompts generally transfer poorly across datasets and models. we find considerable performance variation, which can still persist even for larger instruction-tuned llms on seen tasks and is not explained by perplexity, word sense ambiguity or word frequency.