Unveiling the Lexical Sensitivity of LLM s: Combinatorial Optimization for Prompt Enhancement (2024)

Abstract: Large language models (LLMs) demonstrate exceptional instruct-following ability to complete various downstream tasks. Although this impressive ability makes LLMs flexible task solvers, their performance in solving tasks also heavily relies on instructions. In this paper, we reveal that LLMs are over-sensitive to lexical variations in task instructions, even when the variations are imperceptible to humans. By providing models with neighborhood instructions, which are closely situated in the latent representation space and differ by only one semantically similar word, the performance on downstream tasks can be vastly different. Following this property, we propose a black-box Combinatorial Optimization framework for Prompt Lexical Enhancement (COPLE). COPLE performs iterative lexical optimization according to the feedback from a batch of proxy tasks, using a search strategy related to word influence. Experiments show that even widely-used human-crafted prompts for current benchmarks suffer from the lexical sensitivity of models, and COPLE recovers the declined model ability in both instruct-following and solving downstream tasks.

## introduction language models have achieved remarkable performance in recent years, particularly those referred to as large language models (llms), which contain scaled-up parameters and size @xcite @xcite @xcite . these models demonstrate an exceptional ability to follow human instructions and complete downstream tasks after instruction tuning @xcite . in contrast to masked language models (mlms) like bert @xcite , llms do not require the addition and training of extra layers on top of the pre-trained base model to adapt to * corresponding author. new prompt please identify whether the sentences have the same meaning. ## related work sensitivity to imperceptible changes. the outstanding performance of language models seems to be built upon their excellent understanding of text @xcite @xcite . however, previous works reveal that even imperceptible input perturbations, which do not affect human comprehension, can lead to significant changes in the model's output @xcite @xcite . this property has been widely exploited to create adversarial examples, where small modifications to the embedding or input text can cause the model to generate incorrect answers @xcite @xcite @xcite . therefore, we believe even humans experienced in designing prompts may overlook the performance discrepancies caused by such imperceptible changes. prompt tuning and optimizing. similarly, recent efforts to optimize prompts for llms find that not only the content @xcite but also the format @xcite @xcite @xcite of the prompt, such as the order of examples and phrases, significantly influence the model performance. consequently, this sensitivity of language models to minor changes makes the optimal prompt found by the community increasingly complex. for example, @xcite transforms a prompt of length 6 into one exceeding 900 tokens. however, we argue that also due to this sensitivity, complex variations of the prompt should not be the first operation in prompt optimization, as the performance could be inadvertently constrained by the specific words employed in the prompt. this proposition distinguishes our work from previous studies on prompt optimization: given a prompt that has proven initially effective, we focus on the lexical influence of the prompt on model performance, attempting to recover the potential performance drop caused by lexical choices, rather than creating a prompt that yields optimal results from scratch @xcite @xcite @xcite . ## conclusion in this paper, we demonstrate the notable lexical sensitivity of llms to prompts, which potentially degrades their performance on downstream tasks. we show that even semantically similar prompts located in the neighborhood of the latent representation space may yield very different results. to recover the performance drop caused by the sensitivity, we propose cople, a black-box combinatorial optimization framework that iteratively improves lexical choices in prompts. experiments illustrate the effectiveness of cople in recovering both the model's ability of instruct-following and solving downstream tasks. we believe that carefully checking the word usage is essential before performing complex prompt engineering.