Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions (2024)

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in various NLP tasks. However, previous works have shown these models are sensitive towards prompt wording, and few-shot demonstrations and their order, posing challenges to fair assessment of these models. As these models become more powerful, it becomes imperative to understand and address these limitations. In this paper, we focus on LLMs robustness on the task of multiple-choice questions—commonly adopted task to study reasoning and fact-retrieving capability of LLMs. Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 85% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting. Through a detailed analysis, we conjecture that this sensitivity arises when LLMs are uncertain about the prediction between the top-2/3 choices, and specific options placements may favor certain prediction between those top choices depending on the question caused by positional bias. We also identify patterns in top-2 choices that amplify or mitigate the model’s bias toward option placement. We found that for amplifying bias, the optimal strategy involves positioning the top two choices as the first and last options. Conversely, to mitigate bias, we recommend placing these choices among the adjacent options. To validate our conjecture, we conduct various experiments and adopt two approaches to calibrate LLMs’ predictions, leading to up to 8 percentage points improvement across different models and benchmarks.

## introduction large language models (llms) have demonstrated impressive performance on various tasks, surpassing that of supervised models and, in some original order llm hen house where would i not want a fox? a) hen house b) english hunt c) mountains d) outside bedroom window e) england after reordering where would i not want a fox? a) mountains b) english hunt c) england d) outside bedroom window e) hen house correct response outside bedroom window wrong response figure 1 : gpt-4 sensitivity to reordering options: upon changing the order of choices, gpt-4 changes its prediction from "hen house" to "outside of bedroom window" (the example is from csqa dataset). for instance, llms have shown significant sensitivity to small changes in the prompt @xcite @xcite . therefore, a more comprehensive and conclusive analysis of different aspects that can affect/limit llms' performance is crucial for a fair assessment and their successful real-world adoption. one significant limitation lies in the robustness of llms concerning the arrangement of various components in a prompt, as it directly impacts the assessment of their capability in understanding and reasoning for specific tasks. prior research has demonstrated that llms exhibit sensitivity to the arrangement of few-shot demonstrations @xcite and the order of appearance for responses generated by candidate models when llms are used as referees to evaluate quality @xcite . given these findings, it becomes pertinent to inquire whether llms are also sensitive to the order of elements of the prompts in different tasks. for example, how much does the order of options in multiple-choice question (mcq) answering tasks impact the llms performance. in this paper, we investigate the sensitivity of llms to the order of options in mcqs; using it as a proxy to understand llms sensitivity to the order of prompt elements in in-context learning paradigm. we demonstrate an example of gpt-4's sensitivity to options order in figure 1 , using a sample from the csqa benchmark @xcite . notably, by merely rearranging the placement of options among choices a, c, and e, gpt-4 incorrectly predicts the answer to be "outside bedroom window". within this context, we aim to address the following research questions: (1) to what extent do llms exhibit sensitivity to the order of options in multiple-choice questions? (2) what factors contribute to llms' sensitivity to the order of options? (3) how can we improve llms' robustness to the order of options sensitivity? to answer the first question, we conducted experiments using @xcite , instruct-gpt (text-davinci-003) @xcite , and llama-2-13b (chat version) @xcite on five different multiple-choice question benchmarks. surprisingly, we discovered a substantial sensitivity gap of up to 85% in the zero-shot setting. additionally, in the few-shot setting, we observed that introducing demonstrations to the prompt only led to marginal improvements in llms' robustness if their performance increased. moving on to the second question, we put forth a conjecture that the sensitivity of llms stems from their positional bias, wherein they tend to favor certain placements when uncertain about the answer among the top choices. to validate our conjecture, we analyzed instances where the models' predictions changed upon reordering the options. furthermore, we showed that the complexity of the number of choices, while retaining the top possible answers, had only a gradual impact on the performance. additionally, we discerned patterns in the occurrence of top-2 possible choices that influence the model's probability of selecting a particular option or somewhat mitigate llms' positional bias. for amplifying bias, we found that the optimal strategy involves positioning the top two choices as the first and last options. conversely, to mitigate bias, we recommend placing these choices among the adjacent options. to validate our findings, we conducted qualitative evaluations. addressing the last question, we demonstrated that employing two different calibrating approaches led to a notable improvement in llms' performance, up to 8 percentage points. through these investigations, we contribute to a deeper understanding of how the order of options affects llms' decision-making in mcqs and offer practical solutions to increase their robustness and accuracy in such scenarios. ## background and experimental details this paper focuses on the task of multiple-choice question answering. in mcqs, the objective is to identify the correct answer to a given question from a set of possible options (see figure 1 ). to address this task using llms, we present a prompt in the following format: "choose the answer to the question only from a, b, c, d, and e choices. question: {question}. choices: {op-tions}. answer:" to the models. this in-context framing of multiple-choice questions is consistent with prior research @xcite . additionally, an illustrative example of our prompting approach and more experimental details are presented in appendix. models: we considered three widely-used large language models, llama-2-13b (chat version) @xcite , instructgpt (text-davinci-003) @xcite and @xcite . this selection aimed to represent a diverse range of llms, encompassing varying sizes and both open-source and commercial models. we primarily focus on these models due to their notable superior performance in the context of multiple-choice question answering tasks that require reasoning. data: to investigate the sensitivity of llms to the order of options and the reasons behind this phenomenon, we conducted experiments on five distinct mcq benchmarks. these benchmarks are as follows: csqa @xcite : a commonsense multiple-choice question answering dataset, where each question is accompanied by 5 options. abstract algebra, high school chemistry, and professional law from the mmlu benchmark @xcite : these benchmarks consist of multiple-choice questions with 4 options provided for each question. and, logical deduction from the big-bench dataset @xcite : this benchmark offers multiplechoice questions with 3 options for each question. our selection of these benchmarks was guided by three specific criteria: (1) domain diversity: we aimed to investigate the sensitivity to options order across different domains. (2) varying option numbers: in order to explore the impact of the num- tasks gpt-4 instructgpt llama-2-13b vanila min max vanila min max vanila min max csqa 84.3 -12.6 +10.3 72.3 -24.0 +19.1 62.2 -28.9 +25.5 logical deduction 92.3 -8.1 +5.0 64.0 -39.4 +34.7 53.0 -30.7 +34.7 abstract algebra ## sensitivity to order in this section, we first investigate the sensitivity of llms to the order of options in the zero-shot setting. then, we set out to determine whether introducing demonstrations to the prompt in the fewshot setting can enhance the models' robustness. to quantify sensitivity, we calculate the sensitivity gap, which is the difference between the maximum and minimum llms' performance when using an oracle ordering. in other words, we examine how specific reordering of options affects the models' predictions when the ground truth is known. ## calibrating llms for mcq tasks we conduct an in-depth investigation into how large language models react to changes in the order of options, and investigate the reasons behind their sensitivity to such changes. through our exploration, we have observed that llms are highly responsive to the sequence in which options are presented. this has led us to a critical juncture where we need to focus on methods to improve the models' resilience to variations in options order, ensuring more trustworthy evaluations. one potential solution we have considered is the calibration of llms predictions. the outcomes of calibrating llms predictions to mitigate order sensitivity by taking majority vote over models prediction in 10 random reorders in a simple bootstrapping approach (stickland and @xcite , are provided in we have also incorporated the approach of multiple evidence calibration (mec) introduced by @xcite . in their work, they propose to counteract llms' sensitivity by prompting the model to generate an explanation before providing its prediction. we adopt their provided prompt for solving mcq tasks. the impact of applying mec calibration on mcq tasks are outlined in the results from instructgpt performance reveal that the introduction of mec calibration results in a consistent decrease in model performance. this behavior contradicts the outcomes achieved through majority voting and underscores the unsuitability of mec calibration for multiple-choice question tasks. in the case of gpt-4, the integration of mec calibration also yields contrasting outcomes with respect to majority voting, particularly evident in benchmarks such as csqa, abstract algebra, and high school chemistry. for logical deduction and professional law benchmarks, while both majority voting and mec calibration result in improving the model performance, the amount of improvement differs considerably, thus casting doubt on the reliability of the mec approach in gpt-4 as well. ## related work large language models (llms) show remarkable accomplishments and capabilities on various nlp tasks, including answering multiple-choice questions. in order to ascertain the dependability of llms' proficiency, it becomes imperative to delve into the robustness of their performance when subjected to subtle changes in the input. ## conclusion we investigate the inherent sensitivity of language models to the arrangement of options in multiplechoice questions. upon measuring the intensity of llms sensitivity, our aim was twofold: to pinpoint the underlying source of this sensitivity and propose potential solutions to enhance the models' robustness. our evaluations unequivocally reveal that llms not only exhibit pronounced sensitivity to options order, but also that this sensitivity diminishes only slightly when demonstrations are integrated into the few-shot setting if performance increases. in seeking to uncover the root cause of order sensitivity, we conjecture that the issue arises from llms' positional bias, particularly manifest-ing in uncertain instances. we verify our conjecture by conducting diverse experiments that highlight impactful patterns that either magnify or mitigate this positional bias. finally, to improve the robustness of llms' sensitivity against options order, we consider two calibration techniques leading to up to 8 percentage points improvement across different models and benchmarks.