Is Your Large Language Model Knowledgeable or a Choices-Only Cheater? (2024)

Abstract: Recent work shows that large language models (LLMs) can answer multiple-choice questions using only the choices, but does this mean that MCQA leaderboard rankings of LLMs are largely influenced by abilities in choices-only settings? To answer this, we use a contrast set that probes if LLMs over-rely on choices-only shortcuts in MCQA. While previous works build contrast sets via expensive human annotations or model-generated data which can be biased, we employ graph mining to extract contrast sets from existing MCQA datasets. We use our method on UnifiedQA, a group of six commonsense reasoning datasets with high choices-only accuracy, to build an 820-question contrast set. After validating our contrast set, we test 12 LLMs, finding that these models do not exhibit reliance on choice-only shortcuts when given both the question and choices. Thus, despite the susceptibility of MCQA to high choices-only accuracy, we argue that LLMs are not obtaining high ranks on MCQA leaderboards solely due to their ability to exploit choices-only shortcuts.

## introduction multiple-choice question answering (mcqa) is a popular task to test the knowledge of large language models (llms) @xcite . however, recent work shows that llms surpass majority class baselines in choices-only settingswhen no question and just the choices are given in a prompt @xcite . this raises the question: do models obtain high ranks in mcqa leaderboards due to their pretraining knowledge or their ability to exploit choices-only shortcuts? resolving this query is key to ensure that mcqa leaderboards reliably rank the knowledge of llms. to answer this question, we use a variation of contrast sets-small datasets that test if models "pay attention" to perturbed attributes that should 1 our code is available at https://github.com/ nbalepur/mcqa-artifacts question: some aerosols can decrease temperatures by blocking what? choices: (a) rainfall (b) visibility (c) the sun (d) pressure answer: (c) question: which of the following increases moisture? choices: (a) density (b) the sun (c) wind (d) rain answer: (d) entry d_i entry d_j original mcqa evaluation set question: some aerosols can decrease temperatures by blocking what? choices: (a) the sun (b) rain answer: (a) question: which of the following increases moisture? choices: (a) the sun (b) rain answer: (b) entry d_i entry d_j contrast mcqa evaluation set ... ... alter the model's decision @xcite . for our purposes, we need a contrast set containing pairs of mc entries with identical answer choices, but varied questions that lead to distinct answers. for example, in figure 1 (bottom), the mc entries d i and d j have the same choices of "the sun" and "the rain", but d i has a question that answered by "the sun", and similarly for d j . this design ensures that llms relying only on shortcuts or patterns in the choices, while ignoring questions, can perform no better than random chance. if a model ranks highly on an mcqa dataset but largely drops in rank on a contrast set based on this dataset, it would reveal that this model obtains a high rank on the original dataset primarily by employing choices-only shortcuts. contrast sets are usually built through manual annotation efforts @xcite , as model-generated data can be biased. however, writing mc questions with high-quality distractors is difficult even for experts @xcite . further, rewritten questions can exhibit distributional differences from the original questions, altering the difficulty of the mcqa task. to address this issue, we cast the creation of contrast sets for mcqa datasets to a graph mining task. we treat each mc entry d i in the dataset as a vertex in an undirected graph, and draw edges between entries d i and d j if the gold answer from d i is semantically equivalent to a distractor in d j , and vice versa. for instance, in figure 1 @xcite , the gold answer of "rain" in d i is semantically similar to the distractor of "rainfall" in d j and vice versa, so we draw an edge between d i and d j . thus, an edge (d i , d j ) means that the gold answers in d i and d j can form a set of choices, with questions in d i and d j leading to distinct answers in said choices, mirroring the desired format of our contrast set. we find the maximum matching of this graph to obtain the largest contrast set of distinct mcqa questions derived from the initial dataset. this method overcomes the burden of writing contrast sets, while only minimally using models for semantic equivalence, reducing the risk of model-generated biases. we use our approach to build an 820-question contrast set from six commonsense mcqa datasets from the unifiedqa collection @xcite . we first ask three annotators to assess our contrast set, finding that it has questions with plausible distractors ( §4.1). this finding suggests that we have built a high-quality mcqa contrast set. after verifying the quality of our contrast set, we test 12 llms @xcite @xcite @xcite on the unifiedqa evaluation set and its mined contrast set ( §4.2). our llms surpass random guessing using just the choices on the original evaluation set, aligning with prior work. @xcite . however, when prompted with both the question and choices, llm accuracy rankings between the initial evaluation set and contrast set are highly consistent, with kendall's τ near 0.9. since no llm rank drops markedly, we claim that our tested llms are not ranking highly on mcqa leaderboards solely due to their ability to exploit choices-only shortcuts. despite the susceptibility of mcqa to high choices-only accuracy, we argue the task may still reliably rank llm knowledge. as a result, we recommend that future works continue to explore the behavior of llms in choices-only settings to help explain how llms can adeptly perform mcqa without the question. ## automatic contrast set creation we assume we are given an mcqa dataset d with data entries d @xmath0, c i , a i ), where q i is a question, c i is a list of choices, and a @xmath1. our goal is to build a contrast set d contr from d to probe if llms rely on choice-only shortcuts in mcqa. typically, humans manually create contrast sets @xcite , as model-generated data can be biased @xcite . however, since writing mcqa problems is challenging even for experts @xcite , we seek to automatically mine a contrast set d contr from the original dataset d without model-generated data. to automatically build contrast sets, we need mcqa entry pairs in the style of figure 1 -pairs with the same choices c ′ = {a i , a j }, but questions q i and q j leading to distinct answers a i and a j in c ′ , respectively. we define this format as an entry pair p @xmath2, {a i , a j }, a i ), (q j , {a i , a j }, a j )⟩. creating the largest possible d contr with distinct questions is equivalent to finding the maximum set of unique entry pairs p ij in d. in the next sections, we outline our graph-based approach to mine entry pairs from the original dataset d to form d contr . ## conclusion we find that while llms can perform well in mcqa without access to the question, it does not mean that model rankings on mcqa leaderboards are largely influenced by this ability. this result supports the claim that mcqa can rank the knowledge and ability of llms to reason over both questions and choices. further, we are aligned with recent work that suggests that high choices-only accuracy does not necessarily imply that models are incapable of true reasoning or comprehension, so we hope future works continue to explore what strategies llms may employ to perform well in choices-only settings. our application of graph mining to mcqa sheds light on one way to do this-the automatic construction of contrast setsand we hope similar methods can be applied to other tasks to enhance llm interpretability.