Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? (2024)

Abstract: Multiple-choice question answering (MCQA) is often used to evaluate large language models (LLMs). To see if MCQA assesses LLMs as intended, we probe if LLMs can perform MCQA with choices-only prompts, where models must select the correct answer only from the choices. In three MCQA datasets and four LLMs, this prompt bests a majority baseline in 11/12 cases, with up to 0.33 accuracy gain. To help explain this behavior, we conduct an in-depth, black-box analysis on memorization, choice dynamics, and question inference. Our key findings are threefold. First, we find no evidence that the choices-only accuracy stems from memorization alone. Second, priors over individual choices do not fully explain choicesonly accuracy, hinting that LLMs use the group dynamics of choices. Third, LLMs have some ability to infer a relevant question from choices, and surprisingly can sometimes even match the original question. Inferring the original question is an impressive reasoning strategy, but it cannot fully explain the high choices-only accuracy of LLMs in MCQA. Thus, while LLMs are not fully incapable of reasoning in MCQA, we still advocate for the use of stronger baselines in MCQA benchmarks, the design of robust MCQA datasets for fair evaluations, and further efforts to explain LLM decision-making.1

heading: 1 Introduction
text: Multiple-choice question answering (MCQA) has been key for evaluating large language models (LLMs), valued for its ease of scoring and alignment with human testing protocols (Robinson and Wingate, 2023). In this task, a model is given a question and list of choices as input, and must select the choice that best answers the question. This process is intended to assess if LLMs comprehend, use relevant pretraining knowledge, and reason via
1Our data and code are available at https://github. com/nbalepur/mcqa-artifacts
information from both the question and the choices. Since MCQA is meant to evaluate several model abilities, it is often used to rank LLMs in benchmarks such as the Open LLM leaderboard (Beeching et al., 2023) and the Holistic Evaluation of Language Models (Liang et al., 2023) benchmark.
Given this role of MCQA, it is essential to verify that MCQA accuracy reflects the abilities we intend to measure. While this concern has been studied with prompt sensitivity (Zheng et al., 2024), an unexplored aspect of MCQA-based LLM evaluation is dataset artifacts—patterns or biases in text that models may use as shortcuts, rather than completing the task as intended. Researchers have found that partial-input baselines (Poliak et al., 2018), models trained using just a subset of inputs, can exploit artifacts in parts of the input. Such models include hypothesis-only models in natural language inference (Poliak et al., 2018; Herlihy and Rudinger, 2021) and passage-only models in reading comprehension (Shah et al., 2020; Kaushik and Lipton, 2018). If LLMs rely on artifacts, our datasets may not only fail to assess the skills they are designed to, but we may also overestimate LLMs and face generalizability issues in deployment (Wiegreffe and Marasović, 2021).
Thus, to gauge if MCQA measures LLM abilities as intended, we probe if LLaMA-2 (Touvron et al., 2023), Falcon (Penedo et al., 2023), Phi-2 (Abdin et al., 2023), and Mixtral (Jiang et al., 2024) can use artifacts on three benchmarks: ARC (Clark et al., 2018), MMLU (Hendrycks et al., 2021), and HellaSwag (Zellers et al., 2019). To do so, we test a partial-input choices-only model in MCQA, which omits the question and only sees the choices. Concretely, we propose partial-input prompts, a strategy analogous to trainable partial-input models, where only a subset of inputs are used in a prompt.
For our tested LLMs and datasets, we find that in 11/12 cases, choices-only accuracies significantly surpass majority baselines, with a sizeable gain of
10308
0.33 in LLaMA on HellaSwag. Notably, this is achieved with minimal few-shot examples, ranging from 5 on MMLU to 25 on ARC. To give more insights, we design tests to help explain how LLMs obtain high choices-only accuracy and see if this behavior stems from surface-level shortcuts usually associated with artifacts (Du et al., 2023), or if more advanced reasoning is employed. We test three hypotheses, shown in Figure 1 (right).
1) Memorization: One explanation is that the LLM has been trained on the test set and is recalling answers via memorization (§4). To test this, we create partial-input prompts that give no discriminative information in the choices, and can only be answered if the LLM has already seen the answer (Figure 1, top). These prompts never largely best the majority baseline, leading us to test more complex behaviors to explain choices-only accuracy.
2) Choice Dynamics: With no evidence of memorization, we define two properties of choices that could explain the high choices-only accuracy (§5). First, LLMs may have individual priors on choices, such as favoring choices that contain “always.” Second, LLMs may use the group dynamics of choices, where the models assess a choice based on the other choices, such as favoring an odd-numbered choice if the other choices are even. We refer to these latter behaviors as meta-strategies, where the model reasons over all choices beyond individual priors.
To quantify the impact of individual priors on choices-only accuracy, we prompt the LLM to classify the correctness of each choice in isolation (Figure 1, middle). This prompt only sees one choice at a time and thus the LLM can only use its individual priors. In several cases, the individual priors alone cannot explain the high choices-only accuracy, implying that LLMs reason over all choices
rather than just using cues from individual choices.
3) Abductive Question Inference: Inspired by work using LLMs to verbalize inferences (Hoyle et al., 2023) and given the potential use of metastrategies, we see if LLMs can reason via the metastrategy of abductive question inference (AQI) (§6). We design a two-step prompt where the LLM: 1) abductively infers the missing question from the choices; and 2) answers its own question (Figure 1, bottom). We call it abductive QI as we aim to generate the question that best explains the observed choices (Peirce, 1974; Bhagavatula et al., 2020).
AQI performs similarly to the choices-only prompt, even besting it for 3 LLMs on HellaSwag, and they also exhibit moderate agreement, implying that AQI may be just one of many strategies LLMs use in choices-only settings. We then study the questions generated by LLaMA on ARC. Out of the 43 cases when AQI picks the gold answer and the inferred question is answerable by one of the choices, 42% match the meaning of the original question. Thus, we argue that a successful choicesonly prompt does not always mean that the LLM uses surface-level shortcuts linked with artifacts; in some cases, it may recover the missing question.
Our intention is not to discredit MCQA evaluations. Instead, by highlighting the high choicesonly accuracy of LLMs and investigating several potential explanations, we aim to give suggestions to enhance transparency and robustness in benchmarking, as well as provide insights into the decision-making of LLMs in partial-input settings. Our contributions can be summarized as follows:
1) We assess choices-only prompts with four LLMs and three MCQA datasets, being the first to show that LLMs can exhibit high partial-input accuracy in few-shot settings and with minimal exemplars.
2) We test several hypotheses to help explain this high choices-only accuracy, such as memorization, choice priors, and abductive question inference. 3) We release a thorough black-box MCQA analysis suite to facilitate transparent LLM evaluations.

heading: 2 Experimental Setup

heading: 2.1 MCQA Task Definition
text: Our MCQA task is based on Robinson and Wingate (2023), where the model is given: 1) a question q; and 2) a set of four choices C = {ca, cb, cc, cd}, exactly one of which is correct (i.e. gold choice cg ∈ C). Using these inputs, the LLM must give the letter of the correct option a ∈ {(A),(B),(C),(D)}.
The standard MCQA prompt is a full prompt, where the LLM uses the question q and choices C:
Prompt 2.1: Full Prompt
Question: q Choices:\n(A) ca \n(B) cb \n(C) cc \n(D) cd Answer: a
In the prompt box above, and all prompt boxes in this paper, the non-highlighted text represents the input prompt provided to the model, while the highlighted text represents what the model generates (e.g. the letter of the choice in Prompt 2.1). All of our prompts are few-shot to guide the LLM outputs, leaving zero-shot artifact exploitation for future work. In the few-shot prompts, each exemplar follows the same format shown in the prompt box. For the exemplars, the highlighted text is replaced with the ground truth (Example in Appendix A.4).

heading: 2.2 Models
text: Due to the cost of closed-source LLMs, we study four open-source LLMs: LLaMA-2 70B2 (Touvron et al., 2023), Falcon 40B3 (Penedo et al., 2023), Mixtral 8x7B (Jiang et al., 2024), and Phi-2 (2.7B) (Abdin et al., 2023). Each LLM is run with HuggingFace. We generate with default parameters and set the output length between 5 and 200 tokens.

heading: 2.3 Datasets
text: We select three MCQA benchmarks from the Open LLM Leaderboard (Beeching et al., 2023): ARCChallenge (Clark et al., 2018), MMLU (Hendrycks et al., 2021), and HellaSwag (Zellers et al., 2019), collectively testing factual knowledge, scientific reasoning, and commonsense reasoning. We use
2LLaMA is 8-bit quantized on MMLU to use less memory. 3Falcon is 20-shot on ARC to fit its 2048 token limit.
the entire evaluation sets, but omit the questions in ARC that do not have four choices to ensure all questions have four choices. We follow the Open LLM Leaderboard to pick few-shot examples and use 25 random training examples for ARC, 10 random training examples for HellaSwag, and the 5 given training examples for each subject in MMLU. While our experiments reflect the prompting setups of present MCQA benchmarks, we believe that future works could extend our analysis with zeroshot prompting (Kojima et al., 2022) to limit the influence of priors from the few-shot examples.

heading: 2.4 Evaluation Protocol
text: As defined in §2.1, the LLM aims to generate the letter of the gold answer choice. This black-box setup allows us to study LLM behavior without accessing LLM internals. However, one drawback is that LLMs may produce invalid outputs (e.g. “(E)” when there are four choices), especially when using atypical prompts like the memorization prompts in Figure 1 (top). In these cases, the LLM may not understand the task even with few-shot examples.
One solution is to mark every invalid output as incorrect, but this unfairly penalizes the LLM when the mistake likely stems from the atypical prompt. Thus, for a fair evaluation, when the LLM gives an invalid output, we treat it as random guessing and assign a score of 0.25. In Appendix A.5, we show versions of all experiments where invalid outputs are marked as wrong, which do not alter our claims.

heading: 3 Performing MCQA With No Question
text: To test if LLMs exploit MCQA artifacts, we design partial-input choices-only prompts (§3.1), and show that these prompts often largely best majority baselines4 (§3.2). High partial-input accuracies typically imply artifact exploitation via simple shortcuts (Du et al., 2023). However, in §6 we explore if some of this overperformance can be attributed to the LLM’s ability to infer the original question. In this case, it would be difficult to claim that the model has bypassed the intended MCQA format. Thus, in this section, we establish an initial ceiling for artifact exploitation in the choices-only setting.

heading: 3.1 Prompt Design
text: Artifacts can be uncovered via partial-input models (Poliak et al., 2018)—trained models that omit
4A majority class baseline always predicts the most frequent answer choice found in the dataset.
parts of the input. To adapt this for LLMs, we propose an analogous method: partial-input prompts. In MCQA, the apt partial-input prompt is a choicesonly prompt, where LLMs only use the choices:
Prompt 3.1: Choices-Only Prompt
Choices:\n(A) ca \n(B) cb \n(C) cc \n(D) cd Answer: a
We expect Prompt 3.1 to perform near a majority baseline, as ignoring the question should ideally equal random guessing. An accuracy largely above this suggests that the model may use artifacts.

heading: 3.2 Results
text: In 11/12 cases, the choices-only prompt surpasses the majority baseline significantly, indicating that LLMs may be using artifacts in MCQA benchmarks (Figure 2). Further, larger LLMs tend to have higher choices-only accuracy, implying that more capable LLMs may use artifacts to a higher degree. In Appendix B.1, we study scaling laws with LLaMA to understand this relation more. Overall, these results serve as an initial ceiling for artifact exploitation in choices-only settings, which we use to motivate our subsequent analyses that help attribute where this accuracy may stem from.
Takeaway: Prior work has found artifacts with trained partial-input models (Gururangan et al., 2018), but we are the first to show that LLMs exhibit high choices-only accuracy in MCQA benchmarks, even in few-shot settings with limited exemplars. For researchers seeking to have LLMs per-
form MCQA as intended, where both the question and choices are needed, we recommend three approaches: First, along with typical MCQA metrics, choices-only prompts can be reported as stronger alternatives to majority baselines (Poliak et al., 2018). Second, researchers can design datasets with more robust protocols, such as the Winograd pair format (Levesque et al., 2011), to mitigate the potential for artifacts. Third, HellaSwag has the highest choicesonly accuracy (0.585 with LLaMA) and it is also the only dataset with human-written gold answers and model-written distractors. Thus, the gold answers may contain stylistic cues distinct from the distractors, which can inform strong discriminators like LLMs in partial-input settings. To avoid introducing such artifacts, we advise researchers to use a consistent approach when generating text data.

heading: 4 Hypothesis 1: Memorization
text: Our first hypothesis to explain choices-only accuracy is test set leakage, where the LLM is trained on the test set (Zhou et al., 2023). If this occurred, the LLM could recall the answer via memorization (Huang et al., 2022). To test this, we design prompts only answerable via memorization (§4.1). We assess these prompts and find that memorization cannot explain choices-only accuracy (§4.2).

heading: 4.1 Prompt Design
text: While it is possible to test memorization via contamination analysis (Sainz et al., 2023), this does not reveal how memorization affects MCQA accu-
racy. As a simple solution, we create partial-input prompts where the LLM must return the letter a of the correct choice, but without any discriminative information in the choices. Such prompts are only answerable if the LLM has already been trained on the example, allowing us to quantify how memorization alone impacts choices-only accuracy:
Prompt 4.1: Gold Choices Prompt
Question: q Choices:\n(A) cg \n(B) cg \n(C) cg \n(D) cg Answer: a
Prompt 4.2: Empty Choices Prompt
Question: q Choices:\n(A) \n(B) \n(C) \n(D) Answer: a
Prompt 4.3: No Choices Prompt
Question: q Answer: a
We cannot detect all forms of memorization, as adversarial actors could train an LLM on the test set while shuffling the choice order, bypassing our prompts. However, we assume a non-adversarial setting to test exact memorization and conjecture that if substantial exact memorization had occurred, these prompts would best the majority baseline.

heading: 4.2 Results
text: Figure 3 shows no strong evidence that our LLMs memorized the test sets, as the prompts only barely surpass the majority baseline once. While impossible to rule out memorization entirely, we believe that more complex strategies lead to high choicesonly accuracy, motivating our ensuing analyses on choice dynamics (§5) and question inference (§6).

heading: 5 Hypothesis 2: Choice Dynamics
text: With no evidence of memorization in §4, we study choice dynamics in MCQA. We define two properties of answer choices that could account for the LLMs’ high accuracy in choices-only prompts: 1) Individual Priors: LLMs may learn strong priors over specific choices from in-context learning or pretraining. For instance, an LLM may believe a priori that “Albert Einstein” is often correct, or subtly that choices containing “not” are often wrong, informing its decision in the choices-only setting. 2) Group Dynamics: LLMs may also ground the evaluation of the correctness of a choice based on its relation to surrounding choices. For example, if
an LLM is given a math question with three odd choices and one even choice, the model may reason that the even choice is correct, as it has distinct parity. We define these processes as meta-strategies, where the LLM reasons or makes decisions over a group of choices beyond assessing a single choice, such as inferring the original question (§6) or eliminating similar options (Balepur et al., 2023).
In this section, we tease apart these two factors of choice dynamics. We first design a prompt format that tasks LLMs with classifying the individual correctness of choices (§5.1), isolating the effect of individual priors on choices-only accuracy. Next, we develop a scoring system to make these binary classification scores directly comparable to the accuracy of the choices-only prompt (§5.2). Finally, we assess the relation of individual priors and metastrategies to choices-only accuracy (§5.3).

heading: 5.1 Prompt Design
text: In a full (Prompt 2.1) or choices-only (Prompt 3.1) prompt, the LLMs’ accuracy can be explained by both individual priors and group dynamics, since these prompts include all of the choices in C. To isolate how individual priors alone affect choicesonly accuracy, we prompt the LLMs to classify the correctness of each choice c ∈ C separately. Since these prompts only include a single choice, LLMs cannot use group dynamics. We create two versions of this prompt, when the question q is present or absent, mirroring the full and choices-only prompts:
Prompt 5.1: Individual Full Prompt
Question: q Choice: c Answer: True { if c = cg else } False
Prompt 5.2: Individual Choices-Only Prompt
Choice: c Answer: True { if c = cg else } False
Our other few-shot prompts include all choices, using n MCQA questions as exemplars for an nshot prompt, but the above individual prompts must be adapted to manage the increase of 4n total exemplars (4 choices c per n questions). Thus, to keep an n-shot format, we segment the exemplars such that ⌈n2 ⌉ classify a random distractor as False and ⌊n2 ⌋ classify the gold answer as True, balancing exposure to both True and False labels.

heading: 5.2 Converting Individual Scores
text: To study MCQA when LLMs can use individual priors and group dynamics versus only individual priors, we seek to compare the accuracies of the group full and choices-only prompts (Prompts 2.1, 3.1) against their individual counterparts (Prompts 5.1, 5.2). But the group setting is four-way classification, while the individual setting is binary classification, preventing a direct comparison of the tasks.
Thus, we introduce a function that converts the binary accuracy in the individual setting to a score comparable to the four-way accuracy in the group setting. Based on elimination testing (Ben-Simon et al., 1997), we define score(Ctrue, cg), returning the chance the LLM picks the gold choice cg ∈ C given the choices Ctrue ⊆ C it classifies as True:
score(Ctrue, cg) =    0.25 if |Ctrue| = 0 0 elif cg /∈ Ctrue
1 |Ctrue| elif cg ∈ Ctrue
If the LLM predicts False for all choices (i.e. |Ctrue| = 0), it implies uncertainty and equates to random guessing, giving a score of 0.25. Otherwise, if the LLM does not classify the gold answer cg as True (i.e. cg /∈ Ctrue), the LLM would not select cg when given C, resulting in a score of 0. Lastly, when cg ∈ Ctrue, the LLM’s ability to pick cg is akin to guessing among Ctrue, yielding a score of 1|Ctrue| . With this scoring, we can directly compare the group and individual prompt accuracies.

heading: 5.3 Results
text: In Figure 4, on ARC and MMLU, both the group full prompt and its individual counterpart lead in accuracy, followed by the group choices-only prompt and its individual counterpart. The individual full prompt bests the group choices-only prompt, implying that seeing the question is more helpful than seeing all the choices on these datasets. Further, the individual choices-only prompts underperform the
group choices-only prompts. Thus, the LLMs’ individual priors alone do not fully explain the choicesonly accuracy, suggesting the use of meta-strategies and group dynamics in choices-only settings.
On HellaSwag, the trend varies by model. For LLaMA, the order mirrors ARC/MMLU, but the group choices-only prompt bests the individual full prompt. Hence, for LLaMA, seeing all choices is more informative than the question, and the LLM may still use meta-strategies. In contrast, for Mixtral and Phi, the individual prompts surpass their group versions. This implies that these LLMs may: 1) mainly use individual priors in the choices-only setting; and 2) struggle with the context present in group prompts, due to the dataset’s longer choices.
Takeaway: Partial-input models are often linked to simple shortcuts (Du et al., 2023), such as using statistical cues in individual choices, but we find that LLMs may reason over groups of choices. To study when meta-strategies can effectively be used by LLMs, future works can try to manually control specific properties of distractors (e.g. parity).

heading: 6 Hypothesis 3: Question Inference
text: Given the potential use of meta-strategies (§5), we see if LLMs can use abductive reasoning— giving the best rationale for an observation (Peirce, 1974)—in choices-only settings. Inspired by work using LLMs to verbalize inferences (Hoyle et al., 2023), we test this via abductive question inference (AQI). Below, we design (§6.1), evaluate (§6.2), and qualitatively analyze (§6.3) AQI.

heading: 6.1 Prompt Design
text: We implement AQI via the two-step process of: 1) generating a question qgen using the choices C; and 2) asking the LLM to pick the correct answer a to its own question qgen. Step 1 uses the prompt:
Prompt 6.1: Question Generation Prompt
Choices:\n(A) ca \n(B) cb \n(C) cc \n(D) cd Question: qgen\n Answer: a
We extract qgen from Prompt 6.15 to perform step two of AQI, where the LLM answers qgen:
Prompt 6.2: Self-Ask Prompt
Question: qgen Choices:\n(A) ca \n(B) cb \n(C) cc \n(D) cd Answer: a
In Prompt 6.2, the LLM may ignore qgen, treating it like the choices-only prompt (Prompt 3.1). To test this, we design a baseline where the LLM must answer qrand, a randomly sampled question from the test set unrelated to the current choices C:
Prompt 6.3: Random Question Prompt
Question: qrand Choices:\n(A) ca \n(B) cb \n(C) cc \n(D) cd Answer: a
If the self-ask prompt outperforms the random question prompt, the generated question qgen provides more useful information than a random question qrand, suggesting that the LLM is using qgen.

heading: 6.2 Quantitative Results
text: AQI always results in an accuracy near the choicesonly prompt on ARC and MMLU (Figure 5). One hypothesis is that the LLMs ignore the generated question qgen in the self-ask prompt (Prompt 6.2), but in 11/12 cases, using the random question qrand underperforms qgen. Thus, qgen is of higher average quality6 than qrand, meaning our LLMs can
5The LLM also outputs “Answer: a” after qgen. We initially wanted AQI to be 1 step where qgen acts as a chainof-thought (i.e. merge steps one and two of AQI), but this 1-step process is less effective (see Appendix B.5).
6It is also possible for qrand to be misleading and qgen to just be irrelevant/non-misleading. However, our qualitative analysis (§6.3) reveals that qgen is often answerable by the choices, and thus we claim that it is higher quality than qrand.

heading: Dataset LLaMA Falcon Mistral Phi Random
text: extract information from qgen relevant to answering the question. Further, on HellaSwag for Falcon, Phi, and Mixtral, AQI bests the choices-only prompt. AQI is a reasoning-based strategy and thus, future works can try to reason with other strategies (Huang and Chang, 2023) to see if LLMs can obtain even higher accuracy in choices-only settings.
Since the choices-only prompt and AQI perform similarly, the choices-only prompt may effectively function as the two-step process of AQI. To test this, we use Cohen’s κ (Cohen, 1960) to quantify if the choices-only prompt and AQI answer similar questions correctly and incorrectly. The two strategies exhibit moderate agreement (average κ of 0.32), much higher than a random baseline near 0 (Table 1). Thus, while LLMs may implicitly perform AQI in choices-only prompts, we speculate that they may also employ other strategies jointly, such as reasoning over multiple inferred questions.

heading: 6.3 Qualitative Analysis
text: In our qualitative analysis, we study the behavior of AQI grouped into three research questions below. We use ARC, as MMLU has questions requiring expert knowledge, and HellaSwag contains multisentence questions that are hard to interpret. We study LLaMA’s outputs, but show other LLMs and examples of inferred questions in Appendix B.6. Q1—Why can AQI fail: We study when AQI ends up picking the correct (gold) or incorrect (non-gold) choice, sampling 50 cases of each. The two errors
of AQI we examine are: 1) generating a question qgen that cannot be answered by any choice; and 2) selecting a choice a that does not answer qgen. We compute IP(Ans), the proportion where qgen is answerable by one of the choices, and for each answerable qgen, if the model correctly answers qgen (IP(Corr | Ans)). In Table 2 (left), IP(Ans) and IP(Corr | Ans) are high (over 0.8), even when AQI leads to an incorrect answer. Thus, AQI errors with LLaMA mostly stem from inferring questions related to non-gold choices, rather than generating an unanswerable qgen or incorrectly answering qgen. Q2—Can LLMs infer the original question: For each answerable qgen studied in Q1, we annotate if it matches the meaning of the original question (IP(Match | Ans)). When a is incorrect, qgen never matches the original question, but remarkably, in 43 cases when qgen is answerable and a is correct, it matches the original question 42% of the time (Table 2, right). None of the matches are identical, leading us to believe that this ability is not due to exact memorization. The high rate of matched questions suggests that LLMs may have the ability to verbalize MCQA inferences via abductive reasoning, motivating future works to study the faithfulness of these rationales (Turpin et al., 2023). Q3—When is AQI effective: We explore the effectiveness of AQI based on the nature of the inferred question qgen. We sample 100 AQI cases with LLaMA on ARC and mark each qgen as: 1) unanswerable; 2) answerable and not matching the original question q; or 3) answerable and matching q. To isolate the efficacy of question inference from LLaMA’s ability to answer questions, we have one “oracle” annotator read each qgen and mark all correct choices Cans ⊆ C with access to Google. The annotator can mark multiple options as correct to account for when qgen is ambiguous. The score of this human/LLM team is 0 if the gold answer is not in Cans, and 1|Cans| if the gold answer is in Cans.
Table 3 shows that, as expected, the average human/LLM score is 0 when qgen is unanswerable and 1 when qgen matches q. However, when qgen is answerable and does not match q, it obtains a score of 0.293—above random guessing (0.25). Thus,
even when LLaMA generates a question distinct from the original, it still often pertains to the gold answer, indicating an ability to infer questions relevant to the gold answer beyond random chance.
Takeaway: In some cases, LLMs can use reasoning to reconstruct the question from the choices. AQI is more impressive than the surface-level shortcuts linked with artifacts (Du et al., 2023), but it cannot fully explain choices-only accuracy, meaning that the rest of the choices-only accuracy could be attributed to artifacts and surface-level shortcuts. Thus, while we support artifact-robust evaluations to test LLMs as intended, we also urge researchers to not fully dismiss successful partial-input models (Srikanth and Rudinger, 2022), as such models may still employ impressive, unexpected reasoning.

heading: 7 Related Work

heading: 7.1 Dataset Artifacts
text: Data collection protocols can introduce artifacts exploitable by models (Du et al., 2023). Artifacts stem from many sources, such as biases in crowdworkers (Gururangan et al., 2018; Geva et al., 2019; Parmar et al., 2023) or models in synthetic data (Yu et al., 2023). Model reliance on artifacts poses risks, leading to performance overestimation and generalization issues (Wiegreffe and Marasović, 2021).
One approach to uncover artifacts is partial-input models (Poliak et al., 2018), which omit parts of inputs. Such models include hypothesis-only models in NLI (Poliak et al., 2018; Herlihy and Rudinger, 2021; Srikanth and Rudinger, 2022), passage-only models in reading comprehension (Kaushik and Lipton, 2018; Shah et al., 2020), and question-only models in visual QA (Goyal et al., 2017).
Other methods to find artifacts include text perturbations (Feng et al., 2019; Sugawara et al., 2019; Gardner et al., 2020), adversarial data (Jia and Liang, 2017; Wallace et al., 2019; Morris et al., 2020), and probing experiments (Glockner et al., 2018; McCoy et al., 2019). To limit artifacts and reliably assess models, works have used special training objectives (Belinkov et al., 2019; Mersinias and Valvis, 2022), debiasing procedures (Ravichander et al., 2023), human-AI data collection (Liu et al.,
2022), and context altering to flip model decisions (Srikanth and Rudinger, 2022; Elazar et al., 2023).
Conversely, we show that LLMs can obtain high partial-input MCQA accuracy with few examples.

heading: 7.2 MCQA Decision Making
text: We study three facets of MCQA decision making: memorization, choice dynamics, and question inference. Each has been explored in prior works.
Several works quantify memorization via data extraction attacks (Carlini et al., 2020; Ishihara, 2023; Sainz et al., 2023) and analyzing models after dataset-specific training (Huang et al., 2022; Zhou et al., 2023). In contrast, we design the first partial-input memorization prompts for MCQA.
While many works study how humans make informed or cued MCQA guesses via choice dynamics (Royal and Hedgpeth, 2015; Royal and Stockdale, 2017), we are the first to define similar abilities (i.e. meta-strategies) with LLMs. Other studies assess if LLM pretraining priors can recognize tasks (Pan et al., 2023) and sensitivity to option order in MCQA (Pezeshkpour and Hruschka, 2023), but neither work focuses on dataset artifacts.
Lastly, prior works explore generating entire MC questions (Ch and Saha, 2018), but we are the first to use LLMs to infer the original question from the choices. Other MC strategies have been used with LLMs, such as process of elimination (Ma and Du, 2023; Balepur et al., 2023), where wrong options are removed, and Maieutic Prompting (Jung et al., 2022), involving reasoning over abductive explanations, but not in relation to artifact exploitation.

heading: 8 Conclusion
text: We find that LLMs can perform MCQA without access to the question, a result that can be achieved through few-shot prompting with minimal examples. In pursuing explanations for these results, we release an in-depth black-box evaluation suite, based on our proposed partial-input prompts. As part of our analysis, we show that LLMs still exhibit reasoning abilities in choices-only settings via the meta-strategy of question inference, suggesting that partial-input abilities of LLMs should not be completely attributed to surface-level shortcuts linked to artifacts. Along with software and analysis, we suggest ways to evaluate LLMs on MCQA as intended, where both the question and choices are needed, including: 1) reporting stronger baselines like our proposed choices-only prompt; and 2)
designing robust data creation protocols. In light of our findings, we advocate for more critical discussions around LLM evaluation, allowing us to better understand and interpret the model capabilities our benchmarks are truly assessing.

heading: 9 Limitations
text: One limitation is that our experiments are conducted in a black-box rather than a white-box manner, motivated by the increasing use of closedsource LLMs. This setup allows our analysis to be applicable to any LLM that can provide generated text outputs through prompting. However, we acknowledge that an analysis of logits from open-source LLMs could offer more insights into aspects like confidence, uncertainty, and calibration in artifact exploitation. We believe these directions would be very interesting extensions of our work.
Our experiments also do not use closed-source models like ChatGPT due to resource constraints. Based on the large number of prompts we study, we need to run inference over each MCQA dataset 16 times. This is a very expensive endeavor, as two of our datasets, MMLU and HellaSwag, both have over 10,000 questions in the test set. However, all of our experiments are designed in a black-box fashion and will be made open-source. Thus, we hope if members of the NLP community with access to more resources are interested, they can replicate our experiments with closed-source LLMs.
Lastly, LLMs are sensitive to prompts (Min et al., 2022) and hyperparameters, so it is possible that varied configurations could alter the accuracy in our experiments. Our prompts and hyperparameters were just the first ones we tried, so we believe that LLMs can likely be optimized to obtain even higher choices-only accuracy. Future works can explore these alterations more in-depth and analyze LLM sensitivity in choices-only settings.

heading: 10 Ethical Considerations
text: Through few-shot prompting, we discover that LLMs can achieve high choices-only accuracy in prominent MCQA benchmark datasets. However, our goal is not to discount the adoption of these datasets for evaluation and suggest that they are fundamentally flawed, or discredit the performance of LLMs on these datasets and claim that they are incapable of performing MCQA. Instead, we wish to provide more transparency for MCQA benchmarking and LLM decision-making, which we hope
will inspire future work on: 1) better understanding LLM decision-making in MCQA; 2) reporting choices-only baselines as a stronger alternative to majority class baselines on MCQA evaluation suites; and 3) designing more resilient benchmarks that limit the influence of artifacts.

heading: 11 Acknowledgements
text: We would like to thank members of the CLIP lab at the University of Maryland and external collaborators for their feedback and discussions of this work, including Yu (Hope) Hou, Dayeon (Zoey) Ki, Neha Srikanth, Rupak Sarkar, Shi Feng, and Jordan Boyd-Graber. We are also grateful for our discussions with Hailey Schoelkopf, Stella Biderman, and other members of the EleutherAI community. We also thank the anonymous reviewers for their feedback. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 2236417. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

heading: A Experimental Setup

heading: A.1 Dataset Details
text: All datasets were obtained from their original, publicly-available sources. These datasets are free to use for research purposes, so our research falls within their intended use. ARC uses 25 training examples (20 for Falcon) and 1165 evaluation examples; MMLU uses 5 training examples and 14032 evaluation examples; and HellaSwag uses 10 training examples and 10042 evaluation examples.

heading: A.2 LLM Inference
text: For LLaMA, Falcon, and Mixtral we allocated 24 GPU hours for almost every prompt format and dataset, and used 8 NVIDIA:RTXA5000 GPUs. The notable exception is the two-step inferring the question strategy, which required 48 GPU hours for LLaMA and Falcon, and 96 GPU hours for Falcon, per model and dataset. Phi required around 60 total GPU hours to run all prompt formats per dataset, and used a single NVIDIA:RTXA4000 GPU.
The only parameters changed are: a minimum token generation length of 5, a maximum token generation length of 200, and a stopping criteria when the LLM begins to generate the next few-shot exemplar. For LLaMA-2 on MMLU, we use 8-bit quantization. We did not perform hyperparameter search. All results are obtained from a single run.

heading: A.3 Prompt Creation
text: Given the sensitivity of LLMs to prompt formats, we followed best practices when creating our fewshot prompts. For ARC, we randomly sample 25 examples over the entire training set. The HellaSwag dataset stems from two sources: ActivityNet and wikiHow. Thus, the 10-shot prompt contains 5 randomly sampled examples from the ActivityNet portion of the dataset, and 5 randomly sampled examples from the wikiHow portion. Further, we ensure that each few-shot prompt has a balanced distribution of answer letters and that the order of exemplars is randomized.
All prompts shown in this paper are derived from these few-shot examples, following the format shown in the prompt boxes. Two slight exceptions are when the LLM performs the second step of AQI (Prompt 6.1) and answers a random question (Prompt 6.2). In these prompts, the questions in the few-shot examples are the same as the questions from the Full Prompt (Prompt 2.1), rather
than using model-generated and random questions (respectively), ensuring a fair comparison.

heading: A.4 Prompt Box Example
text: Below, we provide a detailed example to illustrate the application of our prompt boxes. Suppose we have the full prompt (Prompt 2.1) defined in §2.1:
Prompt 2.1: Full Prompt
Question: q Choices: C Answer: a
In the above prompt, the LLM uses the question q and choices C as input and is asked to generate the letter of the answer a. Suppose we have 5 few-shot examples, with questions q1 ,..., q5, list of choices C1, ..., C5, and ground truth answers a1, ..., a5. The expanded few-shot prompt for the prompt box is written below:
Prompt 2.1: Full Prompt Expanded
Question: q1 Choices: C1 Answer: a1
Question: q2 Choices: C2 Answer: a2
Question: q3 Choices: C3 Answer: a3
Question: q4 Choices: C4 Answer: a4
Question: q5 Choices: C5 Answer: a5
Question: q Choices: C Answer:
Using this prompt, the LLM must generate a, which is the highlighted text in the prompt box.

heading: A.5 Output Validity Statistics
text: In Table 4, we display the output validity statistics (i.e. if the model selects a valid letter) of each prompt format, LLM, and dataset used in our experiments. We find that in 100/108 cases, the LLMs produce valid outputs over 95% of the time. This suggests that our few-shot prompt setup typically enables the LLM to produce valid outputs.
Further, we display the results from our experiments if invalid outputs are always marked as in-
correct in Figures 13, 14, and 15. We believe either decision regarding the treatment of invalid options is valid, but we opted for the score of 0.25 to account for task misunderstanding stemming from our atypical prompt formats. Regardless, we provide both versions of the experiments so readers can comprehensively evaluate the LLMs.

heading: A.6 Metrics
text: Accuracy was computed with numpy.7 Cohen’s κ was computed with scikit-learn.8 All metrics are reported from a single run.

heading: B Detailed Results

heading: B.1 Scaling Laws
text: In Figure 6, we perform the same initial experiments on choices-only prompts in §3 with LLaMA2 models of three sizes: 7B, 13B, and 70B. Overall, we find that as model size increases, the accuracy of the choices-only prompt also increases. This again suggests that larger models may exploit artifacts to a higher degree.

heading: B.2 Decoding Strategies
text: In our experiments, we use default parameters to show the out-of-box results of our tested LLMs. To study the effect of decoding strategies, we rerun our choices-only, memorization, and individual prior experiments with LLaMA on all datasets using greedy decoding, displayed in Figures 7, 8, and 9, respectively. We find that our results are mostly consistent, suggesting that the choice of decoding strategy does not significantly influence the results. The one notable difference is that the individual choice-only prompt results in a higher accuracy with greedy decoding on all the datasets. Despite this increase, the individual choice classification still does not fully explain the accuracy of the choices-only prompt, which suggests that LLMs may still use group dynamics and meta-strategies, even with greedy decoding.

heading: B.3 Partial-input Models vs Prompting
text: We compare the performance of our partial-input prompts versus a trainable partial-input model. We train the seq2seq T5-Large model (Raffel et al., 2019) to perform MCQA using the same input sequence format as the full prompt (Prompt 2.1) and
7https://numpy.org/ 8https://scikit-learn.org/
the choices-only prompt (Prompt 3.1). T5 is independently trained on ARC and HellaSwag using 90% of the training set for training (with default parameters) and the other 10% for validation.
The T5 model has less pretraining compared to our LLMs, so a trainable T5 choices-only baseline mainly exploits annotation artifacts (e.g. common words used by crowdworkers), rather than artifacts from pretraining (i.e. world knowledge). If an LLM with the choices-only prompt can outperform T5, it would signify that the LLM is instead leveraging knowledge from pretraining.
In Figure 10, we find that on ARC, T5 is unable to exploit artifacts in the choices-only setting, achieving an accuracy very close to the majority class baseline. This indicates that our tested LLMs can achieve high choices-only accuracy on ARC due to their extensive pretraining, rather than leveraging statistical dataset correlations. On HellaSwag, however, there is a large presence of dataset artifacts, indicated by the high choices-only accuracy of T5 of 0.721. Future works can conduct more in-depth analysis to try to uncover how pretraining relates to artifact exploitation.

heading: B.4 Falcon Individual Evaluation
text: In Figure 11, we show the accuracy of the individual versus group choice experiments from §5.3. Falcon performs very poorly on the independent classification task, achieving scores near the majority class baseline in ARC and HellaSwag. Falcon is one of the weaker models we test with, so we attribute this behavior to the LLM’s less robust overall capabilities compared to our other LLMs.

heading: B.5 One-Step Question Inference
text: Figure 12 displays the accuracy of the two-step versus one-step strategy of abductive question inference (§6); in the latter, the generated question acts as a chain-of-thought (Wei et al., 2022). We find that the one-step strategy consistently underperforms its two-step version, signifying that instructing an LLM to try to infer the question and then answer this question in one step is too difficult of a task.

heading: B.6 Qualitative Analysis
text: In this section, we provide more details regarding the annotation procedure details (appendix B.6.1) as well as additional results that we could not fit into the main body of the paper (appendix B.6.2).

heading: B.6.1 Annotation Procedure Details
text: In Q1, we verify if the model’s selected answer is correct using external tools such as Google search. Further, when the model generates an ambiguous question that is answerable by many choices (e.g. “Which of these is a liquid?” with choices “milk” and “water”), if the model selects any of the correct options we mark it as correct.
In Q2, when we study if the inferred questions match the original question, we define two questions as having the same meaning if they test the same knowledge. To illustrate, in one instance, the generated question is “If there is a high demand for lumber and a low supply of trees, what will most likely happen?” and the original question is “Logging companies cut trees in a forest and send the trees to lumber mills far from the forest. The mills make boards that are used for construction. Some logging companies do not plant tree seedlings after cutting trees. Not planting tree seedlings might affect people who need boards in the future because.” We consider these two questions to be equivalent because they both fundamentally ask the model to reason about what happens if demand for lumber increases but supply decreases.
All annotators in Q1, Q2, and Q3 are the authors, ensuring that the annotations are high-quality.

heading: B.6.2 Additional Results
text: In Table 6, we provide the qualitative evaluation results Q1 and Q2 with all of our tested LLMs (LLaMA, Falcon, Mixtral, and Phi) on ARC. We note that all models showcase a similar trend discussed in §6.2, but Phi is notably much worse at inferring the question. This is expected, given that Phi is the smallest LLM used in our experiments.
We now give further analyses for LLaMA on ARC regarding when AQI is effective (Q3). First, the breakdown of our three cases for the generated question is: 15% as unanswerable, 28% as answerable and matching the original question, and the remaining 57% as answerable and not matching the original question. In terms of question discriminability, we find that 15% can be answered by none of the choices (i.e. unanswerable), 65% can be answered by exactly one of the choices (i.e. perfectly discriminable), and the last 20% can be answered by more than one choice (i.e. ambiguous).
Finally, we provide examples of questions generated from AQI. In Tables 7 and 8, we show generated questions when inferring the question succeeds, and the generated question matches and does
not match the original question, respectively. In Table 9, we show generated questions when inferring the question fails, but the question is still answerable.

heading: C Related Work on MCQA Evaluations
text: Several recent works investigate the reliability of using MCQA to evaluate LLMs. However, the overwhelming majority of these experiments focus solely on prompt sensitivity and LLM robustness (Zheng et al., 2024; Pezeshkpour and Hruschka, 2023; Wang et al., 2024; Alzahrani et al., 2024), collectively pointing out that LLM performance on MCQA can substantially change if the in-context examples, choice order, and symbols used to denote options, among other properties, are altered. Conversely, we focus on the issue of artifact exploitation, allowing us to holistically assess MCQA tasks and datasets, rather than the different ways LLMs can be implemented to perform MCQA.

heading: Future Work
text: Not planting tree seedlings might affect people who need boards in the future because. ” We consider these two questions to be equivalent because they both fundamentally ask the model to reason about what happens if demand for lumber increases but supply decreases. All annotators in Q1, Q2, and Q3 are the authors, ensuring that the annotations are high-quality