Increasing Probability Mass on Answer Choices Does Not Always Improve Accuracy (2023)

Abstract: When pretrained language models (LMs) are applied to discriminative tasks such as multiple-choice questions, they place probability mass on vocabulary tokens that aren’t among the given answer choices. Spreading probability mass across multiple surface forms with identical meaning (such as “bath” and “bathtub”) is thought to cause an underestimation of a model’s true performance, referred to as the “surface form competition” (SFC) hypothesis. This has motivated the introduction of various probability normalization methods. However, many core questions remain unanswered. How do we measure SFC? Are there direct ways of reducing it, and does doing so improve task performance? We propose a mathematical formalism for SFC which allows us to quantify and bound its impact for the first time. We identify a simple method for reducing it—namely, increasing probability mass on the given answer choices by a) including them in the prompt and b) using in-context learning with even just one example. We show this method eliminates the impact of SFC in the majority of instances. Our experiments on three diverse datasets and six LMs reveal several additional surprising findings. For example, both normalization and prompting methods for reducing SFC can be ineffective or even detrimental to task performance for some LMs. We conclude with practical insights for effectively prompting LMs for multiple-choice tasks.

## introduction large pre-trained autoregressive language models (lms) have shown success not only on generation, but also on classification and multiple-choice (mc) tasks with pre-specified answer choices. to succeed on such tasks, one must pay attention to 5 )) does not always translate to better accuracy (right; eq. ( 1 )), as shown here for three different prompt formats (x-axis; §6.3) each with one incontext example. results are averaged across mmlu, openbookqa, and commonsenseqa. including answer choices in the prompt substantially increases probability mass on them. however, high probability mass is surprisingly not always associated with increased accuracy; in fact, it can lead to a substantial drop in performance (e.g., for opt 30b and gpt-3 curie). however, accuracy improvements may be attributable to multiple sources. without a metric to directly measure sfc, it is difficult to assess whether the increased accuracy is, in fact, a consequence of reduced sfc, or something else. to address this gap, we propose a mathematical formalism for studying sfc and use it to investigate the following four research questions. we conclude by leveraging these insights to provide practical recommendations on how to maximize lm accuracy on multiple-choice tasks when using zero-and few-shot prompting. ## related work while various methods have been proposed to improve the accuracy of sequence scoring using probability normalization methods @xcite @xcite , none investigate a direct metric for surface form competition and whether their methods alleviate it. to the best of our knowledge, we are the first to systematically study the role of incontext examples and prompt format on pma, as well as how pma relates to accuracy. @xcite show pmi dc (eq. ( 4 )) improves over sequence scoring accuracy in most cases for gpt-2 and gpt-3 models of various sizes in 0-shot and 4-shot settings. somewhat contradictorily, @xcite find that us-ing a version of pmi dc where the denominator is p θ (x|"answer : "or"a : ") improves task performance on the validation set for only 5 out of 17 datasets investigated. @xcite propose to fit a linear weight matrix and bias vector for classification tasks with a shared label set, such that the labels all have equal probability prior to observing x. malkin et al. (2022) add hyperparameters to eq. (4 ) that are fit on a dataset's validation set, showing further gains at test-time. @xcite propose to score inputs given answer choices, which is mathematically equivalent to pmi dc ( §3.2). this results in lower variance and better worst-case accuracy on multiple-choice tasks in 0-and few-shot settings for gpt-2. @xcite investigate the effect of showing answer choices in the prompt and applying pmi based scoring (though not the combination of the two). they find that the success of one method over the other tends to vary by dataset and model. our results elucidate further that the overall capability of an lm may be a key factor in whether pmi-based scoring improves accuracy or not. ## preliminaries given a task input x, a set of answer choices l, and the correct answer y * ∈ l, the goal of a multiplechoice classification task is to correctly select y * . ## conclusion we take a novel approach to studying the effects of prompt format, in-context examples, and model type on probability assigned to answer choices and its relationship with end task performance, by proposing a new formalization of surface form competition and a quantifiable metric (pma). this is an important step towards understanding and improving the use of lms for discriminative tasks. our findings shed light into the role of probability distributions in model performance. they also challenge intuitive assumptions such as showing answer choices for mc tasks is always beneficial, which is a common practice @xcite @xcite . practical insights: we find that the best way to use vanilla lms in multiple-choice settings is to provide a string prompt without answer choices and apply probability normalization. for instructiontuned models, on the other hand, answer choices should be shown and in an enumerated prompt format, and probability normalization should not be used. more generally, our results reveal that