Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right (2021)

Abstract: Large language models have shown promising results in zero-shot settings. For example, they can perform multiple choice tasks simply by conditioning on a question and selecting the answer with the highest probability. However, ranking by string probability can be problematic due to surface form competition—wherein different surface forms compete for probability mass, even if they represent the same underlying concept in a given context, e.g. “computer” and “PC.” Since probability mass is finite, this lowers the probability of the correct answer, due to competition from other strings that are valid answers (but not one of the multiple choice options). We introduce Domain Conditional Pointwise Mutual Information, an alternative scoring function that directly compensates for surface form competition by simply reweighing each option according to its a priori likelihood within the context of a specific task. It achieves consistent gains in zero-shot performance over both calibrated and uncalibrated scoring functions on all GPT-2 and GPT-3 models on a variety of multiple choice datasets.

## introduction despite the impressive results large pretrained language models have achieved in zero-shot settings @xcite , we argue that current work underestimates the zeroshot capabilities of these models on classification tasks. this is in large part due to surface form competition-a property of generative models that causes probability to be rationed between different valid strings, even ones that differ trivially, e.g., by capitalization alone. such competition can be largely removed by scoring choices according to figure 1 : while humans select from given options, language models implicitly assign probability to every possible string. this creates surface form competition between different strings that represent the same concept. example from commonsenseqa @xcite . domain conditional pointwise mutual information (pmi dc ), which reweighs scores by how much more likely a hypothesis (answer) becomes given a premise (question) within the specific task domain. specifically, consider the example question (shown in figure 1 ): "a human wants to submerge himself in water, what should he use?" with multiple choice options "coffee cup", "whirlpool bath", "cup", and "puddle." from the given options, "whirlpool bath" is the only one that makes sense. yet, other answers are valid and easier for a language model to generate, e.g., "bathtub" and "a bathtub." since all surface forms compete for finite probability mass, allocating significant probability mass to "bathtub" decreases the amount of probability mass assigned to "whirlpool bath." while the total probability of generating some correct answer may be high (i.e., across all valid surface forms), only one of these is a listed option. this is particularly problematic here, because "whirlpool bath" will be much lower probability than "bathtub," due to its rarity. more generally, methods that do not account for surface form competition will favor answers with fewer lexical paraphrases. pmi dc factors out the probability of a specific surface form, by instead computing how much more probable a hypothesis is when conditioned on a premise. we use a domain premise string to estimate the unconditional probability of a hypothesis in a given domain. on commonsenseqa, for example, we compute the probability of each answer option immediately following the string "? the answer is:", and then divide the conditional probability by this estimate to calculate pmi dc . this scaling factor reweighs answer scores according to the surface form competition that is inherent to the domain or task, e.g. completions of the domain premise that are just inherently unlikely will be upweighted more. this allows us to directly measure how much an answer tells us about the question and vice versa (mutual information is symmetric, see §3). valid hypotheses no longer need to compete with each other: both "whirlpool bath" and "bathtub " will be considered reasonable answers to the question, and so both will attain a high score. extensive experiments show that pmi dc consistently outperforms raw, normalized, and calibrated probability scoring methods on zero-shot multiple choice for more than a dozen datasets and it does so for every model in the gpt-2 and gpt-3 families ( §4); this holds true across different possible prompts and in preliminary few-shot experiments as well. to better explain these gains, we use the distinct structure of the copa dataset @xcite to remove surface form competition entirely, showing that all methods perform well in this idealized setting ( §5). additionally, we analyze the only three datasets where pmi dc does worse than other methods and put forward a hypothesis for why normalizing log probabilities works better than raw probabilities ( §6). we conclude with a discussion of how generative models should be used for selection tasks ( §7). surface form competition when applying generative models to multiple choice problems, simply choosing the highest probability answer becomes problematic due to different valid surface forms competing for probability. indeed, recent work in question answering has demonstrated the importance of considering all multiple choice options together @xcite , rather than independently assigning each answer a score and simply choosing the highest. this is a difficult strategy to adapt to left-to-right generative language models, which implicitly choose between all possible strings. using unsupervised language models pretrained on relatively expansive corpora exacerbates surface form competition because such language models generate a much wider distribution than a given question answering dataset contains. pmi work in dialogue has used pmi to promote diversity @xcite @xcite @xcite . recently, brown et al. (2020) used a scoring function resembling pmi dc for zero-shot question answering, though they only use the string "a:" as a prompt for the unconditional probability estimate, whereas we use a task-specific domain premise (see §3 for details). furthermore, brown et al. (2020) only report this scoring method on three datasets (arc, openbookqa, and race, included here) out of the more than 20 tested and do not compare scores with their standard method, averaging loglikelihoods (avg in this work). in contrast, we report a comprehensive comparison on gpt-3 and gpt-2, as well as shedding light on the underlying issue of surface form competition in §5. contextual calibration recently, @xcite describe a new method for calibrating the probabilities of an lm using a learned affine transformation. though geared towards few-shot learning, the authors devise a clever means of using "content free inputs" for zero-shot learning. @xcite calibrate for three forms of bias: (1) majority label bias, (2) recency bias, and (3) common token bias. pmi dc directly compensates for common token bias by dividing by the domain conditional probability of each answer, and performs superior to contextual calibration (cc) in the majority of cases. prompt sensitivity recent work highlights lm sensitivity to inputs, and proposes to consider paraphrases of the prompt to overcome this @xcite , as well as noting that certain trigger tokens @xcite can strongly effect the output of such models. in this work, we focus on the surface form of possible outputs, but do also analyze robustness to different prompts in §4.4. interpreting language models language models tend to model selectional preferences and thematic fit @xcite rather than semantic plausibility @xcite . probability, possibility and plausibility are distinct @xcite , but reporting bias @xcite means that language models only model what people are likely to write (on websites that are easily crawled). pmi dc aims to adjust for these challenges to better measure the underlying agreement between language models and human judgements, but of course is still subject to the limits and biases of the language model used. ## zero-shot scoring strategies this paper does not define any new modeling or finetuning methods. rather, we propose the broad use of pmi dc scoring for any given model and prompt. pmi dc compensates for the fact that different correct answers compete for probability, even though only one will be listed as the correct multiple choice option. we begin by describing the two most common methods currently in use. ## removing surface form competition what if we used the probability of the premise given the hypothesis, p (x|y i ), instead? while we are still measuring the probability of a surface form (e.g. "the bar closed."), it is the same surface form across different options ("it was crowded so", "it was 3 am so"), eliminating the surface form competition. y i and y i can now both attain high scores if they are both correct answers, by causing x to be likely. we call this scoring-by-premise. causal language models like gpt-3 cannot measure this directly, because they are only capable of conditioning on past tokens to predict future tokens. we exploit the structure of the copa dataset to create "copa flipped" via a simple transformation, shown in figure 3 . copa consists of cause and effect pairs (cause so effect, and effect because cause). in the original dataset, whatever comes second (either cause or effect) has two options that a model must choose between. these can be reversed by switching cause and effect, then substituting the natural inverse relation ("because"-→"so" and "so"-→"because" ). ## analysis failure cases there are three datasets where pmi dc does not consistently outperform other methods: hellaswag, arc easy, and boolq. surprisingly, each is dominated by a different method. hellaswag is most amenable to avg. on examination we find that hellaswag is more focused on the internal coherence of the hypotheses, rather than external coherence, i.e. how much a premise and hypothesis match. this is likely due to hel-laswag being generated by gpt-2 @xcite and filtered with bert, as it contains relatively on-topic but intrinsically strange hypotheses that humans can distinguish from natural data. arc easy yields the highest scores to lm, i.e., selecting the highest probability option. @xcite note that arc easy questions can be solved by a retrieval or word co-occurrence baseline, while examples that were answered incorrectly by both were put into the challenge split. this suggests a bias towards a priori likely phrases. manual inspection reveals many stock answers, e.g., "[clouds are generated when] ocean water evaporates and then condenses in the air," supporting our hypothesis. finally, boolq, a reading comprehension dataset in which all answers are either "yes" or "no", is best solved by an unconditional baseline. this is because the dataset presents truly complex questions that require more reasoning than gpt-2 or 3 are capable of out of the box. indeed, none of the methods reported do better than the majority baseline, except pmi dc with the largest gpt-3 model. why does length normalization work? past work offers little explanation for why avg should be a successful strategy, other than the intuition that estimates are strongly length biased and require compensation. length bias may be caused by the final softmax layer of current language models assigning too much probability mass to irrelevant options at each time-step, as noted in open-ended generation, character-level language modeling, and machine translation @xcite @xcite . another (not mutually exclusive) argument is that length normalization may account for unconditional probability in a similar way to pmi dc . length normalization is often measured over byte pair encoding (bpe) tokens @xcite and bpe tends to produce vocabularies where most tokens are equally frequent @xcite . recent evidence suggests that language is approximately uniformly information dense @xcite @xcite . as such, length in bpe tokens may correspond roughly to a unigram estimate of log-probability, supposing that bpe tokens have approximately uniform uni-gram frequency. the adjustment made by avg is still somewhat different than pmi dc , (division of log terms rather than subtraction) but could have a similar effect, if length and probability correlate. ## discussion language models are density estimation functions that assign probability to every possible string, but there are often many strings that could represent a given idea equally well. our key observation is that a generative model assigning probability to a string that represents a certain option isn't equivalent to selecting the concept an option corresponds to. we expect surface form competition anywhere that generative models are used where more than one string could represent the same concept. pmi dc aligns the predictions being made by the model more closely with the actual task posed by multiple choice questions: "choose the hypothesis that explains the premise" rather than "generate the exact surface form of the hypothesis". from this perspective, pmi dc does not go far enough, because the model still cannot consider the given set of options altogether when selecting its choice. this matters when answers interact with each other, e.g., "all of the above". ## conclusion we conduct a large-scale comparison of standard and recent scoring functions for zero-shot inference across all gpt-2 and gpt-3 models. we show that pmi dc consistently outperforms previous scoring functions on a wide variety of multiple choice datasets. we also argue that compensating for surface form competition is the cause of this boost, by demonstrating that other methods work just as well as pmi dc when surface form competition is eliminated. in future work we would like to explore how surface form competition affects generation, as we hypothesize that it may be the cause of overly generic outputs under high model uncertainty.