Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models (2024)

Abstract: Semantic plausibility (e.g. knowing that “the actor won the award” is more likely than “the actor won the battle”) serves as an effective proxy for general world knowledge. Language models (LMs) capture vast amounts of world knowledge by learning distributional patterns in text, accessible via log probabilities (LogProbs) they assign to plausible vs. implausible outputs. The new generation of instruction-tuned LMs can now also provide explicit estimates of plausibility via prompting. Here, we evaluate the effectiveness of LogProbs and basic prompting to measure semantic plausibility, both in single-sentence minimal pairs (Experiment 1) and short context-dependent scenarios (Experiment 2). We find that (i) in both base and instruction-tuned LMs, LogProbs offers a more reliable measure of semantic plausibility than direct zero-shot prompting, which yields inconsistent and often poor results; (ii) instruction-tuning generally does not alter the sensitivity of LogProbs to semantic plausibility (although sometimes decreases it); (iii) across models, context mostly modulates LogProbs in expected ways, as measured by three novel metrics of context-sensitive plausibility and their match to explicit human plausibility judgments. We conclude that, even in the era of prompt-based evaluations, LogProbs constitute a useful metric of semantic plausibility, both in base and instruction-tuned LMs.

## introduction effective language use heavily relies on general world knowledge. to determine which sentence is the most appropriate response in a given situation, 1 code and data are accessible at https://github.com/ carina-kauf/llm-plaus-prob . a language user often needs to establish whether the sentence (e.g., "the actor won the award") plausibly describes the world. in nlp, leveraging world knowledge is important both for specific tasks (such as information retrieval) and for general success of a language model during interactions with a user (such as establishing common ground). language models (lms) are well-positioned to acquire many aspects of general world knowledge by capturing distributional patterns in their training data @xcite . for instance, by observing that "actor" occurs more frequently with "award" than with "battle", the lm might implicitly learn that actors are more likely to win awards than battles. thus, a simple word-incontext prediction objective can enable an lm to acquire vast amounts of world knowledge. we focus on one particular way to assess general world knowledge: estimates of sentence plausibility. plausible sentences conform with world knowledge whereas implausible sentences violate it; thus, the ability to distinguish plausible and implausible sentences is an indicator of underlying world knowledge capabilities. plausibility judgments can be tested using both single sentences (e.g., "the actor won the award" > "the actor won the battle") and setups where plausibility depends on the context of the previous sentences (e.g., "the girl dressed up as a canary. she had a little beak." > "the girl was cute. she had a little beak."). a quantitative metric that has been commonly used to evaluate world knowledge in lmsincluding semantic plausibility-are the log probability scores (logprobs) of the output under the model. logprobs are relatively easy to compute and constitute a direct measure of model behavior (as opposed to more implicit metrics such as decod-ing probe accuracy; @xcite . however, logprobs are sensitive to many different surface-level text properties, such as individual word frequency, output length, and tokenization schemes @xcite @xcite . furthermore, distributional patterns are subject to the reporter bias: people typically communicate new or unusual information rather than trivial or commonly known facts @xcite . thus, the link between logprobs and semantic plausibility is confounded by a variety of factors. the most common way to control for confounds influencing logprobs is by leveraging the minimal pairs setup @xcite @xcite @xcite @xcite @xcite and/or quantifying the effects of multiple contributing factors on the resulting score @xcite , with the rise of instruction-tuned lms @xcite @xcite , it has become possible to directly evaluate lm capabilities via targeted natural language prompting @xcite . thus, we ask: is explicitly prompting instruction-tuned lms for semantic plausibility judgments more effective than using logprobsderived plausibility estimates? and how does instruction tuning affect the logprobs estimates themselves? on the one hand, prompting might provide a better estimate of plausibility by filtering out influences of extraneous factors not mentioned in the prompt. furthermore, instruction tuning might diminish the influence of those factors even at the level of logprobs themselves, leading instruction-tuned models to perform better under either metric. on the other hand, initial direct comparisons of logprobs and prompting measures on different linguistic/semantic knowledge datasets revealed that prompting may, in fact, systematically underestimate the model's internal knowledge by requiring the models not only to solve the task, but also to correctly interpret the prompt and to translate their answer into the desired output format @xcite . as access to logprobs for newer models becomes restricted, it is important to understand what knowledge can be accessed, and what knowledge is inaccessible to the experimenter if prompting is the only way to interact with lms. in addition, some researchers reported that instruction tuning decreases the utility of raw logprobs in domains such as confidence judgments @xcite and prediction of human reading times @xcite , a change that might or might not be compensated by superior prompting performance and that needs to be acknowledged as the field is shifting toward instruction-tuned lms. in this paper, we provide a systematic comparison of semantic plausibility estimates in instructiontuned lms. we test lms' knowledge of plausibility in single-sentence (experiment 1) and contextualized scenarios (experiment 2) and compare implicit (logprobs-based) and explicit (promptingbased) plausibility judgments. we find that: ## experiment 1: single-sentence plausibility judgments in this section, we test lms' knowledge of semantic plausibility in isolated sentences. we compare implicit (logprobs-based) and explicit (prompting-based) judgments derived from the base and instruction-tuned versions of 3 state-ofthe-art lms. we also compare lm scores with human plausibility judgments. ## conclusion overall, we show that, for both base and instruction-tuned models, logprobs remain a more reliable measure of semantic plausibility than naive zero-shot prompting. this is true in scenarios that evaluate both isolated and contextdependent sentence plausibility. even though instruction-tuning has been claimed to align lms and human brain representations @xcite , other studies show that it does not always help for the alignment at the behavioral level @xcite . our results show that the base logprobs estimates for simple world knowledge scenarios do not drastically change as a result of instruction tuning, showing approximately the same amount of implicitly encoded information as representation derived from next-word prediction. in some cases, however, instruction tuning can lead to less alignment of logprobs to human plausibility judgments than those of base model versions. concerning lms' sensitivity to sentence context, we observe that by using logprobs at the level of the target word, all the models perform around 90% with respect to the ground truth and are well aligned to human judgement patterns. however, when using sentence-level logprobs we notice that the models have the tendency to "re-balance" the log likelihoods after processing an unexpected word, with the consequence that semantically anomalous sentences and contextually-licensed ones become harder to distinguish. although it is possible that model-and taskspecific prompts will outperform raw logprobs as a way to estimate sentence plausibility, our work highlights that logprobs are an easy, zero-shot way to assess lms' implicit knowledge. thus, getting a raw logprobs estimate of model performance can provide an initial estimate of whether or not custom prompt-based solutions can be successful or-in some cases-obviate the need for prompt tuning altogether.