Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment (2024)

Abstract: Do LMs infer the semantics of text from co-occurrence patterns in their training data? Merrill et al. (2022) argue that, in theory, sentence co-occurrence probabilities predicted by an optimal LM should reflect the entailment relationship of the constituent sentences, but it is unclear whether probabilities predicted by neural LMs encode entailment in this way because of strong assumptions made by Merrill et al. (namely, that humans always avoid redundancy). In this work, we investigate whether their theory can be used to decode entailment relations from neural LMs. We find that a test similar to theirs can decode entailment relations between natural sentences, well above random chance, though not perfectly, across many datasets and LMs. This suggests LMs implicitly model aspects of semantics to predict semantic effects on sentence co-occurrence patterns. However, we find the test that predicts entailment in practice works in the opposite direction to the theoretical test. We thus revisit the assumptions underlying the original test, finding its derivation did not adequately account for redundancy in human-written text. We argue that better accounting for redundancy related to *explanations* might derive the observed flipped test and, more generally, improve computational models of speakers in linguistics.

## introduction inspired by the empirical capabilities of language models (lms) trained on next-word prediction, recent work has examined if and how linguistic meaning might be inferred from raw text @xcite @xcite . a text corpus is the result of humans using text to communicate information, and doing this efficiently requires following pragmatic principles like avoiding contradictory or redundant sentences. therefore, training to predict whether sentences can co-occur (which can reduce to next-token prediction) might lead lms to represent semantic relationships between sentences @xcite @xcite . but does sentence co-occurrence provide enough signal for lms to learn to represent complex semantic phenomena like entailment? @xcite derive a simple equation by which the entailment relation between two sentences can be detected using their co-occurrence probability in a corpus generated by speakers who avoid redundancy. intuitively, non-redundant speakers will rarely utter entailed sentences, so low co-occurrence probability of two sentences is predictive of their entailment relationship. this means that, in principle, learning to model sentence co-occurrence perfectly requires an lm to implicitly model entailment, and entailment classifications can be extracted from the co-occurrence probabilities of such an lm. however, @xcite s theoretical result has two caveats. first, it assumes an "ideal" lm that perfectly models the likelihood of texts in a language. second, it makes the strong (but theoretically motivated; @xcite assumption that speakers always avoid redundancy. it is thus unclear whether real lms infer a model of entailment from sentence co-occurrence probabilities in their training data, both because lms may misestimate probabilities and because the required assumptions about human speakers may be too simplified. in this work, we empirically evaluate the distributional entailment test from @xcite : can we use it to classify entailment from lm probability estimates? overall, we find across a wide range of entailment benchmarks and lms that a variant of the entailment test consistently detects entailment well above random chance. this suggests that lm probability judgments are sensitive to the relationships between sentence meanings that are reflected in sentence co-occurrence pat-terns, at least to some extent. this further suggests that next-word prediction is a strong enough objective for lms to acquire at least a partial model of entailment relationships between sentences. however, this result comes with a surprise. across many entailment benchmarks, we find that the direction of the test is flipped compared to merrill et al.'s theoretical test: higher co-occurrence probabilities correlate with entailment when the opposite is expected! we take this as evidence against a theory of human speakers based purely on minimizing redundancy. analyzing natural corpora, we find humans are often more redundant than merrill et al.'s non-redundant speakers, which could explain the flipped test. we present a preliminary account of how better accounting for explanations (one observed type of redundancy) might predict the flipped test. overall, our results motivate future work in computational pragmatics accounting for redundancy and are a case study for how the data aggregated about many speakers in lms can be used to test and develop pragmatic theories. ## distributional semantics and the entailment test there is an old debate in linguistics and nlp about whether distributional semantics-the idea that text co-occurrence patterns can contain semantic information-captures semantics in any true sense @xcite . this debate goes back at least to @xcite , who argues that sentence co-occurrences patterns in a corpus could be used as data to build a linguistic theory of semantics, but it has been revisited in recent years in terms of lms. in particular, bender and koller (2020)-in disagreement with harris (1954)-took a strong stance against the claim that lms "understand" language because understanding requires modeling communicative intent or at least conventionalized semantic denotations, both of which do not appear explicitly in the training data for lms. while it is certainly true that lms are only trained on surface forms, counterarguments to bender and koller (2020) have been given for how lms might be able to reconstruct semantic information from their training data. one line of counterarguments @xcite @xcite echoes @xcite , positing that sentence co-occurrence probabilities contain information about semantics because speakers aim to be truthful and informative and are thus unlikely to produce contradictory or redundant pairs of sentences. properly learning which sentences can cooccur (part of lm training) thus amounts to acquiring a semantic representation of which sentences are contradictory or redundant with one another. example 1a is unlikely to be uttered because it contains a contradiction. more subtly, example 1b is unlikely because its second sentence is uninformative given the first, even though they are consistent. in contrast, example 1c is acceptable because it is consistent and adds new information. thus, example 1 suggests sentence co-occurrence is governed by semantic constraints against inconsistency and redundancy. if strong lms correctly model such cooccurrences, they might need an implicit model of sentence semantics to determine these properties. ## entailment test results overall, we find the test predicts entailment on the broad-coverage datasets, but only when the test score is flipped compared to the theoretical test (i.e., a larger score means entailment). however, the pattern is more complicated for the targeted data, where some constructions follow the flipped trend but others follow the original, unflipped test. ## corpus study: characterizing naturalistic linguistic redundancy a surprising finding from the previous section is that the entailment test is robustly flipped: entailed continuations tend to be more likely than nonentailed ones. this suggests the gricean speaker assumed to derive the test may be too simplistic to account for humans. in particular, we hypothesize the disconnect may be because human speakers are explicitly redundant in certain contexts unlike gricean speakers, who always avoid redundancy. we thus search for natural instances of contextually entailed text in corpora to better understand why real human speakers produce redundant sentences. data. to find contextually entailed sentences in different types of discourse, we consider a variety of web domains: book3 @xcite , wikipedia (en) @xcite , multi-news (fabbri et al., 2019) and reuters-21578 @xcite , yahoo! answers topics @xcite , and yelp reviews @xcite . finding contextually entailed text. for each document in each corpus, we construct premise and hypothesis pairs by choosing six contiguous sentences, with the first five as the premise and sixth as the hypothesis. we use entailment classifiers finetuned from t5 @xcite and roberta @xcite to detect entailment pairs and take the intersection of examples considered entailment by both. we then manually filter to remove incorrect entailment pairs (details in §d). ## towards accounting for redundancy we have found that, in practice, the flipped entailment test better detects entailment than the original one and that this trend is also supported by an oracle logistic regression analysis ( §4). our corpus study ( §5) pointed to a possible explanation: the original test relied on the fact that gricean speakers always avoid redundancy, but real humans produce redundant text in certain contexts. quantitatively, the rate of contextually entailed sentences in natural corpora was higher than we would expect if the corpus authors were gricean speakers. qualitatively, specific examples suggested humans are redundant both to repeat important information and for the sake of explanation, i.e., they state entailed summaries or conclusions after a more detailed premise. prima facie, such redundancy could lead to a flipped entailment test if entailed continuations, which are fully redundant, become more likely than other continuations. however, it is crucial to have a more concrete theory of why speakers are redundant to evaluate this and ideally explain why the test direction varies across constructions. we thus consider some possible angles to extend gricean speakers to account for redundant speech acts and whether these extensions predict the flipped test. ## conclusion our results show that sentence co-occurrence probabilities computed by lms can predict entailment relationships, with a stronger effect for better lms. this suggests these lms are implicitly modeling semantic properties of text to some extent in order to predict the next token, in line with harris (1954)'s proposal that sentence co-occurrences can serve as data for building a theory of semantics. however, the best empirical test for entailment we found was flipped compared to @xcite 's theoretical test. this suggests a more nuanced theory of pragmatics beyond gricean speakers is needed to explain how entailment relationships are reflected in sentence co-occurrences. our corpus study revealed that humans in corpora produce more contextually entailed sentences than idealized gricean speakers, suggesting pragmatic theories that better handle redundancy might explain our findings. we took a first step by considering how to model redundancy due to noise tolerance and explanation, but the job is far from done. rather, our findings call for future work that more completely accounts for the pragmatics of redundancy, especially concerning explanations. this can both advance linguistic theory and serve as a foundation for understanding how meaning can be inferred from a corpus, as well as as the potential limits of distributional semantics and lms.