Entailment Semantics Can Be Extracted from an Ideal Language Model (2022)

Abstract: Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. We prove that entailment judgments between sentences can be extracted from an ideal language model that has perfectly learned its target distribution, assuming the training sentences are generated by Gricean agents, i.e., agents who follow fundamental principles of communication from the linguistic theory of pragmatics. We also show entailment judgments can be decoded from the predictions of a language model trained on such Gricean data. Our results reveal a pathway for understanding the semantic information encoded in unlabeled linguistic data and a potential framework for extracting semantics from language models.

## introduction recent advances in building computational models of language have been powered by distributional semantics: the idea that the possible surrounding contexts for a text span encode its meaning (firth, 1957). in particular, large pretrained language models (lms; peters et al., 2018; devlin et al., 2019; brown et al., 2020) have become an integral part of nlp systems: the representations that emerge from training to predict missing words in a text are empirically useful for natural language understanding tasks. despite this empirical progress, bender and koller (2020) argue lms cannot learn to understand the semantics of sentences. this is because of a mismatch between the lm training objectivepredicting missing words in text ("form")-and bender and koller's conception of meaning as the relation of a sentence to the external world. thus bender and koller claim "that the language modeling task, because it only uses form as training data, cannot in principle lead to learning of meaning." in this paper, we argue meaning can be learned from form because the communicative goals of human authors encode semantic information in unlabeled text. we show how this semantic information can be extracted to resolve semantic relations between sentences (e.g., whether one sentence entails another): in this inferentialist sense, ideal lms encode the meaning of sentences. this argument has been raised speculatively by others (michael, 2020; potts, 2020; bommasani et al., 2021), but we will rigorously justify it here with formal results. to give the simplest (and least general) illustration of our argument, we first assume training data is generated by overly idealized uniformly truthful speakers: agents who decide what to say by picking sentences they consider true uniformly at random. foot_1 this very coarsely captures human authors' goal of being informative (rather than misleading) to their listeners (grice, 1975). in theorem 1, we prove a sentence x entails sentence y if and only if, after uttering x, a uniformly truthful speaker is just as likely to say y as to repeat x. thus, entailment semantics can be extracted from probabilistic languages generated by uniformly truthful speakers. uniformly truthful speakers are not a realistic model of humans: while humans favor true sentences to false ones (grice, 1975), not all true sentences are equally likely to be produced. it is a common principle in linguistic theories of pragmatics that human speakers choose their utterances in order to balance two competing objectives: (a) conveying information to their listener and (b) brevity (levinson et al., 1983; grice, 1975). we define a class of gricean speakers who optimize for these objectives, and prove in theorem 2 that x entails y if and only if a simple equation holds in terms of text probabilities produced by such speakers. thus, entailment semantics can be decoded from probabilistic languages generated by gricean speakers. the previous results assume access to a language's ideal likelihood function, but, in practice, one only ever receives a corpus sampled from the language. moving to the corpus setting, we analyze how much data allows approximately computing our derived entailment test using probabilities estimated from sentence frequencies in a corpus. we find that the corpus size needed to guarantee the entailment test holds approximately is inversely related to the likelihood of the sentences. we estimate that approximating the entailment test between 4-word sentences using corpus frequencies is possible with ∼ 10 10 sentences, about the size of the gpt-3 training data (brown et al., 2020). on the other hand, approximating the entailment test for 10-word sentences should be possible with ∼ 10 17 sentences, or ∼ 10 7 gpt-3 corpora. thus, extracting entailment judgments using corpus frequencies requires an infeasible amount of dataeven by modern nlp standards. to overcome this limitation, one might hope to use probabilities estimated by lms to extract entailment judgments between longer sentences that are rare even in a large corpus. with synthetic data generated by gricean speakers, we find that entailment can be decoded from n-gram lm predictions to some extent. however, we speculate that current neural lms may not score the probability of rare text well enough to enable decoding entailment judgments between natural language sentences. in summary, our main contribution is to show a correspondence between the semantics of text and its likelihood, assuming the likelihood function matches models of human text production from linguistic theory. determining whether a sentence in a probabilistic language entails another sentence can be reduced to modeling the probabilities of strings in the language. in practice, entailment judgments between very short sentences can be extracted from corpus frequencies, but this becomes infeasible for slightly longer sentences. lms can in principle be used to extrapolate the likelihood of longer strings, but we hypothesize current lms are not well-suited for doing so well enough to enable extracting entailment from natural language. our theory demonstrates a formal sense in which unlabeled text data encodes linguistic meaning and makes quantitative predictions for (a) how to extract semantics from text corpora and (b) how much data this requires. ## gricean speakers in this section, we will define a new class of speakers who pick sentences in order to be informative to their listener, while also trying to be concise. to do this, we will draw on information theory to formalize what it means for a speaker to be informative. we will then derive a distributional relation that is isomorphic to entailment for gricean speakers, which is a generalization of the relation for uniformly truthful speakers from §3. ## decoding entailment from empirical text frequencies we have so far shown that entailment judgments can be extracted from the sentence probabilities in the ideal distribution p(z). what happens if, more practically, we estimate the probability of a sentence by its frequency in a large corpus sampled from p(z)? we prove this method enables feasible extraction of entailment judgments between very short sentences, but the corpus size may become intractably large for longer sentences. imagine we have a finite corpus of iid sentences {z i } n @xmath6, each sampled from p(z). let p(z) be the empirical frequency of a text z in the corpus, i.e., if π(z, z ′ ) returns whether text z is a prefix of text z ′ , p(z) = 1 n n @xmath7, z i ). since p(z) encodes entailment via our extraction rules, p(z) will encode entailment between sentences if p(z) is close to p(z). a naive notion of closeness is to guarantee, for all ϵ, there exists some number of texts n such that, with high probability, |p(z) -p(z)| < ϵ. but this notion is not strict enough: if p(z) is small, this difference will also be small, even if p(z) is not a good approximation of p(z) on a relative scale. instead, we want to guarantee that p(z)/p(z) converges to 1, or, equivalently, that their difference as log probabilities converges to 0. this ensures that convergence will still be meaningful for low-probability sentences, which most sentences are in natural language. under this standard, rarer sentences take more samples to approximate. define the sentence complexity k p (z) = 1 p(z) . we bound the approximation error in terms of k p (z). @xcite lemma 3 for @xmath8. then, for all x, @xmath9, for all δ > 0, the following holds with probability 1 -δ -2(1 -p(xy)) n , the proof follows analogously to theorem 3. the main improvement of theorem 5 compared to theorem 3 is that the probability the bound holds no longer depends on the unlikely probability p(yy). we also get the benefit that the cost complexity factor has been reduced to only depend on c(xy) and obtain better constants (2 √ 2 instead of 8), although these changes are likely less important than removing the dependence on p(yy). of course, the drawback is that we are assuming access to the cost function c(y). if we have such access, though, the improvements in the bound suggest we may be able to extract entailment from a finite corpus of gricean text with better sample complexity than if we did not.