Predicting Reference: What do Language Models Learn about Discourse Models? (2020)

Abstract: Whereas there is a growing literature that probes neural language models to assess the degree to which they have latently acquired grammatical knowledge, little if any research has investigated their acquisition of discourse modeling ability. We address this question by drawing on a rich psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. The results reveal that, for the most part, the prediction behavior of neural language models does not resemble that of human language users.

## introduction the impressive power of deep learning based language models has inspired a new line of computational psycholinguistics research that examines the extent to which linguistic knowledge lies latent within their distributed networks. this work has primarily focused on linguistic phenomena that syntactic theory tells us requires syntactic knowledge to capture, with mixed results @xcite @xcite @xcite . this paper asks a new question: to what extent do these language models capture the linguistic knowledge required to perform discourse modeling? we are unaware of any work that has addressed this question directly. perhaps the closest research has centered on the winograd schema challenge (wsc) @xcite , which evaluates the ability of systems to employ world knowledge to interpret ambiguous pronouns in minimal pairs that resemble winograd's famous example (1). however, wsc is essentially a 'fill in the blank' problem-solving task, and doesn't evaluate the extent to which systems display humanlike ability to model discourse in an online, incremental fashion. we instead take our inspiration from psycholinguistic work that has focused on this question. for instance, the bayesian model of pronoun interpretation @xcite posits that comprehenders resolve the meaning of a pronoun via bayesian principles by combining their estimates of the speaker's production biases (the likelihood term) with their top-down expectations about which entities are likely to be mentioned next (the prior term, which we refer to as the next-mention bias). @xcite demonstrate that an array of semantic biases (e.g., verb semantics) and pragmatic biases (e.g., coherence relations) that have been claimed to influence pronoun interpretation directly actually do so only indirectly, by conditioning the prior. the role of the prior in the bayesian model is directly analogous to its role in bayesian approaches to tasks such as speech recognition and machine translation, where a language model provides the prior probabilities. we argue that the ability to capture the influence of context on next-mention biases is thus a particularly appropriate task for evaluating the extent to which language models capture discourse modeling knowledge. our focus will be on effects of verb semantics that the psycholinguistic literature has shown to influence next-mention biases. these studies have used a passage com-pletion paradigm, in which experimental participants are presented with context clauses followed by either a full stop (2a) or a conjunction (2b-c), and asked to complete the passage with the first follow-on sentence that comes to mind. ## comparisons and predictions if neural language models latently acquire discourse modeling knowledge, they should be able to distinguish between contexts that are superficially similar but which are known from psychological research to yield significant effects on next-mention biases. we focus on three such contrasts. implicit causality verbs the first comparison is between two kinds of so-called implicit causality (ic) verb, exemplified in (3a-b). sentences with ic verbs generate an expectation that the follow-on sentence will participate in an explanation coherence relation, in which the second sentence provides a cause or reason for the eventuality described by the first @xcite . however, the two types differ in which event participant causality is attributed to. ic1 verbs (3a) have been experimentally shown to generate a strong expectation that the preceding subject will be mentioned next in the follow-on sentence-we heard that john is aggravating, and we now expect to hear why @xcite @xcite @xcite @xcite ; inter alia). ic2 verbs (3b), on the other hand, have been shown to generate a strong expectation that the preceding object will be mentioned next in the follow-on sentence-we heard that mary received praise, and we now expect to hear why. we can then ask: do ic1 and ic2 verbs generate different expectations in language models for next mention in otherwise identical contexts? there are also subsidiary predictions regarding the use of connective prompts as in (2b-c). for both types of ic verbs, because prompts strengthen their biases, since virtually 100% of the continuations will now be explanations rather than 60% as found in full stop prompt conditions @xcite . so we expect to see a higher probability of nextmention of the subject with because prompts for ic1 verbs, and likewise for objects for ic2 verbs. both types of ic verb, however, are known to have a strong bias to the object in result coherence relations @xcite -in which the follow-on describes an effect rather than a cause-which are enforced by the and as a result prompt. for ic1 verbs, therefore, we should see a strong shift toward the object with and as a result prompts compared to full stop prompts. to summarize the predictions: 1a. ic1 contexts with full stop prompts should display a stronger next-mention bias to the subject compared to ic2 contexts. ## experimental setup we evaluated two state-of-the-art, pre-trained autoregressive language models (lms): gpt-2 large @xcite and transformer-xl (dai 2019 ). 2 the experiments were conducted in a zero-shot setting, and the task of generating continuations was reformulated to a next-word prediction task. prior to tokenization, the input stimulus was prepended with a token indicating the beginning of the sentence. additionally, the inputs for transformer-xl were prepended with a padding text to account for the shorter stimulus length. 3 to capture the diversity of ways in which event participants can be mentioned in the context sentence, the twelve frames shown in ## results implicit causality comparison the nextmention biases toward the subject produced by each system in the ic verb conditions are shown in tables 2 and 3 . ## conclusions we set out to evaluate the extent to which neural lms latently acquire the discourse modeling capability necessary to perform a particular type of incremental processing that human language users do: the ability to predict what entities are most likely to be mentioned next. we examined three context pairs with superficially similar linguistic properties that the experimental literature has shown to result in divergent next-mention biases, both with and without connectives. the results were mostly, but not entirely, negative. on the one hand, we found no compelling evidence that the lms are sensitive to any of the three manipulations within the verbal complex in the context sentence. on the other hand, one could argue for preliminary support for the claim that one of the lms-gpt-2-is sensitive to the occurence of the two connectives examined here. future work will be required to assess the extent to which these effects do in fact reflect the acquisition of a latent form of discourse modeling ability. our conclusions, of course, remain preliminary in a number of respects. first, we have analyzed the behavior of only two systems. since each system can be said to stand proxy for a single experimental participant, these results could be argued to be less robust than human language studies, which typically utilize several dozen participants. whereas this limitation is shared with previous work that probes lms for inherently acquired syntactic knowledge, the robustness of the findings would be enhanced by examining a broader range of systems and/or system configurations so as to better capture the kinds of variation found among groups of human participants. second, we have focused here on broad contrasts between context types that have been studied in the psycholinguistic literature. although the stimuli employed were modeled after those used in experimental studies, to improve the robustness of the findings we felt it necessary to compute means over a variety of sentence frames ( third, there are many variations of the studies presented here that could be attempted. examples would include variants that employ longer and more realistic contexts. in this initial investigation we focused on single-sentence contexts so as to hew as closely as possible to previous experimental work. we hope that this short paper will inspire further research that takes next steps in this and a variety of other directions. finally, we want to be clear that we do not claim that the two lms examined have in any sense 'failed' at this task-they were obviously not trained for this purpose. our goal instead was to pose the novel question of to what extent discourse knowledge of the sort examined here may exist latently in the models. that having been said, we consider the identification of alternative language model architectures that are capable of capturing the requisite discourse modeling capability for this task to be an interesting challenge problem for future work.