Discourse structure interacts with reference but not syntax in neural language models (2020)

Abstract: Language models (LMs) trained on large quantities of text have been claimed to acquire abstract linguistic representations. Our work tests the robustness of these abstractions by focusing on the ability of LMs to learn interactions between different linguistic representations. In particular, we utilized stimuli from psycholinguistic studies showing that humans can condition reference (i.e. coreference resolution) and syntactic processing on the same discourse structure (implicit causality). We compared both transformer and long short-term memory LMs to find that, contrary to humans, implicit causality only influences LM behavior for reference, not syntax, despite model representations that encode the necessary discourse information. Our results further suggest that LM behavior can contradict not only learned representations of discourse but also syntactic agreement, pointing to shortcomings of standard language modeling.

## introduction neural network language models (lms), pretrained on vast amounts of raw text, have become the dominant input to downstream tasks @xcite @xcite . commonly, these tasks involve aspects of language comprehension (or understanding). one explicit example is coreference resolution, wherein anaphora (e.g., pronouns) are linked to antecedents (e.g., nouns) requiring knowledge of syntax, semantics, and world-knowledge to match human-like comprehension. recent work has suggested that lms acquire abstract, often human-like, knowledge of syntax (e.g., @xcite @xcite . additionally, knowledge of grammatical and referential aspects linking a pronoun to its antecedent noun (reference) have been demonstrated for both transformer and long short-term memory architectures @xcite . humans are able to modulate both referential and syntactic comprehension given abstract linguistic knowledge (e.g., discourse structure). contrary to humans, we find that discourse structure (at least as it pertains to implicit causality) only influences lm behavior for reference, not syntax, despite model representations that encode the necessary discourse information. the particular discourse structure we examined is governed by implicit causality (ic) verbs @xcite . such verbs influence pronoun comprehension: in (1), she agrees in gender with both sally and mary, so both are possible antecedents. however, english speakers overwhelmingly interpret she as referring to sally in (1-a) and mary in (1-b), despite the semantic overlap between the verbs. verbs that have a subject preference (e.g., frightened) are called subject-biased ic verbs, and verbs with a object preference (e.g., feared) are called objectbiased ic verbs. in addition to pronoun resolution, ic verbs also interact with relative clause (rc) attachment: effects of ic have received renewed interest in the field of psycholinguistics in recent years (e.g., @xcite @xcite @xcite . current accounts of ic claim that the phenomenon is inherently a linguistic process, which does not rely on additional pragmatic inferences by comprehenders (e.g., @xcite . thus, ic is argued to be contained within the linguistic signal, analogous to evidence of syntactic agreement and verb argument structure within corpora. we hypothesize that if these claims are correct, then current lms will be able to condition reference and syntactic attachment by ic verbs with just language data (i.e. without grounding). we tested this hypothesis using unidirectional transformer and long short-term memory network (lstm; @xcite language models. we find that lstm lms fail to acquire a subject/object-biased ic distinction that influences reference or rc attachment. in contrast, transformers learned a representational distinction between subject-biased and object-biased ic verbs that interacts with both reference and rc attachment, but the distinction only influenced model output for reference. the apparent failure of model syntactic behavior to exhibit an ic contrast that is present in model representations raises questions about the broader capacity of lms to display human-like linguistic knowledge. ## related work the ability of lms to encode referential knowledge has largely been explored in the domain of coreference resolution. prior work has suggested that lms can learn coreference resolution to some extent (e.g., @xcite . in the present study, we focus on within-sentence resolution rather than the ability of lms to track entities over larger spans of text (cf. @xcite . previous work at this granularity of coreference resolution has shown lstm lms strongly favor reference to male entities @xcite , for which the present study finds additional support. rather than utilizing a more limited modeling objective such as coreference resolution (cf. cheng and erk, 2020), we followed @xcite in focusing on the representation of referential knowledge by models trained with a general language modeling objective. with regards to linguistic representations, a growing body of literature suggests that lstm lms are able to acquire syntactic knowledge. in particular, subject-verb agreement has been explored extensively (e.g., @xcite @xcite with results at human level performance in some cases @xcite . additionally, work has shown human-like behavior when processing reflexive pronouns, negative polarity items @xcite , center embedding, and syntactic islands @xcite . this literature generally suggests that lms encode some type of abstract syntactic representation (e.g., @xcite . additionally, recent work has shown lms learn linguistic representations beyond syntax, such as pragmatics and discourse structure @xcite @xcite . the robustness of these abstract linguistic representations, however, have been questioned in recent work, suggesting that learned abstractions are weaker than standardly assumed (e.g., @xcite @xcite . the present study builds on these recent developments by demonstrating the inability of lms to utilize discourse structure in syntactic processing. ## language models we trained 25 lstm lms on the wikitext-103 corpus @xcite with a vocabulary constrained to the most frequent 50k words. 1 we used two pretrained unidirectional transformer lms: transformerxl @xcite and gpt-2 xl @xcite . 2 transformerxl was trained on wikitext-103, like our lstm lms, but has more parameters and a larger vocabulary. gpt-2 xl differs from the other models in lacking recurrence (instead utilizing non-recurrent self-attention) and in amount and diversity of training data (1 billion words compared to the 103 million in wikitext-103). as such, we caution against extracting explicit, mechanistic claims from the present study concerning the relationship between learned linguistic knowledge and model configurations and training data. instead, our work points to apparent differences between transformers and lstms with regard to use and acquisition of discourse structure, leaving explanatory principles to further work. ## interactions with reference the results of @xcite suggested that referential contrasts based in grammatical features (e.g., gender) would be easier for models to discern then those purely focused on referential selection (e.g., antecedents with the same gender but differing in preference). to evaluate this claim, we analyzed the degree to which ic verb type (i.e. subject vs. object biased) influenced i) model pronoun preferences when the possible referents differed in gender (e.g., sally feared bob because...), and ii) similarity of model representations between the pronoun and possible referents when they share the same gender (e.g., fred feared bob because he...). our prediction was that ic would have a weaker influence in (ii) than (i). ## interactions with syntactic attachment we turn now to the relationship between ic verbs and syntax. recall the prediction that object-biased ic verbs should interact with rc attachment to license more cases of syntactic attachment to the higher noun compared to lower noun (i.e. chef in anna scolded the chef of the aristocrats who was/were...). ## discussion the present study examined the extent to which discourse structure, determined by implicit causality verbs, could be acquired by transformer and lstm language models (cf. sally frightened mary because she... and sally feared mary because she...). specifically, we evaluated, via comparison to human experiments, whether ic verb biases could influence reference and syntactic attachment in lms. analyses were conducted at two levels of granularity: model behavior (e.g., probability assigned to possible next words) and model representation (e.g., similarity between hidden representations). given the claims in recent literature that implicit causality arises without extra pragmatic inference on the part of human comprehenders, we hypothesized that lms would be able to acquire such contrasts (analogous to their ability to acquire syntactic agreement). we found that lstm lms were unable to demonstrate knowledge of ic either in influencing reference or syntax. however, a transformer (trans-formerxl) trained on the exact same data as the lstm lms was able to partially represent an ic distinction, but model output was only influenced by ic bias when resolving reference, not syntactic attachment. in evaluating a transformer model trained on vastly more data (gpt-2 xl), we found a more robust, human-like sensitivity to ic bias when resolving reference: subject-biased ic verbs increased model preference for subject pronouns and object-biased ic verbs increased model preferences for object pronouns. however, the same mismatch as transformerxl between model representation and model behavior arose in processing syntactic attachment. in contrast to our results, davis and van schijndel (2020a) showed syntactic predictions for lstm lms are influenced by some aspects of discourse structure. a simple explanation for these conflicting results may be that the lms we examined here are unable to learn the syntactic operation of attachment, and thus no influence of discourse can surface. the erasure of number agreement in the final layers of the transformer lms (see section 5.4) provides compelling evidence towards this conclusion. 20 from a theoretical perspective, the present study provides additional support for the centering of implicit causality within the linguistic signal proper. that is, ic bias is learnable, to some degree, without pragmatic inference as hypothesized in section 1 (see also @xcite . the mismatches in syntactic representations and behavior suggest, however, that models ignore the abstract categories that are learned, contrary to human findings (cf. @xcite . we believe a solution may lie in changing model training objectives (i.e. what linguistic unit should be predicted). psycholinguistic studies focusing on the interaction of discourse and syntax have suggested that coherence relations may be the unit of linguistic prediction, in contrast to the next-word prediction used in most language modeling work (see @xcite . we leave to future work an investigation of this suggestion as well as teasing apart the exact role that training data and model architecture play in the interaction between types of linguistic representation. elyce dominique williams. 2020. language experience predicts pronoun comprehension in implicit causality sentences. master's thesis, university of north carolina at chapel hill. thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, r'emi louf, morgan funtowicz, and jamie brew. 2019. huggingface's transformers: state-of-the-art natural language processing. arxiv, abs/1910.03771. a stereotypically gendered nouns used in referential experiments male female man woman boy girl father mother uncle aunt husband wife actor actress prince princess waiter waitress lord lady king queen son daughter nephew niece brother sister grandfather grandmother