Large Language Models Are Partially Primed in Pronoun Interpretation (2023)

Abstract: While a large body of literature suggests that large language models (LLMs) acquire rich linguistic representations, little is known about whether they adapt to linguistic biases in a human-like way. The present study probes this question by asking whether LLMs display human-like referential biases using stimuli and procedures from real psycholinguistic experiments. Recent psycholinguistic studies suggest that humans adapt their referential biases with recent exposure to referential patterns; closely replicating three relevant psycholinguistic experiments from Johnson & Arnold (2022) in an in-context learning (ICL) framework, we found that InstructGPT adapts its pronominal interpretations in response to the frequency of referential patterns in the local discourse, though in a limited fashion: adaptation was only observed relative to syntactic but not semantic biases. By contrast, FLAN-UL2 fails to generate meaningful patterns. Our results provide further evidence that contemporary LLMs discourse representations are sensitive to syntactic patterns in the local context but less so to semantic patterns. Our data and code are available at https://github.com/zkx06111/llm_priming .

## introduction while neural network models, and particularly pretrained large language models have shown excellent performance at particular language processing tasks, many questions remain about the extent to which models optimized for such performance encode, as a side effect, human-like linguistic knowledge and cognitive biases. we know that they do to some extent; existing work has shown, for example, that neural models encode aspects of human-like long-distance number agreement @xcite , incremental syntactic state @xcite , and syntactic generalization more broadly @xcite . in this paper, we examine whether flan-ul2 @xcite and instructgpt @xcite , two representative llms, display adaptation in pronoun interpretation when exposed to consistent referential patterns in the local discourse context. compared with syntactic or lexical knowledge, representing referential knowledge is possibly more complex; we know from psycholinguistic studies that human referential interpretation integrates multiple levels of linguistic structure. humans do not interpret ambiguous pronouns at random but are guided by both syntactic and semantic information. it is well-established that absent other cues humans prefer a syntactic subject in choosing the antecedent of the ambiguous pronoun, i.e., subject bias @xcite . in example (1), she is more likely to be interpreted as the subject ada than the non-subject eva ¶ , even though both referents are possible antecedents for the pronoun. ada 1 received a letter from eva 2 . she 1 ... in sum, people exhibit sensitivity to both syntax (subject bias) and semantics (goal bias) during pronoun interpretation. importantly, these levels of linguistic structure are frequently entwined since both can influence referential interpretation. at times these influences may push in different directions: for instance, ada in [3] is both the syntactic subject and the semantic source. building on a long tradition investigating preferences in pronoun interpretation, recent psycholinguistic studies have probed into the deeper question of the origin of these biases. one hypothesis is that referential biases come from linguistic experience: when a bias occurs very frequently, people will tend to adapt this more frequent referential pattern, both in the immediate exposure as well as more large-scale past experience @xcite . recent evidence has provided support to this idea by demonstrating that recent exposure to certain referential patterns did change people's referential biases. in a series of psycholinguistic studies, johnson and @xcite show that after reading numerous stories that consistently show a particular referential bias, e.g., always referring to the non-subject or source referent, people did have a stronger preference for these primed referents. given this line of psycholinguistic research, the current study investigates the extent to which llms adapt and vary referential biases in pronoun interpretation through exposure to referential patterns in the local context. to do so we replicated actual psycholinguistic experiments from johnson and @xcite in llms using an icl framework and asked whether the responses of llms display adaptation from exposure to referential patterns like human experimental participants. comparing syntactically-motivated to semantically-motivated exposure conditions will allow us to first examine whether llms display human-like subject bias and goal bias and further understand the extent to which llms make use of local frequency changes in discourse representations in these categories. in-context learning refers to llms' ability to learn from demonstrations written in natural language prompts. compared to previous work that has largely examined the encoding of such discourse knowledge using zero-shot inference (upadhye et al., 2020), icl is particularly suitable for experimental simulation since it replicates the naturalistic context in which later responses draw upon exposure to previous examples. foreshadowing our results, we find that instruct-gpt can adapt and thus vary its syntactic bias from exposure to referential patterns in the local context, but the same is not true for semantic bias. given that instructgpt still exhibits a goal bias in spite of local discourse priming, we argue this suggests llms only encode partial semantic knowledge in referential processing. to sum up, our contributions can be summarized as follows: ## related work a growing body of literature has suggested that llms encode rich representations of linguistic structure at various levels, including aspects of syntax, semantics, and reference encoded throughout their representations @xcite . one of the most well-documented lines of this work demonstrates that these models can acquire diverse elements of syntactic knowledge @xcite @xcite . this capacity for encoding linguistic understanding extends to priming effects with psycholinguistic analogies to humans. at the syntactic level, @xcite explored structural priming in various autoregressive llms and found priming effects despite a clear dependence on semantic information. at the semantic level, using english lexical stimuli in bert @xcite , @xcite found that bert does display evidence of sensitivity to semantic priming, though this is localized to more unconstrained contexts and only certain semantic relations. in analyses of llms' linguistic understanding modeled on psycholinguistic experiments, however, the question of discourse knowledge remains relatively under-explored. recent existing work has presented partially conflicting accounts in this area, in particular with regard to how llms may or may not exhibit human-like biases in pronoun interpretation. for example, davis and van schijndel (2020) compared lstm lms and transformer lms behaviors and internal representations in dealing with implicit causality verbs, finding that surprisingly (contrary to humans) implicit causality only influences transformer lms' behavior for reference, but influences neither model for syntactic attachment. @xcite al. (2020) also compared lstm lms and transformer lms in coreference resolution corpora, finding that although lms are much better at grammar, they also captured referential aspects to some extent. this existing work replicates sets of individual stimuli from psycholinguistic experiments in isolation; by contrast, our work takes a more behaviorally-oriented approach by replicating the stimuli, procedure, and even experimental design of a full set of human psycholinguistic studies. we do this to ask a further question: beyond exhibiting baseline human-like referential biases in pronoun interpretation, do llms display adaptation to the frequency of referential patterns in the local context like humans? this question is important because adapting to a referential bias over the course of an experiment requires a sustained representation of the frequency of the pattern, which involves a higher-level understanding of the discourse structure. humans have shown abilities of this kind of adaptation at multiple linguistic levels. for instance, exposure to syntactic structures consecutively affects humans' choice of structures in both the short term and long term via priming effects (e.g., @xcite . exposure to phonological patterns can also guide humans' segmentation patterns (e.g., @xcite . at the semantic and pragmatic levels, humans can also adapt as listeners to speakers' variable choices of uncertainty expressions @xcite . ## methods in this work, we aim to replicate three experiments from johnson and arnold (2022), transferring their designs, procedures, and stimuli as faithfully as possible to the llm context using in-context learning. ## discussion the present study examined whether and to what extent llms display adaptation for pronominal interpretation after exposure to referential patterns by replicating three psycholinguistic experiments. exposed to the same stimuli and study design, and analyzed with the same statistical procedures as the source experiments, we asked whether llms show human-like behaviors and compared the performance of llms-simulated participants with human participants. we firstly found a difference between the capacity of contemporary models to replicate human psycholinguistic experimental designs in an icl framework. while instructgpt was able to correctly answer all the presented comprehension-check content questions, flan-ul2 was not. both models displayed at least some bias towards answering 'yes' in ambiguous cases, but in the case of flan-ul2 this bias resulted in 100% 'yes' answers, rendering meaningful variation impossible to ascertain. this could be a result of structural differences between the models (decoder-only in the case of instructgpt, encoder-decoder in the case of flan-ul2), or perhaps more likely a question of simple model size (175b for instructgpt vs. 20b for flan-ul2). experiments 1a and 2a examined whether llms adapt their syntactic bias from recent exposure to referential patterns like humans, without and with the presence of possibly confounding semantic goal bias. we found that llm's referential biases are indeed sensitive to such exposure in both experiments. in addition, llm did exhibit a goal bias in experiment 2a, in line with previous studies which argue for llm's ability to exhibit human-like semantic bias in pronoun interpretation (e.g., @xcite . experiment 2b examined whether llm can adapt and vary their semantic bias from exposures. in this context, in contrast with the previous two experiments, exposure type did not affect llm behavior at all. this raises the question of why llm would be sensitive to exposure to only referential patterns that exhibit consistent syntactic bias but not semantic bias. an immediate and intuitive answer would be that llm is unable to fully represent semantic knowledge in referential processing. however, given that llm did display a human-like goal bias in pronoun interpretation independent of exposure, this explanation seems unlikely. we suggest that our work provides evidence that llm only partially represents the semantic knowledge involved in referential processing for two reasons. first, adapting referential biases on the basis of exposure in the local context may require representations of higher-level knowledge than merely exhibiting a bias towards certain referents. while the latter may only require knowledge of which features associated with a referent are more frequent or likely in general, adaption requires a sustained awareness of referential pattern frequency as it changes in the local discourse context. though representing knowledge about semantic relations was observed as early as the analogical reasoning task in word2vec @xcite , humanlike extraction of abstract information like a persistent discourse state is more challenging. the model may only be able to identify thematic roles (source or goal) of a referent and associate them with pronoun interpretation, but not to identify a consistent pattern of thematic roles across a discourse. if so, this would explain the strong goal bias we observed in experiment 2. second, it is possible that llm has an extremely strong goal bias that masks the influence of exposures. if so, this suggests that llm over-represents the semantic knowledge encoded in pronoun interpretation. in either case, our results suggest that llm do not encode semantic knowledge in a fully human-like way, even though they do demonstrate some human-like capacities for semantic understanding. although we believe this gap can be mitigated via instruction fine-tuning or chain-ofthought prompting @xcite , these results still suggest we should consider incorporating semantically-informed objectives into selfsupervised pre-training to a greater extent. ## conclusions and future work by replicating a series of psycholinguistic experiments as closely as possible using in-context learning, this paper pioneered whether llms would adapt pronominal interpretation behaviors in a human-like way given exposure to referential patterns in the local discourse context. our work suggests paths forward for replicating psycholinguistic experiments in a more faithful way that allows for comparisons between human and llms' behaviors.