Analysing Reference Production of Large Language Models (2025)

Abstract: This study investigates how large language models (LLMs) produce referring expressions (REs) and to what extent their behaviour aligns with human patterns. We evaluate LLM performance in two settings: slot filling, %KvD the conventional task of referring expression generation, where REs are generated within a fixed context, and language generation, where REs are analysed within fully generated texts. Using the WebNLG corpus, we assess how well LLMs capture human variation in reference production and analyse their behaviour by examining the influence of several factors known to affect human reference production, including referential form, syntactic position, recency, and discourse status. Our findings show that (1) task framing significantly affects LLMs’ reference production; (2) while LLMs are sensitive to some of these factors, their referential behaviour consistently diverges from human use; and (3) larger model size does not necessarily yield more human-like variation. These results underscore key limitations in current LLMs’ ability to replicate human referential choices.

## introduction referring expression generation (reg) is a fundamental sub-task of natural language generation (nlg, @xcite @xcite . at a given point in a discourse, reg generates a referring expression (re) for an intended referent based on the surrounding context @xcite . reg has significant practical value in commercial nlg applications @xcite and remains an active area of research in theoretical linguistics and psycholinguistics @xcite . amid the recent surge of interest in large language models (llms), several corpus-based studies have evaluated llms on the task of reg @xcite . these * corresponding author studies have shown that llms differ significantly from humans, particularly in their inability to capture the variability of human reference use. in this study, we propose a different corpusbased evaluation grounded in the observation that the standard reg task lacks ecological validity: in real-world communication, humans do not first construct surrounding contexts and then produce referring expressions (res) accordingly. they formulate res progressively while producing the text. in this experiment, we want to replicate this behaviour. concretely, building on webnlg @xcite , a widely used nlg dataset, we asked llms to first perform end-to-end nlg and then annotate the res in the texts they generated. we subsequently compared the variation in reference use between the llm-generated texts and the original human-authored corpus. we hope this corpus-based evaluation will shed light not only on how llms' reference use diverges from human language production but also on how task design may influence the referential behaviour of llms. to further investigate the source of these differences, we examine whether factors known to influence human reference production similarly affect llms. specifically, we focus on three factors grounded in linguistic theories of reference: referential status @xcite , syntactic position @xcite , and recency @xcite . we annotated linguistic information related to these factors in both the llm-generated texts and the human-authored webnlg corpus, and compared their respective impacts. ## background in this section, we first provide a brief overview of the reg task and models, followed by an introduction to the webnlg dataset. ## research questions and hypotheses focusing on our main research question how do llms use res differently from human beings, the current study has the following three research questions and hypotheses. first, we refer to the reg task that requires models to generate re given a fixed context as slot filling reg (sf-reg). in contrast, we define a variant of this task in which models perform full language generation (and res are subsequently annotated for analysis) as language generation reg (lg-reg). intuitively, lg-reg is more ecologically valid than sf-reg, as it more closely reflects how humans use language in natural contexts. in this study, we are particularly interested in how well llms capture the variation in human reference use. prior work on the coherence of llm-generated texts @xcite has shown that llms tend to overuse proper names in contexts where pronouns would be more natural, suggesting a lack of human-like referential variation. based on this, we hypothesise that llms fail to reproduce human-like variation in both sf-reg and lg-reg settings. however, we expect their performance on lg-reg to be closer to humans due to the greater ecological validity of the task. we denote this as hypothesis h 1 . furthermore, we expect that the task setting itself significantly affects llms' reference use. that is, the variation in res produced by llms in sf-reg will differ significantly from that in lg-reg. we refer to this as hypothesis h 2 . second, we investigate whether the factors known to influence human reference production affect llms in the same ways. specifically, we examine three well-established factors: referential status, syntactic position, and recency. according to linguistic theories of reference, using pronouns is more likely the shorter the distance between a referent and its antecedent (recency; @xcite ), when the referent is in subject rather than object position (syntactic position; @xcite ), and when it follows a prior mention rather than being introduced for the first time (referential status; @xcite ). we hypothesise that all three would significantly influence llms' reference production (hypothesis h 3 ). however, to account for the observed failure of llms to fully capture human-like variation, we further hypothesise that the effects of these factors on llms would differ significantly from their effects on humanproduced references (hypothesis h 4 ). finally, we are curious about the impact of model size on reference production. according to the scaling law @xcite , llms with more parameters tend to exhibit greater capabilities. we therefore hypothesise that larger llms would more effectively capture the variation in human reference use (hypothesis h 5 ). ## method to test the hypotheses in section 3, we prompted llms to perform both sf-reg and lg-reg on the test set of webnlg, which is a dataset of nlg from resource description framework (rdf) triples (see section 2.2 for more information and examples). unlike previous reg studies using webnlg (e.g., castro ferreira et al. ( 2018a )), we focus exclusively on res for the protagonist in each discourse, since we are more interested in the use of res within a referential chain rather than in single referential mentions. this decision is motivated by the structure of webnlg: since the dataset is built on dbpedia, most res refer to a single main referent. in contrast, res for other referents often occur only once within a discourse and would therefore introduce noise when analysing variation in reference use. for lg-reg, we instructed llms to perform rdf-to-text generation using the prompt shown in figure 4 in appendix a. the example outputs of each model are shown in for sf-reg, although castro ferreira et al. (2018a) provides gold-standard annotations of protagonist res in webnlg, we used our protagonist re annotator to mark these res as well, ensuring a fair comparison across tasks. after identifying the res, we removed them from the original texts and prompted the llms to regenerate them using the instruction shown in figure 6 in appendix a. ## results we first introduce the settings of our experiment and report on our testing of the hypotheses from section 3. ## conclusion this study examined how large language models produce referring expressions under different task framings and whether their behaviour aligns with human patterns. we found that llms' referential behaviour consistently diverges from human use, both in the overall distribution of referential form choices and in response to factors known to affect human reference production. notably, task framing significantly affects llm reference production, but using a more ecologically valid task framing does not lead to better alignment with human patterns. these findings highlight persistent gaps between human and llm reference production.