Improving LLM s’ Learning of Coreference Resolution (2025)

Abstract: Coreference Resolution (CR) is crucial for many NLP tasks, but existing LLMs struggle with hallucination and under-performance. In this paper, we investigate the limitations of existing LLM-based approaches to CR—specifically the Question-Answering (QA) Template and Document Template methods—and propose two novel techniques: Reversed Training with Joint Inference and Iterative Document Generation. Our experiments show that Reversed Training improves the QA Template method, while Iterative Document Generation eliminates hallucinations in the generated source text and boosts coreference resolution. Integrating these methods and techniques offers an effective and robust solution to LLM-based coreference resolution

## introduction coreference resolution involves detecting and clustering different mentions that refer to the same discourse world entity. as a task that requires linguistic and extra-linguistic understanding, it plays a crucial role for many downstream natural language processing tasks, such as information extraction, text summarization, chatbots, and dialogue systems. as a result, cr has garnered significant attention from the nlp community @xcite . the evolution of coreference resolution models can be divided from rule-based and statistical learning approaches to deep learning approaches @xcite . in the past few years, large language models (llms), which implicitly incorporate contextual information and commonsense knowledge, have revolutionized nlp by significantly improving performance across many tasks. this advancement has led researchers to investigate llms' potential for coreference resolution. 1 our code is available here. recent studies have already proved the feasibility of prompting llms to resolve coreferences, with a special focus on zero-and few-shot applications @xcite @xcite @xcite . @xcite have shown that prompt-based llms surpass previous unsupervised systems but still perform worse than state-of-the-art supervised models. in their experiments, two prompt templates were used, namely the question-answering (qa) template and the document template, as shown in figure 1 . specifically, @xcite have focused more on the document template, while @xcite have concentrated on the qa template. experimental results in our comparative study show that the performance of the document template is superior to that of the qa template. however, it is found that the document template method relies on a key assumption: llms do not generate hallucinations when resolving coreferences. but llms do produce hallucinations, which is challenging to match the generated document with the correct places in the original text. for example, in figure 1 , if the llm wrongly generates "there are a candle a wall ..." as "there are a candle a candle a wall ..." with an unnecessary mention "a candle" repeated, it becomes much harder to align these mentions with the original document. through studying the code given by @xcite , we have noticed that they removed examples where llms created hallucinations, only keeping cases where the original text matched correctly (i.e., without hallucinations). however, we have found that hallucinations are quite common, especially in linguistically complex texts, where the problem becomes even more noticeable. although the qa template method is less affected by hallucinations, its performance is weaker than the document template method, making it less reliable for coreference resolution. to overcome the limitations of both the qa and document template methods, we propose two new approaches: reversed training with joint inference, and iterative document generation. our experiments show that the iterative document generation method not only completely removes hallucinations of llms but also improves the final conll score. meanwhile, the reversed training with joint inference method significantly improves the llm's cr performance when using the qa template. ablation studies also show that removing any component of this solution reduces the conll score. overall, our work makes two key contributions: 1. we propose the reversed training and joint inference methods for coreference resolution based on the qa template. ablation study shows that these methods effectively improve llm performance. 2. we propose the iterative document generation method for coreference resolution based on the document template. experiments demonstrate that this method not only enhances llm performance, but also effectively tackles the issue of hallucination in llms when using the document template. ## iterative document generation le and ritter (2024)'s document template provides the complete document with marked candidate mentions and designated character positions. figure 3 : an example of the iterative document generation process of the refined document template. for each iteration, the input consists of a text segment containing a marked mention and its designated position at the end. the llms then predict the cluster id for the marked mention. this predicted id and its following content are appended to the end, creating a new prompt for analyzing the next mention. this sequential process continues till all mentions have been analyzed. the llms must input cluster ids for these candidate mentions simultaneously at the designated positions. however, we discovered that processing the entire document and analyzing all mentions at once led to hallucination-where the llm would generate irrelevant words not present in the document instead of completing the intended task. this issue significantly reduced the model's ua-sa score from approximately 89 (when excluding hallucination results from evaluation) to 59 (when including hallucination results). to address this issue, we modified the full document generation process to generate only one mention's cluster id at a time, as shown in figure 3 . the advantage of generating just one cluster id per step is that it significantly reduces the difficulty of content generation for the llm, helping to avoid hallucinations. the process begins with a text seg-ment containing the document's first marked mention and its designated position. the llm assigns a cluster id to this mention. after filling this predicted cluster id into the designated position, a new text segment is created for analyzing the next mention. this sequential analysis continues until all mentions in the document are processed. during training, the cluster ids are filled with correct values. during inference, they are filled with ids generated by the llms. in prompt design, since only a cluster id is predicted, using just the partial sentence from figure 3 may not be enough for accurate prediction. considering the context is crucial for understanding the coreference relationship of the mention. therefore, based on the document template in figure 1 , we removed the full text from the input and instead added the relevant sentence containing the last mention. unlike in figure 1 , this relevant sentence does not include the mention's position annotation. to ensure consistency with prior work, we use a subset of the test set for comparison with @xcite , and the full test set for comparison with @xcite . the four datasets used in this study are shown as follows: ## conclusion in this paper, we have analyzed the limitations of current approaches to using llms for coreference resolution, particularly the qa and document templates, and then proposed two novel methods: reversed training with joint inference, and iterative document generation. our findings can be summarized as follows. first, reversed training enhances the qa template method. second, iterative doc-ument generation eliminates hallucinations with an improvement in task performance. these methods and techniques proposed in our study form an integrated solution to enhance model learning in llm-based coreference resolution.