L egal C ore: A Dataset for Event Coreference Resolution in Legal Documents (2025)

Abstract: Recognizing events and their coreferential mentions in a document is essential for understanding semantic meanings of text. The existing research on event coreference resolution is mostly limited to news articles. In this paper, we present the first dataset for the legal domain, LegalCore, which has been annotated with comprehensive event and event coreference information. The legal contract documents we annotated in this dataset are several times longer than news articles, with an average length of around 25k tokens per document. The annotations show that legal documents have dense event mentions and feature both short-distance and super long-distance coreference links between event mentions. We further benchmark mainstream Large Language Models (LLMs) on this dataset for both event detection and event coreference resolution tasks, and find that this dataset poses significant challenges for state-of-the-art open-source and proprietary LLMs, which perform significantly worse than a supervised baseline. We will publish the dataset as well as the code.

## introduction identifying event mentions and grouping event mentions based on their coreference relations is a key step for text semantic understanding and necessary for further event structure analysis. however, research on event coreference resolution is mostly limited to news articles as the annotated datasets for event coreference resolution are mostly for this domain @xcite @xcite @xcite . there are a few datasets for other more specific domains, such as twitter @xcite , literature @xcite and biomedical texts @xcite , but these datasets often annotate individual event each party hereby… has been debarred under any…affiliates or personnel becomes debarred or proceedings have been initiated against any of them with respect to debarment each party hereby… has been {event_1 debarred} under any…affiliates or personnel {event_2 becomes} {event_3 debarred} or {event_4 proceedings} have been {event_5 initiated} against any of them with respect to {event_6 debarment}... mentions only and lack event coreference information. ## related works event identification there are many existing datasets annotated with event information, some require the models to identify event mentions and classify them into specific event types @xcite @xcite @xcite , while others require the models to extract event mentions of all types @xcite @xcite @xcite , and legalcore falls into the later category. most of the previous works @xcite @xcite @xcite focus on news domains, but a few datasets have been developed for other specific domains, for example twitter @xcite @xcite , literature @xcite , finance @xcite , and biomedical texts @xcite @xcite @xcite , but none of the previous datasets consider the legal domain. to the best of our knowledge, legalcore is the first dataset for the legal domain annotated with event information. event coreference resolution event coreference resolution is a key and challenging nlp task, and many event coreference datasets have been constructed. previous datasets for event coreference are mostly based on news articles @xcite @xcite @xcite . the most recent large dataset, maven-ere @xcite , contains annotations of event coreference relations as well as other event relations such as temporal relations and causal relations, but the annotated wikipedia articles are still mostly news documents. the event coreference relations annotated in legalcore are expected to enable more studies on event coreference resolution for the legal domain and be highly valuable for developing real-world applications for this important domain. ## dataset construction the legalcore dataset contains 100 legal contract documents that were selected from the cuad dataset @xcite , a public dataset used for identifying key clauses in legal contracts. we will publish our annotated dataset. the dataset construction is done in three phases, namely annotating event mentions, identifying local coreference links, and identifying non-local coreference links, as shown in figure 2 . we follow the annotation guidelines of @xcite for performing the first two phases of annotations, we create our own annotation guidelines for annotating non-local coreference links, and we will publish our data annotation guidelines. ## conclusion and future work we have presented the first event dataset for the legal domain, legalcore, which has been annotated with comprehensive event and event coreference information. we further benchmark mainstream large language models (llms) on this dataset for both event identification and event coreference resolution tasks, and find that this dataset poses significant challenges for both open-source and proprietary llms. for future work, we will extend the dataset to cover other event relations in legal documents, such as temporal relations and causal relations.