Sequential Cross-Document Coreference Resolution (2021)

Abstract: Relating entities and events in text is a key component of natural language understanding. Cross-document coreference resolution, in particular, is important for the growing interest in multi-document analysis tasks. In this work we propose a new model that extends the efficient sequential prediction paradigm for coreference resolution to cross-document settings and achieves competitive results for both entity and event coreference while providing strong evidence of the efficacy of both sequential models and higher-order inference in cross-document settings. Our model incrementally composes mentions into cluster representations and predicts links between a mention and the already constructed clusters, approximating a higher-order model. In addition, we conduct extensive ablation studies that provide new insights into the importance of various inputs and representation types in coreference.

## introduction relating entities and events in text is a key component of natural language understanding. for example, whether two news articles describing hurricanes are referring to the same hurricane event. a crucial component of answering such questions is reasoning about groups entities and events across multiple documents. the goal of coreference resolution is to compute these clusterings of entities or events from extracted spans of text. while within-document coreference has been studied extensively (e.g., @xcite ), there has been relatively less work on the cross-document task. however, growing interest in multi-document applications, such as summarization (e.g., @xcite ; @xcite ) and reading comprehension (e.g., @xcite ; @xcite ), highlights the importance of developing efficient and accurate cross-document coreference models to minimize error-propagation in complex reasoning tasks. in this work we focus on cross-document coreference (cdcr), which implicitly requires withindocument coreference (wdcr), and propose a new model that improves both coreference performance and computational complexity. recent advances in within-document entity coreference resolution have shown that sequential prediction (i.e., making coreference predictions from left to right in a text foot_0 ) achieves strong performance @xcite with lower computational costs. this paradigm is also well suited to real-world streaming settings, where new documents are received every day, since it can easily find corefering events and entities with already processed documents, while most non-sequential models would require a full rerun. in this work, we show how this technique can first be extended to cross-document entity coreference and then adapted to cross-document event coreference. our method is also able to take advantage of the history of previously made coreference decisions, approximating a higher-order model (i.e., operating on mentions as well as structures with mentions). specifically, for every mention, a coreference decision is made not over a set of individual mentions but rather over the current state of coreference clusters. in this way, the model is able to use knowledge about the mentions currently in a cluster when making its decisions. while higher-order models have achieved state-of-the-art performance on entity coreference @xcite , they have been used infrequently for event coreference. for example, @xcite use one chinese restaurant process for wdcr and then a second for cdcr over the within-document clusters. in contrast, our models make within-and cross-document corefer-ence decisions in a single pass, taking into account all prior coreference decisions at each step. our contributions are: (1) we present the first study of sequential modeling for cross-document entity and event coreference and achieve competitive performance with a large reduction in computation time, (2) we conduct extensive ablation studies both on input information and model features, providing new insights for future models. ## related work prior work on coreference resolution is generally split into either entity or event coreference. entity coreference is relatively well studied @xcite , with the largest focus on within-document coreference (e.g., @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite ). recently, @xcite showed that pre-trained language models, in particular bertlarge @xcite , achieve state-of-theart performance on entity coreference. in contrast to prior work on entity coreference, which is primarily sequential (i.e., left to right) and only within-document, our work extends the sequential paradigm to cross-document coreference and also adds incremental candidate composition. there has been less work on event coreference since the task is generally considered harder. this is largely due to the more complex nature of event mentions (i.e., a trigger and arguments) and their syntactic diversity (e.g., both verb phrases and noun-phrases). prior work on event coreference typically involves pairwise scoring between mentions followed by a standard clustering algorithm to predict coreference links @xcite @xcite @xcite , classification over a fixed number of clusters @xcite and template-based methods @xcite . while pairwise scoring (e.g., graphbased models, see §3.7) with clustering is effective, it requires tuned thresholds (for the clustering algorithm) and cannot use already predicted scores to inform later ones, since all scores are predicted independently. to the best of our knowledge, our work is the first to apply sequential models to crossdocument event coreference. although a few previous works attempt to use information about existing clusters through incremental construction @xcite or argument sharing @xcite , these either continue to rely on pairwise decisions or use shallow, noncontextualized features that have limited efficacy. for example, @xcite explore a variant of cluster merging for wd entity conreference only that still relies on scores between individual mentions. additional recent work on wd coreference investigates incremental construction of clusters for prediction @xcite and cluster ranking @xcite in contrast, our method makes coreference decisions between a mention and all existing coreference clusters across multiple documents using contextualized features and so takes advantage of interdependencies between mentions, even across documents, while making all decisions in one pass. ## conclusion in this paper, we propose a new model for crossdocument coreference resolution that extends the efficient sequential prediction paradigm to multiple documents. the sequential prediction is combined with incremental candidate composition that allows the model to use the history of past coreference decisions at every step. our model achieves competitive results for both entity and event coreference and our analysis provides strong evidence of the efficacy of both sequential models and higher-order inference in cross-document settings. in future, we intend to adapt this model to coreference across document streams and investigate alternatives to greedy prediction (e.g., beam search).