Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference (2021)

Abstract: Performing event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full n^2 pairwise comparisons. Existing approaches simplify by considering coreference only within document clusters, but this fails to handle inter-cluster coreference, common in many applications. As a result cross-document coreference algorithms are rarely applied to downstream tasks. We draw on an insight from discourse coherence theory: potential coreferences are constrained by the reader’s discourse focus. We model the entities/events in a reader’s focus as a neighborhood within a learned latent embedding space which minimizes the distance between mentions and the centroids of their gold coreference clusters. We then use these neighborhoods to sample only hard negatives to train a fine-grained classifier on mention pairs and their local discourse features. Our approach achieves state-of-the-art results for both events and entities on the ECB+, Gun Violence, Football Coreference, and Cross-Domain Cross-Document Coreference corpora. Furthermore, training on multiple corpora improves average performance across all datasets by 17.2 F1 points, leading to a robust coreference resolution model that is now feasible to apply to downstream tasks.

## introduction cross-document coreference resolution of entities and events (cdcr) is an increasingly important problem, as downstream tasks that benefit from coreference annotations -such as question answering, information extraction, and summarization -begin interpreting multiple documents simultaneously. yet the number of candidate mentions across documents makes evaluating the full 1 code is available at https://github.com/ helw150/event_entity_coref_ecb_plus n 2 pairwise comparisons intractable @xcite . for single-document coreference, the search space is pruned with simple recency-based heuristics, but there is no natural corollary to recency with multiple documents. most cdcr systems thus instead cluster the documents and perform the full n 2 comparisons only within each cluster, disregarding inter-cluster coreference @xcite @xcite @xcite @xcite . this was effective for the ecb+ dataset, on which most cdcr methods have been evaluated, because ecb+ has lexically distinct topics with almost no inter-cluster coreference. such document clustering, however, keeps cdcr systems from being generally applicable. @xcite shows that inter-cluster coreference makes up the majority of coreference in many applications. @xcite note that document clustering methods are also unlikely to generalize well to real data where documents lack the significant lexical differences of ecb+ topics. these issues present a major barrier for the general applicability of cdcr. human readers, by contrast, are able to perform coreference resolution with minimal pairwise comparisons. how do they do it? discourse coherence theory @xcite @xcite proposes a simple mechanism: a reader focuses on only a small set of entities/events from their full knowledge. this set, the attentional state, is constructed as entities/events are brought into focus either explicitly by reference or implicitly by their similarity to what has been referenced. since attentional state is inherently dynamic -entities/events come into and out of focus as discourse progresses -a document level approach is a poor model of this mechanism. we propose modeling focus at the mention level using the two stage approach illustrated in figure 1 : a high level overview of our system: for a particular mention, candidate coreferring mentions are retrieved from a neighborhood surrounding the mention. these candidate pairs are fed to a pairwise classifier specialized for hard negatives fetched from this space. this allows our method to create a high fidelity coreference graph with minimal pairwise comparison and no a priori assumptions about coreference. we use a bi-encoder for candidate retrieval and a cross-encoder for pairwise classification @xcite . these attentional state neighborhoods aggressively constrain the search space for our second stage pairwise classifier. this classifier utilizes cross-attention between mention pairs and their local discourse features to capture the features important within an attentional state which are comparison specific @xcite . by sampling from attentional state neighborhoods at training time, we train on only hard negatives such as shown in following the recommendations of @xcite , we evaluate our method on multiple event and entity cdcr corpora, as well as on crosscorpus transfer for event cdcr. our method achieves state-of-the-art results on the ecb+ corpus for both events (+0.2 f1) and entities (+0.7 f1), the gun violence corpus (+11.3 f1), the football coreference corpus (+13.3 f1), and the cross-domain cross-document coreference cor-pus (+34.5 f1). we further improve average results by training across all event cdcr corpora, leading to a 17.2 f1 improvement for average performance across all tasks. our robust model makes it feasible to apply cdcr to a wide variety of downstream tasks, without requiring expensive new coreference annotations to enable fine-tuning on each new corpus. (this has been a huge effort for the few tasks that have attempted it like multi-hop qa @xcite and multi-document summarization @xcite .) 2 related work cross-document coreference many cdcr algorithms use hand engineered event features to perform classification. such systems have a low pairwise classification cost and therefore ignore the quadratic scaling and perform no pruning @xcite @xcite . other such systems choose to include document clustering to increase precision, which can be done with very little tradeoff for the ecb+ corpus @xcite . @xcite explore an approach that avoids pairwise classification entirely, instead relying purely on representation learning and clustering within an embedding space. they propose a novel distance based regularization term for their classifier that encourages representations that can other tasks @xcite introduces the concept of a "coarse-to-fine" approach in single document entity coreference resolution. the architecture utilises a bi-linear scoring function to generate a set of likely antecedents, which is then passed through a more expensive classifier which performs higher order inference on antecedent chains. our work extends to multiple documents the idea of using a high recall but low precision pruning function combined with expensive pairwise classification to balance recall, precision, and runtime efficiency. @xcite use a similar architecture to ours to create a highly scalable system for zero-shot entity linking. their method treats entity linking as a ranking problem, using a bi-encoder to retrieve possible entity mentions and then re-ranking the candidate mentions using a cross-encoder. their results confirm that such architectures can deliver state of the art performance while achieving tremendous scale. however, in coreference resolution, mentions can have one, many, or no coreferring mentions which makes treating it as a ranking problem non-trivial and necessitates the novel training and inference processes we propose. ## model our system is trained in multiple stages and evaluated as a single pipeline. first, we train the encoder for the pruning model to define our latent embedding space. then, we use this model to sample training data for a pairwise classifier which performs binary classification for coreference. our complete pipeline retrieves candidate pairs from the attentional state, classifies them using the pairwise classifier, and performs a variant of the agglomerative clustering algorithm proposed by @xcite to form the final clusters, as laid out in figure 2 . ## experiments we perform an empirical study across 3 event and 2 entity english cross-document coreference corpora. ## analysis we analyze the components of our model in isolation to explain the sources of our significant performance gains and bottlenecks which still exist. ## conclusion and future work in this work, we presented a two-step method for resolving cross-document event and entity coreference inspired by discourse coherence theory. we w r p f1 0 84.4 81.4 82.9 1 83.4 86.5 84.9 2 83.1 87.7 85.4 3 87.1 85.3 86. 2 we identify 3 areas of future work: