Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution (2021)

Abstract: We study the problem of event coreference resolution (ECR) that seeks to group coreferent event mentions into the same clusters. Deep learning methods have recently been applied for this task to deliver state-of-the-art performance. However, existing deep learning models for ECR are limited in that they cannot exploit important interactions between relevant objects for ECR, e.g., context words and entity mentions, to support the encoding of document-level context. In addition, consistency constraints between golden and predicted clusters of event mentions have not been considered to improve representation learning in prior deep learning models for ECR. This work addresses such limitations by introducing a novel deep learning model for ECR. At the core of our model are document structures to explicitly capture relevant objects for ECR. Our document structures introduce diverse knowledge sources (discourse, syntax, semantics) to compute edges/interactions between structure nodes for document-level representation learning. We also present novel regularization techniques based on consistencies of golden and predicted clusters for event mentions in documents. Extensive experiments show that our model achieve state-of-the-art performance on two benchmark datasets.

## introduction event coreference resolution (ecr) is the task of clustering event mentions (i.e., trigger words that evoke an event) in a document such that each cluster represents a unique real world event. for example, the three event mentions in figure 1 , i.e., "refuse to sign, "raised objections", and "doesn't sign", should be grouped into the same cluster to indicate their coreference to the same event. a common component in prior ecr models involves a binary classifier that receives a pair of event mentions and predict their coreference @xcite @xcite . to this end, an important step in ecr models is to transform event mention pairs into representation vectors to encode discriminative features for coreference prediction. early work on ecr has achieved feature representation via feature engineering where multiple features are hand-designed for input event mention pairs @xcite . a major problem with feature engineering is the sparsity of the features that limits the generalization to unseen data. representation learning in deep learning models has recently been introduced to address this issue, leading to more robust methods with better performance for ecr @xcite @xcite . as such, there are at least two limitations in existing deep learning models for ecr that will be addressed in this work to improve the performance. first, as event mentions pairs for coreference prediction might belong to long-distance sentences in documents, capturing document-level context between the event mentions (i.e., beyond the two sentences that host the event mentions) might present useful information for ecr. as their first limitation, prior deep learning models for ecr has only attempted to encode document-level context via hand-designed features @xcite that still suffer from the feature sparsity issue. in addition, such prior work is unable to exploit ecr-related objects in documents (e.g., entity mentions, context words) and their connections/interactions (possibly beyond sentence boundary) to aid representation learning. an example for the importance of context words, entity mentions, and their interactions for ecr can be seen in figure 1 . here, to decisively determine the coreference of "raised objections" and "doesn't sign", ecr systems should recognize "trump" and "the donald trump continued to refuse to sign a relief package agreed in congress and headed instead to the golf course…. trump, who is spending the christmas and new year holiday at his mar-a-lago resort in florida, raised objections to the $900bn relief bill only after it was passed by congress last week, having been negotiated by his own treasury secretary steven mnuchin… all these folks and their families will suffer if trump doesn't sign the damn bill. ## related work event coreference resolution is broadly related to works on entity coreference resolution that aim to resolve nouns phrases/mentions for entities @xcite @xcite @xcite . however, resolving event mentions has been considered as a more challenging task than entity coreference resolution due to the more complex structures of event mentions @xcite . our work focuses on the within-document setting for ecr where input event mentions are expected to appear in the same input documents; however, we also note prior works on crossdocument ecr @xcite @xcite @xcite . as such, for within-document ecr, previous methods have applied feature-based models for pairwise classifiers @xcite @xcite , spectral graph clustering @xcite , information propagation @xcite , markov logic networks @xcite , joint modeling of ecr with event detection @xcite @xcite , and recent deep learning models @xcite @xcite @xcite . compared to previous deep learning works for ecr, our model presents a novel representation learning framework based on document structures to explicitly encode important interactions between relevant objects, and representation regularization to exploit the cluster consistency between golden and predicted clusters for event mentions. ## model formally, in ecr, given an input document @xmath0, w 2 , . . . , w n (of n words/tokens) with a set of event mentions @xmath1, e 2 , . . . , e |e| }, the goal is to group the event mentions in e into clusters to capture the coreference relation between mentions. our ecr model consists of four major components: (i) document encoder to words into representation vectors, (ii) document structure to create graphs for documents and learn rich representation vectors for event mentions, (iii) end-to-end resolution to simultaneously resolve the coreference for the entity mentions in d, and (iv) cluster consistency regularization to regularize representation vectors based on consistency constraints between golden and predict event mention clusters. figure 2 presents an overview of our model for ecr. ## conclusion we present a novel end-to-end coreference resolution framework for event mentions based on deep learning. the novelty in our model is twofold. first, document structures are introduced to explicitly capture relevant objects and their interactions in documents to aid representation learning. second, several regularization techniques are proposed to exploit the consistencies between human-provided and machine-generated clusters of event mentions in documents. we perform extensive experiments on two benchmark datasets for ecr to demonstrate the advantages of the proposed model. in the future, we plan to extend our models to related problems in information extraction, e.g., event extraction.