Improving Event Coreference Resolution Using Document-level and Topic-level Information (2022)

Abstract: Event coreference resolution (ECR) aims to cluster event mentions that refer to the same real-world events. Deep learning methods have achieved SOTA results on the ECR task. However, due to the encoding length limitation, previous methods either adopt classical pairwise models based on sentence-level context or split each document into multiple chunks and encode them separately. They failed to capture the interactions and contextual cues among those long-distance event mentions. Besides, high-level information, such as event topics, is rarely considered to enhance representation learning for ECR. To address the above two issues, we first apply a Longformer-based encoder to obtain the document-level embeddings and an encoder with a trigger-mask mechanism to learn sentence-level embeddings based on local context. In addition, we propose an event topic generator to infer the latent topic-level representations. Finally, using the above event embeddings, we employ a multiple tensor matching method to capture their interactions at the document, sentence, and topic levels. Experimental results on the KBP 2017 dataset show that our model outperforms the SOTA baselines.

## introduction within-document event coreference resolution (ecr) is the task of grouping the event mentions (i.e., triggers) that occur in a document into clusters such that each cluster represents a unique realworld event. ecr is essential for information aggregation and can help many downstream applications, such as abstractive summarization @xcite , central event detection @xcite , and discourse parsing @xcite . although there studies employ clustering methods @xcite , most work converts ecr into an event-pair classification task @xcite @xcite , i.e., judging whether two event mentions are coreferent. therefore, event encoding becomes an essential step in ecr. earlier work focuses on extracting hand-crafted features @xcite @xcite , while recent studies encode triggers (i.e., the words that most clearly describe the event), contextual linguistic cues @xcite @xcite , and identified arguments (i.e., the participants of a specific event) @xcite using various neural models. recently, benefiting from the development of the transformer-based models, event encoding @xcite @xcite ) has also changed from using cnn/rnn to bert, spanbert, obtaining better event representations. however, at least two limitations exist in the above studies. first, due to the limitation of encoding length, current studies either adopt variant classical pairwise models based on sentence-level context or split a document into chunks and process them separately to learn embeddings based on segmentlevel context. this way, the ecr models actually predict coreferences based only on the local information. obviously, incorporating document-level contextual cues among event mentions is important for ecr, especially for those long-distance event mention pairs, considering their relations are hard to judge only based on local information. to address the above issue, @xcite implement document structures to capture cross-segment interactions; however, the graph construction relies on predicted entities, entity coreference chains, and dependency trees, which inevitably bring the noise to the model. in this paper, we first introduce a longformer-based @xcite encoder to learn representations based on full document-level context. then, we apply a bert-based encoder with a trigger-mask mechanism to mine detailed cues from sentence-level contexts as a beneficial complement. to our knowledge, this is the first to apply the full-document encoding to ecr. second, most existing ecr methods only use representations obtained by text encoder while ignoring high-level information, such as event topics. since events are usually mentioned repeatedly only when elaborating new aspects or further information (choubey and @xcite , it is challenging to identify coreferent events based only on sentence-level or document-level information, especially for those main events @xcite , which advance the content and form long event chains. previous studies (choubey and huang, 2018; lu and ng, 2020) try incorporating event topic information to improve ecr; however, they use integer linear programming or traditional machine learning models. it is difficult to combine these topic models with deep learning methods directly. to this end, we propose a neural event topic model to infer the latent topic distribution for every event mention. in this way, our ecr model can utilize topic-level associations and interactions to predict coreference chains. finally, we feed these different level representations into a scorer to judge coreferences for all event mention pairs by applying multiple tensor matching methods. the experimental results on kbp 2017 dataset demonstrate the benefits of our proposed model. we summarize the contributions of our work as follows. ## related work coreferent events normally have the same type of triggers and entity-coreferent arguments (i.e., event participants). therefore, resolving event mentions has been considered more challenging than entity coreference resolution due to the more complex event structures @xcite . most of the current work for ecr focuses on event mention representation learning @xcite @xcite . our work focuses on the within-document ecr, where input event mentions appear in the same documents; however, we also note previous studies on cross-document ecr @xcite @xcite , which usually require clustering documents first and then resolving coreferent events mentioned in each document cluster. as such, for within-document ecr, previous methods have applied various machine learning models with hand-engineered features @xcite @xcite , spectral graph clustering @xcite , joint inferencing using integer linear programming or markov logic networks @xcite , joint modeling with multiple related tasks @xcite @xcite , and typical pipeline approaches @xcite @xcite . in particular, @xcite introduce gate mechanisms to mitigate error propagation by controlling the information flows from the input symbolic features. in addition, some studies explore the relevance of event coreferences and discourse structures, e.g., topic structures and content types @xcite @xcite . however, due to the limitation of encoding length, current deep learning methods either adopt classical pairwise models @xcite @xcite or split each document into multiple chunks and encode them separately @xcite . these methods learn event mention representations based only on the local context, making ecr models difficult to determine coreferences between long-distance event mention pairs. in particular, to alleviate the encoding separation caused by splitting documents, tran et al. (2021) construct graph-form document structures to capture the interactions between distant sentences. compared with previous deep learning work for ecr, we present a novel representation learning framework based on both document-level and sentence-level context and apply a neural event topic model to further provide topic-level representations. ## model formally, given an input document @xmath0, t 2 , ..., t n } (of n tokens) and a set of event mentions @xmath1, e 2 , ..., e |e| } in d, ecr seeks to group the event mentions in e into clusters. as such, all event mentions in each cluster are coreferent. figure 1 presents an overview of our ecr model, which consists of four main components: (i) global mention encoder (gme) to learn documentlevel event representations, (ii) local mention encoder (lme) to obtain sentence-level event representations, (iii) event topic generator (etg) to infer latent topic-level event representations, and (iv) tensor matching scorer (tms) to predict the probability that two event mentions are coreferent. ## experimentation in this section, we first introduce the experimental settings and then report the experimental results. ## analysis and discussion in this section, we first analyze the impact of document-level, sentence-level and topic-level information, and then discuss the impact of different tensor matching. ## conclusion in this paper, we first apply a longformer-based encoder to obtain the document-level event embeddings based on full-text context and an encoder with a trigger-mask mechanism to learn sentencelevel event embeddings based on local context. in addition, we propose an event topic generator to infer the latent topic-level representations. finally, using the above event embeddings, we employ a multiple tensor matching method to capture their interactions at the document, sentence, and topic levels. experimental results on kbp 2017 dataset show that our proposed model outperforms previous sota methods. in future work, we will continue to study how to mine the connections between event mentions from more aspects.