2*n is better than n^2 : Decomposing Event Coreference Resolution into Two Tractable Problems (2023)

Abstract: Event Coreference Resolution (ECR) is the task of linking mentions of the same event either within or across documents. Most mention pairs are not coreferent, yet many that are coreferent can be identified through simple techniques such as lemma matching of the event triggers or the sentences in which they appear. Existing methods for training coreference systems sample from a largely skewed distribution, making it difficult for the algorithm to learn coreference beyond surface matching. Additionally, these methods are intractable because of the quadratic operations needed. To address these challenges, we break the problem of ECR into two parts: a) a heuristic to efficiently filter out a large number of non-coreferent pairs, and b) a training approach on a balanced set of coreferent and non-coreferent mention pairs. By following this approach, we show that we get comparable results to the state of the art on two popular ECR datasets while significantly reducing compute requirements. We also analyze the mention pairs that are “hard” to accurately classify as coreferent or non-coreferentcode repo: github.com/ahmeshaf/lemma_ce_coref .

## introduction event coreference resolution (ecr) is the task of finding mentions of the same event within the same document (known as "within-document coreference resolution," or wdcr) or across text (known as "cross-document coreference resolution," or cdcr) documents. this task is used for knowledge graph construction, event salience detection and question answering @xcite . traditionally, ecr is performed on pairs of event mentions by calculating the similarity between them and subsequently using a clustering algorithm to identify ecr relations through transitivity. the pairwise similarity is estimated using a supervised machine learning method, where an 1 code repo: github.com/ahmeshaf/lemma_ce_coref algorithm is trained to distinguish between positive and negative examples based on ground truth. the positive examples are all pairs of coreferent mentions, while the negative examples are all pairs of non-coreferent mentions. to avoid comparing completely unrelated events, the negative pairs are only selected from documents coming from the set of related topics. many coreferent pairs are similar on the surface, meaning that the event triggers (the words or phrases referring to the event) have the same lemma and appear in similar sentences. we can use these features in a heuristic to further classify the positive (p + ) and negative (p -) pairs into four categories: ## related work pre-transformer methods pre-transformer language model-related works in event coreference such as @xcite trained neural models with customized objective (loss) functions to generate richer representations of mentionpairs using "static" embeddings such as contextual word2vec @xcite as well as document-level features such as tf-idf and heuristically-motivated features like mentionrecency, word overlap, and lemma overlap, etc. as such, they improved upon the baselines established by @xcite on the ecb+ corpus. similarly, works such as @xcite suggest both disjoint and joint-clustering of events mentions with their related entity clusters by using a predicate-argument structure. in this, their disjoint model surpassed @xcite by 9.5 f1 points using the conll scorer @xcite whereas their joint model improved upon the disjoint model by 1.2 points for entities and 1 point for events. transformer-based cross-encoding most recent works @xcite @xcite @xcite @xcite in cdcr have shown success in using pairwise mention representation learning models, a method popularly known as cross-encoding. these methods use distributed and contextually-enriched "nonstatic" vector representations of mentions from large, transformer-based language models like various bert-variants to calculate supervised pairwise scores for those event mentions. at inference, such works use variations of incremental or agglomerative clustering techniques to form predicted coreference links and evaluate their chains on gold coreference standards. the methods vary with the context they use for cross-encoding. @xcite al. (2021) in our research, we have focused on the cdlm model from @xcite and their methodology, which uses a combination of enhanced pretraining using the global attention mechanism inspired by @xcite as well as finetuning on a task-specific dataset using pretrained special tokens to generate more semantically-enhanced embeddings for mentions. beltagy et al. (2020) and caciularu et al. (2021) cleverly use the global attention mechanism to linearly scale the oft-quadratic complexity of pairwise scoring of mentions in coreference resolution while also accommodating longer documents (up to 4,096 tokens). previous works such as @xcite , @xcite , @xcite , and @xcite also reduce computation time by strategically using deterministic, rule-based systems along with neural architectures. recently, pruning p - tn for ecr has been shown to be effective by @xcite . they create individual representations for mentions and use them in a bi-encoder method to retrieve potential coreferent candidates, which are later refined using a cross-encoder trained on hard negative examples. in contrast, our approach utilizes a computationally efficient pruning heuristic and trains the crossencoder on a smaller dataset. we also conduct an error analysis on all hard examples that are misclassified by the cross-encoder, which is made feasible by the heuristic. ## datasets we experiment with two popular ecr datasets distinguished by the effectiveness of a lemma heuristic on the dataset. ## system overview there are two major components in our system: the heuristic and the discriminator (cross-encoder) trained on the output of the heuristic. ## experimental setup we describe our process of training, prediction, and hyperparameter choice in this section. ## results we evaluate the event clusters formed using the standard coreference evaluation metrics (muc, b run the baseline results (lh and lh ora ) and the combination of each heuristic with the two discriminators (lh/lh ora + d small /d long ). we compare to previous methods for ecb+ and gvc as shown in the discriminators consistently make significant improvements over the heuristics across both datasets. for ecb+, d long is nearly 2 points better than d small in terms of the conll measure. both d small and d long when coupled with lh ora surpass the state of the art for this dataset. lh +d long beats @xcite but falls short of sota, albeit by only 4 points. on gvc, both fall short of sota @xcite by only 8-9 points on conll f1, with substantially fewer computations. in terms of computational cost-to-performance ratio, as we elaborate in §7.1, our methods outperform all the previous methods. for ecr, where context is key, we would expect better performance from encoders with longer context. d long and d small show this trend for both ecb+ and gvc datasets. however, the gain we get from using the entire document is not substantial for the amount of additional computation required. an interesting line of future work would to automatically detect the core sections in the document that contribute to coreference and then only use that as context for ecr. ## conclusion & future work we showed that a simple heuristic paired with a crossencoder does comparable ecr to more complicated methods while being computationally efficient. we set a upper bound for the performance on ecb+ suggesting improvement with better synonyms pairs detection we can achieve better results. through extensive error analysis, we presented the shortcomings of the crossencoder in this task and suggested ways to improve it. future research directions include applying our method to the more challenging task of crosssubtopic event coreference (e.g., fcc @xcite ) where scalability and compute-efficiency are crucial metrics, making the current heuristicbased mention pair generation process "learnable" using an auxiliary cross-encoder, and incorporating word-sense disambiguation and lemma-pair annotations into the pipeline to resolve lexical ambiguity. an exciting direction for future work made tractable by our work is to incorporate additional cross-encoding features into the pipeline, especially using the latest advancements in visual transformers @xcite @xcite . another important direction is to test our method on languages with a richer morphology than english.