Two Tractable Problems
Year: 2023

Abstract:
Event Coreference Resolution (ECR) is the task of linking mentions of the same event either within or across documents. Most mention pairs are not coreferent, yet many that are coreferent can be identified through simple techniques such as lemma matching of the event triggers or the sentences in which they appear. Existing methods for training coreference systems sample from a largely skewed distribution, making it difficult for the algorithm to learn coreference beyond surface matching. Additionally, these methods are intractable because of the quadratic operations needed. To address these challenges, we break the problem of ECR into two parts: a) a heuristic to efficiently filter out a large number of non-coreferent pairs, and b) a training approach on a balanced set of coreferent and non-coreferent mention pairs. By following this approach, we show that we get comparable results to the state of the art on two popular ECR datasets while significantly reducing compute requirements. We also analyze the mention pairs that are "hard" to accurately classify as coreferent or non-coreferent1.

heading: 1 Introduction
text: Event coreference resolution (ECR) is the task of finding mentions of the same event within the same document (known as “within-document coreference resolution,” or WDCR) or across text (known as “cross-document coreference resolution,” or CDCR) documents. This task is used for knowledge graph construction, event salience detection and question answering (Postma et al., 2018).
Traditionally, ECR is performed on pairs of event mentions by calculating the similarity between them and subsequently using a clustering algorithm to identify ECR relations through transitivity. The pairwise similarity is estimated using a supervised machine learning method, where an
1code repo: github.com/ahmeshaf/lemma_ce_coref
algorithm is trained to distinguish between positive and negative examples based on ground truth. The positive examples are all pairs of coreferent mentions, while the negative examples are all pairs of non-coreferent mentions. To avoid comparing completely unrelated events, the negative pairs are only selected from documents coming from the set of related topics.
Many coreferent pairs are similar on the surface, meaning that the event triggers (the words or phrases referring to the event) have the same lemma and appear in similar sentences. We can use these features in a heuristic to further classify the positive (P+) and negative (P−) pairs into four categories:
1. P+easy: coreferent/positive mention pairs with high surface similarity.
2. P+FN: coreferent/positive mention pairs with low surface similarity. 3. P−hard: non-coreferent/negative mention pairs with high surface similarity. 4. P−TN: non-coreferent/negative mention pairs with low surface similarity
As shown in Figure 1, P+easy represents coreferent mention pairs that can be correctly identified by the heuristic, but P−hard are non-coreferent pairs that might be difficult for the heuristic to identify. Similarly, P−TN (True Negatives) are non-coreferent pairs that the heuristic can correctly infer, but P+FN (False Negatives) require additional reasoning (that Indianapolis is coreferent with Colts) to make the coreference judgement.
Most mention pairs are non-coreferent, comprising all pairs corresponding to P−hard and P − TN. However, we observe that that the distribution of the three categories (P+easy, P − hard, and P + FN) is fairly similar across most ECR datasets, with P−TN causing the imbalance between positive and negative pairs. Previous methods do not differentiate between these four categories and randomly select
the positive and negative pairs to train their coreference systems from this heavily skewed distribution. This makes it challenging for the coreference algorithm to identify coreferent links among a large number of non-coreferent ones. Furthermore, as ECR is performed on n2 number of mention pairs, where n is the number of mentions in the corpus, these methods can become intractable for a large corpus.
To improve the efficiency of the ECR process while achieving near sate of the art (SOTA) results, we divide the problem into two manageable subtasks: a) a heuristic to efficiently and accurately filter out a large number of P−TN as a way of balancing the skewed distribution, and b) an ECR system trained on the balanced set of coreferent and noncoreferent mention pairs (P+easy and P − hard). This approach also eases the analysis of some of the mention pairs that are difficult to classify with an ECR system, which we present in this paper.

heading: 2 Related Work
text: Pre-Transformer Methods Pre-Transformer language model-related works in event coreference such as Kenyon-Dean et al. (2018) trained neural models with customized objective (loss) functions to generate richer representations of mentionpairs using “static” embeddings such as contextual Word2Vec (Mikolov et al., 2013) as well as document-level features such as TF-IDF and heuristically-motivated features like mentionrecency, word overlap, and lemma overlap, etc. As such, they improved upon the baselines established by Cybulska and Vossen (2015) on the ECB+ corpus. Similarly, works such as Barhom et al. (2019) suggest both disjoint and joint-clustering of events
mentions with their related entity clusters by using a predicate-argument structure. In this, their disjoint model surpassed Kenyon-Dean et al. (2018) by 9.5 F1 points using the CoNLL scorer (Pradhan et al., 2014) whereas their joint model improved upon the disjoint model by 1.2 points for entities and 1 point for events. Transformer-based Cross-encoding Most recent works (Meged et al., 2020; Zeng et al., 2020; Cattan et al., 2021; Allaway et al., 2021; Caciularu et al., 2021; Held et al., 2021; Yu et al., 2022a) in CDCR have shown success in using pairwise mention representation learning models, a method popularly known as cross-encoding. These methods use distributed and contextually-enriched “nonstatic” vector representations of mentions from large, Transformer-based language models like various BERT-variants to calculate supervised pairwise scores for those event mentions. At inference, such works use variations of incremental or agglomerative clustering techniques to form predicted coreference links and evaluate their chains on gold coreference standards. The methods vary with the context they use for cross-encoding. Cattan et al. (2021) use only sentence-level context, Held et al. (2021) use context from sentences surrounding the mentions, and Caciularu et al. (2021) use context from entire documents.
In our research, we have focused on the CDLM model from Caciularu et al. (2021) and their methodology, which uses a combination of enhanced pretraining using the global attention mechanism inspired by Beltagy et al. (2020) as well as finetuning on a task-specific dataset using pretrained special tokens to generate more semantically-enhanced embeddings for mentions.
Beltagy et al. (2020) and Caciularu et al. (2021) cleverly use the global attention mechanism to linearly scale the oft-quadratic complexity of pairwise scoring of mentions in coreference resolution while also accommodating longer documents (up to 4,096 tokens). Previous works such as Baldwin (1997), Stoyanov and Eisner (2012), Lee et al. (2012), and Lee et al. (2013) also reduce computation time by strategically using deterministic, rule-based systems along with neural architectures.
Recently, pruning P−TN for ECR has been shown to be effective by Held et al. (2021). They create individual representations for mentions and use them in a bi-encoder method to retrieve potential coreferent candidates, which are later refined using a cross-encoder trained on hard negative examples. In contrast, our approach utilizes a computationally efficient pruning heuristic and trains the crossencoder on a smaller dataset. We also conduct an error analysis on all hard examples that are misclassified by the cross-encoder, which is made feasible by the heuristic.

heading: 3 Datasets
text: We experiment with two popular ECR datasets distinguished by the effectiveness of a lemma heuristic on the dataset.

heading: 3.1 Event Coreference Bank Plus (ECB+)
text: The ECB+ corpus (Cybulska and Vossen, 2014) is a popular English corpus used to train and evaluate systems for event coreference resolution. It extends the Event Coref Bank corpus (ECB; Bejan and Harabagiu (2010)), with annotations from around 500 additional documents. The corpus includes annotations of text spans that represent events, as well as information about how those events are related through coreference. We divide the documents from topics 1 to 35 into the training and validation sets2, and those from 36 to 45 into the test set, following the approach of Cybulska and Vossen (2015).

heading: 3.2 Gun Violence Corpus (GVC)
text: The Gun Violence Corpus (Vossen et al., 2018) is a recent English corpus exclusively focusing on event coreference resolution. It is intended to be a more challenging dataset than ECB+ which has a very strong lemma baseline (Cybulska and Vossen, 2014). It is a collection of texts surrounding a
2Validation set includes documents from the topics 2, 5, 12, 18, 21, 34, and 35
single topic (gun violence) and various sub-topics. Since it does not have coreference links across subtopics, we only consider mention pairs within the sub-topics. We use the data split by Bugert et al. (2021). Table 1 contains the statistics for ECB+ and GVC corpora.

heading: 4 System Overview
text: There are two major components in our system: the heuristic and the discriminator (cross-encoder) trained on the output of the heuristic.
4.1 Lemma Heuristics (LH, LHOra)
A key feature of ECR is its high baseline achieved by comparing the lemmas of mention triggers and sentences. To leverage this feature, we incorporate it as the first step in our coreference resolution system. We utilize spaCy3 to extract the lemmas, a widely-used tool for this task. In addition to matching lemmas of triggers, we also create and utilize a set of synonymous4 lemma pairs that commonly appear in coreferent mention pairs in our training set. This approach allows us to identify coreferent mention pairs that have different triggers and improve the overall recall. The heuristic, LH, only utilizes the synonymous lemma pairs from the training set. We also evaluate the performance of LHOra, which uses synonymous lemma pairs from the entire dataset which means it uses the coreference information of the development and test sets to create synonymous lemma pairs.
For a mention pair (A, B), with triggers (tA, tB), head lemmas (lA, lB) and for a given synonymous lemma pair set (SynP), we consider only lemma pairs that pass any of the following rules:
• (lA, lB) ∈ SynP • lA == lB • tB contains lA 3https://spacy.io/ model en_core_web_md v3.4 4The words need not be synonyms in strict definitions, but
rather appear in coreference chains.
• tA contains lB
For mentions that have matching trigger lemmas/triggers or are synonymous, we proceed by comparing the context of the mentions. In this work, we only compare the mention’s sentence to check for similarities between two mentions. To further refine our comparison, we remove stop words and convert the tokens in the text to their base form. Then, we determine the overlap between the two mentions and predict that the pair is coreferent if the overlap exceeds a certain threshold. We tune the threshold using the development sets.

heading: 4.1.1 Filtering out P−TN
text: Cross-document coreference systems often struggle with a skewed distribution of mention pairs, as seen in Figure 2. In any dataset, only 5-10% of the pairs are corefering, while the remaining 90% are non-coreferent. To address this, we use the heuristic to balance the distribution by selectively removing non-coreferent pairs (P−TN), while minimizing the loss of coreferent pairs (P+FN). We do this by only considering the mention pairs that the heuristic predicts as coreferent, and discarding the non-coreferent ones.
4.1.2 P−hard, P + easy, and P + FN Analysis P+easy and P − hard: As defined earlier, P + easy are the mention pairs that the heuristic correctly predicts as coreferent when compared to the ground-truth, and P−hard are the heuristic’s predictions of coreference that are incorrect when compared to the ground-truth. In §4.2.1, we go through how we fix heuristic’s P−hard predictions while minimizing the errors introduced in terms of P+easy. P+FN: We define a pair as a P + FN only if it cannot be linked to the true cluster through subsequent steps.
As shown in Figure 3, if a true cluster is {a, b, c} and the heuristic discards one pair (a, c), it will not be considered as a P+FN because the coreference can be inferred through transitivity. However, if it discards two pairs {(a,c), (b,c)}, they will both be considered as P+FN. We hypothesize that an ideal heuristic is one that maintains a balance between P+easy and P − hard while minimizing P + FN, and therefore, we tune the heuristic’s threshold accordingly using the development sets of the corpora.
We evaluate the heuristics LH and LHOra by plotting the distributions P+easy, P − hard, and P + FN generated by each for the two corpora. From Figure 4, We observe similar distributions for the test and development sets with the chosen threshold value from the development set. We also observe that LH causes a significant number of P+FN, while LHOra has a minimal number of P+FN. Minimizing the count of P+FN is important as it directly affects
the system’s recall. The distributions of P+easy and P−hard remain balanced across all datasets except when LHOra is used in GVC where there are double the number of P−hard to P + easy. P − hard should be minimized as it can affect the system’s overall precision.

heading: 4.2 Cross-Encoder
text: A common technique to perform ECR is to use Transformer-based cross-encoding (CE) on the mention pair (A, B). This process, depicted in Figure 5, begins by surrounding the trigger with special tokens (<m> and </m>). The mentions are then combined into a single input for the transformer (e.g., RoBERTa). The pooled output of the transformer (ECLS) and the output corresponding to the tokens of the event triggers (EA and EB) are extracted.5 ECLS, EA, EB, and the element-wise product of the mention embeddings (EA⊙EB) are all concatenated to create a unified representation of the mention pair. This representation is used, with a classifier, to learn the coreference score, CE (A, B), between the pair after finetuning the transformer.
5EA and EB represent the sum of the output embedding of each token for event triggers with multiple tokens.
4.2.1 P+easy & P − hard Discriminator (D) The cross-encoder’s encoding is non-symmetric, meaning, depending on the order in which the mentions are concatenated, it will give different coreference scores. In reality, the order should not matter for predicting if the two events are the same or not. We propose a symmetric cross-encoding scorer where we take the average of the scores predicted from both combinations of concatenation. So for a mention pair, p = (A, B), the symmetric cross-encoder coreference scorer (D) is given as:
D(p) = CE(A,B) + CE(B,A)
2 (1)
We employ a cross-encoder with a symmetric scorer, as outlined in Equation 1, as the discriminator for P+easy and P − hard. We conduct experiments utilizing two different Transformer models, RoBERTa (Dsmall) and Longformer (Dlong), which vary in their maximum input capacity.

heading: 5 Experimental Setup
text: We describe our process of training, prediction, and hyperparameter choice in this section.

heading: 5.1 Mention Pair Generation
text: We use the gold mentions from the datasets. Following previous methods, we generate all the pairs (Pall) of mentions (Mv) from documents coming from the same topic. We use gold topics in the training phase and predicted topics through document clustering in the prediction phase (Bugert et al., 2021).

heading: 5.2 Training Phase
text: During the training phase, we leverage LH to generate a balanced set of positive and negative samples, labeled as P+easy and P − hard, respectively. These samples are then used to train our models, Dsmall and Dlong separately, using the Binary Cross Entropy Loss (BCE) function as follows:
L = ∑
p+∈ P+easy, p−∈ P−hard
log D(p+) + log (1− D(p−))
Unlike traditional methods, we do not rely on random sampling or artificial balancing of the dataset. Instead, our heuristic ensures that the positive and negative samples are naturally balanced (as depicted in Figure 6). A side-effect of adopting this approach is that some of the positive samples are
Algorithm 1 Training Phase Require: D: training document set T : gold topics Mv: gold event mentions in D Sv: sentences of the mentions Dv: documents of the mentions G: gold mention cluster map
P ← TopicMentionPairs(Mv, T ) SynP ← SynonymousLemmaPairs(P,G) P+easy,P − hard,P + FN,P − TN ← LH(P,G, SynP, Sv) Dlong ← TrainCrossEncoder(P+easy, P−hard, Dv) Dsmall ← TrainCrossEncoder(P+easy, P−hard, Sv) return SynP,Dlong,Dsmall
excluded in training. We do this to keep the training and prediction phases consistent and, to ensure the cross-encoder is not confused by the inclusion of these hard positive examples.
Additionally, for D with Longformer, we utilize the entire document for training, while for D with RoBERTa, we only use the sentence containing the mention to provide contextual information. We employ the Adam optimizer with a learning rate of 0.0001 for the classifier and 0.00001 for finetuning the Transformer model. This entire process is illustrated in Algorithm 1.
To ensure optimal performance, we train our system separately for both the ECB+ and GVC training sets. We utilize a single NVIDIA A100 GPU
Algorithm 2 Prediction Phase Require: D: testing document set T : gold/clustered topics Mv: gold event mentions in D Sv: sentences of the mentions SynP: synonymous lemma pairs from training Dsmall,Dlong: trained CE discriminators
P ← TopicMentionPairs(Mv, T ) AH,P+ ← LH(P,SynP, Sv) AP ← Dsmall(P+) > 0.5 AP ← Dlong(P+) > 0.5 return ConnectedComponents(AH),
ConnectedComponents(AP)
with 80GB memory to train Dlong with the Longformer model, and a single NVIDIA RTX 3090 GPU (24 GB) for training Dsmall with the RoBERTaBASE model. We train each system for 10 epochs, with each epoch taking approximately one hour for the Longformer model and 15 minutes for the RoBERTa model.

heading: 5.3 Prediction Phase
text: In the prediction phase, we first pass the mention pairs through the heuristic and create an adjacency matrix called AH based on its coreferent predictions. The ones predicted not coreferent by the heuristic are discarded. This step is crucial in terms of making the task tractable. Next, we pass the mention pairs that are predicted to be coreferent by the heuristic through Dsmall and Dlong separately. Using the subsequent coreferent predictions from these models, we generate another adjacency matrix AP. To create event clusters, we use these matrices to identify connected components.
As a baseline, we use the matrix AH to generate the clusters. We then use AP to assess the improvements made by using Dsmall and Dlong over the baseline. This process is illustrated in Algorithm 2. The process takes between 6-10 minutes to run the Longformer model and between 1-2 minutes to run the RoBERTa one.

heading: 6 Results
text: We evaluate the event clusters formed using the standard coreference evaluation metrics (MUC, B3, CEAFe, LEA and CoNLL F1—the average of MUC, B3 and CEAFe Vilain et al. (1995); Bagga and Baldwin (1998); Luo (2005); Luo et al. (2014); Pradhan et al. (2014); Moosavi et al. (2019)). We
run the baseline results (LH and LHOra) and the combination of each heuristic with the two discriminators (LH/LHOra+ Dsmall/Dlong). We compare to previous methods for ECB+ and GVC as shown in Table 2. Bold indicates current or previous SOTA and our best model.
CoNLL F1 scores show that LH and LHOra are strong baselines for the ECB+ corpus, where LHOra surpasses some of the previous best methods. From this, we can say that making improvements in the heuristic by better methods of finding synonymous lemma pairs is a viable solution for tackling ECB+ with a heuristic. However, the heuristics fall short for GVC, where LHOra is only marginally better than LH. This may be due to the lower variation in lemmas in the GVC corpus. We hypothesize methods that can automatically detect synonymous lemma pairs will not be beneficial for GVC, and LH itself is sufficient as a heuristic here.
The discriminators consistently make significant improvements over the heuristics across both datasets. For ECB+, Dlong is nearly 2 points better than Dsmall in terms of the CoNLL measure. Both Dsmall and Dlong when coupled with LHOra surpass the state of the art for this dataset. LH +Dlong beats Cattan et al. (2021) but falls short of SOTA, albeit by only 4 points. On GVC, both fall short of SOTA (Held et al., 2021) by only 8-9 points on CoNLL F1, with substantially fewer computations. In terms of computational cost-to-performance ratio, as we elaborate in §7.1, our methods outperform all the previous methods.
For ECR, where context is key, we would expect better performance from encoders with longer context. Dlong and Dsmall show this trend for both
ECB+ and GVC datasets. However, the gain we get from using the entire document is not substantial for the amount of additional computation required. An interesting line of future work would to automatically detect the core sections in the document that contribute to coreference and then only use that as context for ECR.

heading: 7 Discussion

heading: 7.1 Time Complexity Analysis
text: The heuristic is a very fast process that scales linearly with the number of mentions in a corpus. Specifically, by hashing the lemma pairs and sentence token lemmas, this step performs linear comparisons of mention pairs at prediction. The mention pair cross-encoding with Transformer is a computationally intensive process. A method that encodes all mention pairs in a large corpus can become intractable. Our method, however, is linear in complexity with the number of mentions, as shown in Figure 7, and outperforms previous methods in terms of computational efficiency. While Held et al. (2021)’s cross-encoding at prediction is linear (5*n), their pruning step is quadratic. They rely additionally on training a bi-encoder and a mention neighborhood detector step that requires GPUs.

heading: 7.2 Synonymous Lemma Pairs
text: We have established an upper limit for ECR using the LHOra+ Dlong method for ECB+. Previous methods such as Held et al. (2021), use an oracle coreference scorer after their pruning step. In other words, their oracle assumption involves using a perfect cross-encoder. In contrast, we only use the oracle for pruning by assuming a perfect set of synonymous lemma pairs. This means that
improved pruning methods can lead to better ECR performance. We believe that it is possible to create a more effective synonymous pair detector than LHOra by adopting recent work on predicate class detection (Brown et al., 2014, 2022) that use VerbNet (Schuler, 2005). In future research, we aim to enhance the process of generating synonymous pairs through the use of cross-encoding or additional steps such as word sense disambiguation with the Proposition Bank (Palmer et al., 2005; Pradhan et al., 2022). Identifying the sense of the trigger will help refine the lemma pairs that appear in coreference chains. Additionally, annotating the sense of the trigger is a straightforward process that can be easily incorporated into annotation procedures for new datasets, which is more efficient than coreference annotations.

heading: 7.3 Qualitative Error Analysis
text: We carry out a comprehensive analysis on errors the discriminator makes after the heuristic’s predictions. Unlike previous methods (Barhom et al., 2019) where they sample a subset of mentions to carry out the error analysis, we do so for the entire dataset. By efficiently discarding the large number of P−TN, we are able to isolate the shortcomings of the crossencoder, analyze them and offer solutions. Table 6 in Appendix C lists the various kinds of errors (incorrect and missing links) made by Dsmall on the ECB+ and GVC dev sets.
We find error categories like same-sentence pronouns, weak temporal reasoning, ambiguity due to coreferring entities, misleading lexical similarity, and missed set-member coreferent links. Table 6 in the appendix presents examples of each.
Incorrect links due to same-sentence pronouns like “it” and “this” can be avoided by refining the heuristics-based mention-pair generation process to exclude same-sentence pronouns. Similarly, ambiguous temporal contexts like "Saturday" and "New Year’s Day" that refer to the day of occurrence of the same event in articles published on different dates can be resolved by leveraging more temporal context/metadata where available. Also, errors in lexically-different but semantically similar event mention lemmas can be reduced by leveraging more-enriched contextual representations.
By using the Oracle for pruning, we can focus on where Dsmall falls short in terms of false positives. We first sort the final event clusters based on purity (number of non-coreferent links within the cluster compared to ground truth). Next, we identify
pairs that the discriminator incorrectly predicted to be coreferent within these clusters, specifically focusing on highly impure clusters. We look for these pairs in highly impure clusters and analyze the mention sentences. Our findings are as follows:
• Problems caused when two big clusters are joined through very similar (almost adversarial) examples, e.g., “British hiker” vs. “New Zealand hiker.” This error can be fixed by performing an additional level of clustering, such as, K-means.
• Problems with set-member relations, such as "shootings" being grouped with specific "shooting" events. The sets often include many non-coreferent member events. To address this issue, we can identify whether an event is plural or singular prior to coreference resolution.
• Contrary to the notion that singleton mentions cause the most errors, we found that singletons appear in the least impure clusters. This means the cross-encoder discriminator is good in separating out singletons.

heading: 8 Conclusion & Future work
text: We showed that a simple heuristic paired with a crossencoder does comparable ECR to more complicated methods while being computationally efficient. We set a upper bound for the performance on ECB+ suggesting improvement with better synonyms pairs detection we can achieve better results. Through extensive error analysis, we presented the shortcomings of the crossencoder in this task and suggested ways to improve it.
Future research directions include applying our method to the more challenging task of crosssubtopic event coreference (e.g., FCC (Bugert et al., 2020)) where scalability and compute-efficiency are crucial metrics, making the current heuristicbased mention pair generation process “learnable” using an auxiliary cross-encoder, and incorporating word-sense disambiguation and lemma-pair annotations into the pipeline to resolve lexical ambiguity. An exciting direction for future work made tractable by our work is to incorporate additional cross-encoding features into the pipeline, especially using the latest advancements in visual transformers (Dosovitskiy et al., 2021; Bao et al., 2021; Liu et al., 2021; Radford et al., 2021). Another important direction is to test our method on languages with a richer morphology than English.
Limitations
The most evident limitation of this research is that is has only been demonstrated on English corefernce. Using a lemma-based heuristic requires using a lemmatization algorithm in the preprocessing phase and for more morphologically complex languages, especially low-resourced ones, lemmatization technology is less well-developed and may not be a usable part of our pipeline. Application to more morphologically-rich languages is among our planned research directions.
In addition, all our experiments are performed on the gold standard mentions from ECB+ and GVC, meaning that coreference resolution is effectively independent of mention detection, and therefore we have no evidence how our method would fare in a pipeline where the two are coupled.
A further limitation is that training of the crossencoders still requires intensive usage of GPU hardware (the GPU used for training Longformer is particularly high-end).
Ethics Statement
We use publicly-available datasets, meaning any bias or offensive content in those datasets risks being reflected in our results. By its nature, the Gun Violence Corpus contains violent content that may be troubling for some.
We make extensive use of GPUs for training the discriminator models as part of our pipeline. While this has implications for resource consumption and access implications for those without similar hardware, the linear time complexity of our solution presents a way forward that relies less overall on GPU hardware than previous approaches, increasing the ability to perform event coreference resolution in low-compute settings.

heading: Acknowledgements
text: We would like to express our sincere gratitude to the anonymous reviewers whose insightful comments and constructive feedback helped to greatly improve the quality of this paper. We gratefully acknowledge the support of U.S. Defense Advanced Research Projects Agency (DARPA) FA8750-18-20016-AIDA – RAMFIS: Representations of vectors and Abstract Meanings for Information Synthesis. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of
DARPA or the U.S. government. Finally, we extend our thanks to the BoulderNLP group and the SIGNAL Lab at Colorado State for their valuable input and collaboration throughout the development of this work.

heading: A Ablation Study of Global Attention
text: Table 3 compares Dlong performance with and without Longformer global attention on the ECB+ and
GVC dev sets. This shows a dataset-specific contrast vis-à-vis sequence length where performance with global attention on GVC dev set is only marginally better than without, while the reverse is seen on the ECB+ dev set. More specifically, this suggests that perhaps the “relevant” or "core" context for ECR lies closer to the neighborhood of event lemmas (wrapped by trigger tokens) than the CLS tokens (that use global attention) in both corpora, albeit more so in ECB+. As such, applying global attention to the CLS tokens here encodes more irrelevant context. Therefore, Dlong with Longformer global attention performs less well on ECB+ while being almost comparable to Dlong without global attention on GVC.

heading: B Full Results
text: Table 4 shows complete results for all metrics from all models for within and cross-document coreference resolution on the GVC test set. Table 5 shows complete results for all metrics from all models on the ECB+ test set.

heading: C Qualitative Error Examples
text: Table 6 presents an example of each type of error we identified in the output of our discriminator (Dsmall).

heading: Future Work
text: Future research directions include applying our method to the more challenging task of crosssubtopic event coreference (e. g. , FCC (Bugert et al. , 2020)) where scalability and compute-efficiency are crucial metrics, making the current heuristicbased mention pair generation process “learnable” using an auxiliary cross-encoder, and incorporating word-sense disambiguation and lemma-pair annotations into the pipeline to resolve lexical ambiguity. An exciting direction for future work made tractable by our work is to incorporate additional cross-encoding features into the pipeline, especially using the latest advancements in visual transformers (Dosovitskiy et al. , 2021; Bao et al. , 2021; Liu et al. , 2021; Radford et al. , 2021). Another important direction is to test our method on languages with a richer morphology than English. Limitations
The most evident limitation of this research is that is has only been demonstrated on English corefernce. Using a lemma-based heuristic requires using a lemmatization algorithm in the preprocessing phase and for more morphologically complex languages, especially low-resourced ones, lemmatization technology is less well-developed and may not be a usable part of our pipeline. Application to more morphologically-rich languages is among our planned research directions. In addition, all our experiments are performed on the gold standard mentions from ECB+ and GVC, meaning that coreference resolution is effectively independent of mention detection, and therefore we have no evidence how our method would fare in a pipeline where the two are coupled. A further limitation is that training of the crossencoders still requires intensive usage of GPU hardware (the GPU used for training Longformer is particularly high-end)