Investigating Cross-Document Event Coreference for D utch (2022)

Abstract: In this paper we present baseline results for Event Coreference Resolution (ECR) in Dutch using gold-standard (i.e non-predicted) event mentions. A newly developed benchmark dataset allows us to properly investigate the possibility of creating ECR systems for both within and cross-document coreference. We give an overview of the state of the art for ECR in other languages, as well as a detailed overview of existing ECR resources. Afterwards, we provide a comparative report on our own dataset. We apply a significant number of approaches that have been shown to attain good results for English ECR including feature-based models, monolingual transformer language models and multilingual language models. The best results were obtained using the monolingual BERTje model. Finally, results for all models are thoroughly analysed and visualised, as to provide insight into the inner workings of ECR and long-distance semantic NLP tasks in general.

## introduction with the focus of natural language processing (nlp) applications shifting more towards largescale discourse-oriented tasks, there is a growing need for systems that can model language not only at the word level, but which can also capture longdistance semantic dependencies. event coreference resolution (ecr) has been one of the domains within nlp that has been at the forefront of this transition. the ambition in ecr is to determine whether or not two textual events refer to the same real-life or fictional event. for this to be true, two candidate event mentions should have the same event trigger, which denotes the action performed, and non-contradicting event arguments, which include spatio-temporal information and possible participants to the event. consider the examples below that were taken from two different dutch (flemish) newspaper articles: ## the encore corpus the recently developed encore corpus @xcite provides us with the opportunity to lay the groundwork for crossdocument event coreference in dutch. as far as we know, the encore corpus is the largest annotated cross-document event coreference corpus in existence, not only for the dutch language, but also compared to existing english language corpora. data for the encore corpus was sourced from a large collection of unannotated dutch (flemish) news texts (de clercq, orphée and de bruyne, luna and hoste, veronique, 2020) collected from a variety of online sources during a one-year period. as event coreference data is notoriously sparse, additional measures were taken in order to maximise the total number of coreference links i.e events referring to one another in the corpus. first, named entities were extracted from each of the documents in the aforementioned larger collection. second, articles containing a given number (>5) of unique overlapping entities were grouped together in socalled "event clusters", as it was hypothesized that news texts containing a high number of overlapping named entities are much more likely to contain overlapping events as well. finally, the resulting event clusters were (manually) pruned in order to avoid duplicate and irrelevant news texts. after this process, the corpus totalled 91 event clusters, each containing on average 13 -14 unique documents. ## experimental setup we present baseline results using gold event mentions on the dutch encore corpus. the goal is to correctly reconstruct coreference chains for the events in the documents based on the gold mentions and any spatio-temporal, participant and (meta) linguistic information that was annotated. we report experimental results for both a within and a cross-document coreference resolution task using a variety of algorithms that have shown to perform well throughout the years. the algorithms used for this set of baseline experiments includes both traditional feature-based mention-pair and mentionranking systems, as well as newer monolingual and multilingual transformer models. ## conclusion in this paper, we presented baseline results for dutch ecr on the recently developed encore dataset, which we hope will serve as a benchmark for future investigations into the possibility of developing ecr applications for dutch. we use a selection of both feature-based and transformerbased models that have shown to work well for english ecr and evaluate these for within-document and cross-document coreference. our experiments show that monolingual dutch language models perform best. it should also be noted that multilingual language models perform poorly. this has implications for future work not only in dutch, but possibly for ecr research in other lower-resourced languages. we also present an analysis of our models, confirming earlier observations that semantic similarity features have a large impact on the task of ecr, while discourse features are less effective. additionally, by visualising the attention heads we reveal that transformer architectures can specifically model syntactic and semantic relationships that are important in event coreference. in future work we will progress to the development of an end-to-end dutch ecr system. we will also focus on systems that can accurately model long-distance semantic dependencies, both in context of ecr and language understanding in general.