Leveraging Structural Discourse Information for Event Coreference Resolution in D utch (2023)

Abstract: We directly embed easily extractable discourse structure information (subsection, paragraph and text type) in a transformer-based Dutch event coreference resolution model in order to more explicitly provide it with structural information that is known to be important in coreferential relationships. Results show that integrating this type of knowledge leads to a significant improvement in CONLL F1 for within-document settings (+ 8.6\%) and a minor improvement for cross-document settings (+ 1.1\%).

## introduction large language models (llms) and transformerbased architectures have significantly changed the domain of natural language processing (nlp) in recent years. through pre-training and finetuning masked language models (mlms) such as bert @xcite , state-of-the-art results can be obtained for tasks requiring deep semantic or syntactic knowledge such as readability assessment @xcite , syntactic parsing @xcite and conversational question-answering @xcite . however, despite their apparent dominance over other methods, transformer-based language models are still not the 'one-size-fits-all' solution for a subset of nlp tasks. in particular, discourse-based tasks such as event coreference resolution (ecr) still pose a major challenge. within ecr, the goal is to determine whether or not two textual events refer to the same real-life or fictional event, as is the case in examples 1 and 2 1. frankrijk verslaat belgië in de halve finales van de fifa wereldbeker voetbal en: france beats belgium in the semi-final of the fifa world cup. ## related work there exist two important model paradigms within the domain of ecr. first, mention-ranking approaches focus on finding all possible antecedents for a given event and on generating a ranking of those antecedents based on the likelihood of coreference with the event in question. in lu and ng (2017a) a feature-based probabilistic model was used for within-document ecr. the authors show that lexical features such as full or partial overlap of events and cosine similarity between event mentions are among the most important information sources of the model. in addition, they revealed that distance-based features such as the number of sentences between two events also have a noticeable positive effect on the classifier's performance. a second and more important series of models are mention-pair approaches. this method generates all possible event pairs and reduces the classification to a binary decision (coreferring or not-coreferring) between each event pair. earlier models within this paradigm were entirely featurebased and relied on a series of lexical, structural and logical constraining features. a large variety of classical machine learning algorithms has been tested using the mention-pair paradigm such as decision trees @xcite , support vector machines @xcite and standard deep neural networks @xcite . more recent work has focused on the use of llms and transformer encoders @xcite , with span-based architectures attaining the best overall results @xcite . it has to be noted that mention-pair approaches relying on llms suffer most from the limitations discussed in section 1. therefore, recent studies have attempted, with some success, to integrate insights regarding discourse coherence @xcite and domain-specific document discourse information (choubey et al., 2020) into existing pipelines. research for comparatively lower-resourced languages has generally followed the paradigms and methods described above and has focused on languages such as chinese @xcite , arabic @xcite and dutch @xcite . for each event in our dataset we thus know at which subsection and paragraph level it is located within a given article. additionally, for each event we can derive its text location, depending on where in the article the event is located. there are 5 possible locations, being the article header, article subheader, article introduction, subsection title and paragraph. ## results and discussion tables 1a and 1b show the results of testing various discourse-level embeddings in a within-document and cross-document context, respectively. we evaluate our results using the established conll metric, which is an average of 3 commonly used metrics for coreference evaluation: muc @xcite , b 3 @xcite and ceaf @xcite . we report the average and standard deviations of 3 runs of experiments with different random seeds for the discourse-level input embedding matrices. for the within-document experiments, we see a significant impact on overall performance when including paragraph-and subsection-level information in the fine-tuning process. a combination of paragraph embeddings and subsection embeddings provides the best results. conversely, we find that the inclusion of text location embeddings does not have any noticeable impact on the classification of within-document event coreference. in the cross-document setting, we find that including structural discourse information does not have a significant impact on classifier performance. while including document and paragraph-level embeddings results in a minor improvement over the baseline coreference model, we find that in general including discourse-specific embeddings does not help with cross-document event coreference. ## conclusion and future work in this paper we explored the potential of using discourse-level embeddings in transformer-based models for event coreference resolution. motivated by general linguistic theory on the overall structure of language we integrate paragraph, subsection and text location information in a dutch bert-based mention-pair event coreference algorithm. we find that in within-document contexts the inclusion of discourse-level information has a significant positive effect on overall classifier performance. in particular, the inclusion of paragraph and subsection information consistently leads to better results. results for the cross-document setting show only minimal improvement over the baseline model. in future work, we aim to further develop structurally informed models for event coreference resolution as well as look into improving the existing crossdocument setup.