Parallel Data Helps Neural Entity Coreference Resolution (2023)

Abstract: Coreference resolution is the task of finding expressions that refer to the same entity in a text. Coreference models are generally trained on monolingual annotated data but annotating coreference is expensive and challenging. Hardmeier et al. (2013) have shown that parallel data contains latent anaphoric knowledge, but it has not been explored in end-to-end neural models yet. In this paper, we propose a simple yet effective model to exploit coreference knowledge from parallel data. In addition to the conventional modules learning coreference from annotations, we introduce an unsupervised module to capture cross-lingual coreference knowledge. Our proposed cross-lingual model achieves consistent improvements, up to 1.74 percentage points, on the OntoNotes 5.0 English dataset using 9 different synthetic parallel datasets. These experimental results confirm that parallel data can provide additional coreference knowledge which is beneficial to coreference resolution tasks.

## introduction coreference resolution is the task of finding expressions, called mentions, that refer to the same entity in a text. current neural coreference models are trained on monolingual annotated data, and their performance heavily relies on the amount of annotations @xcite @xcite . annotating such coreference information is challenging and expensive. thus, annotation data is a bottleneck in neural coreference resolution. @xcite have explored parallel data in an unsupervised way and shown that parallel data has latent cross-lingual anaphoric knowledge. figure 1 shows a coreference chain in an english-chinese parallel sentence pair. "acl 2023", "it" in the english sentence, and "acl 2023", "它"(it) in the chinese sentence are coreferential to each other. compared to the two separate monolingual coreferential pairs: <acl 2023, it>, <acl 2023, parallel data has been applied to project coreference annotations in non-neural coreference models @xcite @xcite @xcite . instead, we focus on neural coreference models and ask the following main research question: can parallel data advance the performance of coreference resolution on english, where a relatively large amount of annotations are available? we propose a cross-lingual model which exploits cross-lingual coreference knowledge from parallel data. our model is based on the most popular neural coreference model @xcite , which consists of an encoder, a mention span scorer, and a coreference scorer. we extend these three modules, which are applied to the source-side data, with a target-side encoder and adapters for the mention span scorer and the coreference scorer, allowing these to resolve cross-lingual coreference. as there is no annotated cross-lingual coreference data, the model computes the coreference scores between target spans and source spans without any supervision. we conduct experiments on the most popu-lar ontonotes 5.0 english dataset @xcite . given the english data, we generate 9 different synthetic parallel datasets with the help of pretrained neural machine translation (nmt) models. the target languages consist of arabic, catalan, chinese, dutch, french, german, italian, russian, and spanish. the experimental results show that our cross-lingual models achieve consistent improvements, which confirms that parallel data helps neural entity coreference resolution. neural-coref only models the relation between pairs of mentions. many studies propose to consider entity-level information while predicting clusters @xcite @xcite . however, xu and choi (2020) find that these models considering higher-order inference are not significantly better or even worse. instead, the observed differences can be explained by the powerful performance of spanbert. because these models are expensive in terms of memory and time, especially when using higherdimensional representations. @xcite and @xcite propose models that only keep a limited number of entities in the memory, without much performance drop. @xcite introduce a start-to-end model where the model computes mention and antecedent scores only through bilinear functions of span boundary representations. to cope with the enormous number of spans, @xcite proposes a wordlevel coreference model, where the model first considers the coreference links between single words, and then reconstruct the word spans. all these models are trained on monolingual coreference annotations. in this paper, we introduce a simple model building on the top of neural-coref, which exploits cross-lingual coreference from parallel data in an unsupervised way. 3 coreference models ## experiments due to the page limit, we leave our experimental settings in appendix a. ## conclusions and future work in this paper, we introduce a simple yet effective cross-lingual coreference resolution model to learn coreference from synthetic parallel data. compared to models trained on monolingual data, our crosslingual model achieves consistent improvements, varying from 0.78 to 1.74 percentage points, on the ontonotes 5.0 english dataset, which confirms that parallel data benefits neural coreference resolution. we have shown that the unsupervised crosslingual coreference module can learn limited coreference knowledge. in future work, it would be interesting if we can provide the model some aligned cross-lingual coreference knowledge for supervision, to leverage parallel data better.