Investigating Multilingual Coreference Resolution by Universal Annotations (2023)

Abstract: Multilingual coreference resolution (MCR) has been a long-standing and challenging task. With the newly proposed multilingual coreference dataset, CorefUD (Nedoluzhko et al., 2022), we conduct an investigation into the task by using its harmonized universal morphosyntactic and coreference annotations. First, we study coreference by examining the ground truth data at different linguistic levels, namely mention, entity and document levels, and across different genres, to gain insights into the characteristics of coreference across multiple languages. Second, we perform an error analysis of the most challenging cases that the SotA system fails to resolve in the CRAC 2022 shared task using the universal annotations. Last, based on this analysis, we extract features from universal morphosyntactic annotations and integrate these features into a baseline system to assess their potential benefits for the MCR task. Our results show that our best configuration of features improves the baseline by 0.9% F1 score.

## introduction coreference resolution is the task to identify expressions in a given text that refer to the same entity. while considerable progress has been made in coreference resolution for english @xcite @xcite @xcite , extending this task to multiple languages presents significant challenges due to the linguistic diversity and complexity of different languages. the multilingual coreference resolution (mcr) task @xcite focuses on developing a general and robust system that can effectively handle multiple languages and a wide range of coreference phenomena (e.g., pronoun-drop). recently, @xcite propose a new set of multilingual coreference datasets, coref-ud, built upon the framework of universal dependencies 2 (de marneffe et al., 2021), allowing coreference researchers to conduct cross-linguistic studies across 17 datasets for 12 languages. the datasets serve as resource for the crac 2022 shared task on multilingual coreference resolution @xcite . given the harmonized universal morphosyntactic and coreference annotations, we raise the question whether there are any universal features that are common to all languages and to what extent they can contribute to the development of an mcr system. in this work, we conduct an in-depth investigation into the mcr task by using universal annotations in corefud. first, we analyze ground truth data from different linguistic levels, including mention, entity and document levels, and across different genres, to gain an understanding of coreference across various languages. second, we conduct an error analysis of the most challenging cases that mcr systems fail to resolve. last, based on this analysis, we integrate several features extracted from universal morphosyntactic annotations into a baseline system to examine their potential benefits for the mcr task. to the best of our knowledge, our method represents the first attempt to leverage universal annotations for mcr. our findings reveal: (i) there are indeed commonalities across languages. for example, we observe a common pattern where the closest antecedent of an overt pronoun mainly corresponds to the subject or object position. these commonalities are valuable for potential future research, such as linguistic investigations aimed at further comprehending the linguistic phenomenon of coreference. however, it is important to note that explor-ing universal features is a challenging task due to the inherent variability among languages, e.g., the expression of definiteness. (ii) a common issue encountered in all languages by mcr systems is the difficulty of correctly detecting nominal nouns within some two-mention entities. (iii) our experimental results show that our best configuration of features improves the baseline by 0.9% f1 score. in the field of mcr, there has been notable attention directed towards the research of two types of languages. one prominent area of investigation is around pro-drop languages, such as chinese @xcite @xcite , italian @xcite and arabic @xcite ). another research direction involves the study of morphologically rich languages, such as german and arabic @xcite . in contrast to the aforementioned work, which primarily focuses on enhancing the model's capabilities through technical analysis of specific linguistic phenomena, our research delves into gold annotations to explore multilingual coreference including phenomena like zero pronouns from a linguistic perspective, uncovering valuable insights to foster further research. ## linguistic analyses on corefud 1.1 corefud 1.1 foot_2 is the latest version of corefud @xcite for the crac 2023 shared task on multilingual coreference resolution, including 17 datasets for 12 languages. 4 in the following subsections, we conduct a linguistic study on it by using the ground truth from the training datasets, examining coreference phenomena from different linguistic levels, namely mention, entity and document perspectives, and across different genres, in multiple languages. ## error analysis of mcr systems apart from studying coreference on gold annotations solely, we also investigate the ground truth that the mcr systems failed to address. our particular focus is on two-mention entities, which comprise over 80% of the gold entities where the recall is zero. 9 here, we analyze the predictions of two mcr systems: baseline @xcite , an end-to-end based system, and úfal @xcite , the winning system in the crac 2022 shared task on mcr. foot_9 figure 6 presents the error analysis in a tree structure conducted on úfal. ## modeling with universal annotations based on the findings above, we can gain additional insights and clues regarding mcr. for example, we found that the closest antecedents of overt pronouns are always located in subject position. this pattern is common in nearly all languages as shown in figure 2 @xcite . therefore, we use linguistic information extracted from universal annotations for the purpose of modeling and examine its effectiveness, in the following section. ## discussion and conclusion it has become apparent that leveraging universal morphosyntactic annotations can be advantageous in various ways, like exploring underlying patterns of coreference, performing in-depth analysis and making a contribution to the development of an mcr system. however, there are still languagespecific characteristics that hinder the comprehensive study of multiple languages together, particularly when it involves analyzing intricate aspects of the morphological layer, like definiteness and compound nouns in german. in addition, while multilingual datasets are harmonized to some extent, there are still cases where certain information, such as entity types, is only provided for a limited number of languages. this limitation prevents us from conducting further analyses, such as examining semantic class agreement across languages. we study mcr primarily focusing on identity coreference since it is the most important relation across all datasets. however, it is important to note that there exist various other anaphoric relations, such as bridging and discourse deixis @xcite , that remain unexplored. in this work, we analyze coreference across multiple languages by leveraging the harmonized universal morphosyntactic and coreference annotations in corefud. this analysis provides valuable insights into common features and challenges in mcr. we demonstrate the benefits of incorporating linguistic features for enhancing the mcr system performance.