Coreference Resolution for Italian: Assessing the Impact of Linguistic Components
Authors: Olga Uryupina, Alessandro Moschitti (Università di Trento, Qatar Computing Research Institute, 2014)
Topic: Lexical and Semantic Resources and Analysis

Abstract
This paper presents a systematic evaluation of two linguistic components required to build a coreference resolution system: mention detection and mention description. We compare gold standard annotations against the output of the mod-ules based on the state-of-the-art NLP for Italian. Our experiments suggest the most promising direction for future work on coreference in Italian: we show that, while automatic mention description affects the performance only mildly, the mention de-tection module plays a crucial role for the end-to-end coreference performance. We also show that, while a considerable number of mentions in Italian are zero pronouns, their omission doesn’t affect a general-purpose coreference resolver, sug-gesting that more specialized algorithms are needed for this subtask.

Introduction
Coreference Resolution is an important prerequisite for a variety of Natural Language Processing tasks, in particular, for Information Extraction and Question Answering, Machine Translation or Single-document Summarization. It is, however, a challenging task, involving complex inference over heterogeneous linguistic cues. Several highperformance coreference resolvers have been proposed recently in the context of the CoNLL-2011 and CoNLL-2012 shared tasks (Pradhan et al., 2011; Pradhan et al., 2012). These systems, however, are engineered to process English documents and cannot be directly applied to other languages: while the CoNLL-2012 shared task includes Arabic and Chinese datasets, most participants have not investigated any language-specific approaches and have relied on the same universal algorithm, retraining it for particular corpora. To our knowledge, only very few systems have been proposed so far to provide end-to-end coreference resolution in Italian. In the context of the SemEval-2010 shared task (Recasens et al., 2010), four systems have attempted Italian coreference. Among these toolkits, only BART relied on any language-specific solutions at this stage (Broscheit et al., 2010; Poesio et al., 2010). The TANL system, however, was enhanced with languagespecific information and integrated into the University of Pisa Italian pipeline later on (Attardi et al., 2012). At Evalita 2009 and 2011, different variants of coreference resolution were proposed as shared tasks (Lenzi and Sprugnoli, 2009; Uryupina and Poesio, 2012), in both cases, only one participant managed to submit the final run. One of the bottlenecks in creating highperformance coreference resolvers lies in the complexity of their architecture. Coreference is a deep linguistic phenomenon and state-of-the-art systems incorporate multiple modules for various related subtasks. Even creating a baseline endto- end resolver is therefore a difficult engineering task. Going beyond the baseline is even more challenging, since it is generally unclear how different types of errors might affect the overall performance level. This paper focuses on systematic evaluation of different sub-modules of a coreference resolver to provide a better understanding of their impact on the system’s performance and thus suggest more promising venues for future research. Starting with a gold pipeline, we gradually replace its components with automatic modules, assessing the impact. The ultimate goal of our study is to boost the performance level for Italian. We are focusing on improving the languagespecific representation, leaving aside any comparison between coreference models (for example, mention-pair vs. mention-entity vs. graph-based).

Conclusion
In this paper, we have attempted an extensive evaluation of the impact of two language-specific components on the performance of a coreference resolver for Italian. We show that the mention extraction module plays a crucial role, while the contribution of the mention description model, while still important, is much less pronounced. This suggests that the mention extraction subtask should be in the primary focus at the beginning of the language-specific research on coreference. Our future work in this direction includes developing a robust statistical mention detector for Italian based on parse trees. We also show that zero pronouns can not be handled by a general-purpose coreference resolver and should therefore be addressed by a separate system, combining their extraction and resolution. Finally, our study has not addressed the last language-specific component of the coreference pipeline, the feature extraction module. Its performance cannot be assessed via a comparison with an oracle since there are no perfect gold features. In the future,we plan to evaluate the impact of this component by comparing different feature sets, engineered both manually and automatically.