MODAL: A multilingual corpus annotated for modality
Authors: Malvina Nissim, Paola Pietrandrea (University of Groningen, University of Tours CNRS, 2017)
Topic: Lexical and Semantic Resources and Analysis

Abstract
We have produced a corpus annotated for modality which amounts to approximately 20,000 words in English, French, and Italian. The annotation scheme is based on the notion of \textit {epistemic construction} and virtually language-independent. The annotation is rigorously evaluated by means of a newly developed strategy based on the alignment of the entire epistemic constructions as identified and marked up two annotators. The corpus and the agreement scoring tools are publicly available.

Introduction
Modality is a pervasive phenomenon crucial to language understanding, analysis, and automatic processing AUTHOR. The creation of modality-annotated data would benefit Natural Language Processing in at least two major aspects: (i) factuality detection, consisting in the automatic distinction between propositions that represent factual events and propositions that represent non factual ones; and (ii) sentiment analysis, which involve the processing of extra-propositional aspects of meaning and the detection of polarised judgements. Additionally, the annotation of modality may also have important repercussions in the field of corpus linguistics, as the techniques developed in the automatic treatment of modality can be used to improve our linguistic knowledge of modality itself. As far as the detection of polarised judgments goes, there have been substantial annotation efforts in recent years, exemplified by recurring and increasing sentiment analysis tasks within the context of the Semeval evaluation campaign.. Note that in 2017 within the sentiment analysis track there was also a task on truth detection, which goes to show how closely related the two phenomena indeed are.} Attention has also been given to more specific factuality tasks such as the CoNLL-2010 Shared Task on identifying hedges AUTHOR, and factuality annotation in languages other than English, such as Italian AUTHOR, and Dutch AUTHOR. However, these are annotation efforts involving specific phenomena rather than modality in general. Indeed, a major bottleneck in the creation of modality-annotated resources is the very notion of modality itself, as encapsulating this phenomenon in one exhaustive but workable definition is far from trivial AUTHOR. Building on the function-based proposal advanced in AUTHOR and AUTHOR, we have created a comprehensive annotation scheme for epistemic modality and have applied it to multiple languages. Contextually, we have developed and deployed an evaluation strategy which shows that the corpus is annotated reliably. \paragraph{Summary of contributions} We produced the first multilingual corpus annotated for modality. The annotation scheme is virtually language-independent, and the annotation is evaluated according to a specifically designed methodology which is portable to other tasks where annotators are left with substantial freedom in the selection of the tokens to be marked up. The corpus and the tools for scoring agreement are publicly available.

Conclusion
Modality can be reliably annotated in multiple languages by taking a bottom-up, functional approach paired with a solid annotation scheme, trees to guide the annotators' decisions, and a rigorous evaluation strategy. With this approach, we have produced the first multilingual corpus annotated for modality, which can be potentially used to train modality detection models as well as to further study modality itself. By making all of the data publicly available, and by sharing our annotation experience, we also hope to provide a blueprint for creating modality-annotated resources in yet more languages.