A manually-annotated Italian corpus for fine-grained sentiment analysis
Authors: Marilena Di Bari, Serge Sharoff, Martin Thomas (University of Leeds, 2015)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
This paper presents the results of the annotation carried out on the Italian section of the SentiML corpus, consisting of both originally-produced and translated texts of different types. The two main advantages are that: (i) the work relies on the linguistically-motivated assumption that, by encapsulating opinions in pairs (called appraisal groups), it is possible to annotate (and automatically extract) their sentiment in context; (ii) it is possible to compare Italian to its English and Russian counterparts, as well as to extend the annotation to other languages.

Introduction
Overall, the field of Sentiment Analysis (SA) aims at automatically classifying opinions as positive, negative or neutral AUTHOR. While at first the focus of SA was on the document level (coarse-grained) classification, with the years it has become more and more at the sentence level or below the sentence (fine-grained). This shift has been due to both linguistic and application reasons. Linguistic reasons arise because sentiment is often expressed over specific entities rather than an overall document. As for practical reasons, SA tasks are often aimed at discriminating between more specific aspects of these entities. For example, if an opinion is supposed to be on the plot of a movie, it is not unusual that the user also evaluates actors' performance or director's choices AUTHOR. For SA applications these opinions need to be assessed separately. Also opinions are not expressed as simple and direct assertions, but by using a number of stylistic devices such as pronominal references, abbreviations, idioms and metaphors. Finally, the automatic identification of sarcasm, irony and humour is even more challenging AUTHOR. For all these reasons, fine-grained sentiment analysis is looking at entities that are usually chains of words such as ``noun+verb+adjective'' (e.g. the house is beautiful) or ``adverb+adjective+noun'' (e.g. very nice car) AUTHOR. In addition to the multitude of approaches to fine-grained SA, there is also shortage of multilingual comparable studies and available resources. To close this gap, we designed the SentiML annotation scheme AUTHOR and applied it to texts in three languages, English, Italian and Russian. The proposed annotation scheme extends previous works AUTHOR and allows multi-level annotations of three categories: target (T) (expression the sentiment refers to), modifier (M) (expression conveying the sentiment) and appraisal group (AG) (couple of modifier and target). For example in: “Gli uomini hanno il potere di sradicare la [povertà]T ]AG, ma anche di [[sradicare]M le 105[tradizioni]T ]AG ”. (Men have the power to eradicate poverty, but also to eradicate traditions) \noindent the groups ``sradicare povertà'' (eradicate poverty) and ``sradicare tradizioni'' (eradicate traditions) have an opposite sentiment despite including the same word sradicare (to eradicate). This scheme has been developed in order to facilitate the annotation of the sentiment and other advanced linguistic features that contribute to it, but also the appraisal type according to the Appraisal Framework (AF) AUTHOR in a multilingual perspective (Italian, English and Russian). The AF is the development of the Systemic Functional Linguistics AUTHOR specifically concerned with the study of the language of evaluation, attitude and emotion. It consists of attitude, engagement and graduation. Of these, attitude is sub-divided into affect, which deals with personal emotions and opinions (e.g. excited, lucky); judgement, which concerns author's attitude towards people's behaviour (e.g. nasty, blame); appreciation, which considers the evaluation of things (e.g. unsuitable, comfortable). The engagement system considers the positioning of oneself with respect to the opinions of others, while graduation investigates how the use of language amplifies or diminishes attitude and engagement. In particular, force is related to intensity, quantity and temporality. To the best of our knowledge the AF has only been applied in the case of Italian for purposes not related to computation AUTHOR. This paper is organized as follows: Section 2 describes the annotation scheme and the annotated Italian corpus, Section 3 reports the results and finally Section 4 our conclusions.

Conclusion
In this paper we have described a manuallyannotated corpus of Italian for fine-grained sentiment analysis. The manual annotation has been done in order to include important linguistic features. Apart from extracting statistics related to the annotations, we have also compared the manual annotations to a sentiment dictionary and demonstrated that (i) the dictionary includes only 29.29% of the annotated words, and (ii) the prior orientation given in the dictionary is different from the correct one given by the context in 28.18% of the cases. The original and annotated texts in Italian (along with English and Russian) and the Document Type Definition (DTD) of SentiML to be used with MAE are publicly available. In the meanwhile, the authors are already working on an automatic system to identify and classify appraisal groups multilingually.