The Tenuousness of Lemmatization in Lexicon-based Sentiment Analysis
Authors: Marco Vassallo, Giuliano Gabrieli, Valerio Basile, Cristina Bosco (CREA Research Centre for Agricultural Policies and Bio-economy, Università di Torino, 2019)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
Sentiment Analysis (SA) based on an affective lexicon is popular because straightforward to implement and robust against data in specific, narrow domains. However, the morpho-syntactic pre-processing needed to match words in the affective lexicon (lemmatization in particular) may be prone to errors. In this paper, we show how such errors have a substantial and statistically significant impact on the performance of a simple dictionary-based SA model on data from Twitter in Italian. We test three pre-trained statistical models for lemmatization of Italian based on Universal Dependencies, and we propose a simple alternative to lemmatizing the tweets that achieves better polarity classification results.

Introduction
In the last few years a very large variety of approaches has been proposed for addressing Sentiment Analysis (SA) related tasks. In several approaches, lexical resources play a crucial role: they allow systems to move from strings of characters to the semantic knowledge found, e.g., in an affective lexicon. For achieving this result and calculating the polarity of sentiment, or of some related categories, some shallow morphological analysis has to be applied, which mostly consists in lemmatization. When we refer to standard text, available resources and robust lemmatizers make lemmatization a practically solved issue, but the presence of misspellings, lingo and irregularities makes the application of lemmatization on user-generated content drawn from social media and micro-blogs not equally easy. A possible solution consists in applying supervised machine learning techniques in order to create robust lemmatization models. However, the large manually curated datasets necessary for this task are currently very rare, in particular for languages other than English. For what concerns Italian, a good quality gold standard resource in Universal Dependency has been released which includes texts drawn from micro-blogs, namely PoSTWITA-UD (Sanguinetti et al., 2018). Unfortunately it is not nearly large enough to be of practical use in a supervised machine learning setting. In this paper, we focus on the lemmatization of social media texts, observing and evaluating its impact on SA. The goal of this work is to address the following research questions: what is the impact of lemmatization in SA tasks? Can we classify lemmatization errors and automatically adjust (a relevant portion of) them? We start from the empirical evidence found in a corpus of tweets from the agriculture domain that has initially raised our attention on this problem. After that, we present further experiments on a manually annotated dataset. We further propose some hints about a solution based on an affecting lexicon of inflected forms.

Conclusion
In this paper, we presented an empirical and statistical study on the impact of lemmatization on a NLP pipeline for SA based on an affective lexicon. We found that lemmatization tools need to be used carefully, in order to not introduce too much noise, deteriorating the performance downstream. Then we propose an alternative approach that skips the lemmatization step in favor of a morphologically rich affective resource, in order to alleviate some of the observed issues. We plan on integrating the proposed solutions, including the MAL and an automatic check of the lemma produced by UDpipe, in a pre-processing pipeline based on UDpipe.