Neutral Score Detection in Lexicon-based Sentiment Analysis: the Quartile-based Approach
Authors: Marco Vassallo, Giuliano Gabrieli, Valerio Basile, Cristina Bosco (CREA Research Centre for Agricultural Policies and Bio-economy, Università di Torino, 2024)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
The neutrality detection in Sentiment Analysis (SA) still constitutes an unsolved and debated issue. This work proposes an empirical method based on the quartiles of the polarity distribution for a lexicon-based SA approach. Our experiments are based on the Italian linguistic resource MAL (Morphologically-inflected Affective Lexicon) and applied to two annotated corpora. The findings provided a better detection of the neutral expressions with preserving a substantial overall polarity prediction.

Introduction
Sentiment Analysis (SA) is a well-studied task of Natural Language Processing (NLP), whose main objective is to classify opinions from natural language expressions as positive, neutral, negative or a mixture of those [1]. The neutrality detection in SA is an issue approached in different ways [2, 3, 4], but low agreement on how detecting neutral expressions still exists [4, p.136]. In this paper, we approach neutrality detection in lexicon-based SA, where an affective lexicon provides polarity scores ranging from −𝑎 to +𝑎 with 𝑎 ∈ 𝑁, by using a descriptive statistical method based on the quartiles. To our knowledge, this issue was not investigated so far. We aim at drawing attention towards a better prediction of the neutral expressions. This is done by automatically finding out an optimal interval of neutral scores with a control for the asymmetry of the distribution of the scores across the polarity spectrum. Traditionally, neutrality scores have been assumed to be around point 0, or within a conventionally fixed and algebraically-led interval of [−.5;+.5]. Conversely, it seems more reasonable to postulate that this neutral cluster should lie in a dynamic interval around the zero value. As expected, the [−.5;+.5] interval is indeed insufficient for capturing the neutral values, especially when the polarity scores are symmetrical around the point zero. This is because small positive or negative deviations from zero can be incorrectly classified into their respective polarity if they are neutral. Furthermore, for topics with many controversial opinions, where polarities are indeed dispersed, the misclassification of neutral expressions appears significant, as small positive and negative deviations from zero might be more frequent. As a consequence, the neutral interval also appears to be topic-oriented and thus differs from any SA task, as the topic could, in turn, also influence the symmetry of the distribution of scores. The linguistic counterpart to this phenomenon is that “opinions may be so different that common ground may not be found” [5]. On the other hand, especially in the case of unimodal distributions, the more asymmetrical the polarity scores distribution is, the more the polarities might be positively or negatively skewed, and the less likely a false neutral classification should occur. In the case of multimodal distributions, with multiple possible polarizations, detecting the asymmetry becomes more complex as well as the neutral expressions. But, despite the peculiar situation with the same frequencies for oppositely polarized scores, the more a multimodal distribution is skewed (many different modes/peaks possibly far from zero) the less likely false neutral classifications should again occur.

Conclusion
The asymmetry of a polarity scores distribution seems to be topic-oriented and therefore the neutrality detection for a lexicon-based SA with polarity scores reasonably passes through an optimal interval within the first and the third quartile [𝑄1,𝑄3] that takes this asymmetry into account. The findings of this work stipulated that the quartile-based approach is suitable for any corpus where a task of lexicon-based SA with scores is performed. Hence, we do strongly recommend further experiments on other corpora, both annotated and unannotated, and comparing/integrating this method with others (e.g. Valdivia et al. [4]) for the common objective of detecting neutral expressions. Eventually, it is worthwhile noticing that our methodological framework led us to run experiments on test sets of different sizes in order to consider all potential and reasonable unseen data situations. Alternatively, one could propose a similar experiment with fixed-size test sets, which would have provided more stable, comparable results even with established benchmarks, but on the other hand would also significantly reduce the amount of test data.