A Machine Learning approach for Sentiment Analysis for Italian Reviews in Healthcare
Authors: Luca Bacco, Andrea Cimino, Luca Paulon, Mario Merone, Felice Dell'Orletta (Università Campus Bio-Medico, CNR-ILC, Webmonks s.r.l., 2020)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In this paper, we present our approach to the task of binary sentiment classification for Italian reviews in healthcare domain. We first collected a new dataset for such domain. Then, we compared the results obtained by two different systems, one including a Support Vector Machine and one with BERT. For the first one, we linguistic pre–processed the dataset to extract hand-crafted features exploited by the classifier. For the second one, we oversampled the dataset to achieve better results. Our results show that the SVMbased system, without the worry of having to oversample, has better performance than the BERT-based one, achieving an F1-score of 91.21%.

Introduction
Nowadays, when people want to buy a product or service, they often rely on online reviews of other buyers/users (think of online sales giants like Amazon). Likewise, patients are increasingly relying on reviews on social media, blogs and forums to choose a hospital where to be cured. This behaviour is occurring not only abroad (Greaves et al., 2012; Gao et al., 2012), but also in Italy. This is also demonstrated by the increasing amount of reviews in QSalute1 , one of the most popular Italian ranking websites in healthcare. These reviews are often ignored by hospital companies, which do not exploit the potential of such data to understand patients’ experiences and consequently improve their services. Due to the large amount of data, there is a need for automatic analysis techniques. To meet these needs, we decided to introduce a sentiment analysis system based on machine learning techniques, in order to classify whether a review has positive or negative sentiment. Since such systems require annotated data, the first step was to build a brand-new dataset. We present it in the next section. Then, we developed two systems based on two different classifiers described in Section 3 together with the features extracted from the text. In Sections 4 and 5 we show the experiments conducted during this study, the obtained results and their discussion. Finally, the last section provides concluding remarks and some possible future developments. While there exist several works on affective computing in several domains for the Italian language (Basile et al., 2018; Cignarella et al., 2018; Barbieri et al., 2016), at the time we are writing there are no references in literature that address this particular domain in Italian. Thus, for the best of our knowledge, this is the first work of sentiment analysis on Italian reviews in healthcare.

Conclusion
In this paper, we have introduced a novel system for sentiment analysis for Italian reviews in Healthcare. For the best of our knowledge, this is the first work of this kind in such domain. To do so, we have collected the first dataset for this domain from the web. Then, we have implemented and compared two types of classifiers of the state of the art for such task, the SVM and BERT. Despite the strong dataset imbalance, we have obtained very good results, especially with the SVMbased system, which outperformed the BERTbased one, while maintaining a low computational burden during training. However, there is a chance that increasing the maximum sequence length of BERT it may outperform our best-developed system. Also, recent work (Nozza et al., 2020) has analyzed the contribution of language-specific models, showing in general improvements over BERT multilingual for a wide variety of NLP tasks. For this reason, it might be worth including in future works the use of specific models for Italian, such as GilBERTo6 , UmBERTo7 , and AlBERTo8 . The latter was already used for a sentiment classification task (Polignano et al., 2019). Future works on this dataset may also tackle the task of sentiment classification including the neutral class or sentiment regression of the average scores. Moreover, future research may tackle the task of cataloguing reviews to the area of disease they belong, maybe including other features from metadata such as titles.