Word Embeddings in Sentiment Analysis
Authors: Ruggero Petrolito, Felice Dell'Orletta (Università di Pisa, CNR-ILC, 2018)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In the late years sentiment analyEnglish. In the late years sentiment analysis and its applications have reached growing popularity. Concerning this ﬁeld of research, in the very late years machine learning and word representation learning derived from distributional semantics ﬁeld (i.e. word embeddings) have proven to be very successful in performing sentiment analysis tasks. In this paper we describe a set of experiments, with the aim of evaluating the impact of word embedding-based features in sentiment analysis tasks.

Introduction
Introduction Introduction In the late years sentiment analysis has reached great popularity among NLP tasks. As reported by Mäntylä et al. (2016) the number of papers on this subject has increased signiﬁcantly in the ﬁrst two decades of 21st century, as well as the extent of its applications. A wide variety of technologies has been used to assess sentiment analysis tasks during this period. In the latter years, machine learning techniques proved to be very effective; in particular, in recent years systems based on deep learning techniques represent the state of the art. In this ﬁeld, word embeddings have been widely used as a way of representing words in sentiment analysis tasks, and proved to be very effective. A relevant mirror of the state of the art in sentiment analysis ﬁeld can be found in the SemEval workshops. In the 2015 edition (Rosenthal et al., 2015), most participants used machine learning techniques; in many of the subtasks, the top ranking systems used deep learning methods and word embeddings, like the system submitted by Severyn and Moschitti (2015), which was ranked 1st in subtask A and 2nd in subtask B. In 2016 edition (Nakov et al., 2016), deep learning based techniques, such as convolutional neural networks and recurrent neural networks, were the most popular approach. In 2017 edition (Rosenthal et al., 2017), machine learning methods were very popular, especially support vector machines and deep neural networks like convolutional neural networks and long short-term neural networks. Concerning Italian language, EVALITA conference well represents the state of the art in the natural language processing ﬁeld. In 2016 edition (Barbieri et al., 2016), the top ranking systems used machine learning and deep learning techniques (Castellucci et al. (2016), Attardi et al. (2016), Di Rosa and Durante (2016)). The purpose of this study is to explore ways of using word embeddings to build meaningful representations of documents in sentiment analysis tasks performed on Italian tweets.

Conclusion
Conclusions Conclusions In this work we study the impact of word embedding-based features in the sentiment analysis tasks. We performed several classiﬁcation experiments to investigate the effects on classiﬁcation performances of ﬁve dimensions related to the word embeddings. We tested several different ways of selecting and combining the embeddings and we studied how the performance of a sentiment classiﬁer changes. Despite the lessons learned from this work, several aspects remain to investigate, such as, for example, the tuning of the parameters used to train the embeddings, and new vector combining strategies. References Giuseppe Attardi, Daniele Sartiano, Chiara Alzetta and Federica Semplici. 2016. Convolutional Neural Networks for Sentiment Analysis on Italian Tweets. CLiC-it/EVALITA. Francesco Barbieri, Valerio Basile, Danilo Croce, Malvina Nissim, Nicole Novielli and Viviana Patti. 2016. Overview of the evalita 2016 sentiment polarity classiﬁcation task. Proceedings of Third Italian Conference on Computational Linguistics (CLiC-it 2016) & Fifth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2016). Yoshua Bengio, Réjean Ducharme, Pascal Vincent and Christian Jauvin. 2003. A Neural Probabilistic Language Model. Journal of Machine Learning Research 3 (2003) 1137–1155. Piotr Bojanowski, Edouard Grave, Armand Joulin and Tomas Mikolov. 2016. Efﬁcient Estimation of Word Representations in Vector Space. CoRR abs/1607.04606, 2016. Giuseppe Castellucci, Danilo Croce and Roberto Basili. 2016. Context-aware Convolutional Neural Networks for Twitter Sentiment Analysis in Italian. CLiC-it/EVALITA. Emanuele Di Rosa and Alberto Durante. 2016. Tweet2Check evaluation at Evalita Sentipolc 2016. CLiC-it/EVALITA. Verena Lyding, Egon Stemle, Claudia Borghetti, Marco Brunello, Sara Castagnoli, Felice Dell’Orletta Henrik Dittmann, Alessandro Lenci and Vito Pirrelli. 2013. PAISÀ Corpus of Italian Web Text. Institute for Applied Linguistics, Eurac Research. Mika V. Mäntylä, Daniel Graziotin and Miikka Kuutila. 2016. The Evolution of Sentiment Analysis A Review of Research Topics, Venues, and Top Cited Papers. Computer Science Review, Volume 27, February 2016, Pages 16-32. Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean. 2013. Efﬁcient Estimation of Word Representations in Vector Space. CoRR abs/1301.3781, 2013. Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani and Veselin Stoyanov. 2016. SemEval2016 task 4: Sentiment analysis in Twitter. Proceedings of the 10th international workshop on semantic evaluation (semeval-2016). Fan Rong-En, Chang Kai-Wei, Hsieh Cho-Jui, Wang Xiang-Ruind Lin Chih-Jen. 2008. LIBLINEAR: A Library for Large Linear Classiﬁcation. Journal of Machine Learning Research, 9:1871-1874. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter and Veselin Stoyanov. 2015. Semeval-2015 task 10: Sentiment analysis in twitter. Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), 451– 463. Sara Rosenthal, Noura Farra and Preslav Nakov. 2017. SemEval-2017 task 4: Sentiment analysis in Twitter. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 502–518. Aliaksei Severyn and Alessandro Moschitti. 2015. Unitn: Training deep convolutional neural network for twitter sentiment classiﬁcation. Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), 464–469.