Neural Sentiment Analysis for a Real-World Application
Authors: Daniele Bonadiman, Giuseppe Castellucci, Andrea Favalli, Raniero Romagnoli, Alessandro Moschitti (Università di Trento, Qatar Computing Research Institute, Almawave, 2017)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In this paper, we describe our neural network models for a commercial application on sentiment analysis. Different from academic work, which is oriented towards complex networks for achieving a marginal improvement, real scenarios require flexible and efficient neural models. The possibility to use the same models on different domains and languages plays an important role in the selection of the most appropriate architecture. We found that a small modification of the state-of-the-art network according to academic benchmarks led to a flexible neural model that also preserves high accuracy.

Introduction
In recent years, Sentiment Analysis (SA) in Twitter has been widely studied. Its popularity has been fed by the remarkable interest of the industrial world on this topic as well as the relatively easy access to data, which, among other, allowed the academic world to promote evaluation campaigns, e.g., AUTHOR, for different languages. Many models have been developed and tested on these benchmarks, e.g., AUTHOR. They all appear very appealing from an industrial perspective, as SA is strongly connected to many types of business through specific KPIs. However, previous academic work has not provided clear indications on how to select the most appropriate learning architecture for industrial applications. In this paper, we report on our experience on adopting academic models of SA to a commercial application. This is a social media and micro-blogging monitoring platform to analyze brand reputation, competition, the voice of the customer and customer experience. More in detail, sentiment analysis algorithms register customers' opinions and feedbacks on services and products, both direct and indirect. An important aspect is that such clients push for easily adaptable and reliable solutions. Indeed, multi-tenant applications and sentiment analysis requirements cause a high variability of the approaches to the tasks within the same platform. This should be capable of managing multi-domain and multi-channel content in different languages as it provides services for several clients in different market segments. Moreover, scalability and lightweight use of computational resources preserving accuracy is also an important aspect. Finally, dealing with different client domains and data potentially requires constantly training new models with limited time availability. To meet the above requirements we started from the state-of-the-art model proposed in AUTHOR, which is a Convolutional Neural Network (CNN) with few layers mainly devoted to encoding a sentence representation. We modified it by adopting a recurrent pooling layer, which allows the network to learn longer dependencies in the input sentence. An additional benefit is that such simple architecture makes the network more robust to biases from the dataset, generalizing better on the less represented classes. Our experiments on the SemEval data in English as well as on a commercial dataset in Italian show a constant improvement of our networks over the state of the art. In the following, Section sec:rel-work places the current work in the literature. Section sec:scenario introduces the application scenario. Sections sec:arch and sec:experiments presents respectively our proposal for a flexible architecture and the experimental results. Finally, Section sec:conclusion reports the conclusions.

Conclusion
In this paper, we have studied state-of-the-art neural networks for the Sentiment Analysis of Twitter text associated with a real application scenario. We modified the network architecture by applying a recurrent pooling layer enabling the learning of longer dependencies between words in tweets. The recurrent pooling layer makes the network more robust to unbalanced data distribution. We have tested our models on the academic benchmark and most importantly on our data derived from a real-world commercial application. The results show that our approach works well for both English and Italian languages. Finally, we observed that our network suffers less from the dataset distribution bias.