What Makes a Review helpful? Predicting the Helpfulness of Italian TripAdvisor Reviews
Authors: Giulia Chiriatti, Dominique Brunato, Felice Dell'Orletta, Giulia Venturi (Università di Pisa, CNR-ILC, 2019)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In this paper, we introduce a classification system devoted to predicting the helpfulness of Italian online reviews. It is based on a wide set of features reflecting the different factors involved and tested on different categories of TripAdvisor reviews. For this purpose, we collected the first Italian corpus of online reviews enriched with metadata related to their helpfulness and carried out an in-depth analysis of the most predictive features.

Introduction
Predicting and modeling relevant factors that determine the helpfulness of online reviews have been attracting growing attention in the Natural Language Processing (NLP) community. Both practical applications and the interest in studying human variables underlying the assignment of helpful/unhelpful votes are mainly involved. The identification of product reviews that are useful to customers can be important for several e-business purposes (e.g., the development of product recommendation systems) as well as to investigate persuasive elements that make a review helpful for a review reader (Hong et al., 2012; Park, 2018). Several approaches have been devised, differing at the level of predicting methods (mainly regression or classification algorithms) and the typologies of factors considered, including content elements found within the review and contextual ones referring to user profiles. Although various strategies have already been followed, according to the recent survey by Diaz and Ng (2018), a number of issues are still open and deserve to be explored. Among others, they include i) the need for “more sophisticated textual features” that can be useful to model a writing style typical of helpful reviews, and ii) the lack of studies focused on languages other than English. In this paper, we address these open issues and present a study devoted to predicting Italian review helpfulness with a specific focus on the role played by linguistic features in modeling the style of helpful reviews. Similarly to previous studies, we tackled the task as a text classification problem but with two main novelties. Firstly, we relied on different sets of predictors, considering both lexical (content) and structural features (i.e., morphosyntactic and syntactic) aimed at reconstructing the style of a text (the linguistic “form”). Secondly, we investigated which typology of features are the most effective to predict the helpfulness of online reviews and whether they remain the same across different review categories. Our contribution: i) We collected a corpus of Italian online reviews enriched with metadata related to their helpfulness. ii) We developed the first classification system devoted to predicting the helpfulness of Italian online reviews, based on features modeling both lexical and linguistic factors involved, and tested it in two experimental scenarios, i.e., in- and out-domain with respect to the training category of reviews. iii) We identified and ranked the most predictive features, showing the key role played by linguistic features, especially to predict the helpfulness of reviews belonging to a category very different from the training one.

Conclusion
In this paper, we have presented the first approach to the task of review helpfulness prediction for the Italian language. Two experimental scenarios have been tested in a corpus of TripAdvisor reviews belonging to different categories (restaurants and attractions). In line with previous findings obtained for the English language, we confirmed that lexical information plays a significant role in classifying helpful reviews. In addition, we proved for the first time the highly predictive power of linguistic features modeling the writing style independently from the content. This is particularly true in the two out-domain experiments: in the first case (same category, different geographical area), the classifier based on the linguistic features achieves the same accuracy as the model using lexical features, and it even outperforms all the other configuration models when tested on the most distant review category (restaurants vs attractions). Among the possible future issues that we would like to investigate, an interesting one concerns the role played by metadata features. In the reported results, we showed that star ratings are not relevant when considered alone, but they give a plus when combined with both lexical and linguistic features. Beyond this metadata, we would like to extend the analysis to further user information possibly related to review helpfulness.