HateChecker: a Tool to Automatically Detect Hater Users in Online Social Networks
Authors: Cataldo Musto, Angelo Sansonetti, Marco Polignano, Giovanni Semeraro, Marco Stranisci (Università di Bari Aldo Moro, ACMOS, 2019)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
In this paper we present HATE CHECKER, a tool for the automatic detection of hater users in online social networks which has been developed within the activities of ”Contro L’Odio” research project. In a nutshell, our tool implements a methodology based on three steps: (i) all the Tweets posted by a target user are gathered and processed. (ii) sentiment analysis techniques are exploited to automatically label intolerant Tweets as hate speeches. (iii) a lexicon is used to classify hate speeches against a set of specific categories that can describe the target user (e.g., racist, homophobic, antisemitic, etc.). Finally, the output of the tool, that is to say, a set of labels describing (if any) the intolerant traits of the target user, are shown through an interactive user interface and exposed through a REST web service for the integration in third-party applications. In the experimental evaluation we crawled and annotated a set of 200 Twitter profiles and we investigated to what extent our tool is able to correctly identify hater users. The results confirmed the validity of our methodology and paved the way for several future research directions.

Introduction
According to a recent study, 58% of the Italian population regularly uses online social networks as Twitter, Facebook, Instagram and LinkedIn. Such a huge diffusion of these platforms is providing the users with many new opportunities and services, just think that almost everyone now uses social media to get information, discuss, express opinions and stay in touch with friends. Unfortunately, due to the lack of control and the absence of a clear management of the concept of identity of the users, social networks have become the perfect place to spread hate against minorities and people having different cultures, values and opinions. As pointed out by several works (Mathew et al., 2018), the diffusion of hate speeches in online social media is continuously growing and the countermeasures adopted by the single platforms are neither effective nor timely, even if a big effort is done to make the process of removing hate speeches faster and more precise2 . Accordingly, the research line related to the development of tools and methods for the automatic detection of hate speeches gained more and more attention. Techniques for detecting hate speeches are obviously based on NLP techniques, and range from simple lexicon-based approaches (Gitari et al., 2015) to more sophisticated techniques that exploit word embeddings (Djuric et al., 2015) and deep learning methods (Badjatiya et al., 2017). Similar research attempts were also proposed for the Italian language. One of the most popular initiative is the Italian HateMap project (Musto et al., 2016), a research project that exploits semantic analysis and opinion mining to identify the most-at-risk areas of the Italian country, that is to say, the areas where the users more frequently publish hate speeches. The interest of the research community for the topic was confirmed by the recent work by Bosco et al. (Bosco et al., 2017), who studied hate speech against immigrants, and by Anzovino et al. (Anzovino et al., 2018) who detected misogyny on Twitter. Moreover, as shown by the organization of a specific task in the EVALITA evaluation campaign, an important effort is now devoted to the automatic detection of misogyny (Fersini et al., 2018) and hate speeches in general (Bosco et al., 2018; Basile et al., 2019). In order to continue the investigation in this research line ACMOS3 , a no-profit association based in Torino, recently launched ”Contro l’Odio4 ”, a joint research project with the University of Bari, University of Torino and several local associations. The project aims to develop tools and methodologies to monitor (and hopefully tackle) online hate speeches and intolerant behaviors. One of the outcomes of the research is HATE CHECKER, a tool that aims to automatically identify hater users on Twitter by exploiting sentiment analysis and natural language processing techniques. The distinguishing aspect of the tool with respect to the work we have previously introduced is the focus of the tool itself. Indeed, differently from most of the literature, that focused on the analysis of single Tweets, HATE CHECKER aims to analyze the users as a whole, and to identify hater users rather than hate speeches. Clearly, both the tasks are in close correlation, since techniques to detect hate speeches can be used to detect hater users as well. However, through this work we want to move the focus on the latter since, up to our knowledge, this a poorly investigated research direction. Just think that no datasets of hater users is currently publicly available. To sum up, the contributions of the work can be summarized as follows: • We present a workflow that allows to detect hater users in online social networks; • We evaluate several configurations (on varying of lexicons and sentiment analysis algorithms) of the pipeline and we identified the most effective one to tackle our specific task; • We share the first publicly available dataset for automatic detection of hater users on Twitter. In the following, we will first describe the methodology we designed to implement our system, then we will discuss the effectiveness of the approach by analyzing the results we obtained on a (publicly available) dataset of 200 Twitter users.

Conclusion
In this work we have presented HATE CHECKER, a tool that exploits sentiment analysis and natural language processing techniques to automatically detect hater users in online social networks. Given a target user, the workflow we implemented in our system uses sentiment analysis techniques to identify hate speeches posted by the user and exploits a lexicon that extends that of the Italian Hate Map project to assign to the person one or more labels that describe the nature of the hate speeches she posted. As future work, we plan to arrange a user study, specifically designed for young people, to evaluate the effectiveness of the system as a Quantified Self tool (Musto et al., 2018), that is to say, to improve the awareness of the users towards the behavior of other people in their social network.