Leveraging Multi-domain, Heterogeneous Data using Deep Multitask Learning for Hate Speech Detection (2020)

Abstract: With the exponential rise in user-generated web content on social media, the proliferation of abusive languages towards an individual or a group across the different sections of the internet is also rapidly increasing. It is very challenging for human moderators to identify the offensive contents and filter those out. Deep neural networks have shown promise with reasonable accuracy for hate speech detection and allied applications. However, the classifiers are heavily dependent on the size and quality of the training data. Such a high-quality large data set is not easy to obtain. Moreover, the existing data sets that have emerged in recent times are not created following the same annotation guidelines and are often concerned with different types and sub-types related to hate. To solve this data sparsity problem, and to obtain more global representative features, we propose a Convolution Neural Network (CNN) based multi-task learning models (MTLs) to leverage information from multiple sources. Empirical analysis performed on three benchmark datasets shows the efficacy of the proposed approach with the significant improvement in accuracy and F-score to obtain state-of-the-art performance with respect to the existing systems.

## introduction the continuing rise in the usage of the internet has loaded a large volume of content in social media like twitter with 500 million foot_1 tweets per day and facebook laden with 510k comments foot_2 per minute. these sites are important sources for people to give their opinion on a host of general social topics. the message can be clean, sarcastic, obscene, offensive, rude, hateful, etc. @xcite defined hate speech as a broad umbrella term that describes it as any communication that demeans any person or any group based on race, color, gender, ethnicity, sexual orientation, and nationality. defining hate is, itself, a difficult task as it greatly depends on the demography, i.e. the same content comes under right to speech in some countries, while other countries might adhere to a very strict policy for the same message. in recent times, germany made policy for the social media companies that they would have to face a penalty up to $60 million foot_3 if they failed to remove illegal content on time. denmark and canada have laws that prohibit all the speeches that contain insulting or abusive content that could promote violence and social disorders. the indian government has also urged leading social media sites such as facebook, twitter to take necessary action against hate speech, especially those posts that create social outrage. setting aside legal actions our aim should be to combat these texts by agreeing to a set of standard definitions, guidelines, and practices to remove the content. recently many automated techniques following supervised learning utilizing deep neural networks have been developed. recently shared tasks such as @xcite @xcite have mainly focused on developing multiple-layer identification of offensive languages. the existing prior research towards this direction mainly focused on singletask learning (stl) where classification task on one data set at a time is solved by training the model in stochastic gradient descent approach. however, training of neural networks relies on a large amount of data, and creating a balanced data set seems to be time-consuming, and tedious. as the number of posts showing aggressive tendencies is very less compared to non-aggressive posts, we leverage the concept of homogeneous multi-task learning where we utilized the multiple classification task data sets to be trained jointly to solve the task. although binary classification is very problematic, it filters out harmful messages and provides hateful data to further train the model to classify the data into more fine-grained classes and helps in getting the target and sentiment behind the posts, thus preventing the violation of the right to freedom of speech. the key characteristics of our current work are summarized as follows. (i). we propose a deep multi-task learning framework that leverages information from multiple sources. we experiment on five different variations of cnn based single-task learning (stl) and five different variations of cnn based multi-task learning (mtl) approaches for solving the problem of hate speech classification. (ii). the proposed classification approach can be utilized to obtain hateful or abusive posts to further train any classifier with these data to perform the classification to finer labels. ## related work in recent times, online hate speech detection has attracted the attention of researchers and developers because of its necessity in maintaining social fabrics. in recent times, most of the methods that have emerged are mainly based on classical machine learning and deep learning. @xcite defined hateful tweets as speech that contains abusive speech targeting individuals (e.g. cyberbullying, politician, a celebrity or a product) or a group (a religious group, country, lgbt, gender, organization), etc. @xcite identified the personal attack as a binary classification problem and experimented with logistic regression and multilayer perceptron with word n-grams or char n-grams based features. @xcite observed that including simple n-gram features are more powerful than linguistics-based features, syntactic-based features, as well as word and paragraph embeddings. @xcite highlighted the differences between hateful and offensive languages and that, conflating these two erroneously will make many speakers be hate speakers. they highlighted the need to train the model with hateful data that does not contain any particular keywords or abusive terms to enrich the model with more contextual and knowledge-based features. @xcite provided visualization of attention weights and concluded that the model assigned higher attention to potentially abusive terms when employed with contextual information in comparison to self-attention based features. @xcite utilized bert, that make use of transformer @xcite , an attention mechanism helping to capture the contextual representation between words and sub-words of a sentence that is utilized to perform the classification task on @xcite . (pérez and luque, 2019) leveraged bilstm with a dense layer on top consuming elmo vectors @xcite , and bag of words as additional input to do the classification on the data by @xcite . in this paper, we present a multi-task framework that aims at leveraging information contained in multiple related tasks and improve the classification performance of the hate data sets. ## results, comparison and analysis we report the 5-fold cross validation result for d 1 , d 2 and d 3 in ## conclusion and future work in this paper we have proposed five multi-task learning based approaches for hate speech detection. the proposed approaches has an ability to learn shared features between three different hate speech data sets and also leveraging the knowledge from the data of sentiment analysis tasks. the efficacy of the proposed approach is evident from the fact that it shows a consistent improvement in the f-score and accuracy values over the models working on single-task learning paradigm. the system failure on some cases highlights the need to build a more diverse and robust neural network system to take into account the contextual, demographic as well as the knowledge based features.