Attending the Emotions to Detect Online Abusive Language (2020)

Abstract: In recent years, abusive behavior has become a serious issue in online social networks. In this paper, we present a new corpus for the task of abusive language detection that is collected from a semi-anonymous online platform, and unlike the majority of other available resources, is not created based on a specific list of bad words. We also develop computational models to incorporate emotions into textual cues to improve aggression identification. We evaluate our proposed methods on a set of corpora related to the task and show promising results with respect to abusive language detection.

## introduction nowadays, abusive behavior has become a rising problem in online communities @xcite . such adverse behavior can have serious effects on the physical, mental, and social health of online users, among whom teenagers and young adults are the most vulnerable group. 1 to combat this problem at scale, automated natural language processing (nlp) systems can help identify potentially abusive language. in recent years, there have been several efforts to automate the detection of offensive language across social media platforms. lexical features have been proven to work quite well for this task @xcite . however, such features introduce some bias into the systems by heavily relying on profane words, whereas reports show that most profanities are used in a neutral way in today's teen talks @xcite . the following examples signify the need for linguistically more sophisticated techniques beyond profanity dependent models to detect abusive language: in fact, most of the resources available for abusive language detection have been created based on either a list of bad words or seed words related to abusive topics. in this paper, we aim at tackling this limitation by proposing a new method for sampling the data without focusing on a specific bad word list. we are interested to collect this new dataset from a social media website that is specifically popular among youth, since they are the most vulnerable group of users when it comes to online abuse.we scrape our data from curious cat, foot_0 a semi-anonymous question-answering website, that has increased in popularity among teenagers. this platform provides a way to interact anonymously, which opens the door for digital abuse. on this website, users can choose not to reveal any personal information on their account, as well as post comments/questions on other users' timelines anonymously. additionally, on average, the posts are too short in length. these properties limit both the content of a post, as well as the information about the sender of that post. to overcome the aforementioned challenges within the data, we propose a new methodology to integrate emotional information into textual cues from the input text to decide whether it is offensive or not. our main contributions in this paper are as follows: ## related work abusive language identification and hate speech detection have been addressed by many research papers @xcite . most of the related works have employed feature engineering approaches, and use a combination of different types of lexical, syntactic, semantic, sentiment and lexicon-based features along with classic machine learning algorithms such as support vector machines (svm), and logistic regression @xcite @xcite @xcite . due to the popularity of deep neural networks, multiple studies have recently been conducted in order to explore the performance of these models on the task of aggression identification. most of these studies are focused on hate speech detection within twitter. @xcite use a convolutional neural network (cnn) based model, and investigate different textual and embedding features as the input to the model where word2vec produces the best results. @xcite conduct an extensive evaluation on multiple traditional and deep learning approaches, and report the best results using an ensemble of lstm and gradient boosted decision trees. there are also a few works that try to incorporate user information into the model, using approaches such as graph neural networks @xcite to learn the structure of online communities along with the linguistic behaviors of the users within them. the main limitation of these approaches is that they are not applicable to the social media platforms that offer anonymity options to the users such as curious cat and ask.fm. several research papers have proven that emotion lexicons are helpful features for the tasks of abusive language and hate-speech detection @xcite @xcite @xcite . there is also one study that shows jointly modeling of emotion classification and abuse detection, through a multitask approach, can improve the performance of the latter task @xcite . our methodology has two key differences in contrast to other existing methods: (1) instead of using an ensemble approach, we create unified deep neural architectures that show very promising results across multiple domains, and (2) we do not use any user-level information in our model. therefore, the model can be applied to various online platforms, even those that offer anonymity. ## dataset we collected the data from curious cat, which is a semi-anonymous, question-answer social media platform. curious cat is very popular among the youth and has more than 15 million registered users. on this website, users can choose not to reveal any personal information on their account, as well as post comments/questions on other users' timelines anonymously. the anonymity option available on curious cat opens the door for digital abuse. due to these properties, there are two significant limitations with respect to curious cat data: (1) the post content is usually too short making abuse detection harder, and (2) there is very limited information, if any, about the sender of a post. ## methodology emojis help online users to better express their feelings within the text. with this notion, we hypothesize that emojis are effective tools to provide additional context for online comments, resulting in bet-ter offensive language recognition. for capturing emotions from the text, we use deepmoji @xcite pre-trained on twitter data. as for the output, this model creates a representation for 64 frequently used online emojis that shows how relevant each emoji is to a given text. figure 2 illustrates the top 5 emojis that deepmoji assigned to one neutral and one offensive instances in our curious cat data. both of these comments are very short and include the bad word "die". we can see that deepmoji correctly recognized the tone of the language in both examples. the colors also show the attention weights assigned by deepmoji model. the darker colors indicate higher attention weights. interestingly, the word "die" is attended the most in the offensive instance. in this paper, we examine two different approaches to create the model that combines deep-moji and textual representations to detect whether a given input text is offensive or not. the motivation behind this idea is to exploit emotional representation to better distinguish the use of profanities in an offensive way from a neutral way. both models include the two following main modules: for combining the output of the above mentioned modules, we try two following approaches: concatenation: one popular way to incorporate information into deep neural models is concatenation. in this approach, we pass the output of bilstm to an attention layer, same as @xcite , to aggregate the output hidden states of bilstm into a single vector. within this layer, we calculate the weighted sum of @xmath0, where ## experiments and results we stratified split curious cat data into train and test sets with a 70:30 training to test ratio, and use 20% of the train data as the validation set. for the other corpora, we use the same train, validation, and test folds as used by the original papers. as for preprocessing, we truncate the posts to 200 tokens, and right-pad the shorter sequences with zeros. we use binary cross entropy to compute the loss between predicted and actual labels. to smooth the imbalance problem in the datasets, we add information about class weights to the loss function. the network weights were updated using adam optimizer (kingma and ba, 2015) with a learning rate of 1e -5 . we trained the model over 200 epochs, and reported the test results based on the best macro f1 obtained from the validation set. ## conclusion and future work in this paper, we create a new resource for the task of abusive language detection that does not focus on specific list of bad words. we also propose two different approaches for incorporating emotion information into textual representation by pre-senting end-to-end deep neural models that show very promising results across three existing corpora, and our new corpus for abusive language detection. based on the results, adding emotion information to the model can improve the performance, especially for short and noisy textual data. as for the future work, due to the fact that perceived level of aggression is very subjective to the user, we plan to jointly model the question and answer within a pair for the curious cat and ask.fm data. we believe that the reply that the user provides in response to a received question/comment is a strong indicator whether it was offensive or neutral towards the user. another possible path in order to move the research forward, is to expand this task to the detection of cyberbullying incidents which has also become a growing concern in online communities.