A buse A nalyzer: Abuse Detection, Severity and Target Prediction for Gab Posts (2020)

Abstract: While extensive popularity of online social media platforms has made information dissemination faster, it has also resulted in widespread online abuse of different types like hate speech, offensive language, sexist and racist opinions, etc. Detection and curtailment of such abusive content is critical for avoiding its psychological impact on victim communities, and thereby preventing hate crimes. Previous works have focused on classifying user posts into various forms of abusive behavior. But there has hardly been any focus on estimating the severity of abuse and the target. In this paper, we present a first of the kind dataset with 7,601 posts from Gab which looks at online abuse from the perspective of presence of abuse, severity and target of abusive behavior. We also propose a system to address these tasks, obtaining an accuracy of ∼80% for abuse presence, ∼82% for abuse target prediction, and ∼65% for abuse severity prediction.

## introduction in recent times, online social media (osm) has become an indispensable part of our lives. not only these websites connect billions of people around the world, but they also serve as a platform for expressing opinions and sharing information quickly. however, recently osm platforms have been a subject for criticism over the propagation of fake @xcite and hateful content @xcite . such cases of online abuse have also translated into real world hate crimes. 2 abuse in social media is spread across a wide spectrum from mild expressions of attitudes and beliefs to strong violent threats. inspired by hate theories from anti-defamation league (adl) 3 , we broadly classify forms of abuse as 'biased attitude, 'act of bias and discrimination' and 'violence and genocide'. moreover, abusive content could be targeted at specific individuals (e.g., a politician, a celebrity, etc.) or particular groups (a country, lgbtq+, a religion, gender, an organization, etc.). detection of such abusive content is critical for avoiding its psychological impact on victim communities, and thereby preventing hate crimes. prioritization of particular abuse cases can be done if severity of abuse can be automatically assessed. further, identifying if the abuse target is a person or a large group is critical to predict potential impact set and thereby predict if it could lead to real world crimes along with its scale. hence, in this paper, we propose three abuse prediction tasks: prediction of abuse presence, abuse severity prediction and abuse target prediction. since traditional osm websites are reasonably moderated, finding broadly abusive content is possible. but finding abusive behaviour of differing severity is a 'needle in a haystack' kind of challenge. in contrast to the other osm, gab is relatively unexplored and presents a wider spectrum of online abusive behaviour due to its liberal moderation policy @xcite . hence, we gathered a dataset from gab and contribute the labeled posts to the community in the hope of promoting deeper research on abusive content analysis. gab is an alt-right social media website launched in 2016, which has seen a significant rise in the number of registered users to 1,000,000 users along with a daily web traffic of 5.1 million visits per day by the end of july 2019. 4 our key contributions in this paper are as follows: disclaimer: this paper contains examples of hate content used only for illustrative purposes, reader discretion is advised. ## related work several past works have explored different kinds of online abuse (like racism, sexism etc.) on traditionally studied platforms like twitter @xcite @xcite and on some newer web communities like 4chan and whisper @xcite . but web communities differ from each other through subtleties in language and demographic differences. gab poses an altogether different challenge as it differs from older web groups primarily in its use of online communities to congregate, organize, and disseminate information in weaponized form @xcite . some previous papers @xcite @xcite have presented basic statistical analysis of data extracted from gab. recently, @xcite presented a dataset of 33,776 posts on gab annotated on binary labels hate/non-hate. while some papers have focused on racism versus sexism @xcite , others have focused on sarcasm, cyber-bullying etc. @xcite . initial works in this area focused on feature engineering based methods. with the emergence of deep learning, most of the recent works @xcite @xcite have relied on deep learning techniques for abuse detection. to the best of our knowledge, there is no publicly available corpus or prediction system which focuses on fine-grained abusive language classification across all three tasks: prediction of abuse presence, abuse severity prediction and abuse target prediction. ## abuseanalyzer dataset and results our dataset contains 7,601 gab posts classified on three different aspects: abuse presence or not, abuse severity and abuse target. of the 4120 abusive posts, distribution based on severity is -'biased attitude': 1830, 'act of bias and discrimination': 1807, and 'violence and genocide': 483. for the target classes -389 are in 'individual (second-person)', 1330 in 'individual (third-person)', and 2401 in the 'group' class. the code and dataset are publicly available here foot_3 . data extraction and pre-processing: we obtained a collection of 8.4 million gab posts from http: //files.pushshift.io/gab/ for a period of 4 months from jul to oct 2018. we used a high precision lexicon which consists of racial, sexist, xenophobic, extremist and other derogatory terminologies aggregated from multiple source.we used this to filter 7,601 posts written in english for the annotation process. while we made efforts to strike a balance between abusive versus non-abusive posts, we made no efforts to maintain balance within abuse severity or abuse target classes. annotation procedure: four annotators with fluent english skills were provided clear guidelines (refined iteratively) for annotating the posts across all the three abuse prediction tasks. in case a post could belong to more than one severity classes, annotators were asked to mark the higher severity class (based on life-threatening consequences), to avoid multi-labels. each example was annotated by exactly 3 annotators and all the disagreements were resolved after involving all the annotators. as a measure of inter-annotator agreement, we observed cohen's kappa score @xcite as (1) 0.719 for presence/absence of abuse, (2) 0.720 for presence+target, and (3) 0.683 for presence+severity classification. in each case the kappa score is near 0.7 which is a very good agreement among the annotators. dataset statistics and analysis: unigrams bigrams severity biased attitude lol, white, f*ck, against, killed, twitter, government, usermention, america you are, they are, trying to, illegal alien, going to, to do, to get act of bias and discrimination jews, white, black, muslims, stupid, islam, b*tch, k*ke, evil, rape you are, the jews, of sh*t, jews are, white people, muslims are, a race, white people, a n*gger, a k*ke violence and genocide f*ck, kill, hell, die, b*tch, lol, fight, muslims, white, war to hell, the f*ck, to kill, the b*tch, rid of, kill all, get rid, f*ck the, to die, the holocaust target second person you, your, youre, f*ck, stupid, sh*t, jew, b*tch, yourself, @username you are, if you, are you, you don't, do you, youre a, you just, your own third person he, her, she, his, you, this, b*tch, sh*t, trump, him, @username