Implicitly Abusive Comparisons – A New Dataset and Linguistic Analysis (2021)

Abstract: We examine the task of detecting implicitly abusive comparisons (e.g. “Your hair looks like you have been electrocuted”). Implicitly abusive comparisons are abusive comparisons in which abusive words (e.g. “dumbass” or “scum”) are absent. We detail the process of creating a novel dataset for this task via crowdsourcing that includes several measures to obtain a sufficiently representative and unbiased set of comparisons. We also present classification experiments that include a range of linguistic features that help us better understand the mechanisms underlying abusive comparisons.

## introduction abusive or offensive language is commonly defined as hurtful, derogatory or obscene utterances made by one person to another person. 1 examples are (1)-( 3 ). in the literature, closely related terms include hate speech @xcite or cyber bullying @xcite . while there may be nuanced differences in meaning, they are all compatible with the general definition above. the definition we follow in this work also restricts abusive language to those utterances that are made to deliberately insult the target. a second requirement of an utterance to be considered abusive is that the target itself has to perceive the utterance as abusive. due to the rise of user-generated web content, the amount of abusive language is steadily growing. nlp methods are required to focus human review efforts towards the most relevant microposts. by implicit we understand abusive language that is not conveyed by (unambiguously) abusive words (e.g. dumbass, bimbo, scum). detailed analysis on the output of existing classifiers has also revealed that currently only explicit abuse can be reliably detected @xcite . given that implicit abuse is a challenging problem, we believe that the only reasonable approach to solve this problem is to address specific subtypes individually rather than consider all types of implicit abuse at once. in this paper, we examine implicitly abusive comparisons. a comparison is the act of evaluating two or more things by determining the relevant characteristics of each thing and to determine which characteristics of each are similar/different to the other @xcite . by an abusive comparison, we understand a comparison that is perceived as abusive. in this work, we only consider those comparisons in which no (explicitly) abusive words are contained (6)-(8). those comparisons are referred to as implicitly abusive comparisons. we exclude comparisons with abusive words since they can be easily detected with recent lexical resources for language abuse @xcite . we address abusive comparisons since they make up a large proportion of comparisons on the most related dataset by @xcite . that dataset was created to automatically detect the sentiment of a comparison: one has to distinguish between positive comparisons (you look like a princess), neutral comparisons (you look like your brother) and negative comparisons (you look like a crackhead). we manually annotated the negative comparisons of that dataset. we considered the 359 comparisons that focus on a person 2 (i.e. the 2nd person pronoun you). about 75% of the comparisons were considered abusive, the clear majority (2/3) is implicitly abusive. 25% of the negative person-focused comparisons were non-abusive ( 9 )-(11), which is also a significant proportion. unlike many other datasets for abusive language detection, we create our new dataset with abusive comparisons by inventing instances (i.e. comparisons) rather than by annotating automatically extracted instances. this design choice is necessary since existing datasets contain either insufficient or biased comparisons: the dataset from @xcite includes about 180 implicitly abusive comparisons, however, we found, next to many nearduplicates, a heavy bias towards very few recurring images (e.g. you behave like a child, you look like a monkey). we observed the same phenomenon when extracting comparisons from twitter directly. established datasets for abusive language detection @xcite ) contain just about 30-40 abusive comparisons. our dataset will be created via crowdsourcing. of course, having abusive comparisons be invented this way will inevitably result in some articificial data. however, we think only thus can we produce a dataset of reasonable size that has also a very low degree of bias which are two important requirements to be able to do research on this novel research topic. having crowdworkers invent instances of abusive language may raise ethical concerns. however, the type of abusive language that will be invented in this work is not directed towards specific invididuals or identity groups. therefore, we believe that this procedure is justifiable. in principle, creating morally disputable content as part of research is not unusual. both in plagiarism detection (potthast 2 other foci are fairly unlikely to be abusive. @xcite and deception detection @xcite , a procedure similar to ours is pursued. we frame our task as a binary classification problem in which each instance is to be categorized as either an abusive or some other negative comparison. positive and neutral comparisons are not considered since @xcite already proposed a polarity classifier for comparisons. our contributions in this paper are: ## related work datasets in abusive language detection mostly focus on different targets (e.g. islamophobia @xcite , antisemitism @xcite , misogyny @xcite ), different languages (e.g. spanish @xcite , arabic @xcite , portuguese @xcite ) or different domains (e.g. twitter @xcite , facebook @xcite , wikipedia @xcite ). despite some theoretical work outlining distinct subtypes of abusive language @xcite , there has been little work on datasets that focus on particular subtypes. @xcite present a more detailed overview on abusive language detection. comparisons, particularly figurative comparisons (similes), have been examined with regard to sentiment. @xcite investigate automatic polarity classification of comparisons while component example sentence topic (t) [you] are as smart as a toad. eventuality (e) you [are] as smart as a toad. comparator (c) you are [as] smart [as] a toad. property (p) you are as [smart] as a toad. pattern (t+e+c+p) [you are as smart as] a toad. vehicle you are as smart as [a toad]. niculae and danescu-niculescu-mizil (2014) establish a general correlation between sentiment and comparisons. sentiment is also relevant for the general detection of abusive language (brassard-gourdeau and khoury, 2019). yet the focus on the subset of negative comparisons that are implicitly abusive has not been addressed before. ## linguistic features we present a set of linguistic features that we use for supervised classification. some of them are too difficult to produce automatically. yet we consider them relevant to this work because they may shed more light onto the nature of abusive comparisons. these particular features are produced manually.