Implicitly Abusive Language – What does it actually look like and why are we not getting there? (2021)

Abstract: Abusive language detection is an emerging field in natural language processing which has received a large amount of attention recently. Still the success of automatic detection is limited. Particularly, the detection of implicitly abusive language, i.e. abusive language that is not conveyed by abusive words (e.g. dumbass or scum), is not working well. In this position paper, we explain why existing datasets make learning implicit abuse difficult and what needs to be changed in the design of such datasets. Arguing for a divide-and-conquer strategy, we present a list of subtypes of implicitly abusive language and formulate research tasks and questions for future research.

## introduction abusive or offensive language is commonly defined as hurtful, derogatory or obscene utterances made by one person to another person or group of persons. 1 examples are (1)-(3). in the literature, closely related terms include hate speech @xcite or cyberbullying @xcite . while there may be nuanced differences in meaning, they are all compatible with the general definition above. 2 (1) stop editing this, you dumbass. due to the rise of user-generated web content, the amount of abusive language is growing. nlp methods are required to focus human review efforts towards the most relevant microposts. though there has been much work on abusive language detection in general, comparatively little work has been focusing on implicit forms of abusive language (4)-(5) @xcite . by implicit abuse we understand abusive language that is not conveyed by (unambiguously) abusive words (e.g. dumbass, bimbo, scum). detailed analyses of the output of existing classifiers have also revealed that currently only explicit abuse can be reliably detected @xcite . in this position paper, we want to shed more light on the nature of implicitly abusive language. we identify subtypes of implicit abuse that can be found in existing datasets and the literature. we also outline shortcomings that prevent implicitly abusive language from really being learned on its own terms. with this study, we hope to guide future research on implicitly abusive language. our contributions in this paper are: ## the story so far by far the most prominent classification approaches applied to abusive language detection are supervised learning methods. whereas initially, traditional learning algorithms, such as svms or logistic regression, were among the most popular methods for this task @xcite @xcite , at present, best results are obtained by deep-learning methods, particularly transformers @xcite @xcite . a more detailed summary of the methods explored can be found in @xcite and @xcite . unfortunately, so far there has been little error analysis of system output for abusive language detection. as a consequence, the community is fairly unaware of what types of errors are made and why. the most notable exception is van @xcite who carry out experiments on the dataset of google's toxic comment classification challenge foot_2 and the dataset by @xcite . as prominent errors that a supervised classifier makes, van @xcite list toxicity without swearwords, rhetorical questions and comparisons/metaphorical language. all these phenomena can be subsumed by implicit abuse. unfortunately, the study by @xcite is only of limited help since one of two datasets considered, namely the dataset from the toxic comment classification challenge, contains a high degree of explicitly abusive language ( ## datasets considered in this study recent years have seen a notable increase of datasets for abusive language detection. since a survey would be beyond the scope of this section, we refer the reader to @xcite and @xcite . however, implicit abuse is not covered in these publications. due to the limited space, we only focus on english datasets in this paper. we also just consider the common binary classification task of whether a micropost is abusive or not. ## different subtypes of implicit abuse for each dataset, we manually annotated a random sample of 500 implicitly abusive instances (according to our proxy described in §3) for their subtypes, i.e. 2,500 instances in total. the subtypes we used were either mentioned in previous work @xcite or frequently observed in the examined datasets. in the following, we describe these subtypes: ## the role of machine learning despite the continuing success of machine learning in many areas of nlp, particularly fairly generic methods, we should be careful in considering this the magic bullet for every problem including the detection of implicitly abusive language. already in some subtasks of (explicitly) abusive language detection, machine learning has not produced the anticipated results. for example, supervised learning still produces fairly poor classification performance on the cross-domain detection of abusive language, with lexicon-based approaches performing much stronger @xcite . further, statistical debiasing methods for abusive language detection have also been reported to yield very limited success @xcite . the authors of that research argue that spending more efforts in ensuring a high quality of the datasets during their creation is more worthwhile than applying sophisticated machine learning. we anticipate that there are also some subtasks in the realm of implicit abuse that may not be solved with the help of supervised learning approaches. one such example may be the task of detecting novel or unknown stereotypes. if we compare the two stereotypes (56) and (57), we find that these sentences differ in meaning, sentiment and also in terms of syntactic structure. if we train a classifier on (56) it is unlikely to identify (57) as an instance of the same category due to the lack of similar features. as a consequence, learning-based approaches are unlikely to succeed in this task. although generic supervised methods may always represent a good baseline, the community should also be open that other more linguistically informed approaches can be more effective for particular subtasks in the detection of implicitly abusive language. @xcite demonstrated that mining for a particular linguistic construction is an effective means to recognize a specific type of sarcasm. we envisage that similar approaches may be effective for the detection of implicit abuse. due to the susceptibility of supervised learning to overfitting, we also recommend an experimental set-up in which a cross-domain evaluation is included in order to check whether the resulting classifiers generalize beyond the training data. ## conclusion there are different subtypes of implicit abuse. some of them are frequent in available datasets (e.g. jokes or stereotypes) while others are sparse (e.g. dehumanization or euphemisms). as far as frequent subtypes of implicit abuse (e.g. stereotypes and perpetrators) are concerned, unsuitable sampling causes biases that prevent classifiers from really learning these phenomena. simply adding instances by merging datasets does not solve the problem. it may introduce further detrimental biases. overall, our analysis supports the claim that the currently available datasets are not really suitable for effectively learning implicit abuse. we strongly argue for new datasets that focus on particular subtypes of implicit abuse. this will also facilitate thinking about appropriate negative data. larger datasets are not necessarily the best datasets to train a classifier on, especially if they are dominated by frequently observed words. finally, it may also make sense to learn on smaller units, such as clauses, rather than on entire microposts.