Identifying Implicitly Abusive Remarks about Identity Groups using a Linguistically Informed Approach (2022)

Abstract: We address the task of distinguishing implicitly abusive sentences on identity groups (“Muslims contaminate our planet”) from other group-related negative polar sentences (“Muslims despise terrorism”). Implicitly abusive language are utterances not conveyed by abusive words (e.g. “bimbo” or “scum”). So far, the detection of such utterances could not be properly addressed since existing datasets displaying a high degree of implicit abuse are fairly biased. Following the recently-proposed strategy to solve implicit abuse by separately addressing its different subtypes, we present a new focused and less biased dataset that consists of the subtype of atomic negative sentences about identity groups. For that task, we model components that each address one facet of such implicit abuse, i.e. depiction as perpetrators, aspectual classification and non-conformist views. The approach generalizes across different identity groups and languages.

## introduction abusive language is commonly defined as hurtful, derogatory or obscene utterances made by one person to another person. 1 examples are (1)-( 2 ). in the literature, closely related terms include hate speech @xcite or cyber bullying @xcite . while there may be nuanced differences in meaning, they are all compatible with the general definition above. due to the rise of user-generated web content, the amount of abusive language is growing. nlp methods are required to focus human review efforts towards the most relevant microposts. though there has been much work on abusive language detection in general, there has been little work focusing on implicit forms of abusive language (3)-(4) @xcite . by implicit we understand abusive language that is not conveyed by (unambiguously) abusive words (e.g. bimbo, scum). detailed analyses of the output of existing classifiers have also revealed that currently only explicit abuse can be reliably detected @xcite @xcite . in this paper, we define implicit abuse as those abusive utterances that lack any abusive word according to the largest lexicon of abusive words available, i.e. the lexicon by @xcite . in particular, datasets focusing on abuse towards identity groups (jews, gay people etc.) contain a high degree of implicit abuse. for example, according to @xcite , on the dataset from @xcite , 56% of the abusive instances are implicit, while on the dataset from @xcite , as many as 62% are. so far, existing research on implicitly abusive language detection on identity groups has been limited by various biases on existing datasets @xcite , most prominently the identity-group bias @xcite : mentions of identity groups almost exclusively occur in microposts that are considered abusive. as a consequence, most classifiers erroneously learn identity groups as clues for abusive language. given that implicit abuse is a challenging problem, @xcite argue that the only reasonable approach to solve this problem is to address specific subtypes individually rather than consider all types of implicit abuse at once. in this paper, we address the task of distinguishing implicitly abusive remarks on identity groups (5)-( 7 ) from other negative polar sentences (8)-( 10 ). the task is a binary classification problem. each instance is an individual atomic sentence. we create a novel less biased dataset for this task. in nlp, there is an increasing awareness of the importance of producing such data @xcite . moreover, @xcite find that ensuring the quality of datasets during their creation is considerably more effective than even the most sophisticated statistical debiasing techniques. unlike previous work, we focus on a linguistically informed classification approach and show that this approach is equally effective for different identity groups and can be used to outperform supervised classifiers trained on existing datasets. we consider only negative polar utterances, since implicitly abusive microposts have a predominantly negative sentiment. for instance, on a random sample of 200 implicitly abusive instances from the dataset by @xcite , we could not find a single remark with a positive or neutral sentiment. our contributions are the following: this paper only addresses one subset of implicit abuse. however, we consider this focus appropriate, since it is not trivial to detect these instances. as a comprehensive classifier that can detect all these types, we envisage a meta-classifier that collects predictions of individual classifiers designed for different subtypes of abusive language. all resources created as part of this research are made publicly available. they are contained in the supplementary material foot_1 to this paper, which also includes implementation details. ## related work much of the previous work in abusive language detection follows a one-size-fits-all approach (fortuna and nunes, 2018). surveys on existing datasets do not address implicit abuse @xcite . @xcite present a roadmap on implicit abuse arguing that this type of abusive language has not adequately been addressed in previous work. no classification experiments are presented. next to implicit abuse towards identity groups, they identify as subtypes dehumanization, euphemisms, call for action, multimodal abuse and comparisons. comparisons are also addressed by @xcite who present the first dataset for this subtype along with classification experiments. the comparisons do not target identity groups. therefore, our novel dataset and the comparison dataset comprise different sentence types. @xcite present a study on microaggressions which are comments or actions expressing a prejudiced attitude towards marginalized groups unconsciously. such instances are cases of implicit abuse. since this is a descriptive study no data for classification are introduced. @xcite propose a classification approach for what they call veiled toxicity, an umbrella term for many different subtypes of implicit abuse. the approach is evaluated on the dataset by @xcite which @xcite report to have considerable biases. elsherief et al. (2021) introduce a general dataset for implicit abuse which is sampled from tweets by hate groups. the authors report biases in the dataset, such as the identity-group bias. ## data as a source for our data, we chose twitter since it is a platform that contains a high degree of abusive language. we focused on 4 identity groups that cover a range of different characteristics (religion, sexual orientation and gender) and that can also be frequently found in existing datasets. moreover, they need to occur with sufficient frequency in both languages we are going to examine. the groups are gay people 3 , jews, muslims and women. 4 the abusive utterances we are looking for are essentially stereotypical sentences on identity groups. such remarks typically realize the abused target, i.e. the identity group, as the agent (i.e. logical subject) of the verb (5)-(7). our new dataset focuses only on this argument position since stereotypical remarks usually depict identity groups as the entities performing some action (agent) rather than being affected by it (patient, i.e. logical object). we obtain such utterances by extracting tweets containing mentions of our identity groups followed by a negative polar verb. (this strategy has been proposed by @xcite in order to ensure lexical variability.) the focus on verbs rather than on nouns and adjectives was motivated by the fact that the latter two are more likely to be explicitly abusive words. for example, these parts of speech compose 91% of the lexicon by @xcite . in this work, we are interested in implicit abuse, however. to test the recall of our sampling approach, we inspect two random samples of 200 abusive (atomic) instances from two popular datasets that focus on identity groups @xcite . we find that 80/84% of the instances realize the identity group as an agent. 70/70% of the predicates are verbs, the remainder being adjectives and nouns. of the verbal predicates, 79/92% were negative polar verbs. @xcite recently introduced a dataset similar to ours: it focuses on identity groups and also aims at having annotators create suitable non-abusive data. their goal is to reduce the identity-group bias on their data by a large degree. we refer to this dataset as dynab. we examined the non-abusive instances in dynab for our 4 identity groups ( other expressions, such as gays or queer, were too infrequent. 4 ideally, we would also have included black people as an additional identity group. however, it was not possible to obtain a sufficient amount of implicitly abusive data for this identity group in both languages that we consider in this paper. we implemented the following measures proposed by @xcite for producing less biased data for the detection of implicit abuse. we created a gold standard for english and another, less-resourced language, german. exactly the same sampling procedure was applied to both datasets. however, due to the sparsity of german language content on twitter @xcite , the german dataset is smaller. both datasets were annotated via the crowdsourcing platform prolific. 6 the label of each instance represents the majority vote of 5 different crowdworkers, who were native speakers. we opted for a very high approval rate (i.e. 95% or higher) in order to guarantee a sufficiently high annotation quality. (the supplementary material contains annotation guidelines.) ## supervised classifiers and evaluation we consider roberta @xcite as a baseline for generic supervised classification for english data. for our german data, we use the best transformer according to @xcite . we fine-tune the pretrained models on the given task using the flair framework @xcite . (the supplementary notes contain more details on all classifiers including hyperparameter settings.) as evaluation measures, we use macro-average precision, recall, f1-score. for all classifiers built with transformers, we report the average over 5 training runs (including standard deviation). all other classifiers produce deterministic output. ## linguistically informed classifier we propose a linguistically informed classifier which models 3 component tasks. we describe how this classifier is built for english. the component tasks represent concepts which have been suggested to be predictive for this task @xcite but, so far, could not be tested due to the lack of data. in order to avoid overfitting, each component comes with a separate classifier being built on training data different to the test data of our main task. since we manually labeled our dataset also for each of the component tasks foot_4 we can conduct an intrinsic evaluation of each component, too. in order to have an unbiased annotation, each crowdworker was only allowed to participate in exactly one of our annotation tasks. ## evaluation on english dataset we evaluate the linguistically informed classifier (figure 1 ) on our new english dataset (for implicit abuse) against other classifiers trained on existing datasets. we carry out a cross-dataset evaluation: none of the classifiers, including our linguistically informed classifier, has been trained on our english dataset. given the recent criticism against withindataset evaluation @xcite in which high performance is often the result of overfitting, this is a fairly unbiased set up. as datasets for training supervised baselines, we chose those that focus on implicit abuse @xcite or abuse towards identity groups we also include an oracle version of our linguistically informed classifier, that combines the gold standard annotation for the component tasks ( §5.1- §5.3) rather than the outputs of the respective classifiers. this can be considered the upper bound for the linguistically informed classifier. finally, we also consider a human classifier as a general upper bound. we randomly sampled the judgment of one individual annotator from the crowdsourced gold-standard annotation for the detection of abusive language. this individual judgement may notably differ from the gold standard label which is the majority label of 5 annotators. the strong performance of the oracle version of our linguistically informed classifier (77.7% f1) is proof that our 3 linguistic concepts are predictive of abuse on identity groups. the fact that it outperforms our best automatic solution (73.8% f1) suggests that there is still room for improvement. ## evaluation on german dataset our final experiments focus on our german dataset. as baselines, we consider supervised classifiers ( §4) trained on the german datasets for abusive lan- next to a classifier that replicates our linguistically informed classifier on german data, we also test a cross-lingual classifier. following previous work @xcite , we fine-tune the multilingual transformer xlm-roberta @xcite on our english dataset. since this language model also covers german, the resulting classifier can also be applied on our german dataset. with the linguistically informed classifier we outperform both germeval classifiers. the oracle version is even notably better. these results suggest that the linguistic properties of our 3 components are language independent. the fact that the multilingual transformer performs best indicates that, in general, the type of implicit abuse we address in this work, is valid across different languages. ## discussion as the performance of our oracle classifier shows, even a perfect linguistically informed classifier is still below human performance. we could identify two types of ambiguous utterances in our misclassifications that may be responsible: a few sentences are underspecified as to whether they report facts or reflect the author's opinion being biased by their stereotypical views (26)-( 27 ). only the interpretation as an opinion is perceived abusive. moreover, the prior sentiment of the patient may occasionally depend on the ideology of the reader. for instance, atheists may consider (28) abusive while religious persons would not. similarly, fem-inists and non-feminists may have a different perception of ( 29 ). it may be debatable that unique class labels as we have assigned to ( 26 )-(29) are adequate. one may argue that without further context these ambiguities cannot be properly resolved. a general limitation of our approach is that our data exclusively originate from twitter. therefore, we cannot rule out that certain results reported in this paper only hold for data from this platform. given, however, that we made sure that the data from that platform that we use are not affected by any obvious user or topic biases ( §3) and given that our proposed method works across 4 different identity groups and 2 different languages, we estimate the likelihood that this limitation has significantly affected our results to be very low. another limitation of our work is the focus on atomic sentences in which the identity group is the agent of some negative verb. as we have motivated in §3, our exploratory data analysis suggests that this is the most frequent surface realization of such abusive remarks. however, implicitly abusive remarks targeting identity groups may also be expressed in other ways, such as (30) where the identity group is not an agent of some negative polar verb. while constructions such as (30) are possible, we are unaware of any sampling method that would enable us to capture such constructions. we expect these constructions also to be more infrequent than the more prototypical atomic sentences. therefore, we leave it to future work to address them. ## conclusion we presented a new focused dataset for implicitly abusive remarks among negative polar utterances on identity groups. we identified 3 linguistic properties which allow us to effectively detect such abusive remarks across different identity groups and across different languages. the utterances have to be non-episodic and the identity group is either depicted as a perpetrator or attributed to a non-conformist view. we are also able to notably outperform classifiers trained on previous datasets.