Investigating Annotator Bias in Abusive Language Datasets (2021)

Abstract: Nowadays, social media platforms use classification models to cope with hate speech and abusive language. The problem of these models is their vulnerability to bias. A prevalent form of bias in hate speech and abusive language datasets is annotator bias caused by the annotator’s subjective perception and the complexity of the annotation task. In our paper, we develop a set of methods to measure annotator bias in abusive language datasets and to identify different perspectives on abusive language. We apply these methods to four different abusive language datasets. Our proposed approach supports annotation processes of such datasets and future research addressing different perspectives on the perception of abusive language.

## introduction a challenge that social media platforms are facing in recent years is the large amount of hate speech and other forms of abusive language @xcite . manual monitoring, however, is no longer possible due to the vast volume of usergenerated content. therefore, machine learning models are trained and used by social media platforms, such as facebook, to automatically detect such content @xcite . according to @xcite , these models are a key component of facebook's fight against hate speech. a problem with such machine learning models is that they are vulnerable to bias @xcite . biased models can strongly impair the fairness of a system, which can lead to discrimination @xcite . bias in abusive language detection is already a topic that researchers have started to investigate @xcite . the type of bias we will focus on in this study is annotator bias. this form of bias is a result of an-notators who perceive abusive language differently from each other and have different levels of experience and knowledge @xcite @xcite . we aim to investigate two aspects of annotator bias. (1) assuming that there is only one perspective (one truth) on whether a text is abusive or not, we develop an approach to measure and visualize annotator bias. this approach optimizes the annotation process (e.g., outlier detection, adapting appropriate annotation guidelines). (2) acknowledging multiple valid views on a text (e.g., a group has a more liberal attitude towards abusive texts, while another is stricter), we aim to identify annotator groups to model different, yet valid perspectives. the questions resulting from these research objectives are the following: our contributions are the following: ## related work hate speech and abusive language detection have gained a lot of attention in recent years. a range of different datasets @xcite and shared tasks @xcite @xcite were published to foster research in this area. most of the datasets are commonly labeled by crowdworkers or those in academia with varying expertise @xcite . however, human annotations tend to be subjective and thus inconsistent @xcite , at least if not moderated very strictly. especially for abusive language, @xcite show that individuals interpret hate speech differently. one common method to improve the label quality is presenting each sample to multiple annotators and aggregate their results @xcite . @xcite were the first to propose an approach that incorporates annotator quality into label aggregation. their expectation-maximization (em) algorithm uses the bias matrices to estimate the latent truth. in the matrices the annotator quality is encoded. their seminal work led to further improvements and methods @xcite @xcite . for nlp tasks, @xcite used a customized dawid-skene algorithm to correct for individual biases of crowdworkers and improve model accuracy. however, they did not quantify and inspect the bias of the annotators. in abusive language detection, annotator bias research has focused on how the annotators background influences their annotations. @xcite found models trained on crowd annotations are outperformed by models trained on ex-pert annotations. @xcite emphasized the importance of detailed guidelines to achieve reliable annotations. @xcite showed that classifiers trained on annotations differ in their performance on test data annotated by men or women. al kuwatly et al. (2020) picked up this approach, enhanced it, included other demographics, and found significant differences for annotator's age group and educational background. @xcite observed that posts in african american dialect are more likely to be labeled offensive. similarly, @xcite found that white and non-white workers annotate racially sensitive topics differently. apart from studying the demographic background, researchers also attempt to find groups of annotators with common annotation behavior. @xcite use graph methods to cluster annotators in groups with higher interannotator agreement within groups than across groups. @xcite defined a polarization measure to split annotators in two groups that maximize opposing annotations. to the best of our knowledge, no one has quantified annotator bias at the annotator level. furthermore, the hypothesis of multiple perspectives on abusive language is rarely investigated. ## datasets we use four different abusive english language datasets to demonstrate our proposed approach. it was challenging to find appropriate datasets because our experiment requires unaggregated annotation data. most of the abusive language datasets contain only the agreed upon labels in the documentation and not the individual votes of the annotators. vidgen guest kurrek wulczyn dataset 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 krippendorf's alpha outlier overall krippendorff's alpha formation. the first three datasets @xcite are similar because they are annotated by small groups of annotators (between 6 and 26). furthermore, each document of the three datasets was annotated by two annotators. in case of disagreement, an expert reviewed the votes and decided on the gold label. in contrast, the wulczyn dataset was annotated by many crowdworkersa typical crowdsourcing setup: a group of workers who annotated a small number of documents. each document was annotated by up to 10 annotators. in case of ambiguous annotations, an expert review did not take place. the gold label was determined based on majority vote. for our experiment, we convert all datasets to a binary task (abusive/neutral) to compare the results. figure 1 shows the distributions of the annotators' inter-rater reliability scores in form of krippendorff's alpha. the colored dots represent the overall inter-rater reliability score of each dataset. we see that the overall krippendorff's alphas are all in the same range. the wulczyn dataset, however, exhibits a considerable variance in contrast to the other three datasets. the reason is that 4,053 crowdworkers annotated this dataset, while an instructed small group of workers annotated the other three datasets. therefore, we see many outliers. in the case of the vidgen and kurrek dataset, only one annotator strongly differs from the others. ## methodology our analysis of the annotator bias in the selected abusive language datasets consists of two parts. in the first part, we characterize the annotation behavior based on the deviations of the annotator votes compared with the gold standard of the dataset. in the second part, we visualize the perspectives of different annotator groups on abusive language with the aid of classification models. ## discussion the first part of our study addressing rq1 shows that the proposed approach based on the pessimistic and optimistic scores helps to measure and visualize the difference in the annotation behavior of annotators. in contrast to the inter-rater reliability, our method reveals information about the tendency of the annotators: did they annotate more liberally or stricter than the group average? these findings can be used to understand outliers better, instruct single annotators in the right direction to align them with the rest of the group and/or adapt the annotation guidelines. our approach comprises a range of methods, from an aggregated perspective on all annotations to cluster analyses to evaluations of individual annotators. this variety allows handling of datasets with different numbers of annotators. the proposed approach is unsupervised by itself because it does not require any labeled data. but it can be combined with additional data, as shown by the experiment with the demographic features. we showed that it can help to detect annotator bias caused by different demographic backgrounds. our results are partially in line with the findings from al kuwatly et al. (2020), who examined the same dataset but with a different approach. we confirmed the differences between native and nonnative speakers and between the age groups. in our case, we identified a significant difference between male and female annotators, which al kuwatly et al. (2020) did not find. in contrast to our experiment, they observed a greater difference between educational backgrounds. the reason for the discrepancy can be the different methods. they trained classifiers on different subsets and compared their performances, as we did for the second part of our study. furthermore, they had to group the educational backgrounds to have enough data. consequently, the results can differ. the advantage of our approach over the classifier-based method used by al @xcite and by @xcite on another dataset is that we do not rely on a classifier as we can use the full dataset. the underlying assumption for the first part of the study is that there is only one foundational truth whether a text is abusive or not to demonstrate that we all share the same understanding. in the second part of the study, we had the controversial assumption that there are different perspectives on the perception of abusive language. our goal was to use our proposed method to identify different perspectives and to visualize the differences. by splitting the annotators according to the ratio between the pessimistic and optimistic scores and training different classifiers for these annotators subsets, we showed that there are different perspectives on abusive language. the classifiers of the pessimistic and optimistic annotator subsets per-form well on their own test set and poorly on the test set of the other subset. that means that the perception of abusive language within each group is coherent, but it differs from the perception of the other subset. multiple perspectives on abusive language should be further investigated. @xcite , for example, showed that balancing different perspectives in the training set can improve the classification performance. we can also imagine building classification models that demonstrate different perspectives; each group would have a customized model based on the groups' individual values and perceptions. ## conclusion in this paper, we presented a novel approach to measure and visualize annotator bias purely on their annotation behavior. this approach fosters a better understanding of annotation behavior, detecting outliers, and gaining insights on how to adapt annotation guidelines. furthermore, we showed that there can be different perspectives on abusive language. using our proposed approach, we can identify these perspectives and examine the differences.