Investigating Annotator Bias with a Graph-Based Approach (2020)

Abstract: A challenge that many online platforms face is hate speech or any other form of online abuse. To cope with this, hate speech detection systems are developed based on machine learning to reduce manual work for monitoring these platforms. Unfortunately, machine learning is vulnerable to unintended bias in training data, which could have severe consequences, such as a decrease in classification performance or unfair behavior (e.g., discriminating minorities). In the scope of this study, we want to investigate annotator bias — a form of bias that annotators cause due to different knowledge in regards to the task and their subjective perception. Our goal is to identify annotation bias based on similarities in the annotation behavior from annotators. To do so, we build a graph based on the annotations from the different annotators, apply a community detection algorithm to group the annotators, and train for each group classifiers whose performances we compare. By doing so, we are able to identify annotator bias within a data set. The proposed method and collected insights can contribute to developing fairer and more reliable hate speech classification models.

## introduction a massive problem that online platforms face nowadays is online abuse (e.g., hate speech against women, muslims, or african americans). it is a severe issue for our society because it can cause more than poisoning the platform's atmosphere. for example, @xcite showed a relation between online hate and physical crime. therefore, people have started to develop systems to automatically detect hate speech or abusive language. the advances in machine learning and deep learning have improved these systems tremendously, but there is still much space for enhancements because it is a challenging and complex task @xcite . a weakness of these systems is their vulnerability to unintended bias that can cause an unfair behavior of the systems (e.g., discrimination of minorities) @xcite . researchers have identified different types and sources of bias that can influence the performance of hate speech detection models. @xcite , for example, investigated racial bias in hate speech data sets. @xcite showed that topic bias and author bias of data sets could impair the performance of hate speech classifiers. @xcite examined the impact of political bias within the data on the classifier's performance. to mitigate bias in training data, @xcite and @xcite developed an approach. another type of bias that caught researchers' attention is annotator bias. it is caused by the subjective perception and different knowledge levels of annotators regarding the annotation task @xcite @xcite . such a bias could harm the generalizability of classification models @xcite . especially in the context of online abuse and hate speech, it can be a severe issue because annotating abusive language requires expert knowledge due to the vagueness of the task @xcite . nevertheless, due to the limited resources and the demand for large datasets, annotating is often outsourced to crowdsourcing platforms @xcite . therefore, we want to investigate this phenomenon in our paper. there is already research concerning annotator bias in hate speech and online abuse detection. @xcite examined the relevance of instructing annotators for hate speech annotations. @xcite compared the impact of amateur and expert annotators. one of their findings was that a system trained with data labeled by experts outperforms one trained with data labeled by amateurs. @xcite investigated whether there is a performance difference between classifiers trained on data labeled by males and females. al kuwatly et al. (2020) extended this approach and investigated the relevance of annotators' educational background, age, and mother tongue in the context of bias. @xcite examined racial bias in hate speech data sets and its impact on the classification performance. to the best of our knowledge, no one has investigated annotator bias by identifying patterns in the annotation behavior through an unsupervised approach. that is why we address the following research question in the paper: is it possible to identify annotator bias purely on the annotation behavior using graphs and classification models? our contribution is the following: ## data for our study, we use the personal attacks corpora from the wikipedia detox project @xcite . it contains 115,864 comments from english wikipedia that were labeled whether they comprise personal attack or not. in total, there are 1,365,217 annotations provided by 4,053 annotators from the crowdsourcing platform crowdflower -approximately 10 annotations for each comment. each annotation consists of 5 categories distinguishing between different types of attack: quoting attack, recipient attack, third party attack, other attack, and attack. in our experiments, we only use the 5 th category (attack) because it covers a broader range than the other labels. its value is 1 if "the comment contains any form of personal attack" @xcite . otherwise it is 0. the corpora also contain demographic information (e.g., gender, age, and education) of 2,190 annotators. but this data is not relevant to our study. ## methodology our approach is to group annotators according to their annotation behavior and analyze perfor-mance of classification models trained on annotations from these groups. to do so, we firstly group the annotators according to their annotation behavior using a graph. secondly, we split the data set by the groups and their respective annotations. thirdly, we train classifiers for each annotator group and then compare their performances. the reader can find a detailed description of the steps in the following foot_1 : ## results the experiments show that our proposed method enables the grouping of annotators according to similar annotation behavior. classifiers separately trained on data from the different groups and evaluated with the other groups' test data exhibit noticeable differences in classification performance, which confirms our approach. the detailed results can be found in the following: ## discussion the results show that the proposed method is suitable for identifying annotator groups purely based on annotation behavior. the deviations in interrater agreement rates of the groups and in the classifiers' performances prove this. in regards to the weight functions, we found that both krippendorff's alpha and heuristic function are more suitable than the other functions. both are able to separate the annotators into different groups based on their annotation behavior. however, it is difficult to choose a winner between both because of missing comparability. an advantage of our heuristic function in regards to krippendorff's alpha as weight functions is that it does not have the undefined issue if two annotators assign only one type of label to the comments to be labeled. a potential improvement could be to combine krippendorff's alpha weight function with the heuristic function. the results of our method can be linked to annotator bias in the following manner: an identified annotator group that has a high inter-rater agreement within the group, but poor classification performance on the other test sets indicates that it has a certain degree of bias as the group's annotation behavior differs from the rest. for such insights, we see currently two possible use cases: the novelty of our approach is that it is unsu-pervised and does not require any stipulation of bias that you want to detect in advance. existing approaches, such as @xcite , who investigated gender bias, or @xcite and @xcite , who examined racial bias, defined in their hypothesis which kind of bias they want to uncover. our method, however, does not require any pre-defined categories to detect bias. ## conclusion in this paper, we proposed a novel graph-based method for identifying annotator bias through grouping similar annotation behavior. it differs from existing approaches by its unsupervised nature. but the method requires further research and refinement. to address our limitations, we propose the following future work: firstly, we used only one data set for our study. the approach, however, should be also tested and refined with other data sets. the wikipedia detox project, for example, provides two more data sets with the same structure, but with different tasks (toxicity and aggression). in general, data availability is a challenge of this kind of research because hate speech data sets mostly contain aggregated annotations. therefore, we urge researchers releasing data sets to provide the unaggregated annotations as well. secondly, other approaches for grouping the annotators should be investigated. we used only one community detection method, the louvain algorithm. but there are many more methods, such as the girvan-newman algorithm @xcite and the clauset-newman-moore algorithm @xcite . thirdly, our methods should be extended so that it can handle smaller groups. our current approach requires at least 250 annotators in a group to ensure that we have enough training data. but it would be interesting to investigate smaller groups in the hope that these groups are more coherent in regards to their annotation behavior.