Investigating Sampling Bias in Abusive Language Detection (2020)

Abstract: Abusive language detection is becoming increasingly important, but we still understand little about the biases in our datasets for abusive language detection, and how these biases affect the quality of abusive language detection. In the work reported here, we reproduce the investigation of Wiegand et al. (2019) to determine differences between different sampling strategies. They compared boosted random sampling, where abusive posts are upsampled, and biased topic sampling, which focuses on topics that are known to cause abusive language. Instead of comparing individual datasets created using these sampling strategies, we use the sampling strategies on a single, large dataset, thus eliminating the textual source of the dataset as a potential confounding factor. We show that differences in the textual source can have more effect than the chosen sampling strategy.

## introduction abusive language detection has become an important problem, especially in a world where #black-livesmatter, and where abusive posts on social media need to be found and deleted automatically. however, we also know that the datasets that we currently use for training classifiers are all biased in some way or another. @xcite present one of the first investigations into into bias in different datasets for abusive language detection for english. they compare characteristics of 6 datasets, based on their underlying sampling strategy, their proportion of abusive posts, and the proportion of explicit abuse. the proportion of abusive posts is important for classifiers: if that proportion is too small, classifiers tend default to the majority class baseline. thus, creators of datasets use a range of strategies to increase the number of abusive posts in their data. the specific strategy can have an influence on the proportion of explicitly abusive posts, which tend to be easier to identify as abusive, and overall on classifier performance. we are interested in understanding this interaction better. @xcite distinguish between boosted random sampling and biased topic sampling. boosted random sampling is based on a complete sample, for example all tweets of a specific time frame. then, the number of abusive posts is boosted using different methods, for example by adding more posts by users who have been blocked for being abusive. biased topic sampling, in contrast, samples posts from specific topics, such as soccer or islam, which are known to cause a considerable amount of abuse. @xcite argue that the type of sampling strategy introduces bias into the dataset, and we can assume that the two sampling strategies create different biases: random boosted sampling may create a bias towards specific authors but with a widespread range of topics, and biased topic sampling may create a bias towards specific topics, and potentially specific authors. however, we are often unaware of the exact biases present in such datasets. this is important because first results on debiasing datasets show that these methods work best when we know which bias is present @xcite . in our work, we focus on reproducing the results by @xcite and providing a closer look at the different sampling strategies. while @xcite normalize performance by using a single classifier on all datasets, they do not normalize across different text types. thus, the two sampling strategies have been used on different datasets, which leaves open the question to what degree the differences in bias are due to textual characteristics (wikipedia talkpages, twitter feed, facebook posts), or to the sampling strategies. consequently, we repeat their experiments applying both sampling techniques to the same dataset. we use two datasets from kaggle competitions, of sufficient size to enable us to sample from the same dataset and obtain smaller subsets based on different sampling strategies. we also add an investigation into two variants of biased topic sampling and the out-of-vocabulary rate of the resulting subsets. the remainder of the paper is structured as follows: section 2 explains our research questions, section 3 provides an overview of related work on bias in abusive language detection data, and section 4 discusses our experimental setup, including datasets, lexicons, sampling strategies, the classifier, and evaluation. in section 5, we discuss our findings, and in section 6, we conclude. ## research questions when reproducing the investigation by @xcite , we focus on the following questions: we first need to investigate how diverse the kaggle datasets are, i.e., to what extent sampling a fairly small subset will change the distribution and difficulty of the dataset. thus, we create 3 sampled subsets and compare their results. this is a replication of the question by @xcite , but we first compare the two sampling strategies on samples from the same underlying dataset, the original kaggle dataset also used in their experiments (see section 4.1 for details on the datasets), which is originally based on boosted random sampling. additionally, we repeat the experiment on another, larger kaggle dataset for abusive language detection. since the original kaggle dataset is based on wikipedia talkpages and thus covers topics different from the one covered in other datasets, we could not use the list of topics used by previous approaches for biased topic sampling @xcite @xcite . this leads to the question how dependent results are on the choice of topics. we compare the wide range of topics we used for the previous question to a setting where we use only one specific term to sample. park et al. (2018) discuss methods to decrease the gender bias in abusive language detection. they suggest 3 methods for debiasing, which successfully reduce gender bias in their experiments: debiasing word embeddings, gender swap data augmentation, and fine-tuning using a larger corpus. @xcite , in contrast, focus on racial bias, which is originally introduced by annotator's insensitivities to african-american english (aae), but is then propagated via a trained classifier learning this bias. @xcite show that priming the annotators for dialect and race of the tweet's producer results in fewer aae posts being labeled abusive. @xcite provide a more in-depth analysis, showing that the bias also holds when comparing tweets containing the keywords "n*gga" and "b*tch". there are also approaches to eliminate bias from datasets. for example, @xcite present a method to identify and replace bias sensitive words. ## conclusion and future work we have investigated the interaction between different sampling strategies with classification results for abusive language detection datasets. we have reproduced the two sampling strategies distinguished by @xcite , boosted random sampling and biased topic sampling, but we applied them to the same dataset, in order to eliminate the differences resulting from the textual sources. we have then extended our experiments to a larger dataset to see how much influence the underlying textual source has on the result. we generally found similar trends to @xcite , but much less pronounced, which indicates that the textual source has more influence on the results than the sampling strategy. another important variable is the definition of topic: if we narrow the topic to one word, the proportion of abusive posts increases (but is still well below two of the three dataset using biased topic sampling, waseem and kumar), and the f-score decreases (but is still higher than all of the f-scores reported by wiegand et al. (2019) 6 ). all of our findings emphasize the importance of testing across different datasets. we have also seen the importance of having a high quality lexicon in order to determine the difference between explicit and implicit abuse. for the future, we plan to have a closer look at the datasets since it is still unclear why some of the datasets are more difficult to classify with high accuracy than others, which cannot be explained by class skewing, sampling technique, or proportion of explicit and implicit abuse. additionally, we will experiment with settings in which training and test data have different biases, i.e., if we sampled using different sampling strategies. we will also extend our efforts to create high quality lexicons of explicit abuse. our ultimate goal is to improve classification performance for implicit abuse.