Darkness can not drive out darkness: Investigating Bias in Hate S peech D etection Models (2022)

Abstract: It has become crucial to develop tools for automated hate speech and abuse detection. These tools would help to stop the bullies and the haters and provide a safer environment for individuals especially from marginalized groups to freely express themselves. However, recent research shows that machine learning models are biased and they might make the right decisions for the wrong reasons. In this thesis, I set out to understand the performance of hate speech and abuse detection models and the different biases that could influence them. I show that hate speech and abuse detection models are not only subject to social bias but also to other types of bias that have not been explored before. Finally, I investigate the causal effect of the social and intersectional bias on the performance and unfairness of hate speech detection models.

## introduction over the last decade, there have been attempts to use machine learning models @xcite @xcite @xcite @xcite @xcite @xcite for the task of hate speech and abuse detection. however, those studies focused mainly on enhancing models' performance, without providing any insight into the models' inner workings. in recent years, the research community started to pay more attention to machine learning models' explainability and the biases in these models and the datasets. @xcite describe the term algorithmically infused societies as the societies that are shaped by algorithmic and human behavior. the data collected from these societies carry the same bias in algorithms and humans, like population bias and behavioral bias @xcite . these biases are important in the field of natural language processing (nlp) because unsupervised models like word embeddings encode them during training. @xcite . this includes racial biases @xcite @xcite , gender biases @xcite @xcite , and personality stereotypes @xcite . recent research in social science explains that using racial slurs and third person profanity goes beyond offending individuals or groups of people and that it actually aims at stressing on inferiority of the identity of marginalized groups @xcite . however, the research on bias in nlp have not payed attention to how this type of offensive stereotyping being encoded in machine learning models that are trained on data from social media. so i introduce systematic offensive stereotyping (sos) bias which includes associating offensive terms to different groups of people, especially marginalized people, based on their ethnicity, gender, or sexual orientation. on the other hand, studies that focused on the same type of bias in hate speech detection models studied it within hate speech datasets @xcite @xcite , but not in the widely-used word embeddings which are, in contrast, not trained on data specifically curated to contain offensive content. moreover, the proposed methods to study social biases like gender bias in word embeddings focused on studying the statistical association between words that describe women e.g., wife, mother, sister, girl, woman, and words related to femininity e.g. nurturing, sensitive, and emotional @xcite @xcite . however, social science literature has shown that femininity differs in conceptualization among white and black people @xcite . additionally, the claim that the bias found in the word embeddings influence the nlp downstream tasks has not been proven @xcite . a few studies have used statistical correlation to show that influence @xcite . however, correlation is not causation and causal inferences have not been used to understand the influence of bias that exists in word embeddings, on the downstream task of hate speech detection. the limitations enlisted here could have negative implications as hate speech detection models might learn to associate marginalized groups with extremism and abuse. as a result, these models that were supposed to provide a protective environment for the marginalized people to express themselves are the ones that could lead to silencing them or flagging their content as inappropriate. in this thesis, i aim to understand and investigate the performance and the biases of hate speech and abuse detection models through achieving the following research goals: 1) understand the performance of state-ofthe-art hate speech and abuse detection models. 2) inspect other biases than social stereotypical bias in commonly used static word embeddings. 3) investigate intersectional bias in contextual word embeddings and the causal effect of social and intersectional bias on the task of hate speech detection. ## proposed methods in this section, i describe the proposed methods to achieve my research goals and the outcomes of the research goals that have been achieved. the datasets used in the experiments discussed in sections 3.1 and section 3.2 are described in ## conclusion hate speech and abuse detection is a very important task to provide a safe inclusive environment for people from different backgrounds to express themselves. however, the different types of biases that have been shown in different nlp tasks could have a counter effect on these hate speech and abuse detection models as they could associate minorities with hate and abuse which could lead to flagging their content as inappropriate and silencing which is the exact opposite of the aim of hate speech and abuse detection models. in this thesis, i look at the different biases in hate speech and abuse detection models and what is the influence of that bias on the performance of hate speech detection models and how this bias could harm the model's fairness. this work reveal types of biases other than social bias in some of the most common nlp models. and it gives insight into developing targeted and effective techniques to mitigate the effect of the different biases and to develop fairer hate speech detection models.