Mitigating Biases in Hate Speech Detection from A Causal Perspective (2023)

Abstract: Nowadays, many hate speech detectors are built to automatically detect hateful content. However, their training sets are sometimes skewed towards certain stereotypes (e.g., race or religion-related). As a result, the detectors are prone to depend on some shortcuts for predictions. Previous works mainly focus on token-level analysis and heavily rely on human experts’ annotations to identify spurious correlations, which is not only costly but also incapable of discovering higher-level artifacts. In this work, we use grammar induction to find grammar patterns for hate speech and analyze this phenomenon from a causal perspective. Concretely, we categorize and verify different biases based on their spuriousness and influence on the model prediction. Then, we propose two mitigation approaches including Multi-Task Intervention and Data-Specific Intervention based on these confounders. Experiments conducted on 9 hate speech datasets demonstrate the effectiveness of our approaches.

## introduction over the past few years, hate speech on social media has grown significantly in different forms. this causes serious consequences and societal impact on victims of all demographics @xcite , largely affects their mental health @xcite and even triggers real-world hate crimes @xcite . as a result, automated detection of hate speech becomes especially important and there have been numerous research studies conducted to characterize and detect these hateful or toxic contents @xcite @xcite . however, current hate speech detection approaches often fail to be robust and generalizable, how can the jews be the same as god 's people when they are tring to kill them? label: hateful i wonder if one of these guys is a jew. prediction: hateful trump is the most narcissistic man alive, his purposeful ignorance is stunning. label: hateful trump is wrong when he says that us has never been great. prediction: hateful even to slightly different datasets on the same task, which largely prevents them from being applied in real-world applications @xcite . this might be due to the fact that current models are suffering from the spurious correlations between training data and labels (e.g., hateful) @xcite , which might lead to the biased treatment of vulnerable and minority groups such as african american vernacular english speakers and may exacerbate racism @xcite . one example is shown in figure 1 . the frequent co-occurrence of identity words and the hateful label in the training set can bias the detectors (e.g., fine-tuned bert) to make false predictions during inference. as a result, it is of great need to comprehensively understand and identify the spurious correlations in hate speech detection and further mitigate the bias caused by spurious correlations. a growing amount of recent work has examined the spurious correlation in hate speech detection @xcite @xcite , and found that a large number of tokens that target minority groups are highly correlated with the hateful label @xcite . as a result, methods like masking and removal of such tokens with the help of human annotations @xcite have been proposed to mitigate these spurious correlations, together with finetuned large language models to generate artifacts to augment the training @xcite . despite these successes, they usually focus on identifying token-level spurious patterns in one specific dataset manually @xcite while neglecting spurious correlations beyond tokens (i.e., in sentence levels, shown in the right part of figure 1(b) ) and lacking comprehensive studies across different datasets and domains. moreover, these mitigations are often costly and timeconsuming as human annotation and fine-tuning large language models to generate numerous data are required. thus, a systematic study is needed to first automatically identify spurious correlations in a given hate speech dataset and then effectively and efficiently mitigate them. to fill in this gap, we first conduct a comprehensive analysis to discover spurious correlations from both token-level and sentence-level on 9 hate speech datasets in an automatic way. specifically, we use mutual information with domain knowledge to identify token-level spurious correlations such as identity words and leverage context-free grammar to investigate sentence-level highly correlated grammar patterns. we further propose a novel metric called relative spuriousness (rs) to verify the spuriousness of discovered spurious correlations based on its influences on model prediction. our analysis shows that token-level spurious patterns are usually more general that exist in almost all datasets while sentence-level spurious patterns are more dataset-specific. we further study how spurious correlations cause model bias from a casual perspective (as shown in figure 2 ). one affecting the distribution between plm's pretraining data and vulnerable identities, and another influencing the distribution between vulnerable identities and their context in hate speech datasets. to mitigate these two biases, we propose multi-task intervention (mti) and data-specific intervention (dsi) for bias mitigation. mti tries to mitigate biases from the pre-training corpus through training auxiliary tasks, while dsi focuses on eliminating biases originating from limited data and contains a framework that automatically detects, validates, and mitigates biases through a counterfactual generator. experiments conducted on 9 hate speech datasets and out-of-domain challenge sets demonstrate the effectiveness and robustness of our proposal. to summarize, our contributions are: (1) we automatically identify spurious correlations and comprehensively analyze them in hate speech detection from both token-level and sentence-level across 9 datasets. (2) we introduce a novel metric, relative spuriousness, to evaluate the spuriousness of identified spurious correlations at the sentence level. (3) we study the bias caused by spurious correlations from a causal perspective. (4) we propose two strategies, mti and dsi, to mitigate the biases and show consistent improvements in 9 datasets. ## related work debias hate speech detection recent works (yin and zubiaga 2021; @xcite @xcite @xcite @xcite @xcite have been studying the generalizability and biases for hate speech detection @xcite @xcite . for instance, prior work found that existing hate speech detection models are biased against african american vernacular english speakers @xcite and certain identity words are highly correlated with these hateful labels @xcite . a group of data augmentation methods @xcite @xcite ) are proposed to mitigate the biases. for instance, @xcite propose a simple data augmentation method using a finetuned gpt-2 model to generate 100k hate and non-hate data. ramponi and tonelli find highly correlated tokens and manually categorize them into different groups, and further mask or remove these annotated spurious artifacts. however, human annotations are not practical in large-scale or multi-platform scenarios. different from these prior works that mainly focus on data-specific biases and mitigation methods, our method automatically finds these shortcuts and proposes multi-task interventions on model levels. ## methods this section describes how we identify, understand, and mitigate the spurious correlations in hate speech detection (hs). ## conclusion this work investigates biases in hate speech detection from lexical and sentence levels. apart from the statistical correlation between artifacts and a specific label, we analyze the relative spuriousness