Mitigating Biases in Hate Speech Detection from A Causal Perspective
Authors: Zhehao Zhang
Jiaao Chen
Diyi Yang
Venue: EMNLP 2023 Findings (2023)
Keywords: Hate speech detection
Causal inference
Bias mitigation
TL;DR: We analyze biases in hate speech detection from a causal perspective and propose a new metric and method for bias mitigation based on two confounders.

Abstract: Nowadays, many hate speech detectors are built to automatically detect hateful content. However, their training sets are sometimes skewed towards certain stereotypes (e.g., race or religion-related). As a result, the detectors are prone to depend on some shortcuts for predictions. Previous works mainly focus on token-level analysis and heavily rely on human experts' annotations to identify spurious correlations, which is not only costly but also incapable of discovering higher-level artifacts. In this work, we use grammar induction to find grammar patterns for hate speech and analyze this phenomenon from a causal perspective. Concretely, we categorize and verify different biases based on their spuriousness and influence on the model prediction. Then, we propose two mitigation approaches including Multi-Task Intervention and Data-Specific Intervention based on these confounders. Experiments conducted on 9 hate speech datasets demonstrate the effectiveness of our approaches.

Mitigating Biases in Hate Speech Detection from A Causal Perspective
Zhehao Zhang* Jiaao Chen Diyi Yang*
Dartmouth College Georgia Institute of Technology Stanford University
zhehao. zhang. gr@dartmoth... edu jiaaochen@gatech. diyiyêcs stanford.edu
Abstract
Americans
Semitic
Philippines
Damocrats
Warning: This paper discusses and contains offensive or upsetting content.
Muslims
Palk
Nowadays, many hate speech detectors are built to automatically detect hateful content. However, their training sets are sometimes skewed towards certain stereotypes (e.g., race religion-related). As result, the de- tectors are prone to depend on some short- cuts for predictions. Previous works mainly focus on token-level analysis and heavily rely on human experts' annotations to iden- tify spurious correlations, which is not only costly but also incapable of discovering higher- level artifacts. In this work, we use gram- mar induction find grammar patterns for hate speech and analyze this phenomenon from causal perspective. Concretely, we categorize and verify different biases based on their spuriousness and influence on the model prediction. Then, we propose two mitigation approaches including Multi-Task Intervention and Data-Specific Intervention based these confounders. Experiments conducted on 9 hate speech datasets demon- strate the effectiveness of our approaches. The code is available at httpt::///// /tttps://githustt com/ Com/SALT-NLP BBias_Basaaasss Satealasaataa___eeeeeeeeaaaeeee Hate_Causal.
Japanese
Jews
Women
Cenvicrs
Gay
China
Blacks
Republican
Lesbian
Canadians Ruse on
Orthodox
Training Trumpis is narcissistic alive, his purposeful ignorance is stunning, Label: hazoful
(How the be God people when they tring
to kill them? Label: hateful
Inference
(I wonder if one these Bite
frrump wrong says
(Jew Predistion: hateful
never been great. Prediction: hateral
(b) Spurious
Figure I: (a) Word cloud frequent words in 9 widely used hate speech datasets and an example of how spurious correlations lead the model to make mistakes. The size of a word denotes its frequency of occurrence. (b) An example from Mandl et al., 2019 indicates that both token-level and sentence-level biases can introduce spurious correlations.
even to slightly differrnt datasets on the same task, which largely prevents them from being applied in real-world applications (Vidgen al., 2019). This might be due to the fact that current models are suffering from the spurious correlations between training data and labels (e.g., hateful) (Ramponi and Tonelli, 2022), which might lead to the biased treatment of vulnerable and minority groups such as African American Vernacular English Speakers and may exacerbate racism (Harris et al., 2022a). One example is shown in Figure 1. The frequent co-occurrence of identity words and the hateful la- bel in the training can bias the detectors (c.g.. fine-tuned BERT) to make false predictions during inference. As a result, it is of great need to com- prehensively understand and identify the spurious correlations in hate speech detection and further mitigate the bias caused by spurious correlations.
1 Introduction
Over the past few years, hate speech social me- dia has grown significantly in different forms. This causes serious consequences and societal impact on victims of all demographics (Mathew et al., 2021), largely affects their mental health (Saha et al., 2019) and even triggers real-world hate crimes (Re- lia et al., 2019). As a result, automated detection of hate speech becomes especially important and there have been numerous research studies con- ducted characterize and detect these hateful or toxic contents (Zhou et al., 2021: Lahnala al., 2022; Kotarcic et al., 2022).
However, current hate speech detection ap- proaches often fail to be robust and generalizable,
A growing amount of recent work has examined the spurious correlation in hate speech detection

---

(Zhou et al., 2021; Kennedy et al., 2020; Sap et al.. 2022), and found that large number of tokens that target minority groups are highly correlated with the hateful label (Bender et al., 2).1). As a result, methods like masking and removal of such tokens with the help of human annotations (Ramponi and Tonelli, 2022) have been proposed to mitigate these spurious correlations, together with finetuned large language models to generate artifacts augment the training (Wullach al., 2021; Hartvigsen et al., 2022). Despite these successes, they usually fo- cus on identifying token-level spurious patterns in one specific dataset manually (Bender et al., 2021) while neglecting spurious correlations beyond to- kens (i.e., in sentence levels, shown in the right part of Figure 1(b)) and lacking comprehensive stud- ies across different datasets and domains. More- over, these mitigations are often costly and time- consuming as human annotation and fine-tuning large language models to generate numerous data are required. Thus, a systematic study is needed to first automatically identify spurious correlations in a given hate speech dataset and then effectively and efficiently mitigate them.
originating from limited data and contains frame- work that automatically detects, validates, and mit- igates biases through counterfactual generator: Experiments conducted on 9 hate speech datasets and out-of oo-moo challenge sets demonstrate the effectiveness and robustness of our proposal.
To summarize, our contributions are: (1) We automatically identify spurious correlations and comprehensively analyze them in hate speech de- tection from both token-level and sentence-level across 9 datasets. (2) We introduce novel metric, Relative Spuriousness, to evaluate the spuriousness of identified spurious correlations at the sentence level. (3) We study the bias caused by spurious cor- relations from a causal perspective. (4) We propose two strategies, MTI and DSI, to mitigate the biases and show consistent improvements in 9 datasets.
2 Related Work
Debias Hate Speech Detection Recent works (Yin and Zubiaga 2021; Wiegand et al. 2019; Kennedy et al., 2020; Ma et al., 2020; Gehman et al., 2020; Sap et al., 2020; Dreier et al., 2022; Stanovsky et al., 2019; Sridhar and Yang, 2022;
To fill in this gap, we first conduct a compre- hensive analysis to discover spurious correlations from both loken-level and sentence-level on 9 hate speech datasets in automatic way. Specifically, we use mutual information with domain knowledge identify token-level spurious correlations such identity words and leverage context-free gram- mar to investigate sentence-level highly correlated grammar patterns. We further propose novel met- ric called Relative Spuriousness (RS) to verify the spuriousness of discovered spurious correlations based on its influences on model prediction. Our analysis shows that token-level spurious patterns are usually more general that exist in almost all datasets while sentence-level spurious patterns are more dataset-specifie. We further study how spu- rious correlations cause model bias from a casual perspective (as shown in Figure 2). one affecting the distribution between PLM's pretraining data and vulnerable identities, and another influencing the distribution between vulnerable identities and their context in hate speech datasets. To mitigate these two biases, we propose Multi-Task Interven- tion (MTI) and Data-Specific Intervention (DSI) for bias mitigation. MTT tries to mitigate biases from the pre- pre-training corpus through training auxil- iary tasks, while DSI focuses on eliminating biases
Thakur et al., 2023; Ziems et al., 2023) have been studying the generalizability and biases for hate speech detection (Talat al., 2018; AlKhamissi et al., 2022; Röttger et al., 2022: Bianchi al., 2022). For instance, prior work found that existing hate speech detection models are biased against African American Vernacular English Speakers (Harris et al., 2022b; Sap et al., 2019) and cer- tain identity words are highly correlated with these hateful labels (Bender et al., 2021; ElSherief et al., 2021a). A group of data augmentation methods (Sen et al. 2021; Sen et al., 2022; Hartvigsen et al. 2022) are proposed to mitigate the biases. For instance, Wullach et al. propose a simple data aug- mentation method using a finetuned GPT-2 model to generate 100k hate and non-hate data. Ramponi and Tonelli find highly correlated tokens and man- ually categorize them into different groups, and further mask or remove these annotated spurious artifacts. However, human annotations not prac- tical in large-scale or multi-platform scenarios. Dif- ferent from these prior works that mainly focus on data-specific biases and mitigation methods, our method automatically finds these shortcuts and pro- poses Multi-Tare Interventions on model levels.

---

2.1 Spurious Correlation in NLP
3.1.1 Token-level Spurious Correlations
One specific label (e.g., hateful) might be highly correlated with certain tokens (e.g., "women") the datasets ( (Wang et al., 2022; Ramponi and Tonelli, 2022). As result, models might learn to make biased predictions only based on those token-level correlations while neglecting the whole semantic meaning. In order identify such token-level spurious correlations, we utilize token- level PMI-based searching methods (Ramponi and Tonelli, 2022) to find biased words HS. Specif- ically, following Ramponi and Tonelli, 2022, we compute the PMI using the equation described in Appendix A.1 between every word and the hateful label. After that, we select the highest correlated words for further investigation across 9 datasets. This led large of identity-related words (over 80, shown in Figure an example) that are highly correlated with the hateful labels while the words themselves are neutral. We also observe that such identity words are often common across dif- ferent datasets. Thus, we treat these identity words as general token-level spurious correlations'.
Increasing attention has been focused on spurious correlations in NLP tasks (Tu et al.. 2020; McCoy et al., 2019; Jia and Liang, 2017; Niu et al., 2020; Wang and Culotta, 2020; Vaidya et al., 2020; Clark et al., 2020; Du et al., 2021). A line of work tries evaluate the models' robustness pre-defined shortcuts by proposing challenge test datasets (Mc- Coy et al., 2019; Rosenman al., 2020; Röttger et al., 2021) or finding salient words (Simonyan
et al., 2013; Pezeshkpour et al., 2022; Shrikumar
et al.. 2017: Han et al., 2020). Another line of work also intends to verify the spuriousness of ex- tracted features. Gardner et al., 2021 suggests that all correlations between labels and low-level fea- tures are spurious. Eisenstein, 2022 claims these correlations will naturally appear in the majority of classification tasks and that domain knowledge is required to identify any correlations that may be harmful. Joshi et al. define the spuriousness based on the sufficiency of the feature and counterfactual intervention. However, their definition is based on an unbiased classifier to model the probability of sufficiency, which could not be practical in real applications. Besides, they do not use this metric find spurious artifacts. Instead, they use domain knowledge to pre-define the potential feature. Fur- thermore, previous analyses (Zhang et al., 2018)((() of biases' influences on model predictions are limited. To this end, we propose a novel metric named Rela- tive Spuriousness (RS) which considers the biases' influence the model's decision decision-making making.
3.1.2 Sentence-level Spurious Correlations
Beyond token-level identity words, certain sentence-level patterns might also be highly cor- related with specific labels that might be spuri- ous correlations. For example, in a platform with discussion on politics, patterns such as The wall should and The wall is are usually connected with discontent with the US border wall. A large num- ber of such speeches grammar patterns are completely neutral. To au- tomatically discover these sentence-level spurious correlations, following Friedman et al., 2022, we induce a grammar for HS training data and ob- tain the maximum likelihood trees in an unsuper- vised manner. Specifically, we use probabilistic context-free free grammar (PCFG), which contains the distinguished start symbol S, terminal symbols V (words), non-terminal symbols N and the rules of the form A -> E R, where A E A and B E N VU We use the same parameterization and training methods as Kim et al., 2019. We then compute the mutual information between grammar patterns (non-terminal root) and labels and find
hateful. However, these
3 Methods
This section describes how we identify, under- stand, and mitigate the spurious correlations in Hate Speech Detection (HS).
3.1 Identifying Spurious Correlations
Current HS models often suffer from spurious cor- relations (Ramponi and Tonelli, 2022; Bose et al., 2022; Wang et al., 2022): they tend to utilize pre- diction patterns that hold for the majority examples but do not hold in general. This might cause the model to be biased in applications (Ramponi and Tonelli, 2022; Gardner al., 2021). In this section, we identify potential spurious correlations from two levels across multiple datasets.
'Note that this identification process is automatic and we manually inspect to ensure its quality. To analyze such spu- rious correlations more comprchensively and reduce noises. we adopt another module (a pre-trained NER model) to ensure these highly-correlated words are identity-rolated.

---

Po
Po
G2
C
P
(a) Original SCM
(b) Multi-Task Intervention (MTI) (c) Data Pata-Specifii Intervention (DSI)
Figure 2: SCM (Structural Causal Mode) for HS detection with vulnerable identities. We have omitted the unmeasured variable U. for each variable for brevity. P - L: Using corpus P to pretrain the language model L. (J, C) X: Vulnerable objectives I (e.g., identity words) and their contexts C' constitute the input data X. XI, Y: Inpur the language model I. with data X to output the prediction Y. (a) Two confounders G1 and G2 bias the generation distribution of pretraining corpus and HS data. (b) MTI intervenes with the bias from C1. where Po represents the training corpus of auxiliary tasks. (c) DSI further intervenes with the bias from G2. where Co denotes the newly generated context. To analyze grammar patterns biases, only need change and other parts of the SCM remain the same.
highly-correlane ones. We define the patterns in terms of grammar subtrees. After we apply the above method to 9 hate speech datasets, we find that different datasets have different highly corre- lated patterss (Section 4.5.1 as examples). As result, such patterns can be considered potential data- sata-specificc spurious correlations.
great impact on local probability and almost no impact on global probability. Therefore, it has large Di( y) and small (z y), causing higher RS. Compared with the prior work (Joshi et al., 2022), we consider a feature's s relative influence on the model's prediction locally and globally, filling in the gap of considering features other than Vi, which is reasonable and practical metric to find spuri- ous features. We select the features that have RS higher than certain threshold as data-specific bi- ases. The following sections investigate the origin of the above biases and propose two intervention approaches to mitigate them.
Spuriousness Validation After discovering these sentence-level spurious correlations candi- dates, we further validate their spuriousness to identify clean spurious correlations. We define Relative Spuriousness (RS) based on the necessity of feature's 's influence on the model's prediction. Our definition depends on two properties of spu- rious features: feature's s existence significantly impacts specific label on biased model, and it is not important for unbiased model ideally. Definition (Relative Spuriousness (RS) The RS feature n; for the label is:
3.2 Understanding Spurious Correlations
With the identified spurious correlations in HS, it is then of great need to study how they might be generated and how the models might suffer from them by making biased predictions. In this section, we visualize the relations between spurious correla- tions and the caused bias from a causal perspective (Feder et al., 2021; Hardt et al., 2016; Kilbertus et al., 2017; Vig et al., 2020). Specifically, we use Structural Causal Model (SCM) (Pearl et al., 2000) which is a conceptual model that describes the causal mechanisms of system, to recognize the confounders which variables that influence both the dependent variable and independent vari- able, causing spurious correlations. We use di- rected acyclic graphs (DAGs) G = =VV to de- scribe the causal relationships between different variables. V refers to variable nodes and edges denote causal relation function f.
Pns = Di(x;, y) - (y( v),
Di(2i, 8) = Pi(Y = y/X; = Fj, =
- (X x;) = y|X; = Ez, Y y),
(y(04, v) = PO(Y - x| = ε1, Y,,Y y
- (xx)) = ulx, = 24, -
Here Pt indicates the local probability, where we use the HS detector's softmax output trained on a single HS dataset to model it. Pg represents the global one, where we use the average response of models trained every HS dataset. Note that do not the model trained on all datasets for the class-imbalance problems. D; and D, mean the local and global prohability difference between HS examples with and without feature 2;, respec- tively. Intuitively, for spurious features, it can have
We visualize the SCM for HS detection described in Figure 2a, where identities I and the correspond- ing context C' constitute the input data X. The

---

General Blases (Identity)
Deport JewScor and Muslims.co
Prompt Generate non-hateful sentences containing
Counterfactual Data Jews have 8
Majority-Vote Filter
Jews
Model
PMI-based Bias Search
Model 2
GPT-3
Sata-Specific Blases
HS dataset
Model n
Relative Spuriousness Validation
Prompt Paraphrase the original sentence without
Counterfactual
Data face
oooo Grammar Induction
corporate greed
Trump is the face of...
using Truup
Figure 3: Our proposed pipeline (DSI) is to find spurious features and mitigate these biases. The blue part indicates the framework token-level biases (identity). The yellow part represents framework data-specific biases. indicates intersection. For all the biases, majority-voting filter. We use the counterfactual data along with the original training data to train the HS detector.
off-the-shell GPT-3 the counterfactual generator followed by
evaluation result Y (accuracy Fi score) is gener- ated by PLM L and input X.
corpus P and vulnerable objectives I (caused by G1) in order to resolve two kinds of biases in Sec- tion 3.1. By training with MTI objectives on a wide range of HS, we expect that the original dis- tribution connection of P and I can altered. Secondly, most HS is composed of unregulated sentences. MTI can make the PLM get a more robust representation for HS. Specifically, we in- troduce two auxiliary tasks PLM to alleviate the bias from the pre-training corpus: (1) Masked Language Modeling (Devl) et al., 2019): we randomly mask 15 tokens from hateful sen- tences from HS datasets and let PLM predict the tokens on the masking position with language mod- eling objective. (2) Multi-Task Learning (MTL): We define different tasks different hate speech datasets D; [z]. and train on all the datasets [D,] (see Section 4.1 for details on datasets) simultancously using the same PLM and different classification heads.
Confounders Two confounders G1 and G2 in- fluence their correlated variable's independence of generation distribution, causing spurious correla- tions. The two confounders are as follows.
G1: is the confounder which affects the dis- tribution of both P and vulnerable identities I. For example, the unbalance appearance frequencies of different races of people in the pretraining data for the pretrained language models may constitute this confounder.
G2: influences the distribution of vulnerable objectives and their context C. The training data's s source may compose this confounder. For instance, de Gibert et al. 2018 collect data from Stormfront, a white supremacist forum, where mentions of people of color (POC) usu- ally occur in hateful contexts.
3.3 Mitigating the Bias in HS
Data- -Secifpec Intervention (DSI) To mitigate the influence of G2 and inspired by current progress in counterfactual lcarning (Zeng et al., 2020; Sen al., 2021; Eisenstein. 2022: Garg et al., 2019; Davani al., 2021), we propose a counterfactual generator for DSI. Recent works (Mishra et al., 2022; Ouyang et al., 2022) discover that the power- ful Large Language Models (LLMs) (Brown et al.,
Based on the above two confounders that cause spurious correlations, then propose two inter- vention techniques to mitigate the bias in HS.
Multi-Task Intervention (MTI) The motivation for MTI is twofold. Firstly, MTI tries to mitigate the spurious correlations between PLM's training

---

Method
MM
WSF
Fox
HASOC
ETHOS
TSA
AHS
AOH
7H
Average
davinci-002 zero
48.64
87.13
55.81
66.67
34.15
75.00
44.44
80.00
80.81
63.63
davinci - 002 few
41.86
86.87 45.61
66.67
77.19
31.58
60.00
75.00
70.97
61.75
davinci-003 zero
48.86
59.57
74.43
68.25
84.48
92.42
63.06
78.51
40.26
67.76
davinci-O00 003 few
53.13
73.33
61.22
63,28
86.29
86.61
73.33
79.48
35.21
67.98
ChatGPT zero
52.26
63.94
46.54
52.38
54.73
69.81
33.46
43.33
17.89
48.26
ChatGPT few
33.95
40.46
35.44
53.13
55.50
89.57
40.95
46.86
34.78
47.84
Finetune
48.77
77.49
71.22
65.27
77.77
88.48
93.33
88.42
67.33
75.34
MLLM i
43.12
77.49
70.88
64.49
76.18
87.94
93.21
89.67
68.67
74.63
+ +MTL f
61.63
89.65
84.77
80.28
76.44
93.58
96.06
93.89 79.21
83.95
Table I: Experiment results (macro-Fi Fj score) of two GPT baselines and BERT finetune with two MI.I methods on 9 HS datascts. All Finetune results are averaged over three runs using three random seeds. GPT-3's performances vary greatly across datasets. MTI. has an 8.61 Fi improvement on average compared to Finetune. i mcans our method.
Method
MM
WSF
Fox
HASOC
ETHOS
TSA
AHS
AOH
ZH
Average
Back- -
60.65
90.85
83.41
79.73
76.86
92.67
94.39
93.40
78.82
83.42
Span-cut
58.93
91.10
84.25
81.71
88.85
92.86
95.58
92.30
79.53
85.01
Token-cut V
59.10
91.83
83.86
81.30
88.73
92.57
95.69
92.46
79.24
84.98
Feature-cut
58.54
91.37
86.36
81.13
88.89
92.66
96.44
92.83
80.33
85.39
AEDA
62.71
90.12
82.14
77.50
74.02
92.54
97.23
93.79
76.76
82.98
Masking
58.31
90.65
83.05
80.47
87.71
92.06
95.11
93.37
80.49
84.58
Removal
59.78
90.58
84.68
81.13
89.71
92.95
96.18
91.98
80.94
85.33
DSI
62.71
91.65
87.04
81.39
90.76
93.17
96.52
93.82
80.65
86.41
Table 2: Experiment results (macro-F, score) of token-level( )). sentence-level (4)), and hidden-level (V) data augmentation methods along with two token-level debias methods and DSI on HS datasets. All these methods are bused on MTI. All results are averaged over three runs using three random seeds. We conduct the significant tost following Berg- Torg-Kirkgn Kirkpatrick ct al., 2012 and the average estimate of p-value is 0.003 (< 0.01) between DSI and SOTA baselines, demonstrating the significunt diffcrences.
2020; Touvron et al., 2023; OpenAI, 2023b) with instruction prompt can have outstanding perfor- mances in many NLP tasks. Therefore, we use off-the-shell GPT-3 with instructions as the back- bone of the counterfactual generator. The overall framework of DSI illustrated Figure 3. Af- ter we find spurious features using the method de- scribed in Section 3.1.2, we employ different in- structional prompts for general biases (identity) and data-specific biases. The details on prompt design can be found in Appendix A.4. In practice, we find that the counterfactual data may not always remain the same label as described in the prompt. Thus we further conduct a majority voting with the models trained on different HS datasets. We use a majority vote to make sure the generated counterfactual data are non-hateful, which is a way to further control the quality of the generated data. We then train the HS detector with the original training data plus the counterfactual ones.
4
Experiment
4.1 Datasets and Setup
We conduct experiments 9 datasets 2 includ- ing Multimodal Meme Dataset (text) (Suryawan- shi al.), Hate speech dataset from a white supremacist forum (de Gibert et al., 2018), Fox- News-User-Come Comments (Gao and Huang, 2017). HASOC19 (Mandl et al., 2019), ETHOS (Mollas et al., 2022), Twitter Sentiment Analysis (TSA), Anatomy of Online Hate (AOH) (Salminen et al., 2018 2018), Hate Speech Twitter (Zhang et al., 2018) and Hate-Offennive (AHS) (Davidson et al., 2017). Following previous work (Ramponi and Tonelli, 2022; EISherief et al., 2021b), we adopt BERT- based-cased as the detector backbone model. To better avoid the issue of class imbalances, macro F1 as the primary metric (other metrics such as RS are also explored in Section 4.5.2). More im- plementation details can be found in Appendix A.5.
"Statisties along with generated counterfactual data can be found in Appendix A.2.

---

4.2 Baselines
Methods
Fj score
MTL
82.53
To evaluate the effectiveness of MTI, we compare them with the following baselines": Finetune (De- vlin et al., 2019): finetunes the BERT for hate speech detection (Tran al., 2020). text-davinci- DD2texx--davinn--00///aaaCCC zero/few (Brown et al., 2020; OpenAl, 2023a): uses text-davinci- 002/eex-davinii-033/ChaaaaCCC
+AEDA
81.87
+ BBack-Trans
81.19
+Span-cut
83.89
+ Token-cut
82.54
+ Feature-cut
82.55
along with text prompt to model the classification problem into conditional generation problem (with few exam- ples). MLM Finetune: first conducts Masked Language Modeling (MLM) (Devlin al., 2019) hateful speeches get more robust represen- tation before finetuning. MTL + Finetune first conducts Multi-Task-Leeariing (MTL) before fine- tuning. Specifically, we train 9 datasets simultane- ously with separate label space in MTL.
+Masking
81.87
+ Removal
79.01
+DSI (Ours)
88.49
Table 3: Experiment results of different methods on the
challenge set. Our proposal DSI can have a more noticeable improvement over other baselines on OOD analysis.
the finetune- based model can learn these abnormal structures through training data. Furthermore, for the two MTI methods, MTL can further consis- tently improve the performance (8.61 F1 score), indicating the importance of getting a robust rep- resentation of HS. Besides, MTL is orthogonal other data-specific approaches (DA, data-specific debias methods). Therefore, the following experi- ments Table are all based on MTL.
To evaluate the effectiveness of DSL, we use the following methods baselines based MTL Finetune (best models): AEDA (Karimi al., 2021): randomly inserts punctuation marks into the original text for data augmentation. Back translation (Edunov et al., 2018): first translates sentences into certain intermediate languages and then translates them back. Cutoff (Shen et al., 2020): masks tokens, spans, and features in the input space. Artiffacts-removal masking (Ram- poni and Tonelli, 2022): removes (or masks) any occurrence of spurious lexical artifacts (based on PMI) for training and validation data.
From Table 2, observe that our proposed DSI outperform other baselines average and have noticeable improvement (2.46 Fi score) over MTL. Besides, find that token-level DA meth- ods can even impair the overall performance, in- dicating that these methods are not as effective as in other tasks. On the other hand, three hidden- level DA methods do have positive impact on the performance. Besides, previous debias methods such as artifact removal or masking also boost the performance but the improvement is subtle.
4.3
Main results
The main experiment results are shown in Table and 2, respectively. As shown in Table 1, different
observe that LLMs' performances datasets vary greatly. Besides, the performance of few-shot LLMs can not outperform zero-shot con- sistently, which different from other tasks. As a result, these results indicate that the off-the-sher
4.4 Generalization Analysis
Joshi al. claim that building "challenge set" to see the intervention of the input cause model pre- dictions to vary expectedly
LLMs is not always a good HS detector, which is consistent with the results in (Ziems et al., 2023). One of the reasons behind this may be that HS's grammatical structure may not conform the stan- dard grammatical norms. For example, the authors HS online hardly use complete sentences. As a result, HS datasets contain large proportion of such unregulated sentences, which may confuse the off-the-shell GPT model. Among the LLMs we use, text-davinci-003 performs the best while Chat- GPT has the worst performance. On the other hand,
standard method of testing a model's robustness. Hence, to verify the robustness of our method, we construct an OOD challenge set using non-hateful counterfactual data generated by GPT-3 and hateful data from CO- NAN (Bonaldi, Helena and Dellantonio, Sara and Tekiroğlu, Serra Sinem and Guerini, Marco, 2022). Details on OOD datasets can be found in Appendix A.3. Given MTL's strong performances in Table 1, we evaluate other baselines based on MTI., as shown in Table 3. Our proposed DSI outperforms other methods by a larger margin than that in Ta-
IImplementation details can be found in Appendix A.6.2

---

ble 2, indicating the effectiveness of our method's robustness and great generalization ability.
baseline model that solved by our approach. As a result, we conduct the following case study on one of our datasets used in this work, i.e., the white supremacist forum dataset (de Gibert et al., 2018). We randomly sample four false positive sentences where the baseline model makes mistakes in the test set. As shown in Table 4, the baseline model can not conrectly identify some complex non-hateful ex- amples containing identity words. For example, the sentences containing ethnicity words (e.g., jews, Blacks, hebrews, and Arabs) confuse the base- line detector. Besides, gender-related words (e.g.. women) may also make the model more likely to classify the sentence as hateful. However, these sentences are non-hateful based on their neutral contexts. Inspired by saliency methods (Simonyan et al., 2013; Sundararajan et al., 2017; Smilkov et al., 2017: Balkir et al., 2022), we compute the probability differences between original sentences and identity-masked sentences to better interpret the model prediction. The average differences in the above samples are 0.76 and 0.04 for the base- line detector and our proposal, indicating that the impact of these identity words has been consider- ably reduced by our approach.
4.5 Deep Dive of Sentence-level Biases
This part takes a deep dive into these sentence-level biases via qualitative analyses and visualization of their relative spuriousness distributions.
4.5.1 Potential Data-Specific Biases
After conducting grammar induction and investigal- ing 180 grammar patterns, we observe the follow- ing categories with high PMI with the hateful label: (I) Absolute expression (8) A large number of HS sentences contain absolute statements, where patterns like all the, all other, any other, etc. fre- quently occur. (2) Hashtag before 'bad' words: (7.2%) In online datasets, hashtags are common patterns. We observe that hashtags before some bad words (e.g., # dickhead, # Liar, # Thief etc.) are highly correlated with hateful labels. (3) Aggres- sive actions (6.7% ): A large number of HS contain radical action words (e.g., anything, kill your- self. etc.). (4) All capitalization (2.8%); )) All-caps patterns such as FUCK YOU and HAVE NEVER are more likely to be found in hateful sentences. In addition to the aforementioned types, there are other types of spurious patterns without cohesive themes in HS (see Table 6 for some examples). We also find that different datasets have different pat- terns. However, not all of the above patterns spurious. The following section illustrates the RS distribution of these data-specific biases.
Conclusion
This work investigates biases in hate speech detec- tion from lexical and sentence levels. Apart from the statistical correlation between artifacts and a specific label, we analyze the relative spuriousness of the feature based on its impact on local and global models. We find that most highly-correlated pattern features do not have high RS. After that, analyze the generation process of HS biases from a causal view. We identify two confounders that cause the biases and propose Multi-Task In- tervention from the model level and Data-Specific Intervention from the data level to mitigate them. Noticeable performance improvements on nine HS datasets and label-balance challenge set indicate the effectiveness and robustness of our approach.
4.5.2 Bias Distribution Analysis
For potential data-specific biases, we use RS de- scribed in Scction 3.1.2 to validate their spurious- ness. We visualize the distribution of these bi- ases based on our proposed RS in Figure 4 in the Appendix. We find that most data-specific pat- terns' RS is positive, indicating that these highly- correlated artifacts do make the model more likely to predict a particular label. However, the experi- ment result shows that most of their RS is less than 0.2. As a result, most of the data-specific biases' impact on the HS detector's ss output is minimal. This is because most of these biased patterns do not con- tain hateful semantics. Although they are highly correlated with a specific label statistically, PLM can ignore them some extent during inference.
Limitations
Our work is subject to few limitations. First, our experiments are limited to English datasets. However, multilingual hate speeches are also fre- quently found on many social media platforms. There are still many challenges in languages other than English, especially some minor languages. A thorough examination of our methods' effec-
4.6
Case Study of Token-Level Biases
To further analyze our proposal's effectiveness on identity biases, we analyze the errors made by the

---

Examples
Before
After
This is going to stir up a lot of yellow hatred for the hebrews
hateful
non-hateful
I wonder if one of these guys is a Jew
hateful
non-hateful
What you say about Blacks Arabs and Jews are LIES.
hateful non-hateful
Are they talking about the African women with huge lip plates?
hateful non-hateful
Table 4: Case study on white supremacist forum dataset. The identity words are highlighted. Before the intervention, the model makes mistakes on these non-hateful examples. Our proposed intervention can address such problems.
tiveness in languages other than English is nec- essary, which we leave as future work. Second, we use BERT-base-cased as the PLM backbone for most HS detectors following Ramponi and Tonelli, 2022, and we add GPT-3, ChatGPT and coxt-davincii-nn
References
Badr AlKhamissi, Faisal Ladhak, Srini Tyer, Ves Stoy- anov, Zornitsa Kozareva, Xian Li, Pascale Fung, Lam- bert Mathias, Asli Celikyilmaz, and Mona Diab. 2022. Token: Task decomposition and knowledge infusion for few-shot hate speech detection.
as baselines. Other PLMs of different scales architectures' robustness on various biases needs be verified. Moreover, we only examine and mitigate biases explicit HS datasets following previous works. However, bi- ases in implicit HS (EISherief et al., 2021b) are also important and we leave it as future work. In addition, our proposed RS and mitigation methods are applicable to any other text classification tasks, which we leave future work. Besides, diverse prompt designs for counterfactual generators also need to be validated in future works. In this work, we only focus on single-turn HS detection tasks following previous works without considering user contexts (Yu et al., 2022), which is another practi- cal setting. Finally, we find that some data is falsely annotated or based on different datasets. We do not modify them for fair comparisons. However, it is necessary to uniform the criteria for HS and correct problematic annotations. For token-level biases, al- though a large number of them are identity-related there are still other highly-correlated tokens. We do not investigate them and leave it as future work.
Esma Balkir, Isar Nejadgholi, Kathleen Fraser, and Svet- lana Kiritchenko. 2022. Necessity and sufficiency for explaining text classifiers: A case study in hate speech detection. Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, pages 2672-26622 2686, Seattle, United States. Association for Computational Linguistics.
Emily M. Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dan- gers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, EAceTT '21, page 610-623, New York, NY, USA. Association for Computing Machinery.
Taylor Berg- King-Kikkattick, David Burkett, and Dan Klein. 2012. An empirical investigation of statistical signifi- cance in NLP. In Proceedings of the 2012 Joint Confer- ence on Empirical Methods in Natural Language Pro- cessing and Computational Natural Language Learn- ing. pages 995-1005, Jeju Island, Korea, Association for Computational Linguistics.
Federico Bianchi, Stefanie Anja Hills, Patricia Rossini, Dirk Hovy, Rehekah Tromble, and Nava Tintarev. 2022. "it's not just hate": A multi-dimensional perspective on detecting harmful speech online.
Ethics Statement
Bonaldi, Helena and Dellantonio, Sara and Tekiroglu, Serra Sinem and Guerini, Marco. 2022. Human- Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering. arXiv preprint arXiv:2211. 1.03433.
Our proposals in the research aim to mitigate biases and accurate detection of HS. The main datasets utilized in the study are open-access and publicly available. The offensive terms and identity-related words included as examples are mainly used to help researchers analyze the models more effectively. We do not involve annotators in this process given the sensitive nature of this work, and to reduce exposure of hateful content human participants. We also add a content warning in the beginning of this paper to warn readers.
Tulika Bose, Nikolaos Aletras, Irina Illina, and Do- minique Fohr. 2022. Dynamically refined regulariza- tion for improving cross-corpora hate speech detection. In Findings of the Association for Computational Lin- guistics: ACL 2022, pages 372-382, Dublin, Ireland. Association for Computational Linguisties.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda

---

Askell, Sandhini Agarwal, Ariel Herbert-Voor Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Πya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Asso- ciates, Inc.
In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489- 500, Brussels, Belgium. Association for Computational Linguisties.
Jacob Eisenstein. 2022. Informativeness and invariance: Two perspectives on spurious correlations in natural lan- guage. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies. pages 4326-4331, Seattle, United States. Association for Computational Linguistics.
Ke-Li Chiu, Annie Collins, and Rohan Alexander. 2021. Detecting hate speech with gpt-3.
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaish- navi Anupindi, Jordyn Scybolt, Munmun De Choudhury, and Diyi Yang. 2021a. Latent hatred: benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345-363.
Christopher Clark. Mark Yatskar, and Luke Zettlemoyer. 2020. Learning to model and ignore dataset bias with mixed capacity ensembles. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2020, pages 3031-3045, Online. Association for Computa- tional Linguistics.
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaish- navi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021b. Latent hatred: benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345-363, Online and Punta Cana, Dominican Republic. Association for Computa- tional Linguistics.
Aida Mostafazadeh Davani, Ali Omrani, Brendan Kennedy, Mohammad Atari. Xiang Ren, and Morteza Dehghani. 2021. Improving counterfactual generation for fair hate speech detection.
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the 1thh International AAAI Conference on Web and Social Media, ICWSM 17, pages 512-515.
Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Dous Doughty Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E, Roberts, Brandon M. Stewart, Victor Veitch, and Diyi Yang. 2021. Causal inference in natural language pro- cessing: Estimation, prediction, interpretation and be- yond. CoRR, abs/2109.00725.
Ona de Gibert, Naiara Perez, Aitor García-Pare Pablos, and Montse Cuadros. 2018 Hate Speech Dataset from White Supremacy Forum. In Proceedings of the 2nd Workshop on Abusive Language Online (AIW2), pages 11-20, Brussels, Belgium. Association for Computa- tional Linguistics.
Wikimedia Foundation. Acl 2019 fourth conference on machine translation (wmt19), shared task: Machine translation of news.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. 2019, BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, Minneapo- lis, Minnesota. Association for Computational Linguis- ties.
Dan Friedman. Alexander Wettig, and Danqi Chen. 2022. Finding dataset shortcuts with grammar induc- tion. In Proceedings of the 2022 Conference Empir- ical Methods in Natural Language Processing, pages 4345-4363
Lei Gao and Ruihong Huang. 2017. Detecting online hate speech using context aware models. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANIP 2017, pages 260- 266, Varna, Bulgaria. INCOMA Ltd.
Sarah Dreier, Emily Gade, Dallas Card, and Noah Smith. 2022. Patterns of bias: How mainstream media opera- tionalize links between mass shootings and terrorism. Political Communication, 39:1 1-21.
Matt Gardner, William Merrill, Jesse Dodge, Matthew Peters, Alexis Ross, Sameer Singh, and Noah A. Smith. 2021. Competency problems: On finding and removing artifacts language data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1801-1813, Online and Punta Cana, Dominican Republic. Association for Computational Linguisties.
Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. 2021. Towards interpreting and mit- igating shortcut learning behavior of NLU models. In Proceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics: Human Language Technologies. pages 915- 929, Online. Association for Computational Linguistics.
Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019. Counterfactual fair- ness in text classification through robustness.
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding aranssranntation at scale.

---

Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityrompts: Evaluating neural toxic degeneration in language mod- els. In Findings of EMNLP.
Akbar Karimi, Leonardo Rossi, and Andrea Prati. 2021. AEDA: An easier data augmentation technique for text classification. In Findings of the Association for Compu- tational Linguistics: EMNLP 2021. pages 2748-2754, Punta Cana, Dominican Republic. Association for Com- putational Linguistics.
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning, In Proceedings of the 34th International Conference on Machine Learning- Volume 70, ICML' 17, page 1243-1252. JMLR.org.
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Da- vani, Morteza Dehghani, and Xiang Ren. 2020. Con- textualizing hate speech classifiers with post-hoc expla- nation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5435-5442, Online. Association for Computational Lin- guisties.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Short Papers), pages 107-112, New Orleans, Louisiana. Association for Computational Linguistics.
Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding discrimination through causal reasoning. In Advances Neural Infor- mation Processing Systems, volume 30, Curran Asso- ciates, Inc.
Xiaochuang Han, Byron C. Wallace, and Yulia Tsvetkov. 2020. Explaining black box predictions and unveiling data artifacts through influence functions. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5553-5563, Online. Association for Computational Linguisties.
Yoon Kim. Chris Dyer. and Alexander Rush. 2019. Compound probabilistic context-free grammars for grammar induction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguis- tics, pages 2369-2385, Florence, Italy. Association for Computational Linguistics.
Moritz Hardt, Eric Price. Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc.
Ana Kotarcie Dominik Hangartner, Fabrizio Gilardi, Selina Kurer, and Karsten Donnay. 2022. Human-int -ntthe---ttee----- loop hate speech classification in a multilingual context.
Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, and Diyi Yang. 2022a. Exploring the role of grammar and word choice in bias toward african american english (aac) in hate speech classification. In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAceT '22, page 789-798, New York, NY, USA. Association for Computing Machinery.
Allison Lahnala, Charles Welch, Béla Neuendorf, and Lucie Flek. 2022. Mitigating toxic degeneration with empathetic data: Exploring the relationship between toxicity and empathy. In Proceedings of the 2022 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, pages 4926-4933 Seattle, United States. Association for Computational Linguisties.
Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, and Diyi Yang. 2022b. Exploring the role of grammar and word choice in bias toward african american english (aac) in hate speech classification. In 2022 ACM Conference Fairness, Accountability, and Transparency, pages 789-798.
Xinyao Ma. Maarten Sap, Hannah Rashkin, and Yejin Choi. 2020. Powertransformer: Unsupervised con- trollable revision for biased language correction. In EMNLP.
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generater dataset for adversarial and implicit hate speech detection. In Pro- ceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pages 3309-3326, Dublin, Ireland, Association for Computational Linguistics.
Thomas Mandl, Sandip Modha, Prasenjit Majumder. Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identifica- tion in indo-european languages. In Proceedings of the 11th Forum for Information Retrieval Evaluation, FIRE '19, page 14-17, New York, NY, USA. Association for Computing Machinery.
Robin Jia and Percy Liang. 2017. Adversarial exam- ples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Meth- ods in Natural Language Processing, pages 2021-2031 Copenhagen, Denmark. Association for Computational Linguistics.
Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 35(11))14867-14875
Nitish Joshi, Xiang Pan, and He He. 2022. Are all spurious features natural language alike? an analysis through a causal lens.
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristies in natural language inference. In Proceedings of the

---

57th Annual Meeting of the Association for Computa- tional Linguistics. pages 3428-3448, Florence, Italy. Association for Computational Linguisties.
2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 3702-3710, Online. Association for Computational Linguisties.
Swaroop Mishra, Daniel Khashabi, Chiua Baral, Yejin Choi, and Hannaneh Hajishirzi. 2022. Reframing in- structional prompts to GPTk's 's language. In Findings of the Association for Computational Linguistics: ACL 2022. pages 589-6889998 612. Dublin, Ireland. Association for Computational Linguistics.
Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguis- tics and the 11th International Joint Conference on Nat- ural Language Processing (Volume 1: Long Papers), pages 41-58, Online. Association for Computational Linguisties.
Toannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2022. ETHOS: a multi-label hate speech detection dataset. Complex & Intelligent Systems.
Paul Röttger, Debora Nozza, Federico Bianchi, and Dirk Hovy. 2022. Data-efficient strategies for expanding hate speech detection into under-resourced languages.
Xing Niu, Prashant Mathur, Georgiana Dinu, and Yaser Al-Onaizan. 2020. Evaluating robustness to input per- turbations for neural machine translation. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages Association for Computational Linguisties.
Koustuv Saha, Eshwar Chandrasekharan, and Munmun De Choudhury. 2019. Prevalence and psychological effects of hateful speech in online college communities. In Proceedings of the 10th ACM Conference on Web Science, WebSci '19, page 255-264, New York, NY, USA. Association for Computing Machinery.
Online.
OpenAl. 2023a. Chatgpt.
OpenAl. 2023b. Gpt-4 technical report.
Joni Salminen, Hind Almerekhi, Milica Milenković, Soon-gyo Jung, Jisun An, Haewoon Kwak, and Bernard Jansen. 2018. Anatomy of online hate: Developing taxonomy and machine learning models for identifying and classifying hate in online news media. Proceedings of the International AAAI Conference on Web and Social Media, 12(1).
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal. Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Train- ing language models to follow instructions with human feedback.
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019, The risk of racial bias in hate speech detection. ACL.
Judea Pearl et al. 2000. Models, reasoning and in- ference. Cambridge, UK: CambrridgeUnivvesisyPeesss 19(2).
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In ACI..
Pouya Pezeshkpour, Sarthak Jain, Sameer Singh, and Byron Wallace. 2022. Combining feature and instance attribution to detect artifacts. In Findings of the Associ- ation for Computational Linguistics: ACL 2022, pages 1934-1946, Dublin, Ireland. Association for Computa- tional Linguistics.
Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 5884-5906, Seattle, United States. Association for Computational Linguis- ties.
Alan Ramponi and Sara Tonelli. 2022. Features or spurious artifacts? data-centric baselines for fair and robust hate speech detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, pages 3027-3040, Seattle, United States. Association for Computational Linguistics.
Indira Sen, Mattia Samory, Fabian Flöck, Claudia Wag- ner, and Isabelle Augenstein. 2021. How does coun- terfactually augmented data impact models for social computing constructs? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 325-344, Online and Punta Cana, Dominican Republic. Association for Computational Linguisties.
Kunal Relia, Zhengyi Li. Stephanic H Cook, and Rumi Chunara. 2019. Race, ethnicity and national origin- based discrimination in social media and hate crimes across 100 us cities. In Proceedings of the International AAAI Conference on Web and Social Media, volume 13, pages 417-427.
Shachar Rosenman, Alon Jacovi, and Yoav Goldberg. 2020. Exposing Shallow Heuristics of Relation Extrac- tion Models with Challenge Data. In Proceedings of the
Indira Sen, Mattia Samory, Claudia Wagner, and Is- abelle Augenstein. 2022. Counterfactually augmented data and unintended bias: The case of sexism and hate

---

speech detection. In Proceedings of the 2022 Confer- ence of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies, pages 47 4716-4726, Seattle, United States. Association for Computational Linguistics.
Thanh Tran, Yifan Hu, Changwei Hu, Kevin Yen, Fei Tan, Kyumin Lee, and Se Rim Park. 2020. HABER- TOR: An efficient and effective deep hatespeech detec- tor. In Proceedings of the 2020 Conference on Empirt- cal Methods in Natural Language Processing (EMNLP), pages 7486-7502, Online. Association for Computa- tional Linguistics.
Dinghan Shen. Mingzhi Zheng, Yelong Shen, Yanru Qu and Weizhu Chen. 2020. A simple but tough- to-beat data augmentation approach for natural lan- guage understanding and generation. arXiv preprint arXiv:2009.1 13878.
Lifu Tu, Garima Lalwani, Spandana Gella, and He He. 2020. An empirical study on robustness to spurious correlations using pre-trained language models.
Avanti Shrikumar, Peyton Greenside, and Anshul Kun- daje. 2017. Learning important features through prop- agating activation differences. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, page 31 5145-3153. Jor..
Ameya Vaidya, Feng Mai, and Yue Ning. 2020. Empiri- cal analysis of multi-task learning for reducing model bias toxic comment detection.
Bertie Vidgen, Alex Harris, Dong Nguyen, Rebekah Tromble, Scott Hale, and Helen Margetts. 2019, Chal- lenges and frontiers in abusive content detection. In Proceedings of the Third Workshop on Abusive Lan- guage Online, pages 80-93, Florence, Italy. Association for Computational Linguistics.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisser- man. 2013. Deep inside convolutional networks: Visu- alising image classification models and saliency maps.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: removing noise by adding noise.
Jesse Vig. Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388-1 1148-1 Curran Associates, Inc.
Rohit Sridhar and Diyi Yang. 2022. Explaining toxic text via knowledge enhanced text generation. In Pro- ceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pages 811-88 I-826, Seattle, United States. Association for Computational Linguistics.
Tianlu Wang. Rohit Sridhar, Diyi Yang, and Xuezhi Wang. 2022. Identifying and mitigating spurious cor- relations for improving robustness in NLP models. In Findings of the Association for Computational Linguis- tics: NAACL 2022, pages 1719-1729, Seattle, United States. Association for Computational Linguistics.
Gabriel Stanovsky. Noah A. Smith, and Luke Zettle- moyer. 2019. Evaluating gender bias in machine transla- tion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1679- 1684, Florence, Italy. Association for Computational Linguistics.
Zhao Wang and Aron Culotta. 2020. Identifying spuri- ous correlations for robust text classification. In Find- ings of the Association for Computational Linguistics: EMNLP 2020, pages 3431-3440, Online. Association for Computational Linguistics.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017, Axiomatic attribution for deep networks. In Proceed- ings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learn- ing Research, pages 33 3319-3328. PMLR.
Michael Wiegand, Josef Ruppenhofer, and Thomas Kleinbauer. 2019. Detection of Abusive Language: the Problem of Biased Datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume I (Long and Short Papers), pages 602-608, Minneapolis, Minnesota. Association for Computational Linguistics.
Shardul Suryawanshi. Bharathi Raja Chakravarthi. Mi- hael Arcan, and booktitle = Buitelaar, Paul. Multimodal meme dataset (multioff) for identifying offensive con- tent in image and text.
Zeerak Talat, James Thorne, and Joachim Bingel. 2018. Bridging the gaps: Multi task learning for domain trans- fer of hate speech detection.
Tomer Wullach, Amir Adler, and Einat Minkov. 2021. Fight fire with fire: Fine-tuning hate detectors using large samples of generated hate speech. In Findings of the Association for Computational Linguisties: EMNLP 2021, pages 4699-4705, Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Himanshu Thakur, Atishay Jain, Praneetha Vaddamanu, Paul Pu Liang, and Louis-Philippe Morency. 2023. Lan- guage models get a gender makeover: Mitigating gender bias with few-shot data interventions.
Wenjie Yin and Arkaitz Zubiaga. 2021. Towards gen- eralisable hate speech detection: a review on obstacles and solutions.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Bap- tiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and effi- cient foundation language models.
Xinchen Yu, Eduardo Blanco, and Lingzi Hong. 2022. Hate speech and counter speech detection: Conversa- tional context does matter. In Proceedings of the 2022

---

Conference of the North American Chapter of the As- sociation for Computational Linguisties: Human Lan- guage Technologies, pages 18-5930. Seattle, United States. Association for Computational Linguisties.
Dataset
# #train
# dev
# test
MM
445
149
149
WSF
6421
2141
2141
Fox
916
306
306
Xiangji Zeng, Yunliang Li, Yuchen Zhai, and Yin Zhang. 2020. Counterfactual generator: weakly-supervised method for named entity recognition. In Proceedings of the 2020 Conference Empiricul Methods in Natu- ral Language Processing (EMNLP), pages 7270-7280, Online. Association for Computational Linguistics.
HASOC
3150
1171
1171
ETHOS
598
200
200
TSA
19176
6393
6393
AHS
14869
4957
4957
AOH
1932
645
644
ZH
14869
4957
4957
Ziqi Zhang, D. Robinson, and Jonathan Tepper. 2018. Detecting hate speech on twitter using a convolution gru based deep neural network.
Table 5: Statistics of datasets in this work. These datasets are used in MLM, MTL and the main clas- sification experiments.
Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah Smith. 2021. Challenges in automated debiasing for toxic language detection. In Proceedings of the 16th Conference of the European Chapter of Association for Computational Linguistics: Main Vol- ume, pages 3153-3155, Online. Association for Com- putational Linguistics.
Each group contains 15 hateful and non-hateful sentences. The total size of the challenge set is 150.
Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2023. Can large lan- guage models transform computational social science?
A.4 Details for Counterfactual Generator
For token-level general biases (identity), we use the following example prompt for GPT-3.
A Appendix
A.1 Point-wise Mutual Information (PMI)
Generate 20 non non-hateful sentences con- taining 'Jews':
According to Gururangan et al., PMI is defined as following:
For data-specific biases, we use the following prompt.
PMI(t, c) log c(l,c)
p(l, ((..)p(..c)
We use this equation to find general (identity) bi- ases in hate speech datasets.
Paraphrase the original sentence with- out using 'Trump is':
Original sentence: Trump is the face corporate greed. And the poor love him? Ironic.
A.2 Dataset Statistics
Table 5 illustrates the statistics of datasets for MLM, MTL, and baseline models. The number of generated counterfactual examples after major- ity filtering are { MM: 631, WSF: 779, Fox: 558, HASOC: 872, ETHOS: 873, TSA: 250, AHS: 579, AOH: 1072, ZH: 520 J. Because many hate speech datasets do not have a specific split, we employ 60-20-20 split for them.
Paraphrased sentence:
We set the temperature 1, max_tokent 4000, top_p P to 1.0, frequency pennalty to 1, and pres- for GPT-3. The RS threshold
ence_ sice_penalty Drrquuncy_penatty
we select 0.2 for data-specific artifacts.
A.5 Implementation Details
A.3 Details on OOD Datasets
Following ElSherief et al., 2021b and Ramponi and Tonelli, 2022, we use a grid search to find the most suitable hyperparameters including learning rate [2e-5, 3e-5, 5e-5, 8e-6, 5e-6], batch size [4, 8, 10] and the number of epochs [1, 2, 3, 4]. We repeat every experiment three times using differ- random seeds and select the best models based on average macro Fi on validation sets. Other hyperparameters are set to the default one Hug- gingFace transformers' trainer, All experiments are
We find that the phenomenon of label imbalance exists for many current HS datasets. Specifically, a large proportion of HS datasets contain much more hateful data instances than non-hateful ones in both train and test sets. As result, to further mit- igate the influence of such label imbalance, we con- struct a label-balance OOD challenge set contain- ing five frequently occurring vulnerable groups, in- cluding Muslims, Women, LGBT, POC, and Jews.

---

conducted NVIDIA RTX 2080 Ti with 11GB memory, Every run of experiments can be finished within an hour.
ever, this combination can not yield noticeable performance improvement.
A.7
Examples of Data-Specific Biases
A.6 Set up for Baselines
Apart from the categories in Section 4.5.1, large number of data-specific biases can hardly be cate- gorized. a result, we do not list it on the tables. We listed Top-5 data-specific biases in Table 6.
A.6.1 Prompt Design for GPT-3 Baselines
Following Chiu et al., 2021, we use the following prompts for LLMsbasclines: Zero-shot:
Is the following text hateful? Answer yes or no.
'Men can see beauty, so they don't want to sleep with black women or make goblin ba- bies.'
Answer:
Few-shot:
Classify the following texts into 'non-hateful' or 'hateful':
"My best friend is muslim. And he is the best man in the world': non-hateful
'He seems like the very talkative taxi driver': non-hateful
'Men can see beauty, so they don't want to sleep with black women or make goblin ba- bies.': hateful.
"Islam a religion of hate. These propaganda type videos are dishonest. Simply read the
Quran and Hadith.': hateful.
'Kill all muslims':
A.6.2 Other Detail for Baselines
For AEDA, we generate four additional examples for augmentation for one original sentence. For LLMs' baselines, we randomly select 60 examples from the test sets and use the powerful text-davinci- 002, text-davinci-O0 and texi-chat-davinei-000 through openai API and set the temperature to max_tokens to 5, top_p P to 1.0, seapueccy_penallee to O, temperature to 0.7 and presence yecnictyyyeeyyyyyeee to 0. For MLM, we randomly mask 15% tokens and apply the default hyperparameters from the Hug- gingFace transformers trainer. Back translation is one of the commonly-used sentence-level DA methods. We use German as the intermediate lan- guage and Fairseq (Gehring al., 2017) machine translation model pretrained on WMT19 (Founda- tion). We also try to combine two kinds of MTI including MTL and MLM by some weights. How-

---

Root Examples
MI
#Non-hate
#Hate
% Majority
ban this world cup, a and, better use to
0.16
19
557
96.42
help, Trump is, wall that could have been a.23 million waste, are nation, a piece, at the bottom, used to celebrate man, ...
0.03
0
253
100.0
ICC shift this world another, called national, pity on your, live in this,
31
0.08
222
99.11
the worst, a fucking, in the, Shift this, has a, ban Rain,..
0.03
35
167
82.67
live in, him for, known sick of, tired of, called as, along with,
0.03
50
188
78.99
Table 6: Examples of data sete--peeifii patterns in HASOC19. Root: the non non-terminal root of PCFG. Examples: example grammar patterns of a certain root. MI: mutual information between the root and hateful label. #Non-hate: number non-hateful sentences containing such grammar pattern. Hate: number hateful sentences containing such grammar pattern. % Maaj: percentage of hateful examples.
Histagram Distribution for Relative Spuriousness
10
-0.2
0.0
0.2
0.4
0.6
Relative Spuriousness
Figure 4: RS distribution of data-specific biases. A large proportion of data-specific artifacts remain positive, while most are around zero. This result indicates that most data-specific patterns like those in Appendix A.7 have subtle influence model predictions.


Reviews:
[{"review_id": "5iQVf3g1S4", "reviewer": "Reviewer_5jqy", "summary": "", "questions": "Question A: I'm confused about the MTI method. Does it refer to applying both MLM and then MTL sequentially? Or is each of these two independently referred to as MTI? If the first case is correct, why wasn't the result of MTI (MLM + MTL) reported in table 1?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful and positive feedback! We are encouraged that you found our proposed mitigation methods **valuable** and **impactful** for challenging hate speech detection. We are also glad that you highlighted the **novelty** of our proposed metric of “Relative Spuriousness” to identify sentence-level spuriousness which was ignored by previous works and the **detailed statistics** on the correlation between detected token or grammar biases.\n\n- **W1: The DSI experiment should be analyzed separately to validate the effectiveness of biases high RS (sentence-level biases).**\n    - We appreciate your constructive feedback and acknowledge the importance of validating DSI’s effectiveness independently. In response to your comments, we have conducted the following **additional experiments** to **separately analyze** the DSI’s performance. To be specific, for sentence-level biases, we collect the non-hateful data points that contain highly biased grammar patterns (see Table 6 as examples) towards the label of hateful in the test set as the evaluation of the sentence-level bias mitigation. Similarly, for token-level biases, we collect the non-hateful data points that have highly correlated identity words to evaluate token-level bias mitigation. We then train BERT with the original training set and with DSI augmentation respectively. We report the average difference in the accuracy of these two methods in the following table:\n    \n        | Dataset | MM | WSF | Fox | HASOC | ETHOS | TSA | AHS | AOH | ZH | Average |\n        | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n        | Acc difference (sentence-level) | 20.93 | 8.54 | 11.43 | 23.52 | 17.39 | 7.69 | 14.28 | 12.97 | 5.08 | 13.54 |\n        | Acc difference (token-level) | 25.58 | 38.89 | 21.42 | 13.79 | 35.89 | 17.39 | 23.21 | 12.50 | 14.46 | 22.57 |\n\n        The experiment result indicates that DSI can **noticeably mitigate** sentence-level token-level biases. Comparatively, token-level DSI is more effective.\n    \n- **Q1: Does MTI (Multi-Task Intervention) refer to MLM (Masked Language modeling) and then MTL (Multi-Task Learning) sequentially or those two independently?**\n    - As described in the paraphrase starting from **line 344**, MLM and MTL are two types of MTI, which are independently presented in our experiments. Precisely, MTI is an abstract Intervention derived from causal analysis while MLM and MTL are two solutions to achieve such Intervention. We will further clarify these concepts in the following camera-ready version.\n- Typos, Grammar, Style, and Presentation Improvements\n    - For the MLI in Table 3, it is a typo and we will fix it to MTL as described in lines 491 and 492. We will fix all the citations from the arXiv version to the published version and other typos (Figure 3’s caption and line 102) in the camera-ready version.\n    - **The rationale behind the choices of G1 and G2 with the proposed method to mitigate them.**\n        - The choice of G1 comes from the **pre-training data** of PLMs. As the pre-training data of PLMs such as BERT are mostly from books or the internet, the distribution of different identities can be hugely diverse. For example, the identity words of minority groups can occur several orders of magnitude less frequently than other groups. As a result, such imbalance distribution (G1) can bring biases to PLMs. To mitigate such biases from the pretraining data, we propose MTI of training auxiliary tasks using a diverse range of HS datasets under the assumption that minorities occur more frequently in HS datasets.\n        - The choice of G2 comes from **hate speech training data**. In most HS data, the distribution of different identities in hateful and non-hateful contexts is not even (see **lines 336-339** as an example, G2 can be the platform of data source). As a result, training with such data can bring biases to certain identities such as Muslims or Jews. To mitigate such biases from the HS training data, we propose DSI to augment the current HS dataset with counterfactual data from both token-level and sentence-level to change the original distribution.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed methods effectively enhance the performance of hate speech (HS) tasks  mitigating the biases and exhibit creativity in their approach. The authors utilize GPT to generate non-hate speech data containing the identified biases, which serves as a means to mitigate biases. Overall, both proposed methods are valuable and impactful, leading to improvements. Through some straightforward steps, they manage to elevate the model's performance in tackling the challenging task of hate speech detection. They also introduce a novel method to identify sentence-level spurious.", "reasons_to_reject": "Considering that they introduce a new type of potential bias (sentence-level) and propose a metric to measure their spuriousness, it would be essential  to validate and show the accuracy of these metrics. Since the RS distribution figure suggests that these biases exert a subtle influence, it's possible that the improvement achieved by the DSI method is mainly due to the mitigation of token-level biases . I believe that the DSI experiment should be analyzed separately for both sentence-level and token-level counterfactual generated examples to see the effectiveness of mitigating biases with high RS.", "questions_to_authors": "Question A: I'm confused about the MTI method. Does it refer to applying both MLM and then MTL sequentially? Or is each of these two independently referred to as MTI? If the first case is correct, why wasn't the result of MTI (MLM + MTL) reported in table 1?", "paper_topic_and_main_contributions": "The authors introduce the concept that artifacts related to specific labels are not confined solely to individual tokens; they can also extend to sentence-level elements, such as phrases within datasets. To explore this, they utilize a probabilistic context-free grammar tree to identify potential grammar candidates and employ pointwise mutual information (PMI) to highlight the most correlated grammar patterns. These patterns are identified as sentence-level (data-specific) spurious correlations. To validate their spuriousness and distinguish clean spurious correlations, they introduce a novel metric named \"Relative Spuriousness.\" This is accomplished by measuring the disparity between local and global model scores, where the score is calculated as the difference between model results for inputs with and without the candidate spurious patterns. Additionally, they introduce \"identity spurious,\" referring to identity words (tokens) with the highest PMI. Addressing two distinct types of bias, they present two mitigation methods. To counter identity bias at the token level, they again pre-train BERT through a masked language model (MLM) objective on nine hate speech datasets. They believe these token-level biases stem from biases in the underlying training data of pre-trained language models (PLMs). Subsequently, the retrained PLMs are fine-tuned on various hate speech tasks simultaneously, resulting in superior performance compared to baseline methods for different hate speech datasets. The second method (DSI) involves counterfactual learning, generating non-hate speech samples containing identified biases (both token-level and sentence-level) using GPT. By fine-tuning the model on this augmented dataset, they achieve improved performance in comparison to other augmentation techniques.\n\n\n\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "I would appreciate the inclusion of statistics regarding the correlation between detected token or grammar biases and the corresponding labels in the mentioned datasets. It would be beneficial to have this information, even if it's presented in an appendix.\n\nIn lines 32 and 33, the reference (Re033 lia et al., 2019) is inaccurately addressed, as it pertains to the arXiv version instead of the published version in AAAI. This issue also occurs in line 240. While I haven't checked all the references, it's advisable to verify that all references are cited with their original published versions to ensure accuracy.\n\nIn lines 491 and 492, you mentioned using MTL as a baseline in Table 3, but in Table 3, you wrote MTI. This is confusing.\n\nThe caption of Figure 3 mentions the presence of black and blue lines, but there are no such lines visible in the actual figure.\n\n\nLine 102 mentions \"as shown in Figure 3.2,\"; however, Figure 3.2 is not a direct reference to the expected figure, which I assume might be Figure 2. Instead, it refers to the section that discusses the content related to it.\n\nI believe it would be beneficial to provide a more comprehensive explanation behind the choices of G1 and G2, along with the rationale for applying the proposed methods to mitigate their effects."}}, {"review_id": "jtMkPxr1Vk", "reviewer": "Reviewer_W9Qi", "summary": "", "questions": "what is the justification of using only the text of multimodal data? would the classification results still be meaningful? ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful feedback! We appreciate that you find our proposed methods’ **great justification by causal models** and solid experiments with **sufficient baselines** and **in-depth generalization analysis**.\n\n- **W1: The definition of hate speech is not clarified.**\n    - We acknowledge that a clear definition of hate speech is important. However, it is a **challenging** task. In fact, the definition of hate speech is neither universally accepted nor are individual facets of the definition fully agreed upon[1]. As a result, different datasets may have subtly different criteria for hate speech. Our work uses **widely-used** public benchmarks following previous works [2][3][4].\n- **W1: No information on the quality of generated counterfactual data.**\n    - As described in **lines 385-391** and the **rightmost** part of **Figure 3**, to ensure the generated counterfactual data are non-hateful, we add majority-vote filtering after raw data generation. To further evaluate data quality, we conducted an **additional human study** on the generated counterfactual data (we randomly sampled 60 counterfactual data points from every dataset). Following Hartvigsen et al.[5], we first ask the annotators to guess whether the statement’s author was a human or an AI system. Then we ask annotators to measure the harmfulness of text on a **1-5 scale** with 1 being clearly benign and 5 indicating very hateful. The result of the human study is that **92%** of the generated data are considered human-generated by annotators. Besides, the average toxicity measure of **1.02** indicates that most of our generated data are non-hateful.\n- **Q1: What is the justification for using only the text of multimodal data?**\n    - We follow the practice of the **original Multimodal Meme Dataset paper**[6] where they conduct separate analyses on the textual part of their dataset using text-only models. As described in [6], the motivation for their practice is that the textual parts of this dataset are the captions of the memes, which contain **all the textual information** on the images which can be sufficient to determine if the meme is hateful for a large number of cases. (Their experiment results show **comparable** performances of text-only and multimodal classifiers.)\n- Typos, Grammar, Style, and Presentation Improvements\n    - The trump example in Figure 1 is from HASOC19[7] and is labeled as hateful. We will add the source in the Figure caption in the camera-ready version.\n    - The expression of $P_l (Y(X \\neq x_i)| X_i = x_i, Y = y)$ is the probability of $y$ if $x_i$ is deleted for those that originally have the feature $x_i$ with the label $y$.\n    \n\n[1] MacAvaney, Sean, et al. \"Hate speech detection: Challenges and solutions.\" PloS one 14.8 (2019): e0221152.\n\n[2] Ilia Markov, Nikola Ljubešić, Darja Fišer, and Walter Daelemans. 2021. Exploring Stylometric and Emotion-Based Features for Multilingual Cross-Domain Hate Speech Detection. In Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 149–159, Online. Association for Computational Linguistics.\n\n[3] Markov, T., Zhang, C., Agarwal, S., Eloundou Nekoul, F., Lee, T., Adler, S., Jiang, A., & Weng, L. (2023). A Holistic Approach to Undesired Content Detection in the Real World. Proceedings of the AAAI Conference on Artificial Intelligence, 37(12), 15009-15018.\n\n[4] Steve Durairaj Swamy, Anupam Jamatia, and Björn Gambäck. 2019. Studying Generalisability across Abusive Language Detection Datasets. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 940–950, Hong Kong, China. Association for Computational Linguistics.\n\n[5] Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3309–3326, Dublin, Ireland. Association for Computational Linguistics.\n\n[6] Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. 2020. Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text. In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, pages 32–41, Marseille, France. European Language Resources Association (ELRA).\n\n[7] Mandl, Thomas, et al. \"Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages.\" Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation. 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- causal model justifies both proposed methods.\n- each method was tested and proven effective against a variety of reasonable baselines.\n- demonstrated good generalisation with new testing data.\n", "reasons_to_reject": "- definition of hate speech is not clarified. debatable understanding of what hate speech is occasionally, mixing identity words and slurs, e.g. table 4 and section 4.5.2 , 'blacks' used as nouns to refer to people are not simply identity words and the sentences are hence not neutral. it is also questionable whether their influence to model prediction should be mitigated.\nunclear, inconsistent definitions are already a big problem in this field. a paper with good methodologies but lack proper understanding of the task can be worrying. \n- no information on quality of generated counterfactual data, e.g. any manual inspection, small batch annotation. ", "questions_to_authors": "what is the justification of using only the text of multimodal data? would the classification results still be meaningful? ", "paper_topic_and_main_contributions": "- aims to reduce spurious correlation in hate speech detection.\n- identifies token and sentence- level ones with pmi and pcfg respectively. \n- reasoned with two types of confounders, which justified two types of intervention proposed: multi-task and data-specific(counterfactual learning). ", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "cite the final published version of papers if possible e.g. 'Finding Dataset Shortcuts with Grammar Induction' "}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- i think more people would agree that figure 1 top right trump example shouldn't be hateful than otherwise. either way when presenting examples, it's better to include source e.g. which dataset.\n- expressions 272-275 are confusing, although the paragraph after it makes sense.  i suppose Pl(Y (X ̸= xi) = y|Xi = xi, Y = y) would be probability without feature xi but how does each element correspond to elements of that sentence? e.g. X ̸= xi would mean the input does not equal xi rather than not contain\n- typos? vulnerable objectives -> identities\n- many mentions of '9 datasets' before they are introduced. would be helpful to point to the dataset section when they are first mentioned"}}, {"review_id": "CC6miNCd8B", "reviewer": "Reviewer_CfXf", "summary": "", "questions": "A. For spuriousness validation, why an unbiased model is equal to the average models trained on every HS dataset? Will there be situations where biases exist in every HS dataset? Why don't you use datasets from other domains for this step?\n\nB. In line 388, what's the rationale that the authors choose to use majority vote? \n\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "ThThank you for your thoughtful review! We are grateful that you found our work’s research problem **important** and the **potential impact** of our method **on future work**. We are also pleased that you found out our method is **well-grounded from a causal perspective** and its **effectiveness with great performance**.\n\n- **W1: Metric choices are not well-justified**\n    - As described in **lines 404-409**, following **previous works**[1], we use macro-F1 as the main metric for evaluations, which is a **widely accepted metric** to evaluate the performance of an HS detection system. We acknowledge that it is not designed to directly evaluate how well biases are mitigated. However, as macro-F1 score gives a fair representation of the classifier's performance across **all** classes, which can reflect how biases are mitigated (prediction from hateful to non-hateful for non-hateful data containing short-cuts) to a certain degree. Besides, to make up for this limitation, we conduct a comprehensive **case study (Section 4.6 from line 541)** to analyze how well the biases are mitigated through our approaches both **qualitatively (Table 4)** and **quantitatively (line 565)**.\n- **Q1: Why do we treat the averaged model as an “unbiased model”? Will there be biases in every HS dataset? Why don't we use other domains’ data for this?**\n    - We treat the averaged model as an “unbiased model” under the assumption that different datasets are more likely to have different data-specific biases (adopted by **many other works** such as [2] to conduct cross-dataset analysis on spurious correlations).\n    - Yes, there will be biases in almost every HS dataset. As described in **Section 3.1.1 (line 206)**, the bias of identity words exists in almost all hate speech datasets. However, we use domain knowledge to find such biases following previous works [1][3][4] instead of Relative Spuriousness (RS). Besides, as shown in **Figure 3**, we only use RS to detect **data-specific biases (yellow part of the figure)**.\n    - Using datasets from other domains will deteriorate the BERT’s representation of hate speech. As a result, the classifier can not model the probabilities described in the expression from lines 271-275 well.\n- **Q2: What is the rationale for using a majority vote in line 388?**\n    - As described in **lines 385-387**, we use a majority vote to make sure the generated counterfactual data are non-hateful, which is a way to further **control the quilty** of the generated data.\n- Typos, Grammar, Style, and Presentation Improvements\n    - We will fix the typo in the caption of Figure 3 in the camera-ready version. The blue part indicates the framework for token-level biases (identity). The yellow part represents the framework for data-specific biases.\n    - We will add more detailed explanations such as the response to Q2 to Line 386-389 in the camera-ready version.\n    \n\n[1] Alan Ramponi and Sara Tonelli. 2022. Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3027–3040, Seattle, United States. Association for Computational Linguistics.\n\n[2] Tianlu Wang, Rohit Sridhar, Diyi Yang, and Xuezhi Wang. 2022. Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1719–1729, Seattle, United States. Association for Computational Linguistics.\n\n[3] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The Risk of Racial Bias in Hate Speech Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678, Florence, Italy. Association for Computational Linguistics.\n\n[4] Maximilian Wich, Jan Bauer, and Georg Groh. 2020. Impact of Politically Biased Data on Hate Speech Classification. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 54–64, Online. Association for Computational Linguistics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is overall well-structured.\n2. The problem of hate bias does exist in current content moderation algorithms, and this paper works toward this problem.\n3. As all the methods are automated, it is easier for future work to replicate and build on this work. The hate mitigation methods achieve good performance. \n4. Studying this problem from a causal perspective is well-grounded and reasonable.", "reasons_to_reject": "The main weakness of this paper is the metric choices and to me they are not well-justified in the paper.\nIt is unclear how an unbiased model can be considered as the average responses from models trained on every HS dataset. \nThe intuitions of the metric design may be better grounded on a theory or any previous work (line 254-268). ", "questions_to_authors": "A. For spuriousness validation, why an unbiased model is equal to the average models trained on every HS dataset? Will there be situations where biases exist in every HS dataset? Why don't you use datasets from other domains for this step?\n\nB. In line 388, what's the rationale that the authors choose to use majority vote? \n\n", "paper_topic_and_main_contributions": "This paper works toward the problem of mitigating biases in hate speech detection.\nThe authors propose a way to automatically identify and validate spuriousness. \nThey develop two methods to mitigate the biases by solving this problem from a causal perspective.\nThe results show consistent improvements in 9 hate speech datasets. \n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "line 175: claim -> claims\n\nFigure 3: I cannot see the black lines.\n\nLine 386-389: I struggle to follow it and would suggest to add more explanations. "}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an approach to mitigate bias in hate speech detection approaches by automatically finding likely non-causal correlates.

Overall strengths include the focus on a current and important problem, a well-motivated approach with a potentially novel "spuriousness" measure, and good performance across multiple datasets at least for the metrics shown. Weaknesses include: choice of evaluation metrics, lack of validation for non-causal confounds/sentence-level spuriousness, and a lack of clear definition of the task (hate speech detection).
Decision: Accept-Findings