Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes (2022)

Abstract: To tackle the rising phenomenon of hate speech, efforts have been made towards data curation and analysis. When it comes to analysis of bias, previous work has focused predominantly on race. In our work, we further investigate bias in hate speech datasets along racial, gender and intersectional axes. We identify strong bias against African American English (AAE), masculine and AAE+Masculine tweets, which are annotated as disproportionately more hateful and offensive than from other demographics. We provide evidence that BERT-based models propagate this bias and show that balancing the training data for these protected attributes can lead to fairer models with regards to gender, but not race.

## related work hate speech research has focused on dataset curation @xcite @xcite @xcite and dataset analysis @xcite @xcite . in our work, we further analyze datasets to uncover latent biases. it has been shown that data reflects social bias inherent in annotator pools @xcite @xcite . work has been conducted to identify bias against aae @xcite @xcite and gender @xcite . research has also been conducted in identifying disparities in performance across social groups, with machine learning algorithms underperforming for certain groups @xcite @xcite . @xcite investigated whether bias along the intersectional axis exists in @xcite . while @xcite focused on bias within a single dataset, in our work we generalize to multiple hate speech datasets. we also examine classifier behavior and methods to mitigate this bias. ## datasets five english datasets were used: three hate speech datasets (davidson, founta and ha-texplain), one dataset of tweets labeled for race (groenwold) and one for gender (volkova). we adopt the definitions of @xcite for hate speech (defined as speech that contains expressions of hatred towards a group or individual on the basis of protected attributes like ethnicity, gender, race and sexual orientation) and offensive speech (speech that contains offensive language but is not hateful). ## intersectionality statistics in in hatexplain, aae and sae are more balanced, but there is instead imbalance between genders. for masculine and feminine speech, passages are neutral at rates of 43% and 61% respectively. in davidson, sae+feminine speech is viewed as more offensive than sae+masculine (48% vs. 19%), while in hatexplain, sae+masculine is more hateful than sae+feminine (34% vs. 16%). finally, when comparing genders in aae speech, we see that while aae+feminine contains a larger percentage of offensive tweets (for example, in founta, 69% vs. 54% and in hatexplain, 50% vs. 21%), aae+masculine contains disproportionately more hateful speech (in davidson, 7% vs. 5%, in founta, 28% vs. 9% and in hatexplain, 19% vs. 6%). overall, aae and masculine speech is annotated as more offensive and hateful than sae and feminine speech. further analyzing aae, aae+masculine is viewed as more hateful than aae+feminine. ## bias in bert we investigate to what extent data bias is learned by bert. we compare our findings against a dataset balanced for race and gender, to examine whether balanced data leads to fairer models. namely, we compare a randomly sampled with a balanced set the davidson dataset. 6 in the balanced set we sample the same number of aae and sae tweets (3000) and the same number of masculine and feminine tweets (1750). we also include 8000 neutral 6 founta and hatexplain were not considered for this study as they do not contain enough aae examples to make confident inferences. in the randomly sampled set, there is an imbalance both for gender and race. for gender, while masculine tweets are more hateful (3% vs. 1%), feminine tweets are more offensive (71% vs. 63%). for race, aae is marked almost entirely as offensive (94%), while sae is split in neutral and offensive (53% and 41%). in the sae subset of tweets, there is an imbalance between genders, with sae+feminine being marked disproportionately more often as offensive than sae+masculine (54% vs. 19%). ## conclusion in our work, we analyze racial, gender and intersectional bias in hate speech datasets. we show that tweets from aae and aae+masculine users (as classified automatically) are labeled disproportionately more often as offensive. we further show that bert learns this bias, flagging aae speech as significantly more offensive than sae. we perform interpretability analysis using lime, showing that the inability of bert to differentiate between variations of the n-word across dialects is a contributing factor to biased predictions. finally, we investigate whether training on a dataset balanced for race and gender mitigates bias. this method shows mixed results, with gender bias being mitigated more than racial bias. with our work we want to motivate further investigation in model bias not only for the usual gender and racial attributes, but also for their intersection.