Socially Responsible Hate Speech Detection: Can Classifiers Reflect Social Stereotypes? (2023)

Abstract: Recent studies have shown that hate speech technologies may propagate social stereotypes against marginalized groups. Nevertheless, there has been a lack of realistic approaches to assess and mitigate biased technologies. In this paper, we introduce a new approach to analyze the potential of hate-speech classifiers to reflect social stereotypes through the investigation of stereotypical beliefs by contrasting them with counter-stereotypes. We empirically measure the distribution of stereotypical beliefs by analyzing the distinctive classification of tuples containing stereotypes versus counter-stereotypes in machine learning models and datasets. Experiment results show that hate speech classifiers attribute unreal or negligent offensiveness to social identity groups by reflecting and reinforcing stereotypical beliefs regarding minorities. Furthermore, we also found that models that embed expert and context information from offensiveness markers present promising results to mitigate social stereotype bias towards socially responsible hate speech detection.

## introduction while artificial intelligence (ai) technologies have generated unprecedented opportunities for society, they have also introduced new forms of perpetuating inequality and heightened threats to human rights and well-being (un, 2023). in this context, the investigation of hate speech (hs) is undoubtedly important since the proposition of automated systems has implications for unprejudiced societies. nevertheless, researchers have constantly observed that these technologies are being developed with scarce consideration of their potential social biases, which may perpetuate social inequalities when propagated at scale 1 warning: this paper contains examples of offensive content and stereotypes. it does not reflect our way of thinking. ## definitions here, we describe in detail the definitions of hate speech and social stereotypes used in this paper. ## results section 6.1 presents an error analysis of ml models, roc curves (see figure 2 ), and an evaluation in terms of precision, recall, and f1-score (see ## towards socially responsible hate speech detection as shown in figure 3 , the bow, bert, and fasttext are the models that more reflected social stereotypes. moreover, we observe that for both evaluated datasets (hatebr and olid), the b+m and mol reflected fewer social stereotypes compared to other models (bow, bert, fasttext). observe that the mol and b+m consist of context-aware methods for hate speech detection @xcite . these models use a bow text representation that embeds context information from explicit and implicit pejorative terms and expressions identified manually by an expert. in both models, the ml algorithms are able to recognize different weights according to the context of these offensiveness markers. for example, "stupid", which is mostly used in a pejorative context (e.g. "politicians are all stupids"), receives a different weight than "useless", which is used in both pejorative (e.g. the government is useless), and non-pejorative (e.g. this smartphone is useless) contexts. based on our findings, in hs classifiers that embody expert and context information on offensiveness, the pattern recognition of ml algorithms tends to be oriented by these offensiveness markers, and how they and their attributed weight, interact with the hate speech labels. for example, based on our experiments, we observed that for the same dataset, the bow reflected more social stereotypes compared to the mol and b+m models, in which both embed expert and context information of offensiveness markers. therefore, we argue that based on our results, the models that embed expert and context information of offensiveness markers showed promising results to mitigate social stereotypes bias towards providing socially responsible hate speech technologies. ## final remarks and future work since a human-based distinctive classification of social stereotypes and counter-stereotypes provides evidence of socially biased thinking, we introduce a new approach to analyze the potential of hs classifiers to reflect social stereotypes against marginalized groups. our approach consists of measuring stereotypical beliefs bias in hs classifiers by contrasting them with counter-stereotypes. specifically, we first implemented different ml text representations and evaluated them on two different datasets in english and portuguese from twitter and instagram data. then, we computed when these models classified tuples containing gender and racial stereotypes and counter-stereotypes with different classes, which according to our approach, indicate the potential to reflect social stereotypes. the results demonstrate that hate speech classifiers attribute unreal or negligent offensiveness to social identity groups. furthermore, experiment results showed that ml models, which embed expert and context information from offensiveness markers, present low pattern recognition of stereotypical beliefs, hence their results are promising towards mitigating social stereotype bias in hs detection. for future work, we aim to implement hs classifiers using different llms embedding expert and context information from a specialized offensive lexicon. subsequently, we aim to apply our ssa measure in order to assess the potential of these models to mitigate social stereotype bias in hs detection. we also aim to extend our dataset of tuples. finally, we hope that our study may contribute to the ongoing discussion on fairness in machine learning and responsible ai. ## ethical statements the datasets used in this paper were anonymized. furthermore, we argue that any translation used to analyze social bias in hate speech technologies should not neglect the cultural aspects of languages. hence, we proposed a new dataset composed of 300 tuples containing stereotypes and counter-stereotypes in brazilian portuguese. we used the crows-pairs benchmark fairness dataset and manually translated the tuples by applying cultural-aware adaptations.