Hate Speech Classifiers are Culturally Insensitive (2023)

Abstract: Increasingly, language models and machine translation are becoming valuable tools to help people communicate with others from diverse cultural backgrounds. However, current language models lack cultural awareness because they are trained on data representing only the culture within the dataset. This presents a problem in the context of hate speech classification, where cultural awareness is especially critical. This study aims to quantify the cultural insensitivity of three monolingual (Korean, English, Arabic) hate speech classifiers by evaluating their performance on translated datasets from the other two languages. Our research has revealed that hate speech classifiers evaluated on datasets from other cultures yield significantly lower F1 scores, up to almost 50%. In addition, they produce considerably higher false negative rates, with a magnitude up to five times greater, demonstrating the extent of the cultural gap. The study highlights the severity of cultural insensitivity of language models in hate speech classification.

## introduction the current nlp models are trained on culturally biased datasets, so they lack sociocultural diversity @xcite . there is recent research emphasizing the importance of developing models that are more generalized to other languages and cultures @xcite @xcite . hate speech detection poses an extra challenge because it is crucial to consider the impact of inherent social and cultural differences for this task @xcite . however, current approaches tend to overlook cultural differences, underscoring the need for more nuanced and culturally sensitive approaches to develop models that can address the hate speech classifiers ground truth translated text prediction hate if you want to insist on gender equality, agree to the female conscription system. (original text: 남녀평등 주장할 거면 여성징병제에도 동의하라고) non-hate quantitative & qualitative analysis translated dataset @xmath0. with communication across cultural and linguistic barriers becoming increasingly common in the online landscape, an effective cross-cultural hate speech classifier is necessary. this classifier should identify hate speech that incorporates diverse cultural nuances and variations, regardless of the language. however, to the best of our knowledge, no research has yet addressed this critical necessity. this study aims to evaluate cross-cultural hate speech classifiers. we investigate cultural disparities in hate speech detection, explicitly focusing on the cultures of korean, arabic, and englishspeaking countries. to achieve this goal, we develop hate speech classifiers for each language and evaluate their performance on translated datasets from other cultures. the experiment overview can be seen in figure 1 . we also perform sample-level analysis within the misclassified texts, providing insights into the reasons for poor classification performance on the datasets from different cultures. through our analysis, we identify the limitations of current methodologies that fail to address the complexity of cross-cultural communication and perpetuate cultural divides. our experiment revealed that the f1 scores of hate speech classifiers evaluated on datasets from other cultures decremented by 26% to 48%, and the false negative rate (fnr) increased about two to five times larger. this result shows that models trained in a single language are deficient in detecting hate speeches from other cultures. deeper examinations of false negative samples showed that the limited performance was likely due to the differences in target groups, sociocultural backgrounds, and even the standards of hate speech. ## related work recent research has focused on developing multilingual hate speech detection datasets and models. several approaches have been proposed to address the scarcity of datasets in different languages, such as building multilingual hate speech corpora @xcite @xcite and implementing cross-lingual methods that incorporate translated data or multilingual embeddings @xcite @xcite @xcite . additionally, transfer learning on multilingual models like xlm-r has been utilized to take advantage of large english datasets and crosslingual contextual word embeddings @xcite . however, most of these approaches did not consider the cultural differences among datasets. they did not examine the model's cross-cultural detection ability, where the model could detect hate speech from other cultures. challenges in building a hate speech classifier in multilingual or multicultural settings include variations in targets of hate speech among countries and cultures @xcite , and the need to consider cultural discrepancies and diverse backgrounds. current studies have not fully addressed these issues, as some have used translated texts and maintained ground truth labeling without considering cultural differences @xcite @xcite . another consideration is that word senses may differ based on dialect, sociolect, language, and culture @xcite @xcite . therefore, incorporating cultural diversity is crucial in handling linguistically varied and cross-cultural hate speech. researchers have proposed various methods for adapting hate speech detection models to different cultural contexts @xcite @xcite , but there is still limited research on cross-cultural hate speech detection. some methods include using multi-task learning on hate speech datasets from different cultures @xcite and building new datasets that contain different targets of hate @xcite . while @xcite has evaluated knowledge transfer performance across different datasets from different cultural backgrounds in the same language, it lacked a deeper analysis of the cultural differences behind poor performance. in contrast, this paper includes a thorough analysis of sociocultural backgrounds and differences between hate speech datasets from different cultures and explores the reasons behind the poor performance in various language settings. ## datasets from different cultures this study evaluates the cross-cultural performance of hate speech classifiers trained on korean, arabic, and english datasets. we translate the datasets to compare the cross-cultural performance of the classifiers in different cultural settings. the datasets represent each culture, allowing for a more nuanced analysis of the performance of hate speech classifiers. we use the training and validation sets of these datasets for training and test sets for evaluations, including the cross-cultural experiment. since the korean dataset does not have training, validation, and test sets separated, we divide the entire dataset by the ratio of 8:1:1. ## culture representative model training to ensure that the hate speech classifiers accurately represent the cultures of their respective languages, they must achieve high performance on datasets from their language. to address this, we use monolingual models pretrained in each of the three languages and finetune them. the following sections contain descriptions of each model and the results of finetuning. specific training details are in appendix b.1. we use the best model for each language for cross-cultural evaluation in section 5, and ## cross-cultural evaluation the current study aimed to evaluate the crosscultural performance of different hate speech classifiers and explore the factors responsible for their poor performances. another common tendency was decreased false positive rate (fpr). this could be due to the lack of understanding of other cultures leading the models to follow the majority label of the training dataset and to predict some instances as non-hate incorrectly. another possible reason is that hate speech classifiers tend to have identity term bias @xcite ), but they may not have the bias for unknown targets of hate from different cultures. our area of interest was the increase in false negative rate (fnr) of the cross-cultural evaluation results, up to five times higher than that of the original dataset. the findings revealed that the poor performance of the models is not only due to differences in the target of hate but also due to variations in the standard across cultures. ## conclusion in this paper, we investigated the cross-cultural performance of monolingual hate speech classifiers for korean, english, and arabic languages by evaluating the classifiers' performance on translations of hate speech datasets from other languages. our deep analysis of model performance and false negative samples revealed the limitations of classifiers trained in a single language, including their inability to understand the sociocultural background of other cultures. this lack of understanding resulted in many samples being predicted as non-hate speech, highlighting the need for cross-cultural evaluation of hate speech classifiers. our research also demonstrated standard differences in hate of general target groups across cultures. our findings underscore the importance of crosscultural evaluation of hate speech classifiers and sample-level analysis to identify their weaknesses in a cross-cultural context. adopting this approach will enable models to accurately detect hate speech from diverse cultures in global online communities. as such, our research highlights the need for more culturally sensitive approaches to developing hate speech classifiers to address the challenges posed by linguistic and cultural diversity in online spaces.