Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments (2022)

Abstract: Building on current work on multilingual hate speech (e.g., Ousidhoum et al. (2019)) and hate speech reduction (e.g., Sap et al. (2020)), we present XTREMESPEECH, a new hate speech dataset containing 20,297 social media passages from Brazil, Germany, India and Kenya. The key novelty is that we directly involve the affected communities in collecting and annotating the data – as opposed to giving companies and governments control over defining and combatting hate speech. This inclusive approach results in datasets more representative of actually occurring online speech and is likely to facilitate the removal of the social media content that marginalized communities view as causing the most harm. Based on XTREMESPEECH, we establish novel tasks with accompanying baselines, provide evidence that cross-country training is generally not feasible due to cultural differences between countries and perform an interpretability analysis of BERT’s predictions.

## introduction much effort has been devoted to curating data in the area of hate speech, from foundational work @xcite to more recent, broader @xcite as well as multilingual @xcite approaches. however, the demographics of those targeted by hate speech and those creating datasets are often quite different. for example, in @xcite , 66% of annotators are male and in @xcite , 82% are white. this may lead to unwanted bias (e.g., disproportionately labeling african american english as hateful @xcite ) and to collection of data that is not representative of the comments directed at target groups; e.g., a white person may not see and not figure 1 : overview of hate speech data collection. instead of querying for data on our own, we work with fact-checkers advocating for targeted communities who collect and label data as they organically come across it. this inclusive approach results in datasets more representative of online speech the communities are exposed to. see §3.2 for definition of xtremespeech labels. an example from our dataset is the kenyan social media post "… we were taught that such horrible things can only be found in luo nyanza." the luo are an ethnic group in kenya; nyanza is a kenyan province. the post is incendiary because it suggests that the luo are responsible for horrible things, insinuating that retaliation against them may be justified. only a group of people deeply rooted in kenya can collect such examples and understand their significance. xtremespeech. in this paper, we present xtremespeech, a new hate speech dataset containing 20,297 social media passages from brazil, germany, india and kenya. the key novelty is that we empower the local affected communities (as opposed to companies and governments) to collect and annotate the data, thus avoiding the problems inherent in approaches that hire outside groups for hate speech dataset creation. in more detail, we built a team of annotators from fact-checking groups from the four different countries. these annotators both collected and annotated data from channels most appropriate for their respective communities. they were also involved in all phases of the creation of xtremespeech, from designing the annotation scheme to labeling. our inclusive approach results in a dataset that better represents content targeting these communities and that minimizes bias against them because fact-checkers are trained to be objective and know the local context. figure 1 gives a high-level overview of data collection and annotation for xtremespeech. xtremespeech also is a valuable resource because existing hate speech resources are not representative for problematic speech on a worldwide scale: they mainly cover western democracies. in contrast, our selection is more balanced, containing three countries from the global south and one western democracy. we present a data statement (see @xcite ) in appendix a. ## experiments we establish xtremespeech baselines for large pretrained models and traditional machine learning models (details in appendix e). as introduced in §1, we address three novel tasks: predicting the extremity of speech (extremity), whether a passage should be removed or not (removal) and the target of extreme speech (target). unless noted otherwise, our measure is microaveraged f1. we split each country set 80:10:10 into train:dev:test, sampling equally for all labels. 5 in tables 5 , 6 , 7, 8, 9 we show results on the development set (test set results in appendix g). we evaluate both multilingual (mbert, xlm-r @xcite ) and monolingual (langbert) models. each monolingual model was pretrained on the local language we are using for each corresponding country; e.g., the indian model was pretrained on hindi. for finetuning and classification with bert-based models, a task-specific head is added that takes as input the [cls] token representation. ## conclusion we have presented xtremespeech, an extreme speech dataset, containing 20,297 passages from brazil, germany, india and kenya. we capture both granular levels of extremity and targets of extreme speech by engaging a team of annotators from within the affected communities. in a collaboration of anthropologists and computational linguists, we established a community-based framework, with the goal of curating data more representative of real-world harms. we introduce baselines for three novel tasks, including extreme speech and target group classification. we give experimental support for the intuition that extreme speech classification is dependent on cultural knowledge and that current nlp models do not capture this. finally, we perform interpretability analysis on bert's predictions to reveal potential deficiencies, showing that models rely heavily on keywords and names of marginalized groups. we hope our community-driven work will contribute to the effective elimination of extreme speech against target groups, not just in western democracies, but in a greater variety of countries worldwide.