A Multilingual Dataset of Racial Stereotypes in Social Media Conversational Threads (2023)

Abstract: In this paper, we focus on the topics of misinformation and racial hoaxes from a perspective derived from both social psychology and computational linguistics. In particular, we consider the specific case of anti-immigrant feeling as a first case study for addressing racial stereotypes. We describe the first corpus-based study for multilingual racial stereotype identification in social media conversational threads. Our contributions are: (i) a multilingual corpus of racial hoaxes, (ii) a set of common guidelines for the annotation of racial stereotypes in social media texts, and a multi-layered, fine-grained scheme, psychologically grounded on the work by Fiske, including not only stereotype presence, but also contextuality, implicitness, and forms of discredit, (iii) a multilingual dataset in Italian, Spanish, and French annotated following the aforementioned guidelines, and cross-lingual comparative analyses taking into account racial hoaxes and stereotypes in online discussions. The analysis and results show the usefulness of our methodology and resources, shedding light on how racial hoaxes are spread, and enable the identification of negative stereotypes that reinforce them.

## introduction racial hoaxes (rhs) are "a communicative act oriented to spread fallacious information against a social group" @xcite . as social media have become a dominant means of communication, investigating them is crucial for tackling the spread of rhs. we approach this task combining * the first three authors contributed equally. psychological and computational linguistics methods with a multilingual, cross-cultural perspective (italian, spanish, and french). in particular, rhs can contribute to the diffusion of stereotypes about people belonging to the outgroup, i.e., a social group with features that differ from the ingroup @xcite and are, thus, more vulnerable. even common, naive users are as likely to become spreaders of rhs as malicious users @xcite . in this paper, we cover a specific theme: anti-immigrant stereotypes. the discursive construction of immigrants and refugees in user interaction on social media has been studied by @xcite , who has shown how racial expressions and overt racism are becoming increasingly normalized, thus leading to prejudices and racial stereotypes and, eventually, even harmful acts. overall, the attention to these topics is relatively new in the nlp community, and thus, there is still a meaningful lack of annotated resources for the development of automatic tools to detect stereotypes and related phenomena. among the few research contributions in this direction, @xcite organized the second edition of haspeede at evalita 2020, asking participants to automatically detect hate speech and stereotypes in italian tweets and headlines. similarly, in the detests shared task at iberlef 2022, @xcite proposed a 10-label classification task for the identification of stereotypes in spanish; and finally, during irostereo at pan/clef 2022, @xcite proposed an author profiling task regarding stereotype spreaders and studied the link with irony in english. recently, for french, @xcite investigated how to improve gender hate speech classification by leveraging stereotype detection based on multitask architectures. however, such related works only focus on monolingual contents, without considering multilingual settings from which cross-cultural differences and similarities in the expression of stereotypes can emerge. furthermore, most of the related work limits the scope of investigation to the mere presence/absence of stereotypes in a single text, without diving into the finer-grained features that arise from psychological studies @xcite @xcite , and without taking into account their propagation in social media conversational threads. considering the gaps in current related work, we propose a cross-cultural, and multilingual perspective for studying racial hoaxes and stereotypes. in this work, our original contributions are: ## from racial hoaxes to reactions in order to collect reactions to racial hoaxes on social media, we first created the multilingual racial hoaxes corpus (mrhc), a list of 239 rhs in three languages: italian, spanish, and french. given the difficulty of spotting them automatically, we collected the entries of the mrhc manually. depending on the language, different factchecking websites or newspapers commenting on hoaxes were used as a source for manually extracting the mrhc between 2019 and 2021. for instance, for italian we used the debunking sites bufale.net and butac.it; for spanish mald ita.es and newtral.es; and finally for french factuel.afp.com and lemonde.fr/les-decod eurs. ## cultural and linguistic analyses in this section, we describe the comparative analyses we performed to extract analogies and differences in the expression of stereotypes and the forms of discredit in the reactions to rhs among the three subsets. ## conclusion and future work in this paper, we presented the first outcomes of a study of the stereotypes that are spread through racial hoaxes, with the aim of creating nlp resources and tools to automatically detect them. in order to address this challenging task, we started with an examination of the psychological and computational literature on fake news and stereotypes. this helped us to build the mrhc, the first multilingual corpus of racial hoaxes, which includes rhs in italian, spanish, and french, classified according to the topic of the news they spread. we designed a multi-layered annotation scheme for the annotation of racial stereotypes that takes into consideration the conversational thread extracted from social media. we applied it for the first time to a newly created multilingual dataset of twitter reactions to rhs. thanks to the outcomes of the annotation procedure, we were able to perform cross-cultural and cross-language analyses of these texts that are shaped in a twitter conversational structure. the results show that the presence of stereotype is, in general, lower within the rhs domain, with respect to its percentage in other pre-existing more general-purpose datasets (e.g., the ones developed within shared tasks). other relevant findings show that, even if the first source rh contains a stereotype, in the following replies in the conversational thread, the presence of stereotypes decreases. additionally, the dimension of implicitness was shown to be highly dependent on the dimension of contextuality in this domain. contentwise, from an observation of rhs' topics, crossed with a lexical analysis (counting the most relevant tokens and expressions in each language subset), the outcomes show how the presence of stereotypes is linked to words that are typically grounded within the specificities of a certain language or culture. finally, it can be observed that people who continue to spread a stereotypical view, originated in the source tweet and throughout the replies-toreplies, typically use polarized expressions that are in line with the original rh that generated the full conversational thread. thanks to the resources and framework elaborated in this study, it will be possible to investigate the spread of racial stereotypes on social media in a finer-grained way from a computational perspective and in a multilingual context. furthermore, these steps are essential for developing computational tools for the automatic detection and classification of racial stereotypes in real-life scenarios.