Analysis of Implicit Stereotypes: A Corpus-Based Study for Italian
Authors: Wolfgang S. Schmeisser-Nieto , Giacomo Ricci, Simona Frenda, Mariona Taulé, Cristina Bosco (Universitat de Barcelona, Università di Torino, Heriot-Watt University, Aequa-tech, 2024)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
Detecting stereotypes is a challenging task, particularly when they are not expressed explicitly. In this study, we applied an annotation schema from the literature designed to formalize implicit stereotypes. We analyzed implicit stereotypes about immigrants in two datasets: StereoHoax-IT and SterheoSchool, which are created from different sources. StereoHoax-IT consists of reactions on Twitter to specific hoaxes aimed at discriminating against immigrants, while SterheoSchool includes comments from teenagers on fake news generated in psychological experiments. We describe the annotation process, annotator disagreements, and provide both quantitative and qualitative analyses to shed light on how implicitness characterizes stereotypes in different texts. Our findings suggest that implicit stereotypes are often conveyed through logical linguistic relations, such as entailment and behavioral evaluations of immigrants.

Introduction
Various recent NLP studies have focused on detecting stereotypes online, often in conjunction with forms of abusive language AUTHOR. The importance of tackling this phenomenon is due to its impact on social structures and the power of individuals. Therefore, detecting stereotypes can prevent their emergence and spread, and thereby have a positive impact on our society. In social psychology, a stereotype has been defined as a set of beliefs about others perceived as belonging to a different social group AUTHOR. It oversimplifies the features of the group and generalizes a particular feature, applying it to all its members AUTHOR. In contrast to the emotional component of prejudice and the behavioral component of discrimination, a stereotype is associated with the cognitive component of the triad AUTHOR. In language, stereotypes can be expressed explicitly or implicitly AUTHOR. Explicit stereotypes deliver a straightforward message, clearly revealing the associated traits, often using derogatory adjectives AUTHOR. In contrast, implicit stereotypes are more nuanced and indirect, requiring the reader to infer their meaning AUTHOR. These implicit stereotypes can be communicated through linguistic devices such as metaphor and irony AUTHOR, negation AUTHOR, or entailments AUTHOR. Recently, efforts have been made to formalize the strategies for expressing implicit stereotypes, with the goal of establishing standardized criteria for annotators AUTHOR. An example of explicit stereotype is "{[Gli immigrati] buttano via il cibo che gli danno per poi andare a mangiare i poveri cani, dove finiremo!"} } (extracted from StereoHoax-IT corpus), in which the generalization of the target group and the association with an action is expressed in a present tense with a habitual aspect. On the other hand, in the example "{Come noi rispettiamo loro e il colore della loro pelle, così loro che abitano nei nostri paesi dovrebbero portare rispetto nei nostri confronti."} } (SterheoSchool corpus), the stereotype is not overtly manifested, but it must be inferred through the evaluation of the in-group and an exhortative sentence. From a computational linguistics perspective, concerns have been raised about how to detect and process stereotypes, a task often considered closely related to the detection of abusive language or hate speech AUTHOR. \\ Alongside research on hate speech, the study of stereotype detection has increased, particularly within evaluation tasks AUTHOR. However, the detection of implicit stereotypes remains a significant challenge AUTHOR. There are several works that deal with stereotypes in more complex narratives, such as microportraits AUTHOR and political debates AUTHOR. The detection of implicitness has also been studied with reference to several other phenomena, in particular those characterized by subjectivity, such as irony AUTHOR. In this paper, we analyze the implicit manifestation of stereotypes targeting immigrants, using a well-defined annotation schema proposed by \citet{schmeisser2022criteria} and tested on a subset of comments from Spanish newspapers (DETESTS AUTHOR). This schema represents different criteria for determining the implicitness of stereotypes in an attempt to formalize the concept. Disentangling strategies of implicitness presents a significant challenge, often resulting in the identification of multiple categories within the same text. Our main contributions consist of expanding the annotation with topics of stereotypes about immigrants AUTHOR and the strategies to implicitness AUTHOR, as well as testing this schema on two existing Italian datasets. These datasets share the same domain as those used for Spanish, stereotypes about immigrants, and include data extracted from Twitter (now X) as reactions to specific hoaxes (StereoHoax-IT) and comments written by high school students to two examples of fake news artificially created within psychological experiments (SterheoSchool) as described in AUTHOR. Analyzing the annotated texts, we noted that implicit stereotypes appear to be conveyed especially through logical linguistic relations like entailment and the behavioral evaluation of immigrants in both datasets. Moreover, in most cases, the annotators needed to use contextual information to determine the presence of stereotypes. For example, in this case "{Che centra lui e Italiano!, può essere massacrato!}" } (StereoHoax-IT) the author of the message expresses a stereotype complaining that foreigners enjoy better treatment than Italians, who can indeed be "macellati" (slaughtered). The rest of the paper is organized as follows: Sections~sec:datasets and~sec:annotation describe the datasets and the annotation applied; Sections~sec:statistics and~sec:qualitative present quantitative and qualitative analyses of the annotated data; and Section~sec:conclusions summarizes the results and provides guidance regarding future work.

Conclusion
In this paper, we applied an annotation scheme for analyzing the implicitness of stereotypes against immigrants according to two main dimensions (i.e., topics and strategies for making the content implicit) to the Italian StereoHoax-IT and SterheoSchool corpora. Adding these two layers of annotation allowed us to observe that annotators need to use contextual information to determine the presence of stereotypes especially, when specific strategies have been used by the author of the message (irony/sarcasm, extrapolation, entailment/evaluation, and imperative/exhortative). Moreover, implicit stereotypes appear to be conveyed mainly through logical linguistic relations such as the entailment and behavioral evaluation of immigrants and, in fewer cases, via `imperative/exhortative', `irony/sarcasm' and `extrapolation.' As future work, we plan to perform a comparative analysis with the datasets in Spanish, which have already been annotated with this schema, in order to understand cultural analogies and differences in portraying immigrants as threats, enemies or victims.