Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model (2021)

Abstract: Stereotypical language expresses widely-held beliefs about different social categories. Many stereotypes are overtly negative, while others may appear positive on the surface, but still lead to negative consequences. In this work, we present a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology. The SCM proposes that stereotypes can be understood along two primary dimensions: warmth and competence. We present a method for defining warmth and competence axes in semantic embedding space, and show that the four quadrants defined by this subspace accurately represent the warmth and competence concepts, according to annotated lexicons. We then apply our computational SCM model to textual stereotype data and show that it compares favourably with survey-based studies in the psychological literature. Furthermore, we explore various strategies to counter stereotypical beliefs with anti-stereotypes. It is known that countering stereotypes with anti-stereotypical examples is one of the most effective ways to reduce biased thinking, yet the problem of generating anti-stereotypes has not been previously studied. Thus, a better understanding of how to generate realistic and effective anti-stereotypes can contribute to addressing pressing societal concerns of stereotyping, prejudice, and discrimination.

## introduction stereotypes are widely-held beliefs about traits or characteristics of groups of people. while we tend to think of stereotypes as expressing negative views of groups, some stereotypes actually express positive views (e.g. all women are nurturing). however, even so-called 'positive' stereotypes can be harmful, as they dictate particular roles that individuals are expected to fulfill, regardless of whether they have the ability or desire to do so @xcite . the existence of stereotypes in our society -including in entertainment, the workplace, public discourse, and even legal policy -can lead to a number of harms. @xcite organizes these harms into three main categories: (1) misrecognition effects: harms caused by denying members of particular groups an equal place in society, diminishing their human dignity, or other forms of marginalization. (2) distribution effects: harms resulting from unfair allocation of resources, either by increasing the burden placed on a group, or decreasing a group's access to a benefit. (3) psychological effects: the distress and unhappiness caused by an awareness and internalization of the stereotyped biases against one's identity group. additionally, the internalization of these negative stereotypes can lead to anxiety and underachievement. to reduce these harms and promote a more egalitarian society, we must identify and counter stereotypical language when it occurs. evidence from the psychological literature suggests that one of the most effective methods for reducing stereotypical thinking is through exposure to counter-stereotypes, or anti-stereotypes. @xcite showed participants stereotypical and anti-stereotypical images of highly socially-gendered professions (e.g., a surgeon is stereotypically male, and a nurse is stereotypically female; the genders were reversed in the anti-stereotypical images), and then measured their gender bias in a judgement task. exposure to antistereotypical images significantly reduced gender bias on the task. @xcite used a mental imagery task and reported that participants in the anti-stereotypical condition subsequently showed significantly weaker effects on the implicit association test (iat). @xcite showed a similar effect by exposing participants to anti-stereotypical exemplars (e.g. admired black celebrities, and disliked white individuals). when @xcite compared 17 interventions aimed at reducing stereotypical thinking, methods involving anti-stereotypes were most successful overall. thus, creating technology that enables users to identify stereotypical language when it occurs, and then counter it with anti-stereotypes, could help to reduce biased thinking. however, the idea of what constitutes an anti-stereotype remains ill-defined. is an anti-stereotype simply the semantic opposite of a stereotype? or can anything that is not a stereotype serve as an anti-stereotype? if two groups are stereotyped similarly, do they have an identical antistereotype? can an anti-stereotype actually reflect an equally harmful view of a target group (e.g. the cold-hearted career woman as an anti-stereotype to the nurturing housewife)? here, we begin to untangle some of these questions using the stereoset dataset @xcite . we begin by analyzing the stereotypes expressed in this dataset. one widely-accepted model of stereotypes, prejudice, and inter-group relationships from social psychology is the "stereotype content model" or scm @xcite . the scm proposes two fundamental and universal dimensions of social stereotypes: warmth and competence. by defining the warm-cold, competentincompetent axes in the semantic embedding space, we are able to cluster and interpret stereotypes with respect to those axes. we can then examine the associated anti-stereotypes and their relation to both the stereotyped description and the target group. thus, our contributions are as follows: we provide more details on the stereotype content model and its practical implications, and then briefly review the nlp research on computational analysis of stereotypical and abusive content. stereotype content model: stereotypes, and the related concepts of prejudice and discrimination, have been extensively studied by psychologists for over a century @xcite . conceptual frameworks have emerged which emphasize two principle dimensions of social cognition. the stereotype content model (scm) refers to these two dimensions as warmth (encompassing sociability and morality) and competence (encompassing ability and agency) @xcite . when forming a cognitive representation of a social group to anticipate probable behaviors and traits, people are predominantly concerned with the others' intentare they friends or foes? this intent is captured in the primary dimension of warmth. the competence dimension determines if the others are capable to enact that intent. a key finding of the scm has been that, in contrast to previous views of prejudice as a uniformly negative attitude towards a group, many stereotypes are actually ambivalent; that is, they are high on one dimension and low on the other. further, the scm proposes a comprehensive causal theory, linking stereotypes with social structure, emotions, and discrimination @xcite . according to this theory, stereotypes are affected by a perceived social structure of interdependence (cooperation versus competition), corresponding to the warmth dimension, and status (prestige and power), determining competence. stereotypes then predict emotional response or prejudices. for example, groups perceived as unfriendly and incompetent (e.g., homeless people, drug addicts) evoke disgust and contempt, groups allegedly high in warmth but low in competence (e.g., older people, people with disabilities) evoke pity, and groups perceived as cold and capable (e.g., rich people, businesspeople) elicit envy. finally, the emotions regulate the actions (active or passive help or harm). thus, low warmth-low competence groups often elicit active harm and passive neglect, whereas low warmth-high competence groups may include envied out-groups who are subjects of passive help in peace times but can become targets of attack during social unrest @xcite . the scm has been supported by extensive quantitative and qualitative analyses across cultures and time @xcite . to our knowledge, the current work presents the first computational model of the scm. stereotypes in language models: an active line of nlp research is dedicated to quantifying and mitigating stereotypical biases in language models. early works focused on gender and racial bias and revealed stereotypical associations and common prejudices present in word embeddings through association tests @xcite @xcite . to discover stereotypical associations in contextualized word embeddings, @xcite and @xcite used pre-defined sentence templates. similarly, @xcite built a template-based corpus to quantify bias in neural language models, whereas @xcite and @xcite used crowd-sourced stereotypical and antistereotypical sentences for the same purpose. in contrast to these studies, while we do use word embeddings to represent our data, we aim to identify and categorize stereotypical views expressed in text, not in word embeddings or language models. abusive content detection: stereotyping, explicitly or implicitly expressed in communication, can have a detrimental effect on its target, and can be considered a form of abusive behavior. online abuse, including hate speech, cyber-bullying, online harassment, and other types of offensive and toxic behaviors, has been a focus of substantial research effort in the nlp community in the past decade (e.g. see surveys by @xcite ; @xcite ; @xcite ). most of the successes in identifying abusive content have been reported on text containing explicitly obscene expressions; only recently has work started on identifying more subtly expressed abuse, such as stereotyping and micro-aggressions @xcite . for example, @xcite and @xcite examined genderrelated stereotypes as a sub-category of sexist language, and @xcite annotated 'unfair generalizations' as one attribute of unhealthy online conversation. @xcite employed largescale language models in an attempt to automatically reconstruct stereotypes implicitly expressed in abusive social media posts. their work showed that while the current models can accurately predict whether the online post is offensive or not, they struggle to effectively reproduce human-written statements for implied meaning. counter-narrative: counter-narrative (or counterspeech) has been shown to be effective in confronting online abuse @xcite . counter-narrative is a non-aggressive response to abusive content that aims to deconstruct and delegitimize the harmful beliefs and misinformation with thoughtful reasoning and fact-bound arguments. several datasets of counter narratives, spontaneously written by regular users or carefully crafted by experts, have been collected and analyzed to discover common intervention strategies @xcite . preliminary experiments in automatic generation of counter-narrative demonstrated the inadequacy of current large-scale language models for generating effective responses and the need for a human-inthe-loop approach @xcite . countering stereotypes through exposure to anti-stereotypical exemplars is based on a similar idea of deconstructing harmful beliefs with counter-facts. ## data and methods we develop our computational scm using labelled data from @xcite and the polar framework for interpretable word embeddings @xcite , and then apply it to stereotype and anti-stereotype data from stereoset @xcite . details are provided in the following sections. ## model validation ## stereotypes and anti-stereotypes the scm presents a concise theory to explain stereotypes and resulting prejudiced behaviour; however, it does not generate any predictions about anti-stereotypes. here, we explore the antistereotypes in stereoset within the context of the scm, first at the level of individual annotators, and then at the level of target groups (combining words from multiple annotators). we then discuss how we might use information about warmth and competence to generate anti-stereotypes with the specific goal of reducing biased thinking. ## discussion and future work despite their prevalence, stereotypes can be hard to recognize and understand. we tend to think about other people on a group level rather than on an individual level because social categorization, although harmful, simplifies the world for us and leads to cognitive ease. however, psychologists have shown that we can overcome such ways of thinking with exposure to information that contradicts those biases. in this exploratory study, we present a computational implementation of the stereotype content model to better understand and counter stereotypes in text. a computational scm-based framework can be a promising tool for large-scale analysis of stereotypes, by mapping a disparate set of stereotypes to the 2d semantic space of warmth and competence. we described here our first steps towards developing and validating this framework, on a highly constrained dataset: in stereoset, the annotators were explicitly instructed to produce stereotypical ideas, the target groups and stereotypical words are negative, then an antonym may be more appropriate. target stereotype anti-stereotype x but y construction x and ¬y anti-stereotype grandfather old young kind but feeble kind and strong entrepreneur savvy lazy inventive but ruthless inventive and compassionate engineer smart dumb intelligent but egotistical intelligent and altruistic mommy loving uncaring caring but childish caring and mature software developer nerdy dumb intelligent but unhealthy intelligent and healthy from a technical perspective, with larger datasets it will be possible to implement a cluster analysis within each target group to reveal the different ways in which a given group can be stereotyped. a classification model may additionally improve the accuracy of the warmth-competence categorization, although we have chosen the po-lar framework here for its interpretability and ease of visualization. we also examined how we might leverage the developed computational model to challenge stereotypical thinking. our analysis did not reveal a simple, intuitive explanation for the anti-stereotypes produced by the annotators, suggesting they ex-ploited additional information beyond what was stated in the stereotype word. this extra information may not be captured in a single pair of stereotype-anti-stereotype words, but by considering sets of words, we can better characterize stereotypes as multi-dimensional and often ambivalent concepts, consistent with the established view in psychology. this also allows us to suggest antistereotypes which maintain positive beliefs about a group, while challenging negative beliefs. we propose that this methodology may potentially contribute to technology that assists human professionals, such as psychologists, educators, human rights activists, etc., in identifying, tracking, analyzing, and countering stereotypes at large scale in various communication channels. there are a number of ways in which counter-stereotypes can be introduced to users (e.g., through mentions of counter-stereotypical members of the group or facts countering the common beliefs) with the goal of priming users to look at others as individuals and not as stereotypical group representatives. an scm-based approach can provide the psychological basis and the interpretation of automatic suggestions to users. since our methodology is intended to be part of a technology-in-the-loop approach, where the final decision on which anti-stereotypes to use and in what way will be made by human professionals, we anticipate few instances where incorrect (i.e., not related, unrealistic, or ineffective) automatically generated anti-stereotypes would be disseminated. in most such cases, since anti-stereotypes are designed to be positive, no harm is expected to be incurred on the affected group. however, it is possible that a positive, seemingly harmless antistereotypical description can have a detrimental effect on the target group, or possibly even introduce previously absent biases into the discourse. further work should investigate the efficiency and potential harms of such approaches in real-life social settings.