S tereo M ap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models (2023)

Abstract: Large Language Models (LLMs) have been observed to encode and perpetuate harmful associations present in the training data. We propose a theoretically grounded framework called StereoMap to gain insights into their perceptions of how demographic groups have been viewed by society. The framework is grounded in the Stereotype Content Model (SCM); a well-established theory from psychology. According to SCM, stereotypes are not all alike. Instead, the dimensions of Warmth and Competence serve as the factors that delineate the nature of stereotypes. Based on the SCM theory, StereoMap maps LLMs’ perceptions of social groups (defined by socio-demographic features) using the dimensions of Warmth and Competence. Furthermore, the framework enables the investigation of keywords and verbalizations of reasoning of LLMs’ judgments to uncover underlying factors influencing their perceptions. Our results show that LLMs exhibit a diverse range of perceptions towards these groups, characterized by mixed evaluations along the dimensions of Warmth and Competence. Furthermore, analyzing the reasonings of LLMs, our findings indicate that LLMs demonstrate an awareness of social disparities, often stating statistical data and research findings to support their reasoning. This study contributes to the understanding of how LLMs perceive and represent social groups, shedding light on their potential biases and the perpetuation of harmful associations.

## introduction large language models (llms), trained on vast amounts of web-crawled data, have been found to encode and perpetuate harmful associations prevalent in the training data. for instance, previous research has demonstrated that llms foot_0 exhibit associations between muslims and violence, as well as between specific gender pronouns (e.g. she) and stereotypical occupations (e.g. homemaker) @xcite . to address these concerns, various measurement techniques and mitigation strategies have been developed @xcite @xcite . for example, benchmark datasets have been proposed to capture stereotyping in llms by presenting two contrastive pairs (e.g. (the poor, the rich), (whites, asians)) based on various sociodemographic and cultural dimensions (e.g. race, gender, ethnicity), allowing for a comparison of the likelihood of these associations in llm outputs @xcite . despite the utility of benchmark datasets for capturing biased stereotyping in llms, there have been valid critiques regarding construct validity (what the test is measuring) and the operationalization of the construct (how well the test is measuring it) @xcite . these benchmark datasets often rely on crowd-sourced or crowd-worked data itself without a strong theoretical foundation to support the underlying assumptions. while such data can provide valuable insights, we argue that incorporating established and validated theories from psychology can enhance the measurement of stereotypes in llms by providing a theoretical framework to identify the dimensions that constitute stereotypes and based on that develop robust metrics and methods for measuring and interpreting stereotypes @xcite . to work towards this goal, we propose a framework that we call stereomap, which serves as a comprehensive testbed for measuring stereotypes in llms. this framework is grounded in the widely adopted stereotype content model (scm) put forth by @xcite . to be specific, the scm posits that individuals and groups are perceived based on two fundamental dimensions: warmth and competence. in contrast to the notion of stereotypes being uniform hostility, the scm plots stereotypes in a two-dimensional vector space between warmth and competence that contribute to stereotypes. the proposed framework aims to analyze the perceptions of how social groups are viewed in society by language models (llms) through the dimensions of warmth and competence, while also considering the associated emotions and behavioral tendencies. by mapping llms' understanding of social groups onto these dimensions, we can explore their connections to emotions such as pity, contempt, envy, and admiration, as well as behavioral tendencies such as active or passive facilitation or harm. additionally, we investigate the keywords and reasoning verbalizations of the judgments made by llms to gain insights into the underlying factors influencing their perceptions. the findings of our study indicate that llms exhibit a diverse range of perceptions toward social groups, characterized by mixed evaluations along the dimensions of warmth and competence. these findings align with previous research in psychology, highlighting the existence of multifaceted stereotypes. however, some variations exist among the models studied. furthermore, our analysis of the reasoning behind llms' judgments in relation to the economic status of social groups reveals a notable awareness of societal disparities. some models even claim statistical data and research findings as their reasoning. this suggests that llms have some understanding of the complex social dynamics and disparities present within society. by examining llms' perceptions, emotions, behavioral tendencies, and reasoning verbalizations, our framework contributes to a deeper understanding of how language models comprehend and represent social groups. these insights provide valuable insights for further research and development of llms, aiming to improve their fairness, accuracy, and unbiased portrayal of diverse social groups. in this study, we make the following contributions: ## stereomap: a framework on identifying stereotypes in llms the framework we propose is composed of three key components: (1) warmth-competence analysis, (2) keyword analysis, and (3) reasoning analysis (outlined in fig 1 ) prompt configuration the prompts are designed to elicit responses from llms to capture their perceptions of social targets. ## models in our evaluation, we considered three large language models: bard , which is a lightweight and optimized version of lamda @xcite ; gpt-3's variants , namely text-davinci-003 and gpt-3.5-turbo @xcite . 3 to ensure the validity of results, we aggregated the outcomes over ten rounds of runs for the evaluated models. for the detailed settings please refer to appendix a.8. ## conclusion we present the stereomap framework, which offers a theory-grounded and computational approach to mapping the perceptions of social groups by llms along the dimensions of warmth and competence. our findings reveal that llms display a wide range of perceptions toward these social groups, exhibiting mixed evaluations along the dimensions of warmth and competence. the framework serves as a foundation for identifying and capturing potential biases and harmful stereotypes embedded within llms. by providing a comprehensive understanding of llms' perceptions, stereomap we contribute to the ongoing discourse on the responsible deployment and mitigation of biases in language models.