Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis (2024)

Abstract: Prior work has explored the possibility of using the semantic information obtained from embedding representations to quantify social stereotypes, leveraging techniques such as word embeddings combined with a list of traits (Garg et al., 2018; Charlesworth et al., 2022) or semantic axes (An et al., 2018; Lucy et al., 2022). However, these approaches have struggled to fully capture the variability in stereotypes across different conceptual domains for the same social group (e.g., black in science, health, and art), in part because the identity of a word and the associations formed during pre-training can dominate its contextual representation (Field and Tsvetkov, 2019). This study explores the ability to recover stereotypes from the contexts surrounding targeted entities by utilizing state-of-the-art text embedding models and adaptive semantic axes enhanced by large language models (LLMs). Our results indicate that the proposed pipeline not only surpasses token-based methods in capturing in-domain framing but also effectively tracks stereotypes over time and along domain-specific semantic axes for in-domain texts. Our research highlights the potential of employing text embedding models to achieve a deeper understanding of nuanced social stereotypes.

## introduction social stereotypes, representing the associations attributed to social groups (e.g., white, black, religious), are deeply embedded in and perpetuated by human languages. these stereotypes are both reflected in everyday language use and contribute to the reinforcement of societal biases. consequently, a growing topic of interest in nlp is whether and how computational techniques can be used to quantify these associations at scale. various methodologies have been developed to explore and measure these social biases in language. for instance, by calculating cosine similarities between the embeddings of traits (e.g., unhealthy, weak) and social groups within word embeddings @xcite @xcite , researchers can uncover how stereotypes emerge and persist across society. another related approach involves projecting social group embeddings along opposed semantic dimensions (i.e., semantic axes, such as beautiful -ugly @xcite ) to reveal tendencies toward particular semantic dimensions, suggesting certain stereotypes. social stereotypes are multifaceted and can intersect across different social groups or vary across different domains. for instance, @xcite investigated how racial stereotypes in the us persist in both academic and non-academic contexts, such as music and sports. their research demonstrated that the same social group can be associated with different stereotypes depending on the domain. similarly, @xcite found that asian american women performed better on a verbal test when their female identity was made salient but performed worse when their asian identity was emphasized. these findings suggest that domain-specific stereotypes significantly impact performance outcomes. such complexity underscores the importance of understanding in-domain stereotypes, which are stereotypes specific to particular contexts or domains. in-domain stereotypes are particularly challenging to analyze due to the need for contextual specificity in identifying associations with a target group. @xcite attempted to address this by replacing target social group words with neutral words (e.g. "person") and projecting neutral words' contextual embeddings onto semantic axes. however, this approach often resulted in the neutral word's identity dominating the analysis, leading to similar semantic poles (i.e., one end of one semantic axis) across different occupational categories before filtering. in this work, we develop a novel pipeline leveraging off-the-shelf llms and text embedding models to explore in-domain stereotypes. our pipeline first enhances semantic axes in two ways: (1) to address the gap where existing broad semantic axes fail to capture domain-specific variations in stereotypes, we utilize llms to generate more comprehensive and relevant axes. this approach allows us to include important contextual nuances, such as globalization -nationalism in economic analyses. (2) employing multiple pruning methodologies to refine existing semantic axes, ensuring inappropriate words are trimmed to avoid semantic confusion. then, whereas prior work has calculated associations with semantic axes using token embeddings, we explore whether these associations can be better modeled by embedding the context surrounding a target entity mention. using off-the-shelf text embedding models, we embed the context with target entity masked and adaptive semantic axes to measure group-and domain-specific stereotypes along these axes. we conduct extensive evaluations using automatic validation metrics and human evaluators, demonstrating that: (1) text embedding models encode semantic axes with greater consistency compared to previous token-based embeddings from bert; (2) our pipeline captures in-domain stereotypes that better align with human annotations compared to previous approaches; and (3) in a case study of us news discourse, our pipeline effectively captures general stereotypes, contrasts between countries, and changes in associational biases corresponding to real-world events along specific axes of interest. our results show that this innovative approach allows for a more nuanced and precise understanding of stereotypes within specific domains. our codes are available at https://github.com/qcznlp/adaptive_axes 2 background and related work ## proposed pipeline -adaptive axes in this section, we describe our adaptive axes pipeline. specifically, the construction of semantic axes relies on three main steps -(1) building embedding models, (2) building semantic axis poles and getting semantic axis vectors, and (3) projecting target vectors on semantic axes to show stereotypes. our study revisits these steps, aiming to construct a pipeline with high generalizability and accuracy. our general framework is shown in figure 1 . ## pipeline validation in this section, we present a series of experiments validating the effectiveness of our pipeline in two key areas: (1) do text embedding models encode semantic axes effectively? (2) does our pipeline capture stereotypes closely aligned with human intuitions? additionally, we conduct experiments to understand whether target-masked text embedding models could accurately predict affective information in appendix b. to address the first question, we employ uae-large-v1 @xcite , a model fine-tuned on bert-large, and sfr-embedding-mistral @xcite , which is based on mistral-7b @xcite , representing two state-of-the-art approaches to text embedding. each word or phrase is processed by these models without additional prompts to obtain candidate embeddings. for the second question, to avoid directly modeling the specific word of interest, we utilize attention masks and include 20 surrounding tokens (or all available tokens if fewer than 20) for context modeling. if the targeted social groups appear multiple times, all instances are masked before being processed by the embedding model. to introduce more domain-specific semantic axes, for our case studies we use @xcite to generate 13 new axes in the following domains: politics and governance, global trade and economics, and culture and education. these domain-specific axes enable a more precise analysis of stereotypes and their variations across different contexts. our domain-specific semantic axes are attached in appendix c. to effectively classify news articles in the news on the web corpus @xcite into various categories to mine domain-specific stereotypes, we use the zero-shot classification system @xcite 2019) and a list of candidate labels (global trade and economic, politics and governance, cultural and education, and none of above) to classify us news articles. two authors manually annotated 100 random news articles and find an average classification accuracy of 82%, which is sufficient to scale across the corpus. ## case study #1 in this section, we turn to real-world case studies with adaptive axes to ask one research question: how are different countries generally framed across various domains in us news discourse? ## case study #2 a key theoretical advantage of our pipeline is its flexibility in embedding new, user-specified semantic axes, whether they are individual words or phrases. moreover, we demonstrate that these domain-specific axes capture relatively significant variances among all axes within the in-domain texts. in this section, we conduct a case study to investigate the efficacy of new semantic axes curated by a large language model (llm) and quantify temporal changes along these axes. specifically, we seek to answer: do these newly introduced axes effectively quantify social framing? how do these axes correspond to real-world events? by addressing these questions, we aim to evaluate the capacity of these axes to reflect shifts in societal narratives over time. ## discussion in this paper, we introduce a novel pipeline that leverages text embedding models to encode both augmented semantic axes and news discourse contexts related to target social groups. by combining these elements, we project contextual embeddings onto semantic axes embeddings to gain insights into the underlying stereotypes present within the data. the study of social stereotypes is inherently complex and multifaceted. recent research has focused on the evolution of intersectional stereotypes-those that emerge at the intersection of multiple social categories-in various text sources @xcite . we identify that these text sources are themselves highly diverse, spanning a range of social domains such as culture, politics, and economics, as well as repositories like google books, coha, and common crawl @xcite . this diversity creates a layered and intricate landscape of social stereotypes within word embedding models. to address this complexity, we systematically categorize the text sources by domain and apply our pipeline to show how representations of social groups differ significantly across these domains. our findings contribute to the discourse on stereotype analysis by underscoring the nuanced and context-dependent nature of stereotypes embedded in text, thus advancing a more granular understanding of how these representations are framed. our pipeline enhances the capabilities of existing semantic axes by using text embedding models to encode arbitrary semantic axes. the high semantic accuracy of these models makes the axes more reliable for capturing subtle meanings and nuances in language. however, our approach still relies heavily on word co-occurrences within texts. for example, in the trade domain, the term overseas frequently appears, while canada is closely associated with north/northern, as determined by cosine similarity scores. these associations reflect general geographical information rather than stereotypes. this highlights a crucial limitation: while the pipeline effectively captures broad contextual relationships, it is less equipped to automatically discern stereotypical language from neutral or descriptive terms. more targeted methods will be needed in future work to accurately retrieve stereotypeladen language specifically related to social groups. such methods could involve refining the selection of semantic axes to focus on traits or attributes typically linked to social biases, or using advanced filtering techniques that distinguish between general context and stereotype-driven discourse. this refinement will allow the pipeline to move beyond identifying generic associations and instead focus on retrieving and analyzing language that specifically conveys stereotypes, thus providing more precise insights into the framing of social groups within various discourse contexts.