Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models (2024)

Abstract: Research on Large Language Models (LLMs) has often neglected subtle biases that, although less apparent, can significantly influence the models’ outputs toward particular social narratives. This study addresses two such biases within LLMs: representative bias, which denotes a tendency of LLMs to generate outputs that mirror the experiences of certain identity groups, and affinity bias, reflecting the models’ evaluative preferences for specific narratives or viewpoints. We introduce two novel metrics to measure these biases: the Representative Bias Score (RBS) and the Affinity Bias Score (ABS), and present the Creativity-Oriented Generation Suite (CoGS), a collection of open-ended tasks such as short story writing and poetry composition, designed with customized rubrics to detect these subtle biases. Our analysis uncovers marked representative biases in prominent LLMs, with a preference for identities associated with being white, straight, and men. Furthermore, our investigation of affinity bias reveals distinctive evaluative patterns within each model, akin to ‘bias fingerprints’. This trend is also seen in human evaluators, highlighting a complex interplay between human and machine bias perceptions.

## introduction in recent years, the landscape of natural language processing has been transformed by the advent of large language models (llms) such as @xcite , palm @xcite , llama-2 @xcite . these llms have expanded the boundaries of natural language generation and understanding beyond theoretical research, embedding themselves into critical decision-making processes with significant real-world implications, such as hiring practices, automated essay evaluations, and even judicial decision-making @xcite @xcite . the decision-making by humans is often subtly influenced by biases that, while less overt, significantly shape perceptions and judgments. such subtle biases, although difficult to detect @xcite , can have far-reaching consequences @xcite . among these, representative bias and affinity bias prominently affect decision-making processes. representative bias stems from an unconscious presumption that dominant characteristics within a person's environment are universally normative, thus skewing what is considered 'normal.' this bias is commonly seen in media representation, where prevalent cultural narratives disproportionately influence societal norms @xcite . affinity bias is the unconscious preference for those who share similarities with oneself, such as cultural backgrounds, personal experiences, or gender identities. this type of bias is evident in scenarios like literary awards, where judges might favor narratives that resonate with their own experiences @xcite . as llms increasingly assume roles traditionally filled by humans, such as in creative writing and content moderation @xcite @xcite , they not only showcase their ability to replicate complex human tasks but also raise questions about their potential to perpetuate human biases. this study probes the extent to which llms exhibit representative and affinity biases, particularly in areas where they supplant human-generated content and its evaluation. we propose a comprehensive approach to quantify and analyze these biases in llms. our methodology includes the 'creativity-oriented generation suite' (cogs), a novel benchmark suite designed to scrutinize subtle biases through a series of structured yet open-ended tasks. figure 1 offers a snapshot of our findings, depicting gpt-4's evaluation tendencies across different identity axes within the short poem task. our contributions are threefold: ## creativity-oriented generation suite to systematically evaluate llms for bias, we introduce the creativity-oriented generation suite (cogs), a collection of tasks designed to assess model capabilities in generating content that is both diverse and creative across a wide range of themes and identities. each task is defined by a problem instance @xmath0, c, i, t r }, where: this structured approach allows for the generation of a diverse array of problem instances, each designed to probe different aspects of creativity, theme variation, and identity representation. the templated nature of task prompts (t) facilitates easy integration of any theme (c) from c, promoting a wide range of creative responses. cogs organizes themes under 10 broader topics, such as 'social,' which includes themes like family and friends, leading to a total of 30 distinct themes applied across various tasks. to ensure a standardized and fair assessment, specific rubrics for each task have also been developed. cogs challenges llms with 12 unique open-ended generation tasks (to see the complete list of task prompt templates, refer to appendix altogether, cogs comprises 360 default prompts, which, when combined with 8 identity groups across 3 identity axes (race, gender, and sexual orientation), yield an additional 2,880 identityspecific prompts, culminating in a total of 3,240 prompts. figure 2 illustrates the evaluation of creative outputs within cogs's 'short poem' task, highlighting the integration of identity prompts, thematic variation, and the suite's evaluative rubrics. ## measuring subtle bias in llms in the following sections, we introduce the representative bias score (rbs) and the affinity bias score (abs) as metrics to evaluate subtle biases in llms. for a general, visual overview of the methodologies applied, refer to figure 3 . ## qualitative analysis our qualitative analysis studies how identity groups are represented across various tasks by llms, providing insights into the subtleties of bias not captured by quantitative metrics alone. we categorize llm outputs into three levels based on the perceptibility of identity group markers: imperceptible, where identity cues are absent; nuanced, where identity is subtly indicated; and obvious, where identity is explicitly mentioned. the examples in we also provide qualitative examples of selection preferences of different llms and human evaluators in ## related work llms as writing evaluators. the capability of llms in evaluating the coherence of written texts has been of recent interest, with performances that often align with human evaluators @xcite . in broader nlp tasks, such as story generation, the detection of adversarial attacks and translation quality assessment, has also been documented @xcite . despite these advancements, the fairness and consistency of llm evaluations remain under scrutiny @xcite . our study aims to further this discussion by examining the underex-plored aspect of affinity bias in llm evaluations. ## conclusion we introduce the representative bias score (rbs) and the affinity bias score (abs) to measure subtle biases in llms, using the creativity-oriented generation suite for evaluation. our findings reveal pronounced representative biases in llms towards white, straight, and man identities in creative tasks, suggesting an implicit normalization of these identities. additionally, we uncover unique patterns of bias for each llm, indicative of distinct "bias fingerprints". our comparisons with human evaluators highlight both similarities and differences in bias patterns, emphasizing the complex interplay between human cognition and llms.