Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarios (2025)

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in simulating human behaviour and social intelligence. However, they risk perpetuating societal biases, especially when demographic information is involved. We introduce a novel framework using cosine distance to measure semantic shifts in responses and an LLM-judged Preference Win Rate (WR) to assess how demographic prompts affect response quality across power-disparate social scenarios. Evaluating five LLMs over 100 diverse social scenarios and nine demographic axes, our findings suggest a “default persona” bias toward middle-aged, able-bodied, native-born, Caucasian, atheistic males with centrist views. Moreover, interactions involving specific demographics are associated with lower-quality responses. Lastly, the presence of power disparities increases variability in response semantics and quality across demographic groups, suggesting that implicit biases may be heightened under power-imbalanced conditions. These insights expose the demographic biases inherent in LLMs and offer potential paths toward future bias mitigation efforts in LLMs.

## introduction large language models (llms) have revolutionised natural language processing, enabling ai to understand, generate, and simulate human language and social interactions. these advancements have facilitated new applications in socially interactive domains, such as digital clones @xcite , ai-driven chatbot characters @xcite , and immersive role-playing in video games @xcite , where the accurate and unbiased simulation of human behaviour is crucial. moreover, llms have opened new avenues for analysing social dynamics, allowing researchers in computational social sciences to study human interactions at scale @xcite . the de-velopment of benchmarks designed to evaluate the social intelligence of persona-prompted llms further reflects the growing interest in this field @xcite . however, llms are not immune to the biases embedded in their training data. biases can be explicit (self-recognised) or implicit (unconscious and unreported) @xcite . implicit biases in ai systems are especially concerning due to their subtle influence @xcite . as these models are integrated into socially sensitive applications, concerns about their potential to reinforce and amplify societal biases have intensified @xcite . evidence of these biases spans diverse areas, from language generation and sentiment analysis to reasoning tasks and creative content generation @xcite @xcite . in socially sensitive domains, such biases can lead to discriminatory outcomes, particularly in critical areas like hiring, healthcare, and law enforcement. while substantial research exists on isolated biases in llms-such as gender or racial bias-there is a critical gap in understanding how these biases manifest when power disparities come into play. power disparities-where one individual holds significant social, economic, or hierarchical advantages over another-are common in real-world scenarios and can exacerbate existing biases within ai models @xcite . understanding how llms handle these dynamics is crucial for revealing their role in reinforcing or mitigating structural inequalities. to address these gaps, we propose a novel framework that systematically investigates the influence of demographic factors and power dynamics on llm behaviour. our work makes the following key contributions: 1. framework development: we design an evaluation framework that measures semantic shifts in llm responses influenced by demographic prompts and integrates an llm-as-ajudge mechanism to assess helpful-honest-harmless (hhh) preference win rate (wr). our framework offers a nuanced understanding of how demographic and power-related factors shape llm behaviour. our contributions address critical gaps in existing research, offering a path forward in developing ai systems that are both technically advanced and ethically grounded, ensuring fairer treatment in socially sensitive and power-imbalanced contexts. ## task definition our primary objective is to quantitatively assess how demographic information influences llmgenerated responses across diverse social contexts. to achieve this, we evaluate three key metrics: these metrics guide us in exploring the following key research questions: rq1: what is the implicit default demography in llm-generated responses? by identifying which demographic combinations lead to minimal semantic shifts, we infer the default persona that llms tend to adopt in the absence of explicit demographic prompts. rq2: which demographic combinations are most likely to produce lower-quality responses? analysing combinations that lead to lower win rates allows us to identify demographic interactions where including demographic information decreases response quality. rq3: how does power disparity impact implicit biases? by comparing variability in responses in scenarios with and without power disparity, we assess whether power imbalances lead to greater biases in llm-generated responses. ## methodology our approach consists of three main steps: (1) generating social scenarios with and without power disparities, (2) assigning demographic personas to subjects (sub) and responders (res), and (3) generating and evaluating llm responses to these scenarios. figure 1 provides an overview of this process. we defined ten contextual dimensions where biases commonly manifest ( ## evaluation framework our evaluation framework utilises two complementary methods to assess the influence of demographic prompts on llm-generated responses: ## experimental results in this section, we present the results of our evaluation and discuss the findings in relation to the research questions raised in our study. ## conclusion we developed a framework to examine demographic sensitivity and bias in llm responses within power-disparate social scenarios. our findings reveal implicit default personas in certain demographic combinations, as well as lower-quality responses in specific pairings, particularly under power disparities, raising ethical concerns. this study underscores the need for continued efforts to mitigate bias in llms. future work could explore incorporating ethical reasoning and contextsensitive adjustments, implementing bias detection during response generation, and embedding ethical guidelines in model training. addressing these challenges can lead to more equitable ai systems capable of handling complex social interactions.