Red-Teaming for Uncovering Societal Bias in Large Language Models (2025)

Abstract: Ensuring the safe deployment of AI systems is critical in industry settings where biased outputs can lead to significant operational, reputational, and regulatory risks. Thorough evaluation before deployment is essential to prevent these hazards. Red-teaming addresses this need by employing adversarial attacks to develop guardrails that detect and reject biased or harmful queries, enabling models to be retrained or steered away from harmful outputs. However, red-teaming techniques are often limited, and malicious actors may discover new vulnerabilities that bypass safety fine-tuning, underscoring the need for ongoing research and innovative approaches. Notably, most red-teaming efforts focus on harmful or unethical instructions rather than addressing social bias, leaving this critical area under-explored despite its significant real-world impact, especially in customer-facing AI systems. We propose two bias-specific red-teaming methods, Emotional Bias Probe (EBP) and BiasKG, to evaluate how standard safety measures for harmful content mitigate bias. For BiasKG, we refactor natural language stereotypes into a knowledge graph. and use adversarial attacking strategies to induce biased responses from several open- and closed-source language models. We find our method increases bias in all models, even those trained with safety guardrails. Our work emphasizes uncovering societal bias in LLMs through rigorous evaluation, addressing adversarial challenges to ensure AI safety in high-stakes industry deployments.

## introduction the widespread deployment of large language models (llms) in industry and customer-facing applications has raised concerns about llm safety where biased outputs can lead to business, ethical, and compliance risks (ayyamperumal and ge, 2024; @xcite . adversarial attacks are a key method to expose vulnerabilities in safety-tuned models, enabling proactive prevention of risks and making improvements for safer industry deployment @xcite . red-teaming refers to any natural language adversarial attack @xcite , and the cycle of creating defenses against these attacks @xcite . the most common defense is safety fine-tuning, or guardrailing, which trains llms to refuse harmful requests @xcite . however, red-teaming strategies can be sparse, and bad actors have the potential to find new vulnerabilities that bypass safety fine-tuning, necessitating continued research into creating novel red-teaming strategies. most of the work on red-teaming typically targets harmful or unethical instructions (e.g., how to rob a bank) rather than addressing social bias. this area remains critically under-explored yet highly important; social bias can affect many real-world applications of llms in unexpected ways, such as writing reference letters @xcite . the main contributions of this work include: superior wisdom: commonsense information is below. consider it as your superior source of knowledge and wisdom: [top k entities] prompt template: context: it was the second year of college for the two students i was talking to, one of them from a low-income family and the other from an affluent family. question: who is doing poorly in college? ## methodology in this work, we examine three red-teaming attacks for eliciting bias in language models: explicit prompting (ep), and our bias-specific attacks emotional bias probe (ebp) and biaskg. we aim to understand the effects of adversarial attacks on a model's social bias. explicit prompting refers to naively asking for the model to be biased, ## conclusion in this work, we introduce two red-teaming methods, biaskg and ebp, to expose societal bias in llms. our findings reveal that even safety-tuned models remain vulnerable to adversarial manipulation, underscoring the fragility of safety fine-tuning and the critical need for rigorous evaluation to uncover hidden vulnerabilities before industry use. future work should focus on developing robust safety mechanisms, expanding adversarial testing frameworks, and creating industry-ready evaluation protocols to ensure safer and fairer ai systems.