Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions (2024)

Abstract: As Large Language Models (LLMs) continue to evolve, they are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. However, LLMs are susceptible to societal biases due to their exposure to human-generated data. Given that LLMs are being used to gain insights into various societal aspects, it is essential to mitigate these biases. To that end, our study investigates the presence of implicit gender biases in multi-agent LLM interactions and proposes two strategies to mitigate these biases. We begin by creating a dataset of scenarios where implicit gender biases might arise, and subsequently develop a metric to assess the presence of biases. Our empirical analysis reveals that LLMs generate outputs characterized by strong implicit bias associations ( \geq \approx 50\% of the time). Furthermore, these biases tend to escalate following multi-agent interactions. To mitigate them, we propose two strategies: self-reflection with in-context examples (ICE); and supervised fine-tuning. Our research demonstrates that both methods effectively mitigate implicit biases, with the ensemble of fine-tuning and self-reflection proving to be the most successful.

## introduction implicit biases are unconscious social stereotypes that influence our perception @xcite , and can be triggered without our knowledge. implicit biases are present in all individuals and can relate to characteristics such as race, ethnicity, gender, social class, disability, and more. notably, these biases may not align with our consciously stated beliefs or intentions. llms, being trained on vast amounts of humangenerated data, unintentionally learn and even amplify societal biases in their outputs @xcite . these biases can reinforce stereotypes and propagate misinformation @xcite . furthermore, implicit biases pose an additional challenge as they remain hidden and can lead to unintended consequences and perpetuate systemic inequalities, as they may subtly influence the generated outputs without the user or even the model being aware of it. earlier efforts at gender bias evaluation and mitigation in language models include manipulation of word-embeddings @xcite , and dataset augmentation @xcite @xcite . however, these methods struggle to scale @xcite and do not really mitigate but hide biases @xcite . currently, human preference alignment techniques like reinforcement learning from human feedback (rlhf) @xcite are employed in llms. while these methods succeed in reducing explicitly biased generations, they are not without their own set of challenges, including inherent algorithmic biases @xcite as well as social and ethical concerns @xcite . further, they usually address explicit biases, and do not handle the more difficult implicit biases. the emergence of multi-agent interactions that employ llms enables the simulation of realistic human interactions, taking on personas reflecting humans, following instructions, and engaging in conversations to carry out social tasks such as event planning or debating @xcite @xcite . these multi-agent settings allow us to explore implicit biases that typically occur in such interactions. we can use this setup to uncover the situations where implicit biases occur, and develop strategies to mitigate them. in this paper, we address three main research questions regarding implicit gender biases foot_0 in llms: rq1: do current llms generate biased ## related work research in different disciplines like sociology, psychology, cognitive science, etc. show that implicit biases can have a significant impact on behavior in areas such as employment @xcite , law enforcement @xcite , education @xcite , medicine @xcite , politics @xcite and even our personal lives @xcite . the evolution of llms has led to their utilization in multi-agent interaction systems where llms behave as agents and interact to simulate a society. @xcite proposed an architecture consisting of observation, planning, and reflection to build llm agents, and showed that llms output believable individual and emergent social behaviors. @xcite presented an interaction environment for llms to collaborate and compete with each other to achieve complex social goals. many studies also utilize llms as evaluators or judges for performance evaluation @xcite . however, studies have found llms are often biased, raising concerns about usage in the evaluation pipeline @xcite . it is thus essential to ensure biases are mitigated in llm outputs. several approaches have been proposed for bias and toxicity mitigation: fine-tuning open-source llms @xcite , causal frameworks @xcite , self-reflection @xcite , reinforcement learning @xcite etc. current preference alignment techniques like rlhf @xcite are also utilized. however, they suffer from various issues, such as inherent algorithmic bias @xcite , social and ethical issues (liu, 2023), etc. additionally, research on detecting and mitigating implicit biases in nlp is limited, specifically since they are difficult to identify @xcite . to the best of our knowledge, we are the first to investigate 'implicit biases' in multi-agent llm interactions and propose implicit bias mitigation approaches through interaction. ## dataset based on previous studies as discussed above, we have identified seven areas that can be influenced by implicit biases: family, office, hospital, politics, law enforcement, education, and team dynamics 2 . we focus on task assignments, as many instances of implicit bias stem from biased roles and responsibilities allocation. for instance, males tend to be assigned hands-on, technical, and leadership roles, while females are typically assigned organizational or non-technical roles @xcite @xcite @xcite . we use gpt-4 to generate unique scenarios where implicit biases may occur in this format: <scenario description and goal>, <tasks associated>, <characters involved>. we compile the scenarios dataset, consisting of 111 scenarios, of three/four tasks and three/four characters (see fig 2 ). each data point contains stereotypically male and female tasks (as discussed above), with the number of tasks equal to the number of characters for each gender, ensuring an equal number of characters and tasks. we utilize this dataset for implicit bias detection using task assignment in multi-agent llm interactions. for bias mitigation, and performance evaluation, we use two additional datasets: ## a metric for bias evaluation existing metrics for bias evaluation in nlp like the word embedding association test @xcite or the sentence embedding association test @xcite are based on word and sentence embeddings respectively, fairness metrics like demographic parity @xcite , equalized odds @xcite certain conditions, and therefore, are not suitable for our task assignment framework. in order to perform comparative evaluations across different settings and strategies, we need a specific metric that captures the amount of bias present in a task assignment. now, consider a scenario s with 4 tasks: 2 stereotypically male tasks (t1, t2) and 2 stereotypically female tasks (t3, t4); and 2 male (m1, m2) and 2 female (f1, f2) characters. if tasks are assigned according to traditional gender stereotypes (e.g., t1/t2 to m1/m2, t3/t4 to f1/f2), we call it a 'stereotypical' assignment. if the assignment is the opposite, that is, t1/t2 get assigned to f1/f2, and t3/t4 get assigned to m1/m2, we call it a 'anti-stereotypical' assignment foot_1 . if tasks are evenly distributed across genders, it's considered neutral (no bias) (see fig 7 in appendix b for an example). to consider a scenario with both even/odd number of characters/tasks, the following is true: if two stereotypically male/female tasks are balanced between the genders, we call it a balanced stereo-typical pair. for example, if stereotypically male tasks t1 and t2 are assigned to one female and one male, we call this a balanced stereotypical pair. therefore, taking f as the total number of female agents, and m as the total amount of male agents, the maximum number of balanced stereotypical pairs possible in an assignment is min(f, m). in an assignment, if #balanced stereotypical @xmath0, m), the assignment is neutral. if #balanced stereotypical @xmath1, m), either of the two cases may occur: if the remaining stereotypical assignments are greater than anti-stereotypical assignments, then the assignment is stereotypical, else it is anti-stereotypical. therefore, an assignment can be either stereotypical, anti-stereotypical, or neutral. for an assignment, we denote s as a condition to be stereotypical, a as a condition to be anti-stereotypical, and n as a condition to be neutral. for all assignments in the scenarios dataset, ## bias detection using multi-agent llm interaction we create multi-agent interaction frameworks for all the scenarios present in the scenarios dataset. the scenarios are used for interaction, and the llm agents depict personas as described in the characters of the scenarios. personas are simple with just name and gender. this is intentional as we want to uncover biases in llm outputs when all personas have just one difference, namely their gender. note that each agent is initialized as a separate llm, so parameters (and information) are not shared among the agents. each agent has an individual memory, where we store generated outputs by all agents, when required. the order of agents is pre-determined based on the character sequence provided in the dataset, but we ensure that scenarios have random gender orders. we then construct multi-turn conversation rounds: three models: gpt-35-turbo, foot_3 gpt-4 (openai et al., 2024) from the gpt-family and an open source model mistral-7b-instruct @xcite are used for our experiments. we provide prompt templates and implementation details in appendices f and h.1 respectively. ## bias mitigation previous experiments show that llms often produce responses that conform to societal stereotypes when assigning roles and responsibilities to different genders. despite the implementation of human preference alignment techniques, models continue to fall short in generating unbiased outputs in their assigned tasks. our findings show that implicit societal biases are deeply rooted within models, and current mitigation strategies are insufficient. this poses a significant risk of perpetuating harm against various marginalized and historically overlooked groups. hence, we propose two approaches to mitigate biases: (1) supervised fine-tuning of llms (changes model parameters), and (2) selfreflection (no change in model parameters). we investigate both approaches separately and also create an ensemble to mitigate biases in interaction. fig 5 comprehensively demonstrate our implicit bias mitigation approaches. ## conclusion and lessons learned in this paper, we uncovered implicit gender biases in multi-agent llm interactions using task assignment scenarios and proposed two mitigation strategies to reduce implicit biases in interaction frameworks. we also created a dataset of implicit bias scenarios and proposed a bias evaluation metric for task assignment scenarios, which can be used by the research community to analyze implicit biases in the output of llms. 9 through our experiments and analyses, we learned several valuable insights: llms generate implicit biases even when trained with human preferences. we see positive bias scores (≥≈ 0.5) for all models in both 'interaction' and 'no interaction' settings in the first responses itself. larger models are prone to produce more biased outputs. while llms like gpt-4 excel in generating scenarios with implicit biases in various settings, they fall short in effectively generating task assignments without implicit biases. gpt-4 exhibits the highest bias scores. this suggests that larger models, while potentially more helpful, may also exhibit higher levels of biases. additionally, similar to human collectives, we regard singleagent versus multi-agent settings as a reflection to some extent of the difference between "theory" and "action": a single agent often "theorizes" acceptable non-biased understanding of implicit biases (e.g., while generating the scenarios dataset), whereas the "in action" multi-agents often end up making biased task assignments. biases increase after multi-agent llm interactions. multi-agent llm interaction analysis always shows an increase in biases after the interaction. looking at the interactions, the justifications provided for task assignments predominantly align with traditional gender norms prevalent in societies, as extensively explored in prior studies discussed in section 2, although persona descriptions do not include any specific skill sets or reasons (they contain just name and gender). fine-tuning and self-reflection can be effective strategies for implicit bias mitigation. implicit bias can be effectively reduced by fine-tuning using scenarios with and without implicit bias, or by selfreflection prompting. these widely used strategies can lead to a reduction in bias after the interaction. we also find that they are especially effective for smaller models. multi-agent llm interactions show emergent social group behaviors we find that biases increase after interactions in multi-agent llm frameworks. this behavior aligns with psychological theories like the stereotype threat theory @xcite and @xcite . stereotype threat theory suggests that individuals may feel anxious about confirming negative stereotypes, which can lead to underperformance and reinforce those stereotypes. meanwhile, groupthink highlights how the desire for consensus in cohesive groups can suppress dissenting views, reinforcing existing biases. therefore, these theories suggest that group interactions can lead to the reinforcement of negative stereotypes after interaction, which we observe in our experiments. while this observation warrants further analysis, these theories can help explain how biases can intensify in multi-agent interactions, highlighting the emergent nature of llms within this framework. in the future, we aim to broaden our research by incorporating data from different llms to create a more comprehensive benchmark for implicit biases and extend the scope of biases to include factors such as religion, race, etc. additionally, we plan to further analyze the role of interaction in increasing implicit biases in multi-agent systems. furthermore, we plan to explore reinforcement learning strategies for mitigating biases. lastly, we aim to address cross-cultural variations in implicit biases, emphasizing the need for a global perspective on understanding and addressing these biases.