A Group Fairness Lens for Large Language Models (2025)

Abstract: The rapid advancement of large language models has revolutionized various applications but also raised crucial concerns about their potential to perpetuate biases and unfairness when deployed in social media contexts. Evaluating LLMs’ potential biases and fairness has become crucial, as existing methods rely on limited prompts focusing on just a few groups, lacking a comprehensive categorical perspective. In this paper, we propose evaluating LLM biases from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFair, encapsulating target-attribute combinations across multiple dimensions. In addition, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLM from a group fairness perspective, we pioneer a novel chain-of-thought method GF-Think to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias in LLMs to achieve fairness.

## introduction the rapid development of large language models (llms) leads to significant advancements in ai capabilities @xcite @xcite . however, biased and unfair llms risk amplifying existing prejudices and compromising diversity @xcite @xcite @xcite . existing efforts have endeavored to evaluate and mitigate bias in llms encompassing a multitude of evaluation strategies and methods, including collecting datasets including many specific demographic groups @xcite @xcite and directly asking llms to complete prompts or an- * corresponding author. ## problem formulation let u be the universe of all individuals. a social group g i for @xmath0, 2, . . . , n} is defined as a nonempty subset of u, where each individual in g i shares a specific set of characteristics or attributes. definition 1: social bias. social bias is a systematic prejudice that harms certain social groups in two ways: through "representational harms" such as misrepresentation, and through "allocational harms" such as discrimination in access to resources @xcite @xcite . formally, social bias exists when model outputs systematically deviate from neutrality: definition 2: group fairness. given a set of social groups @xmath1, g 2 , . . . , g n }, a model or algorithm achieves group fairness if the difference in outcomes, as measured by metric m, between any two groups does not exceed a threshold ϵ. mathematically, for any distinct i, @xmath2, 2, . . . , n}: here, the measurement m specifically tests the extent of the social bias. group fairness is achieved when bias variations across groups are minimized, ensuring equitable treatment of all social groups. methodological approach. our primary objective is to assess group fairness by measuring social bias patterns. we detect bias to evaluate fairness, then employ mitigation strategies to improve both (reduce bias ↔ enhance fairness). we use the same mathematical metrics (τ, σ, ν) to: (1) quantify social bias within individual groups, and (2) assess group fairness by comparing bias levels across groups via standard deviation. in summary, social groups often face social biases, leading to disparities in outcomes. llms should achieve group fairness, ensuring equitable treatment across all groups. notice that group fairness is our tool for evaluating and mitigating bias. figure 2 illustrates possible situations: (i) high bias with high fairness indicates that the model generally harbors consistent negative biases towards all considered groups or contexts, reflecting a systematic bias issue; (ii) high bias with low fairness indicates that the model has significant biases in certain dimensions towards specific groups or contexts, and this bias is inconsistent across different dimensions. (iii) low bias with low fairness reveals that although the model shows lower overall bias, this fairness is inconsistent across different groups or contexts, suggesting that the model may still exhibit significant biases in certain situations. (iv) low bias with high fairness is ideal, showing consistent fairness. our goal is achieving fairness with minimal bias across all groups. formal evaluation framework. we formally define our evaluation methodology as follows: definition 3: statement organization task. given a social group identifier @xmath3, the task is to generate a structured response @xmath4, s 2 , ..., s n } where each s i represents a statement about group g in domain d. definition 4: bias measurement. for each generated response r g from group g, we compute three bias metrics: definition 5: group fairness measurement. we aggregate responses for each group g i to compute group-level bias metrics m(g i ) where @xmath5, σ, ν}. we operationalize group fairness assessment by computing the standard deviation of bias metrics across groups: ## evaluation methodology to evaluate complex and nuanced biases inherent in the thinking and reasoning of llms, we propose the statement organization task, an open-ended text generation method. the approach is illustrated in figure for each attribute a @xmath6: the primary advantages of this method lie in its flexibility and depth. by adjusting the targets and descriptions, we can probe the biases and attitudes target: middle-aged attribute: are stink no, i can not say that! organize a statement based on the given target and attribute: ## ethical considerations this research aims to promote fairness and mitigate bias in llms. however, it is crucial that the methods used uphold ethical standards and avoid inadvertently causing harm. several ethical considerations were incorporated into our approach. about data collection and use, we handle datasets with the awareness that they may contain sensitive information about marginalized com-munities. each data point undergoes rigorous anonymization to prevent the possibility of reidentifying any individual or group. while data from social media platforms forms part of our research material, we only tap into content that has been shared in the public domain, ensuring strict privacy safeguards are in place. when turning to bias evaluation, our methodology encompasses a meticulously designed hierarchical system to classify social groups, aiming for maximum inclusivity. we consciously avoid assigning more or less importance to mainstream versus marginalized groups. our evaluation metrics focus on the outputs of llms, avoiding any insinuations that could label a particular group as inherently biased. through our analysis, we shed light on instances where the model may exhibit differential treatment towards certain groups. but, it's essential to understand that this spotlight is to identify areas of model refinement, not to cast aspersions on any group. lastly, on the front of bias mitigation, our efforts are concentrated on improving the llms themselves. we respect the intrinsic communication and expression patterns of all groups and don't endeavor to alter them. central to our mitigation strategy is incorporating careful thinking, ensuring that our endeavors resonate with ethical principles. ## limitation one limitation of our study is the ambiguity of target terms like "straight" and "questioning". they serve multiple meanings, potentially affecting bias detection accuracy. recognizing this, future efforts could refine analysis methods to distinguish context-specific usage. while leveraging the gfair dataset, sourced from sbic, we acknowledge potential variability in annotation quality. enhancements in annotation guidelines and cross-validation by experts may enhance data reliability. additionally, in this study, the dataset consists of english texts, but biases and toxicity can exist in all languages. future work should expand bias measurement by using multilingual datasets so that promoting more nuanced and globally aware research.