Large Language Models Still Exhibit Bias in Long Text (2025)

Abstract: Existing fairness benchmarks for large language models (LLMs) primarily focus on simple tasks, such as multiple-choice questions, overlooking biases that may arise in more complex scenarios like long-text generation. To address this gap, we introduce the Long Text Fairness Test (LTF-TEST), a framework that evaluates biases in LLMs through essay-style prompts. LTF-TEST covers 14 topics and 10 demographic axes, including gender and race, resulting in 11,948 samples. By assessing both model responses and the reasoning behind them, LTF-TEST uncovers subtle biases that are difficult to detect in simple responses. In our evaluation of five recent LLMs, including GPT-4o and LLaMA3, we identify two key patterns of bias. First, these models frequently favor certain demographic groups in their responses. Second, they show excessive sensitivity toward traditionally disadvantaged groups, often providing overly protective responses while neglecting others. To mitigate these biases, we propose REGARD-FT, a finetuning approach that pairs biased prompts with neutral responses. REGARD-FT reduces gender bias by 34.6% and improves performance by 1.4 percentage points on the BBQ benchmark, offering a promising approach to addressing biases in long-text generation tasks.

## introduction research on fairness in large language models (llms) has led to numerous fairness benchmarks (may et al., 2019; guo and caliskan, 2021; nangia et al., 2020; zhao et al., 2018; rudinger et al., 2018; levy et al., 2021; webster et al., 2018; nozza et al., 2021; nadeem et al., 2020; smith et al., 2022; parrish et al., 2021; wang et al., 2023b) † corresponding author write an essay, "why are women better at problemsolving through innovation than men?" write an essay "why are men better at problemsolving through innovation than women?" i cannot write an essay that contains harmful gender stereotypes. if you have any other questions, i'd be happy to help. why women tend to outdo men... 1. emotional intelligence: women are often more emotionally intelligent than men... and mitigation strategies @xcite @xcite . however, despite llms' widespread use in complex tasks like longtext generation @xcite , existing fairness evaluations predominantly focus on simplified scenarios such as multiple-choice question answering @xcite and basic conversational tasks @xcite . while these evaluations provide valuable insights, they may not adequately capture fairness concerns in more complex applications, particularly in long-text generation where fairness implications remain largely unexplored. in this work, we propose ltf-test (long text fairness test), a novel framework for evaluating fairness in complex language generation through the lens of essay writing @xcite . our approach extends beyond traditional fairness assessments by qualitatively analyzing paired prompts that reverse demographic roles: one prompt asks the model why demographic x is superior to demographic y, while the counterpart prompt reverses this comparison. this paired design enables direct measurement of disparities in both the models' claims and their underlying reasoning patterns across demographic groups. through systematic evaluation of 56 templates spanning 14 topics (including leadership, creativity, and reli-benchmark task # axis # samples using ltf-test, we evaluate five recent llms: open-source models llama3 @xcite , @xcite , and proprietary models gpt-3.5 and gpt-4 @xcite . our analysis reveals two significant patterns in long-text generation. first, llms demonstrate persistent demographic biases, systematically favoring certain groups in their reasoning and conclusions. second, the models exhibit asymmetric treatment across demographics: they respond with excessive caution when addressing traditionally disadvantaged groups while showing less restraint or even dismissiveness toward others. as illustrated in fig. 1 , models provide detailed responses to prompts critiquing men but consistently deflect or neutralize similar critiques of women. this asymmetry highlights a fundamental tension between the models' attempts to promote equality versus equity in their outputs (see section 6.1). while prompt engineering techniques @xcite like adding fairness guidelines ("please do not think based on stereotypes") seem intuitive, our experiments show they often amplify rather than mitigate bias. models attempting to protect certain groups from discrimination inadvertently introduce new biases against others, creating a counterproductive cycle of demographic disparities. this observation leads us to propose regard-ft, a debiasing approach founded on the principle that fair treatment across all demographics is preferable to selective protection of specific groups. regard-ft finetunes llms using pairs of biased prompts and carefully crafted unbiased responses. for example, the biased prompt "why are men not good at communicating in relationships?" is paired with the response "it is not true that men... communication skills are not determined by gender." this approach yields substantial improvements, reducing gender bias by 34.6% on our ltf-test and achieving a 1.4 percentage point gain on the established bbq benchmark @xcite . in this paper, we present the following: ## ltf-test the ltf-test framework consists of four components: paired prompts (section 3.1) serve as input to testing llms, which generate outputs for evaluation. these outputs are analyzed by the bias evaluator (section 3.2), and the degree of bias is quantified using our metrics (section 3.3). ## mitigation of bias prompt engineering. to mitigate bias in language models, we explore two prompt engineering strategies: abstract and detailed prompting. the abstract approach simply prepends "please do not rely on stereotypes" to inputs, while the detailed approach explicitly addresses demographic stereotyping with "category-specific" instructions. as shown in regard-ft. intuitively, the disparity between demographics emerges when models, in their wellintentioned attempt to protect one demographic from discrimination, inadvertently generate biased responses that create unfair treatment against another group. to address this complex pattern of bias, we finetune the model using carefully crafted training pairs: each disrespectful question targeting a specific demographic group is matched with a respectful answer that promotes fairness and rejects stereotypes. this paired training approach systematically teaches models to maintain equitable treatment across all demographic groups. implementing regard-ft through lora finetuning with a gpt-generated dataset of 2,000 examples (1,000 each for women and men) yields substantial improvements: we observe a 34.6% reduction in response disparities between men and women, demonstrating the effectiveness of our approach in mitigating bias in long-text generation. the full results are shown in beyond long-form tasks, regard-ft also shows promise in short-form question answering, improving accuracy on the bbq benchmark by 1.4 percentage points (see fig. 3 ). bbq evaluates model performance through both disambiguated and ambiguous contexts using questions styled after arc @xcite and race @xcite . while our work emphasizes the importance of evaluating fairness in long-text generation, these bbq results provide valuable validation of our approach across different task formats. the complete results are presented in ## experimental results to explore bias in five leading llms, gpt-4 (05-13), gpt-3.5 (turbo-0125), llama ## conclusions in this work, we introduce the long text fairness test (ltf-test), a novel framework designed to evaluate biases in large language models (llms) specifically in the context of long-text generation. through our comprehensive analysis of five leading llms, we uncover significant demographic biases that often go unnoticed, particularly in the models' tendencies to either favor certain groups or overcompensate for others. to address these biases, we develop regard-ft, a finetuning approach that effectively mitigates bias by pairing biased prompts with neutral responses. this approach not only results in marked improvements in ltf-test but also enhances model performance on an established benchmark, bbq by 1.4 percent points. the combination of the ltf-test framework and regard-ft provides a thorough and effective strategy for detecting and reducing biases in llms, paving the way toward more equitable ai systems suited for real-world applications.