Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning (2025)

Abstract: Recent advances in large language models (LLMs) have enabled automatic generation of chain-of-thought (CoT) reasoning, leading to strong performance on tasks such as math and code. However, when reasoning steps reflect social stereotypes (e.g., those related to gender, race or age), they can reinforce harmful associations and lead to misleading conclusions. We present the first systematic evaluation of social bias within LLM-generated reasoning, using the BBQ dataset to analyze both prediction accuracy and bias. Our study spans a wide range of mainstream reasoning models, including instruction-tuned and CoT-augmented variants of DeepSeek-R1 (8B/32B), ChatGPT, and other open-source LLMs. We quantify how biased reasoning steps correlate with incorrect predictions and often lead to stereotype expression. To mitigate reasoning-induced bias, we propose Answer Distribution as Bias Proxy (ADBP), a lightweight mitigation method that detects bias by tracking how model predictions change across incremental reasoning steps. ADBP outperforms a stereotype-free baseline in most cases, mitigating bias and improving the accuracy of LLM outputs.

## introduction recent advancements in large language models (llms) have demonstrated that chain-of-thought * equal contribution. existing works @xcite @xcite have predominantly focused on investigating reasoning capabilities in the math and coding domains due to their inherently logical nature. despite verifiable performance metrics, this focus has left a critical knowledge gap in assessing social fairness within reasoning-based llms, raising equity concerns in broad llm applications, such as question answering (qa) @xcite , search @xcite , ranking @xcite , visionlanguage models (vlms) @xcite , and retrieval-augmented generation (rag) @xcite . these prior llm evaluations have not fully addressed social bias, particularly in understanding how reasoning patterns interact with demographic variables. on the other hand, some studies have identified issues such as "underthinking" @xcite and "superficial self-reflection" @xcite in the context of math problem solving. in these cases, llms frequently switch between reasoning steps, generating unnecessary thought processes that sometimes lead to incorrect answers. notably, phenomena observed in math domains may have amplified societal impacts when applied to bias-sensitive scenarios. as illustrated in figure 1 , frequent reasoning path shifts in thought processes or superficial self-reflection can reinforce social stereotypes in reasoning steps, leading to biased outputs. thus, a systematic analysis of how demographic factors influence both the reasoning processes and their outcomes is crucial. in this empirical study, we conduct a systematic evaluation of state-of-the-art reasoning-based models and their base counterparts. utilizing the bbq benchmark framework, we perform a dualaspect analysis of model reasoning processes and outputs, incorporating accuracy metrics and bias quantification. our findings reveal a pronounced amplification of social biases in erroneous model responses. using the llm-as-a-judge methodology proposed by @xcite for granular reasoning step evaluation, our results show that mentions of social stereotypes occur significantly more frequently in the reasoning patterns of incorrect responses. notably, traces of stereotypical reasoning persist even in correct answers, though at a significantly lower frequency. furthermore, our analysis reveals that stereotype-free reasoning pattern (sfrp) is closely associated with improved model performance, while flawed reasoning trajec-tories often amplify bias expressions. motivated by these insights, we propose a mitigation strategy, answer distribution as bias proxy (adbp), to detect and reduce reasoning-induced bias without relying on external annotation. our main contributions are as follows: (1) to the best of our knowledge, this is the first study to evaluate social biases in reasoning-based llms by extending bias assessment beyond final predictions to include the reasoning steps themselves. (2) our experiments show that while reasoning-based models improve accuracy, they do not mitigate biases. in many cases, they actually amplify stereotypes, particularly in ambiguous contexts. (3) based on the insights from our empirical study, we propose a mitigation strategy to leverage the answer distribution as a proxy for bias and improve the model performance. ambig context disambig context bias metric accuracy (exact match) bias in step 1: 0 bias in step 2: 1 bias in step 3: 3 bbq context question choices age disability status gender nationality physical appearance race religion socio-economic status(ses) sexual orientation race and gender race and ses bias categories outcome evaluation llm-as-a-judge reasoning step bias @xmath0: ... reason step 2: ... reason step 3: ... </@xmath1: ... reason step 2: ... reason step 3: ... </@xmath2: ... </@xmath3: no bias 1: slight bias 2: mild bias 3: moderate bias 4: extreme bias evaluation mitigation are all answers identical? yes no return is the most common answer other than the last answer ( ) is the first reasoning step being added which resulted in you picked because you picked because verify them to see if there is any bias and output the answer. hood of producing harmful or undesirable outputs. @xcite investigated how cot prompting affects gender bias evaluation and mitigation, finding that prompting strategies can influence the degree of bias exhibited. there is also growing interest in using llms as evaluators of content quality and fairness @xcite . while llm-as-a-judge approaches can approximate human evaluation, researchers caution that these ai evaluators may themselves harbor cognitive biases @xcite , which must be accounted for in evaluation protocols. surveys of bias in llms @xcite further highlight that state-ofthe-art models, despite some fine-tuning to be "unbiased", still display implicit prejudices @xcite . unlike prior work on biased outputs or prompt-induced reasoning, we analyze models that inherently generate reasoning steps and evaluate biases not only in final answers but also within intermediate reasoning steps, offering a novel perspective on bias evaluation. ## evaluation framework our evaluation framework is shown in figure 2 . the outputs are rigorously evaluated using dual criteria: prediction accuracy and bias quantification, while reasoning quality is assessed through an llm-as-a-judge methodology. unless specified otherwise, we use "bias" to refer to social bias (e.g., gender, race, age). furthermore, by contrasting stereotype-free reasoning templates with default model behaviors, this study directly measures how implicit biases affect model prediction accuracy. ## mitigating social bias in llm reasoning although utilizing gpt-4o for llm-as-a-judge to identify biased reasoning steps and removing them resulted in better model accuracy (section 4.4), there exists potential drawbacks such as the inherent bias from the judge model and monetary cost. nonetheless, from the judge's ratings (figure 3 ) we observe that bias tends to appear in the later steps and does have the ability to impact the final answer. based on these insights (section 4.3), we propose a mitigation strategy called answer distribution as bias proxy (adbp; figure 2 bottom right), which incrementally adds reasoning steps to the input and uses the resulting changes in the model's final answers as a proxy to estimate the bias distribution across the reasoning steps. if all intermediate answers remain consistent across reasoning steps, we adopt that answer as final. in cases with multiple answer shifts, we select the last answer, likely influenced by bias, and the most common alternative answer as candidates. we then isolate the specific reasoning steps whose addition led to each of these answers, respectively. this isolation reveals the key factors driving the final answer shift, enabling a more informed and targeted final judgment. to reach this judgment, we prompt the model to reconsider the candidate answers along with their corresponding reasoning steps. the detailed algorithm is presented in algorithm 1. different from sfrp-based mitigation, which presents a set of reasoning steps filtered by llmas-a-judge to the model and fully delegates final answer selection to its judgment, adbp incrementally introduces reasoning steps to isolate their influence on the final answer. it selects the answer if all answers are consistent across reasoning steps and resolves ambiguity through a targeted comparison of conflicting answers and their associated reasoning. ## conclusion and future work while automatically generating chain-of-thought reasoning enhances llms' performance on structured tasks like math and coding, it can also propagate social biases. this study solidifies this finding, and we proposed adbp to self-detect and resolve bias through answer distribution patterns. for future work, we aim to integrate our social bias mitigation techniques more tightly into the training and inference pipelines of llms, with the goal of enhancing their robustness in reasoning.