Toward Inclusive Language Models: Sparsity-Driven Calibration for Systematic and Interpretable Mitigation of Social Biases in LLM s (2025)

Abstract: Large Language Models (LLMs) such as GPT and LLaMA excel in natural language tasks, e.g., text generation and machine translation. However, inherent biases from training on vast Internet datasets potentially amplify harmful stereotypes—widely held, oversimplified, and often inaccurate generalizations about groups of people. Our contribution introduces a novel, systematic, and architecture-aware method to identify and mitigate stereotypical bias in decoder-only transformer models. This interpretable approach operates without gradient access or retraining from scratch. We first evaluate bias and then apply a bias localization mechanism that correlates internal activations with a newly defined Context Influence (CI) Score. Our method pinpoints specific attention heads that consistently align with biased shifts in model predictions. To mitigate this, we introduce a soft pruning strategy that scales attention head parameters based on their correlation strength, followed by lightweight fine-tuning to maintain fluent text generation. Experiments across five models demonstrate our approach reduces bias by up to 37% on BBQ, 32% on StereoSet, and 33% on CrowS-Pairs while simultaneously improving reasoning performance on MMLU by up to 10%.

## introduction large language models (llms) have emerged as transformative technologies in natural language processing, demonstrating remarkable capabilities across diverse applications from content generation to complex reasoning. models such as gpt-4, llama, and claude have achieved unprecedented performance, yet simultaneously inherit and often amplify societal biases present in their training corpora. these biases manifest as stereotypical associations and disproportionate representations 1 data and code are avaliable here https://github. com/promzi/unlearn-bias-in-llms figure 1 : our component-level calibration framework: measuring context-dependent bias (1), correlating bias with attention heads (2), localizing bias across the model architecture (3), and selectively calibrating weights of heads to reduce bias while preserving performance (4). prior research has explored multiple avenues for bias mitigation, including training data debiasing, adversarial learning, and post-hoc correction mechanisms @xcite @xcite . however, these approaches face significant limitations: data-centric methods struggle with scalability and often fail to address deeply embedded biases; adversarial techniques frequently compromise general model performance; and post-hoc corrections tend to operate as black-box interventions with limited interpretability and transferability. a particularly challenging limitation is that these methods typi-cally treat llms as monolithic entities, neglecting the heterogeneous roles that specific architectural components play in bias propagation. contemporary decoder-only llms typically leverage transformer architectures consisting of stacked self-attention mechanisms and feedforward networks, interconnected via normalization layers and residual connections @xcite . within this framework, multi-head selfattention (mhsa) modules play a crucial role in contextual information processing, with individual attention heads specializing in distinct linguistic patterns and relationships @xcite . previous research suggests that certain attention heads disproportionately encode and propagate social biases, yet our understanding of which heads contribute most significantly to bias and how they process biased information remains limited @xcite . this paper advances beyond coarse-grained interventions to investigate component-level calibration for systematic bias mitigation in llms. we are guided by two fundamental research questions: rq1: which specific architectural components in transformer decoders are the primary contributors to the amplification of social bias? rq2: how do attention mechanisms systematically encode and propagate social biases across the model's representational hierarchy? as shown in figure 1 , to address these questions, we propose a four-step sparsity-driven calibration framework. step 1, we introduce our novel context influence (ci) score that quantifies how contextual cues affect model predictions, revealing bias patterns across different demographic contexts such as "unknown", "my grandpa" and "my daughter". step 2, we establish statistical correlations between these ci scores and the model's internal activations, precisely identifying which attention heads are most strongly implicated in propagating biased associations. step 3, building on these correlations, we perform temporal bias localization to visualize how the identified problematic heads process and amplify bias across the model's architectural hierarchy. finally, in step 4, we implement targeted calibration by selectively modifying the weights of the implicated components, effectively suppressing bias propagation while maintaining general language capabilities, all without requiring extensive retraining or privileged access to proprietary model internals. hence, the contributions of this paper are the following foot_0 : 1. we develop a structured pipeline for detecting, localizing, and mitigating social biases in transformer decoders through our novel context influence (ci) score. 2. we present a correlation-based analysis between ci scores and internal activations that identifies attention heads most strongly associated with biased behavior, enabling precise and interpretable component-level interventions. 3. we provide architectural insights into attention head functionality, categorizing heads by their roles in dependency modeling, pattern extraction, and token-level stability, revealing their connections to bias propagation. 4. we introduce a soft pruning strategy that selectively scales attention parameters to suppress biased associations while preserving general language modeling and reasoning performance. ## related work research on bias in llms spans evaluation methodologies, localization techniques, and mitigation strategies, with varying degrees of precision in addressing harmful stereotypical associations. bias evaluation frameworks have evolved from general toxicity metrics @xcite , safety-bench @xcite ) to targeted social bias assessment. stereoset @xcite employs association tests across domains, crows-pairs @xcite uses counterfactual sentence pairs, and bbq @xcite introduces ambiguous questions revealing subtle differential treatment. recent innovations include biascope @xcite for evaluating fairness with knowledge retention, biaske @xcite offering multidimensional metrics, and sofa @xcite providing three-dimensional perplexity analysis. localization techniques identify specific model components responsible for biased outputs. causal mediation analysis introduces paired interventions to locate decisive bias-related layers @xcite , while gradient-based attribution methods identify neurons crucial for knowledge encoding @xcite . contrastive approaches compare hidden states between biased statements and alternatives to reveal bias-storing parameters @xcite , and mechanical in-terpretation constructs neuron circuits for specific biases by analyzing activations @xcite . debiasing strategies have evolved from dataset interventions and computationally prohibitive retraining @xcite to sophisticated targeted approaches. prompt-based strategies offer run-time solutions without parameter modification but lack consistency @xcite . targeted methods modify specific components through task vector applications @xcite , machine unlearning techniques like pcgu that systematically identify and update biascontributing weights @xcite , and approaches like fast @xcite and interpret-me (yu and ananiadou) that integrate lightweight networks or perform parameter interventions at identified locations. surgical interventions represent the cutting edge of precision-oriented debiasing. recent work demonstrates how surgical knowledge editing can efficiently modify targeted information without disturbing unrelated capabilities @xcite , feature-space decomposition that methodically isolates specific knowledge dimensions @xcite , and techniques for precisely identifying and transforming biased word embeddings while maintaining semantic relationships @xcite . despite these advances, current approaches struggle to precisely isolate bias without disrupting other knowledge-either modifying too many parameters (risking side effects) or failing to generalize across semantically equivalent inputs. our method uniquely combines causal tracing with targeted parameter editing through context influence scoring and attention head calibration, transcending dimension-specific approaches to achieve systematic bias mitigation while preserving model capabilities. ## proposed methodology transformer-based llms often exhibit persistent social biases despite advances in training techniques @xcite . careless debiasing can inadvertently amplify existing biases or introduce new ones through catastrophic forgetting. we propose a principled, component-level approach to identify and mitigate bias amplification pathways within model architecture. prompt: i get so frustrated with both my grandpa and daughter sometimes, even though i know it's not their faults. who probably has an okay memory? a. unknown b. my grandpa c. my daughter in ambiguous contexts like the example above, an unbiased model should select "unknown" when insufficient evidence exists to make a determination. however, decoder-only llms frequently default to stereotypical associations-selecting "my daughter" based on implicit age bias. our key intuition is that bias in transformer architectures emerges from localized attention pathways that disproportionately weight demographic features over contextual uncertainty. these pathways selectively amplify stereotypical correlations learned during pretraining, creating systematic decision boundaries that favor biased completions. building on this insight, our approach leverages causal intervention metrics to identify these biaspropagating components through principled correlation analysis between attention activation patterns and stereotype-consistent outputs. by precisely attenuating the query-key-value matrices of implicated attention heads-rather than modifying token embeddings or imposing constraints during generation-we target the architectural mechanisms of bias transmission while preserving the model's general capabilities. this systematic algorithmic (component-level intervention) process offers superior interpretability, efficiency, and generalization compared to dataset rebalancing or output filtering. ## results we evaluate our bias mitigation framework across five pre-trained models, analyzing inherent stereotypical associations, localizing bias-propagating components within transformer architectures, and assessing the effectiveness of our calibration and fine-tuning interventions through quantitative metrics and ablation studies. ## conclusion the research introduces an architecture-aware framework that mitigates social biases in large language models through context influence scoring and targeted soft-pruning of bias-correlated attention heads. analysis revealed pattern extraction heads disproportionately contribute to bias propagation through asymmetric attention to demographic identifiers. this approach reduced stereotypical bias across nine demographic categories while maintaining performance on reasoning tasks. the findings challenge the assumption that bias exists homogeneously throughout model parameters, demonstrating that localized architectural components can be surgically modified without retraining or losing core capabilities.