Fine-tuning LLM s with Cross-Attention-based Weight Decay for Bias Mitigation (2025)

Abstract: Large Language Models (LLMs) excel in Natural Language Processing (NLP) tasks but often propagate societal biases from their training data, leading to discriminatory outputs. These biases are amplified by the models’ self-attention mechanisms, which disproportionately emphasize biased correlations with sensitive tokens, like “he” or “she”, reflecting the sensitive attributes such as gender and race. To address this issue, we propose a novel fine-tuning method, called Cross-Attention-based Weight Decay (CrAWD), which modifies the LLM architecture to mitigate bias. CrAWD introduces a cross-attention mechanism between an input sequence and a sensitive token sequence, enabling the model to identify and selectively decay the attention weights of tokens associated with sensitive tokens. This reduces the influence of biased association on the model’s generation while maintaining task performance. Evaluations on real-world datasets demonstrate the effectiveness of our proposed CrAWD method. Notably, our method can handle multiple sensitive attributes by adjusting the sensitive token sequence, and it does not require full knowledge of sensitive tokens presented in the dataset, underscoring CrAWD’s versatility in promoting fair LLMs across various applications.

## introduction large language models (llms) trained on vast datasets have demonstrated remarkable capabilities on different natural language processing (nlp) tasks, including text generation @xcite and, text classification @xcite . however, their widespread usage in real-world settings has also raised concerns about the propagation of societal biases embedded in their training data @xcite . these biases can result in discriminatory outputs, affecting downstream applications in significant ways @xcite . addressing bias in llms is essential for building more equitable and responsible ai systems. bias in the nlp tasks is reported as biased or stereotypical associations with some sensitive attributes, such as gender, race, language, religion, etc. @xcite . for llms, the bias arises from the training data and is then encoded in the models' internal architectures. pre-training on large corpora drawn from the web often means the data contains inherent societal biases related to race, gender, and other sensitive attributes @xcite . pre-trained llms also exhibit significant bias amplification, meaning that even subtle biases in the training data become more pronounced in the model's output @xcite . the transformer-based models allocate attention weights to different tokens during training using the self-attention mechanisms @xcite . when exposed to biased training content, the model can disproportionately focus on biased associations with sensitive tokens, such as gendered pronouns or racially coded words. this leads to biased predictions, especially in tasks like text classification and generation @xcite . for example (shown in figure 1 ), llm learns a biased association between gender and occupations from the training data via self-attentions @xcite . for the next token generation, "my friend works at the hospi-tal. he is a ___", it generates "doctor" with a high probability due to the high attention weights from "hospital" and "he", where the association between "he" and "doctor" is biased. a fair llm should generate tokens based mainly on the relevant context of "hospital". various approaches have been proposed to mitigate bias in llms, ranging from data-level interventions to model-level adjustments. data-level methods include balancing the training data or removing biased examples @xcite , while model-level techniques focus on architectural modifications or post-processing steps that reduce the impact of biased outputs @xcite . for example, @xcite measures bias as explicit mentions of gender pronouns. their fine-tuning method directly adds a gender probability loss, derived from their bias metric, to the total loss to mitigate bias. bias mitigation finetuning to minimize a specific evaluation metric requires prior knowledge of the evaluation task and sensitive tokens, which lack generalizability across different types of biases in llms. there is no single metric to summarize all types of biases in llms. these studies focus on bias defined by specific model output, which limits their mitigation methods to generalize to other tasks. instead of a specific metric on the model output, we believe that the model's internal self-attention mechanism plays an important role in encoding and perpetuating bias. to address these challenges, we propose a bias mitigation technique, called cross-attention-based weight decay (crawd), which modifies the llm architecture to reduce the influence of biased association during llm fine-tuning. specifically, our approach introduces a cross-attention mechanism between an input sequence and a reference sequence consisting of sensitive tokens, such as "he" and "she". this cross-attention enables the model to identify the input tokens that have strong correlations with sensitive tokens. by selectively decaying the attention weights of these tokens during finetuning, we can scale down their biased influence on the model's predictions without significantly degrading the overall performance of correctness. the contribution of this work is as follows: our codes are available at https://github.com/farsheedhaque/crawd 2 related work bias in nlp systems has emerged as a significant concern, particularly as language models become increasingly integral to a wide range of applications. @xcite and @xcite demonstrated that societal biases like gender, racial, and cultural stereotypes, which may lead to discriminatory outcomes are embedded in model representations. large-scale datasets, often sourced from unfiltered content on the internet, are a primary contributor to bias in language models. these datasets frequently reflect societal prejudices, which models inadvertently learn and sometimes amplify during text generation @xcite . as @xcite highlighted, such biases do not just persist but may intensify in model outputs, leading to harmful stereotypes being reproduced at scale. to address this issue, researchers have developed several tools to measure bias in large language models (llms). notable among these are the stereoset dataset @xcite and the crows-pairs dataset @xcite , both designed to assess the presence of societal bias in generated text. in addition to measuring bias, various mitigation techniques have been explored. one popular approach involves generation-based strategies, such as zero-shot @xcite and few-shot prompting @xcite @xcite , where carefully crafted prompts are employed to guide models toward more equitable outputs. another strategy is the use of chain of thought (cot) reasoning, which has been shown to improve the fairness of model generations by making the model's decisionmaking process more transparent and deliberate @xcite . while these approaches have made significant strides in mitigating bias during text generation, there has been limited research on fine-tuning techniques specifically aimed at addressing bias during model training. for instance, @xcite proposed metrics to assess both explicit and implicit gender bias in generated text, incorporating these metrics into the loss function as a regularization technique to reduce bias. similarly, @xcite introduced an auxiliary model that predicts protected attributes, using the negative log-likelihood from these predictions as an energybased constraint to minimize the influence of biased tokens on the output. contrastive learning has also been employed as a bias mitigation technique, where stereotypical data points are pushed away while non-stereotypical data points are pulled closer in the model's latent space @xcite . ## bias in llms from the attention perspective large language models (llms) are pre-trained on vast corpora of data, which often contain social biases associated with sensitive attributes such as gender, race, or ethnicity. these biases are embedded in the model during pre-training and can propagate into downstream tasks. we aim to mitigate such bias embedded in the pre-trained llms using a fair fine-tuning method. we use the next token generation task for llm fine-tuning. given an input sequence @xmath0, where t i denotes the i-th token in the sequence and n is the sequence length, the llm predicts the next token t n+1 by adjusting its parameters θ to maximize the conditional probability p θ (t n+1 |t 1 , t 2 , . . . , t n ). llms utilize the selfattention (sa) mechanism to learn a robust and contextual representation of each token @xcite . sa mechanism assigns attention weights to all input tokens {t 1 , t 2 , . . . , t n } based on their contextual relevance to the target token t n+1 . specifically, the self-attention mechanism is defined as: our goal is to mitigate the effect of the sensitive token t s during fine-tuning, ensuring that the representation learned from the self-attention mechanism does not disproportionately emphasize the biased associations, thus reducing bias in the model's output while maintaining overall performance in downstream tasks. ## cross-attention-based weight decay in this section, we propose a novel fair fine-tuning method, called cross-attention-based weight decaying (crawd), to mitigate bias in pre-trained llms. the core mechanism in crawd is a cross-attention (ca) mechanism between input sequences and reference sensitive tokens. the crossattention mechanism helps to identify potentially biased associations. through weight decay, the fine-tuned llm tries to deemphasize these biased associations. in the end, it learns a debiased contextual representation for downstream tasks. ## conclusion in this work, we introduced cross-attention-based weight decay (crawd), a method for mitigating multiple types of bias in llms. crawd uses crossattention mechanism to identify and reduce biased associations with sensitive attributes during finetuning, without requiring prior knowledge of bias. our approach successfully reduces bias in model outputs while preserving performance, offering a versatile and effective solution for promoting fairness in diverse nlp tasks across various domains.