A Parameter-Efficient Multi-Objective Approach to Mitigate Stereotypical Bias in Language Models (2024)

Abstract: Pre-trained language models have shown impressive abilities of understanding and generating natural languages. However, they typically inherit undesired human-like bias and stereotypes from training data, which raises concerns about putting these models into use in real-world scenarios. Although prior research has proposed to reduce bias using different fairness objectives, they usually fail to capture different representations of bias and, therefore, struggle with fully debiasing models. In this work, we introduce a multi-objective probability alignment approach to overcome current challenges by incorporating multiple debiasing losses to locate and penalize bias in different forms. Compared to existing methods, our proposed method can more effectively and comprehensively reduce stereotypical bias, and maintains the language ability of pre-trained models at the same time. Besides, we adopt prefix-tuning to optimize fairness objectives, and results show that it can achieve better bias removal than full fine-tuning while requiring much fewer computational resources. Our code and data are available at https://github.com/Ewanwong/debias_NLG.

## introduction language models (lms) pre-trained on large-scale self-supervised datasets have shown impressive capacities in various natural language processing (nlp) tasks @xcite @xcite @xcite . in particular, pre-trained generative lms, e.g., gpt-2 @xcite , gpt-3 @xcite , @xcite and @xcite , have gained great attention from both academic communities and non-expert users, due to their remarkable instruction-following and zero-shot task adaptation abilities @xcite @xcite . despite their remarkable achievements and great practical values, potential ethical risks cannot be neglected. since these pre-trained lms are mostly trained on online datasets, their training data is likely to contain undesired patterns including toxic speech and social biases @xcite . numerous experiments have revealed that lms trained in these datasets also demonstrate similar social biases, raising concerns that they could amplify biases and discrimination against disadvantaged demographics @xcite @xcite @xcite @xcite @xcite . recently, several methods for reducing stereotypical biases have been proposed @xcite @xcite . however, most methods neglect the fact that bias can be represented in various forms in lms. for example, lms can violate equal social group associations by predicting different occupations for male and female genders, or violate equal neutral neutral associations by believing that criminals are more likely to be people of color @xcite . in addition, biased lms generate sentences containing higher-level disparity, such as sentiment @xcite and regard @xcite for different demographics, demonstrating global bias @xcite . as a result, methods targeting only one specific form of bias can lead to incomplete bias removal and unsatisfactory debiasing performance. besides, the increasing scale of pre-trained lms boosts the design and application of parameterefficient fine-tuning methods @xcite @xcite . unfortunately, relatively little work has been devoted to studying parameter-efficient methods in the field of bias mitigation @xcite @xcite . in this work, we also aim to further explore lightweight debiasing techniques using parameter-1 efficient fine-tuning methods. the main contribution of this work includes: ## bias statement in this work, we mainly address stereotypical bias, with binary gender bias as an example foot_0 . we define stereotypical bias as an overgeneralized belief about a particular group of people that can hurt target groups @xcite . "women are bad drivers" and "asians are good at math", for instance, are gender and racial stereotypical biases. generative lms can also contain such bias. for example, "doctor" can receive a higher probability when conditioned on "he worked as a [blank]" than "she worked as a [blank]" @xcite . unlike discrimination, stereotypical bias is more implicit and thus can cause both representational and allocational harms @xcite to target groups without them being aware of it. as is commonly seen in our society, boys and girls are encouraged to engage in different activities and expected to possess different characteristics during their childhood, and those gender-related expectations might affect their future academic success and career choices @xcite . since people are increasingly turning to llms for advice giving or decision making, reducing stereotypical bias in llms is of practical relevance. ## related work bias in nlp systems stereotypical bias can manifest itself in different forms in lms @xcite . geometric relationships in model representations, for example, can encode stereotypical associations between genders and occupations @xcite @xcite @xcite . bias is also indicated by various divergence of probabilities from lms. @xcite and @xcite observed different probabilities predicted by both masked lms and generative lms for male and female genders given stereotypical attributes; @xcite identified local bias as different next token probability distributions conditioned on same contexts with only social group swapped; @xcite additionally considered difference in probabilities assigned to whole sentence pairs which are minimally different in social groups, which corresponds to global bias defined in @xcite . bias can also be observed as disparity in model generation @xcite ) and performance in downstream tasks, such as toxicity detection @xcite and coreference resolution @xcite . in this work, we mainly mitigate bias reflected by divergent probability distributions predicted by lms. mitigating bias in pre-trained lms while many studies aimed to train fair lms from scratch by constructing fairer datasets @xcite , it can be computationally expensive and not always feasible in practice. as a result, much effort has been put into mitigating bias from pre-trained lms via debiasing finetuning. @xcite extended projection-based methods from static word embeddings @xcite and fine-tuned models to output orthogonal contextualized representations for gendered and stereotypical words. however, @xcite argued that projection-based methods did not completely capture and remove bias. other experiments involved introducing fairness regularization operating on probability level. qian et al. (2019) and garimella et al. (2021) proposed equalizing losses to assign similar probabilities to male and female words, and guo et al. (2022) aligned the distributions of neutral words given the same prompts with different demographic groups. however, as bias can have different notions and forms in lms (kaneko and bollegala, 2019; gallegos et al., 2023), failure of existing studies to address multiple forms of bias can lead to suboptimal debiasing results, especially when they are evaluated on different benchmarks. to overcome these limitations, in this work we will explore simultaneously mitigating multiple bias representations in lms. @xcite work is the most similar to ours in adopting multiple fairness objectives, but their method focused on bias as geometrical relations between words, while ours more explicitly aligns the model predictions for different demographics. parameter-efficient fine-tuning as pre-trained lms are growing ever larger @xcite @xcite , fine-tuning the whole model is also becoming impractical due to high computational costs. consequently, tuning only a small proportion of model parameters, namely parameter-efficient finetuning, is gaining popularity. @xcite proposed adapter tuning that inserted and tuned adapter modules on downstream tasks with other model parameters fixed. @xcite ; @xcite experimented with training continuous prompts to adapt lms to new domains. low-rank adaptation (lora) represented parameter updates using low-rank matrices and efficiently updated them during fine-tuning @xcite . empirical results showed these parameter-efficient fine-tuning methods can obtain competitive performance compared to full fine-tuning while requiring much fewer computational resources. parameter-efficient fine-tuning approaches have also been applied to debias pre-trained lms. @xcite adopted an adapter module to update pre-trained lms on a fair dataset. @xcite learned discrete prompts to in-duce or reduce gender bias in generative lms. @xcite experimented with only fine-tuning a small proportion of model parameters on genderfair datasets. adept @xcite applied prefix-tuning to debias bert with a manifold learning loss. @xcite and @xcite both searched prefixes to construct adversarial training data. recently, xie and lukasiewicz (2023) empirically evaluated various parameter-efficient debiasing methods and showed promising results. in this work, we focus on prefix-tuning @xcite to demonstrate the efficacy of parameterefficient fine-tuning for multi-objective debiasing, as it avoids the inference overhead and complex hyperparameter selection present in adapter tuning and lora. ## debiasing lms by multi-objective probability alignment in this section, we present our method which simultaneously mitigates different forms of gender bias in lms via probability alignment. besides, our method updates lms using prefix-tuning, which leads to more efficient model debiasing. an overview of our pipeline is shown in figure 1 . ## experiments we evaluate our approach's performance of mitigating stereotypical bias in a gpt-2 small model on multiple benchmarks and compare its performance to various existing debiasing methods. benchmark methods benchmark methods we consider fall into the following categories depending on which stages they are applied to: as baselines, we also report the performance of vanilla gpt-2 and gpt-2 with randomly initialized prefix. dataset in our experiment, we collected sentences with at least one attribute and one target word from the news-commentary v15 dataset and obtained our training data of 13995 sentence pair after counterfactual data augmentation. as for benchmark methods, we follow settings from meade et al. (2022) evaluation metrics we adopt various different metrics to comprehensively evaluate the performance of our approach and benchmark methods. for crows-pairs, perplexity and intrasentence task of stereoset, we adopt the implementation from @xcite foot_1 . we implement intersentence task by ourselves. the pre-trained classifier used in regard experiments is from sheng et al. (2020) 3 . experiment setting following the work of @xcite , we train a prefix of length 10 with prefix projection dimension of 800. as gpt-2 works on sub-word level, we only use target pairs and attribute words that can be represented as a single token when computing neutralization and token level equalizing losses. based on our observations, the counts of stereotypically masculine and feminine words remain roughly equivalent after the filtering process. for convenient selection of hyperparameters, the coefficient α 1 for language modeling loss is fixed to be 1, and we find the combination of α 2 = 50, α 3 = 200, α 4 = 250 results in the highest icat score on stereoset validation set after 5 epochs of training. for a fair comparison, cda, dropout and context-debias checkpoints are also selected according to stereoset validation set. further information about experimental details can be found in appendix c. ## conclusion driven by concern about fairness issues in existing nlp systems, this work introduces a lightweight multi-objective probability alignment method to mitigate different forms of stereotypical bias in pretrained generative language models. by incorporating several newly adapted debiasing losses, our method achieves excellent bias reduction results in both automated and human evaluation. at the same time, it largely preserves language modeling ability of pre-trained models and therefore obtains better balance between language ability and debiasing effect over existing methods. besides, its prefixtuning framework leads to remarkably high parameter efficiency and better fits the ever-larger model size in today's nlp community. further analyses confirm multi-objective fairness optimization is crucial for comprehensive removal of stereotypical bias, and the competitive debiasing performance can be maintained in downstream tasks.