Can Machine Unlearning Reduce Social Bias in Language Models? (2024)

Abstract: Mitigating bias in language models (LMs) has become a critical problem due to the widespread deployment of LMs in the industry and customer-facing applications. Numerous approaches revolve around data pre-processing and subsequent fine-tuning of language models, tasks that can be both time-consuming and computationally demanding. As alternatives, machine unlearning techniques are being explored, yet there is a notable lack of comparative studies evaluating the effectiveness of these methods. In this work, we explore the effectiveness of two machine unlearning methods: Partitioned Contrastive Gradient Unlearning (PCGU) applied on decoder models, and Negation via Task Vector, and compare them with Direct Preference Optimization (DPO) to reduce social biases in open-source LMs such as LLaMA-2 and OPT. We also implement distributed PCGU for large models. It is empirically shown, through quantitative and qualitative analyses, that negation via Task Vector method outperforms PCGU and is comparable to DPO in debiasing models with minimum deterioration in model performance and perplexity. Negation via Task Vector reduces the bias score by 25.5% for LLaMA-2 and achieves bias reduction of up to 40% for OPT models. Moreover, it can be easily tuned to balance the trade-off between bias reduction and generation quality, unlike DPO.

## introduction the widespread integration of language models (lms) into various everyday and industry applications has raised significant concerns on the trustworthiness of such models @xcite , for generating toxic, unfair, and harmful outputs. although numerous pre-processing techniques have been suggested to create unbiased datasets @xcite , the challenge is that specific pre-training data is not disclosed, making pre-trained models susceptible to intrinsic biases by default. on the other hand, an alternative approach to mitigating bias involves retraining the model on secure, unbiased data. however, this can be computationally expensive. as a result, the focus has been shifted to techniques that work to nullify the model's inherent bias. multiple techniques for mitigating bias exist, yet there is a lack of comparative studies to evaluate their respective advantages and disadvantages. in this study, we explore and compare different debiasing approaches through both quantitative and qualitative analyses. one approach is based on machine unlearning @xcite . it involves selectively forgetting unwanted data (or concepts) in a trained model while retaining useful information and maintaining computational efficiency. we compare two machine unlearning methods, partitioned contrastive gradient unlearning (pcgu) @xcite and unlearning via task vectors @xcite to a popular alignment-based approach using direct preference optimization (dpo) @xcite , which aligns the model to human preferences. we conduct experiments on the opt @xcite and llama-2 models @xcite . social bias. we focus on social bias that is characterized by deliberate or unintentional discriminatory attitudes or actions toward individuals, groups, or specific ideas and beliefs, resulting in prejudiced or unfair treatment @xcite . our main contributions are highlighted below: ## related work there have been different machine unlearning approaches used in the literature @xcite @xcite that focus on updating the learned behaviour of the model. @xcite propose using task vectors to steer the behavior of neural networks by specifying the direction in the weight space of a pre-trained model. similarly, @xcite propose machine learning for privacy in lms using the unlikelihood training objective to target token sequences with minimal impact on the performance of llms. partitioned contrastive gradient unlearning (pcgu) @xcite ) method debiases pretrained masked language models by systematically searching through a pre-trained masked language model to find the weights that contribute to bias and optimizes them. another line of research uses influence functions for debiasing @xcite . influence functions are used to estimate how training examples impact predictions during testing. for extended related work, refer to appendix d. ## methodology we adapt and extend the pcgu method to debias decoder models, unlike previous work @xcite , which used pcgu on encoder models only. also, in contrast to previous research, we cover additional protected groups beyond gender. see appendix a.1 for details of pcgu method. data. due to the autoregressive nature of decoder models, the protected group term (e.g., he/she for gender) cannot be positioned in the middle of the sentence. hence, we leverage the bias benchmark for qa (bbq) dataset @xcite ( ## conclusion in this paper, we compare two unlearning techniques to an alignment based approach to address social biases in language models, specifically opt and llama-2. our empirical findings highlight the ability of the negation via tv method to reduce bias, while maintaining overall model performance. it also provides greater flexibility compared to dpo based alignment by varying the scaling coefficient, which is not available for dpo. we also extend the pcgu approach for decoder-based models but observe mixed results across model families in terms of bias reduction, which we may further investigate in our future work. we hope that our work will ben-7 @xmath1, since at higher values of k the generations become incoherent (see figure 1 -left ).