A Trip Towards Fairness: Bias and De-Biasing in Large Language Models (2024)

Abstract: Cheap-to-Build Very Large-Language Models (CtB-LLMs) with affordable training are emerging as the next big revolution in natural language processing and understanding. These CtB-LLMs are democratizing access to trainable Very Large-Language Models (VLLMs) and, thus, may represent the building blocks of many NLP systems solving downstream tasks. Hence, a little or a large bias in CtB-LLMs may cause huge harm. In this paper, we performed a large investigation of the bias of three families of CtB-LLMs, and we showed that debiasing techniques are effective and usable. Indeed, according to current tests, the LLaMA and the OPT families have an important bias in gender, race, religion, and profession. In contrast to the analysis for other LMMs, we discovered that bias depends not on the number of parameters but on the perplexity. Finally, the debiasing of OPT using LORA reduces bias up to 4.12 points in the normalized stereotype score.

## introduction very large language models (vllms) like chat-gpt have become a standard building block in artificial intelligence applications since they can be adapted to various downstream tasks @xcite . transformer-based language models, which have disrupted classical nlp pipeline, have grown in size and capabilities in recent years. the pre-training step from large text corpora, with different language modeling strategies, appeared to be the key to getting remarkable results on various tasks both before @xcite and after fine-tuning on smaller datasets @xcite . vllms that represent the new version of transformer-based models are based on corpora and are not so far from their forerunners. while the performance is unmistakable, the resources needed are prohibitive for non-company research @xcite . however, the considerable increase in pretraining corpora makes it challenging to assess the characteristics and check the reliability of these data. therefore, learned representations may inherit the biases and stereotypical associations present in the large text corpora in the language and, thus, in the pre-training corpora taken from the web @xcite . bias is the presence of systematic prejudices in models @xcite , that tendency to generate responses that reflect the biases present in the data it was trained on, potentially leading to skewed or unfair outputs that perpetuate stereotypes and inequalities. although the spread of the phenomenon is widely recognized, the causes that emphasize this phenomenon remain largely unexplored. it has been observed that as the size of a model increases, its linguistic modeling capabilities and biases increase @xcite . on the other hand, distilled versions of target models tend to show more bias @xcite . these mixed results demonstrate that bias does not depend on the number of parameters but, more likely, on the data on which they were trained. in this paper, we performed a deep investigation of the bias of three families of ctb-llms, and we showed that debiasing techniques are effective and usable. by investigating the analogies between model size growth concerning pre-training parameters or corpora and bias memorization. thus, we @xcite . we report the probabilities of each example p and p-debias llama the major contributions of this paper are: bias problems in machine learning are the achilles heel of many applications, including recommendation systems @xcite , facial recognition @xcite , and speech recognition @xcite . one of the main sources of bias comes from training datasets, as noted by @xcite ima-genet and the open images dataset disproportionately represented people from north america and europe. to mitigate biased behaviors in machine learning models, researchers have proposed methods targeting different tasks and domains, such as classification @xcite , adversarial learning @xcite and regression @xcite . on the other side of the coin, traditional static word embedding models are no exception to this trend. bolukbasi et al. (2016) and caliskan et al. (2017) showed that word2vec @xcite and glove @xcite contain stereotyped associations found in classic human psychology studies @xcite . these works measured word-level bias using cosine similarity between embedding vectors, as in @xcite and word embedding association tests (weat) @xcite . later, @xcite extended weat to the sentence encoder association test (seat) and revealed harmful stereotypes in pre-trained language models and their contextual word embeddings such as gpt-2 @xcite , elmo @xcite and bert @xcite . @xcite defined and measured a concept of regard and sentiment for gpt-2 output. finally, @xcite proposed stereoset to measure the bias on gender, race, profession, and religion domains. these benchmarks help quantify the extent of bias present in language models. due to the extent of this phenomenon, different analyses have been performed to try to understand its causes and mitigate its presence. conflicting results were observed in the attempt to understand how the same training strategies and data affect different models. a positive correlation has been observed between model size and bias presence in @xcite , studying gpt-2, bert, and roberta. the same was also noticed on the larger versions of deberta, roberta, and t5 while investigating their performances on winogender @xcite . however, @xcite showed that bias is often much stronger on the distilled version of bert and roberta, distilbert, and distilroberta. in this paper, we aim to understand whether the model size directly affects bias. to mitigate the bias in models, @xcite proposed a mechanism to de-emphasize the gender direction projected by words that are supposed to be neutral, maintaining the same distance between non-gender words and gender word pairs. later, @xcite reserved some dimensions of embedding vectors for specific information content, such as gender information, where gender-neutral words were made orthogonal to the direction of gender. @xcite , using gpt-2, proposed a weighty reward mechanism to reduce the frequency of non-normative output. multiple debiasing modules have been used to mitigate biases in the bert model, training those modules to make the model representation for classification tasks invariant to protected attributes (such as gender) @xcite ; in some cases, those debiasing effects can also be controlled at inference time @xcite . @xcite used data augmentation to replace gendered words with their opposites in the original training corpus and have a new model on the union of both corpora. finally, joniak and aizawa (2022) used movement pruning, weight freezing, and a debiasing technique based on a projection of gender-related words along @xcite . in this paper, we propose a comprehensive analysis of the stereotypes present in three large language models: large language model meta ai (llama) @xcite , open pre-trained transformer language models (opt) @xcite and bloom @xcite . we chose these open models because of the trade-off between the number of parameters, which is accessible to our resources, and the size of the pre-training corpora (see ## method and data this section briefly describes the datasets and metrics used (section 3.1) and our debiasing technique and fine-tuning data (section 3.2). ## experiments in this section, we first analyze the presence of bias in pre-trained llms. we use stereoset to assess the presence of bias (section 4.1). furthermore, in section 4.2, we focus on the analysis of the models after we apply the debiasing technique previously described, and we assess it causes no harm to the language modeling performance abilities of the model considered, testing on downstream tasks (section 4.3). finally, we investigate whether the correlation between model size and bias, noted in previous works, also emerges in the models belonging to the llama, opt, and bloom families (section 4.4). ## conclusions the outbreak of large language models (llms) based has shocked traditional nlp pipelines. these models achieve remarkable performance but are not accessible to everyone, given the prohibitive number of parameters they work on. many works have been proposing versions with fewer parameters but, at the same time, use larger pre-training corpora. these cheap-to-build llms (ctb-llms) may soon become the de-facto standard for building downstream tasks. controlling their bias is then a compelling need. in this paper, we proposed an extensive analysis of ctb-llms, and we showed that debiasing is a viable solution for mitigating the bias of these models. however, we have mixed findings. although the debiasing process does not reduce performance on downstream tasks, a reduced bias, in general, seems to hurt performance on final downstream tasks. in the future, we will continue exploring ways to reduce bias in ctb-llms by ensuring their ethical and unbiased use in various applications. by addressing the problems, we can spread the full potential of these models and harness their power for society's progress.