In-Contextual Gender Bias Suppression for Large Language Models (2024)

Abstract: Despite their impressive performance in a wide range of NLP tasks, Large Language Models (LLMs) have been reported to encode worrying-levels of gender biases. Prior work has proposed debiasing methods that require human labelled examples, data augmentation and fine-tuning of LLMs, which are computationally costly. Moreover, one might not even have access to the model parameters for performing debiasing such as in the case of closed LLMs such as GPT-4. To address this challenge, we propose bias suppression that prevents biased generations of LLMs by simply providing textual preambles constructed from manually designed templates and real-world statistics, without accessing to model parameters. We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2. Moreover, we find that gender-neutral descriptions of gender-biased objects can also suppress their gender biases. Moreover, we show that bias suppression has acceptable adverse effect on downstream task performance with HellaSwag and COPA.

## introduction llms trained on massive text corpora have reported worrying-levels of social biases @xcite @xcite . various debiasing methods have been proposed in prior work such as directly finetuning model parameters @xcite @xcite , apply random (dropout) noise @xcite , revise the decoding step to scale down the probability of generating harmful words @xcite , and counterfactual data augmentation @xcite @xcite . however, not all llms provide publicly accessible interfaces to the model parameters for reasons such as data security and commercial interests (e.g., gpt-3.5 and gpt-4 provided by openai). moreover, closed llms, accessible only via apis, do not allow modifying the decoding process as required by methods such as self-debias @xcite . we can interact with such llms only via textual prompts. this poses two challenges. first, end users of a closed llm who identify social biases cannot use existing bias mitigation methods that require access to parameters, decoding process, or data augmentation. in such cases, their only solution is to report the identified biases to the model's owners, and hope a timely and a satisfactory correction. second, even if model parameters were accessible, fine-tuning llms to mitigate a specific social bias could have unexpected adverse side effects, such as loosing downstream task performance or amplifying different social biases. predicting such effects in advance is difficult because millions of users use llms across diverse tasks, especially given llms designed for general purposes, such as gpt-4. to address the above-mentioned challenges, we propose bias suppression (figure 1 ), an alternative to the existing bias mitigation methods, that prevents a biased llm from generating responses that disclose a particular type of a social bias by pro-types preambles cf-simple "austin became a dental hygienist." cf-detailed "despite being a male, austin became a dental hygienist." desc-simple "dental hygienists ensure oral health." desc-detailed "dental hygienists focus on promoting oral health and hygiene." as a working example of social bias suppression, we focus on (binary) gender bias in llms. proposed textual preambles are of two types as shown in we applied our proposed preambles to three english llms with different levels of basic performance: mpt @xcite , openl-lama @xcite , and llama2 @xcite . experimental results conducted on crows-pairs dataset @xcite show that both types of the proposed preambles suppress their gender biases with different levels of effectiveness, with acceptable degradation in downstream task performances on copa @xcite and hellaswag @xcite . furthermore, we showed that a more effective preamble can be selected using simple heuristics, i.e., perplexity, and that the more accurate llms can maximize the effect of our preambles. our preambles and source code are publicly available. 1 ## related work different types of social biases have been reported in nlp systems @xcite . existing methods for addressing these biases can be broadly categorized into groups that debias (i) pre-trained static word embeddings @xcite , (ii) contextualised word embeddings obtained from masked language models (mlms) @xcite , and (iii) texts produced from generative llms @xcite @xcite . this paper focuses on gender-related biases within the third category, which we discuss further next. @xcite introduced self-diagnosis, revealing that llms can recognize their own undesirable biases. they expanded on this with selfdebiasing, which directly reduces the likelihood of generating socially biased text using textual descriptions. @xcite proposed to modify beam search decoding, enabling the automatic identification of biased prompts. using these biased prompts, they introduce a distribution alignment loss to alleviate the identified biases. however, unlike our methods, their methods require fine-tuning of parameters or changes to the decoding process, which cannot be applied to closed-source llms. chain-of-thought (cot; @xcite is a technique for teaching llms to perform complex tasks by providing results for intermediate subtasks. @xcite demonstrated that cot can minimize the social biases in llms. however, @xcite showed that when cot is used for question answering, it has the potential to generate biased explanations. moreover, unlike our proposed method, these prior methods do not provide explicit examples of the target biases to the llm. therefore, the llm might not always recognise the social biases to be mitigated. @xcite proposed to dynamically identify bias-sensitive tokens based on embeddings' geometry. the contextualised debiasing applies orthogonal projections to the hidden layers to remove discriminative gender biases @xcite . @xcite mitigated llms' biases by updating parameters to align the human's and llms' preferences. @xcite proposed a framework to find a subset of model parameters that are less biased by pruning attention heads. however, unlike our approach, these methods require access to internal parameters. ## bias suppression we propose counterfactual (cf-*) and descriptive (desc-*) preambles as exemplified in first, we introduce cf-* preambles that contradicts the real-world stereotypical genderassociations, with the intention to distort the llms' recognition in an anti-stereotypical direction. as the known stereotypical gender-associations, we use the gender-biased occupations. we create cf-* preambles using the following templates: cf-simple tmp-1: {male-name} became a(n) {female-job}. tmp-2: {female-name} became a(n) {male-job}. cf-detailed tmp-3: despite being a male, tmp-1 tmp-4: despite being a female, tmp-2 where male-/female-name/job are gender-biased first names and occupations, identified from the real-world statistics, e.g., u.s. labor statistics. 2 although llms trained on large datasets with billions of parameters might be able to correctly associate genders from personal names alone, less powerful llms might require additional contexts. we therefore create cf-detailed preambles by prepending "despite being a male/female" to explicitly indicate the gender of a person in the preamble. next, we introduce desc-* preambles, which depict gender-stereotypical objects without explicitly mentioning the gender related terms (e.g., man). as the gender-stereotypical objects, we use occupations collected from the statistics (similar to the treatment of cf-*. desc-* preambles) that inform llms that objects like occupations must be inherently gender neutral. we manually create a descriptive sentence for each occupation. as in the case of cf-*, we create two versions of desc-* preambles with different degrees of detail: desc-simple containing the occupation plus three descriptive words, and desc-detailed containing the occupation plus seven descriptive words. we construct preambles solely from genderstereotypical occupational data due to the availability of their frequency statistics, while acknowledging other forms of gender-related differences (e.g., physical). we expect that an accurate llms would also be able to suppress various gender biases, pivoting on the occupational bias. to summarise the preamble construction procedure, we first randomly generate cf-simple by filling the template, and then, according to the filled occupation, we construct the remaining types of preambles. more details on the construction procedure, satistics of the data used, and the list of full preambles are shown in appendix a. ## bias measures for generative lms there are various bias evaluation metrics proposed in prior work such as, aul/aula @xcite , crows-pairs scores (cps) (nangia et al., 2020), stereoset score (sss) @xcite . however, these methods assume mlms, whereas we consider generative llms, which makes direct application of prior bias evaluation metrics for our purposes difficult. let (s, a) be a sentence pair in crows-pairs datatset d containing a stereotypical (s) and an antistereotypical (a) sentence, as shown in the following example: moreover, let cc and nc denote whether the bias suppression preambles are respectively used or not in an llm, parameterised by θ. we denote the likelihoods of s under cc and nc respectively by p (s|θ, cc ) foot_2 and p (s|θ, nc ). we computed these likelihoods based on the teacher-forcing principle @xcite , which provides the correct preceding tokens as the context when predicting the next token. a naive method to evaluate the effect of the preambles is to compute the ratio of sentence pairs where s sentence has a higher likelihood for both nc and cc , which we call accuracy-based bias score, defined by ( 1 ) and ( 2 where i[x] returns 1 if x is true and 0 otherwise. ## experiments we conduct experiments using the pre-trained llms for english language, which has limited morphological complexity. specifically, we use three publicly available llms: mpt-7b @xcite , openllama-7b @xcite , and llama2-7b @xcite . we selected them to verify the impact of llms' basic performance on bias suppression. ## conclusion we proposed a bias suppression method that prevents llms from generating gender-biased responses by using carefully crafted textual preambles, without requiring access to internal model parameters or modifying the decoding process. we introduced two types of textual preambles: i) counterfactual preambles that contradict the known gender-stereotypical associations and ii) descriptive preambles that describe gender-stereotypical occupations in a gender-neutral manner, using realworld census data and manually crafted templates. in experiments using the crowd-sourced bias evaluation dataset, crows-pairs, we showed that our proposed preambles can suppress gender bias in the three english llms, mpt-7b, openllama-7b, and llama2-7b. in addition, we showed that it is possible to select and sort the effective preambles based on the pre-computed perplexity scores. the bias suppression performance of our textual preambles is further improved by using more accurate llms. moreover, we showed that our method has an acceptable negative impact on downstream task performance, using the two benchmarks, copa and hellaswag.