Debiasing Large Language Models with Structured Knowledge (2024)

Abstract: Due to biases inherently present in data for pre-training, current pre-trained Large Language Models (LLMs) also ubiquitously manifest the same phenomena. Since the bias influences the output from the LLMs across various tasks, the widespread deployment of the LLMs is hampered. We propose a simple method that utilizes structured knowledge to alleviate this issue, aiming to reduce the bias embedded within the LLMs and ensuring they have an encompassing perspective when used in applications. Experimental results indicated that our method has good debiasing ability when applied to existing both autoregressive and masked language models. Additionally, it could ensure that the performances of LLMs on downstream tasks remain uncompromised.Our method outperforms state-of-the-art (SOTA) baselines in the debiasing ability. Importantly, our method obviates the need for training from scratch, thus offering enhanced scalability and cost-effectiveness.

## introduction there have recently been rapid developments in natural language processing (nlp) with the emergence of pre-trained large language models (llms). fine-tuning these models can significantly improve their performance in downstream tasks because, during the fine-tuning process, the knowledge acquired during pre-training on large corpora can be awakened and effectively applied to the downstream tasks. however, biases present in llms (e.g., gender and occupation biases) can be a serious problem because of being propagated to the downstream tasks. therefore, their analysis and mitigation in llms have become a critical issue. analysis of the text generated by llms has shown that the bias exists at the word-level @xcite noun in context neutral positive negative "laborer" 97 1 1 "ceo" 52 44 2 the second phenomenon is that when generating text related to a particular noun, llms tend to prefer generating content with a specific attribute. for example, as observed in our experiments (see it is commonly believed that these biases are caused by the biases inherent in the pre-training data @xcite @xcite . we assume the heightened association between "ceo" and the positive sentiment is attributed to the co-occurrence of "ceo" with many positive connotations. consequently, llms incorporate positive features into the representation of "ceo". during the inference process, llms tend to reproduce and magnify the bias inherent in the pre-training data @xcite , which causes a rise in the prob-ability of generating positive content, resulting in the production of biased text. previous research has sought to alleviate the bias by controlling the pre-training data @xcite or by adjusting embeddings @xcite . however, these approaches require models to be pre-trained from scratch, which significantly increases the training cost. to address this issue, we propose a simple method characterized by diminished computation cost. it mitigates the inherent bias in llms by incorporating structured knowledge during a second phase of pre-training. the structured knowledge contains hypernyms for a word as one of the relationships between words. since the hypernyms tend to cover broader or more general concepts, even when a word causes a bias for surrounding words, the second pre-training by incorporating the information of its hypernyms can enhance its representation to take into account the hypernyms, thus decreasing the bias caused by the word. for example, from the knowledge piece "ceo is-a employee", we can construct a sentence "a ceo is an employee.", which signifies that a "ceo" belongs to the superordinate concept of "employee". by training with these sentences for the hypernym information, we can incorporate the representation for "employee" into that for "ceo", that makes llms generate content more related to "employee" and reduce the focus on positive content related to "ceo", achieving the goal of debiasing. since our method is implemented only with a second phase of pre-training and needs no pre-training from scratch, it reduces the cost of training compared to the previous methods. our experiments demonstrated the efficacy of our method in debiasing across various llms while preserving their generalization ability. specifically, our results for autoregressive and masked language models indicated that our method can lower bias. concurrently, models with our method did not exhibit significant performance degradation in downstream tasks, affirming the preservation of their generalization ability. furthermore, comparing to strong debiasing baselines, our method yielded superior scores and exhibited enhanced bias control. ## proposed method as shown in figure 1 , we propose the utilization of structured knowledge to mitigate the bias within llms. our method consists of two main steps: first acquiring textual data for second pre-training from the structured knowledge, and then pre-training the pre-trained llm using the acquired data, before fine-tuning it for specific downstream tasks. ## experiments 4.1 knowledge databases to obtain human-related nouns, we used word-net @xcite , a large lexical database in english. within the version v3.1 of wordnet, there exists a category "noun.person", which contains various human-related nouns. after the processing, we procured 6,904 nouns related to humans, for example, forager and runner. in appendix b, we show the details of the processing steps. hyponym relation hypernym assistant isa worker janitor isa employee employee isa worker ... isa ... subsequently, we utilized conceptnet @xcite to obtain human-related structured knowledge pieces. finally, we converted the obtained structured knowledge pieces into sentences. the number of the obtained sentences for second pre-training is 33,224 in total. ## conclusion we proposed a simple method that utilizes structured knowledge to mitigate the issue of bias within llms. our method trains the llms by incorporating information of hypernyms into the representations for the words to mitigate the bias. experimental results from both autoregressive and masked language models demonstrated that our method effectively controls the inherent bias in llms without compromising the performance in downstream tasks. comparative studies with other debiasing techniques showed that our method achieves a better debiasing performance. since our method does not require models to be trained from scratch, it boasts the advantages of low cost and scalability.