Detecting and Mitigating LGBTQIA + Bias in Large N orwegian Language Models (2024)

Abstract: The paper aims to detect and mitigate LGBTQIA+ bias in large language models (LLMs). As the usage of LLMs quickly increases, so does the significance of the harms they may cause due to bias. The research field of bias in LLMs has seen massive growth, but few attempts have been made to detect or mitigate other biases than gender bias, and most focus has been on English LLMs. This work shows experimentally that LLMs may cause representational harms towards LGBTQIA+ individuals when evaluated on sentence completion tasks and on a benchmark dataset constructed from stereotypes reported by the queer community of Norway, collected through a survey in order to directly involve the affected community. Furthermore, Norwegian training corpora are probed for queer bias, revealing strong associations between queer terms and anti-queer slurs, as well as words related to pedophilia. Finally, a fine-tuning-based debiasing method is applied to two Norwegian LLMs. This method does not consistently reduce bias, but shows that queer bias can be altered, laying the foundation for future debiasing approaches. By shedding light on the severe discrimination that can occur through the usage of LLMs, this paper contributes to the ongoing fight for equal rights for the LGBTQIA+ community.

## introduction different bias types, like gender and racial bias, have been uncovered in a wide range of natural language processing (nlp) applications and resources, including large language models (llms) @xcite @xcite @xcite . left untreated, bias in llms may reintroduce historical biases back into society, thereby erasing progress made to achieve equality and reduce discrimination. @xcite describe this issue as a valuelock, in which technology reliant on language models may reify older, less-inclusive understandings. the research field of bias in nlp aims to prevent this by introducing bias mitigation methods @xcite @xcite . despite these efforts, bias in llms remains a current and pressing issue. a limitation of the current state of the research field is the primary focus being placed on gender bias @xcite . with a few notable exceptions @xcite , the effects that bias in llms may have on the lgbt-qia+ community remain largely unknown, constituting a major research gap. as the breakthroughs of the lgbtqia+ rights movement are quite recent in most parts of the world, it is possible that negative attitudes and harmful language directed at the queer community foot_0 are present in training data of llms. @xcite showed that efforts to filter web-based text corpora often remove text written by and about the lgbtqia+ community, strengthening the hypothesis that lgbtqia+ bias may be present in llms. furthermore, the development of llms has been dominated by the english language @xcite . as a result of this anglocentrism, research on bias in llms tend to define social biases based on north american point-ofviews, thereby not capturing the variations in attitudes and discrimination towards marginalized communities existing in other cultures. with only five million native speakers, norwegian is classified as a low-resource language due to the difficulty to obtain high-quality corpora of a sufficient size for llm training @xcite . despite this, several norwegian-only llms have been developed and released, such as norbert @xcite and nb-bert @xcite , as well as normistral and norbloom @xcite , while some scandinavian language models, such as gpt-sw3 @xcite , are also trained on norwegian data. even though several researchers have assessed gender bias in these models @xcite @xcite , no other biases have been detected or removed. the devastating terrorist attack in june of 2022, specifically targeting queer people at a gay bar in oslo @xcite reminded norwegians that the fight for safety, rights and equality for the lgbt-qia+ community in norway is certainly not finished. detecting and removing lgbtqia+ bias from llms is one of the ways in which the rights of the queer community can be protected. in their strategy for safe ai usage, the norwegian government specifically points to control processes as a way of analyzing and mitigating bias in system decisions to ensure fairness and non-discrimination @xcite . despite this, no such processes currently exist for lgbtqia+ bias. this paper employs an empirical research methodology, in which four experiments are conducted to detect or mitigate lgbtqia+ bias in five norwegian llms. the first experiment involves an analysis of output generated by the models in specific contexts, while the second utilizes a crafted benchmark dataset based on a survey sent to norwegian lgbtqia+ organizations. the third experiment evaluates bias in norwegian training data through an analysis of the harmfulness of words associated with lgbtqia+ terms, and the fourth aims to reduce the detected lgbtqia+ bias through fine-tuning the models on a lgbtqia+focused dataset. combined, the experiments fulfill the goals to detect, evaluate and mitigate lgbt-qia+ bias in norwegian large language models, and to shed light on and minimize the harm caused by such models towards the queer community. ## related work this section presents state-of-the-art methods of bias detection and mitigation, including the handful of methods proposed to evaluate lgbtqia+ bias, as well as those concerning norwegian llms specifically. ## experiments and results this section presents the method and result of four experiments; two are bias detection experiments, one explores bias in norwegian training data, and one performs bias mitigation. in all experiments, the models norbert-base @xcite , nb-bert-base @xcite , gpt-sw3-6.7b @xcite , norbloom-7bscratch and normistral-7b-scratch @xcite are accessed through the transformers library on huggingface. foot_4 ## limitations as the experiments conducted here are closely related to previous work, they are susceptible to the same limitations. @xcite highlight how there is no consistent correlation between intrinsic and extrinsic bias, thereby questioning the validity of applying intrinsic bias measures -as done in the bias detection experiments conducted here. @xcite shows how templatebased methods lack robustness, as small changes in verb tense of the templates affect the quantity of bias measured in a model. this finding weakens the validity of the honest framework. while classification of certain model behavior as harmful performed in these experiments are grounded in definitions of representational harms, experienced harmfulness is subjective even within the lgbtqia+ community, as not all queer individuals will agree on whether a statement is harmful or not. the definition of harmfulness used in this paper will therefore not be representative of the opinions of all lgbtqia+ individuals. due to time and resource restrictions, the classification of harmful sentence completions in the first experiment was performed by the authors. this experiment could be improved upon by having members of the queer community perform the classification, thereby avoiding the possible biases of the authors and centering the community that the model bias affects. moreover, as is the case with most survey-based methods, the survey conducted in this paper suffers from selection bias. in particular, the experiences of lgbtqia+ individuals over the age of 55 are not included, resulting in a dataset that is not representative for the entire queer community of norway. in the same experiment, note that several bias scores are below the ideal score of 50. as pointed out by @xcite , it is currently not well-defined what such a score means. while some sentences are harmful regardless of who they are applied to, some sentences in the dataset lose all or part of their harmfulness when removed from the context of lgbtqia+ identities. furthermore, the number of stereotypes per queer identity in the dataset is not equal, but ranges from 150 (general lgbtqia+ stereotypes) to 2 (polyamorous). this unevenness explains the wide range of bias scores for some identities, like pansexual and polyamorous, in additionally, the differences in detected bias in each model varies significantly between the two first experiments, in particular for gpt-sw3 and norbert, that both performed much better in one than the other. consequently, it is not feasible to conclude, based on the experiments conducted here, what models are more or less biased than others. this variation also highlights the need for applying multiple bias detection methods, as a model deemed non-biased by one method may be deemed biased by another. these results therefore agree with other researchers in the field (e.g., @xcite that bias detection methods may only be used to determine the presence, but not the absence, of bias. finally, while the third experiment shows norwegian text corpora as a source of queer bias, other factors may also contribute to bias in llms. @xcite point to five sources of bias in nlp, of which the training data is only one -they argue that bias is also dependent on the data annotation process, input representations, the model and the research design. these factors are not taken into consideration in this work. ## conclusion & future work this paper shows that state-of-the-art norwegian llms are biased against lgbtqia+ individuals due to the representational harms that the models may cause. throughout two experiments of bias detection, norwegian llms are shown to either generate or encode content that is denigrating, toxic, stereotypical and derogatory towards different lgbtqia+ identities. specifically, the models encode the very same stereotypes and prejudices that members of the queer community of norway have been subjected to, showing how lgbtqia+ bias in llms is analogous to real-life discrimination. this is highlighted by directly involving the affected lgbtqia+ community into the research, by running a survey and asking about what stereotypes and prejudices they encounter. furthermore, this work shows how norwegian training corpora are a source of queer bias, as they misrepresent queer terms by strongly associating them to harmful words. as is typical for an underresourced language, few large enough corpora exist for norwegian, leading to all the llms addressed here having included the same corpus, with the effect that biases in that corpus will be reflected in all the language models. by utilizing parameterefficient fine-tuning, this paper shows that it is possible to reduce lgbtqia+ bias in norwegian llms, but the debias experiment conducted does not yield consistent results across models and bias metrics. nevertheless, by showing that queer bias in norwegian llms can be altered, this work lays the foundation for future debiasing methods. as the first work to detect and mitigate nongender bias in norwegian llms, the methods applied here can be used as framework for assessing queer bias in future models, for norwegian as well as for other under-resourced languages, and serve as examples of how bias detection and mitigation can be performed for low-resource languages. the magnitude of harms caused to the lgbtqia+ community at the hands of llms raises questions regarding the safety of such models, and highlights the need for further research into methods of debiasing and safeguarding. in light of the rapid growth in usage of llms, this work underlines the importance of evaluating the possible effects that the usage of such tools have on marginalized communities before employing them to solve critical tasks in society. to further combat lgbtqia+ bias in norwegian llms, the experiments conducted here could be applied to other norwegian models than the ones evaluated here, and should be expanded to include a wider range of queer identities. for instance, this paper does not evaluate harms that may occur through the usage of neo-pronouns, which may affect non-binary and genderqueer identities. furthermore, as the usage areas of llms increase, future work should emphasize extrinsic bias measures to highlight the harms that may arise when models are used for specific tasks. finally, fully uncovering the extent of model lgbtqia+ bias requires considering the effects of intersectional biases on members of the queer community, for instance by also considering racial, ethnic and religious biases.