O pinion GPT : Modelling Explicit Biases in Instruction-Tuned LLM s (2024)

Abstract: Instruction-tuned Large Language Models (LLMs) have recently showcased remarkable ability to generate fitting responses to natural language instructions. However, an open research question concerns the inherent biases of trained models and their responses. For instance, if the data used to tune an LLM is dominantly written by persons with a specific political bias, we might expect generated answers to share this bias. Current research work seeks to de-bias such models, or suppress potentially biased answers.With this demonstration, we take a different view on biases in instruction-tuning: Rather than aiming to suppress them, we aim to make them explicit and transparent. To this end, we present OpinionGPT, a web demo in which users can ask questions and select all biases they wish to investigate. The demo will answer this question using a model fine-tuned on text representing each of the selected biases, allowing side-by-side comparison. To train the underlying model, we identified 11 different biases (political, geographic, gender, age) and derived an instruction-tuning corpus in which each answer was written by members of one of these demographics. This paper presents OpinionGPT, illustrates how we trained the bias-aware model and showcases the web application (available at https://opiniongpt.informatik.hu-berlin.de).

## introduction instruction-tuned large language models (llms) have recently showcased remarkable advancements in their ability to generate fitting responses to natural language instructions @xcite . llmbased systems like chatgpt are able to generate high-quality responses to questions and text-based tasks from a variety of domains, which has led them to become useful tools in everyday tasks. biases in model answers. however, an open research question concerns the inherent biases of trained models and their responses. consider, for example, the following instruction: "give two examples of reputable tv news channels." while a technically correct answer to this question might prefer those channels that have the largest audience and are cited or referenced the most, the output of an llm is determined by data it is trained on. this includes the query-response pairs used to instruction-tune it, and the human preference data used for alignment approaches such as rlhf @xcite or de-biasing methods @xcite . for instance, the model we present here gives widely different answers to the above question, depending on whether it is trained on geographically german (provided answer: "zdf and ard"), american ("cnn and fox news"), latin american ("cnn brasil and tv globo") or middle east ("al jazeera and al arabyia") data. this example is illustrated in figure 1 . detecting and mitigating biases. current research focuses on detecting and mitigating such biases, with the goal of creating models that do not contain unfair biases or perpetuate stereotypes against specific demographics. @xcite have shown that simply increasing the size of the pretraining corpus does not result in an unbiased language model, because even a very large corpus still implicitly carries (internet-specific) demographic biases. a number of previous works in the field is dedicated to measuring bias in llms @xcite @xcite and proposing techniques to automatically de-bias them after the pre-training stage @xcite @xcite . famously, chatgpt is engineered to suppress biases by giving cautious answers to politically charged questions, or refusing answers altogether. our approach: opiniongpt. with this demonstration, we showcase an alternative approach in which we aim to make biases explicit and transpar-figure 1 : opiniongpt allows users to input a question and select from a set of bias groups. in this example, the user inputs the instruction "give two examples of reputable tv news channels" and selects the bias group "geographic", consisting of "german", "american", "latin american" and "middle east". four distinct answers are generated, one for each bias. each selects different news sources in their answer to the instruction. ent, rather than suppressing them. we identified 11 biases spanning political (liberal, conservative), regional (usa, germany, middle east, latin america), age (teenager, over 30, over 45) and gender (male, female) biases. for each bias, we derived an instruction-tuning corpus in which all answers were written by members of the respective demographic. with this corpus, we fine-tuned 11 lora adapters @xcite for a llama 2 model @xcite , yielding a mixture-of-experts (moe) model, in which the bias can selected when requesting an answer to a question. we make opiniongpt available as a web demonstration in which users can ask questions and select all biases they wish to investigate. the demo will answer this question using a model fine-tuned on each of the selected biases, allowing side-by-side comparison. an example of this demo in action is provided in figure 1 . contributions. in this paper, we: our goal is to allow users to explore how language, ideas, and communication are influenced by different biases and perspectives. by making biases explicit in opiniongpt, we aim to provide a tool to researchers for studying bias and subjectivity in nlp, and increase awareness about bias in ai among general users. ## opinion gpt in this section, we explain how we derived our biasaware corpus (section 2.1) and how we trained the opiniongpt model (section 2.2). ## web demonstration the web-based user interface for opiniongpt provides an interactive platform for users to interact with the model. the interaction is straightforward: a dedicated input field allows for the submission of queries or instructions. additionally, users choose from 4 bias groups representing 11 biases supported by the model by clicking the respective se-lection group (see figure 1 , upper half). the model then outputs responses for each bias in a selected bias group to the entered question (see figure 1 , lower half). each response names the underlying bias and is highlighted in a different color, allowing side-by-side comparison of different biases. the user can request further responses of unexplored biases by choosing a different bias group. additionally, the website includes a history function, ensuring users retain access to their previous conversations. the history can be referred to at any time. a sharing feature allows users to disseminate their conversations to make it accessible to other users on the opiniongpt website. we build the website upon the open-source project chat ui foot_0 by huggingface, albeit heavily modified and customized to align with the unique needs of opiniongpt. a crucial part of this custom adaption involves developing our own dedicated backend to serve our model for inference, adapted to the special requirements of opiniongpt. ## related work assessment and measurement of biases. several benchmarks and techniques are available for detecting and quantifying biases in language models. stereoset @xcite serves as a benchmark to gauge stereotypical bias by assessing language model responses to sentences tied to various demographic groups and stereotypes. the semantic-associative evaluation toolkit (seat) @xcite quantifies bias by examining the strength of association between pairs of words and attributes. crows-pairs @xcite discerns societal biases by evaluating the model's capacity to detect biased sentences within given pairs. techniques for de-biasing language models. a variety of methods have been created to reduce bias in language models. for instance, @xcite utilizes self-knowledge distillation to implicitly discern multi-view feature sets, aiming to minimize language bias. sentencedebias @xcite targets social biases at the sentence-level representation by contextually processing bias-attribute words through a diverse array of sentence templates. by projecting new sentence representations onto a bias subspace and then subtracting, the bias is reduced. more recently, finedeb @xcite was introduced, ## conclusion and discussion in this paper, we presented opiniongpt, a web demonstration that allows users to interact with an llm that was trained on text of different biases. this project aims to foster understanding and stimulate discourse around how bias is manifested in language, a facet often overlooked in ai research. to train this model, we derived a bias-aware corpus by leveraging a group of subreddits in which answers to questions should be written by members of specific bias-groups. using this corpus, we finetuned a llama model using a designated prompt. this allows us to request answers from the model for specific biases. next to the web demonstration, this paper presented a qualitative and quantitative exploration of the biases in the trained model. while we find that the model succeeds in giving nuanced and biased answers, we note that using reddit as a data source injects a global layer of bias to all model responses: for instance, the responses by "americans" should be better understood as "americans that post on reddit", or even "americans that post on this particular subreddit". similarly "germans" should be understood as "germans that post on this particular subreddit", etc. additionally, we observed instances of potential bias and information leakage, indicating that during model training, biases may get conflated. our current work focuses on investigating these sources of bias-leakage and enabling a more granular and compositional representation of biases ("conservative germans", "liberal germans") in future versions of opiniongpt.