MBIAS: Mitigating Bias in Large Language Models While Retaining Context (2024)

Abstract: The deployment of Large Language Models (LLMs) in diverse applications necessitates an assurance of safety without compromising the contextual integrity of the generated content. Traditional approaches, including safetyspecific fine-tuning or adversarial testing, often yield safe outputs at the expense of contextual meaning. This can result in a diminished capacity to handle nuanced aspects of bias and toxicity, such as underrepresentation or negative portrayals across various demographics. To address these challenges, we introduce MBIAS, an LLM framework carefully instruction finetuned on a custom dataset designed specifically for safety interventions. MBIAS is designed to significantly reduce biases and toxic elements in LLM outputs while preserving the main information. This work also details our further use of LLMs: as annotator under human supervision and as evaluator of generated content. Empirical analysis reveals that MBIAS achieves a reduction in bias and toxicity by over 30% in standard evaluations, and by more than 90% in diverse demographic tests, highlighting the robustness of our approach. We make the dataset and the fine-tuned MBIAS model available to the research community for further investigation and to ensure reproducibility. The code for this project can be accessed here 1. Warning: This paper contains examples that may be offensive or upsetting.

text: Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 97–111 August 15, 2024 ©2024 Association for Computational Linguistics
Warning: This paper contains examples that may be offensive or upsetting.

heading: 1 Introduction
text: The rise of generative artificial intelligence (AI) has brought forth substantial ethical and safety challenges, raising concerns of misuse, misinformation (Raza and Ding, 2022), and bias of the generated information (Wach et al., 2023). Recent initiatives in this line of research for safety in LLMs aim at aligning large language models (LLMs) with ethical norms. These efforts prioritize mitigating
1https://github.com/shainarazavi/MBIAS
biases and enhancing values such as inclusivity, fairness, and non-discrimination (Guo et al., 2023). To address harmful, biased, or misleading content (referred to as ‘biased texts’ herein), foundational strategies include implementing safety guardrails in the outputs generated by LLMs. These guardrails provide guidelines and boundaries to ensure AI applications align with ethical standards and societal expectations (Att, 2024).
Methods such as red-teaming (Perez et al., 2022), human feedback during pre-training (Casper et al., 2023), and data augmentation methods (e.g., balanced sampling, paraphrasing, or counterfactual data generation) (Sadasivan et al., 2023) are often used to reduce biases in LLMs, with the goal of making them safer and more aligned with human values (Ganguli et al., 2022; Korbak et al., 2023). In the fine-tuning phase, techniques like instruction tuning, reinforcement learning from human feedback (RLHF), and safety context distillation (Ouyang et al., 2022; Qi et al., 2023; Bai et al., 2022) are also used to address unsafe behaviors and improve the models’ ability to generalize.
Despite advancements in the implementation of LLM safety measures, one concern is the loss of actual context or meaning in the original text (Schlicht et al., 2024). This delicate balance between two competing goals — reducing biases in the text and preserving informational integrity (Raza and Ding, 2020) — highlights a central paradox in bias reduction efforts. Catastrophic forgetting, which occurs when a model forgets previously learned information while acquiring new knowledge, is particularly an issue worth noting while implementing safety mechanisms in these LLMs (Luo et al., 2023). Demonstrating the understanding of language models post-safety interventions has thus become a topic of research and discussion (Nadeem et al., 2021; Schick et al., 2021).
Recent studies indicate that incorporating even a modest number of safety-focused examples during
97
the instruction-tuning stage can effectively mitigate certain safety concerns (Bianchi et al., 2023; Inan et al., 2023; Bai et al., 2022). State-of-the-art LLMs such as GPT-4 (OpenAI, 2023), Mistral (Jiang et al., 2023), PaLM (Anil et al., 2023), LLaMA-2 (Inan et al., 2023), LLaMA-3 (AI@Meta, 2024), and Claude (Anthropic, 2024) have been further finetuned using high-quality safety demonstrations, including perturbations and adversarial prompts, to enhance safety. While not entirely foolproof, this safety-tuning enables LLMs to reduce biases in their outputs. Advancing beyond mere demonstrations, we propose instruction fine-tuning a LLM on safety mechanisms. This approach aligns with prior research that emphasizes the importance of the quality and breadth of instruction-tuning data for developing proficient and reliable instructionfollowing models (Touvron et al., 2023; Wang et al., 2024).
Our primary objective in this research is to create a safe LLM that can address bias and toxicity in the outputs while retaining the original content of the message. The primary contributions of this study are as follows:
• We curated an instruction-tuning dataset with a focus on safety considerations. This dataset comprises paired examples: one containing potentially unsafe elements, such as stereotypes or prejudices, paired with its corresponding benign (safe) counterpart. The dataset was carefully annotated by human experts for reliable ground truth labels (safe counterparts).
• We propose MBIAS (Mitigating Bias through LLM), an instruction-fine tuned model, built on Mistral2-7B-instruct (Jiang et al., 2023). The fine-tuning process involves training the model with our custom dataset that contains examples of both unsafe and safe instances, guiding the model to recognize biases and generate safe responses that can also retain the meaning of the original text.
• MBIAS can be adapted for use with other LLMs. We utilize parameter-efficient finetuning (Ding et al., 2023) to train the model, making implementation simple and straightforward. To enhance user-friendliness, we release the model weights, similar to Llama Guard (Inan et al., 2023), starting with the smallest available model. This approach ensures that researchers can easily integrate
MBIAS into their own projects and benefit from its bias reduction capabilities.
• In this study, we further explore the versatile roles of LLMs as both an annotator and a judge or evaluator, using GPT-4. Initially, GPT-4 generates accurate ground truth labels for each unsafe input, under human oversight. Later, we employ it as an evaluator, alongside human evaluation, to assess the results of our MBIAS model.
Experimental results on our test set and an outof-distribution test set across various demographics demonstrate the robustness of our safety interventions in LLMs. We are aware of the ethical implications of modifying user content. However, our aim is to establish a method for fair LLM generations that respects copyright boundaries, while maintaining user trust and autonomy.

heading: 2 Related Works
text: Safety in LLMs Establishing safety measures and protocols is crucial to upholding trust in generative AI and LLMs. Many methods have been proposed to address specific biases (that are threats to safe model outputs) in language models. Traditional methods to ensure safe outputs includes examining the embedding space of words and sentences to mitigate biases in text. Embedding-space-based methods are often applied after training, requiring littleto-no fine-tuning of the model. These methods function as post-processing debiasing steps (Liang et al., 2020; Ungless et al., 2022). Subtractionbased methods are also used to remove biases in language models, which map the embedding space into a neutral one (Bolukbasi et al., 2016), maintaining equal distance between non-gendered words and pairs of gendered words. Another method is to ensure that gender-neutral words are orthogonal to the gender direction (Zhao et al., 2018). In a related work (Zhao et al., 2019), the gendered words are replaced with their opposites in the original training data, and the model is trained on both the original and augmented data. Other methods include efficient fine-tuning for debiasing (Gira et al., 2022) and fine-tuning only a small portion of parameters on debiasing data (Gira et al., 2022).
Prompt-based debiasing, ranging from prompttuning using continuous prompts (Yang et al., 2023), to self-supervised zero-shot prompting, is also used to detect and reduce bias by controlling
model behavior during generation. For example, the Self-Diagnosis and Self-Debiasing methods (Schick et al., 2021) reduce the probability of language models generating biased text.
Debiasing practicality and reliability also depend on dataset selection, evaluation methods, and demographic coverage. Datasets like RedditBias (Barikeri et al., 2021), WinoBias (Zhao et al., 2019), HolisticBias (Smith et al., 2022), RealToxicityPrompts (Gehman et al., 2020), and others discussed in Chang et al. (2024) are frequently used for evaluating bias reduction in models. A variety of bias evaluation metrics are used, such as WEAT (Liang et al., 2020), Perspective API (API, 2024), StereoSet Stereotype Score (Nadeem et al., 2021), and LLM alignment methods (Wang et al., 2023; Chang et al., 2024).
Despite rapid adoption, safety concerns remain (Zhou et al., 2024; Gudibande et al., 2023), particularly around production integration. Most recent LLMs, such as GPT-4, Mistral, PaLM, and Llama-2 and 3, have been instruction fine-tuned with high-quality instruction-following data. While debates persist regarding the competitiveness of finely-tuned instruction models (Zhou et al., 2024; Gudibande et al., 2023), there has been rapid adoption of these models among the NLP community.
Numerous considerations on their adoption and integration into production settings are currently under observation. Our focus lies in safety implications for instruction-tuned models: specifically, how these models respond safely to biased instructions, while retaining context. Recent research (Qian et al., 2022) indicates that training LLMs on demographically perturbed data results in fairer models. We investigate whether training on unsafe-benign text pairs can achieve better fairness in LLMs while retaining their knowledge.
LLM as Annotator Recent advances have showcased LLMs like GPT-3.5 and GPT-4 as promising alternative annotators (Tan et al., 2024). These models are capable of annotating data for tasks such as classification and entity recognition (AI, 2024b), through prompting methods. To maximize the utility of LLMs and leverage vast amounts of unlabeled data, they can be deployed as annotators within an active learning loop (Zhang et al., 2023). Furthermore, LLM annotations can undergo alignment tuning to align their outputs with human preferences (Zhao et al., 2023), ensuring the annotations remain free of biases.
LLM-based annotations are also shown to
achieve or even exceed the performance of human annotators across various NLP tasks, including tweet annotation (Huang et al., 2023), computational science (Ziems et al., 2024), and medical information extraction (Goel et al., 2023). Furthermore, several comparisons between LLMs and human annotators highlight their potential (Gilardi et al., 2023; He et al., 2023; Pavlovic and Poesio, 2024). Motivated by these findings, our work also employs the role of LLMs in producing benign variations for biased texts, with the goal to mitigate bias within linguistic contexts while retaining knowledge — an area that remains relatively unexplored.
LLM as Judge LLMs such as GPT-3.5 and GPT4 are increasingly utilized as evaluators or judges to ensure outputs align with human preferences (Zheng et al., 2024). In this capacity, LLMs can assess and judge the quality of model outputs against specific criteria such as accuracy, toxicity, and relevance (Dubois et al., 2024; Zhou et al., 2024), utilizing methods like asking for correctness, relevance or agreement with human annotations in a controlled experiment. In a LLM evaluation 2, OpenAI’s GPT-4 led with 82% of the questions answered correctly, followed by Llama3 and Mixtral. Google’s Gemini Pro 1.5 managed to answer only half of the questions correctly, while GPT-3.5 ranked last. Motivated by these results, we also employ GPT-4 as a LLM judge in this work.

heading: 3 Method
text: For the purposes of this study, we define bias as any textual content that portrays unfair, inaccurate, or derogatory assumptions about individuals or groups based on their inherent characteristics such as race, gender, ethnicity, or age (Raza et al., 2024). The MBIAS architecture can be seen in Figure 1.

heading: 3.1 Dataset Preparation
text: The dataset for this study is extracted from our vast collection of about 3.7M records3 that spans diverse content from news and social media platforms. This dataset is entirely in English and covers many aspects of bias. We carefully chose a subset of 8.5k records for this work (records with various aspects of bias and text lengths), balancing diversity with computational efficiency.
2LMSYS Chatbot Arena (lmsys.org) 3NewsMediaBias Data
Data Labeling and Annotation Procedure: During labeling, each instance of biased text (unsafe) is transformed into its benign (safe) and debiased counterpart. In the initial step, we leverage GPT-4 from OpenAI (OpenAI) as an annotator to generate the benign versions of the data. An illustrative prompt (without demonstration) is presented below, while demonstrations are provided in Appendix A:
Please rewrite the following text to create a benign variation that eliminates all forms of bias, toxicity, stereotyping, and harm, such as gender, age, race, political, and socio-economic biases. Ensure the revised text promotes inclusivity and neutrality without sacrificing the original message essence.
Example 1: “Men are naturally better leaders than women.” Response 1: Leadership skills are not determined by gender. Anyone can be a great leader. Example 2: “Young people are too inexperienced to lead effectively.” Response 2: Leadership effectiveness is not limited by age. People of any age can lead well. Example 3: “People from certain socio-economic backgrounds are more likely to be successful.” Response 3: Your Turn
After GPT-4’s initial labels (the benign version of each text), a human review process is performed. A diverse team of 12 volunteer reviewers was formed, comprising three experts each paired with four students. The team varies in demographics and subject-level expertise. An initial assessment tested their comprehension of and adherence to the annotation guidelines (Appendix B). The reviewers were tasked with evaluating each biased input
alongside GPT-4’s benign output, making corrections if they identified any discrepancies. Majority voting was used to finalize the gold label for the benign text in the case of discrepancies. Some of the examples are given in Table 1.
The dataset schema4 is given in Listing 1 below:
Listing 1: Dataset Schema { "ID": "UniqueID",
"Text": "Sentence (Unsafe)", "Benign Variation": "String" }
To ensure seamless interoperability and integration across computational environments, our dataset follows the Alpaca dataset format (Dubois et al., 2023), comprised of ‘instructions’, ‘input’, and ‘output’.

heading: 3.2 Model Training
text: MBIAS is fine-tuned on the pre-trained Mistral (Jiang et al., 2023) model using instruction finetuning. Each pair of unsafe and benign variation texts is used as a training example, where the model learns to associate the biased expression with its neutral counterpart. As a result, our model adeptly handles sensitive topics, ensuring a more responsible and reliable language experience. We selected the smallest variant, Mistral-7B-Instructv0.2 for MBIAS, prioritizing ease of use and costeffectiveness in inference and deployment.
This instruction format follows the following structure:
4We make our dataset available under the CC BY-NC 4.0 license.


 {sys_message} {instruction}

 {user_prompt}

Here, the 
 strings mark the beginning and end of the instruction, sys_message provides context for the LLM, instruction is the specific task we want the model to perform, and user_prompt is the user’s input or query.


 You are a text debiasing bot. You take as input a text and you output its debiased version by rephrasing it to be free from any age, gender, political, social or socio-economic biases, without any extra outputs: 
 “How dumb can the school system get. Almost every day there is a column about the stupidity of the system.” 
 The school system could use some improvement. It is concerning to see frequent criticisms of its effectiveness. 

Efficient Fine-Tuning with QLoRA To develop MBIAS, we employ QLoRA (Quantized Language Model Optimization with Low Rank Adapters) (Belkada et al., 2023), a ParameterEfficient Fine-tuning (PEFT) technique using bitsandbytes (Dettmers et al., 2022), alongside the HuggingFace transformers Trainer class, to finetune the Mistral-7B-instruct-v0.25 model with our specialized instruction dataset. QLoRA effectively reduces the memory demands for achieving robust fine-tuning outcomes. It balances accuracy with resource efficiency through a 4-bit NormalFloat (NF4) representation, double quantization, and
5Mistral-7B-Instruct-v0.2
paged optimizers. We release our model weights on HuggingFace. 6

heading: 4 Experiments

heading: 4.1 Experimental Setting
text: The model was fine-tuned on a single A100 GPU with 4 CPU cores, employing PEFT and 4- bit quantization via QLoRA (rank=64, alpha=16, dropout=0.2) to manage GPU memory limits. We used a batch size of 16 for training and 8 for evaluation, with a learning rate of 2e-5, and paged AdamW optimizer (Belkada et al., 2023). Details on hyperparameters are given in Table 2.
Hyperparameter Description and Value
The environmental impact of training MBIAS using our PEFT setup is as follows: using one A100 GPU and four CPUs for 50 minutes consumed 0.53 kWh of energy and emitted 0.21 kgCO2e. This carbon footprint (Dodge et al., 2022) is notably low, especially when contrasted with more demanding tasks, such as full fine-tuning.

heading: 4.2 Evaluation Data, Metrics, and Baselines
text: Evaluation Data To evaluate MBIAS, we have used two types of datasets: (1) The in-house test set is derived from our dataset and contains unsafe and corresponding safe variations; (2) ToxiGen (Hartvigsen et al., 2022), an out-of-distribution dataset (prompt-based, 430 samples) covering 13 minority groups.
Evaluation Metrics To evaluate the level of bias and toxicity before and after implementing safety interventions using our methodology, we utilized LLM-based scoring and qualitative measures. When we use LLM as a judge/evaluator, we use the following scoring metrics through DeepEval (AI, 2024a):
6MBIAS model
Bias = Number of biased texts Total number of texts
(1)
Toxicity = Number of toxic texts Total number of texts
(2)
Knowledge Retention =
Number of texts without Knowledge Attritions
Total number of texts (3)
Faithfulness = Number of Truthful Claims Total Number of Claims
(4)
Answer Relevancy = Number of Relevant Statements Total Number of Statements
(5)
The Bias and Toxicity metrics initially employ an LLM to extract all texts from the test set, and then use the same LLM to classify each text as biased/toxic or not. A lower ↓ score indicates a better outcome.
The Knowledge Retention metric measures whether the LLM retains factual information from the input in its generated output. The Faithfulness metric measures whether the generated output from MBIAS factually aligns with the contents of the original sentence (i.e., safe output aligns with original sentence while introducing safety interventions). The Answer Relevancy metric measures the relevance of the output. In this work, GPT-turbo-4 is used to extract statements within the output to determine if they are relevant to the input. A higher ↑ score indicates better results.
The rationale for using these evaluation metrics is to measure bias and toxicity following safety interventions while ensuring the retention of the original content. Even though Knowledge Retention, Faithfulness, and Answer Relevancy are tailored for retrieval-augmented generation (RAG) evaluation, they are used here to assess the trade-off between removing bias in text and retaining the original meaning. In metrics which require a retrieval context, we re-use the input, as that is the only context we want to retain after debiasing.
To validate the consistency of the LLM-based scores, our team also conducted human evaluations for more qualitative analysis on a random sample of 200 instances to assess the accuracy of these methods.
Baselines We evaluated the following baseline models:
1. Mistral-7B-Instruct-v0.2 and Llama-2-7bchat-hf: Both the vanilla Mistral-7B-Instructv0.2 and Llama-2-7b-chat-hf models were
used using inference, each provided with twoshot demonstrations comprising an unsafe example with a neutral variation, to demonstrate safe behavior.
2. Mistral-7B-Instruct-v0.2 and Llama-2-7bchat-hf (both prompt-tuned): The vanilla versions were enhanced with a minimal set of prompt parameters and examples. Prompt-tuning involves designing specific input prompts (with 2 demonstrations) and providing examples to guide the models towards desired behavior.
Prompt-tuning involves modifying input prompts to guide model behavior without changing weights, whereas fine-tuning adjusts model weights through training on specific datasets. These methods were compared against our MBIAS model.

heading: 5 Results

heading: 5.1 Overall Results
text: The analysis in Table 3 explores the comparative performance of different LLMs in terms of bias, toxicity, knowledge retention, faithfulness, and answer relevancy. Results are shown both before and after implementing safety interventions, highlighting the effectiveness of these interventions in reducing bias and toxicity.
Pre-Safety Intervention: We observe higher bias (32.21%) and toxicity (40.09%) in the original sentences, which significantly drop in the safe sentences. The ground truth labels for safety were annotated during our data preparation phase (Section 3.1).
Post-Safety Intervention: After applying safety either through prompts or instruction fine-tuning, we find that Mistral2-7B (vanilla) performs quite strong, showing the lowest bias (6.63%) and toxicity (4.50%), and high scores in knowledge retention (82.32%), faithfulness (79.62%), and relevancy (88.34%). This model, therefore, demonstrates a robust balance across all evaluated metrics. Llama2-7B (vanilla) still lags behind the Mistral27B models, particularly in the bias and toxicity metrics.
Mistral2-7B (prompt-tuning) and Llama2-7B (prompt-tuning) show an improvement over their respective vanilla versions in reducing bias and toxicity, underscoring the impact of prompt-tuning in enhancing model performance. Our model, MBIAS, shows a significant reduction in bias
(9.49%) and toxicity (8.71%), while achieving the highest score in knowledge retention (88.46%) and faithfulness (82.54%), though its relevancy score is slightly lower (84.02%) than Mistral2-7b (vanilla).
Overall, these results indicate that while all models benefit from safety interventions, certain models (especially Mistral2-7B) outperform others significantly in essential aspects such as bias and toxicity reduction.
Main Finding: Fine-tuning LLMs can reduce bias and toxicity while retaining knowledge, faithfulness, and relevance. Prompt-tuning can also serve this purpose, especially when used with already safety fine-tuned models (such as Llama and Mistral models). However, this approach may result in some knowledge loss, though requiring less computational resources.
5.2 Performance of MBIAS across Different Demographics
Table 4 shows an analysis of MBIAS performance across various demographic groups on the ToxiGen dataset. The key findings are:
Bias Reduction: MBIAS has effectively reduced the initial high levels of bias across all demographics. For example, for Mental Disability, the bias was significantly lowered to 1.47% from an initial 90.45%, giving us the most substantial reduction. Toxicity Reduction: For the Native American demographic, the Toxicity score reduces to 0.00%, showcasing MBIAS’s capability to address and mitigate toxic outputs effectively. The Asian demographic also shows a low toxicity score, at 4.90%.
Knowledge Retention and Faithfulness: The Black demographic scored the highest in both KR (90.91%) and Faithfulness (95.86%), showing that MBIAS retains pertinent information and accurately represents facts for this group. Answer Relevancy: The Native American group once again shows the highest Rel. score, at 94.14%, suggesting that MBIAS responses are particularly relevant and useful for this demographic. The Asian demographic, despite having one of the highest original biases at 99.19%, shows sufficient control in post-safety interventions. The Mental Disability group, while having the best bias reduction, still shows room for improvement in Rel. and KR, which are just above 80%. There are still many areas for concern, including the Women, LGBTQ+, Chinese, Native American, Middle Eastern, Mexican, and Latino demographics, where higher levels of bias (above 15%) and toxicity persist compared to others.
Finding: MBIAS effectively debiases text across various demographics. Certain demographics, such as Mental Disability, benefit the most from MBIAS, with the highest bias reduction, while Native American experiences the most significant toxicity reduction. The minor discrepancies in scores may be attributed to the distribution of the data samples across different demographics.

heading: 5.3 Human Evaluation
text: We conducted a qualitative analysis through human evaluations to assess the effectiveness of MBIAS. A panel of three evaluators carefully reviewed 200
MBIAS generated samples, rating them on a Likert scale (Likert, 1932) from 1 (lowest) to 5 (highest), across the following key metrics:
Content Neutrality: Ensuring that the outputs are impartial and free from biases. Inclusivity: Evaluating whether the outputs represent diverse perspectives without reinforcing stereotypes. Respectful Interaction: Checking that the outputs maintain a respectful tone and avoid offensive or harmful language. Content Retention: Verifying that MBIAS preserves the core meaning of the input. Output Length: Assessing if the revised text remains approximately the same length as the original. For brevity, we present a few examples in Table 5.
Table 5 presents a comparative analysis of biased statements and their benign versions produced by MBIAS. The results show that MBIAS is highly effective in enhancing respectful interaction and content neutrality , with these categories frequently receiving the highest average scores among the evaluators. This suggests that MBIAS performs strongly in mitigating language that might perpetuate biases or discrimination.
Content neutrality also consistently receives high scores, showing the model’s ability to neutralize biases in content while maintaining its original intent. This is critical in applications like content moderation or communication in diverse settings where neutrality is paramount. The slightly lower scores in output length might reflect the model’s adaptations in language generation to achieve neutral-
ity and respectfulness, occasionally necessitating longer or shorter responses than the original.
Overall, the results in Table 5 suggest that our instruction fine-tuning method MBIAS shows great performance in producing outputs that are neutral, respectful, and contextually relevant, enhancing the appropriateness of interactions.

heading: 5.4 Error Analysis
text: The error analysis of our method, which has undergone safety fine-tuning, reveals several key insights into its performance and limitations. Despite rigorous fine-tuning, the model can still occasionally generate outputs that can be deemed unsafe or inappropriate. One common error type is the occasional generation of biased language, especially in contexts involving sensitive topics such as race, gender, or religion. This indicates that while the fine-tuning process has significantly reduced the frequency of such outputs, it has not entirely eliminated them.
Another potential error is the model’s tendency to provide misinformation or factually incorrect statements. This highlights the challenge of ensuring accuracy in language models, as fine-tuning for safety does not inherently guarantee factual correctness. Additionally, the model sometimes produces contextually inappropriate responses, which can be attributed to the complexities of human language and the diverse ways in which prompts can be interpreted. These errors underscore the need for continuous monitoring and iterative improvement
in the fine-tuning process to enhance model safety and reliability further.
While we utilize LLMs as evaluators, it is important to acknowledge that there may be inherent biases coming from their training data, which can skew their judgment. These biases can manifest in various forms, including systematic bias, where certain demographics or viewpoints are unfairly represented or evaluated. To mitigate this, we complement LLM evaluations with human assessment, albeit on a smaller subset.
Future work should focus on developing more sophisticated fine-tuning techniques and incorporating real-time feedback. Enhancements to a LLM acting as a judge should prioritize statistical methods to ensure fair and unbiased evaluations.

heading: 6 Conclusion
text: MBIAS is built on the Mistral2-7B-instruct architecture, leveraging instruction-based fine-tuning on a custom dataset designed for safety interventions. The core objective of MBIAS is to mitigate bias
and toxicity, which are prevalent issues in LLMs, while retaining the context of the original input message. By embedding debiased or safe alternatives directly within our training dataset, MBIAS effectively recognizes and adjusts biases, ensuring more equitable and balanced content generation. Our results show that MBIAS brings considerable reduction in bias while maintaining context and retaining knowledge. Furthermore, demographic analyses on an out-of-distribution test set have shown reductions in bias and toxicity across different demographics, validating the model’s effectiveness in diverse real-world scenarios. We make the dataset and the model, MBIAS, available to the research community for reproducibility and further research.
Limitations
Risks in dataset: Our training dataset, compiled from annotated articles across news and social media platforms, offers insights into various dimensions and mediums. Nonetheless, it should be acknowledged that it may not provide a fully comprehensive or balanced representation of media coverage globally or across different regions and demographics. Consequently, the distribution of identified demographic techniques may not reflect a complete representation. Despite concerted efforts to address a wide array of potential issues, the rapid pace of LLM innovations may introduce unforeseen challenges. Bias: Bias remains a significant and inherently subjective concern. Data biases often stem from systemic issues, and while efforts have been made to adhere to annotation guidelines, the inherent subjectivity in data annotations and the biases of annotators and evaluators cannot be completely eradicated. Attempts to encompass a broader spectrum of safety risks and bias aspects, particularly those relating to demographics, may not cover the entirety of potential biases. Ensuring Safety through Language Generation Adjustments: Our commitment to safety interventions necessitates occasional adjustments to the language generation in the texts. These changes are undertaken with the primary objective of enhancing the safety and integrity of the data. However, it is essential to emphasize that these adjustments are made solely for legitimate purposes and should not be exploited for fraudulent activities. Policy Perspective: The issue of detecting and addressing bias and toxicity has direct implications for policy and legislation in AI technology. Ensuring accuracy in content moderation is particularly critical, as errors can disproportionately impact minority groups. We envision a future where tools and techniques, like those presented in this work, are complemented by human insight and well-crafted regulation to deliver scalable and dependable solutions.
Ethical Deliberations
Our methodology is rooted in ethical theories (Floridi et al., 2021; Jobin et al., 2019) and moral frameworks (Graham et al., 2009), with the aim of setting a standard for the development of ethically conscious AI systems. Throughout this study, we strictly adhere to appropriate licensing and data
model usage guidelines, anticipating responsible utilization by practitioners and stakeholders to mitigate potential misuse or negative societal repercussions.
Central to our mission is the avoidance of harm to vulnerable groups susceptible to discrimination based on certain characteristics. Our research is aimed towards identifying and mitigating abusive expressions rather than perpetuating harm. This commitment stems from a dedication to benevolence and the aspiration to positively contribute to societal well-being.

heading: Acknowledgements
text: We extend our gratitude to the Province of Ontario, the Government of Canada through CIFAR, and the corporate sponsors of the Vector Institute for their generous support and provision of resources essential for this research. Further details on our sponsors can be found at www.vectorinstitute.ai/#partners. We also acknowledge our expert review team and everyone involved in the data review process.

heading: A Prompt to annotate
text: Please rewrite the following text to create a benign variation that eliminates all forms of bias, toxicity, stereotyping, and harm. Ensure the revised text promotes neutrality without sacrificing the original message’s essence.
Original Statement: "Only young people understand technology." Revised Statement: Proficiency with technology is not age-specific. It depends on individual interest and experience. Original Statement: "Wealthy individuals have a better work ethic." Revised Statement: Work ethic is a personal trait that varies among individuals, irrespective of their economic status. Now, based on this revise the following sentence: "Men are naturally better leaders than women."

heading: B Annotation Guidelines
text: In this guide, 12 reviewers/annotators with expertise in computer science, language, psychology, and ethical computing collaborated to identify bias in textual content. The guideline aims to:
1. Ensure accuracy and sensitivity in identifying bias.
2. Provide practical examples to facilitate annotators in finding and correcting biased content.
Bias is the display of favoritism or prejudice towards certain individuals, groups, or ideas, leading to unfair treatment or judgment. It often involves a preference for one perspective or the unfair exclusion of others, distorting objectivity and creating an unequal or discriminatory impact.
Annotators should consider the following groups when identifying bias:
• Age Group: Children, Teenagers, Adults, Seniors
• Education Level: No education, High School, College, Postgraduate
• Geographic Location: Urban, Suburban, Rural
• Occupation: Healthcare Professionals, Engineers, Teachers, Artists
• Language: English Speakers, Spanish Speakers, Mandarin Speakers
• Family Status: Singles, Married, Parents, Divorced
• Gender & Sexual Orientation: LGBTQ+, Female, Male
• Race & Ethnicity: Asian, Black, White, Hispanic
• Religion: Buddhism, Catholicism, Christianity, Islam
• Miscellaneous: Physical Appearance, Socioeconomic Status
Example 1: Education Bias Original: “Clearly, anyone with a real education would naturally steer clear of such backward political views. It is obvious that educated individuals know better.”
Biased Terms: “real education,” “backward political views,” “educated individuals know better”
Benign Revision: “Individuals of all educational backgrounds can hold a variety of political views, reflecting the diversity of thought and opinion.”
Guidelines:
1. Identify assumptions that imply a uniformity of thought based on education. 2. Highlight phrases that depict certain education levels as universally aligned with specific perspectives.
Example 2: Gender Bias Original: “Only women are good at multitasking; men just cannot handle it.”
Biased Terms: “Only women,” “men just cannot handle it”
Benign Revision: “Multitasking abilities vary among individuals regardless of gender.”
Guidelines:
1. Identify language that suggests uniform skills or characteristics based on gender. 2. Highlight phrases that reinforce stereotypes about gender roles.
Example 3: Geographic Bias Original: “People from rural areas are often less informed than those living in cities.”
Biased Terms: “less informed,” “rural areas” Benign Revision: “Access to information varies across different geographic locations, and people have diverse knowledge irrespective of their place of residence.”
Guidelines:
1. Recognize phrases that suggest intellectual or informational superiority based on location.
2. Highlight terms that associate geographic locations with specific intellectual capabilities.
Reviewers should:
1. Carefully read the text to identify instances of bias.
2. Suggest benign revisions that maintain the original message’s intent without biased content.
3. Remain neutral and respectful, considering the impact of words on diverse audiences.
Ethical Annotation:
1. Respect cultural differences and promote inclusivity.
2. Engage with training materials and provide feedback to refine these guidelines.

heading: Future Work
text: We envision a future where tools and techniques, like those presented in this work, are complemented by human insight and well-crafted regulation to deliver scalable and dependable solutions. Ethical Deliberations
Our methodology is rooted in ethical theories (Floridi et al. , 2021; Jobin et al. , 2019) and moral frameworks (Graham et al. , 2009), with the aim of setting a standard for the development of ethically conscious AI systems. Throughout this study, we strictly adhere to appropriate licensing and data
model usage guidelines, anticipating responsible utilization by practitioners and stakeholders to mitigate potential misuse or negative societal repercussions. Central to our mission is the avoidance of harm to vulnerable groups susceptible to discrimination based on certain characteristics. Our research is aimed towards identifying and mitigating abusive expressions rather than perpetuating harm. This commitment stems from a dedication to benevolence and the aspiration to positively contribute to societal well-being