ChatGPT Based Data Augmentation for Improved Parameter-Efficient Debiasing of LLMs

Authors: pengrui  han
rafal  kocielnik
adhithya  saravanan
roy  jiang
anima  anandkumar
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 05:38:16

Abstract: Large Language models (LLMs), while powerful, exhibit harmful social biases. Debiasing is often challenging due to computational costs, data constraints, and potential degradation of multi-task language capabilities. This work introduces a novel approach utilizing ChatGPT to generate synthetic training data, aiming to enhance the debiasing of LLMs. We propose two strategies: Targeted Prompting, which provides effective debiasing for known biases but necessitates prior specification of bias in question; and General Prompting, which, while slightly less effective, offers debiasing across various categories. We leverage resource-efficient LLM debiasing using adapter tuning and compare the effectiveness of our synthetic data to existing debiasing datasets. Our results reveal that: (1) ChatGPT can efficiently produce high-quality training data for debiasing other LLMs; (2) data produced via our approach surpasses existing datasets in debiasing performance while also preserving internal knowledge of a pre-trained LLM; and (3) synthetic data exhibits generalizability across categories, effectively mitigating various biases, including intersectional ones. These findings underscore the potential of synthetic data in advancing the fairness of LLMs with minimal retraining cost.

Main paper: 
 introduction: 
 Recent advancements in Large Language Models (LLMs) have significantly improved Natural Language Processing (NLP) capabilities. However, concerns about fairness have emerged (Bender et al., 2021b) , highlighting that LLMs may inherit and amplify real-world biases like racial and gender biases (Kirk et al., 2021) , toxicity (Gehman et al., 2020) , and misinformation (Weidinger et al., 2022) due to their training on vast human-generated text data. This issue is critical when LLMs are applied in sensitive areas like healthcare, job recruitment, or criminal prediction, where such biases could lead to widespread discriminatory outcomes.Considerable efforts have been made in recent research to debias LLMs. However, with the large size of these models, social bias mitigation appears to be particularly challenging (Xie & Lukasiewicz, 2023; Brown et al., 2020; Hoffmann et al., 2022) . Traditional methods are computationally expensive as they often require model retraining (Tokpo et al., 2023) . On top of that, retraining on limited data can lead to lowering LLM's general language capabilities due to catastrophic forgetting (Fatemi et al., 2023) . On the contrary, recent parameter-efficient methods (He et al., 2022; Ding et al., 2022; Xie & Lukasiewicz, 2023 ) offer a good alternative as they only require minor and targeted parameter adjustments. While more efficient, these approaches heavily rely on the quality of training data (Delobelle et al.) and may offer a limited generalization, posing a challenge for comprehensive bias reduction (Li et al., 2022) .Our Approach In this work, to bolster the robustness of light-weight debiasing, we propose a method to systematically prompt ChatGPT (Ouyang et al., 2022) to generate synthetic training data for LLM debiasing (Fig. 3 ). This is achieved using two distinct prompting strategies: targeted prompting and general prompting, complemented by an auxiliary method, loss-guided prompting. The first one is meant to debias models for a concrete category, which requires prior knowledge about the social bias to target, whereas the latter one has the potential to offer comprehensive debiasing and helps assess the generalizability of synthetic data to unknown social bias categories.In particular, targeted prompting creates data specifically to address a particular category of bias, while general prompting generates data intended to be useful for mitigating bias across a range of categories. We conducted extensive evaluations on the impact of bias mitigation using our synthetic dataset through the parameter-efficient method of adapter tuning (Houlsby et al., 2019) across racial, gender, and religious bias. We also show promising results in debising models for challenging intersectional categories based on a recent BiasTestGPT dataset (Kocielnik et al., 2023b) .Findings Our primary findings include: • Our synthetic data effectively mitigates bias in popular LLMs (Fig. 1 ). On GPT2 and BERT, our approach surpassed the performance of the recent Wikipedia-based dataset from Xie & Lukasiewicz (2023) . Specifically, our best method enhances bias mitigation by an average of 6.4% on GPT-2 and 1.7% on BERT. Detailed results are in (Tables 2, 3 , 5). • Our method generalizes broadly reducing bias across GPT-2 family models by: 8.2% in LLaMA-3B, 5.8% in both OPT-350m and GPT-Neo-125m (Fig. 2 ). • We also show promising results for challenging intersectional category related to Mexican Females from Kocielnik et al. (2023b) where we lower bias on GPT-2 by 12.9% (Table 6 ). • As a result of our debiasing strategies, the general language model capabilities (LMS) in GPT-2 models are either slightly improved or minimally diminished (less than 1.3%). For BERT models, the variation in LMS is within 2.5%. • Debiasing performance is improved with much less data, speeding up training by up to 60 times compared to Wikipedia-based baselines (Xie & Lukasiewicz, 2023) .We contribute by: • Introducing a novel approach to bolster the robustness of parameter-efficient debiasing by prompting ChatGPT to generate high-quality synthetic debiasing data. • Proposing two methods for synthetic data generation for debiasing: targeted -providing superior debiasing but requiring prior knowledge of social bias definition, and generalmitigating a range of social biases without prior knowledge but at the cost of reduced overall effectiveness. • We further experiment with a variation of targeted prompting, a loss-guided prompting, that yields promising initial results on the BERT model. • We share the code in our GitHub repository.  
related work: 
 2.1 Social Bias in Large Language Models (LLMs) Biases can lead to harmful outcomes and may emerge at different stages of constructing and deploying an LLM (Suresh & Guttag, 2019) . The primary causes include inherent biases in the training data and those introduced during the model construction stage, where the chosen architecture and algorithms may propagate certain biases (Ferrara, 2023; Bender et al., 2021a; Baumann et al., 2023) .Since LLMs like ChatGPT are trained on vast amounts of real-world data, limitations in current data preprocessing techniques can result in the retention of various biases within the pre-training dataset. These biases may include human stereotypes, underrepresentation of certain social groups, racism, hate speech, and other forms of bias (Ferrara, 2023; Zhao et al., 2019b; Bender et al., 2021a; He et al., 2023) . Despite significant efforts to cleanse and curate datasets, the massive volume and complexity of the data often mean that some undesirable elements and imbalances in the representation persist. These data issues can potentially introduce biases to the language models during the training stage, and thus reproduce or even reinforce existing societal prejudices and inequalities (Li et al., 2023) .The model architecture and training algorithm themselves can introduce additional bias. Modern LLMs are designed to generalize from their training data to new, unseen inputs. This generalization is a double-edged sword. While it enables the models to understand text and sentences they have not previously encountered and to generate seemingly relevant responses to a wide range of queries, it also carries the risk of capturing only the central mass of the observed distribution, while failing to correctly model the tails. This can lead to perpetuating stereotypes and biased inferences (Ferrara, 2023) . Bias mitigation in LLMs often involves a compromise in general performance and language capabilities. This trade-off represents a fundamental challenge in LLM bias mitigation efforts (Li et al., 2023) .Studying and mitigating biases in LLMs has become increasingly important. Substantial effort has been devoted to addressing both algorithmic and training data bias. For algorithmic bias, prior work focused on developing model-agnostic human-understandable methods for detecting and mitigating biases (Richardson et al., 2023; DiCiccio et al., 2023) . However, a significant challenge lies in the scarcity of suitable datasets, often relying on a limited number of benchmark datasets (Baumann et al., 2023) . Additionally, these efforts do not extend to addressing biases inherent in the training data.On the other hand, more general debiasing methods have been developed. These include novel algorithms (Yu et al.; Ma et al., 2020) , leveraging pre-trained language models for generating gender variants of given texts (Jain et al., 2022) , using unsupervised pipelines to curate and refine instances mentioning stereotypes (Gaci et al., 2023) , increasing the amount of data used for training (Liang et al., 2020b; Schick et al., 2021a; Wang et al., 2022) , and employing extra prompting to suppress social biases (Oba et al., 2023) . However, these debiasing techniques often still rely on manual templates and are limited by the availability of the data (Delobelle et al.) . Additionally, existing datasets have been shown to exhibit issues related to data quality and reliability (Blodgett et al., 2021) . Furthermore, curated datasets are often difficult to expand and generalize. Consequently, using them for debiasing could lead to overfitting specific social bias categories.More importantly, even in the presence of proper data, the large-scale training methods that are not parameter-efficient or using post-hoc method (Liang et al., 2020a; Schick et al., 2021b) can be costly and potentially compromise the general language capabilities of an LLM (Xie & Lukasiewicz, 2023; Fatemi et al., 2023) . This makes the process of debiasing a challenging endeavor. In this context, recent work of Xie & Lukasiewicz (2023) is noteworthy. They evaluated parameter-efficient debiasing methods on two popular language models, BERT (Devlin et al.) and GPT-2 (Radford et al., 2019) , against gender, racial, and religious biases using existing datasets. Although the methods are more computationally efficient, they rely on a limited subset of the data from Wikipedia which can be used for counterfactual augmentation. The evaluation is also limited in scope to small LLMs. We contrast the effectiveness of the debiasing approach in this work to our own approach which utilizes dynamically generated synthetic data for bias mitigation. We further expand the debiasing to larger modern LLMs.As our debasing approach utilizes synthetic data generation, we provide a brief background of related methods in this area. Synthetic data plays a crucial role in scientific research. Commonly, it serves two primary purposes: 1) to emulate real-world data, enabling the creation of large datasets while preserving privacy (Angwin et al., 2022; Baumann et al., 2023) , and 2) to generate test cases for evaluation purposes (Le Quy et al., 2022) .Traditionally, synthetic data generation has mainly relied on learning probability distributions from real data (Baumann et al., 2023) . However, this approach often faces limitations, particularly when real data is scarce (Liu et al., 2024) . Learning from limited data can also be costly and inflexible, requiring complex modeling techniques or human-designed frameworks (Guo & Chen, 2024 ; CRACS-INESC TEC & Faculty of Sciences, 2022). In contrast, generative AI, particularly LLMs trained on vast real-world data, can synthesize new data that closely resembles real data in both structure and content (Veselovsky et al., 2023) . Previous studies have shown that LLMs can be used as effective training data generators for many applications (Borisov et al., 2022; Chintagunta et al., 2021; Sun et al., 2024) . Recent research (Yu et al., 2024; Wang et al., 2023) further demonstrates that it is possible to control the quality and dimensions of the data during generation to achieve better outcomes. However, few studies have systematically developed strategies to generate efficient data specifically targeted for debiasing. In this work we leverage ChatGPT's knowledge of social bias to generate highly effective and generalizable data for mitigating bias in LLMs. 
methodology: 
 Figure 3 introduces our debiasing framework and prompting strategies for synthetic data generation used for parameter-efficient LLM debiasing.Targeted Prompting: In the targeted prompting approach, we prompt ChatGPT to produce sentences that aim to debias a specific category. Our first step is to identify the category of bias we aim to mitigate. The generation process consists of two primary components: term generation and sentence generation. Initially, we prompt ChatGPT to produce social group terms related to the chosen bias category by providing a few sample terms. Subsequently, we prompt ChatGPT to create anti-stereotyped sentences using the generated terms. We instruct ChatGPT to generate sentences that counter prevailing stereotypes associated with a particular social group (e.g., race-related terms). The desired output format is communicated by asking ChatGPT to produce sentences that connect a social group term with an antistereotyped attribute. Each generated sentence, "S", should also indicate the corresponding social group term, "T", and attribute term, "A", following the format: S, T, A. The previously generated terms serve as references for ChatGPT during this process.To ensure the quality of the produced data, we include additional specific instructions. For instance, we ask ChatGPT to diversify the terms used and to produce sentences with varying levels of complexity. All relevant terms can be found in Appendix 21 and 23. Examples of sentences and visualizations of terms are presented in Table 1 and Fig. 4 respectively. The prompts used for ChatGPT are detailed in Appendix 17.The General Prompting approach aims to produce data that mitigates biases across various categories. Consequently, during the generation process, we afford ChatGPT greater freedom. We neither select specific bias categories nor generate social group terms. Instead, we directly prompt ChatGPT to create anti-stereotypical sentences that counteract stereotypes, adhering to the ["S","T","A"] format previously detailed. All terms are located in Appendix 22 and 24. Meanwhile, examples of sentences and visualization of terms are in Table 1 and Fig. 4 . The ChatGPT prompts are in Appendix 17. We formalize Targeted and General Prompting in Algorithm 1.We observed diminished effectiveness and a more pronounced trade-off between debiasing performance and language ability in models outside the GPT family, such as BERT, when using synthetic data generated from ChatGPT. This could be due to out-of-distribution generations from ChatGPT that harm the pre-trained knowledge of BERT in the course of further pre-training. A phenomenon known as catastrophic forgetting (Luo et al., 2023) . To address this, we aim to guide ChatGPT to generate more in-distribution sentences for the given LLM. We select 50 samples exhibiting the highest and lowest loss, respectively under given LLM, from the generated data for each category. These samples, along with their corresponding loss scores, were then provided back to ChatGPT. This approach guides ChatGPT to generate data that is more in-distribution.Loss-Guided Prompting is an auxiliary method used to generate more in-distribution data for targeted and general prompting, its format follows these two strategies, and we do not present it separately in the Table 1 . We formalize Loss-guided Prompting in Algorithm 2.Training Methodology: We train LLMs using synthetic data generated by GPT4 through adapter tuning (Houlsby et al., 2019) . Adapter tuning freezes all the original parameters of an LLM and introduces, additional adapter layers into the original model architecture. Only these are trained for downstream applications. For GPT-2 and other GPT2 family models, we modify the sentence to position the attribute word at the end, employing the Causal Language Model loss as our training objective. For the BERT model, we mask the attribute word within the sentence and use the Masked Language Modeling (MLM) loss as the objective. 
experiment: 
 Metrics and Datasets: To align with Xie & Lukasiewicz (2023) , we use the CrowS-Pairs (Nangia et al., 2020) and the StereoSet (Nadeem et al., 2021) intrasentence datasets. The CrowS-Pairs dataset comprises pairs of contrasting sentences, one of which is more stereotyped than the other. The StereoSet intrasentence dataset contains entries each composed of a stereotyped sentence, an anti-stereotyped sentence, and an unrelated sentence. The differences among these sentences are solely the attribute word. The CrowS-Pairs dataset contains 262, 105, and 516 entries for gender, religion, and race, respectively. For the StereoSet intrasentence set, there are 1026, 623, and 3996 examples respectively. For bias evaluations, we adopt the "stereotype score (SS)" from Meade et al. (2022) . It quantifies the preference of an LLM for a stereotypical association over an anti-stereotypical one, with an ideal score being 50% for an unbiased model. To assess a model's general language capability, we use the "language modeling score (LMS)" from Nadeem et al. (2021) . It measures the model's preference for meaningful associations over unrelated ones, with an ideal score of 100%.Training Details: For the targeted prompting experiment, we generated three datasets for every category of bias, each containing 500 targeted sentences. For the general prompting experiment, we produced three datasets, each with 500 general sentences. We also tested various data sizes and selected 500 based on the optimal balance between debiasing performance and language ability impact. The performance graph for different data sizes is included in the Appendix 19. To represent masked and autoregressive language models, and to align with Xie & Lukasiewicz (2023) , we chose to debias BERT (Devlin et al.) and GPT-2 Targeted Prompting Gender "Love the Godfather not for his power, but for his genuine kindness.""Godfather" "Kindness"Race "The Scottish community has been a beacon of innovation in various fields.""Scottish" "Innovation"Religion "Many students find the Quran to be enlightening.""Quran" "Enlightening"General Prompting General "Just because she's a cheerleader doesn't mean she isn't the top student in her calculus class.""Cheerleader" "Calculus""She found that the skateboarder was also a connoisseur of classical music.""Skateboarder" "Classical"Table 1 : Example data of Targeted and General Prompting, including sentence, subject word, and attribute word for each example. Additional examples can be found in Appendix 16. (Radford et al., 2019) . To show the generalizability of our method, we also experimented with other GPT2 family models: Llama 3b v2 (Touvron et al., 2023) (the latest version of LLaMA-3B model), OPT-350m (Zhang et al., 2022) , and GPT-Neo-125m (Gao et al., 2020) . We use Adapter Hub (Pfeiffer et al., 2020) and the code from Xie & Lukasiewicz (2023) .We trained Llama 3b v2 model on a Google Colab A100 GPU. All other experiments were conducted on a Google Colab V100 GPU. Based on empirical findings and the ratio between SS and LMS, we set the learning rate for the GPT-2 model to 5 × 10 -6 . For BERT, the learning rate was set to 1 × 10 -5 . For Llama 3b v2 and OPT-350m, we used 5 × 10 -5 , and for GPT-Neo-125m: 5 × 10 -4 . For each bias category or for general debiasing, we conducted the experiments for the three datasets separately and reported the average outcomes as well as the standard deviations.Baseline: For GPT-2 and BERT, we compare our debiasing approach, which uses synthetic datasets via adapter tuning, with other parameter-efficient methods and existing datasets, focusing particularly on the work of Xie & Lukasiewicz (2023) . In their study, the authors down-sample 20% of the English Wikipedia as the debiasing corpus and augment it counterfactually for training (Zhao et al., 2019a; Zmigrod et al., 2019; Webster et al., 2020) . The debiased corpus is then used with three distinct parameter-efficient methods: prefix tuning (Li & Liang, 2021) , prompt tuning (Lester et al., 2021) , and adapter tuning (Houlsby et al., 2019) . For other models in the GPT-2 family, due to the lack of relevant prior work for comparison, we assessed the effectiveness of debiasing by comparing the debiased versions of the models to their original versions with weights before our debiasing. 
results: 
 Mitigating Racial Bias: Table 2 shows that our synthetic data surpasses all other parameter-efficient methods that utilize English Wikipedia for GPT-2 models in mitigating racial bias. For BERT, our results are in line with the baselines. Our general debiasing achieves the best SS for StereoSet and yields results comparable to others for the SS on CrowS-Pairs, with the difference being less than 3%. In terms of language capability, our We present the average bias score with standard deviations from 3 runs paired with the % change compared to the model prior to debiasing. The first column lists the dataset and the parameter-efficient method employed. "Wiki-debiased" is baseline dataset from recent work (Xie & Lukasiewicz, 2023) . "Synthetic-targeted" and "Synthetic-general" refer to our synthetic data generated via targeted and general prompting. "Prefix", "Prompt", and "Adapter" denote the three parameter-efficient methods.For instance, "+Synthetic-targeted + Adapter" means debiasing with synthetic data from targeted prompting using the adapter tune method. For both CrowS-Pairs and StereoSet datasets, a score closer to 50 (SS) is optimal, reflecting less bias. For the Language Model Score (LMS), a higher score is indicative of enhanced language capabilities. The positive direction of change is denoted in blue, while the negative is in red. The best score under each metric is marked in bold and underscored.synthetic targeted approach secures the highest score on the GPT-2 model. For BERT, while our approach is outperformed by the baselines, the difference remains within 3.5%.For the other models in the GPT-2 family, both targeted and general prompting strategies mitigate bias across both metrics, achieving an average reduction of 5.7% for targeted debiasing and 5.4% for general debiasing. Meanwhile, language ability is well-preserved: it is either slightly improved (approximately 0.5%) or minimally diminished (less than 1.3%).As seen in Table 3 , our synthetic data outperforms all other methods in the baseline for the GPT-2 model. While it slightly underperforms in LMS, the difference is marginal, at under 2%. For the BERT model, with the incorporation of loss-guided prompting, our synthetic data achieves the best results compared to all other methods in the baseline. In terms of LMS, the discrepancy is less than 2.5%.For the other models in the GPT-2 family, general debiasing proves highly effective, yielding an average bias reduction of 7.2%. However, targeted debiasing is less effective, achieving an average reduction of only 0.7%. In terms of LMS, it is well preserved, exhibiting a variation of only 1.0% compared to the original LMS.Mitigating Gender Bias: Our approach effectively reduces gender bias (Table 5 ). On GPT-2, our targeted data achieves the best SS on Stereoset, and our general data outperforms the baseline in the average SS score. In the case of BERT, although we did not surpass the baseline, with the implementation of loss-guided prompting, we still achieved an average bias reduction of 3.9% in loss-guided targeted debiasing and 2.6% in general debiasing. For the LMS, the difference is around 2.5%. Our method is also highly effective in reducing gender bias in other models in the GPT-2 family. We achieve an average reduction of 7.5% with targeted debiasing and 5.8% with general debiasing. The LMS varies within a margin of 1.0% compared to the original model.Debiasing is Effective: Across all three categories of bias, our synthetic data, generated through both targeted, general, and loss-guided prompting, has demonstrated its effectiveness under different metrics. On GPT-2, our targeted debiasing approach reduced social bias by an average of 10.2% on StereoSet and 7.9% on CrowS-Pairs, while general debiasing achieved reductions of 5.3% and 5.1%. These figures surpass our baseline, which achieved average reductions of 2.5% on StereoSet and 2.2% on CrowS-Pairs. For BERT, our general debiasing approach reduced biases by 1.8% and 4.9%, exceeding existing methods with 1.6% and 3.2% reductions. However, targeted debiasing was less effective for BERT, showing no improvement on StereoSet and a 1.6% reduction on CrowS-Pairs. We addressed this by introducing a loss-guided targeted approach for BERT, enhancing results to 2.1% on StereoSet and 4.5% on CrowS-Pairs, thereby surpassing the baseline.We demonstrated broad generalizability in bias reduction across various GPT family models, including LLaMA-3B, OPT-350m, and GPT-Neo-125m, across three bias categories. For LLaMA-3B, bias was reduced by 7.1% and 6.8% using targeted and general strategies, respectively, on StereoSet, and by 6.2% and 9.7% on CrowS-Pairs. On OPT-350m, reductions were 5.3% and 4.6% on StereoSet, and 3.3% and 4.1% on CrowS-Pairs. GPT-Neo-125m showed decreases of 4.9% and 6.0% on StereoSet, and 1.0% and 5.7% on CrowS-Pairs.Targeted Prompting Usually More Effective: Targeted prompting is more effective than general prompting in most cases. This is in line with our expectations that more prior knowledge leads to more robust debiasing. On the other hand, general debiasing compromises a bit of effectiveness in exchange for a broader range of bias mitigation.Debiasing & Language Capability Trade-off: A noticeable trade-off emerges between language proficiency and bias mitigation when working with the BERT model. Although this trade-off was reduced through loss-guided prompting, it still presents an important focus of future exploration. Training costs-both in terms of time and memory-are substantially reduced. With smaller dataset than the baselines, we expedite the training process by approximately a factor of 60. We frequently secure results that match or surpass the baselines and original models in terms of bias mitigation and language ability. 
synthetic dataset analysis: 
 Dataset Similarity: A natural concern arises that ChatGPT may know and merely reproduce the original test sets. We analyzed the similarity between the generated synthetic data and the test set. We compared the original StereoSet test set, the StereoSet development set, a different dataset, our synthetic dataset, and another StereoSet development set for various bias categories to check the uniqueness of our synthetic data. Table 7 in the Appendix reveals that for our synthetic dataset, the similarity matches that of a different dataset. For the targeted synthetic dataset, there is a pronounced similarity in terms of social group terms. This is anticipated because generating an extensive list of corresponding social group terms inevitably results in numerous overlaps and analogous terms. The authors of StereoSet manually ensured that the development and test sets did not share the same social group terms. We refrained from doing this to avoid referring to the test set.Unseen Biases: To further ensure our synthetic data is not overfitting to the existing datasets, we use BiasTestGPT (Kocielnik et al., 2023b) , which generates varied test sentences for different social categories and attributes through ChatGPT. While this dataset uses ChatGPT for sentence generation, the crucial social group and attribute terms defining bias categories are taken from psychology-backed studies from Guo & Caliskan (2021) .We examine these biases for GPT-2 and BERT respectively (Table 6 in Appendix 14). For GPT-2, our debiasing effectively mitigates bias in a variety of categories including similar, intersectional, and less related categories. In the case of BERT, we observe a clear trade-off between language ability and bias mitigation, which aligns with our previous experiments.We performed additional analysis of the potential toxicity and negative sentiment using unbiased ToxicBERT (Hanu & Unitary team, 2020) and VaderSentiment (Hutto & Gilbert, 2014) . Table 4 shows that the scores for our synthetic data are very low, far below the 0.5 threshold for considering a text toxic (on a scale from 0.0 to 1.0). This is even for the top 95% percentile of highest-scoring generations. A manual inspection of the 55 sentences (out of all 6k generated) that scored above 0.5 on any of the dimensions revealed that toxicity detectors may be overly sensitive to certain keywords and contexts that, while potentially sensitive, are not inherently negative. Terms like "arrogant" or "angry," or references to specific nationalities or religious texts (e.g., "Serbian," "African," "the Analects of Confucianism"), seem to trigger automated flags, despite being used in a neutral or positive context. Similarly, unique combinations of roles and attributes meant to counter stereotypes seem misinterpreted (e.g., "garbage collector and a specialist in aero-dynamics"). Further details and scores for other dimensions of disruptive contents are presented in Table 9 in Appendix 20. 
discussion: 
 We introduced synthetic data generation via targeted and general prompting to debias Large Language Models (LLMs). Our findings offer several avenues for deeper exploration.Our methodologies-targeted versus general prompting-vary in their approach and effectiveness across models. Targeted Prompting provides specificity in debiasing certain categories, while General Prompting offers a broader spectrum of bias mitigation. Notably, the effectiveness of these strategies demonstrated variation across models, such as GPT-2 and BERT, and different bias categories. One potential explanation is the difference in model architectures, affecting how each processes training data. Another reason could be the variance in training data, where different datasets or preprocessing methods influence the model's behavior. Finally, the specificity of bias categories might play a role, with targeted prompting being more effective for well-defined biases and general prompting for more complex or subtle biases. Further investigation is needed here.Understanding Trade-offs: We observe a trade-off between language capability and bias mitigation, particularly pronounced in the BERT model (see graph in Appendix 15). This might be because the synthetic data is generated by ChatGPT, which significantly differs from BERT. We generate more in-distribution data through loss-guided prompting, which mitigates the issue, supporting the hypothesis. Nevertheless, the trade-off between the debiasing performance and the language ability is a fundamental problem (French, 1999) . Understanding this trade-off becomes pivotal in different use cases and applications.Evaluating Synthetic Data's Universality: Our similarity analysis underscores the uniqueness of our synthetic data, ensuring it isn't merely a reproduction of known datasets. Some robustness against different biases in another dataset -BiasTestGPT, suggests broader applicability. This is particularly relevant in an ever-evolving societal landscape with shifting norms and biases (Linegar et al., 2023; Kocielnik et al., 2023a) . We note that further exploration of prompting strategies should investigate a good trade-off between prior knowledge of bias and the universality of the generated debiasing data. 
limitations: 
 Our evaluation primarily relies on benchmarks and datasets with a North American English focus, which may not fully represent global biases. The effectiveness of our debiasing might vary in tasks outside our testing scenarios. There's also a concern that ChatGPT's exposure to test sets could have impacted our synthetic datasets (Prabhumoye et al., 2021) . We investigated this possibility by checking if the synthetic data merely replicates known datasets and by experimenting with a newer dataset -BiasTestGPT. Nevertheless, alignment with test sets may still exist. Moreover, the dynamic nature of societal biases, which continually evolve, may require updates of our datasets. Our focus on explicit biases may overlook subtler ones (Goethals et al., 2024) . This emphasizes the need for careful interpretation of our results and continuous improvement in debiasing approaches. 
conclusion: 
 This paper presents two new methods for generating synthetic data to reduce social bias in LLMs more efficiently: general and targeted prompting. These methods outperform the recent work using parameter-efficient debiasing in bias mitigation and training efficiency.They also preserve language model capabilities. Our work highlights the potential of synthetic data in making LLMs fairer and suggests future research directions, including improving synthetic data generation, applying our approach to other domains such as vision, and exploring its broader applications beyond fairness. Lowering bias in a language model is likely to impact its general language proficiency. This represents a fundamental challenge in the field of language model fairness. 
appendix -performance graph for different data sizes: 
 Figure 6 : Performance across different data sizes. The 200 data size yields minimal debiasing performance, while the 1000 data size significantly impairs the model's language capability. Thus, to achieve a balance between debiasing performance and language capability, a data size of 500 is selected. 
appendix -additional details of toxicity and sentiment analysis: 
 VaderSentiment (Hutto & Gilbert, 2014) produces individual positive, negative, and neutral sentiment component scores and a compound score. We report just the values for the negative sentiment component as these are the most relevant here. Aside from general Toxicity presented in Table 4 , we also analyzed all the dimensions provided by ToxicBert (Hanu & Unitary team, 2020) -Severe Toxicity, Identity Attack, Insult, Sexually Explicit, Obscene, and Threat. We present the results for these other dimensions in Table 9 .  
appendix -subject words for targeted prompting data: 
 Table 12 : Subject Words for Religious Bias Data through Targeted PromptingReligion Generation 1 analects( 9) druidry( 9) voodoo( 8) torah( 7) guru granth sahib( 7) shamanism( 7) zen( 7) sufism( 7) gospel( 7) talmud( 6) taoism(6) baha'i(6) book of mormon( 6) rastafarianism( 6) wicca( 6) santeria( 6) mahayana( 6) kabbalah( 6) hasidism( 6) yazidism( 6) deism( 6) pantheism( 6) unitarianism( 6) mennonite( 6) mosque( 6) church( 6) tao te ching(6) kitáb-i-aqdas( 6) alevism( 6) avesta( 6) shinto( 6) candomblé( 6) vajrayana( 6) druze( 6) quran( 5) buddhism( 5) christian( 5) jainism( 5) sikhism( 5) hadith( 5) catholic( 5) orthodox( 5) paganism( 5) native american church(5) falun gong(5) dianetics( 5) theravada( 5) coptic( 5) gnosticism( 5) monotheism( 5) presbyterianism( 5) amish(5) jehovah's witnesses(5) synagogue( 5) temple( 5) monastery( 5) ritual( 5) bektashi( 5) agnosticism( 5) atheism( 5) animism( 5) nichiren( 5) wahhabism( 5) ahmadiyya( 5) calvinism( 5) seventh-day adventist(5) society of friends( 5) universalism( 5) dualism( 5) baptism( 5) hindu( 5) protestant( 4) zoroastrianism( 4) kojiki( 4) lutheran( 4) pilgrimage( 4) umbanda( 4) samaritanism( 4) polytheism( 4) manichaeism( 4) anglicanism(4) church of satan( 4) tenrikyo( 4) bible( 4) mandaeanism( 4) islam( 3) shia( 3) quakerism( 3) scientology( 3) sunni( 3) mormonism( 3) confucianism( 3) upanishads( 2) lutheranism( 1) pagan( 1) centers( 1) puranas( 1) tantrism(1) bhagavad gita(1) hare krishna(1) shaktism( 1) vaishnavism( 1) shaivism( 1) sankhya( 1) vedanta( 1) advaita( 1) rigveda( 1) samaveda( 1) atharvaveda( 1) brahmanas( 1) aranyakas(1) Religion Generation 2 baha'i( 9) candomblé( 9) wicca( 8) sufism( 8) jainism( 7) talmud( 7) protestant( 7) zoroastrianism( 7) kojiki( 7) tao te ching( 7) analects( 7) kitáb-i-aqdas( 7) voodoo( 7) animism( 7) paganism( 7) druidry( 7) shamanism( 7) church of satan( 7) mosque( 7) bible( 6) hadith( 6) orthodox( 6) avesta( 6) taoism( 6) rastafarianism( 6) nichiren( 6) zen( 6) kabbalah( 6) gospel( 6) synagogue( 6) hindu( 5) quran( 5) buddhism( 5) torah( 5) christian( 5) sikhism( 5) guru granth sahib( 5) islam( 5) sunni( 5) shia( 5) catholic( 5) shinto( 5) confucianism( 5) mormonism( 5) book of mormon( 5) santeria( 5) umbanda( 5) native american church(5) samaritanism( 5) tenrikyo( 5) theravada( 5) mahayana( 5) vajrayana( 5) wahhabism( 5) ahmadiyya( 5) coptic( 5) gnosticism( 5) druze( 5) alevism( 5) bektashi( 5) deism( 5) polytheism( 5) universalism( 5) quakerism( 5) calvinism( 5) mennonite( 5) seventh-day adventist(5) jehovah's witnesses( 5) scientology( 5) temple( 5) church( 5) monastery( 5) pilgrimage( 5) ritual( 5) mandaeanism(4) falun gong( 4) dianetics( 4) hasidism( 4) yazidism( 4) agnosticism( 4) atheism( 4) pantheism( 4) monotheism( 4) dualism( 4) manichaeism( 4) unitarianism(4) society of friends( 4) lutheran( 4) anglicanism( 4) presbyterianism( 4) amish( 4) baptism( 4) atheist( 1) agnostic( 1) churches(1) Religion Generation 3 wicca( 10) voodoo( 8) zen( 8) sufism( 8) jainism( 7) guru granth sahib( 7) kabbalah( 7) gospel( 7) church( 7) torah( 6) talmud( 6) hadith( 6) taoism( 6) tao te ching( 6) confucianism( 6) analects( 6) book of mormon( 6) rastafarianism( 6) animism( 6) paganism( 6) church of satan( 6) vajrayana( 6) hasidism( 6) coptic( 6) gnosticism( 6) druze( 6) yazidism( 6) alevism( 6) atheism( 6) deism( 6) pantheism( 6) manichaeism( 6) unitarianism( 6) universalism( 6) calvinism( 6) lutheran( 6) presbyterianism( 6) mennonite( 6) scientology( 6) temple( 6) pilgrimage( 6) ritual( 6) quran( 5) buddhism( 5) christian( 5) catholic( 5) orthodox( 5) avesta(5) kojiki( 5) baha'i(5) kitáb-i-aqdas(5) druidry( 5) shamanism( 5) santeria(5) candomblé( 5) umbanda( 5) mandaeanism(5) falun gong(5) dianetics( 5) tenrikyo(5) nichiren( 5) theravada( 5) mahayana( 5) wahhabism( 5) ahmadiyya( 5) bektashi(5) agnosticism( 5) polytheism( 5) monotheism( 5) dualism(5) society of friends(5) anglicanism(5) amish( 5) baptism(5) seventh-day adventist(5) synagogue( 5) mosque( 5) monastery( 5) hindu( 4) sikhism( 4) sunni( 4) protestant( 4) shinto( 4) mormonism(4) native american church(4) samaritanism(4) jehovah's witnesses(4) bible( 4) islam( 3) shia( 3) zoroastrianism (3) quakerism( 3) presbyterian(1) 
appendix -attribute words for general prompting data: 
 Table 13 : Subject Words for General Prompting DataGeneral Generation 1 librarian( 5) nun( 4) ceo( 4) rapper( 4) biker( 4) accountant( 4) lawyer( 4) bartender( 4) bodybuilder( 4) punk( 4) skateboarder( 4) desert( 4) boxer( 4) politician( 4) model( 4) tattoos( 3) football player( 3) africa( 3) hijab (3) truck driver( 3) petite( 3) monk( 3) janitor( 3) soldier( 3) comedian( 3) mechanic( 3) butcher( 3) software engineer( 3) wrestler( 3) carpenter( 3) physicist( 3) mathematician( 3) men( 3) fisherman( 3) pilot( 3) farmer( 3) baker( 3) age( 2) teenager( 2) wealthy( 2) city( 2) scientist( 2) muscular( 2) gamer( 2) beauty queen(2) construction worker(2) rugby player( 2) actor( 2) surfer( 2) firefighter( 2) prison guard( 2) cowboy( 2) goth( 2) cab driver(2) basketball player( 2) cheerleader( 2) slums( 2) banker( 2) athlete( 2) judge( 2) chef( 2) rocker( 2) insurance agent( 2) seamstress( 2) architect( 2) detective( 2) surgeon( 2) journalist( 2) teacher( 2) fishermen( 2) gamers( 2) asian( 2) australian( 2) arctic( 2) blonde( 2) fashionista( 2) dancer( 2) sailor( 2) astronaut (2) tattoo artist(2) flight attendant( 2) barista( 2) drummer( 2) cashier( 2) plumber (2) wall street( 2) priest( 2) coal( 2) fireman( 2) male(1) woman( 1) asians( 1) blind(1) tech valley(1) immigrant( 1) disability( 1) traditional( 1) overweight(1) fashion model( 1) height( 1) housewife(1) police officer(1) astrophysicist( 1) millionaire(1) sumo wrestler(1) hip-hop artist(1) saleswoman( 1) princess( 1) developer( 1) powerlifter( 1) motorcyclist( 1) metalworker(1) security guard(1) tattooed( 1) vet( 1) manager( 1) miner( 1) consultant( 1) podiatrist( 1) engineer( 1) radiologist(1) bus driver(1) painter( 1) receptionist( 1) anaesthesiologist( 1) engineers(1) football team( 1) politicians( 1) dancers( 1) grandparents( 1) bodybuilders( 1) chefs( 1) writers( 1) farmers(1) fashion models(1) construction workers(1) software developers(1) musicians( 1) artists( 1) lawyers( 1) mathematicians( 1) firemen( 1) economists(1) rugby players(1) soldiers (2) business executives(1) doctors( 1) teenagers( 2) philosophers( 1) teachers(1) truck drivers( 1) pilots( 1) nurses( 1) architects( 1) astronauts( 1) veterinarians( 1) bankers( 1) actors( 1) journalists( 1) children (2) elderly women(1) carpenters(1) marathon runners(1) boxers( 1 5) farmer( 5) janitor( 5) plumber( 5) cheerleader(4) truck driver( 4) barista( 4) boxer( 4) librarian( 4) fisherman( 4) hairdresser( 4) accountant( 4) hip-hop artist(4) dj( 4) bodybuilder( 4) waitress( 4) gamer( 3) mechanic( 3) rapper( 3) flight attendant( 3) metalhead( 3) florist( 3) cab driver( 3) principal( 3) firefighter( 3) punk rocker(3) pastry chef( 3) banker( 3) fashion designer( 3) zookeeper( 3) cashier( 3) tattoo artist( 3) lifeguard( 3) butcher( 3) clown( 3) bartender (3) rugby player( 2) athlete( 2) ballet dancer( 2) countryside( 2) kindergarten teacher( 2) teenager( 2) soldier( 2) businessman( 2) software engineer( 2) politician( 2) wrestler( 2) hipster( 2) goth( 3) chef( 2) beauty queen( 2) cop( 2) stay-at-home mom (2) receptionist( 2) surgeon( 2) football player( 2) artist( 2) dentist( 2) housekeeper (2) bus driver( 2) electrician( 2) car mechanic( 2) veterinarian( 2) model( 2) ceo( 2) bikers( 2) rappers( 2) farmers( 3) elderly( 2) ceos( 2) inner city (2) nun( 2) bouncer( 2) actress( 2) fast-food worker( 2) maid( 2) firefighters (2) sumo wrestler( 2) wheelchair( 2) surfer( 2) valet( 2) preschool teacher( 2) gardener (2) window washer( 2) intern( 2) stuntman( 2) custodian( 2) tailor( 2) graffiti artist (2) supermodel( 2) drummer( 2) hijab( 2) taxi driver(2) mma fighter( 2) monk (2) pop star( 2) fashionista( 2) nail technician( 2) bricklayer( 2) miner( 2) street vendor (2) shepherd( 2) monks( 2 6) janitor(5) tattoo artist( 5) accountant( 5) mechanic( 5) surfer( 5) rapper( 5) boxer( 5) librarian( 5) butcher(5) truck driver(4) dancer( 4) city( 4) farmer( 4) biker(4) taxi driver(4) construction sailor(4) skateboarder(4) software developer(4) banker( 4) carpenter( 4) bartender( 4) firefighter( 4) ceo (3) fashion model( 3) basketball player( 3) gamer( 3) martial artist( 3) detective( 3) politician( 3) electrician( 3) teenager( 3) chef( 3) plumber(3) flight attendant(3) actor( 3) gardener( 3) cheerleader( 3) barista(3) graffiti artist( 3) corporate( 3) fisherman( 3) nun( 3) male( 2) rural( 2) grandmother( 2) mathematician( 2) immigrants( 2) physicist( 2) linebacker( 2) opera singer( 2) comedian( 2) weightlifter( 2) pilot( 2) urban( 2) soldier( 2) motorcyclist( 2) scientist( 2) animator(2) small town (2) football player( 2 2) ghettos( 2) bellboy( 2) magician( 2) blonde( 2) hijab( 2) model( 2) wealthy( 2) housewife( 2) hairstylist( 2) miner( 2) postman( 2) baker( 2) receptionist( 2) lifeguard( 2) military( 2) coal miner( 2) desert(2) kindergarten teacher(2) mountain climber( 2) fashion( 2) sumo wrestler( 2) drummer( 2) fathers(1) senior citizen(1) men(1) software engineer(1) teenagers(1) preschool teacher(1) ceo's son(1) introverts( 1) barber(1) corporate lawyer(1) boy( 1) mountains( 1) saleswoman( 1) millennial( 1) fireman( 1) biologist( 1) gymnast( 1) journalist( 1) dentist( 1) painter( 1 7) meticulous( 7) nurturing( 6) analytical( 6) tech-savvy(6) gentle( 5) compassionate( 5) tenacious( 5) agile( 5) strategic( 4) innovative( 4) humble( 4) adventurous( 4) empathetic( 4) profound( 4) culinary( 4) prodigy( 4) fashion( 3) martial( 3) poetic( 3) leadership( 3) romantic( 3) driver( 3) robotics( 3) wise( 3) logical( 3) graceful( 3) audacious( 3) physicist( 3) empathy( 2) baking( 2) dance( 2) grounded( 2) physics( 2) ballet( 2) climbing( 2) weightlifting( 2) yoga( 2) action( 2) gourmet( 2) boxing( 2) video( 2) eloquent( 2) salsa( 2) pastry( 2) skincare( 2) virtuoso( 2) environmental( 2) emotional( 2) resourceful( 2) courageous( 2) protective( 2) shrewd( 2) calm( 2) patient( 2) cheerful( 2) mature( 2) imaginative( 2) attentive( 2) creative( 2) insightful( 2) skillful( 2) resilient( 2) humorous( 2) lively( 2) articulate( 2) candid( 2) jovial( 2) boisterous( 2) tactical( 2) intuitive( 2) whimsical( 2) flair( 2) sagacious( 2) voracious( 2) adept( 2) proficient( 2) astute( 2) erudite( 2) dexterous( 2) formidable( 2) brilliant( 2) artist( 2) entrepreneur( 2) mountaineer( 2) gardening( 2) dancer( 2) coder( 2) poet( 2) champion( 2) master( 2) warrior( 2) opera( 2) astrophysicist( 2) engineer( 2) astronomer( 2) architect( 2) marine( 2) athlete( 2) pilot( 2) biologist( 2) florist( 2) mechanic( 2) engineering( 2) stoic( 1) mechanical( 1) computer( 1) engine( 1) intuition( 1) commanding( 1) sew( 1) meditate( 1) historical( 1) music( 1) calligraphy( 1) astrophysics( 1) electronic( 1) aesthetic( 1) chess( 1) animation( 1) woodworking( 1) ornate( 1) sports( 1) pottery( 1) electric( 1) operatic( 1) basketball( 1) virtual( 1) graffiti( 1) code( 1) diving( 1) business( 1) violin( 1) detective( 1) ethereal( 1) punk( 1) architectural( 1) tech( 1) languages( 1) painting( 1) DJs( 1) mathematical( 1) bioengineering( 1) exploration( 1) flamenco( 1) blues( 1) skateboarder( 1) surreal( 1) AI( 1) sculpting( 1) artisanal( 1  
 
 Appendix: Appendix -Attribute words for Targeted Prompting Data 


Decision: The paper proposed a prompting method to generate anti-stereotype sentences (500 sentences for each of the three types of biases: rase, gender, religion) as data augmentation. The synthetically generated data is then used in adapter tuning of GPT-2 and BERT models to de-bias them. 

The paper presents an interesting and effective idea and a good amount of meaningful comparison experiments. I think there is enough novelty in the paper. 

While the authors addressed some of the reviewer's concerns (the most effective part of the author's response was by providing the automatic toxicity analysis), the paper could be further improved by taking some of the reviewer's constructive criticism as suggestions for improvement, in particular:

(1) provide more in-depth analysis (reviewer fjaf):  In addition to the post-hoc analysis that focuses on automatic experiment results or analysis, provide some more direct analysis, including manual analysis or human evaluation, about the quality of these anti-stereotype sentences. Explain more clearly, why and where exactly your method works better than others, and where it may fail (i.e., limitations of the proposed method). 

(2) update Section 2.2 to provide a more direct/comprehensive review of debiasing methods (reviewer P4Tc): include more related works and provide a discussion about the connection and difference from Attrprompt (Yu, Yue et al. 2024).

I recommend the paper for acceptance, with slight reservations on whether the authors will be willing to address the above two issues.

Reviews: review: paper_summary: paper_summary: This paper introduces a novel approach that utilizes ChatGPT to generate synthetic training data, with the aim of improving the debiasing process of Large Language Models (LLMs). The authors propose two strategies: Targeted Prompting, which effectively addresses known biases but requires prior specification of the bias in question, and General Prompting, which offers debiasing across various categories albeit with slightly less effectiveness.

The paper leverages resource-efficient LLM debiasing using adapter tuning and compares the effectiveness of the synthetic data generated by the approach to existing debiasing datasets. The results reveal several key findings: Firstly, ChatGPT can efficiently produce high-quality training data for debiasing other LLMs. Secondly, the data generated via their approach outperforms existing datasets in terms of debiasing performance while also preserving the internal knowledge of a pre-trained LLM. Thirdly, synthetic data demonstrates generalizability across categories, effectively mitigating various biases.
strength_weakness: strength_weakness: The proposed approach lacks technical novelty, as it relies on relatively simple yet effective prompting.

The study could benefit from more in-depth case studies and analysis to help readers better understand the advantages and disadvantages of the proposed approach.
questions: questions: No questions.
paper_summary: paper_summary: The paper introduces an interesting approach to debiasing large language models (LLMs) using synthetic data generated via ChatGPT, employing targeted and general prompting strategies alongside an auxiliary loss-guided prompting method. By adapting adapter tuning, this technique efficiently mitigates biases such as racial, gender, and religious prejudices with minimal degradation of language capabilities. Comparative evaluations against traditional datasets demonstrate that this method not only competes favorably but often exceeds existing debiasing strategies in effectiveness while substantially reducing computational costs and training time. The findings suggest that synthetic data can significantly advance the fairness of LLMs.
strength_weakness: strength_weakness: 1. While with a slightly different motivation, the method of this work seems to be similar to other attributed prompting-based debiasing methods like Attrprompt [1]. [1] Yu, Yue, et al. "Large language model as attributed training data generator: A tale of diversity and bias." Advances in Neural Information Processing Systems 36 (2024).
2. Existing related works in Section 2.2 seem to focus on debiasing methods before LLMs. Could you provide a comprehensive review of debiasing methods for LLMs as baselines?
questions: questions: See weakness
paper_summary: paper_summary: This paper proposes a novel data augmentation framework for the purpose of social bias mitigation by leveraging the generative power of ChatGPT. The motivation for this paper is reasonable and well supported. Furthermore, improving the quality of a debiasing corpus is a straightforward and effective solution. However, the experimental results of BERT are not very strong.
strength_weakness: strength_weakness: - One possible issue is the generated sentences might still be biased or toxic, even though ChatGPT is prompted not to be. Therefore, the paper could be improved by including validation results about the harmfulness of those generations.
- Given that the LoRA or adaptor tuning is a computationally cheap solution for debiasing, this means that we can anticipate that the adaptation layer can debias the learned internal knowledge by referring to the debiasing corpus. How many adaptation layers do we need for this purpose? What is the gap between a solution that debiases the whole network and the solution with an adaptation layer? Debiasing a small BERT model would be expensive enough, though it would strengthen the paper to have other references or tiny experiments for this. However, if the authors have hardware limitations, please ignore this.
- I am not that surprised about the potential ineffectiveness of the generated sentences for mitigating social bias from smaller LLMs, such as BERT or GPT-neo. Borrowing from the research outcomes of theoretical understanding towards in-context learning [1], if we consider that pretraining/in-context learning is to learn/locate a class of function w.r.t. the training corpus, the learned function classes for ChatGPT and BERT should be very different because of the model scales (or maybe other additional factors). I would appreciate it if our authors can suggest some solutions in this respect. A relevant case might be self-alignment [2] or self-instruction [3].

[1] Mao, Haitao, et al. "A Data Generation Perspective to the Mechanism of In-Context Learning." arXiv preprint arXiv:2402.02212 (2024).
[2] Sun, Zhiqing, et al. "Principle-driven self-alignment of language models from scratch with minimal human supervision." Advances in Neural Information Processing Systems 36 (2024).
[3] Wang, Yizhong, et al. "Self-Instruct: Aligning Language Models with Self-Generated Instructions." Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Comments: title: Additional analysis around toxicity
comment: We once again appreciate all the comments and we hope our rebuttal helped clarify and address the remaining concerns. Please let us know if we can further clarify anything. We would also like to point to the additional analysis of Toxicity, which we performed in response to the latest comments from Reviewer fjaf.
We once again appreciate all the comments and we hope our rebuttal helped clarify and address the remaining concerns. Please let us know if we can further clarify anything.
We would like to, once again, thank the reviewer for relating to our rebuttal and providing additional comments. Regarding the potential bias, we would like to point to the fact that the same synthetic dataset (synthetic-general) is effective at debiasing across several bias categories in different datasets, which suggests it doesn’t introduce new biases while mitigating others.

However, in light of the concerns around the potential toxicity of our generations, we performed additional analysis using the unbiased [ToxicBert]([https://huggingface.co/unitary/toxic-bert) model (i.e., Detoxicty) as well as a non-neural model for sentiment analysis - [VaderSentiment]([https://github.com/cjhutto/vaderSentiment). We present these results in the table below:

| Evaluation Model |    Synthetic Data|     Mean Score (SD) | Top 95% percentile |
| :---- | :----  |-----------------:|----------------------: |
| ToxicBERT *(toxicity)*  | General   | 0.0086 ± 0.0539 | 0.0216 |
|                  | Gender   |   0.0312 ± 0.1147 | 0.1480 |
|                  |  Race  | 0.0177 ± 0.0632 | 0.2943 |
|                  | Religion | 0.0123 ± 0.0614 | 0.0314|
|VaderSentiment  | General | 0.0081 ± 0.0404 | 0.0000 |
| *(negative component)*                        | Gender   | 0.0318 ± 0.0862 | 0.2421 |
|                          | Race | 0.0039 ± 0.0277 | 0.0000 |
|                          | Religion | 0.0049 ± 0.0350 | 0.0000 |

The Toxicity and VaderSentiment scores take the value from 0.0 to 1.0 on a continuous scale with values close to 0.0 representing very little toxicity and lack of negative sentiment respectively. We can see that the scores for our synthetic data are very low, way below 0.50 general threshold for considering a text toxic. This is even for the top 95% percentile of highest-scoring generations. This suggests that our synthetic generations are not toxic. We will be happy to add this analysis to the paper.

*Methodology notes:* VaderSentiment produces individual positive, negative, and neutral sentiment component scores and a compound score. We report just the values for the negative sentiment component as these are the most relevant here. We also analyzed other dimensions provided by ToxicBert such as Identity Attack, Insult, and Threat. The scores were the highest for Toxicity (but still very low in overall terms) so we report these. 

We hope this additional analysis helps to alleviate concerns about toxicity.
Thank you for your reply, I have no further questions.  I agree with Reviewer n66S that  the generated sentences might still be biased or toxic. I will remain my scores.