PEFTDebias : Capturing debiasing information using PEFTs
Authors: Sumit Agarwal
Aditya Srikanth Veerubhotla
Srijan Bansal
Venue: EMNLP 2023 Main (2023)
Keywords: paramter-efficient finetuning (PEFT)
bias mitigation
debias
bias
gender
group
language model
debiasing
LoRA debias
prompt debias
adapter debias
TL;DR: PEFTDebias is a novel approach for parameter efficient debiasing of language models, consisting of upstream and downstream phases, and various PEFTs for efficient bias across different axes.

Abstract: The increasing use of foundation models highlights the urgent need to address and eliminate implicit biases present in them that arise during pretraining. In this paper, we introduce PEFTDebias, a novel approach that employs parameter-efficient fine-tuning (PEFT) to mitigate the biases within foundation models. PEFTDebias consists of two main phases: an upstream phase for acquiring debiasing parameters along a specific bias axis, and a downstream phase where these parameters are incorporated into the model and frozen during the fine-tuning process. By evaluating on four datasets across two bias axes namely gender and race, we find that downstream biases can be effectively reduced with PEFTs. In addition, we show that these parameters possess axis-specific debiasing characteristics, enabling their effective transferability in mitigating biases in various downstream tasks.

PEFTDebias: Capturing debiasing information using PEFTs
Aditya Srikanth Veerubhotla' Srijan Bansal Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA (sumita, adityasv, jeandree@andrew cmu edu
Sumit Agarwal
Abstract
Pronpe
Adapier
U Upstream
LoRA
Foundation Modal
PEFT
The increasing use of foundation models high- lights the urgent need to address and eliminate im- plicit biases present in them that arise during pre- training. In this paper, we introduce PEFTDebias, novel approach that employs parameter-efficient fine-tuning (PEFT) mitigate the biases within foundation models. PEFTDebias consists of two main phases: an upstream phase for acquiring de- biasing parameters along specific bias axis, and a downstream phase where these parameters are incorporated into the model and frozen during the fine-tuning process. By evaluating on four datasets across two bias axes namely gender and race, we find that downstream biases can be effectively re- duced with PEFTs. In addition, show that these parameters possess axis-specific debiasing charac- teristics, enabling their effective transferability in mitigating biases in various downstream tasks. To ensure reproducibility, we release the code to do our experiments!.
SFT
Corpora
Pre Praiingnn (MLM)
CDA
PEFT (Axis)
Evaluation Templates
Measure Intrinsic Bias
PSFT Thandbr
Task Corpara (Axis)
Tuning
Fine Tunal Modei A
PEFT
Evaluntion Templates
Mensure (FPRD, TPR pup), Perf
Downstream
The figure illustrates our proposed PEFT- Debias method to debias the fine-tumed model, which consists of two main phases upstream phase where debiasing parameters are acquired through CDA-based PEFT training axis corpora, evaluated using intrinsic metrics, downstream phase, where the debiased PEFT injected into trainable model and kept frozen during the fine-tuning process on a task corpora. Bias is mea- sured using extrinsic metries along same axis.
Figure I:
1 Introduction
In recent years, it has become evident that founda- tion models such BERT or GPT-3 (Devlin et al., 2019; Brown et al., 2020) are susceptible to a range of stereotypical societal biases (Jentzsch and Tu- ran, 2022) such as sexism (gender) (Kurita et al., 2019) and racism (race) (Ahn and Oh, 2021), that are present in the training data. Such bias axes can lead to unfair discriminatory outcomes (Web- et al., 2021; Barikeri al., 2021) in various socio-technical scenarios.
higher likelihood of subsequent harmful effects, concept known bias transfer hypothesis (Boluk- basi et al., 2016; Caliskan al., 2017). However, most approaches for bias mitigation are primarily applied during fine-tuning to reduce bias in specific downstream tasks or datasets (Park et al., 2018; Zhang et al.. 2018). It involves incorporating aux- iliary training objectives (Jin et al., 2021), anno- tation of bias attributes (Liang et al., 2020) and task-specific fairness metrics (Zhang al., 2020), which poses a challenge for the expanding commu- nity of fine-tuning language models.
Recent research (Ladhak et al., 2023) suggests that biases acquired during pre-training can propa- gate to downstream models, resulting in superficial text dependencies and potential implicit bias, and a
"Equal contribution, names ordered randomly. http://www.....uuiiaag////////te
Previous studies have attempted to address this

---

issue by first debiasing the model and then fine- tuning for a specific task. This process referred to as upstream debiasing by Jin et al. (2021), and entails fine-tuning the model on upstream tasks while incorporating bias-attribute annotations for debiasinn. Subsequently, the model is fine-tuned for the target downsiream task. Nevertheless, this approach possesses certain limitations: (i) it re- quires annotated bias attributes for the upstream task as well as supervised data for both tasks and (ii) there is no guarantee that the model will exhibit reduced bias in the downstream task (Steed et al., 2022). This uncertainty arises due to the fact that modifying all parameters of the debiased upstream model might result in the loss of debiased represen- tations. This phenomenon is commonly referred to fairness forgetting (Lauscher et al., 2021). Inspired by the promising outcomes of PEFT methods, which effectively capture debias infor- mation and yield competitive results compared to full model-tuning (Kumar et al., 2023; Lauscher et al., 2021), we hypothesize that employing PEFTs for debiasing on an upstream bias axis could be a viable approach to mitigate bias in founda- tion model for any downstream task on the same bias axis. To address this, we present a novel method called PEFTDebias. This approach uti- lizes PEFTs to capture debiasing information by training the model on axis-specific data during the upstream stage. Subsequently, in the downstream task, the model is fine-tuned while keeping the PEFTs frozen, thereby preserving the upstream debiasing information along that axis. Our contri- bution can be summarized as:
2019), dropout regularization (Webster al., 2020), null-space projection (Ravfogel al., 2020), adver- sarial training (Liu et al., 2020), contrastive learn- ing (He et al., 2022). However, these techniques necessitate expensive additional annotation, such as the inclusion of protected attributes, along with the task data. Conversely, (Jin et al., 2021) demon- strate debiasing using only task data, showing its potential for improving generalization. In contrast, (Steed al., 2022) indicate that debiasing lan- guage model (LM) prior to fine-tuning does not guarantee unbiasedness in the resulting fine-tuned model. Jin et al. (2021) investigate the transfer- ability of debiasing techniques. They begin by applying bias mitigation to pre-trained model through fine-tuning and subsequently employ it for downstream fine-tuning.
Lauscher et al. (2021); Kumar et al. (2023) show that PEFT methods like Adapters (Houlsby et al., 2019), can be used to debias language models (LMs) while keeping the LM backbone frozen. Hauzenberger et al. (2023) present a method to do debiasining by identifying sparse subnetworks that correspond to different bias axes, which can subsequently be composed. A notable advantage of these approaches the reduced computational cost and environmental impact associated with debias- ing LMs (Hessenthaler et al., 2022). Additionally, it holds the potential for preventing catastrophic forgetting of pre-trained knowledge caused by fine- tuning (Kirkpatrick et al., 2017). However, these techniques are typically applied during the down- stream phase and possess the limitations discussed earlier.
We explore the efficacy of training PEFT pa- rameters along a specific bias axis by utilizing axis-based data to transfer bias information to downstream tasks aligned with that axis.
Bias Factors and Datasets
We validate our hypothesis by conducting valida- tion on two widely recognized factors of social bias: gender stereotyping and racial identifiers. To ad- dress occupation based gender stereotypes, we uti- lize the BiasBios dataset (De-Arteaga et al., 2019). For the bias related to race, we address the issue of elevated occurrences of false positive outcomes in hate speech predictions using GHC (Kennedy et al., 2018). To show our generalizibility of capturing debiasing information along specific axis using PEFTs, we show transfer to datasets MNLI (multi genre NLI) (Williams et al., 2018) and LHC (large hate corpus) (Toraman et al., 2022) along gender and race axis respectively.
We evaluate the effectiveness of various PEFT methods in mitigating social biases to deter- mine whether certain PEFT techniques are more efficient than others.
We examine the transfer capabilities of PEFTs across different datasets to mitigate social bi- ases along specific axes.
2 Related Work
Several debiasing methods have been proposed in conjunction with the downstream task, including counterfactual data augmentation (Zmigrod et al.,
In order to assess the effectiveness of our debi-

---

asing techniques in mitigating gender and racial biases, we utilize two intrinsic bias benchmarks, namely CrowS-Pairs (Nangia et al., 2020) and StereoSet (Nadeem et al., 2021), during the ini- tial phase of our evaluation, referred to as the upstream stage. StercoSet evaluates a language model's stereotypical associations by employing fill-in-the-blee blank problems with intra-sentence ex- amples across different bias categories. CrowS- Pairs an intra-sentence dataset of minimal pairs that compares the language model's masked token probabilities of sentences with disadvantaged or advantaged races fulfilling or violating stereotypes.
the viability of multiple PEITTs, including Adapters (Pfeiffer al., 2021), Prompt Tuning (Lester al., 2021), LoRA (Hu et al., 2021), and Sparse Fine- tuning (Ansell et al., 2022) (refer A.2).
4.1 Upstream Phase
Counterfactual Data Augmentation (CDA) (Zmi- grod et al., 2019) is a data-based debiasing tech- nique that swaps attribute words pertaining to a bias (e.g, he/she for binary gender). Parameter efficient debiasing with Adapters (Lauscher et al., 2021) has demonstrated the effectiveness of using CDA to capture debiasing information while minimiz- ing the number of parameters. Consequently, our study aims to explore the application of CDA using PEFT methods for obtaining debiasing parameters. Specifically, we utilize a PEFT to perform CDA on axis-specific data. We extract attribute words from particular axis and apply them through CDA to obtain debiasing PEFT parameters. Our hypoth- esis posits that these parameters will proficiently capture task-agnostic debiasing information that is specific to the designated axis.
In the subsequent downstream stage, evaluate the performance gap of PEFTs across different pro- tected attributes within the specific domain using extrinsic bias metries. To measure gender bias, we adopt the method proposed by De-Arteaga et al. (2019) to calculate the gender gap in the True Posi- tive Rate (TPR) for each occupation (TPR-GAP). To assess racial bias, we compute the False Pos- itive Rate Difference (FPRD) by comparing the FPR of examples mentioning protected racial at- tributes to the overall FPR. We calculate FPRD for both the in-domain data and the Identity Phrase Templates Test Sets (IPTTS) (Zhang et al., 2020), which consist of 77k instances. These instances comprise hate and non-hate sentences that men- tion 25 racial identifiers and are generated using predefined templates. To measure transferability, we evaluate MNLI using FN (fraction of neutrals) in Bias-NLI (Dev al., 2019), NLI dataset to measure gender bias, and LHC using IPTTS.
4.2 Downstream Phase
To enable the transferability of debiasing PEFT pa- rameters across datasets, we propose learning debi- asing parameters during the upstream phase and in- jecting them into a trainable language model while keeping PEFT parameters frozen during down- stream task fine-tuning, Our hypothesis is that this set of frozen parameters will retain the upstream debiasing effect and safeguard the model against acquiring biases during task finetuning. Conse- quently, effectively mitigates biases along the specific axis in the finetuned model.
4 Methodology
Kumar et al. (2023) demonstrates that incorpo- rating adapters in debiasing during the finetun- ing process helps. However, transferring adapters between different datasets/tasks is not feasible due to the need to learn data-specific modules. While Lauscher et al. (2021) indicate that learning adapters in the upstream phase contributes to better results during downstream fine-tuning. We propose a novel approach called PEFTDebias which com- bines elements from both aforementioned meth- ods. It consists of two main phases: the upstream phase, responsible for selecting debiasing param- eters through PEFTss and the downstream phase, which employs the debiased PEFTs for task debi- asing during tieettnnng, as illustrated in Figure I and outlined in pseudo-code A.3. We investigate
Results
Our experimental setup is described A.4. We present three sets of results: evaluation of the upstream and downstream phases on the same datasets, and the transferability to other datasets.
5.1 Upstream Phase
In Table 1, we present the results of our experi- ments in the upstream setting. The results clearly indicate that the utilization of PEFTs with CDA not only enhances the performance of LM, but also diminishes intrinsic bias. Remarkably, both the Prompt Tuning and Adapter techniques demon- strate substantial debiasing effectiveness while ei- ther preserving or even enhancing the LM score

---

when compared to other techniques. For BiasBios, Prompt Tuning shows the highest performance in bias intrinsic scores CrowS and StereoSet.
PEET
BiasBios (Gender)
GIC (Race)
ACC1
TPR
[PRD
IPRD
FT
81.29
13.05
68.76
1.01
0.01
Full-Debias
81.27
12.86
62.48
1,07
0,08
PEFT
SS LM 1
SS Score J.
CrowS
Adapter
81.28
13.22
67.39
0.78
0,02
BiasBios
Eval: Gender
Prompt
81.10
11.98
67.15
0.54
0,01
LoRa
81.28
13.67
66.91
0.73
0.12
BERT
85.68
60.03
57.25
SFT
81.34
12.04
65.06
0.59
0.25
Full-Dehias
85.74
60.28
54.96
PEFT
BiasBios
>> MNLI
GHC >) LHC
+ Adapter
86.45
57.1
53.82
Prompt
85,54
56.64
51.91
ACC1
FN
F1 7
FPRD.
FERDIPTTS
LoRa
86.21
58.85 85
54.20
FT
80.52
0.02
91.06
0.34
0.03
SFT
86.22
$7.9
55.34
Full-Debias
80.13
0.02
91.63
0.32
0.00
GHC
Eval: Race
Adapter
80.11
0.02
91.47
0.33
0.01
Prompt
80.01
0.21
91.2
0.34
0.00
BERT
83.88
57.06
62.33
LoRa
80.3
0.02
91.18
0.32
0.01
Full-Debias
84.01
57.03
45.63
SFT
80.25
0.01
91.63
0.31
0.01
Adapter 85.88
58.56
55.15
Prompt 85.73
58.78
52.62
Table 2: Task performance and extrinsic bias matrix re- sults in the downstream setting on the BiasBios (gender) and GHC (race) datasets; same those used during the upstream phase (above) and transfer setting on different MNLI (gender) and LHC (race) datasets (below)
LoRa 84.89
58.20
56.12
SFT
85.42
58.91
54.76
Table Results in the Upstream setting using BERT as the LM and CDA for performing Debiasing.
parameters during the finetuning process for a cor- responding task in the same axis on MNLI and LHC. By comparing these results with the ones reported in Table 2, observe that the perfor- mance of the transferred debiasing parameters is comparable that of full finetuning (FT). While parameters learned from the same task data ex- hibit the least bias, as indicated by the FPRD and FPRDIPTTS metrics, Table 2 demonstrates that com- parable performance can still be achieved through transfer. Notably, the SFT and Prompt Tuning out- perform full finetuning on in-domain FPRD metrics when it comes transfer which also aligns with our findings from previous experiments. In case of MNLI, the performance remains similar to that of full finetuning while Prompt Tuning showing impressive performance for bias scores calculated using BiasNLI. This indicates that agno-igi axis-based patch generated by PEFTs work ef- fectively to debias along the same axis across different datasets.
5.2 Downstream Phase
The results of the downstream experiments are pre- sented in Table 2 where the dataset used in the upstream phase is same the one in the down- stream phase, demonstrating that the PEFTs attain comparable task performance to the BERT base- line (within a 5% margin) with a significant im- provement in extrinsic bias metric. This observa- tion suggests that is possible to achieve efficient debiasing without significant performance loss. Among the PEFTs, Prompt Tuning stands out for its superior ability to reduce bias. This find- ing implies that Prompt Tuning effectively debiases the model in the upstream phase while maintaining its task performance, possibly due minimal mod- ifications inside the language model (Ding et al., 2022) during forward pass as compared to other PEFTs. Additionally, both BiasBios and GHC ex- hibit positive correlation between upstream debi- asing performance and downstream bias reduction. This correlation indicates that upstream debiasing can effectively transfer downstream tasks using PEFTs, facilitating bias mitigation across similar axes. We also study in detail the reduction in bias BiasBios dataset A.5
Conclusion & Future Work
This research paper introduces PEFTDebias, a novel debiasing approach that utilizes PEFTs to mitigate the biases. PEFTDebias involves two phases: an upstream phase for learning debiasing PEFTSs along specific bias axes, and downstream phase where these PEFT's are incorporated into the model and kept frozen while fine-tuning. Experi- mental results highlight the effectiveness of Prompt
5.3 PEFT Transfer
To evaluate the task-agnostic nature of the learned upstream debiasing parameters along a specific axis, we conduct experiments where we apply these

---

Tuning for downstream debiasing and the transfer- ability of axis-specific debiasing parameters in mit- igating biases across different tasks. Future work includes extending our technique for generative models and tasks, as well as exploring the com- position of multiple bias axes (Jin et al., 2021) to address various biases in datasets.
from making definitive claims about the debiasing techniques outlined in this paper or applying them directly in real-world settings.
Acknowledgement
We thank Professors Emma Strubell and Maarten Sap for their valuable guidance and feedback on this work.
7 Limitation
Our research specifically targeted the debiasing of BERT, widely used language model, and did not encompass other foundational language mod- els such as GPT-3 limiting its scope to the spe- cific context of BERT and its associated biases. We demonstrated the effectiveness of our debias- ing techniques on downstream classification tasks. However, it is important to note that these findings may not directly translate to generative language models, as they approach every task as a genera- tion problem. To extend the applicability of our approaches to the broader landscape of all founda- tional language models, further analysis and inves- tigation would be necessary. We focus our study on mitigating the biases within the dataset, and do not focus on the biases in the annotation of the task labels.
References
Jaimeen Ahn and Alice Oh. 2021. Mitigating language- dependent ethnic bias in BERT. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pages 533-549, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vulić. 2022. Composable sparse fine-tuning for cross- lingual transfer. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Voluee 1: Long Papers), pages 1778-1796, Dublin, Ireland. Association for Computational Lin- guistics.
Soumya Barikeri, Anne Lauscher, Ivan Vulić, and Goran Glavaš. 2021. RedditBias: A real- world resource for bias evaluation and debiasing of conversational lan- guage models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers), pages 1941-1955, Online. Association for Computational Linguistics.
8 Ethical Considerations
In this research, employed binary gender defi- nition while examining gender bias in pre-trained language models. However, we acknowledge that gender is non-binary and recognize the importance using a more flexible definition in future studies on gender bias drawing inspiration from previous research (Dinan et al., 2020). Likewise, our inves- tigation of racial bias is limited to specific set of biased attribute words, representing narrow defini- tion. It is important to note that we did not explore the potential reduction in harm through the imple- mentation of our debiasing techniques in real-world scenarios. Furthermore, we want to emphasize that all the intrinsic bias benchmarks used in this study possess only positive predictive power. This means that they can identify biased models but cannot con- firm model as unbiased. For instance, a stereo- type score of 50% StereoSet or CrowS-Pain does not necessarily indicate an unbiased model. The extrinsic measures also rely on few words or templates and cannot comprehensively capture all the stereotypical variations used by humans, Due to these considerations, we urge readers to refrain
Tolga Bolukbasi, Kai- Kai-Wei Chang. James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS' 16, page 4356 23644436, Red Hook, NY, USA. Curran Associates Inc.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 13:7-1711
Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semanties derived automatically from language corpora contain human-like biases. Science, 386(6333)
Maria Dr-Arteaga, Alexey Romanov, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios. In Proceedings of the Conference on Fairness, Account- ability, and Transparency. ACM.

---

Sunipa Dev, Tao Li, Jeff Phillips, and Vivek Srikumar. 2019. On measuring and mitigating biased inferences of word embeddings.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang. Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint 188.0088..
Sophie Jentzsch and Cigdem Turan. 2022. Gender bias in BERT measuring and amalysing biases through sentiment rating in a realistic downstream classifica- tion task. In Proceedings of the 4th Workshop on Gen- der Bias in Natural Language Processing (GeBNLP). pages 184 199, Seattle, Washington. Association for Computational Linguistics.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 1 (Long and Short Pupers), J. pages 4171-4186, -4166, Minneapolis. Minnesota. Association for Computational Linguistics.
Xisen Jin, Francesco Barbieri, Brendan Kennedy, Aida Mostafazadeh Davani, Leonardo Neves, and Xiang Ren. 2021. On transferability of bias mitigation ef- fects in language model fine- finettuing. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 3770-3783, Online. Association for Computational Linguistics.
Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multi- dimensional gender bias classification. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 314-331, Online. Association for Computational Lin- guistics.
Brendan Kennedy, Mohammad Atari, Aida M Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo- Wiightmmn, and Elaine Gonzalez. 2018. Introducing the gab hate corpus: Defining and applying hate-based rhetoric to social media posts at scale.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zong- han Yang. Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, Jing Yi. Weilin Zhao, Xiaozhi Wang. Zhiyuan Liu, Hai-Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. 2022. Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness. Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Grabska-Barwinska. et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, (14(13):3521-3)
Lukas Hauzenberger. Shahed Masoudian, Deepak Kumar, Markus Schedl, and Navid Rekabsaz. 2023. Modular and on-demand bias mitigation with attribute-removal subnetworks. In Findings of the As- sociation for Computational Linguisties: ACL 2023, pages 6192-6214, 44 Toronto, Canada. Association for Computational Linguistics.
Deepak Kumar, Oleg Lesota, George Zerveas, Daniel Cohen, Carsten Fickhoff, Markus Schedl, and Navid Rekabsaz. 2023. crrameter-efficien modularised bias mitigation via AdapterFusion. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2738-275 1, Dubrovnik, Croatia. Association for Computational Linguistics.
Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Danqi Chen. 2022. Mabel: Attenuating gender bias using textual entailment data. arXiv preprint arXiv:2210.1499:
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in con- textualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Lan- guage Processing, pages 166-172, Florence, Italy. Association for Computational Linguisties.
Marius Hessenthaler, Emma Strubell, Dirk Hovy, and Anne Lauscher. 2022. Bridging fairness and environ- mental sustainability in natural language processing. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 7817-7836, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, and Tat- sunori Hashimoto. 2023. When do pre-training bi- ases propagate to downstream tasks? a case study in text summarization. In Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, pages 3206- 3219, Dubrovnik, Croatia. Association for Computa- tional Linguistics.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In In- lernational Conference on Machine Learning, pages 2790-2799. PMLR.
Anne Lauscher, Tobias Lueken, and Goran Glavaš. 2021. Sustainable modular debiasing of language models.

---

In Findings of the Association for Computational Linguistics: EMNLP 2021. pages 4782-4799 Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
the Association for Computational Linguistics, pages 7237-7256, Online. Association for Computational Linguisties.
Ryan Steed, Swetasudha Panda, Ari Kobren, and Michael Wick. 2022. Upstream Mitigation Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 3524-3542, Dublin, Ireland. Association for Computational Linguistics.
Brian Lester, Rami AI-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficier prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045-3059, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Moreney. 2020. Towards dehiasing sentence representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 5502-5515, Online, Association for Computational Linguistics.
Cagri Toraman, Furkan Şahinuç. and Eyup Yilmaz. 2022. Large-scale hate speech detection with cross- domain transfer. In Proceedings of the Thirteenth Language Resources and Evaluation Conference. pages 2215-2225, Marseille, France. European Lan- guage Resources Association.
Haochen Liu, Wentao Wang. Yiqi Wang, Hui Liu, Zi- tao Liu, and Jiliang Tang. 2020. Mitigating gender bias for neural dialogue generation with adversarial learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 893-903, Online. Association for Computational Linguistics.
Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler. Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2021. Measuring and reducing gendered correlations in pre-trained models.
Kellie Webster, Xuezhi Wang. Ian Tenney, Alex Beu- tel. Emily Pitler, Ellie Pavlick, Jilin Chen, and Slav Petrov. 2020. Measuring and reducing gen- dered correlations in pre-trained models. ArXiv, abs/2010.06032.
Moin Nadeem, Anna Bethke, and Siva Reddy, 2021. StereoSet: Measuring stereotypical bias in pretrained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pages 5356-5371, Online. Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122. Association for Computational Linguistics.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020, CrowS-pairs: A chal- lenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Lnguage Processing (EMNLP), pages 1953 1967,197, Online. As- sociation for Computational Linguistics.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES '18 page 335-340 New York, NY, USA. Association for Computing Machinery.
Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Re- ducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2799-2804, Brussels, Belgium. Association for Com- putational Linguistics.
Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Con- ghui Zhu, and Tiejun Zhao. 2020. Demographics should not be the reason of toxicity: Mitigating discrimination in text classifications with instance weighting. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 4134 4145, Online. Association for Computa- tional Linguistics.
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Con- ference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487-503, Online. Association for Computational Lin- guistics.
Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmenta- tion for mitigating gender stereotypes languages with rich morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651 1666-1 Florence, Italy, Asso- ciation for Computational Linguistics.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoay Goldberg, 2020. Null it out: Guard- ing protected attributes by iterative nullspace projec- tion. In Proceedings of the 58th Annual Meeting of

---

A Appendix
task while keeping the PEFT frozen to obtain Opss The final debiased task-specifie model is the union of the axis-specific PEFT and the foundation model (fFM U ΦPEFT)
A.1 Bias Axes & Attribute Words
We describe the bias axes and attribute words that we will use in our studies. We mention two differ- ent biases, gender and race. Hereby, we present list of some attribute word examples as well along with the biases.
A.4 Experimental Setup
We used pre-trained BERT (Devlin et al., 2018) as the starting point for all of our models. We also applied text normalization to GHC datasets to remove URLs and user mentions using tweet based processing 2. For the upstream experiments, we trained our models with MLM and CDA on the BiasBios dataset and the other datasets using learning rate of le and batch size of 128 and 32 respectively. We ran MLM for 10,000 steps and evaluated the models every 1,000 steps. We selected the models with the lowest loss for our experiments. For the downstream experiments, we used batch size of 32 and trained our models for 10 epochs. We ensured that all PEFTs have similar number of parameters, being 1% of the base LM, to keep them comparable. For the downstream ex- periments, we used a batch size of 32 and trained our models for 10 epochs. We chose the models with the best task metrics for analysis. For GHC and Stormfront datasets, which had few hateful ex- amples compared to non-hateful ones, we weighted the loss of hateful examples by factor of 10 for GHC and 6.7 for Stormfront, based on their propor- tions in the data. We compared methods with two baselines: BERT in the pre-trained setting and BERT in the fine-tuned setting (Full-Debias). Our implementation is based on the AdapterHub 3,
Gender (actor, actress), (boy, girl), (brother, sister),
(he, she)
Race (black, caucasian, asian), (african, caucasian, asian), (black, white, asian)
A.2 Paramter Efficient Fine-Tunina (PEFT)
We explore the use of multiple PEFTs, Adapters: (Pfeiffer et al., 2021) which are task-specific mod- ules inserted between transformer layers, Prompt Tuming: (Lester et al., 2021) which involves incor- porating task vectors (prompts) into the in- put sequence, LoRA: (Hu et al., 2021) which inte- grates trainable low-rank matrices into transformer layers in order to approximate weight updates, and Sparse Fine Tuning: (Ansell et al., 2022) builds upon the Lottery Ticket Hypothesis (LTH) to select a sparse sub-network based on the parameters that undergo the most sigmificant changes.
A.3 Algorithm
Algorithm PEFTDebias training algorithm
Require: D. (ax) unlabelled
Require: 1) = =(x,,))) ~ P(X, Y) labelled
Initialize AFM
Initialize ÓPEFT
A.5 Reduction in bias
1t Upstream stage */ OPEFT* +- Debios(Orme OPEET, Du. 4)
We conducted a comparison of the TPR-GAP per- formance of CDA debiasing techniques using FT and Prompt Tuning on the BiasBios dataset (see Figure/ 2, specifically focusing on occupations cat- egorized as male and female. Our findings indicate that debiasing with Prompt Tuning yields better sults compared to FT, evidenced by a decrease in the TPR for gender-dominant professions. We ob- served that certain female-dominated professions such as dietitian and interior designer exhibit re- duced correlation with the female gender, while male-dominated professions like surgeon and co- median also demonstrate a decrease in correlation with the male gender. Although we did not ob- serve significant changes in the gap for professions
1" Downstream stage
7Pm FT " Dr) Di)
return FrM U øPErT"
Our algorithm for debiasing is described in 1. Our method requires an unlabeled in-domain cor- pus D, for upstream debasing and a labeled corpus Dj for task-specific fine-tuning in the downstream phase. We use pretrained foundation model FFM- and set of PEFT parameters ΦPEFT which will be used for debiasing the model. In the upstream stage, the backbone model is kept frozen and do- main and axis-specific PEFT parameters d/err" for the axis A are obtained. These are then used to finetune the foundation model on the downstream
llink script Phttss//uddaperrhuubm//

---

like rapper and psychologist, we encountered an issue of over-correction, resulting in reversed gap for poet and accountant. This discrepancy can be attributed to the limited number of examples avail- able for these particular professions.
FT Prompt
TPR GAP
dialitian
designer
physician
yaga
dhiropracio
comedian
surgeon
Figure 2: Comparing the TPR-GAP performance of CDA debiasing using FT and Prompt Tuning on the Biasbios dataset across different occupations.


Reviews:
[{"review_id": "NAu0kCuJq4", "reviewer": "Reviewer_LNGT", "summary": "", "questions": "-", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for \tproviding your valuable feedback. We would like to provide some clarifications for our approach.\n\n> Comprehending the operation of the proposed method is challenging, even though the paper is concise.\n \nIn the final version of our draft, we will add a detailed pseudocode for our approach in the appendix, and review the wording of our method to make it more clear to understand.\n\n> Additionally, despite the inclusion of \"Parameter efficient\" in the name of the proposed method, there appears to be a lack of performance evaluation specifically addressing its efficiency. To enhance the comprehension of the proposed method's effectiveness, it would be beneficial to incorporate an analysis that delves into the computational costs associated with the approach.\n\n1. Through our experimentation we find that PEFTs help capture debiasing information along a particular axis. The efficiency of our approach lies in the limited number of parameters (1% of the total parameters) utilized in the Upstream debiasing stage, which is a one-off experiment within a specific domain. \n2. The downstream phase operates akin to full fine-tuning, with the PEFT parameters being frozen. This approach results in comparable computational time (within 2%) during the downstream phase on BiasBios, the largest dataset for our experiments. Our observations indicate that training times are similar. Additional details regarding computational analysis will be included in the appendix of the final camera-ready draft.\n3. By demonstrating transfers to various datasets, we also indicate that these debiasing parameters could be efficiently applied across diverse datasets without needing to retrain the parameters. This not only saves a significant amount of time but also benefits when applying them to multiple datasets simultaneously.\n\n> Would be hard pressed to reproduce the results. The contribution depends on data that are simply not available outside the author's institution or consortium; not enough details are provided.\n\nTo ensure the reproducibility of our results, we will share the repository that was used to generate the results presented in the final camera-ready version of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method not only improves the performance of the language model but also effectively reduces inherent bias. This indicates that the method possesses the capability to enhance both debiasing and fundamental performance aspects.", "reasons_to_reject": "Comprehending the operation of the proposed method is challenging, even though the paper is concise. Utilizing pseudo-code could potentially offer clearer insight into the workings of both the upstream and downstream phases, as compared to relying solely on Figure 1.\n\nAdditionally, despite the inclusion of \"Parameter efficient\" in the name of the proposed method, there appears to be a lack of performance evaluation specifically addressing its efficiency. To enhance the comprehension of the proposed method's effectiveness, it would be beneficial to incorporate an analysis that delves into the computational costs associated with the approach.", "questions_to_authors": "-", "paper_topic_and_main_contributions": "The paper's authors suggest an effective approach for fine-tuning, specifically a parameter-efficient fine-tuning method, to mitigate bias issues in trained models. They make use of axis-based data to harness information relevant to downstream tasks that align with these axes. This approach consists of two primary phases. Initially, during the upstream phase, they employ a parameter-efficient fine-tuning method to identify debiasing parameters. Subsequently, in the downstream phase, the method employs these debiased parameters for fine-tuning. The authors demonstrate the effectiveness of their proposed algorithm through evaluations in various social bias tasks, highlighting its superiority.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "-", "Typos_Grammar_Style_and_Presentation_Improvements": "-", "Justification_for_Ethical_Concerns": "-"}, "extra_text": {}}, {"review_id": "BafN57sq12", "reviewer": "Reviewer_A5DV", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our work. To address your question, this works aims to present a preliminary study on evaluating the role of PEFT based techniques for debiasing LMs. While this study only shows results on Encoder-only models, our approach can be easily extended to other architectures such as Decoder-only and Encoder-Decoder architectures. We will leave these explorations to future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This work focus on an important and foundation task that try to mitigate biases in pre-trained models.\n2. Clear motivation and comprehensive experiments, the experiments provide a sufficient support for its claims.\n", "reasons_to_reject": "The work mainly conducts experiments on encoder-based model (i.e., Bert), but has no idea of generative models, like GPT.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work focus on debiasing pre-trained models, and proposes a two-stage approach called PEFTDebias, which training PEFT parameters along a specific bias axis while keeping base model frozen in the upstream task, then fine-tune the base model on downstream tasks while keeping the PEFT parameters frozen. Experimental results on several PEFT methods and different datasets show that the proposed methods effectively reduce the downstream biases and have transferability in various datasets.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MoQLbcGznL", "reviewer": "Reviewer_xV67", "summary": "", "questions": "No question.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating our work. We would like to provide clarifications for some of your concerns and questions.\n\n> The method is only evaluated on BERT and could have benefited from evaluation on more LLMs.\n\nThis is the same point as question 1 from Reviewer 2, and is addressed in our response there.\n\n> Missing References:\nModular and On-demand Bias Mitigation with Attribute-Removal Subnetworks Lukas Hauzenberger, Shahed Masoudian, Deepak Kumar, Markus Schedl, Navid Rekabsaz In Findings of the Association for Computational Linguistics: ACL (Findings of ACL), 2023\n\nThank you very much for pointing out this work, we will surely include this in the camera-ready draft of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Well written paper with a clear and to-the-point objective which perfectly fits to the scope of a short paper.\n- The introduced approach, while relatively simple, has practical benefits, particularly in terms of exploiting LLM debiasing techniques in downstream tasks.\n- Extensive experiments in terms of various PEFT approaches.", "reasons_to_reject": "- The method is only evaluated on BERT and could have benefited from evaluation on more LLMs.", "questions_to_authors": "No question.", "paper_topic_and_main_contributions": "The work explores the important topic of exploiting upstream debiasing in downstream tasks, and proposes leveraging parameter-efficient fine tuning (PEFT) methods with a specific downstream training strategy. The upstream bias mitigation is done using CDA, learned through various PEFT architectures appended to BERT, and evaluated on BiasBIOs, GHC, as well as MNLI and LHC. The results show that upstream bias mitigation when freezed during downstream training can transform its effectivity.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks\nLukas Hauzenberger, Shahed Masoudian, Deepak Kumar, Markus Schedl, Navid Rekabsaz\nIn Findings of the Association for Computational Linguistics: ACL (Findings of ACL), 2023"}}]

Author rebuttal:

Meta-review: In their work “PEFTDebias : Capturing debiasing information using PEFTs“, the authors propose to use parameter-efficient fine-tuning (PEFT) for two-stage debiasing of language models. They evaluate several PEFT variants on BERT and four datasets across two bias dimensions (gender, race), and demonstrate that their methodology is effective for reducing biases. 

Overall, the reviewers appreciate this work (soundness, excitement). Given that this is a shortpaper submission, the scope is generally feasible. However, they also note that the claims would be stronger with results on more/ other base models and that more emphasis on the efficiency aspect (as suggested by the title of this work) would enrich the discussion.
Decision: Accept-Main