Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks
Year: 2023

Abstract:
Societal biases are reflected in large pre-trained language models and their fine-tuned versions on downstream tasks. Common in-processing bias mitigation approaches, such as adversarial training and mutual information removal, introduce additional optimization criteria, and update the model to reach a new debiased state. However, in practice, end-users and practitioners might prefer to switch back to the original model, or apply debiasing only on a specific subset of protected attributes. To enable this, we propose a novel modular bias mitigation approach, consisting of stand-alone highly sparse debiasing subnetworks, where each debiasing module can be integrated into the core model on-demand at inference time. Our approach draws from the concept of diff pruning, and proposes a novel training regime adaptable to various representation disentanglement optimizations. We conduct experiments on three classification tasks with gender, race, and age as protected attributes. The results show that our modular approach, while maintaining task performance, improves (or at least remains onpar with) the effectiveness of bias mitigation in comparison with baseline finetuning. Particularly on a two-attribute dataset, our approach with separately learned debiasing subnetworks shows effective utilization of either or both the subnetworks for selective bias mitigation.

heading: 1 Introduction
text: A large body of research evidences the existence of societal biases and stereotypes in pre-trained language models (PLMs) (Zhao et al., 2019; Sheng et al., 2019; Rekabsaz et al., 2021), and their potential harms when used in down-stream tasks (Blodgett et al., 2020; De-Arteaga et al., 2019; Rekabsaz and Schedl, 2020; Stanovsky et al., 2019). Common in-processing approaches to bias mitigation update a model’s (typically all) parameters to satisfy specific attribute erasure criteria through optimization methods such as adversarial train-
ing (Elazar and Goldberg, 2018; Rekabsaz et al., 2021), and mutual information reduction (Colombo et al., 2021). These methods are shown to be effective in reducing the footprint of protected attributes (e.g., gender, race, etc.) in the resulting model.
However when using such debiasing models in practice and in specific use-cases, system designers or end-users might still prefer to instead use the original model, or a debiased variation in respect to a particular subset of protected attributes. This can be due to various reasons such as the nature of a given input, preference of an individual end-user, or fairness-utility trade-off considerations. For instance, while a bias-aware model should indeed be agnostic to genders when the input is about genderneutral occupations (such as nurse or CEO), certain topics like pregnancy may specifically require gender information for a correct model decision.1 Also as shown in previous studies (Zerveas et al., 2022; Biega et al., 2018; Rekabsaz et al., 2021), since improving fairness on specific tasks may come with the cost of performance degradation, it is necessary to provide on-demand control over whether to impose fairness/debiasing criteria. Using existing approaches, this would require maintaining and deploying multiple large parallel models for every protected attribute, resulting in overly complex and resource-heavy pipelines and increased latency.
To address this, we introduce a novel modular bias mitigation approach using sparse weightdifference networks. In our approach, the required changes in a model’s parameters for erasing a bias attribute are stored in a decoupled subnetwork, trained simultaneously by a debiasing and a sparsification objective. At inference time, adding each debiasing module to the core model results in delivering debiasing qualities to a model’s prediction in respect to the corresponding protected attribute. Our approach extends the principle idea
1See also the discussion in Krieg et al. (2023) about the need to separate bias-sensitive queries from “normal” ones.
6192
of diff pruning (Guo et al., 2021) introduced for parameter-efficient task training to bias mitigation, by viewing the objective of erasing a protected attribute as a stand-alone diff module. This module replaces fine-tuning by training only a small set of parameters added to the corresponding PLM’s parameters, to deliver bias mitigation of a specific protected attribute. We further propose a novel procedure to train such debiasing subnetworks separately, and to selectively add an arbitrary set of them to the core model at inference time (§3).
Our approach can be applied to any debiasing and representation disentanglement method, provided that its objective has separate learning signals for the task and each protected attribute. In comparison with adapter networks (Rebuffi et al., 2017; Houlsby et al., 2019), as shown by Guo et al. (2021) and also evidenced in our experiments, even more parameter-efficiency can be provided. Additionally, since our approach extends the base model with diff subnetworks, the resulting model is expected to perform (at least) as good as the fine-tuning variation, avoiding possible performance degradations. The modularity of our approach supports separating the process of developing debiasing solutions for a task from using them, such that stand-alone debiasing modules can be created and shared, and later be utilized in a final system on-demand.
We evaluate our approach on three bias mitigation tasks: occupation prediction from biographies involving gender (De-Arteaga et al., 2019), hate speech detection with dialect-based race as sensitive attribute (Founta et al., 2018); and mention prediction in tweets with two attributes of gender and age of the authors (Pardo et al., 2016). The last dataset particularly enables the study of combining independently trained debiasing modules (details in §4). The evaluation results show that our approach, due to learning the subnetworks specialized on the narrow functionality of debiasing an attribute, provides on par or better debiasing performance in comparison with strong baselines. Additionally, we observe that on the mention detection task, learning the debiasing subnetworks post-hoc to model training provides effective results when combining the two (independently trained) subnetworks at inference time. Remarkably, these results are achieved with debiasing subnetworks of maximum 1% size of the core model (BERT-Base), in some cases (e.g., for gender attribute) even only 0.01% (details in §5).

heading: 2 Related Work

heading: 2.1 Parameter-efficient and Modular Training
text: The concept of subnetworks is grounded in the lottery ticket hypothesis (Frankle and Carbin, 2019), stating that in deep neural networks, one can find many sparse subnetworks with a capacity comparable to that of the base network. Zhou et al. (2019) show that spotting such subnetworks through binary masks can in fact be seen as a form of model training. Zhao et al. (2020) further consider the magnitude of the parameters for pruning and filter out the ones lower than a specific threshold. Guo et al. (2021) use L0-regularization (Louizos et al., 2018) to reduce the number of active neurons. Hu et al. (2021) propose low-rank adaptation via rank decomposition matrices. These methods are extended by structural pruning approaches with additional architectural constraints, commonly led to a higher parameter-efficiency (Lagunas et al., 2021; Jaszczur et al., 2021).
Recent studies exploit subnetworks in the context of multi-task learning. Wortsman et al. (2020) isolate the learning signal of each task in a separate masking subnetwork, while Ben-Zaken et al. (2021) only finetune the bias weights. Xu et al. (2021) learn a subset of parameters via masking out the gradients of other parameters during backwardpass. Guo et al. (2021) suggest learning a sparse diff subnetwork for each task, whose parameter values are added to the corresponding parameters of the base network. An alternative architecture are adapter networks, first introduced in the context of multi-task learning (Rebuffi et al., 2017; Houlsby et al., 2019; Stickland and Murray, 2019), and are then extended in respect to their parameter efficiency (Rücklé et al., 2021; Han et al., 2021a), architectural variations (Mahabadi et al., 2021), and transfer learning capacity (Pfeiffer et al., 2021). As stated by (Sung et al., 2021), adapters (in their original form) slightly increase the inference cost of a model in comparison with the original form or pruning-based variations. Our proposed approach contributes to this line of research by extending this concept to on-demand bias mitigation.

heading: 2.2 Fairness & Bias Mitigation in NLP
text: Several studies explore methods for debiasing PLMs, such as linearly projecting embeddings into the space with minimum correlations to protected features (Ravfogel et al., 2020; Kaneko and Bollegala, 2021; Bolukbasi et al., 2016), utilizing a
STEP 0 (task) STEP 1 (protected attribute 1) STEP # (protected attribute #)…
distribution alignment loss (Guo et al., 2022), or penalizing bias by utilizing the encoded information in models (Schick et al., 2021). Adversarial training, originally introduced in domain adaptation (Ganin et al., 2016; Ganin and Lempitsky, 2015) is utilized in the context of fair representation learning (Xie et al., 2017; Madras et al., 2018), and later to erase demographic data from text classifiers (Elazar and Goldberg, 2018; Barrett et al., 2019; Han et al., 2021b; Wang et al., 2021), information retrieval models (Rekabsaz et al., 2021), and recommendation systems (Ganhör et al., 2022). Mutual information removal is an alternative approach, which minimizes the approximate upper bound of the common information between the task and protected attributes (Cheng et al., 2020; Colombo et al., 2021). Our work utilizes these optimizations to learn a novel modular on-demand bias mitigation approach.
Few recent studies explore parameter-efficient training for bias mitigation. Lauscher et al. (2021) approach debiasing PLMs using a stack of adapters. While shown effective in practice, the adapters in the higher levels inherently depend on the ones in the lower levels and cannot be learned nor utilized stand-alone. Zhang et al. (2021) approach bias mitigation with binary masks applied to the base network. More recently and in the context of removing spurious shortcuts in natural language understanding datasets, Meissner et al. (2022) train sparse binary masks on a finetuned model. Our work extends this line of research by encapsulating concept erasure modules in separate diff subnetworks for each protected attribute, and selectively
applying them to the base model at inference time.
3 Modular Debiasing with Diff Subnets
We start with defining the general approach to model bias mitigation. We consider an arbitrary PLM denoted by fθ with the set of parameters θ. The model learns the task τ using the loss function Lτ . The predictions of fθ might be sensitive to the variations in any of the k protected attributes of the set P = {ρ1, ..., ρk}. The bias mitigation objective is to make the model invariant to these variations while maintaining the effectiveness on the task, approached by defining the debiasing loss Lρi for the protected attribute ρi. We discuss two realizations of this loss function in Section 3.2. A debiased model in respect to attributes P is achieved by training on the following loss function:
Ltotal = Lτ + Lρ1 + ...+ Lρk (1) Using Ltotal, one can finetune all parameters of the model (Elazar and Goldberg, 2018; Rekabsaz et al., 2021), or utilize any parameter-efficient training such as adapters (Lauscher et al., 2021), diff pruning (Guo et al., 2021), or binary masks (Zhang et al., 2021). We use some of these methods as baselines, explained in the following sections. In the remainder of this section, we introduce our Modular Debiasing with Diff Subnetworks (MODDIFFY), explain its training and inference procedure, and describe two debiasing optimization methods.

heading: 3.1 MODDIFFY
text: We aim to encapsulate the debiasing functionality of the protected attribute ρi, provided by the signal
of the corresponding loss Lρi , into the sparse diff subnetwork characterized by the set of parameters δρi . Each parameter in δρi corresponds to a parameter in θ, such that adding δρi to the corresponding parameters in θ results in debiasing the model f .
To learn the model, let us consider a training data item in form of ⟨x, yτ , yρ1 , ..., , yρk⟩, where x is the input, yτ the task label, and yρi denotes the label of the corresponding protected attribute ρi. Figure 1 depicts the MODDIFFY approach with adversarial bias mitigation optimization. We first optimize for the task by encoding x into the vector zτ using f(.;θ). An arbitrary decoder network denoted as gτ uses zτ to predict the task output, and task loss is calculated using cross entropy (CE), as formulated below:
zτ = f(x;θ), L(0)total = CE(gτ (zτ ), yτ ) (2)
This loss updates θ as well as the parameters of gτ . While in our experiments we opt for fully finetuning fθ, in practice the model can be trained using any parameter-efficient method.
Next, we iterate over the protected attributes, and in each step learn the corresponding debiasing diff subnetwork. Specifically in step i dedicated to the protected attribute ρi, we learn the set of sparse additional parameters δρi such that by being added to θ, the resulting f(.;θ + δρi) model is debiased. Learning δρi is characterized by three loss functions. The first loss, Ltaskρi maintains the task performance when the task output is predicted from the altered encoder formulated below:
zρi = f(x;θ + δρi), Ltaskρi = CE(gτ (zρi), yτ ) (3)
The second is the debiasing loss Ldebiasρi , defined based on zρi and the label of the corresponding protected attribute yρi . We discuss adversarial bias removal and mutual information reduction as two possible realization of this representation disentanglement loss in Section 3.2. The third loss imposes the sparsity constraint, defined as the L0 regularization of δρi . The L0 loss aims to reduce the number of non-zero parameters, namely the term ∑|δρi |
j=1 1{δρi,j ̸= 0}, and is realized with the differentiable approximation proposed by Louizos et al. (2018). Following Guo et al. (2021), δρi is decomposed into the element-wise multiplication of two sets of parameters: δρi = mρi ⊙ wρi . The parameter set wρi stores the magnitude changes (diff values), and mρi learns to mask out the parameters.
mρi is characterized by the hard concrete distribution (Guo et al., 2021) with (logαρi , 1) parameters, and γ < 0 and ζ > 1 hyperparameters. This results in the following formulation:
LL0ρi = |δρi |∑
j=1
σ ( logαρi,j − log (− γ
ζ )
) (4)
where σ denotes the sigmoid function, and as in Guo et al. (2021) β is set to 1. The masking network can be simply reduced by assigning a mask to a group of parameters (such as a weight matrix, or a layer) instead of each individual one.
Putting all together, the objective of MODDIFFY at step i > 0 is defined as:
L(i)total = Ltaskρi + Ldebiasρi + LL0ρi (5)
We should note that while each subnetwork is trained independently from the others, the output embeddings zρi are passed to the same decoder network gτ . This design choice forces the learned embeddings to remain in the same embedding space, making it possible to add multiple (independently trained) subnetworks together to the core model. Another aspect is that the sparsity rate of each resulting subnetwork is not fixed and may vary due to the factors such as the hyperparameter setting, and the extent of encoded information content. To achieve a fixed sparsity rate, we further apply magnitude pruning by only keeping a fixed portion of the parameters with the largest absolute values, and finetuning the resulting parameters with the task and debiasing loss terms.
We conduct optimization with Ltotal under two training regimes. In the first one referred to as MODDIFFY-PAR, we repeat the mentioned training steps for each training batch. The second approach referred to as MODDIFFY-POST trains debiasing subnetworks post-hoc to training the core model. While MODDIFFY-PAR accommodates for a higher flexibility in optimization by training the networks in parallel, MODDIFFY-POST provides the practical benefits of learning various debiasing solutions for an already trained core model.
Finally at inference time, one can use the core model in its original form f(x;θ), or in combination with any of the debiasing subnetworks in the form of f(x;θ + δρi). Our approach also enables simultaneously debiasing all or any subset of the protected attributes by adding their corresponding subnetworks to the core network, for instance
as in f(x;θ + δρ1 + ... + δρk). We should note that, although these subnetworks reside in the same distributional space, they might affect each others functionality, depending on the inherent nature of the biases as well as the possible correlations between the protected attributes. We will examine this case in the next sections on a dataset with two protected attributes of gender and age.

heading: 3.2 Bias Mitigation Objectives
text: We explain two bias mitigation optimization methods used in MODDIFFY in the following.
Adversarial Bias Removal This method first defines a new classification head hρi for each protected attribute ρi. This head receives zρi as input, predicts the corresponding protected attribute, and calculates the cross entropy loss function Ldebiasρi . This loss needs to remove the information of ρi from f but train hρi such that it can effectively predict the protected attribute. This optimization forms the min-max game: Ldebiasρi = minf maxhρi CE(hρi(zρi), yρi). A common approach to turn this loss into a minimization problem is by using gradient resversal layer (GRL) (Ganin and Lempitsky, 2015) added before the debiasing heads. GRL multiplies the gradient of Ldebiasρi with a factor of −λi, and thereby simplifies the learning process to a standard gradient-based optimization, formulated below:
Ldebiasρi = minf,hρi CE(hρi(zρi), yρi)
Mutual Information (MI) Reduction This approach represents a family of algorithms that aims to remove the mutual information of the encoded embeddings of the task and protected attributes. Maximum Mean Discrepancy (MMD), first introduced in the context of domain adaptation (Gretton et al., 2012; Tzeng et al., 2014), offers a realization of MI reduction by minimizing the ability to separate the subsets belonging to two protected attributes. In particular, given a set of data points X split into two subsets XAρi and X B ρi according to the values of the (binary) protected attribute ρi, MMD minimizes the distance between the encoded embeddings of the subgroups with the following loss formulation:
Ldebiasρi = (∑
xA∈XAρi ϕ(f(xA)) |XAρi | − ∑ xB∈XBρi ϕ(f(xB)) |XBρi | )2
where ϕ is the feature map kernel defined as a linear combination of multiple Gaussian kernels.

heading: 4 Experiment Design
text: Datasets We evaluate our approach on three datasets on the tasks of occupation prediction, hate speech detection, and mention prediction; involving protected attributes of gender, age, and race dialect. The first dataset is BIOS (De-Arteaga et al., 2019) which contains short biographies used to predict a person’s job, where the name and any indication of the person’s gender (such as pronouns) in the biography are omitted. The BIOS dataset contains around 430K data points with 28 occupations, and two protected attribute classes (female/male). The second dataset is FDCL18 (Founta et al., 2018) for hate speech detection, containing a set of tweets each classified as hateful, abusive, spam, or none. As discussed in Xia et al. (2020), hate speech might have a strong correlation with dialect-based racial bias. Following previous studies (Sap et al., 2019; Ravfogel et al., 2020), we assign race dialect labels of African American and White American to FDCL18 using the probabilistic model developed by Blodgett et al. (2016), resulting in the dataset of approximately 62K data points. The third dataset is PAN16 (Rangel et al., 2016) containing a set of tweets accompanied with the labels of gender and age of the authors. The task’s objective is to predict mentions (whether another user is mentioned in a tweet). PAN16 provides approximately 200K data points with binary task classes (mention, no mention), as well as two gender labels and five age groups. Further details on the three datasets are provided in Appendix A.
Models and Baselines We conduct the experiments on the following models and baselines. FINETUNE: finetuning all parameters of the PLM on the task without any bias mitigation objective. FINETUNE-DEBIAS: the same model as FINETUNE but with the bias mitigation objective. ADAPTER: learning the task with an adapter network while the rest of the PLM’s parameters are kept frozen. ADAPTER-DEBIAS: the same model as ADAPTER but the adapter is trained on both task and bias mitigation objectives. DIFFPRUN: using a diff network to learn the task while PLM parameters remain unchanged. DIFFPRUN-DEBIAS: same as DIFFPRUN but the diff network is trained on both task and bias mitigation objectives. MODDIFFY-POST: our introduced post-hoc approach
where the debiasing modules are learned after training the model. We use FINETUNE as the base model for MODDIFFY-POST, whose parameters are kept frozen during post-hoc training. MODDIFFY-PAR: our introduced parallel approach where the debiasing modules are learned together with the base model finetuned on the task. Additionally, to provide a comprehensive view on bias mitigation methods, we evaluate the datasets on the INLP (Ravfogel et al., 2020) approach using the implementation and suggested hyperparameter setting. The evaluation results of the INLP method are separately reported in Table 8 in Appendix B. As the PLM encoder for all models, we using two versions of BERT (Devlin et al., 2019) with different sizes, namely BERT-Mini (Turc et al., 2019) and BERT-Base. This provides us a more comprehensive picture regarding the effect of encoder size and number of involved parameters on the bias mitigation methods.
All debiasing models are separately trained according to the adversarial bias mitigation and mutual information reduction methods. We particularly opt for a non-linear adversarial head with two fully connected layers and the tanh activation. Furthermore, to improve the capacity of adversarial learning, we initialize 5 instances of hρi and calculate the average of the loss for the backward pass. In MI reduction, in the case of protected attributes with more than two classes, we turn the multi-classes setting to multiple one-versus-rest splits. For the models with diff subnetworks, we conduct preliminary experiments to find proper thresholds for magnitude pruning that improves
sparsity as much as possible without sacrificing performance. For BERT-Base and BERT-Mini, we set the minimum sparsity threshold to 99% and 95% (maximum size of 1% and 5%), respectively. We note that these ratios are the lower bounds, and the L0 regularization may by itself reach a higher sparsity. The complete details of our hyperparameters setting and training procedure are explained in Appendix A. Our code and trained resources are available in https://github.com/CPJKU/ ModularizedDebiasing.
Evaluation Metrics We evaluate the performance of the classifiers on the core task using the accuracy metric. We evaluate bias mitigation based on the concept of fairness through blindness, namely by examining whether models are agnostic about the protected attributes. Concretely following previous works (Elazar and Goldberg, 2018; Barrett et al., 2019), we report the leakage of a protected attribute in terms of the performance of a strong probe (or attacker) network. To train the probe, we freeze the model’s parameters, and train a new classification head (two-layer feed-forward layer with a tanh activation) to predict the protected attribute from the z encoding vector. For each evaluation, we train an ensemble of 5 probes for 40 epochs with early stopping if validation loss does not increase over 5 epochs, and report the results of the best performing probe. We report the performance of the probe in terms of balanced/macro accuracy (average of per-class accuracy scores). Balanced accuracy has the benefit of better reflecting the performance of the methods when considering minority groups, particularly given the unbalanced
distributions over protected labels in the datasets. To account for possible variabilities, we repeat every experiment five times and report the mean and standard deviation.

heading: 5 Results and Analysis
text: Single-attribute Evaluation Table 1 reports the evaluation results of the BIOS and FCDL18 datasets on BERT-Base using adversarial bias removal and mutual information (MI) reduction. The results of the same experiments on BERT-Mini are shown in Table 6 in Appendix B.
Starting from task accuracy, the models using subnetworks (variations of DIFFPRUN and MODDIFFY shown at the lower part of the table) consistently perform the same as the fully finetuned models on both datasets and debiasing methods. We observe a slight decrease in performance for the adapter-based models on FCDL18.
Looking at the leakage probing performance of bias attributes, MODDIFFY models show better
performance (lower values) among the subnetworkbased models on both datasets, and overall on BIOS.2 In particular, MODDIFFY models outperform the directly comparable baselines FINETUNEDEBIAS and DIFFPRUN-DEBIAS on all configurations, indicating the benefits of learning separate debiasing modules on bias mitigation performance. This indeed comes with the core advantages of MODDIFFY models in proving modularized and on-demand bias mitigation.
The results also show that MODDIFFY-POST, while (as expected) slightly weaker than MODDIFFY-PAR, provides competitive bias mitigation performance (i.e., on par with DIFFPRUN-DEBIAS). Finally, comparing between debiasing optimizations, MI reduction shows consistently worse bias
2Particularly on FCDL18, we observe high variations, which requires us to more cautiously interpret results. We assume that this is due to the small size of this dataset especially in the learning regimes with many parameters on BERT-Base, as this effect is less pronounced on BERT-Mini (Table 6 in Appendix B).
mitigation performance in comparison with adversarial training, particularly on FCDL18 where the protected attribute has more than two labels.
Two-attribute Evaluation The results on PAN16 using BERT-Base are reported in Table 2, and the same experiments on BERT-Mini in Table 7 in Appendix B. In this experiment, every debiasing model is trained based on either gender, age, or simultaneously on both gender and age, shown with the subscripts G, A, G&A, respectively. An additional evaluation is indicated with MODDIFFY-*G + dittoA, which refers to adding the two separately trained gender and age debiasing subnetworks to the core model at inference time. In fact, for the experiments of each MODDIFFY model indicated with “+ditto”, G, and A, we train only one model with gender and age subnetworks, and then add the respective subnetwork(s) to the core model.
Looking at task performance results, similar to the previous datasets the models perform on par with fully finetuning, except the adapter-based models which in this case significantly underperform in both optimization methods. Regarding bias mitigation performance, we observe similar patterns to the ones discussed on the other datasets: MODDIFFY models particularly MODDIFFY-PAR show the least attribute leakage among the subnetworkbased models consistently, and also over all models with MI reduction. This reaffirms the benefits of modularizing debiasing of the attributes separately from the task. The results also indicate the existence of a correlation between the gender and age attributes in this dataset, such that debiasing each attribute also results in a decrease in leakage of the other attribute. The overall best results are achieved on the models that simultaneously optimize on both attributes (G&A).
Finally, let us have a closer look at the results of applying the two independently trained subnetworks. On both training regimes, we observe on par debiasing performance between MODDIFFY-*G+ dittoA and the corresponding results with one subnetwork, namely the ones of gender and age debiasing in MODDIFFY-*G and MODDIFFY-*A, respectively. These results indicate the viability of our approach to effectively merge subnetworks at inference time.
Subnetworks analysis We further investigate the achieved sparsity rate of the subnetworks, keeping
in mind that the maximum capacity of the debiasing subnetworks on BERT-Base is set to 1%. The size of a subnetwork indicates the amount of information or in fact modifications needed to be applied, in order to debias a protected attribute. Table 3 reports the percentage of the number of non-masked parameters in every layer, and also overall, in the subnetworks of the MODDIFFY-PAR models regarding the protected attributes. The results show interesting patterns in respect to various protected attributes: the gender attribute on both PAN16 and BIOS dataset require much smaller subnetworks, such that the subnetwork on PAN16 is only 0.01% of the size of the core model. The age attribute appears to be a more complex topic in the underlying PLM, as it fully uses the 1% maximum capacity. Looking across the layers, the results show that debiasing the gender attribute is mostly handled in the lower transformer layers (particularly on BIOS), while debiasing age and dialect attributes mostly happens at the higher layers.
In Appendix C, we further discuss this topic for all models on the level of individual weight matrices. Moreover, in Appendix D we investigate to what extent the subnetworks of a model across several runs affect on the same set of parameters.

heading: 6 Conclusion
text: We propose MODDIFFY, a novel bias mitigation approach which enables integration of an arbitrary subset of the debiasing modules at inference time. Our method encapsulates the functionality of bias mitigation in respect to a protected attribute into a separate magnitude-difference subnetwork, which can then be applied to the core model on-demand. Our experiments on three classification tasks show
that MODDIFFY improves bias mitigation achieved by separating the debiasing from the task network, and effectively mitigates the bias of two (and possibly more) attributes when their respective subnetworks are simultaneously utilized.

heading: 7 Limitations
text: An important limitation of our work concerns the definition of the protected attributes in the datasets used for evaluation. In particular, gender in BIOS and PAN16 is limited to the binary female/male, lacking an inclusive and nuanced definition of gender. Similarly in FDCL18, we consider only two dialects of African American and White American, while clearly this definition is limited and noninclusive. Furthermore as in previous work (Sap et al., 2019; Ravfogel et al., 2020; Zhang et al., 2021), the labels of this protected attribute are assigned through a probabilistic model, and hence the dataset might not represent the nuances and traits of the real-world.
The second limitation regards reaching strong conclusions on the generalizability of the multiattribute setting for MODDIFFY over any possible number of protected attributes or subset of them. Our multi-attribute experiments are conducted on one dataset with two attributes of gender and age, particularly due to the lack of available suitable datasets. Hence, Further studies (as well as more suitable datasets) are required for achieving a more comprehensive picture on the topic.
Finally, we should also highlight two general limitations, shared with the other related studies in the area of model bias mitigation. First, we should consider that the aim of representation disentanglement optimizations is to reduce the existing correlations in the model with the protected attributes based on the observed data. These data-oriented approaches might lack effective generalization, particularly when the model is evaluated in other domains or out-of-distribution data. Second, our bias mitigation evaluation is grounded in the notion of fairness through blindness, and the debiasing optimization methods are designed to support this form of fairness. The effects of our method on other possible definitions of fairness are therefore left for future work.

heading: 8 Acknowledgment
text: This work received financial support by the Austrian Science Fund (FWF): P33526 and DFH-23;
and by the State of Upper Austria and the Federal Ministry of Education, Science, and Research, through grants LIT-2020-9-SEE-113 and LIT-2021YOU-215.

heading: A Experiment Settings – Additional Details
text: In FDCL18 dataset, we use the TwitterAAE model (Blodgett et al., 2016) to assign racial dialect classes. The TwiiterAAE model predicts four racial classes, African American, White American, Hispanic, and Others. We labeled a tweet as African American or White American if the prediction score was greater than 0.5. For PAN16 dataset, following (Sap et al., 2019) we balanced the task labels and sampled 200K data. The age groups of this dataset are 18-24, 25-34, 35-49, 50-64, and 65+.
We randomly split the dataset into train, validation, and test set with the proportions 63:12:15 for BIOS, 63:12:15 for FDCL18, and 80:5:15 on PAN16. We use the validation set for hyperparameter tuning, and the best result on the validation set is evaluated on test set for the final results. The validation and test sets in all datasets follow the same distribution as the whole dataset. To address the unbalancedness of the dataset and the potential problems in adversarial learning, we apply upsampling only on the training sets of BIOS and FDCL18 datasets, to balance the protected attribute labels within each task label. For instance, genders are balanced in the dentist class by repeating the data items of the minority subgroup.
Adversarial heads consist of five classifiers with different initialization. The loss for the five classifiers is averaged and accuracy is measured via majority vote. All baseline models are trained for 20 epochs. All DIFFPRUN and MODDIFFY model variants are trained for 30 epochs as they need to account for the two phases of diff pruning, where a model requires more training to recover its performance after the magnitude pruning step. We fix the learning rate of BERT weights to 2e−5 and the learning rate for the classifier heads to 1e−4. We set the batch size to 64 for all experiments. We keep other diff -specific hyperparameters the same as suggested by Guo et al. (2021). Adapter baselines follow Pfeiffer et al. (2021) with reduction factor of two. Rest of the hyperparameters are same as the subnetwork-based models. Table 4 reports the hyperparameters of our experiments.

heading: B Additional Results
text: The results of all models using BERT-Mini are shown in Table 6 for BIOS and FCDL18 datasets, and in Table 7 for PAN16. Table 8 reports the evaluation results of the INLP method.

heading: C Sparsity rates of subnetworks
text: We visualize the percentage of non-masked parameters of the subnetworks in BERT-Base for each parameter matrix in Figures 2, 3 and 4 for MODDIFFY-PAR, MODDIFFY-POST, and DIFFPRUNDEBIAS, respectively. In addition to the discussion in Section 5, we observe in these detailed figures that the LayerNorm module of the last Transformer block generally has a high density. We assume
that this is due to the additive nature of these diff - based methods, as changing the weight magnitude through adding a subnetwork requires rescaling the final output.

heading: D Consistency in finding subnetworks
text: Figures 5, 6 and 7 show the percentage of common non-masked parameters in subnetworks on a particular weight matrix/layer across 5 runs for MODDIFFY-PAR, MODDIFFY-POST, and DIFFPRUN-DEBIAS, respectively. We report the percentage of overlap between the subnetworks of two, three, four, and five runs, separated with the “/”. The results show that the equivalent subnetworks across various runs (with different initialization seeds) seem to be largely separated. This results are consistent with the observations on the lottery ticket hypothesis on large neural networks (Chen et al., 2020).
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Left blank.

heading: 3 A2. Did you discuss any potential risks of your work?
text: Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Left blank.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4, 7 and Appendix A

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4, 7 and Appendix A

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section 4 and Appendix A

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4 and Appendix A
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4 and Appendix A
C 3 Did you run computational experiments? Section 4, 5, and Appendix A

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4 and Appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5, and Appendix B
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? No response.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: The effects of our method on other possible definitions of fairness are therefore left for future work