Balancing out Bias: Achieving Fairness Through Balanced Training (2022)

Abstract: Group bias in natural language processing tasks manifests as disparities in system error rates across texts authorized by different demographic groups, typically disadvantaging minority groups. Dataset balancing has been shown to be effective at mitigating bias, however existing approaches do not directly account for correlations between author demographics and linguistic variables, limiting their effectiveness. To achieve Equal Opportunity fairness, such as equal job opportunity without regard to demographics, this paper introduces a simple, but highly effective, objective for countering bias using balanced training.We extend the method in the form of a gated model, which incorporates protected attributes as input, and show that it is effective at reducing bias in predictions through demographic input perturbation, outperforming all other bias mitigation techniques when combined with balanced training.

## introduction natural language processing (nlp) models have achieved extraordinary gains across a variety of tasks in recent years. however, naively-trained models often learn spurious correlations with other demographics and socio-economic factors @xcite @xcite , leading to disparities across author demographics in contexts including coreference resolution, sentiment analysis, and hate speech detection @xcite @xcite . two popular approaches for mitigating such biases are: (1) balancing each demographic group in training, either explicitly via sampling @xcite or implicitly via balancing losses for each group @xcite 1 code available at https://github.com/ hanxudong/achieving_fairness_through_ balanced_training/ @xcite ; and (2) removing demographic information from learned representations @xcite @xcite . while balancing methods have been shown to be successful, they have not been tested extensively in nlp. in this paper, we focus on equal opportunity fairness (eo: @xcite ), which requires non-discrimination across demographics within the "advantaged" outcome labels, and adapt three balanced training approaches for debiasing. in addition, we propose a new objective for balanced training, which can be used for proxy optimization of eo fairness. we first provide a theoretical justification for our approach, and then conduct experiments on two benchmark datasets which show that our proposed objective is highly effective at achieving eo fairness while maintaining competitive accuracy. even when the training data is balanced, ignoring demographic-specific features can lead to bias @xcite , due to differences in language use across demographics @xcite . there is thus a fine line to be walked in terms of optimizing for linguistic variables associated with different demographic groups (potentially boosting overall model accuracy), and ensuring model fairness. inspired by work in domain adaptation on learning domain-specific representations that generalize across domains @xcite , we propose a gated model, which incorporates author demographics as an input to generate group-specific representations but also generalizes across demographic groups. we show that when combined with instance reweighting during training, this technique leads to substantial bias reductions over leading debiasing techniques, typically with higher predictive accuracy. we also introduce a second means of bias reduction through tailoring gating coefficients of the trained model, which al-lows for fine-tuning of the accuracy-fairness tradeoff. our experiments over two benchmark datasets for language debiasing show that our techniques are competitive with much more complex state-ofthe-art methods for debiasing in situations where the demographic attribute is not known at test time, and provide substantial gains over the state-of-theart when the protected attribute is observed. ## balanced training despite their simplicity and versatility, balanced training approaches have received limited attention in prior work in nlp. in this section, we propose a novel objective for balanced training, which we show to be a proxy for the eo. we further review three balanced training approaches, discuss their objectives, and highlight their differences over our proposed method. ## demographic factors improve fairness ignoring demographic-specific features can lead to bias even when the training data has been balanced @xcite . instead, as suggested by @xcite , addressing demographic factors is essential for nlp to get closer to the goal of human-like language understanding, and increase fairness. our approach to dealing with this is, rather than removing demographic information, to use a gated model that uses demographic labels as input. as can be seen in figure 2 , the gated model consists of (1+|g|) encoders: one shared encoder, and a dedicated encoder for each demographic group in g. 2 formally, let e denote the shared encoder, e j denote the encoder for the j-th demographic group, c denote the classifier, and g i be a 1-hot input such that g i,j is 1 if the instance (x i , g i , y i ) belongs to the j-th group, and 0 otherwise. the prediction for an instance is: the two inputs are concatenated and input to the classifier c. intuitively, the shared encoder learns a general representation, while each group-specific encoder captures group-specific representations. our setting differs from other debiasing methods in that we assume the demographic attribute is available at training and prediction time, while techniques such as adversarial training @xcite and @xcite only require the attribute for training. this richer input allows for more accurate predictions, courtesy of the demographic-specific encoder, but limits applica-bility at test time. for better applicability, we also relax this requirement by replacing demographic factors with a non-informative prior in section 4.7. ## experimental results in this section we first introduce our experimental settings, and then report and discuss our results. in appendix b, we provide detailed settings for reproducing our experiments. ## related work fairness much work on algorithmic fairness has focused on group fairness, i.e., disparities in error rates across groups defined by protected attributes, such as gender, age, or race. many criteria have been proposed for group fairness, such as statistical parity @xcite and equal opportunity @xcite . broadly speaking, fairness can be classified into three categories: independence, separation, and sufficiency @xcite , with the most recent work addressing separation criteria, i.e, potential correlations between main task labels and protected attributes. mitigating bias many approaches for bias mitigation haven been proposed recently, including removing protected information form hidden representations @xcite @xcite , preprocessing data to remove bias @xcite @xcite , modifying the training algorithm @xcite , and post-hoc correction @xcite . in the context of nlp, the best results have been achieved through protected information removal. iterative nullspace projection (ravfogel et al. ( 2020 )) takes hidden representations and projects them onto the nullspace of the weights of a linear classifier for each protected attribute. the classifier training and projection are carried out over multiple iterations to more comprehensively remove protected information. another popular approach is adversarial training, which jointly optimizes the removal of sensitive information and main task performance, through the incorporation of adversarial discriminator(s) to identify protected attributes from the hidden representations @xcite @xcite . differentiated adversarial learning @xcite ) uses an ensemble of adversaries for each protected attribute, subject to an orthogonality constraint. comparison with mixture of experts one line of work that is similar to our gated model is mixture of experts @xcite . technically, the gated model is similar to the moe model in the sense that an expert can be largely aligned with a group-specific encoder in our model. however, there are several key differences: (1) instead of making independent predictions by each expert, our group-invariant encoder acknowledges the shared patterns across demographic groups; (2) moe employs an extra softmax gating network to mix experts' predictions, while our method does an argmax based on group labels; and (3) we use the group-specific information jointly together with the group-invariant encoder' outputs for making the final predictions while the moe model has one output layer for each expert. ## conclusions and future work this paper proposed the adoption of balanced training approaches to mitigate bias, and demonstrated their effectiveness relative to existing methods, as well as their ability to further enhance existing methods. we also proposed a gated model based on demographic attributes as an input, and showed that while the simple version was highly biased, with a simple bayesian extension at inference time, the method was highly effective at mitigating bias. for future work, it is important to consider settings where there are multiple protected attributes, such as author age, gender, and ethnicity. a simple extension would be to treat g as being intersectional classes, defined as the cartesian product of the multiple demographic groups. e.g., k binary groups would result in 2 k intersectional classes.