Systematic Evaluation of Predictive Fairness (2022)

Abstract: Mitigating bias in training on biased datasets is an important open problem. Several techniques have been proposed, however the typical evaluation regime is very limited, considering very narrow data conditions. For instance, the effect of target class imbalance and stereotyping is under-studied. To address this gap, we examine the performance of various debiasing methods across multiple tasks, spanning binary classification (Twitter sentiment), multi-class classification (profession prediction), and regression (valence prediction). Through extensive experimentation, we find that data conditions have a strong influence on relative model performance, and that general conclusions cannot be drawn about method efficacy when evaluating only on standard datasets, as is current practice in fairness research.

## introduction and background naively-trained models have been shown to encode and amplify biases in the training dataset, and exhibit performance disparities across author demographics @xcite @xcite . various methods have been proposed to mitigate such biases, such as balanced training @xcite , adversarial debiasing @xcite , and null-space projection @xcite . however, experiments have largely been conducted on a handful of benchmark datasets such as moji sentiment analysis @xcite and bios biography classification @xcite , under a narrow set of data conditions. in this paper, we systematically explore the impact of data conditions on model accuracy and fairness, synthesising the following data conditions over real-world datasets: (1) target label (im)balance; (2) protected attribute (im)balance; (3) target label-protected attribute (im)balance (also known as "stereotyping"); and (4) target label arity. consistent with the literature on fairness in nlp, we primarily focus on classification tasks, but also include preliminary text regression experiments. in doing so, we develop a novel framework for comprehensively evaluating the performance of debiasing methods under a range of data conditions, and use it to evaluate eight widely-used debiasing methods. our experimental results show that there is no single best model. debiasing methods that account for both class disparities and demographic disparities are generally more robust, but are less effective in multi-class settings. for the regression task, our experiments indicate that existing debiasing approaches can substantially improve fairness, and that simple linear debiasing outperforms more complex methods. ## related work in this section, we first describe different fairness criteria, then examine work which has evaluated the effectiveness of debiasing methods from different perspectives. fairness criteria studies in the fairness literature have proposed several definitions of fairness capturing different types of discrimination, such as group fairness @xcite @xcite , individual fairness @xcite @xcite , and causality-based fairness @xcite . in this work, we focus on group fairness, where a model is considered to be fair if it performs identically across different demographic subgroups. to quantify how predictions vary across different demographic subgroups, demographic parity @xcite @xcite , equal opportunity @xcite , and equalized odds @xcite @xcite are widely-used notions. we present these in a setting where there are exactly two protected attribute labels (a "privileged" and "under-privileged" subpopulation), consistent with how they are traditionally defined. demographic parity ensures that models achieve the same positive prediction rate for the two demographic subgroups, not taking the ground-truth target label into consideration. equal opportunity requires that models achieve the same true positive rate across the two subgroups for instances with a positive label. equalized odds goes one step further in requiring that models achieve not only the same level of true positive rate but also the same level of false positive rate across the two groups. aligned with key applications such as loan approvals, most fairness metrics assume binary classification and focus on one label (e.g., loan approved.) when turning attention to a multi-class classification scenario, equal opportunity is a natural choice, as it can be easily reformulated by assigning the positive class to each candidate class under a 1-vs-rest formulation. ## methods here we describe the methods employed to manipulate the dataset distributions for classification tasks, and then describe how we adopt debiasing methods to a regression setting. ## experiments in this section we describe general settings across all experiments. in appendix a, we provide full experimental details and dataset statistics. ## binary classification the task is to predict the binary sentiment (happy and sad) of a given english tweet, as determined by the (redacted) emoji used in the tweet. each tweet is also associated with a binary protected attribute, reflecting the ethnicity of the tweet author, as captured in the register of the english: standard american english (sae) and african american english (aae). we use the widely-used twitter emoji dataset @xcite @xcite , denoted as moji. the training dataset is balanced in terms of both sentiment and ethnicity in general, but skewed in terms of sentiment-ethnicity combinations, p (aae|happy) = p (sae|sad) = 0.8. for ease of comparison with previous work @xcite , we refer to the cb interpolation as varying "stereotyping" (p ′ (z|y)) with balanced target class distribution. to explore the effects of target class distribution and stereotyping, we further experiment in various controlled settings: (1) varying class ratio (p ′ (y)) without stereotyping (p ′ (z|y) = 0.5); (2) varying stereotyping with imbalanced target class distribution; and (3) varying class ratio with stereotyping. finally, we summarise our findings with respect to the effectiveness and robustness of various debiasing methods over different class-stereotyping compositions. ## multi-class classification we next turn to our second dataset, which is a multiclass classification task with natural imbalance in both target labels and protected groups. the dataset consists of online biographies, labeled with one of 28 occupations (target labels) and binary author gender (protected label), and the task is to predict the occupation from the biography text @xcite ). ## regression we finally turn to the regression setting. the task is to predict the valence (sentiment) of a given facebook post, where each post is assigned a valence score by two trained annotators in the range 1-9 and the task is to predict the average of the two scores @xcite . additionally, each post is associated with a binary authorship gender label. 3 in our experiments, results are reported based on 5-fold cross-validation. ## general discussion and recommendations so far, we have shown that there is no single best model across different data conditions, and data conditions should be a key consideration in fairness evaluation. in this section, we divide debiasing methods into three families, and summarize their robustness to skewed training data distributions. balancing demographics in the training dataset ds and rw are representatives of this family, and are simple and effective. in addition, such methods are flexible as the training dataset is pre-processed before model training, and any candidate models on the original dataset can be applied to the debiased dataset. however, ds methods are sensitive to group sizes. considering an extreme setting where the smallest subset in the training dataset has 0 instances, i.e., d c,@xmath0, ds will result in an empty training set. for instance, the group size distribution is highly skewed for the regression task, and ds resulted in @xmath1. similar problems are associated with up-sampling methods, which can increase the training set size dramatically. in addition, when considering multiple protected attributes, such as intersectional groups and gerrymandering groups @xcite , the number of groups increases exponentially with the number of protected attributes to be considered. as a result, the joint distributions can be highly skewed, and these two families of methods (resampling and reweighting) may not be appropriate choices. lastly, skewed protected label distributions in the training dataset is not the only source of bias @xcite . for example, as shown in figure 1 the vanilla model trained over balanced versions (p ′ (aae|happy) = 0.5) of the moji dataset is less fair than the vanilla model trained over a biased dataset where p ′ (aae|happy) = 0.4. learning fair hidden representations adv, dadv, and inlp represent a family of methods that learn fair representations through unlearning discriminators. since the training and unlearning of discriminators do not take into account target class information, these methods are robust to the number of classes and naturally generalize to regression tasks. however, these methods are not capable of modelling conditional independence for the equal opportunity criterion without taking target class into consideration, resulting in worse dto than other debiasing methods over moji (section 5). to achieve equal opportunity fairness, different discriminators can be trained for each target class to capture conditional independence @xcite . but training targetspecific discriminators assumes target labels to be discrete, which is sensitive to the number of classes. another limitation of this family of methods is associated with the discriminator learning: the discriminator can also suffer from long-tail learning problems, i.e. skewed demographics, and lead to biased estimations of protected information. the unlearning of biased discriminators limits the method's contribution to bias mitigation, which can be seen from figures 3 and 4 in section 5. minimising loss disparities across demographic groups fairbatch, eo cla , and eo glb provide a practical approximation of expected fairness in using empirical risk-based objectives, and directly optimize for empirical risk parity during training. similar to balanced training approaches, resampling and reweighting are also used in mitigating loss disparities, where fairbatch adjusts resampling probabilities for batch selection, and eo cla and eo glb assign instances different weights depending on the demographic group they belong to. however, minimising loss disparities can be more flexible than balanced training -for example, instance weights are dynamically adjusted by eo cla and eo glb , and can take on negative values to aggressively reduce a bias towards favouring of over-represented groups. conversely, drawbacks associated with resampling and reweighting also apply to this family. for example, fairbatch indeed broke down (an error raised) when d c,@xmath2. minimising loss difference is also less efficient in multi-class settings, as it adjusts weights based on class information during training, making optimisation harder. ## conclusion in this work, we presented a novel framework for investigating different classification dataset distributions with a single parameter, and used it to systematically examine the effectiveness of debiasing methods in binary classification and multiclassification settings based on real-world datasets. we also presented preliminary analysis of debiasing methods in a regression setting, including proposing a method for adapting existing debiasing methods to regression tasks. based on extensive experimentation over three datasets, we found that there was no single best model. debiasing methods that account for both class and demographic disparities are generally more robust, but are less efficient at achieving fairness in multi-class settings. for the regression task, we demonstrated that existing debiasing approaches can substantially improve fairness, and that the simple linear debiasing method outperforms more complex techniques. in summary, there is no universal best debiasing method across all tasks, and data conditions have a large impact on different models. as such, we propose that future research adopts our evaluation framework as a means of more comprehensively evaluating debiasing methods.