Year: 2021

Abstract:
Bias mitigation algorithms aim to reduce the performance disparity between different protected groups. Existing techniques focus on settings where there is a small number of protected groups arising from a single protected attribute, such as skin color, gender or age. In real-world applications, however, there are multiple protected attributes yielding a large number of intersectional protected groups. These intersectional groups are particularly prone to severe underrepresentation in datasets. We conduct the first thorough empirical analysis of how existing bias mitigation methods scale to this setting, using large-scale datasets including the ImageNet People Subtree and CelebA. We find that as more protected attributes are introduced to a task, it becomes more important to leverage the protected attribute labels during training to promote fairness. We also find that the use of knowledge distillation, in conjunction with group-specific models, can help scale existing fair learning methods to hundreds of protected intersectional groups and reduce bias. We show on ImageNet’s People Subtree that combining these insights can further reduce the bias amplification of fair learning algorithms by 15% —a surprising reduction given that the dataset has 196 protected groups but fewer than 10% of the training dataset has protected attribute labels.

heading: 1 INTRODUCTION
text: In many real-world applications, there is a significant potential for harm when the predictive properties and performance of machine learning models vary across different demographic populations. This is indeed the case for many applications including facial analysis (Buolamwini & Gebru, 2018), ad delivery (Sweeney, 2013) and search engines (Noble, 2018). Recent works aim to address this by designing learning algorithms that guarantee or approximate formal notions of algorithmic fairness (Mehrabi et al., 2021). However, there still remains a gap between theory and practice.
This gap is especially pronounced in the case of intersectional fairness. In the real-world, it is critical to simultaneously protect multiple attributes such as gender, skin color and age. Due to increasing disadvantage along the intersecting dimensions of protected attributes (Crenshaw, 1989; Foulds et al., 2020), it is also important to protect the intersections of these attributes. However, there has been scarce work on intersectional and subgroup fairness in deep learning contexts, with a large portion of the literature still focusing on single (binary) protected attribute settings, such as protecting the Male/Not-Male attribute in the CelebA Dataset (Liu et al., 2015).
In this work, we focus on intersectional fairness and seek to scale deep learning bias mitigation techniques to multiple protected attributes with hundreds of intersectional groups. Approaches such as training separate classifiers for each group (Wang et al., 2020), upweighting the losses of samples based on the group size, optimizing for the worst-case group outcome (Sagawa et al., 2020), or upweighting errors in early epochs (Liu et al., 2021) have all been shown to be effective in the single attribute setting. However, it is not clear whether they are effective given the intersection of multiple protected attributes, where there is a combinatorial number of protected groups and fairness for individual protected attributes does not imply intersectional fairness (Figure 1b). We address the question of how to scale existing bias mitigation techniques to this important but challenging setting.
Summary of Contributions:
1. We perform a thorough empirical analysis of existing deep learning bias mitigation techniques to explore their applicability in multi-attribute fairness settings. We find that as the number of protected attributes (k) increases, attribute labels become necessary to mitigate biases.
2. We conduct the first study of bias mitigation on the ImageNet People Subtree, with 196 protected groups but fewer than 10% of training data having protected attribute labels. In this challenging setting, we show existing bias mitigation methods can reduce bias amplification (Zhao et al., 2017) by 9% against empirical risk minimization (ERM). We identify two key issues with scaling up to so many intersectional groups: model complexity and overfitting to limited attribute labels.
3. We address these scaling challenges by proposing a novel regularization method: Knowledge Distillation of Independent Models as Regularization (DIR). DIR regularizes a single student classifier with teacher classifiers trained specifically for each intersectional group to implicitly incorporate attribute label information whenever it’s available. In contrast to existing techniques with complexity linear in the number of protected groups, DIR has a constant inference complexity and model size when used as a stand-alone bias mitigation algorithm. We further demonstrate that DIR can regularize other bias mitigation algorithms, reducing bias amplification by an additional 15% on ImageNet.
We first conduct a thorough evaluation of existing bias mitigation algorithms through a series of fair learning tasks on the CelebA dataset (Liu et al., 2015). Each round introduces a new protected attribute and evaluates whether existing methods can protect the resulting intersectional groups. Our findings demonstrate that, even with hundreds of intersectional groups, there are bias mitigation algorithms which can significantly reduce bias. However, they all require access to protected attribute labels during training. While bias mitigation methods which do not require attribute labels are occasionally effective in single-attribute settings, as suggested in prior literature (Liu et al., 2021; Shrestha et al., 2021), we show they are not effective in multi-attribute scenarios.
We then consider the more challenging task of learning a fair classifier on the ImageNet People Subtree, which Yang et al. (2020) labels for 196 intersectional groups. This setting is especially challenging for two reasons. First, our previous results suggest that access to protected attribute labels during training is necessary to effectively protect intersectional groups. However, such labels are scarce on our ImageNet dataset, with less than 10% of training datapoints having such information. This leaves bias mitigation algorithms prone to overfitting on the few available attribute labels. Second, bias mitigation algorithms often have a model size and complexity that scale linearly with the number of protected groups, which, in turn, explode combinatorially due to intersectionality.
We address these challenges by proposing Knowledge Distillation of Independent Models as Regularization (DIR). DIR uses group-specific teacher classifiers to train a single student classifier that, at each datapoint, mimics the output of the teacher classifier corresponding to the datapoint’s groundtruth group. DIR, whose model complexity is independent of the number of protected attributes (k), is an efficient alternative to existing bias mitigation algorithms which scale linearly with k.
DIR also provides a regularization scheme for other bias mitigation techniques, reducing overfitting by implicitly incorporating attribute label information into learning objectives. Using DIR, we can significantly mitigate bias towards intersectional groups on ImageNet, empirically demonstrating a 22% reduction in bias amplification over standard empirical risk minimization.

heading: 2 RELATED WORK
text: Algorithmic Fairness Prior works have explored algorithmic bias in a number of real-world applications. Deep learning models, including commercial image classifiers (Buolamwini & Gebru, 2018) and natural langauge processing (Alvi et al., 2018; Bolukbasi et al., 2016; Garg et al., 2018), have been the subject of significant scrutiny. Recent literature have proposed methods for bias mitigation (Edwards & Storkey, 2015; Ramaswamy et al., 2021; Ryu et al., 2018; Wang et al., 2020; Zhao et al., 2017), approximately optimizing metrics like worst-case accuracy (weighted on the worst-off group), mean accuracy (balanced over protected groups), bias amplification scores (Zhao et al., 2017), and intersectional bias scores (Foulds et al., 2020). Most build on existing techniques in classical fairness, such as reductions approaches, adversarial models, fairness through awareness, and importance weighting (Agarwal et al., 2018; Saerens et al., 2002; Dwork et al., 2012; Zhang et al., 2018). Many also borrow on techniques from robust learning (Adragna et al., 2020; Liu et al., 2021; Sagawa et al., 2020), causal inference (Arjovsky et al., 2020; Creager et al., 2021; Kusner et al., 2017; Madras et al., 2019), and representation learning (Pezeshki et al., 2020).
Intersectional Multi-Attribute Fairness One of our contributions is replicating existing results (Wang et al., 2020; Shrestha et al., 2021; Liu et al., 2021; Sagawa et al., 2020) and extending their empirical analysis to analagous multi-attribute settings. Shrestha et al. (2021) is the most related work of multi-attribute fairness, but only compare existing algorithms against new sets of datasets. In contrast, we focus on the ImageNet and CelebA datasets and observe algorithms as we tune the number of protected attributes (on a fixed dataset). We reach conflicting findings with Shrestha et al. (2021). While methods that do not access protected attribute labels may appear to be effective in certain single-attribute settings, their performance deteriorates in multi-attribute settings. We also find that a reweighting baseline is competitive with methods proposed by Wang et al. (2020). Beyond the deep learning settings considered by our paper and the aforementioned literature, Kang et al. (2021) considers a variational method for multi-attribute fairness in classical learning settings.
Fairness with Partial Unknown Attributes In many settings, it may not be practical to label the protected attributes of all datapoints. The setting where only part of the dataset has attribute labels has been explored in previous works: Dai & Wang (2021) targets a specific graph neural network setting and Ho et al. (2020) provides a basic analysis of adversarial fairness. Some works have looked at the case where no labels are available (Chen et al., 2019; Hashimoto et al., 2018). Others have looked at learning proxy labels from a different task (Kallus et al., 2020; Awasthi et al., 2021). Our work focuses on the partial unknown attributes case and provides an in-depth analysis with extensive experiments and larger coverage of methodologies.

heading: 3 PRELIMINARIES
text: We formalize our problem as learning a model h : X → Y , where X is the input space and Y a finite label space. We seek to learn a h that is fair with respect to am-dimensional protected attribute vector A where m is the number of protected attributes. Fairness datasets are thus in tuples (x, a, y) with image x ∼ X , protected attribute vector a ∈ A, and single/multi-label label y ∈ Y . For example, a protected attribute vector might look like a = (male, caucasian, teenager, not veteran).
For the entirety of this paper, “labeled/unlabeled” refers to whether the protected attribute labels A are available—we always assume the input features X and class labels Y are available. Unlabeled data refers to datapoints where X and Y are available, but A is not.
To be fair with respect to the protected attributesA, we want to be fair to the intersectional groupsG arising from A. The number of such groups is generally combinatorial in m. An example is shown in Figure 1(a). We formalize fairness among groups by reviewing two common definitions.
First, demographic parity concerns parity in Pr(ŷ | g), the probability of predicting a label ŷ ∈ Y for a datapoint belonging to a group g ∈ G. This definition of fairness requires that, over some fixed
data distribution, the predictions of a model h are probabilistically independent of the protected group membership. Formally, ∀g′ ∈ G, ŷ ∈ Y : Pr(ŷ | g) = Pr(ŷ | g′). This objective is generally more applicable to settings where the predicted class Y is a decision (e.g., issuing a loan).
Second, the equalized odds definition refines the demographic parity constraint to enforce parity in Pr(ŷ | g, y), the probability of predicting a label ŷ ∈ Y , for a datapoint with true label y ∈ Y belonging to group g ∈ G. This definition enforces, for instance, equivalent accuracies for each group. Formally, ∀g′ ∈ G, ŷ, y ∈ Y : Pr(ŷ | g, y) = Pr(ŷ | g′, y).

heading: 3.1 FAIRNESS METRICS
text: Given the difficulty of analyzing neural networks, we use metrics that approximate the margin by which notions of fairness are violated. For equalized odds, two common surrogate metrics include the importance-weighted accuracy and the worst-case accuracy. Importance-weighted accuracy (or “reweighted accuracy”) is the accuracy weighted such that each group has equal representation.
AccU (h) = Ex,y,g∼(X,Y,G) [ 1
|g| 1[y = h(x)]
] (1)
Worst-case accuracy is the accuracy of the group with the lowest accuracy.
AccW (h) = min g′∈G Ex,y,g∼(X,Y,G);g=g′ [1[y = h(x)]] (2)
Similarly, bottom-quartile accuracy is defined as the average of the bottom quartile of group accuracies. Papers based on robust learning often use worst-case accuracy metrics (Sagawa et al., 2020; Liu et al., 2021). However, more general papers also use importance weighted loss, such as Wang et al. (2020). We also define such reweighted and worst-case metrics for Mean Average Precision (mAP), which we use in place of accuracy for multi-label settings.
For demographic parity, two common metrics include bias amplification (Zhao et al., 2017) and intersectional bias (Foulds et al., 2020). Bias amplification measures the difference between a group’s ground-truth representation for a label class versus the group’s predicted representation:
sy := Pr(g | h(x))− Pr(g | y) where g := argmaxg′∈G Pr(g′ | h(x)). (3)
A positive score indicates that the model amplifies biases in the data. Generally, we seek a bias amplification score close-to or below zero. However, differences between negative bias amplification scores are non-informative. Bias amplification have been widely adopted by prior works like Wang et al. (2020), but has been found to be limited in what it captures (Wang & Russakovsky, 2021).
The intersectional bias score (Foulds et al., 2020) more directly measures violations of the demographic parity constraint. Larger intersectional bias indicates greater demographic parity violations.
y := max g1,g2∈G argmin s.t. exp(− ) ≤ Pr(h(x) = y | g1) Pr(h(x) = y | g2) ≤ exp( ) (4)
In contrast to bias amplification, we seek as low an intersectional bias score as possible. However, due to noise, intersectional bias scores can be non-informative when different methods have very similar scores. Since these metrics are defined for each class label y, in multi-class and multi-label settings we will occasionally average the metrics over their top quartile of class labels.

heading: 3.2 METHODS FOR ANALYSIS
text: Table 1 lists the bias mitigation algorithms explored in this paper. We consider both “unlabeled” and “labeled” methods, which are distinguished by whether they use protected group information during training. Our definition of “unlabeled” methods still assume access to class labels; they instead lack access to protected attribute labels. Abbreviations of unlabeled methods are prefixed with a “U-”. We include empirical risk minimization (“U-ERM”) as a baseline. Other unlabeled methods we consider include spectral decoupling (Pezeshki et al., 2020) and Just Train Twice (Liu et al., 2021).
For labeled methods, we include weighted ERM where the risk in Equation 1 is approximated with a maximum-likelihood-estimate of group importance weights. We also include robust optimization methods that use adversarial importance weights to either maximize the worst-case group accuracy (GDRO) (Sagawa et al., 2020) or eliminate protected group information (Adversarial) (Edwards & Storkey, 2015). We also include invariant risk minimization (IRM) (Arjovsky et al., 2020) for completeness. We include the domain independent and discriminative algorithms (Wang et al., 2020), which train a collection of group-specific classifiers and ensemble them for inference.

heading: 4 ANALYSIS TOWARDS MULTIPLE PROTECTED ATTRIBUTES
text: In this section, we analyze how different bias mitigation techniques perform in settings with multiple protected attributes. We build on the CelebA dataset (Liu et al., 2015) experiment settings considered in Wang et al. (2020), Liu et al. (2021), and Shrestha et al. (2021). Specifically, we expand their single-attribute experiments by incrementally introducing additional protected attributes. The Appendix includes additional replication results alongside other competitive baselines.
Experimental Setup. We consider two primary experiment settings: (1) a binary classification task, CelebA-SL, on CelebA’s blond hair class label, taken from Liu et al. (2021) and Shrestha et al. (2021). (2) a multi-label classification task, CelebA-ML, on 33 out of 40 of CelebA’s class labels, taken from Wang et al. (2020). For CelebA-ML, the 7 omitted class labels are reserved as protected attributes and specified in the following paragraph.
For both CelebA-SL and CelebA-ML, we train residual networks (He et al., 2016) using the algorithms listed in Table 1. We evaluate how these methods perform as new protected attributes are introduced by repeating the learning task while each time introducing a new attribute to protect. Let k denote the number of protected attributes. Since every CelebA attribute is binary, every protected attribute that is added doubles the number of intersectional groups. The protected attributes that are added are taken from a fixed set to reduce variance. The pool of potential protected attributes is, in order: Pale Skin, Male, Narrow Eyes, Big Nose, Young, Straight Hair, and Attractive. These at-
tributes were selected pseudorandomly (i.e., without tuning) for their problematic associations with protected demographics. Further details on dataset and model hyperparameters are in the Appendix.
Primary Results. CelebA-SL and CelebA-ML are depicted in Figures 2, 3 with the key findings: 1. We can mitigate intersectional bias and bias amplification, even for hundreds of intersectional
groups, using labeled methods like Domain Independent and Weighted ERM. 2. In contrast to prior findings by Liu et al. (2021); Shrestha et al. (2021), we find that unlabeled
methods generalize poorly to other fair learning tasks and, even if they perform competitively for small k (number of protected attributes), they have bad intersectional bias for large k.
3. There is no consistently superior bias mitigation algorithm. Methods like GDRO, IndependentSP, and Weighted ERM may each perform best depending on the learning task, performance/bias metric, and the value of k.
Intersectional Bias Mitigation On Celeba-SL (Figure 2), Weighted ERM and GDRO are the best overall methods, with similar accuracy, intersectional bias, and bias amplification for k ≤ 3. Both methods successfully reduce bias amplification and intersectional bias (versus U-ERM) by a significant margin without losing accuracy. On CelebA-ML (Figure 3), Weighted ERM and IndependentSP are the best overall methods, with Independent-SP outperforming Weighted ERM in terms of bottom quartile mAP (59% vs 53% for k = 3) and intersectional bias (5.5 vs 6.2 for k = 7). Again, these best methods for CelebA-ML successfully reduce bias amplification and intersectional bias (versus U-ERM) by a significant margin without significant loss of accuracy. These results suggest that bias reduction is possible, even under highly intersectional settings with large k, but the optimal choice of a bias mitigation algorithm is context-dependent. All the best methods we found in this section are labeled methods.
Unlabeled Methods Prior works found that unlabeled methods perform competitively with labeled methods when protecting a single attribute on CelebA-SL (Shrestha et al., 2021; Liu et al., 2021). Appendix A.1 replicates their experiments with highlights in Tables 2-3. Table 3 confirms that U-JTT yields a competitive worst-case accuracy and bias amplification (85% vs Weighted ERM’s 89% accuracy; -0.041 vs -0.055 bias amp). Table 2 shows U-SD also yields competitive worst-case accuracy and bias amplification (82% vs GDRO’s 83% accuracy; -0.057 vs -0.065 bias amp).
However, we find these prior results are fragile. Figures 2-3 shows U-JTT has consistently middleof-the-pack bias scores and low reweighted accuracies for all k. Even on Figure 2’s k=1 case, JTT’s 88% reweighted accuracy is 3%+ below others. The only difference between Figure 2’s k=1 case and Table 3 is that the former protects “Is/Not Pale Skin” attribute instead of “Is/Not Male”. In Figure 2 (CelebA-SL), U-SD fares better with competitive reweighted accuracy (92% vs weighted ERM’s 92%), bias amplification (0.001 vs weighted ERM’s 0.003), and intersectional bias scores (0.3 vs weighted ERM’s 0.3) for k=1. However, for k ≥ 3, the intersectional bias of U-SD is
consistently greater than the labeled baseline, Weighted ERM: 1.9 vs 1.5 for k = 5 and 3.0 vs 2.6 for k = 7. U-SD was not evaluated on CelebA-ML as its hyperparameters are intractable to tune for large numbers of label classes. These negative results highlight the limitation of unlabeled methods and thus the importance of protected attribute labels, particularly in the multi-attribute domain.
Intersectional Bias and Bias Amplification Trends For both CelebA-SL and CelebA-ML, intersectional bias scores are generally increasing in k (see the right-most plots of Figures 2, 3). The noise of the metric itself, however, does not grow significantly with the number of protected attributes. Bias amplification scores are significantly more noisy (see all lines in Figure 2 and the U-JTT line in Figure 3). We also find that bias amplification is uninformative for CelebA-SL with all methods but Independent-SP having a bias amplification score below zero. However, we attribute this to the binary classification nature of CelebA-SL where, although there is class imbalance (85% to 15%), there is a sufficient number of datapoints per class that there is little risk of underfitting with ResNets. Bias amplification remains non-trivial on the multi-label CelebA-ML and, as we will see later, is also non-trivial on the multi-class prediction ImageNet task.

heading: 5 SCALING TO REAL-WORLD DATASETS
text: We now consider learning fair models on the ImageNet People Subtree, with protected attribute labels annotated by Yang et al. (2020). This dataset is a realistic benchmark for intersectional bias mitigation, featuring 196 intersectional groups arising from 3 protected attributes: gender, skin color, and age. In addition, only 15,981 images of the dataset’s 124,693 images have protected attribute labels—posing a challenge with label scarcity. We found in Section 4 that bias mitigation is possible with hundreds of intersectional groups but that labeled methods are imperative. Can we still effectively mitigate bias on ImageNet with hundreds of protected groups but scarce attribute labels?
We first analyze the performance of existing labeled and unlabeled bias mitigation algorithms on ImageNet. We then improve on those results by proposing a general knowledge distillation based approach (DIR) that helps scale bias mitigation algorithms to the ImageNet People Subtree.

heading: 5.1 BIAS MITIGATION TECHNIQUES ON THE IMAGENET PEOPLE SUBTREE
text: Experimental Setup. For this section, we use the ImageNet People Subtree annotated by Yang et al. (2020). We have 130,554 images for training (5,861 of which have attribute labels). We have 5,327 images for validation and 5,327 for test, all with attribute labels. 3 protected attributes (gender, skin color, age) are annotated with 196 corresponding intersectional groups. This raises the issue of zero-shot fairness, as 237 out of 284 synsets on the people subtree have zero representation of dark-skinned females (contrast this to the only 3 out of 284 synsets that lack light skinned male representation). The methods considered in this section are as described in Table 1 with the exception of weighted ERM, which we modify to square-root all importance weights–this is to remedy the high-variance importance weights that arise on ImageNet.
Existing Methods on ImageNet. Table 4 shows the performance of various labeled and unlabeled methods on ImageNet. The key result is that, using existing labeled bias mitigation methods, we can
significantly mitigate bias. Weighted ERM and Independent-SP both outperform standard ERM in terms of reweighted accuracy (47%, 47% vs 45%) and bias amplification (0.076, 0.074 vs 0.082) by a significant margin. The intersectional bias metric is uninformative, i.e., all methods are within a standard deviation of each other.
We also find that unlabeled methods U-JTT and U-SD (45%, 37% accuracy) are significantly outperformed by Weighted ERM and Independent-SP (47% accuracies). U-JTT and U-SD’s bias amplification and intersectional bias scores are within a standard deviation of Weighted ERM and Independent-SP’s. This corroborates our Section 4 findings of the limited applicability of unlabeled methods in multi-attribute settings where protected attribute information is critical.
Pretraining. An important note about the results in Table 4 is that all methods are pretrained with ERM on the entire training set, including data points without attribute labels. This is justified in Table 5 which compares Table 4 results against their not-pretrained counterparts. Pretraining is important for obtaining reasonable performance, improving reweighted accuracy (15% → 47% for Weighted ERM and 17% → 47% for Independent-SP) and reducing bias amplification (0.096 → 0.076 and 0.143 → 0.074 respectively). Surprisingly, pretraining increases intersectional bias (2.16 → 2.69 for Weighted ERM and 2.27 → 2.69 for Independent-SP). This is a flaw of the metric. Low accuracy, high uncertainty neural networks act like a random classifier, which have a intersectional bias close to zero (Eq 4); this is not the case with bias amplification, for instance.

heading: 5.2 KNOWLEDGE DISTILLATION FOR SCALING LABELED BIAS MITIGATION TECHNIQUES
text: Our findings in Section 5.1 demonstrate that bias mitigation methods, including Weighted ERM and Independent-SP, can successfully mitigate bias in multi-attribute settings. However, we encountered two key issues from scaling to ImageNet.
Model Size. The number of protected groups grows combinatorially with the number of protected attributes. This poses an issue for methods which have a model size linear in the number of protected groups, including Independent-SP, Adversarial, and Discriminative. Their inference complexity increases combinatorially. This necessitated low batch sizes for the experiments in Table 4.
Overfitting. Given the large number of protected groups but scarce protected attribute information, labeled bias mitigation algorithms are prone to overfit to available attribute information. For instance, methods which estimate importance weights over protected groups, e.g. Weighted ERM, are prone to high-variance weight estimates. This also affects methods, like Independent-SP and Discriminative, which use protected attribute labels to selectively update network weights: weights associated with underrepresented groups may overfit to the scarce datapoints labeled with the group.
Knowledge Distillation of Independent Models as a Regularization. We propose Knowledge Distillation of Independent Models as Regularization (DIR) to address these challenges. Specifically, we develop a two-stage bias mitigation approach, where we distill a set of group-specific
classifiers into a single classifier. This provides: (1) a bias mitigation method with O(1) inference complexity but the same inductive biases as the O(k) complexity Domain Independent algorithm; and (2) a form of regularization that can be used to augment any labeled bias mitigation algorithm.
GivenG protected groups, the Domain Independent (Independent-SP) algorithm (Wang et al., 2020) trains a specialized model hg for each protected group g, training on the same X → Y task but exclusively on datapoints labeled with g. Given an input x, the algorithm directly sums the outputs of the set of hg ,
z = ∑ g∈G hg(x), (5)
where z is final output of the Domain Independent model. Instead of summing over the specialized classifiers {hg | g ∈ G}, we propose a distillation-based approach that trains a single classifier hd with the following loss for each datapoint (x, y, g) ∼ (X,Y,G):
`(hd(x), y) + λKL(hd(x), hg(x)), (6)
where hg are already trained, `(·, ·) is some loss function and λ is the weight for the distillation term. This formulation yields a single classifier hd that is regularized by the second term in Equation (6), which incentivizes the output hd(x) to be similar to that of hg , the specialized classifier for the datapoint’s protected group g. We refer to this scheme as DIR. When ` is the original task loss function, e.g. cross-entropy, Equation 6 is an approximation of the Domain-Independent algorithm withO(1) rather thanO(k) model complexity. However, we can also use DIR to augment other bias mitigation algorithms. For instance, we can augment Weighted ERM by selecting its importance weighted loss as `. This yields the regularized objective∑
i
wi`(h(xi), yi) + λKL(f(xi), hgix(xi)), (7)
where wi denotes the importance weight estimate used by Weighted ERM, gix the protected group of xi and hgix the datapoint’s group-specific classifier.
Experimental Results. Table 6 depicts the performance gains on ImageNet from using DIR to augment Weighted ERM and Independent-SP. The second and fourth rows correspond to models trained with Equation (6) and Equation (7), respectively. DIR reduces bias amplification for both Independent-SP (0.074 → 0.063) and Weighted ERM (0.076 → 0.073) while maintaining the accuracy gains realized by these two labeled methods. Intersectional bias is again uninformative.

heading: 6 ETHICAL CONCERNS & CONCLUSION
text: In this paper, we conduct a thorough review of bias mitigation deep learning algorithms in multiattribute settings. We also propose a knowledge distillation framework for improving bias mitigation on the protected-label-scarce ImageNet dataset. In many applications of machine learning technology that directly or indirectly interface with protected classes, the insights we find concerning multi-attribute fairness may help better inform interventions for bias mitigation.
Due to the nature of the datasets that our experiments use, there are a number of potential ethical concerns arising from the use of protected attribute labels. Dataset schemas used commonly in deep learning literature, such as the subjective “Attractive/Not-Attractive” attribute adopted by the CelebA dataset, can be problematic. In addition, there are privacy and representation concerns inherent in the process of labeling protected attributes. For a more detailed discussion, we refer readers to Yang et al. (2020); Paullada et al. (2020); Birhane & Prabhu (2021) which further explore issues in fair dataset design, especially concerning bias and privacy.

Review summaries:
This paper conducts an empirical analysis of existing bias mitigation methods on two large datasets CelebA and ImageNet where there are multiple sensitive attributes and some unavailable protected labels. The results show the existing can mitigate intersectional bias at scale but the unlabeled methods generalize poorly. This paper further proposes a knowledge distillation of independent models as regularization method (DIR) which is able to augment into other bias mitigation algorithms.

This paper studies the fairness of deep learning models in classification tasks with a large number of intersectional groups. The paper has two primary contributions:  1. The paper includes an empirical study that shows the inherent difficulty of this setting (i.e., due to the lack of labels), and the limitations of existing approaches.  2. The paper presents a new approach to train deep models to settings with many intersectional groups called Knowledge Distillation of Independent Models as Regularization (DIR).

 This paper provides an empirical investigation of deep learning bias mitigation methods, focusing on two problems: intersectionality and missing sensitive attribute labels. The paper evaluates several bias mitigation approaches on the Celeb dataset and on ImageNet, the latter having few sensitive attribute labels. The main takeaways from their empirical analysis is that the best performing mitigation method depends on the context and mitigation methods that operate without using the sensitive attribute do not perform well in terms of intersectionality. The paper also proposes a new distillation-based approach to resolve the runtime complexity problem that models like Domain Independent exhibit. On ImageNet their method significantly reduces bias amplification but there are no differences in group-reweighed accuracy or intersectional bias.