Scaling Fair Learning to Hundreds of Intersectional Groups

1 INTRODUCTION . In many real-world applications , there is a significant potential for harm when the predictive properties and performance of machine learning models vary across different demographic populations . This is indeed the case for many applications including facial analysis ( Buolamwini & Gebru , 2018 ) , ad delivery ( Sweeney , 2013 ) and search engines ( Noble , 2018 ) . Recent works aim to address this by designing learning algorithms that guarantee or approximate formal notions of algorithmic fairness ( Mehrabi et al. , 2021 ) . However , there still remains a gap between theory and practice . This gap is especially pronounced in the case of intersectional fairness . In the real-world , it is critical to simultaneously protect multiple attributes such as gender , skin color and age . Due to increasing disadvantage along the intersecting dimensions of protected attributes ( Crenshaw , 1989 ; Foulds et al. , 2020 ) , it is also important to protect the intersections of these attributes . However , there has been scarce work on intersectional and subgroup fairness in deep learning contexts , with a large portion of the literature still focusing on single ( binary ) protected attribute settings , such as protecting the Male/Not-Male attribute in the CelebA Dataset ( Liu et al. , 2015 ) . In this work , we focus on intersectional fairness and seek to scale deep learning bias mitigation techniques to multiple protected attributes with hundreds of intersectional groups . Approaches such as training separate classifiers for each group ( Wang et al. , 2020 ) , upweighting the losses of samples based on the group size , optimizing for the worst-case group outcome ( Sagawa et al. , 2020 ) , or upweighting errors in early epochs ( Liu et al. , 2021 ) have all been shown to be effective in the single attribute setting . However , it is not clear whether they are effective given the intersection of multiple protected attributes , where there is a combinatorial number of protected groups and fairness for individual protected attributes does not imply intersectional fairness ( Figure 1b ) . We address the question of how to scale existing bias mitigation techniques to this important but challenging setting . Summary of Contributions : 1 . We perform a thorough empirical analysis of existing deep learning bias mitigation techniques to explore their applicability in multi-attribute fairness settings . We find that as the number of protected attributes ( k ) increases , attribute labels become necessary to mitigate biases . 2 . We conduct the first study of bias mitigation on the ImageNet People Subtree , with 196 protected groups but fewer than 10 % of training data having protected attribute labels . In this challenging setting , we show existing bias mitigation methods can reduce bias amplification ( Zhao et al. , 2017 ) by 9 % against empirical risk minimization ( ERM ) . We identify two key issues with scaling up to so many intersectional groups : model complexity and overfitting to limited attribute labels . 3 . We address these scaling challenges by proposing a novel regularization method : Knowledge Distillation of Independent Models as Regularization ( DIR ) . DIR regularizes a single student classifier with teacher classifiers trained specifically for each intersectional group to implicitly incorporate attribute label information whenever it ’ s available . In contrast to existing techniques with complexity linear in the number of protected groups , DIR has a constant inference complexity and model size when used as a stand-alone bias mitigation algorithm . We further demonstrate that DIR can regularize other bias mitigation algorithms , reducing bias amplification by an additional 15 % on ImageNet . We first conduct a thorough evaluation of existing bias mitigation algorithms through a series of fair learning tasks on the CelebA dataset ( Liu et al. , 2015 ) . Each round introduces a new protected attribute and evaluates whether existing methods can protect the resulting intersectional groups . Our findings demonstrate that , even with hundreds of intersectional groups , there are bias mitigation algorithms which can significantly reduce bias . However , they all require access to protected attribute labels during training . While bias mitigation methods which do not require attribute labels are occasionally effective in single-attribute settings , as suggested in prior literature ( Liu et al. , 2021 ; Shrestha et al. , 2021 ) , we show they are not effective in multi-attribute scenarios . We then consider the more challenging task of learning a fair classifier on the ImageNet People Subtree , which Yang et al . ( 2020 ) labels for 196 intersectional groups . This setting is especially challenging for two reasons . First , our previous results suggest that access to protected attribute labels during training is necessary to effectively protect intersectional groups . However , such labels are scarce on our ImageNet dataset , with less than 10 % of training datapoints having such information . This leaves bias mitigation algorithms prone to overfitting on the few available attribute labels . Second , bias mitigation algorithms often have a model size and complexity that scale linearly with the number of protected groups , which , in turn , explode combinatorially due to intersectionality . We address these challenges by proposing Knowledge Distillation of Independent Models as Regularization ( DIR ) . DIR uses group-specific teacher classifiers to train a single student classifier that , at each datapoint , mimics the output of the teacher classifier corresponding to the datapoint ’ s groundtruth group . DIR , whose model complexity is independent of the number of protected attributes ( k ) , is an efficient alternative to existing bias mitigation algorithms which scale linearly with k. DIR also provides a regularization scheme for other bias mitigation techniques , reducing overfitting by implicitly incorporating attribute label information into learning objectives . Using DIR , we can significantly mitigate bias towards intersectional groups on ImageNet , empirically demonstrating a 22 % reduction in bias amplification over standard empirical risk minimization . 2 RELATED WORK . Algorithmic Fairness Prior works have explored algorithmic bias in a number of real-world applications . Deep learning models , including commercial image classifiers ( Buolamwini & Gebru , 2018 ) and natural langauge processing ( Alvi et al. , 2018 ; Bolukbasi et al. , 2016 ; Garg et al. , 2018 ) , have been the subject of significant scrutiny . Recent literature have proposed methods for bias mitigation ( Edwards & Storkey , 2015 ; Ramaswamy et al. , 2021 ; Ryu et al. , 2018 ; Wang et al. , 2020 ; Zhao et al. , 2017 ) , approximately optimizing metrics like worst-case accuracy ( weighted on the worst-off group ) , mean accuracy ( balanced over protected groups ) , bias amplification scores ( Zhao et al. , 2017 ) , and intersectional bias scores ( Foulds et al. , 2020 ) . Most build on existing techniques in classical fairness , such as reductions approaches , adversarial models , fairness through awareness , and importance weighting ( Agarwal et al. , 2018 ; Saerens et al. , 2002 ; Dwork et al. , 2012 ; Zhang et al. , 2018 ) . Many also borrow on techniques from robust learning ( Adragna et al. , 2020 ; Liu et al. , 2021 ; Sagawa et al. , 2020 ) , causal inference ( Arjovsky et al. , 2020 ; Creager et al. , 2021 ; Kusner et al. , 2017 ; Madras et al. , 2019 ) , and representation learning ( Pezeshki et al. , 2020 ) . Intersectional Multi-Attribute Fairness One of our contributions is replicating existing results ( Wang et al. , 2020 ; Shrestha et al. , 2021 ; Liu et al. , 2021 ; Sagawa et al. , 2020 ) and extending their empirical analysis to analagous multi-attribute settings . Shrestha et al . ( 2021 ) is the most related work of multi-attribute fairness , but only compare existing algorithms against new sets of datasets . In contrast , we focus on the ImageNet and CelebA datasets and observe algorithms as we tune the number of protected attributes ( on a fixed dataset ) . We reach conflicting findings with Shrestha et al . ( 2021 ) . While methods that do not access protected attribute labels may appear to be effective in certain single-attribute settings , their performance deteriorates in multi-attribute settings . We also find that a reweighting baseline is competitive with methods proposed by Wang et al . ( 2020 ) . Beyond the deep learning settings considered by our paper and the aforementioned literature , Kang et al . ( 2021 ) considers a variational method for multi-attribute fairness in classical learning settings . Fairness with Partial Unknown Attributes In many settings , it may not be practical to label the protected attributes of all datapoints . The setting where only part of the dataset has attribute labels has been explored in previous works : Dai & Wang ( 2021 ) targets a specific graph neural network setting and Ho et al . ( 2020 ) provides a basic analysis of adversarial fairness . Some works have looked at the case where no labels are available ( Chen et al. , 2019 ; Hashimoto et al. , 2018 ) . Others have looked at learning proxy labels from a different task ( Kallus et al. , 2020 ; Awasthi et al. , 2021 ) . Our work focuses on the partial unknown attributes case and provides an in-depth analysis with extensive experiments and larger coverage of methodologies . 3 PRELIMINARIES . We formalize our problem as learning a model h : X → Y , where X is the input space and Y a finite label space . We seek to learn a h that is fair with respect to am-dimensional protected attribute vector A where m is the number of protected attributes . Fairness datasets are thus in tuples ( x , a , y ) with image x ∼ X , protected attribute vector a ∈ A , and single/multi-label label y ∈ Y . For example , a protected attribute vector might look like a = ( male , caucasian , teenager , not veteran ) . For the entirety of this paper , “ labeled/unlabeled ” refers to whether the protected attribute labels A are available—we always assume the input features X and class labels Y are available . Unlabeled data refers to datapoints where X and Y are available , but A is not . To be fair with respect to the protected attributesA , we want to be fair to the intersectional groupsG arising from A . The number of such groups is generally combinatorial in m. An example is shown in Figure 1 ( a ) . We formalize fairness among groups by reviewing two common definitions . First , demographic parity concerns parity in Pr ( ŷ | g ) , the probability of predicting a label ŷ ∈ Y for a datapoint belonging to a group g ∈ G. This definition of fairness requires that , over some fixed data distribution , the predictions of a model h are probabilistically independent of the protected group membership . Formally , ∀g′ ∈ G , ŷ ∈ Y : Pr ( ŷ | g ) = Pr ( ŷ | g′ ) . This objective is generally more applicable to settings where the predicted class Y is a decision ( e.g. , issuing a loan ) . Second , the equalized odds definition refines the demographic parity constraint to enforce parity in Pr ( ŷ | g , y ) , the probability of predicting a label ŷ ∈ Y , for a datapoint with true label y ∈ Y belonging to group g ∈ G. This definition enforces , for instance , equivalent accuracies for each group . Formally , ∀g′ ∈ G , ŷ , y ∈ Y : Pr ( ŷ | g , y ) = Pr ( ŷ | g′ , y ) .

Summary:  This paper provides an empirical investigation of deep learning bias mitigation methods, focusing on two problems: intersectionality and missing sensitive attribute labels. The paper evaluates several bias mitigation approaches on the Celeb dataset and on ImageNet, the latter having few sensitive attribute labels. The main takeaways from their empirical analysis is that the best performing mitigation method depends on the context and mitigation methods that operate without using the sensitive attribute do not perform well in terms of intersectionality. The paper also proposes a new distillation-based approach to resolve the runtime complexity problem that models like Domain Independent exhibit. On ImageNet their method significantly reduces bias amplification but there are no differences in group-reweighed accuracy or intersectional bias.