Adaptive Sampling for Minimax Fair Classification

1 Introduction . Machine learning ( ML ) models are increasingly being applied for automating the decision-making process in several sensitive applications , such as loan approval and employment screening . However , recent work has demonstrated that discriminatory behaviour might get encoded in the model at various stages of the ML pipeline , such as data collection , labelling , feature selection , and training , and as a result , adversely impact members of some protected groups in rather subtle ways ( Barocas and Selbst , 2016 ) . This is why ML researchers have started to introduce a large number of fairness measures to include the notion of fairness in the design of their algorithms . Some of the important measures of fairness include demographic parity ( Zemel et al. , 2013 ) , equal odds and opportunity ( Hardt et al. , 2016 ; Woodworth et al. , 2017 ) , individual fairness ( Dwork et al. , 2012 ) , and minimax fairness ( Feldman et al. , 2015 ) . The minimax fairness is particularly important in scenarios in which it is necessary to be as close as possible to equality without introducing unnecessary harm ( Ustun et al. , 2019 ) . These scenarios are common in areas such as healthcare and predicting domestic violence . A measure that has been explored to achieve this goal is predictive risk disparity ( Feldman et al. , 2015 ; Chen et al. , 2018 ; Ustun et al. , 2019 ) . Instead of using the common approach of putting constraints on the norm of discrimination gaps , Martinez et al . ( 2020 ) has recently introduced the notion of minimax Pareto fairness . These are minimax classifiers that are on the Pareto frontier of prediction risk , i.e. , no decrease in the predictive risk of one group is possible without increasing the risk of another one . In this paper , we are primarily interested in the notion of minimax fairness in terms of the predictive risk . However , instead of studying the training phase of the ML pipeline , our focus is on the datacollection stage , motivated by Jo and Gebru ( 2019 ) and Holstein et al . ( 2018 ) . In particular , we study the following question : given a finite sampling budget , how should a learner construct a training set consisting of elements from different protected groups in appropriate proportions to ensure that a classifier trained on this dataset achieves minimax fairness ? Our work is motivated by the following scenario : suppose we have to learn a ML model for performing a task ( e.g. , loan approval ) for inputs belonging to different groups based on protected attributes , such ∗Electrical and Computer Engineering Department at UCSD ( { shshekha , grfields , tjavidi } @ ucsd.edu ) †Google Research ( ghavamza @ google.com ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . as race or gender . Depending upon the distribution of input-label pairs from different groups , the given task may be statistically harder for some groups . Our goal is to ensure that the eventual ML model has optimal predictive accuracy for the worst-off group . We show that , under certain technical conditions , this results in a model with comparable predictive accuracy over all groups . One approach for this would be to train separate classifiers for each group . However , this is often not possible as the group-membership information may not be available at the deployment time , or it may be forbidden by law to explicitly use the protected characteristics as an input in the prediction process ( Lipton et al. , 2017 , § 1 ) . To ensure having a higher proportion of samples from the harder groups without knowing the identity of the hard or easy groups apriori , we consider this problem in an active setting , where a learner has to incrementally construct a training set by drawing samples one at a time ( or a batch at a time ) from different groups . Towards this goal , we propose and analyze an adaptive sampling scheme based on the principle of optimism , used in bandits literature ( e.g. , Auer et al . 2002 ) , that detects the harder groups and populates the training set with more samples from them in an adaptive manner . We also wish to note that bias in ML has multi-faceted origins and that our work here addresses dataset construction and can not account for bias introduced by model selection , the underlying data distribution , or other sources as discussed in Hooker ( 2021 ) , Suresh and Guttag ( 2019 ) . We also endeavor to ensure minimax fairness , but in some contexts another notion of fairness , such as those mentioned above , may be more appropriate or equitable . In general , application of our algorithm is not a guarantee that the resulting model is wholly without bias . Our main contributions are : 1 ) We first propose an optimistic adaptive sampling strategy , Aopt , for training set construction in Section 3 . This strategy is suited to smaller problems and admits theoretical guarantees . We then introduce a heuristic variant ofAopt in Section 4 that is more suitable to practical problems involving CNNs . 2 ) We obtain upper bounds on the convergence rate of Aopt , and show its minimax near-optimality by constructing a matching lower bound in Section 3.1 . 3 ) Finally , we demonstrate the benefits of our algorithm with empirical results on several synthetic and real-world datasets in Section 5 . Related Work . The closest work to ours is by Abernethy et al . ( 2020 ) , where they propose an -greedy adaptive sampling strategy . They present theoretical analysis under somewhat restrictive assumptions and also empirically demonstrate the benefits of their strategy over some baselines . We describe their results in more detail in Appendix C.1 , and employ the tools we develop to analyze our algorithm to perform a thorough analysis of the excess risk of their strategy under a much less restrictive set of assumptions and to show some necessary conditions on the value of their exploration parameter , . We find comparable empirical results for both algorithms given sufficient tuning of their respective hyperparameters , we report some of these results in Section 5 and compare the algorithms in Appendix C. In another related work , Anahideh et al . ( 2020 ) propose a fair adaptive sampling strategy that selects points based on a linear combination of model accuracy and fairness measures , and empirically study its performance . These results , however , do not obtain convergence rates of the excess risk of their respective methods and only offer implementations for small-scale datasets . The above results study the problem of fair classification in an active setting and target the datacollection stage of the ML pipeline . There are also works that take a passive approach to this problem and focus of the training phase of the pipeline . Agarwal et al . ( 2018 ) design a scheme for learning fair binary classifiers with fairness metrics that can be written as linear constraints involving certain conditional moments . This class of fairness metrics , however , do not contain the minimax fairness measure . Diana et al . ( 2021 ) propose a method for constructing ( randomized ) classifiers for minimax fairness w.r.t . the empirical loss calculated on the given training set . Similarly , Martinez et al . ( 2020 ) derive an algorithm for learning a Pareto optimal minimax fair classifier , under the assumption that the learner has access to the true expected loss functions . Thus , the theoretical guarantees in Diana et al . ( 2021 ) and Martinez et al . ( 2020 ) hold under the assumption of large training sets . Our work , in contrast , constructs the training set incrementally ( active setting ) from scratch while carefully taking into account the effects of the finite sample size . Finally , we note that the data-collection strategies proposed in our paper can , in principle , be combined with the training methods presented in Martinez et al . ( 2020 ) and Diana et al . ( 2021 ) to further guarantee the ( minimax ) fairness of the resulting classifier . We leave the investigation of this approach for future work . Besides data-collection and training , there have been studies in the fair ML literature on other aspects of the ML pipeline , such as pre-processing ( Celis et al. , 2020 ) , learning feature representations ( Zemel et al. , 2013 ) , post-processing ( Hardt et al. , 2016 ) , and model documentation ( Mitchell et al. , 2018 ) . We refer readers to a recent survey by Caton and Haas ( 2020 ) for detailed description of these results . 2 Problem Formulation . Consider a classification problem with the feature ( input ) space X , label set Y , and a protected attribute set Z = { z1 , . . . , zm } . For any z ∈ Z , we use Pz ( x , y ) as a shorthand for PXY |Z=z ( X = x , Y = y | Z = z ) to denote the feature-label joint distribution given that the protected feature value is Z = z . We also use Ez [ · ] as a shorthand for the expectation w.r.t . Pz . A ( non-randomized ) classifier is a mapping f : X 7→ Y , which assigns a label y ∈ Y to every feature x ∈ X . For a family of classifiers F ⊂ YX , a loss function ` : F × X × Y 7→ R , and a mixture distribution over the protected attributes π ∈ ∆m , we define the π-optimal classifier fπ as fπ ∈ arg min f∈F Eπ [ ` ( f , X , Y ) ] : = ∑ z∈Z π ( z ) Ez [ ` ( f , X , Y ) ] . ( 1 ) When π lies on the corners of the simplex ∆m , i.e. , π ( z ) = 1 for a z ∈ Z , we use the notation fz instead of fπ . For π in the interior of ∆m , it is clear that the risk of fπ on group z , i.e. , Ez [ ` ( fπ , X , Y ) ] , must be larger than the best possible classification loss for PXY |Z=z . In this paper , our goal is to develop an active sampling scheme to find the fair mixture distribution π∗ in a minimax sense ( minimizing the maximum risk among the groups ) , i.e. , π∗ ∈ arg min π∈∆m max z∈Z L ( z , fπ ) : = Ez [ ` ( fπ , X , Y ) ] . ( 2 ) The active sampling problem that we study in this paper can be formally defined as follows : Problem 1 . Suppose F denotes a family of classifiers , ` a loss function , n is a sampling budget , and O : Z 7→ X ×Y an oracle that maps any attribute z ∈ Z to a feature-label pair ( X , Y ) ∼ PXY |Z=z . The learner designs an adaptive sampling scheme A that comprises a sequence of mappings ( At ) nt=1 with At : ( X × Y ) t−1 7→ Z to adaptively query O and construct a dataset of size n. Let πn ∈ ∆m denote the resulting empirical mixture distribution over Z , where πn ( z ) = Nz , n/n and Nz , n is the number of times that A samples from Pz in n rounds . Then , the quality of the resulting dataset is measured by the excess risk , or sub-optimality , of the πn-optimal classifier fπn , i.e. , Rn ( A ) : = max z∈Z L ( z , fπn ) − max z∈Z L ( z , fπ∗ ) , ( 3 ) where π∗ is the fair mixture distribution defined by ( 2 ) . Hence , the goal of the learner is to design a strategy A which has a small excess riskRn ( A ) . Informally , the algorithm should adaptively identify the harder groups z and dedicate a larger portion of the overall budget n to sample from their distributions ( see Section 2.2 for an illustrative example ) .

Summary: This paper proposes an adaptive sampling scheme to construct the training dataset which allows us to learn classifiers that are fair in a minimax sense. They proved theoretical results on the excess risk of their algorithm, and also proposed heuristic extensions of the algorithm to be applicable to real-world large scale problems. They validate their algorithm on a series of synthetic and real-world datasets, showing an improved performance over empirical/uniform sampling schemes, which also verified their theoretical findings.