Adaptive Sampling for Minimax Fair Classification

1 Introduction . Machine learning ( ML ) models are increasingly being applied for automating the decision-making process in several sensitive applications , such as loan approval and employment screening . However , recent work has demonstrated that discriminatory behaviour might get encoded in the model at various stages of the ML pipeline , such as data collection , labelling , feature selection , and training , and as a result , adversely impact members of some protected groups in rather subtle ways ( Barocas and Selbst , 2016 ) . This is why ML researchers have started to introduce a large number of fairness measures to include the notion of fairness in the design of their algorithms . Some of the important measures of fairness include demographic parity ( Zemel et al. , 2013 ) , equal odds and opportunity ( Hardt et al. , 2016 ; Woodworth et al. , 2017 ) , individual fairness ( Dwork et al. , 2012 ) , and minimax fairness ( Feldman et al. , 2015 ) . The minimax fairness is particularly important in scenarios in which it is necessary to be as close as possible to equality without introducing unnecessary harm ( Ustun et al. , 2019 ) . These scenarios are common in areas such as healthcare and predicting domestic violence . A measure that has been explored to achieve this goal is predictive risk disparity ( Feldman et al. , 2015 ; Chen et al. , 2018 ; Ustun et al. , 2019 ) . Instead of using the common approach of putting constraints on the norm of discrimination gaps , Martinez et al . ( 2020 ) has recently introduced the notion of minimax Pareto fairness . These are minimax classifiers that are on the Pareto frontier of prediction risk , i.e. , no decrease in the predictive risk of one group is possible without increasing the risk of another one . In this paper , we are primarily interested in the notion of minimax fairness in terms of the predictive risk . However , instead of studying the training phase of the ML pipeline , our focus is on the datacollection stage , motivated by Jo and Gebru ( 2019 ) and Holstein et al . ( 2018 ) . In particular , we study the following question : given a finite sampling budget , how should a learner construct a training set consisting of elements from different protected groups in appropriate proportions to ensure that a classifier trained on this dataset achieves minimax fairness ? Our work is motivated by the following scenario : suppose we have to learn a ML model for performing a task ( e.g. , loan approval ) for inputs belonging to different groups based on protected attributes , such ∗Electrical and Computer Engineering Department at UCSD ( { shshekha , grfields , tjavidi } @ ucsd.edu ) †Google Research ( ghavamza @ google.com ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . as race or gender . Depending upon the distribution of input-label pairs from different groups , the given task may be statistically harder for some groups . Our goal is to ensure that the eventual ML model has optimal predictive accuracy for the worst-off group . We show that , under certain technical conditions , this results in a model with comparable predictive accuracy over all groups . One approach for this would be to train separate classifiers for each group . However , this is often not possible as the group-membership information may not be available at the deployment time , or it may be forbidden by law to explicitly use the protected characteristics as an input in the prediction process ( Lipton et al. , 2017 , § 1 ) . To ensure having a higher proportion of samples from the harder groups without knowing the identity of the hard or easy groups apriori , we consider this problem in an active setting , where a learner has to incrementally construct a training set by drawing samples one at a time ( or a batch at a time ) from different groups . Towards this goal , we propose and analyze an adaptive sampling scheme based on the principle of optimism , used in bandits literature ( e.g. , Auer et al . 2002 ) , that detects the harder groups and populates the training set with more samples from them in an adaptive manner . We also wish to note that bias in ML has multi-faceted origins and that our work here addresses dataset construction and can not account for bias introduced by model selection , the underlying data distribution , or other sources as discussed in Hooker ( 2021 ) , Suresh and Guttag ( 2019 ) . We also endeavor to ensure minimax fairness , but in some contexts another notion of fairness , such as those mentioned above , may be more appropriate or equitable . In general , application of our algorithm is not a guarantee that the resulting model is wholly without bias . Our main contributions are : 1 ) We first propose an optimistic adaptive sampling strategy , Aopt , for training set construction in Section 3 . This strategy is suited to smaller problems and admits theoretical guarantees . We then introduce a heuristic variant ofAopt in Section 4 that is more suitable to practical problems involving CNNs . 2 ) We obtain upper bounds on the convergence rate of Aopt , and show its minimax near-optimality by constructing a matching lower bound in Section 3.1 . 3 ) Finally , we demonstrate the benefits of our algorithm with empirical results on several synthetic and real-world datasets in Section 5 . Related Work . The closest work to ours is by Abernethy et al . ( 2020 ) , where they propose an -greedy adaptive sampling strategy . They present theoretical analysis under somewhat restrictive assumptions and also empirically demonstrate the benefits of their strategy over some baselines . We describe their results in more detail in Appendix C.1 , and employ the tools we develop to analyze our algorithm to perform a thorough analysis of the excess risk of their strategy under a much less restrictive set of assumptions and to show some necessary conditions on the value of their exploration parameter , . We find comparable empirical results for both algorithms given sufficient tuning of their respective hyperparameters , we report some of these results in Section 5 and compare the algorithms in Appendix C. In another related work , Anahideh et al . ( 2020 ) propose a fair adaptive sampling strategy that selects points based on a linear combination of model accuracy and fairness measures , and empirically study its performance . These results , however , do not obtain convergence rates of the excess risk of their respective methods and only offer implementations for small-scale datasets . The above results study the problem of fair classification in an active setting and target the datacollection stage of the ML pipeline . There are also works that take a passive approach to this problem and focus of the training phase of the pipeline . Agarwal et al . ( 2018 ) design a scheme for learning fair binary classifiers with fairness metrics that can be written as linear constraints involving certain conditional moments . This class of fairness metrics , however , do not contain the minimax fairness measure . Diana et al . ( 2021 ) propose a method for constructing ( randomized ) classifiers for minimax fairness w.r.t . the empirical loss calculated on the given training set . Similarly , Martinez et al . ( 2020 ) derive an algorithm for learning a Pareto optimal minimax fair classifier , under the assumption that the learner has access to the true expected loss functions . Thus , the theoretical guarantees in Diana et al . ( 2021 ) and Martinez et al . ( 2020 ) hold under the assumption of large training sets . Our work , in contrast , constructs the training set incrementally ( active setting ) from scratch while carefully taking into account the effects of the finite sample size . Finally , we note that the data-collection strategies proposed in our paper can , in principle , be combined with the training methods presented in Martinez et al . ( 2020 ) and Diana et al . ( 2021 ) to further guarantee the ( minimax ) fairness of the resulting classifier . We leave the investigation of this approach for future work . Besides data-collection and training , there have been studies in the fair ML literature on other aspects of the ML pipeline , such as pre-processing ( Celis et al. , 2020 ) , learning feature representations ( Zemel et al. , 2013 ) , post-processing ( Hardt et al. , 2016 ) , and model documentation ( Mitchell et al. , 2018 ) . We refer readers to a recent survey by Caton and Haas ( 2020 ) for detailed description of these results . 2 Problem Formulation . Consider a classification problem with the feature ( input ) space X , label set Y , and a protected attribute set Z = { z1 , . . . , zm } . For any z ∈ Z , we use Pz ( x , y ) as a shorthand for PXY |Z=z ( X = x , Y = y | Z = z ) to denote the feature-label joint distribution given that the protected feature value is Z = z . We also use Ez [ · ] as a shorthand for the expectation w.r.t . Pz . A ( non-randomized ) classifier is a mapping f : X 7→ Y , which assigns a label y ∈ Y to every feature x ∈ X . For a family of classifiers F ⊂ YX , a loss function ` : F × X × Y 7→ R , and a mixture distribution over the protected attributes π ∈ ∆m , we define the π-optimal classifier fπ as fπ ∈ arg min f∈F Eπ [ ` ( f , X , Y ) ] : = ∑ z∈Z π ( z ) Ez [ ` ( f , X , Y ) ] . ( 1 ) When π lies on the corners of the simplex ∆m , i.e. , π ( z ) = 1 for a z ∈ Z , we use the notation fz instead of fπ . For π in the interior of ∆m , it is clear that the risk of fπ on group z , i.e. , Ez [ ` ( fπ , X , Y ) ] , must be larger than the best possible classification loss for PXY |Z=z . In this paper , our goal is to develop an active sampling scheme to find the fair mixture distribution π∗ in a minimax sense ( minimizing the maximum risk among the groups ) , i.e. , π∗ ∈ arg min π∈∆m max z∈Z L ( z , fπ ) : = Ez [ ` ( fπ , X , Y ) ] . ( 2 ) The active sampling problem that we study in this paper can be formally defined as follows : Problem 1 . Suppose F denotes a family of classifiers , ` a loss function , n is a sampling budget , and O : Z 7→ X ×Y an oracle that maps any attribute z ∈ Z to a feature-label pair ( X , Y ) ∼ PXY |Z=z . The learner designs an adaptive sampling scheme A that comprises a sequence of mappings ( At ) nt=1 with At : ( X × Y ) t−1 7→ Z to adaptively query O and construct a dataset of size n. Let πn ∈ ∆m denote the resulting empirical mixture distribution over Z , where πn ( z ) = Nz , n/n and Nz , n is the number of times that A samples from Pz in n rounds . Then , the quality of the resulting dataset is measured by the excess risk , or sub-optimality , of the πn-optimal classifier fπn , i.e. , Rn ( A ) : = max z∈Z L ( z , fπn ) − max z∈Z L ( z , fπ∗ ) , ( 3 ) where π∗ is the fair mixture distribution defined by ( 2 ) . Hence , the goal of the learner is to design a strategy A which has a small excess riskRn ( A ) . Informally , the algorithm should adaptively identify the harder groups z and dedicate a larger portion of the overall budget n to sample from their distributions ( see Section 2.2 for an illustrative example ) .

Summary: The paper exploits active sampling methods to construct a training dataset that yields fair classifiers in a minimax sense: minimising the maximum risk of any of the groups of interest. The proposed method constructs a dataset iteratively by drawing samples from the group with highest upper confidence bound (UCB) for the predictive risk of the current classifier. At each iteration, the classifier is retrained (updated) on the extended dataset, and the process is repeated until a sampling budget is exhausted. The authors provide theoretical bounds for the convergence of the algorithm, which are proven to be tighter than those of previous $\epsilon$-greedy approaches.