Training individually fair ML models with sensitive subspace robustness

1 INTRODUCTION . Machine learning ( ML ) models are gradually replacing humans in high-stakes decision making roles . For example , in Philadelphia , an ML model classifies probationers as high or low-risk ( Metz & Satariano , 2020 ) . In North Carolina , “ analytics ” is used to report suspicious activity and fraud by Medicaid patients and providers ( Metz & Satariano , 2020 ) . Although ML models appear to eliminate the biases of a human decision maker , they may perpetuate or even exacerbate biases in the training data ( Barocas & Selbst , 2016 ) . Such biases are especially objectionable when it adversely affects underprivileged groups of users ( Barocas & Selbst , 2016 ) . In response , the scientific community has proposed many mathematical definitions of algorithmic fairness and approaches to ensure ML models satisfy the definitions . Unfortunately , this abundance of definitions , many of which are incompatible ( Kleinberg et al. , 2016 ; Chouldechova , 2017 ) , has hindered the adoption of this work by practitioners . There are two types of formal definitions of algorithmic fairness : group fairness and individual fairness . Most recent work on algorithmic fairness considers group fairness because it is more amenable to statistical analysis ( Ritov et al. , 2017 ) . Despite their prevalence , group notions of algorithmic fairness suffer from certain shortcomings . One of the most troubling is there are many scenarios in which an algorithm satisfies group fairness , but its output is blatantly unfair from the point of view of individual users ( Dwork et al. , 2011 ) . In this paper , we consider individual fairness instead of group fairness . Intuitively , an individually fair ML model treats similar users similarly . Formally , an ML model is a map h : X → Y , where X and Y are the input and output spaces . The leading notion of individual fairness is metric fairness ( Dwork et al. , 2011 ) ; it requires dy ( h ( x1 ) , h ( x2 ) ) ≤ Ldx ( x1 , x2 ) for all x1 , x2 ∈ X , ( 1.1 ) where dx and dy are metrics on the input and output spaces and L ≥ 0 is a Lipschitz constant . The fair metric dx encodes our intuition of which samples should be treated similarly by the ML model . We emphasize that dx ( x1 , x2 ) being small does not imply x1 and x2 are similar in all respects . Even if dx ( x1 , x2 ) is small , x1 and x2 may differ in certain problematic ways , e.g . in their protected/sensitive attributes . This is why we refer to pairs of samples x1 and x2 such that dx ( x1 , x2 ) is small as comparable instead of similar . Despite its benefits , individual fairness was dismissed as impractical because there is no widely accepted fair metric for many ML tasks . Fortunately , there is a line of recent work on learning the fair metric from data ( Ilvento , 2019 ; Wang et al. , 2019 ) . In this paper , we consider two data-driven choices of the fair metric : one for problems in which the sensitive attribute is reliably observed , and another for problems in which the sensitive attribute is unobserved ( see Appendix B ) . The rest of this paper is organized as follows . In Section 2 , we cast individual fairness as a form of robustness : robustness to certain sensitive perturbations to the inputs of an ML model . This allows us to leverage recent advances in adversarial ML to train individually fair ML models . More concretely , we develop an approach to audit ML models for violations of individual fairness that is similar to adversarial attacks ( Goodfellow et al. , 2014 ) and an approach to train ML models that passes such audits ( akin to adversarial training ( Madry et al. , 2017 ) ) . We justify the approach theoretically ( see Section 3 ) and empirically ( see Section 4 ) . 2 FAIRNESS THROUGH ( DISTRIBUTIONAL ) ROBUSTNESS . To motivate our approach , imagine an auditor investigating an ML model for unfairness . The auditor collects a set of audit data and compares the output of the ML model on comparable samples in the audit data . For example , to investigate whether a resume screening system is fair , the auditor may collect a stack of resumes and change the names on the resumes of Caucasian applicants to names more common among the African-American population . If the system performs worse on the edited resumes , then the auditor may conclude the model treats African-American applicants unfairly . Such investigations are known as correspondence studies , and a prominent example is Bertrand & Mullainathan ’ s celebrated investigation of racial discrimination in the labor market . In a correspondence study , the investigator looks for inputs that are comparable to the training examples ( the edited resumes in the resume screening example ) on which the ML model performs poorly . In the rest of this section , we formulate an optimization problem to find such inputs . 2.1 FAIR WASSERSTEIN DISTANCES . Recall X and Y are the spaces of inputs and outputs . To keep things simple , we assume that the ML task at hand is a classification task , so Y is discrete . We also assume that we have a fair metric dx of the form dx ( x1 , x2 ) 2 , 〈x1 − x2 , Σ ( x1 − x2 ) 〉 1 2 , where Σ ∈ Sd×d+ . For example , suppose we are given a set of K “ sensitive ” directions that we wish the metric to ignore ; i.e . d ( x1 , x2 ) 1 for any x1 and x2 such that x1 − x2 falls in the span of the sensitive directions . These directions may be provided by a domain expert or learned from data ( see Section 4 and Appendix B ) . In this case , we may choose Σ as the orthogonal complement projector of the span of the sensitive directions . We equip X with the fair metric and Z , X × Y with dz ( ( x1 , y1 ) , ( x2 , y2 ) ) , dx ( x1 , x2 ) +∞ · 1 { y1 6= y2 } . We consider d2z as a transport cost function on Z . This cost function encodes our intuition of which samples are comparable for the ML task at hand . We equip the space of probability distributions on Z with the fair Wasserstein distance W ( P , Q ) = infΠ∈C ( P , Q ) ∫ Z×Z c ( z1 , z2 ) dΠ ( z1 , z2 ) , where C ( P , Q ) is the set of couplings between P and Q . The fair Wasserstein distance inherits our intuition of which samples are comparable through the cost function ; i.e . the fair Wasserstein distance between two probability distributions is small if they are supported on comparable areas of the sample space . 2.2 AUDITING ML MODELS FOR ALGORITHMIC BIAS . To investigate whether an ML model performs disparately on comparable samples , the auditor collects a set of audit data { ( xi , yi ) } ni=1 and solves the optimization problem maxP : W ( P , Pn ) ≤ ∫ Z ` ( z , h ) dP ( z ) , ( 2.1 ) where ` : Z×H → R+ is a loss function , h is the ML model , Pn is the empirical distribution of the audit data , and > 0 is a small tolerance parameter . We interpret as a moving budget that the auditor may expend to discover discrepancies in the performance of the ML model . This budget forces the auditor to avoid moving samples to incomparable areas of the sample space . We emphasize that equation 2.1 detects aggregate violations of individual fairness . In other words , although the violations that the auditor ’ s problem detects are individual in nature , the auditor ’ s problem is only able to detect aggregate violations . We summarize the implicit notion of fairness in equation 2.1 in a definition . Definition 2.1 ( distributionally robustly fair ( DRF ) ) . An ML model h : X → Y is ( , δ ) distributionally robustly fair ( DRF ) WRT the fair metric dx iff maxP : W ( P , Pn ) ≤ ∫ Z ` ( z , h ) dP ( z ) ≤ δ . ( 2.2 ) Although equation 2.1 is an infinite-dimensional optimization problem , it is possible to solve it exactly by appealing to duality . Blanchet & Murthy showed that the dual of equation 2.1 is supP : W ( P , Pn ) ≤ EP [ ` ( Z , h ) ] = infλ≥0 { λ + EPn [ ` cλ ( Z , h ) ] } , ` cλ ( ( xi , yi ) , h ) , supx∈X ` ( ( x , yi ) , θ ) − λdx ( x , xi ) . ( 2.3 ) This is a univariate optimization problem , and it is amenable to stochastic optimization . We describe a stochastic approximation algorithm for equation 2.3 in Algorithm 1 . Inspecting the algorithm , we see that it is similar to the PGD algorithm for adversarial attack . Algorithm 1 stochastic gradient method for equation 2.3 Require : starting point λ̂1 , step sizes αt > 0 1 : repeat 2 : draw mini-batch ( xt1 , yt1 ) , . . . , ( xtB , ytB ) ∼ Pn 3 : x∗tb ← arg maxx∈X ` ( ( x , ytb ) , h ) − λdx ( xtb , x ) , b ∈ [ B ] 4 : λ̂t+1 ← max { 0 , λ̂t − αt ( − 1B ∑B b=1 dx ( xtb , x ∗ tb ) ) } 5 : until converged It is known that the optimal point of equation 2.1 is the discrete measure 1n ∑n i=1 δ ( Tλ ( xi ) , yi ) , where Tλ : X → X is the unfair map Tλ ( xi ) ← arg maxx∈X ` ( ( x , yi ) , h ) − λd2x ( x , xi ) . ( 2.4 ) We call Tλ an unfair map because it reveals unfairness in the ML model by mapping samples in the audit data to comparable areas of the sample space that the system performs poorly on . We note that Tλ may map samples in the audit data to areas of the sample space that are not represented in the audit data , thereby revealing disparate treatment in the ML model not visible in the audit data alone . We emphasize that Tλ more than reveals disparate treatment in the ML model ; it localizes the unfairness to certain areas of the sample space . We present a simple example to illustrating fairness through robustness ( a similar example appeared in Hashimoto et al . ( 2018 ) ) . Consider the binary classification dataset shown in Figure 1 . There are two subgroups of observations in this dataset , and ( sub ) group membership is the protected attribute ( e.g . the smaller group contains observations from a minority subgroup ) . In Figure 1a we see the decision heatmap of a vanilla logistic regression , which performs poorly on the blue minority subgroup . The two subgroups are separated in the horizontal direction , so the horizontal direction is the sensitive direction . Figure 1b shows that such classifier is unfair with respect to the corresponding fair metric , i.e . the unfair map equation 2.4 leads to significant loss increase by transporting mass along the horizontal direction with very minor change of the vertical coordinate . Comparison with metric fairness Before moving on to training individually fair ML models , we compare DRF with metric fairness equation 1.1 . Although we concentrate on the differences between the two definitions here , they are more similar than different : both formalize the intuition that the outputs of a fair ML model should perform similarly on comparable inputs . That said , there are two main differences between the two definitions . First , instead of requiring the output of the ML model to be similar on all inputs comparable to a training example , we require the output to be similar to the training label . Thus DRF not only enforces similarity of the output on comparable inputs , but also accuracy of the ML model on the training data . Second , DRF considers differences between datasets instead of samples by replacing the fair metric on inputs with the fair Wasserstein distance induced by the fair metric . The main benefits of this modifications are ( i ) it is possible to optimize equation 2.1 efficiently , ( ii ) we can show this modified notion of individual fairness generalizes .

Summary: This paper proposes a new definition of algorithmic fairness that is based on the idea of individual fairness. They then present an algorithm that will provably find an ML model that satisfies the fairness constraint (if such a model exists in the search space). One needed ingredient for the fairness constraint is a distance function (or "metric") in the input space that captures the fact that some features should be irrelevant to the classification task. That is, under this distance function, input that differ only in sensitive attributes like race or gender should be close-by. The idea of the fairness constraint is that by perturbing the inputs (while keeping them close with respect to the distance function), the loss of the model cannot be significantly increased. Thus, this fairness constraint is very much related to robustness.