Pareto Optimality in No-Harm Fairness

1 INTRODUCTION . Machine learning algorithms play an important role in decision making in society . When these algorithms are used to make high-impact decisions such as hiring , credit-lending , predicting mortality for intensive care unit patients , or classifying benign/malign skin lesions , it is paramount to guarantee that these decisions are both accurate and unbiased with respect to sensitive attributes such as gender or ethnicity . A model that is trained naively may not have these properties by default ; see , for example Barocas & Selbst ( 2016 ) . In these critical applications , it is desirable to impose some fairness criteria . Much of the fairness in machine learning literature attempts to produce algorithms that satisfy Demographic Parity , which aims to make algorithm ’ s predictions independent of the sensitive populations ( Louizos et al . ( 2015 ) ; Zemel et al . ( 2013 ) ; Feldman et al . ( 2015 ) ) ; or Equality of Odds or Equality of Opportunity , which aims to produce predictions that are independent of the sensitive attributes given the ground truth ( Hardt et al . ( 2016 ) ; Woodworth et al . ( 2017 ) ) . Notions of Individual Fairness have also been advanced ( Dwork et al . ( 2012 ) ; Joseph et al . ( 2016 ) ; Zemel et al . ( 2013 ) ) . These notions of fairness can be appropriate in many scenarios , but in domains where quality of service is paramount , such as healthcare , we argue that it is necessary to strive for models that are as close to fair as possible without introducing any unnecessary harm to any subgroup ( Ustun et al . ( 2019 ) ) . Even if the overall fairness goal is a potentially harmful , zero-gap classifier , pursuing first a Pareto-fair classifier and later applying other harmful methodologies ensures that all possible non-harmful trade-offs are covered before explicitly degrading performance , therefore minimal harm is introduced to the decision . In this work we make use of the concept of Pareto optimality to measure and analyze discrimination ( unfairness ) in terms of the difference in predictive risks across sub-populations defined by our sensitive attributes , a fairness metric that has been explored in other recent works such as Calders & Verwer ( 2010 ) ; Dwork et al . ( 2012 ) ; Feldman et al . ( 2015 ) ; Chen et al . ( 2018 ) ; Ustun et al . ( 2019 ) . We examine the subset of models from our hypothesis class that have the best trade-offs between sub-population risks , and select from this set the one with the smallest risk disparity gap . This is in direct contrast to common post-hoc correction methods like the ones proposed in Hardt et al . ( 2016 ) ; Woodworth et al . ( 2017 ) , where noise is potentially added to the decisions of the best performing sub-population . While this latter type of approach diminishes the risk-disparity gap , it does so by degrading performance on advantaged groups , the previously disadvantaged groups do not directly benefit from this treatment . Since our proposed methodology does not require test-time access to sensitive attributes , and can be applied to any standard classification or regression task , it can also be used to reduce risk disparity between outcomes , acting as an adaptive risk equalization loss compatible with unbalanced classification scenarios . Main Contributions . We formalize the notion of no-harm risk fairness1 using Pareto optimality ( Mas-Colell et al . ( 1995 ) ) , a state of resource allocations from which it is impossible to reallocate without making one subgroup worse . We show that finding a Pareto-fair classifier is equivalent to finding a model in our hypothesis class that belongs to the Pareto front ( the set of all Pareto optimal allocations ) with respect to the sub-population risks with the smallest possible risk disparity . This general notion is already amenable to non-binary sensitive attributes . We analyze the fairness performance trade-offs that can be expected from different approaches with an illustrative example . We provide a concrete algorithm that promotes fair solutions belonging to the Pareto front ; this algorithm can be applied to any standard classifier or regression task that can be trained using ( Stochastic ) Gradient Descent . We show that if the goal is to obtain a zero-gap classifier , first recovering the fairest Pareto optimal solution and then adding harmful post-hoc corrections ensures the lowest risk levels across all subgroups . We demonstrate how our methodology performs on synthetic and real tasks such as inferring income status in the Adult dataset Dua & Graff ( 2017a ) irrespective of their ethnicity or gender , predicting ICU mortality rates in the MIMIC-III dataset from hospital notes Johnson et al . ( 2016 ) , classifying skin lesions in the HAM10000 dataset Tschandl et al . ( 2018 ) , and assessing credit risk on the German Credit dataset Dua & Graff ( 2017b ) . 2 RELATED WORK . There is an extensive body of work on fairness in machine learning . Following Friedler et al . ( 2019 ) , we compare our methodology against the works of Feldman et al . ( 2015 ) ; Kamishima et al . ( 2012 ) ; Zafar et al . ( 2017 ) . Our method shares conceptual similarities with Zafar et al . ( 2017 ) ; Woodworth et al . ( 2017 ) ; Agarwal et al . ( 2018 ) , with differences on how we define our fairness objective and adapt it to work with standard neural networks . Although optimality is often discussed in the fairness literature , it is usually in the context of utility-discrimination tradeoffs . To the best of our knowledge , this is the first work to discuss optimality with respect to subgroup risks on a unified classifier , a distinction that disallows extreme performance degradation in the pursuit of fairness . The work presented in Hashimoto et al . ( 2018 ) discusses decoupled classifiers as a way of minimizing group-risk disparity , but simultaneously cautions against this methodology when presented with insufficiently large datasets . The works of Chen et al . ( 2018 ) ; Ustun et al . ( 2019 ) also empirically report the disadvantages of decoupled classifiers as a way to mitigate risk disparity . Here we argue for the use of a single classifier because it allows transfer learning between diverse sub-populations . We do not need access to the sensitive attribute during test time , but in cases where this is possible , we instead choose to incorporate it as part of our observation features . The work of Chen et al . ( 2018 ) uses the unified bias-variance decomposition advanced in Domingos ( 2000 ) to identify that noise levels across different sub-populations may differ , making perfect fairness parity impossible without explicitly degrading performance on one subclass . Their methodology attempts to bridge the disparity gap by collecting additional samples from high-risk sub-populations . Here we modify our classifier loss to bridge the disparity gap without inducing unnecessary harm , which could prove to be synergistic with their methodology . 3 PROBLEM STATEMENT . Consider we have access to a datasetD = { ( xi , yi , ai ) } ni=1 containing n independent triplet samples drawn from a joint distribution ( xi , yi , ai ) ∼ P ( X , Y , A ) where xi ∈ X are our input features ( e.g. , images , tabular data , etc . ) , yi ∈ Y is our target variable , and ai ∈ A indicates group membership or sensitive status ( e.g. , ethnicity , gender ) ; our input features X may or may not explicitly contain A . Let h ∈ H be a classifier from a compact hypothesis class H trained to infer y from x , h : X → Y ; and a loss function ` : Y ×Y → R. We define the class-specific risk of classifier h on subgroup a as Ra ( h ) = EX , Y |A=a [ ` ( h ( X ) , Y ) ] . The risk discrimination gap between two subgroups a , a′ ∈ A is 1In this paper we use “ no-harm ” to mean “ no-unnecessary-harm , ” in other words , the system doesn ’ t degrade performance on any class unless it is strictly necessary to improve performance in a disadvantaged class . measured as Γa , a′ ( h ) = |Ra ( h ) −Ra′ ( h ) | , and we define the pairwise discrimination gap vector as ~ΓA ( h ) = { Γa , a′ ( h ) } a , a′∈A . Our goal is to obtain a classifier h ∈ H that minimizes this gap without causing unnecessary harm to any particular group in A . To formalize this notion , we define : Definition 3.1 . Dominant risk vector : A vector r′ ∈ Rk is said to dominate r ∈ Rk , noted as r r′ , if ri ≥ r′i , ∀i = 1 , ... , k and ∃j : rj > r′j ( i.e. , strict inequality on at least one component ) . Definition 3.2 . Dominant risk classifier : Classifier h′ is said to dominate h , noted as h h′ , if the risks vector r′ = { Ra ( h′ ) } |A|a=1 dominates r = { Ra ( h ) } |A| a=1 . Definition 3.3 . Pareto front : We define the Pareto front as P ( H , A ) = { h ∈ H : @ h′ ∈ H | h h′ } . This means that there is no other classifier in H that is at least as good in all risks and strictly better in at least one of them . It is the set of classifiers such that improving one group ’ s risk comes at the cost of increasing other ’ s . The Pareto front defines the best achievable trade-offs between population risks Ra ( h ) . This definition is already suited for classification and regression tasks where the sensitive attributes are categorical . Constraining the classifier to be in the Pareto front disallows laziness , there exists no other classifier in the hypothesis classH that is at least as good on all class-specific risks and strictly better in one of them . As shown in Chen et al . ( 2018 ) ; Domingos ( 2000 ) , the risk can be decomposed in bias , variance and noise for some loss functions , where the noise is the smallest achievable risk for infinitely large datasets ( Bayes-optimal risk ) . If the noise differs between sensitive groups , zero-discrimination ( perfect fairness ) can only be achieved by introducing bias or variance . Literature on fairness has focused on putting constraints on the norm of discrimination gaps ( Zafar et al . ( 2017 ; 2015 ) ; Creager et al . ( 2019 ) ; Woodworth et al . ( 2017 ) ) . We follow a similar criteria in Definition 3.4 and define the Pareto-fair classifier as the classifier in the Pareto front that minimizes ||~ΓA ( h ) ||∞ ( the maximum risk discrimination gap ) . Note that one could alternatively choose to find the Pareto classifier that minimizes the maximum subgroup risk . Definition 3.4 . Pareto-fair classifier and Pareto-fair vector : A classifier h∗ is an optimal Paretofair classifier if it minimizes the discrimination gap among all Pareto front classifiers , h∗ = arg min h∈P ( H , A ) ||~ΓA ( h ) ||∞ . The Pareto-fair vector r∗ ∈ R|A| is defined as r∗ = { Ra ( h∗ ) } a∈A . Even when perfect equality of risk is desirable , Pareto classifiers still serve as useful intermediaries . To this end , Lemma 3.1 , shows that applying a mechanism for reaching equality of risk on a dominated classifier h ∈ H leads to equal or worse risks than applying it to a Pareto classifier hp ∈ P ( H , A ) that dominates h. Lemma 3.1 . If h 6∈ P ( H , A ) → ∃hp ∈ P ( H , A ) : hp h ∧ Ra ( hERp ) ≤ Ra ( hER ) ∀a , with hER an equal-risk classifier : Ra ( hER ) = max a′∈A Ra′ ( h ) , ∀a and hERp : Ra ( hERp ) = max a′∈A Ra′ ( hp ) . To exemplify these notions graphically , Figure 1 shows a scenario with binary sensitive attributes a and binary output variable y where none of the Pareto front classifiers achieve equality of risk . Here the noise level differs between subgroups , and the Pareto-fair vector r∗ is not achieved by either a Naive classifier ( minimizes expected global risk ) , or a classifier where subgroups are re-sampled to appear with equal probability ( rebalanced Naive classifier ) . Note that the amount of performance degradation required to enforce perfect fairness starting from the Naive classifier is higher than when starting from the Pareto-fair vector . Our objective is to find the Pareto-fair classifier h∗ as in Definition 3.4 . In particular , we will give regularity conditions on the hypothesis class H and the loss function ` ( h ( X ) , Y ) . It can be shown that for a sufficiently rich class of hypothesis functions H and for risk functions Ra ( h ) that are convex with respect to h , the space of risk vectors is convex ( see Geoffrion ( 1968 ) ; Koski ( 1985 ) ; Miettinen ( 2012 ) ) . Under these conditions , we can find an auxiliary loss function φ : R|A| → R defined in terms of the subgroup risks { Ra ( h ) } a∈A ( denoted from here on as r ∈ R|A| ) that has a global minima on r∗ . To prove the existence of loss function φ with the desired global minimum r∗ , it is convenient to think of the convex set of risk vectors as the intersection of a convex Pareto set ( defined by a convex Pareto front and all risks that are dominated by it ) , and an additional convex set Ω . We further require both this Pareto set and Ω to be smooth , so that we can apply standard tools from smooth convex optimization and prove that for every risk vector r′ in the Pareto front there exists a function φ that has r′ as its global optima . This is formalized in Lemma 3.3 . Definition 3.5 . Pareto field : A function P : Ω → R is a Pareto field over a convex set Ω ⊂ Rk if P ∈ C1 is a continuously differentiable function such that∇iP ( r ) > 0 ∀r ∈ Ω , ∀i = 1 , . . . , k. Lemma 3.2 . Let Ω ⊂ Rk be a convex set , and P : Ω → R a Pareto field . Then the set D = { r ∈ Ω : P ( r ) = 0 } is a Pareto set in Ω , and the set D+ = { r ∈ Ω : P ( r ) > 0 } is the set of dominated points , i.e. , D+ = { r ∈ Ω : ∃r′ ∈ D | r r′ } Lemma 3.3 . Let Ω ⊂ Rk be a convex set defined by Ω = { r ∈ Rk : gc ( r ) ≥ 0 ∀ c ∈ { 1 , . . . , C } , gc continuously differentiable } ; let P : Ω → R be a convex Pareto field with corresponding proper Pareto set D = { r ∈ Ω : P ( r ) = 0 } . Let r̂ ∈ D and φ ( r , µ ) =∑k i=1 ri + µi ( ri − c ) +2 , with c < r̂ , where we denote ( x ) +2 = max2 ( x , 0 ) . There exists a set of vectors µ̂ 0 such that : r̂ = arg min r∈Rk φ ( r , µ̂ ) s.t . : P ( r ) ≥ 0 , gc ( r ) ≥ 0∀c ∈ { 1 , . . . , C } . Where C is the number of constraints that characterized Ω . Proofs for all lemmas are given in the supplementary material , Section A.1 . Lemma 3.3 motivates our loss function to take the form φ ( r , µ ) = ∑|A| i=1 ri+µi ( ri−c ) +2 , with |A| the number of sensitive groups . The challenge is to find µ∗ such that our Pareto-fair vector r∗ minimizes φ ( r , µ∗ ) . In Section 4 we provide an algorithm to approximately recover the Pareto-fair classifier h∗ by adaptively searching for µ∗ and optimizing a classifier on φ ( · , µ ) using tools from standard Stochastic Gradient Descent .

Summary: This paper introduces a new kind of algorithmic fairness framework where the focus is on first finding a fair classifier that does "no harm" and then in a subsequent step potentially allow doing harm in order to achieve even fairer outcomes. Fairness is here understood as risk disparity: how different are the risks achieved by our model in the various subgroups. The risk is task-dependent and can be something like a cross-entropy loss for classification problems. The goal is to have similar risks in the subgroups that correspond to sensitive attributes.