Pareto Optimality in No-Harm Fairness

1 INTRODUCTION . Machine learning algorithms play an important role in decision making in society . When these algorithms are used to make high-impact decisions such as hiring , credit-lending , predicting mortality for intensive care unit patients , or classifying benign/malign skin lesions , it is paramount to guarantee that these decisions are both accurate and unbiased with respect to sensitive attributes such as gender or ethnicity . A model that is trained naively may not have these properties by default ; see , for example Barocas & Selbst ( 2016 ) . In these critical applications , it is desirable to impose some fairness criteria . Much of the fairness in machine learning literature attempts to produce algorithms that satisfy Demographic Parity , which aims to make algorithm ’ s predictions independent of the sensitive populations ( Louizos et al . ( 2015 ) ; Zemel et al . ( 2013 ) ; Feldman et al . ( 2015 ) ) ; or Equality of Odds or Equality of Opportunity , which aims to produce predictions that are independent of the sensitive attributes given the ground truth ( Hardt et al . ( 2016 ) ; Woodworth et al . ( 2017 ) ) . Notions of Individual Fairness have also been advanced ( Dwork et al . ( 2012 ) ; Joseph et al . ( 2016 ) ; Zemel et al . ( 2013 ) ) . These notions of fairness can be appropriate in many scenarios , but in domains where quality of service is paramount , such as healthcare , we argue that it is necessary to strive for models that are as close to fair as possible without introducing any unnecessary harm to any subgroup ( Ustun et al . ( 2019 ) ) . Even if the overall fairness goal is a potentially harmful , zero-gap classifier , pursuing first a Pareto-fair classifier and later applying other harmful methodologies ensures that all possible non-harmful trade-offs are covered before explicitly degrading performance , therefore minimal harm is introduced to the decision . In this work we make use of the concept of Pareto optimality to measure and analyze discrimination ( unfairness ) in terms of the difference in predictive risks across sub-populations defined by our sensitive attributes , a fairness metric that has been explored in other recent works such as Calders & Verwer ( 2010 ) ; Dwork et al . ( 2012 ) ; Feldman et al . ( 2015 ) ; Chen et al . ( 2018 ) ; Ustun et al . ( 2019 ) . We examine the subset of models from our hypothesis class that have the best trade-offs between sub-population risks , and select from this set the one with the smallest risk disparity gap . This is in direct contrast to common post-hoc correction methods like the ones proposed in Hardt et al . ( 2016 ) ; Woodworth et al . ( 2017 ) , where noise is potentially added to the decisions of the best performing sub-population . While this latter type of approach diminishes the risk-disparity gap , it does so by degrading performance on advantaged groups , the previously disadvantaged groups do not directly benefit from this treatment . Since our proposed methodology does not require test-time access to sensitive attributes , and can be applied to any standard classification or regression task , it can also be used to reduce risk disparity between outcomes , acting as an adaptive risk equalization loss compatible with unbalanced classification scenarios . Main Contributions . We formalize the notion of no-harm risk fairness1 using Pareto optimality ( Mas-Colell et al . ( 1995 ) ) , a state of resource allocations from which it is impossible to reallocate without making one subgroup worse . We show that finding a Pareto-fair classifier is equivalent to finding a model in our hypothesis class that belongs to the Pareto front ( the set of all Pareto optimal allocations ) with respect to the sub-population risks with the smallest possible risk disparity . This general notion is already amenable to non-binary sensitive attributes . We analyze the fairness performance trade-offs that can be expected from different approaches with an illustrative example . We provide a concrete algorithm that promotes fair solutions belonging to the Pareto front ; this algorithm can be applied to any standard classifier or regression task that can be trained using ( Stochastic ) Gradient Descent . We show that if the goal is to obtain a zero-gap classifier , first recovering the fairest Pareto optimal solution and then adding harmful post-hoc corrections ensures the lowest risk levels across all subgroups . We demonstrate how our methodology performs on synthetic and real tasks such as inferring income status in the Adult dataset Dua & Graff ( 2017a ) irrespective of their ethnicity or gender , predicting ICU mortality rates in the MIMIC-III dataset from hospital notes Johnson et al . ( 2016 ) , classifying skin lesions in the HAM10000 dataset Tschandl et al . ( 2018 ) , and assessing credit risk on the German Credit dataset Dua & Graff ( 2017b ) . 2 RELATED WORK . There is an extensive body of work on fairness in machine learning . Following Friedler et al . ( 2019 ) , we compare our methodology against the works of Feldman et al . ( 2015 ) ; Kamishima et al . ( 2012 ) ; Zafar et al . ( 2017 ) . Our method shares conceptual similarities with Zafar et al . ( 2017 ) ; Woodworth et al . ( 2017 ) ; Agarwal et al . ( 2018 ) , with differences on how we define our fairness objective and adapt it to work with standard neural networks . Although optimality is often discussed in the fairness literature , it is usually in the context of utility-discrimination tradeoffs . To the best of our knowledge , this is the first work to discuss optimality with respect to subgroup risks on a unified classifier , a distinction that disallows extreme performance degradation in the pursuit of fairness . The work presented in Hashimoto et al . ( 2018 ) discusses decoupled classifiers as a way of minimizing group-risk disparity , but simultaneously cautions against this methodology when presented with insufficiently large datasets . The works of Chen et al . ( 2018 ) ; Ustun et al . ( 2019 ) also empirically report the disadvantages of decoupled classifiers as a way to mitigate risk disparity . Here we argue for the use of a single classifier because it allows transfer learning between diverse sub-populations . We do not need access to the sensitive attribute during test time , but in cases where this is possible , we instead choose to incorporate it as part of our observation features . The work of Chen et al . ( 2018 ) uses the unified bias-variance decomposition advanced in Domingos ( 2000 ) to identify that noise levels across different sub-populations may differ , making perfect fairness parity impossible without explicitly degrading performance on one subclass . Their methodology attempts to bridge the disparity gap by collecting additional samples from high-risk sub-populations . Here we modify our classifier loss to bridge the disparity gap without inducing unnecessary harm , which could prove to be synergistic with their methodology . 3 PROBLEM STATEMENT . Consider we have access to a datasetD = { ( xi , yi , ai ) } ni=1 containing n independent triplet samples drawn from a joint distribution ( xi , yi , ai ) ∼ P ( X , Y , A ) where xi ∈ X are our input features ( e.g. , images , tabular data , etc . ) , yi ∈ Y is our target variable , and ai ∈ A indicates group membership or sensitive status ( e.g. , ethnicity , gender ) ; our input features X may or may not explicitly contain A . Let h ∈ H be a classifier from a compact hypothesis class H trained to infer y from x , h : X → Y ; and a loss function ` : Y ×Y → R. We define the class-specific risk of classifier h on subgroup a as Ra ( h ) = EX , Y |A=a [ ` ( h ( X ) , Y ) ] . The risk discrimination gap between two subgroups a , a′ ∈ A is 1In this paper we use “ no-harm ” to mean “ no-unnecessary-harm , ” in other words , the system doesn ’ t degrade performance on any class unless it is strictly necessary to improve performance in a disadvantaged class . measured as Γa , a′ ( h ) = |Ra ( h ) −Ra′ ( h ) | , and we define the pairwise discrimination gap vector as ~ΓA ( h ) = { Γa , a′ ( h ) } a , a′∈A . Our goal is to obtain a classifier h ∈ H that minimizes this gap without causing unnecessary harm to any particular group in A . To formalize this notion , we define : Definition 3.1 . Dominant risk vector : A vector r′ ∈ Rk is said to dominate r ∈ Rk , noted as r r′ , if ri ≥ r′i , ∀i = 1 , ... , k and ∃j : rj > r′j ( i.e. , strict inequality on at least one component ) . Definition 3.2 . Dominant risk classifier : Classifier h′ is said to dominate h , noted as h h′ , if the risks vector r′ = { Ra ( h′ ) } |A|a=1 dominates r = { Ra ( h ) } |A| a=1 . Definition 3.3 . Pareto front : We define the Pareto front as P ( H , A ) = { h ∈ H : @ h′ ∈ H | h h′ } . This means that there is no other classifier in H that is at least as good in all risks and strictly better in at least one of them . It is the set of classifiers such that improving one group ’ s risk comes at the cost of increasing other ’ s . The Pareto front defines the best achievable trade-offs between population risks Ra ( h ) . This definition is already suited for classification and regression tasks where the sensitive attributes are categorical . Constraining the classifier to be in the Pareto front disallows laziness , there exists no other classifier in the hypothesis classH that is at least as good on all class-specific risks and strictly better in one of them . As shown in Chen et al . ( 2018 ) ; Domingos ( 2000 ) , the risk can be decomposed in bias , variance and noise for some loss functions , where the noise is the smallest achievable risk for infinitely large datasets ( Bayes-optimal risk ) . If the noise differs between sensitive groups , zero-discrimination ( perfect fairness ) can only be achieved by introducing bias or variance . Literature on fairness has focused on putting constraints on the norm of discrimination gaps ( Zafar et al . ( 2017 ; 2015 ) ; Creager et al . ( 2019 ) ; Woodworth et al . ( 2017 ) ) . We follow a similar criteria in Definition 3.4 and define the Pareto-fair classifier as the classifier in the Pareto front that minimizes ||~ΓA ( h ) ||∞ ( the maximum risk discrimination gap ) . Note that one could alternatively choose to find the Pareto classifier that minimizes the maximum subgroup risk . Definition 3.4 . Pareto-fair classifier and Pareto-fair vector : A classifier h∗ is an optimal Paretofair classifier if it minimizes the discrimination gap among all Pareto front classifiers , h∗ = arg min h∈P ( H , A ) ||~ΓA ( h ) ||∞ . The Pareto-fair vector r∗ ∈ R|A| is defined as r∗ = { Ra ( h∗ ) } a∈A . Even when perfect equality of risk is desirable , Pareto classifiers still serve as useful intermediaries . To this end , Lemma 3.1 , shows that applying a mechanism for reaching equality of risk on a dominated classifier h ∈ H leads to equal or worse risks than applying it to a Pareto classifier hp ∈ P ( H , A ) that dominates h. Lemma 3.1 . If h 6∈ P ( H , A ) → ∃hp ∈ P ( H , A ) : hp h ∧ Ra ( hERp ) ≤ Ra ( hER ) ∀a , with hER an equal-risk classifier : Ra ( hER ) = max a′∈A Ra′ ( h ) , ∀a and hERp : Ra ( hERp ) = max a′∈A Ra′ ( hp ) . To exemplify these notions graphically , Figure 1 shows a scenario with binary sensitive attributes a and binary output variable y where none of the Pareto front classifiers achieve equality of risk . Here the noise level differs between subgroups , and the Pareto-fair vector r∗ is not achieved by either a Naive classifier ( minimizes expected global risk ) , or a classifier where subgroups are re-sampled to appear with equal probability ( rebalanced Naive classifier ) . Note that the amount of performance degradation required to enforce perfect fairness starting from the Naive classifier is higher than when starting from the Pareto-fair vector . Our objective is to find the Pareto-fair classifier h∗ as in Definition 3.4 . In particular , we will give regularity conditions on the hypothesis class H and the loss function ` ( h ( X ) , Y ) . It can be shown that for a sufficiently rich class of hypothesis functions H and for risk functions Ra ( h ) that are convex with respect to h , the space of risk vectors is convex ( see Geoffrion ( 1968 ) ; Koski ( 1985 ) ; Miettinen ( 2012 ) ) . Under these conditions , we can find an auxiliary loss function φ : R|A| → R defined in terms of the subgroup risks { Ra ( h ) } a∈A ( denoted from here on as r ∈ R|A| ) that has a global minima on r∗ . To prove the existence of loss function φ with the desired global minimum r∗ , it is convenient to think of the convex set of risk vectors as the intersection of a convex Pareto set ( defined by a convex Pareto front and all risks that are dominated by it ) , and an additional convex set Ω . We further require both this Pareto set and Ω to be smooth , so that we can apply standard tools from smooth convex optimization and prove that for every risk vector r′ in the Pareto front there exists a function φ that has r′ as its global optima . This is formalized in Lemma 3.3 . Definition 3.5 . Pareto field : A function P : Ω → R is a Pareto field over a convex set Ω ⊂ Rk if P ∈ C1 is a continuously differentiable function such that∇iP ( r ) > 0 ∀r ∈ Ω , ∀i = 1 , . . . , k. Lemma 3.2 . Let Ω ⊂ Rk be a convex set , and P : Ω → R a Pareto field . Then the set D = { r ∈ Ω : P ( r ) = 0 } is a Pareto set in Ω , and the set D+ = { r ∈ Ω : P ( r ) > 0 } is the set of dominated points , i.e. , D+ = { r ∈ Ω : ∃r′ ∈ D | r r′ } Lemma 3.3 . Let Ω ⊂ Rk be a convex set defined by Ω = { r ∈ Rk : gc ( r ) ≥ 0 ∀ c ∈ { 1 , . . . , C } , gc continuously differentiable } ; let P : Ω → R be a convex Pareto field with corresponding proper Pareto set D = { r ∈ Ω : P ( r ) = 0 } . Let r̂ ∈ D and φ ( r , µ ) =∑k i=1 ri + µi ( ri − c ) +2 , with c < r̂ , where we denote ( x ) +2 = max2 ( x , 0 ) . There exists a set of vectors µ̂ 0 such that : r̂ = arg min r∈Rk φ ( r , µ̂ ) s.t . : P ( r ) ≥ 0 , gc ( r ) ≥ 0∀c ∈ { 1 , . . . , C } . Where C is the number of constraints that characterized Ω . Proofs for all lemmas are given in the supplementary material , Section A.1 . Lemma 3.3 motivates our loss function to take the form φ ( r , µ ) = ∑|A| i=1 ri+µi ( ri−c ) +2 , with |A| the number of sensitive groups . The challenge is to find µ∗ such that our Pareto-fair vector r∗ minimizes φ ( r , µ∗ ) . In Section 4 we provide an algorithm to approximately recover the Pareto-fair classifier h∗ by adaptively searching for µ∗ and optimizing a classifier on φ ( · , µ ) using tools from standard Stochastic Gradient Descent .

Summary: This paper considers the notion of "no-harm" group fairness, i.e. trying to reduce the risk gap between minority and majority groups without excessive reduction in performance on the majority groups. Authors formalize the problem by defining a Pareto fair classifier, i.e. one that minimizes the risk gaps between groups and belongs to the family of Pareto classifiers containing the classifier minimizing the empirical risk. Authors suggest an optimization procedure for finding the Pareto fair classifier and demonstrate its performance on multiple datasets.