P ARETO O PTIMALITY IN N O -H ARM F AIRNESS
Anonymous authors
Paper under double-blind review
A BSTRACT
Common fairness deﬁnitions in machine learning focus on balancing various notions of disparity and utility.
In this work we study fairness in the context of risk disparity among sub-populations.
We introduce the framework of Pareto-optimal fairness , where the goal of reducing risk disparity gaps is secondary only to the principle of not doing unnecessary harm, a concept that is especially applicable to high-stakes domains such as healthcare.
We provide analysis and methodology to obtain maximally-fair no-unnecessary-harm classiﬁers on ﬁnite datasets.
We argue that even in domains where fairness at cost is required, no-unnecesary-harm fairness can prove to be the optimal ﬁrst step.
This same methodology can also be applied to any unbalanced classiﬁcation task, where we want to dynamically equalize the misclassiﬁcation risks across outcomes without degrading overall performance any more than strictly necessary.
We test the proposed methodology on real case-studies of predicting income, ICU patient mortality, classifying skin lesions from images, and assessing credit risk, demonstrating how the proposed framework compares favorably to other traditional approaches.
I NTRODUCTION
Machine learning algorithms play an important role in decision making in society.
When these algorithms are used to make high-impact decisions such as hiring, credit-lending, predicting mortality for intensive care unit patients, or classifying benign/malign skin lesions, it is paramount to guarantee that these decisions are both accurate and unbiased with respect to sensitive attributes such as gender or ethnicity.
A model that is trained naively may not have these properties by default; see, for example Barocas & Selbst (2016).
In these critical applications, it is desirable to impose some fairness criteria.
Much of the fairness in machine learning literature attempts to produce algorithms that satisfy Demographic Parity, which aims to make algorithm’s
predictions independent of the sensitive populations (Louizos et al.
Zemel et al.
; Feldman et al.
or Equality of Odds or Equality of Opportunity, which aims to produce predictions that are independent of the sensitive attributes given the ground truth (Hardt et al.
; Woodworth et al.
Notions of Individual Fairness have also been advanced (Dwork et al.
; Joseph et al.
; Zemel et al.
These notions of fairness can be appropriate in many scenarios, but in domains where quality of service is paramount, such as healthcare, we argue that it is necessary to strive for models that are as close to fair as possible without introducing any unnecessary harm to any subgroup (Ustun et al.
Even if the overall fairness goal is a potentially harmful, zero-gap classiﬁer, pursuing ﬁrst a Pareto-fair classiﬁer and later applying other harmful methodologies ensures that all possible non-harmful trade-offs are covered before explicitly degrading performance, therefore minimal harm is introduced to the decision.
In this work we make use of the concept of Pareto optimality to measure and analyze discrimination (unfairness) in terms of the difference in predictive risks across sub-populations deﬁned by our sensitive attributes, a fairness metric that has been explored in other recent works such as Calders & Verwer (2010); Dwork et al.
; Feldman et al.
; Chen et al.
Ustun et al.
We examine the subset of models from our hypothesis class that have the best trade-offs between sub-population risks, and select from this set the one with the smallest risk disparity gap.
This is in direct contrast to common post-hoc correction methods like the ones proposed in Hardt et al.
; Woodworth et al.
(2017), where noise is potentially added to the decisions of the best performing sub-population.
While this latter type of approach diminishes the risk-disparity gap, it does so by degrading performance on advantaged groups, the previously disadvantaged groups do not directly beneﬁt from this treatment.
Since our proposed methodology does not require test-time access to sensitive attributes, and can be applied to any standard classiﬁcation or regression task, it can also be used to reduce risk disparity between outcomes, acting as an adaptive risk equalization loss compatible with unbalanced classiﬁcation scenarios.
Main Contributions.
We formalize the notion of no-harm risk fairness 1 using Pareto optimality (Mas-Colell et al.
(1995)), a state of resource allocations from which it is impossible to reallocate without making one subgroup worse.
We show that ﬁnding a Pareto-fair classiﬁer is equivalent to ﬁnding a model in our hypothesis class that belongs to the Pareto front (the set of all Pareto optimal allocations) with respect to the sub-population risks with the smallest possible risk disparity.
This general notion is already amenable to non-binary sensitive attributes.
We analyze the fairness performance trade-offs that can be expected from different approaches with an illustrative example.
We provide a concrete algorithm that promotes fair solutions belonging to the Pareto front; this algorithm can be applied to any standard classiﬁer or regression task that can be trained using (Stochastic) Gradient Descent.
We show that if the goal is to obtain a zero-gap classiﬁer, ﬁrst recovering the fairest Pareto optimal solution and then adding harmful post-hoc corrections ensures the lowest risk levels across all subgroups.
We demonstrate how our methodology performs on synthetic and real tasks such as inferring income status in the Adult dataset Dua & Graff (2017a) irrespective of their ethnicity or gender, predicting ICU mortality rates in the MIMIC-III dataset from hospital notes Johnson et al.
(2016), classifying skin lesions in the HAM10000 dataset Tschandl et al.
(2018), and assessing credit risk on the German Credit dataset Dua & Graff (2017b).
R ELATED W ORK
There is an extensive body of work on fairness in machine learning.
Following Friedler et al.
(2019), we compare our methodology against the works of Feldman et al.
; Kamishima et al.
Zafar et al.
Our method shares conceptual similarities with Zafar et al.
Woodworth et al.
(2017); Agarwal et al.
(2018), with differences on how we deﬁne our fairness objective and adapt it to work with standard neural networks.
Although optimality is often discussed in the fairness literature, it is usually in the context of utility-discrimination tradeoffs.
To the best of our knowledge, this is the ﬁrst work to discuss optimality with respect to subgroup risks on a uniﬁed classiﬁer, a distinction that disallows extreme performance degradation in the pursuit of fairness.
The work presented in Hashimoto et al.
(2018) discusses decoupled classiﬁers as a way of minimizing group-risk disparity, but simultaneously cautions against this methodology when presented with insufﬁciently large datasets.
The works of Chen et al.
Ustun et al.
(2019) also empirically report the disadvantages of decoupled classiﬁers as a way to mitigate risk disparity.
Here we argue for the use of a single classiﬁer because it allows transfer learning between diverse sub-populations.
We do not need access to the sensitive attribute during test time, but in cases where this is possible, we instead choose to incorporate it as part of our observation features.
The work of Chen et al.
(2018) uses the uniﬁed bias-variance decomposition advanced in Domingos (2000) to identify that noise levels across different sub-populations may differ, making perfect fairness parity impossible without explicitly degrading performance on one subclass.
Their methodology attempts to bridge the disparity gap by collecting additional samples from high-risk sub-populations.
Here we modify our classiﬁer loss to bridge the disparity gap without inducing unnecessary harm, which could prove to be synergistic with their methodology.
P ROBLEM S TATEMENT
Consider we have access to a dataset D
= { ( x i , y i , a i ) } ni
=1 containing n independent triplet samples drawn from a joint distribution ( x i , y i , a i ) ∼ P ( X, Y, A )
where x i ∈
X are our input features (e.g., images, tabular data, etc.),
y i ∈ Y is our target variable, and a i ∈ A indicates group membership or sensitive status (e.g., ethnicity, gender); our input features X may or may not explicitly contain A .
Let h ∈ H be a classiﬁer from a compact hypothesis class H trained to infer y from x , h : X → Y ; and a loss function (cid:96) :
Y × Y → R .
We deﬁne the class-speciﬁc risk of classiﬁer h on subgroup a as
R a ( h ) =
E X,Y |
A = a [ (cid:96) ( h ( X ) , Y )] .
The risk discrimination gap between two subgroups a, a (cid:48) ∈
A is measured as Γ a,a (cid:48) ( h ) =
| R a ( h ) − R a (cid:48)
( h ) | , and we deﬁne the pairwise discrimination gap vector as (cid:126) Γ A ( h )
=
{ Γ a,a (cid:48) ( h ) } a,a (cid:48) ∈A .
Our goal is to obtain a classiﬁer h ∈ H that minimizes this gap without causing unnecessary harm to any particular group in A .
To formalize this notion, we deﬁne:
Deﬁnition 3.1.
Dominant risk vector:
A vector r (cid:48)
∈
R k is said to dominate r ∈ R k , noted as r (cid:31) r (cid:48) , if r i ≥ r (cid:48)
i , ∀ i
= 1 , ..., k and ∃ j : r j > r (cid:48)
j (i.e., strict inequality on at least one component).
Deﬁnition 3.2.
Dominant risk classiﬁer: Classiﬁer h (cid:48) is said to dominate h , noted as h (cid:31) h (cid:48) , if the risks vector r (cid:48)
=
{ R a ( h (cid:48) ) } |A| a =1 dominates r
= { R a ( h ) } |A| a =1 .
Deﬁnition 3.3.
Pareto front: We deﬁne the Pareto front as P ( H , A ) =
{ h ∈ H : (cid:64) h (cid:48)
∈ H | h (cid:31) h (cid:48) } .
This means that there is no other classiﬁer in H that is at least as good in all risks and strictly better in at least one of them.
It is the set of classiﬁers such that improving one group’s risk comes at the cost of increasing other’s.
The Pareto front deﬁnes the best achievable trade-offs between population risks R a ( h ) .
This definition is already suited for classiﬁcation and regression tasks where the sensitive attributes are categorical.
Constraining the classiﬁer to be in the Pareto front disallows laziness, there exists no other classiﬁer in the hypothesis class H that is at least as good on all class-speciﬁc risks and strictly better in one of them.
As shown in Chen et al.
; Domingos (2000), the risk can be decomposed in bias, variance and noise for some loss functions, where the noise is the smallest achievable risk for inﬁnitely large datasets (Bayes-optimal risk).
If the noise differs between sensitive groups, zero-discrimination (perfect fairness) can only be achieved by introducing bias or variance.
Literature on fairness has focused on putting constraints on the norm of discrimination gaps (Zafar et al.
; Creager et al.
; Woodworth et al.
We follow a similar criteria in
Deﬁnition 3.4 and deﬁne the Pareto-fair classiﬁer as the classiﬁer in the Pareto front that minimizes || (cid:126)
Γ A ( h ) || ∞ (the maximum risk discrimination gap).
Note that one could alternatively choose to ﬁnd the Pareto classiﬁer that minimizes the maximum subgroup risk.
Deﬁnition 3.4.
Pareto-fair classiﬁer and Pareto-fair vector:
A classiﬁer h ∗ is an optimal Paretofair classiﬁer if it minimizes the discrimination gap among all Pareto front classiﬁers, h ∗ = arg min
h ∈P ( H , A ) ||
(cid:126)
Γ A ( h ) || ∞ .
The Pareto-fair vector r ∗ ∈ R |A| is deﬁned as r ∗ =
{ R a ( h ∗ ) } a ∈A .
Even when perfect equality of risk is desirable, Pareto classiﬁers still serve as useful intermediaries.
To this end, Lemma 3.1, shows that applying a mechanism for reaching equality of risk on a dominated classiﬁer h ∈ H leads to equal or worse risks than applying it to a Pareto classiﬁer h p ∈ P ( H , A ) that dominates h .
Lemma 3.1.
If h (cid:54)∈ P ( H , A ) → ∃ h p ∈ P ( H , A ) : h p (cid:31) h ∧ R a ( h ERp ) ≤ R a ( h ER ) ∀ a , with h ER an equal-risk classiﬁer : R a ( h ER )
= max a (cid:48) ∈A R a (cid:48) ( h ) , ∀ a and h ERp :
R a ( h ERp )
= max a (cid:48) ∈A R a (cid:48)
( h p ) .
To exemplify these notions graphically, Figure 1 shows a scenario with binary sensitive attributes a and binary output variable y where none of the Pareto front classiﬁers achieve equality of risk.
Here the noise level differs between subgroups, and the Pareto-fair vector r ∗ is not achieved by either a
Naive classiﬁer (minimizes expected global risk), or a classiﬁer where subgroups are re-sampled to appear with equal probability (rebalanced Naive classiﬁer).
Note that the amount of performance degradation required to enforce perfect fairness starting from the Naive classiﬁer is higher than when starting from the Pareto-fair vector.
Our objective is to ﬁnd the Pareto-fair classiﬁer h ∗ as in Deﬁnition 3.4.
In particular, we will give regularity conditions on the hypothesis class H and the loss function (cid:96) ( h ( X ) , Y ) .
It can be shown that for a sufﬁciently rich class of hypothesis functions H and for risk functions R a ( h ) that are convex with respect to h , the space of risk vectors is convex (see Geoffrion (1968); Koski (1985);
Miettinen (2012)).
Under these conditions, we can ﬁnd an auxiliary loss function φ : R |A| → R deﬁned in terms of the subgroup risks { R a ( h ) } a ∈A (denoted from here on as r ∈ R |A| ) that has a global minima on r ∗ .
To prove the existence of loss function φ with the desired global minimum r ∗ , it is convenient to think of the convex set of risk vectors as the intersection of a convex Pareto set (deﬁned by a convex
Pareto front and all risks that are dominated by it), and an additional convex set Ω .
We further require both this Pareto set and Ω to be smooth, so that we can apply standard tools from smooth convex optimization and prove that for every risk vector r (cid:48) in the Pareto front there exists a function φ that has r (cid:48) as its global optima.
This is formalized in Lemma 3.3.
Deﬁnition 3.5.
Pareto ﬁeld: A function P : Ω → R is a Pareto ﬁeld over a convex set Ω ⊂ R k if
P ∈ C 1 is a continuously differentiable function such that ∇ i P ( r ) > 0 ∀ r ∈ Ω , ∀ i = 1 , . . .
, k .
Lemma 3.2.
Let Ω ⊂ R k be a convex set, and P : Ω → R a Pareto ﬁeld.
Then the set D = { r ∈ Ω : P ( r ) = 0 } is a Pareto set in Ω and the set D + = { r ∈ Ω : P ( r ) > 0 } is the set of dominated points, i e., D + = { r ∈ Ω : ∃ r (cid:48)
∈ D | r (cid:31) r (cid:48) }
Lemma 3.3.
Let Ω ⊂
R
k be a convex set deﬁned by Ω = { r ∈ R
k : g c ( r ) ≥ 0 ∀ c ∈ { 1 , . . .
, C } , g c continuously differentiable } ; let P : Ω → R be a convex Pareto ﬁeld with corresponding proper Pareto set D = { r ∈ Ω : P ( r ) = 0 } .
Let ˆ r ∈ D and φ ( r , µ ) =
(cid:80) ki =1 r i + µ i ( r i − c ) +2 , with c < ˆ r , where we denote ( x ) +2 = max 2 ( x, 0) .
There exists a set of vectors ˆ µ (cid:31)
such that:
[Equation]
Where C is the number of constraints that characterized Ω .
Proofs for all lemmas are given in the supplementary material, Section A.1.
Lemma 3.3 motivates our loss function to take the form φ ( r , µ ) =
(cid:80) |A| i =1 r i + µ i ( r i − c ) +2 , with |A| the number of sensitive groups.
The challenge is to ﬁnd µ ∗ such that our Pareto-fair vector r ∗ minimizes φ ( r , µ ∗ ) .
In Section 4 we provide an algorithm to approximately recover the Pareto-fair classiﬁer h ∗ by adaptively searching for µ ∗ and optimizing a classiﬁer on φ ( · , µ ) using tools from standard Stochastic Gradient Descent.
O PTIMIZATION M ETHODS
Recall that we wish to recover the Pareto-fair classiﬁer h ∗ within our hypothesis class.
From Lemma 3.3, there exists a loss function of the form
[Equation]
(1) such that h ∗ =
arg min h ∈H φ ( h ; µ , c ) .
Note that we can state the loss function directly on the risk vectors r , or implicitly on the classiﬁer h .
Since R a ( h ) is differentiable with respect to h , φ ( h ; µ , c ) can be directly minimized using gradient descent on h for any choice of values µ , c .
Let D Tr =
{ x, y, a } N T i =1 , D Val = { x, y, a } N V i =1 be our training and validation datasets respectively.
The proposed implementation of the Pareto-fair framework is formalized in Algorithm 1 where we specify how to update the penalty coefﬁcients (cid:126)µ, c .
Given: h θ , (cid:96), D Tr , D Val , n µ , n p , n max , γ > 0 , k ≥ 1 , ξ, ζ ∈ (0 , 1) , lr , B µ ← 1 , µ ∗ ← 1 , µ count ← 0 , e count ← 0 , c ← 0 , Γ ∗ ← ∞ , h ∗ ← h
θ while e count ≤ n max and µ count ≤ n µ do µ count ← µ count + 1 , e count ← e count +h θ , r Val ← AdaptiveOptimize ( h θ , (cid:96), µ , c, D Tr , D Val , n p , lr , B ) //
Optimize current loss // Check that solution is Pareto efficient and reduces fairness gap if || (cid:126) Γ A ( h ) || ∞ < Γ ∗ and r Val is not dominated by previous validation risks then h
[Equation]
[Equation] [Equation]
[Equation]
←else lr ← ζ lr, µ ← µ ∗ ,
[Equation]
← γξ, h θ ← h ∗ end µ a (cid:48) ← (1 + γ ) µ a (cid:48) end //
Exit loop due to excessive iterations or no improvement in fairness
Return: h ∗
We regularly check that reductions in the fairness gap generalize to the validation set; we additionally check if the trade-offs are in the non-dominated solution set (i.e., we have not observed a universally better classiﬁer during training).
Algorithm 2 (AdaptiveOptimize, shown in Section A.2) summarizes how we perform stochastic gradient descent steps with early stopping in-between µ , c updates.
Lemma A.5 (shown in supplementary material, Section A.1) shows that this algorithm is convergent for |A|
= 2 when the minimization step for a ﬁxed µ , c is performed exactly.
To conclude this section, we stress that the proposed framework is independent of the desired algorithm class H and loss function (cid:96) ; these are kept from the original application.
The Pareto-fair classiﬁer uses the same inputs as the Naive classiﬁer, with parameters that have been optimized towards Pareto fairness.
Code will be made available.
E XPERIMENTS AND R ESULTS
We applied the methodology described in Section 4 to learn a Pareto-fair classiﬁer (classiﬁer in the group-risk Pareto front with the smallest risk disparity).
We ﬁrst validate our methodology on synthetic data with known Pareto-fair classiﬁers.
Observations are drawn from a Gaussian mixture model where each sensitive attribute is encoded by a corresponding Gaussian mode, and target attributes are binary.
We demonstrate our methodology on publicly available fairness datasets, and show how the risk disparity gaps are subsequently reduced.
Where applicable, we compare our results against the methodologies proposed in Zafar et al.
; Kamishima et al.
; Hardt et al.
; Feldman et al.
5. S YNTHETIC D ATA
We tested our approach on synthetic data where the observations are drawn from conditional Gaussian distributions X |
A = a ∼ N ( µ A , 1) , the target variable y is a conditional Bernoulli variable with distribution Y | X
= x, A = a ∼ Ber (cid:0) f a ( x ) (cid:1) with f a ( x ) =
ρ lowa 1 [ x ≤ c a ] + ρ higha 1 [ x > c a ] (cid:1) , and | A | = 3 .
We used Brier Score (BS, refer to Section A.4) as our loss function.
Under these conditions, the Bayes-optimal classiﬁer for each subgroup is h ( x ) =
f a ( x ) .
The subgroup risk function for any classiﬁer h can be computed as R a ( h ) =
E X |
A = a [ Y − h
( X )) 2 ] =
E X |
A = a [ f a ( X )(1 − h ( X ))
+ (1 − f a ( X ))
h
( X ) 2 ] .
Network details are given in the supplementary material, Section A.6.
Figure 2 shows the analytically-derived Pareto-fair classiﬁer, as well as the trade-off obtained by the proposed algorithm.
We remark that the Pareto-fair classiﬁer in this scenario cannot be achieved using linear logistic regression, something that can be seen from the functional form of the Bayes Optimal classiﬁer derived in Section A.8 in supplementary material.
From the center graph in Figure 2 we can see that the approximated Pareto-Fair risks (dashed lines) are close to the theoretical best (solid lines).The achieved maximum risk discrimination is also close to the theoretical best.
The algorithm is able to correctly trade risks from the two advantaged subgroups to reduce the risk of the worst performing subgroup.
5. R EAL D ATASETS
We evaluate the performace of our algorithm on mortality prediction (MIMIC-III), skin lesion classiﬁcation (HAM10000), income prediction (Adult), and credit lending (German).
We also compared with several state of the art methods (Zafar et al.
; Kamishima et al.
; Hardt et al.
Feldman et al.
and with a model trained to minimize the average risk (Naive) and one that undersamples the minority class (Naive Balanced or NaiveB).
A description of these methods is provided in Section A.3 of Supplementary material.
Metrics used for evaluation include accuracy (Acc), conﬁdence (Conf), expected and maximum calibration error (ECE and MCE respectively),
Brier score (BS) and cross-entropy (CE).
Values for every metric are reported per sensitive attribute.
The dataset average is reported as (sample mean), and the group-normalized average as (group mean).
The best and worst performing groups are also shown as (group max) and (group min), and the worst case difference between subgroups is reported as (disc).
For an in-depth description of the the metrics and datasets used for evaluation, refer to Sections A.4 and A.5 of Supplementary material.
Here we present a comparison of the performance in terms of accuracy and ECE but an extensive comparison is available in additional results Section A.7 in supplementary material.
Similarly to Chen et al.
; Zafar et al.
; Hardt et al.
; Woodworth et al.
(2017), we omit the sensitive attribute a ∈ A from our observation features.
Our method trains a single classiﬁer for the entire dataset to avoid needing test-time access to sensitive attributes whenever possible.
All classiﬁers shown in this section are implemented using neural networks and/or logistic regression, we used either cross-entropy or Brier score as our training loss depending on the dataset.
For details on the architecture and hyperparameters used on each dataset, refer to the supplementary material,
Section A.6.
We show how the proposed Pareto-fair approach produces well calibrated models that reduce group disparities across several metrics.
5.2. P REDICTING M ORTALITY IN I NTENSIVE C ARE P ATIENTS
Medical decisions in general and mortality prediction in particular are examples where notions of fairness among sub-populations are of paramount importance, and where ethical considerations make no-harm fairness a very attractive paradigm.
To that end, we used clinical notes collected from adult ICU patients at the Beth Israel Deaconess Medical Center (MIMIC-III dataset) Johnson et al.
(2016) to predict patient mortality.
We study fairness with respect to age (adult or senior), ethnicity (white or nonwhite), and outcome (alive/deceased).
This leads to a total of 8 sensitive groups.
We included outcome (alive/deceased) as a sensitive sub-population criteria to demonstrate a case where sensitive attributes would not be available at test-time, and because in our experiments patients who ultimately passed away on ICU were under-served by a Naive classiﬁer.
Table 1 shows empirical accuracies and expected calibration errors of all tested methodologies.
It is important to note how imbalanced these groups are by looking at the ’ratio’ column.
Here one can see that 56.7% of the samples correspond to the majority class (alive, white, senior) and only 0.4% to the minority
[Table]
class (deceased, nonwhite, adult).
Our methodology produces low discrimination classiﬁers with high group accuracies, for example, ParetoFair trained with brier score loss (BS) increased the classiﬁcation accuracy of the most under-served group by over 50% while reducing the accuracy of the best-served group by 12% when compared to the Naive classiﬁer.
5.2. S KIN L ESION C LASSIFICATION
The HAM10000 dataset Tschandl et al.
(2018) collects over 10 , 000 dermatoscopic images of skin lesions over a diverse population.
Lesions are classiﬁed according to diagnostic categories.
We found that a Naive classiﬁer exhibited almost no measurable discrimination based on age or race on this dataset.
We instead chose to use the diagnosis class as both the target and sensitive variable, casting balanced risk minimization as a particular use-case for Pareto fairness.
This is possible since our methodology does not require test-time access to sensitive labels.
It was not possible to show comparisons against Hardt et al.
(2016) since the sensitive attribute is perfectly predictive of the outcome.
Table 3 shows accuracies and ECE for all tested methodologies.
The Pareto-fair classiﬁer has the overall best calibration results and smallest accuracy disparities across methodologies.
Comparisons against other methods were not possible because the target labels are non-binary.
[Table]
5.2. I NCOME P REDICTION AND C REDIT R ISK A SSESMENT
We tested the proposed method on the Adult UCI dataset Dua & Graff (2017a) and on the German Credit dataset Dua & Graff (2017b).
In the Adult UCI dataset the goal is to predict a person’s
[Table]
income, which can be an important factor on meaningful decisions such as credit lending.
In the
German Credit dataset the goal is predicting credit risk.
We select gender and ethnicity as our sensitive attributes.
To compare ourselves against state of the art methods Zafar et al.
; Feldman et al.
; Kamishima et al.
we binarize the sensitive attributes into White-Male and Other when dealing with ethnicity and gender simultaneously, or Male and Female when dealing with gender, and use the uniﬁed testbed provided in Friedler et al.
We limit our hypothesis class H to linear logistic regression to compare evenly against these standard baselines.
Results on both datasets are shown in Table 5.
D ISCUSSION
There exists a rich literature of fairness in machine learning in general, and risk-based fairness in particular.
Here we explore a relatively untapped sub-problem where the goal is to reduce risk disparity gaps in the most ethical way possible (i.e., minimizing unnecessary harm).
Unlike other works in the area, our problem investigates on how to reduce this disparity gap without collecting additional data samples, using the entirety of the available training data to produce an algorithm that is maximally fair with respect to sub-populations, and that does not necessarily require test-time access to sensitive attributes.
We provide a concrete algorithmic adaptation to any standard classiﬁcation or regression loss to bridge this disparity gap at no unnecessary harm, and demonstrate its performance on several realworld case studies.
Even for applications where the need for strict fairness outweighs the need for no-harm classiﬁers, this methodology can be applied before any post-hoc corrections to ensure that the risk disparity gap is closed in the most risk-efﬁcient way for all involved sub-populations.
The proposed algorithm does not sweep through different disparity constraint values, as previously done in related works, making it a simpler alternative.
As an avenue of future research, it could be of interest to analyze if we can automatically identify high-risk sub-populations as part of the learning process and attack risk disparities as they arise, rather than relying on preexisting notions of disadvantaged groups or populations.
We strongly believe that no-unnecessary-harm notions of fairness are of great interest for several applications, especially so on domains such as healthcare and lending, where decisions are highly impactful.
A A PPENDIX A. P ROOFS
Here we restate the Lemmas shown in Section 3 along with a sketch of the proofs.
Lemma 3.1 If h (cid:54)∈ P ( H , A ) → ∃ h p (cid:31) h ∈ P ( H , A ) : R a ( h ERp ) ≤ R a ( h ER ) , ∀ a , where h ER is a equality of risks classiﬁer in H such that R a ( h ER ) = max a (cid:48) ∈A R a (cid:48) ( h ) , ∀ a and h ERp :
R a ( h ERp )
= max a (cid:48) ∈A R a (cid:48)
( h p ) , ∀ a .
Proof.
If h p dominates h → R a ( h ERp )
= max a (cid:48) ∈A R a (cid:48)
( h p ) ≤ max a (cid:48) ∈A R a (cid:48)
( h ) =
R a ( h ER ) ∀ a ∈ A .
Deﬁnition A.1.
Dominant vector:
A vector r (cid:48)
∈
R k is said to dominate r ∈ R k , noted as r (cid:48) (cid:31) r , if r i ≥ r (cid:48)
i , ∀ i
= 1 , ..., k and ∃ j : r j > r (cid:48)
j (i.e., strict inequality on at least one component).
Deﬁnition A.2.
Pareto ﬁeld:
A function P : Ω → R is deﬁned as a Pareto ﬁeld over a convex set Ω ⊂ R k if P ∈ C 1 is a continuously differentiable function such that ∇ i P ( r ) > 0 ∀ r ∈ Ω , ∀ i = 1 , . . . , k .
Lemma A.1.
Let Ω ⊂ R k be a convex set, and P : Ω → R a Pareto ﬁeld.
Then the set D = { r ∈ Ω : P ( r ) = 0 } is a proper Pareto set in Ω , and the set D + = { r ∈ Ω : P ( r ) > 0 } is the set of dominated points, i.e., D + = { r ∈ Ω : ∃ r (cid:48)
∈ D | r (cid:31) r (cid:48) } .
Proof.
First we prove that D
= { r ∈ Ω : P ( r ) = 0 } is a Pareto set.
Assume by contradiction that there exists r , r (cid:48)
∈ D | r (cid:48) (cid:31) r , then r (cid:48) − r = c = (cid:80) i
δ i e i
δ i ≥ 0 ∀ i ∈ { 1 , . . .
, k } , ∃ j ∈ { 1 , . . .
, k } | δ j > 0 .
e i is a standard basis vector.
Using the Gradient theorem we have
[Equation]
which directly contradicts r (cid:48) , r ∈ D .
No point in set D is dominated by another point in set D , making set D a proper Pareto set.
To show that the set D + is the set of dominated points, we ﬁrst note that for all r ∈ Ω | r (
cid:48) ∈ D, r (cid:31) r (cid:48)
we have P ( r ) > P ( r (cid:48) ) using the same arguments as above, meaning that the set of dominated points is included in D + .
Similarly, for all r ∈ Ω | r (cid:48) ∈ D, r (cid:48) (cid:23) r we have P ( r (cid:48) ) ≥ P ( r ) , meaning that non-dominated points are not a part of set D + .
Lemma A.2.
Let Ω ⊂ R |A| be a convex set deﬁned by Ω = { r ∈ R |A| : g c ( r ) ≥ 0 ∀ c ∈ { 1 , . . .
, C } , g c continuously differentiable } ; let P : Ω → R be a convex Pareto ﬁeld with corresponding Pareto set D = { r ∈ Ω : P ( r ) = 0 } .
Let r ∗ ∈ D and φ ( r , µ ) = (cid:80) |A| i =
1 r i + µ i ( r i − c ) 2+ , with c < r ∗ .
There exists a set of µ ∗ (cid:31)
such that:
[Equation]
Proof.
By hypothesis, both Ω and { r ∈ Ω : P ( r ) ≥ 0 } are convex, so the intersection is also convex.
For µ (cid:31) 0 , φ ( r , µ ) is a convex function of r .
Under these hypothesis, the Karush-KuhnTucker (KKT) conditions are necessary and sufﬁcient to recover a global minimizer.
So it sufﬁces to ﬁnd a set µ ∗ such that the KKT conditions are exactly satisﬁed at r ∗ .
Let J ∗ =
{ j : g j ( r ∗ ) = 0 } be the set of indices of active g c constraints at r ∗ , note that by hypothesis, r ∗ ∈ D and therefore P ( r ∗ ) is always active.
Focusing on the ﬁrst KKT condition on the active set we get ∇ φ ( r ∗ , (cid:126)µ )
= λ ∇ P ( r ∗ )
+ (cid:80)
j ∈ J ∗ η j ∇ g j ( r ∗ ) , in matrix form:
[Equation]
By hypothesis, we have r i − c > 0 and therefore
[Equation]
In particular, for η j = 0 ∀ j and µ 1 > [ ∇ P i ( r ∗ ) ∇ P 1 ( r ∗ ) − 1]
[Equation]
which satisﬁes the KKT conditions and produces the required minimizer r ∗ for all points r ∗ ∈ D .
Lemma A.3.
Let Ω ⊂ R 2 be a convex set deﬁned by Ω = { r ∈ R 2 : g c ( r ) ≥ 0 ∀ c ∈ { 1 , . .
, C } , g c continuously differentiable } ; let P : Ω → R be a convex Pareto ﬁeld with corresponding Pareto set D = { r ∈ Ω : P ( r ) = 0 } .
Let (cid:15) > 0 , r ∗ = arg min r ∈ D [max i r
i − min
j r j ] then c
= min( r 1 , r 2 ) − (cid:15)
< min( r ∗ 1 , r ∗ 2 ) ∀ (
r 1 , r 2 ) ∈ D.
Proof.
case r ∗ 1 = r ∗ 2 : Assume r ∈ D : min i r i > r ∗ 1 .
Then r (cid:31) r ∗ , which contradicts the hypothesis r ∈ D .
case r ∗ 1 (cid:54) =
r ∗ 2 : Without loss of generality, we can take r ∗ 1 > r ∗ 2 .
We can then prove by couterpositive that r ∈ D implies r 1 > r 2 .
Assume by contrapositive that there exists r (cid:48)
∈ D : r (cid:48) 2 > r (cid:48) 1 .
By hypothesis D is a continuous curve, and we can ﬁnd a path R ( t ) : R (0) =
r ∗ , R (1) =
r (cid:48) , R ( t ) ∈
D ∀ t ∈ [0 , 1] , which would imply there exists ˆ t : R (ˆ t ) =
(ˆ r , ˆ r ) , in which case r ∗ 1 = r ∗ 2 , which contradicts r ∗ 1 (cid:54) =
r ∗ 2 .
We then have that for r ∈ D , r 1 > r 2 .
Assume by counterpositive that r 2 = min ( r 1 , r 2 ) > min ( r ∗ 1 , r ∗ 2 ) =
r ∗ 2 then r 1 < r ∗ 1 (since both r , r ∗ ∈ D ), but then | r 1 − r 2 | < | r ∗ 1 − r ∗ 2 | which contradicts the hypothesis.
Putting both statements together, we proved that if r ∗ 1 > r ∗ 2 , then ∀ r ∈ D, min( r 1 , r 2 ) =
r 2 ≤ min ( r ∗ 1 , r ∗ 2 ) =
r ∗ 2 .
Lemma A.4.
Let Ω ⊂ R 2 be a convex set deﬁned by Ω = { r ∈ R 2 : g c ( r ) ≥ 0 ∀ c ∈ { 1 , . .
, C } , g c continuously differentiable } ; let P : Ω → R be a convex Pareto ﬁeld with corresponding Pareto set D = { r ∈ Ω : P ( r ) = 0 } .
Let µ = ( µ 1 , µ 2 ) > 0 , µ (cid:48)
= ( µ 1 , µ (cid:48) 2 ); µ (cid:48) 2 > µ 2 .
Deﬁne φ ( r , µ, c ) =
(cid:80) 2 i =
r
i + µ i ( r i − c ) 2+ and
[Equation]
r
[Equation]
µ
[Equation]
With c < r 2 , then, r (cid:48) 1 ≥ r 1 , r (cid:48) 2 ≤ r 2 , with equality if and only if r (cid:48)
=
ˆ r =
[Equation]
Proof.
By hypothesis, we have
[Equation]
since by hypothesis φ ( r , µ, c ) ≤ φ ( r (cid:48) , µ, c ) , it follows r (cid:48) 2 ≤ r 2 , and therefore r (cid:48) 1 ≥ r 1 , because they both belong to the Pareto set.
Case r (cid:48) =
r .
To analyze when the equality arises, note that the tangent to D at r is τ ( r )
= ( ∇ 2 P ( r ) , −∇ 1 P ( r )) and that r + δτ ( r ) , δ > 0 is a valid search direction if r (cid:54) =
ˆ r ( r does not minimize r 2 ).
Also note that ∇ φ ( r , µ (cid:48) , c ) =
∇ φ ( r , µ, c ) + (0 , 2( µ (cid:48) 2 − µ 2 )( r 2 − c )
+ )
By contradiction, assume r (cid:54) =
ˆ r , then r + δτ ( r ) , δ > 0 is a valid search direction and (cid:104) ( δτ ( r )) , ∇ φ ( r , µ (cid:48) , c (cid:105) > 0 , which implies r (cid:48) (cid:54) =
arg min
[Equation]
φ
[Equation] [Equation]
, c ) and contradicts the hypothesis.
Therefore, if r (cid:48) =
r it follows r (cid:48)
=
r = ˆ r
Lemma A.5.
Let Ω ⊂ R |A| be a convex set deﬁned by Ω = { r ∈ R |A| : g c ( r ) ≥ 0 ∀ c ∈ { 1 , . . .
, C } , g c continuously differentiable } ; let P : Ω → R be a convex Pareto ﬁeld with corresponding Pareto set D = { r ∈ Ω : P ( r ) = 0 } .
Let r ∗ = arg min r ∈ D (max i r i − min
j r j ) and φ ( r, µ, c )
=
(cid:80) |A| i =1 r i + µ i ( r i − c ) 2+ .
Let µ 0 > 0 be an initial penalty vector, r 0 = arg min r ∈ D (cid:80)
i r i , (cid:15) > 0 , c
= min
i r
i − (cid:15) , γ > 0 , ξ ∈ (0 , 1) .
Deﬁne the following auxiliary variables
[Equation]
We deﬁne the following iteration procedure compute Eq
If Γ k = Γ † terminate
While Γ † > Γ k γ ← ξγ
Recompute Eq
End While µ k +1 = µ † · ( r † − c k ) + ( r † − c † ) + r k +1 =
r †
The procedure is convergent to r ∗ for |A| = 2 .
Proof.
From Lemma A.3 we have c k < min i r ∗ i ∀ k , we can then apply Lemma A.2 and state that there exists a set µ ∗ k such that r ∗ =
arg min s.t.
P ( r ) ≥ 0 r ∈ Ω φ ( r, µ ∗ k , c k ) .
We split the proof in two scenarios, one where Γ ∗ = 0 and then Γ ∗ > 0 , since they show different convergence properties.
case Γ ∗ = 0 .
Assume without loss of generality that r k 1 = arg max i r ki , from Lemma
A.4 we have µ k 1 < µ k ∗ 1 , and further, for all µ † 1 ∈ ( µ k 1 , µ k ∗ 1 ) , we have r k 1 < r † 1 < r ∗ 1 .
Since |A| = 2 and r k , r † , r ∗ all belong to D , we also have r k 2 > r † 2 > r ∗ 2 , which implies Γ k > Γ † > Γ ∗ = 0 .
The subiteration procedure is always successful, and we get a sequence of iterates such that Γ k > Γ k +1 , a strictly decreasing sequence bounded below by 0 , which implies asymptotic convergence to r ∗ .
Procedure never terminates with probability 1 (event µ k ∈ µ ∗ k has probability 0 ).
case Γ ∗ > 0 Assume without loss of generality that r ∗ 1 < r ∗ 2 .
Following the arguments from Lemma
A.3 we observe that r 1 < r 2 ∀ ( r 1 , r 2 ) ∈ D .
In these conditions, we have that µ k 2 µ k 1 < µ k +12 µ k + and c k +1 > c k .
We will re-derive the KKT conditions in this scenario to show that convergence is guaranteed for µ k 2 > µ 2 ∗ , and that this µ 2 ∗ is independent of c k , meaning that convergence to the optimal r ∗ can be achieved in a ﬁnite number of steps.
Let ∇ P ( r ∗ ) =
( dP 1 , dP 2 ) > 0 , with corresponding tangent vector τ ∗ = ( dP 2 , − dP 1 ) , for r ∗ to be the risk with minimal gap, r ∗ + δτ ∗ must be an infeasible descent direction for δ > 0 .
Therefore, there is an active constraint from Ω , g ( r ) , with ∇ g ( r ∗ ) =
( dg 1 , dg 2 ) such that (cid:104) ( dg 1 , dg 2 ) , ( dP 2 , − dP 1 ) (cid:105) < 0 .
Deriving the ﬁrst KKT conditions, we get
[Equation]
and in terms of λ , η
[Equation]
Note that dg 2 dP 1 − dP 2 dg 1 > 0 from (cid:104) ( dg 1 , dg 2 ) , ( dP 2 , − dP 1 ) (cid:105) < 0 .
To recover a valid set of
Lagrange multipliers η, λ > 0 , it sufﬁces to have
[Equation]
where we used that c 0 < c k < (cid:15) ∀ k .
Therefore, for µ k 2 µ k 1 large enough, r k = r ∗ and the algorithm terminates.
A. A LGORITHMIC DETAILS
Here we provide details on how we optimize our adaptive loss in-between penalty update ( µ a ) steps, shown in Algorithm 2.
A. M ETHODS
We compare the performance of the following methods:
Given: h θ , L, (cid:126)µ, c, D Tr , D Val , n p , lr , B p ← 0 , φ ∗ ← ∞ h ∗ ← h
θ while p ≤ n p // Making loss progress do for { ( x i , y i , a i } Bi =1 ∈ D
Tr // Run one epoch of SGD on training set do
I a =
{ i ∈ { 1 , . . .
, B } ∧ a i = a } , ¯ R B a ( h θ ) = 1 | I a | (cid:80) i ∈ I a L ( h θ ( x i ) , y i ) //
Empirical risks φ B ( h θ ) =
(cid:80) a ∈A ¯ R B a ( h θ ) + µ a ( ¯ R B a ( h θ ) − c )
+ θ ← θ − lr ∇ θ φ B ( h θ ) //
Gradient step end if φ Val ( h θ ) < φ ∗ //
Evaluate improvement on Val and update target risks then h ∗ ← h θ ; φ ∗ ← φ Val ( h θ ); p ← 0 else p ← p + 1 ; end
R Val ← ¯ R Val ( h ) end
Return: h ∗ , R Val
Kamishima.
Kamishima et al.
(2012) uses logistic regression as a baseline classiﬁer, and requires numerical input (observations), and binary sensitive attribute and target variable.
Fairness is controlled via a regularization term with a tuning parameter η that controls trade-off between fairness and overall accuracy.
η is optimized via grid search with η ∈ (0 , 300) as in the original paper.
We report results on the hyperparameter conﬁguration that produces the smallest accuracy disparity between sensitive subgroups.
Feldman.
Feldman et al.
(2015) provides a preprocessing algorithm to sanitize input observations.
It modiﬁes each input attribute so that the marginal distribution of each coordinate is independent on the sensitive attribute.
The degree to which these marginal distributions match is controlled by a λ parameter between 0 and 1.
It can handle numerical and categorical observations, as well as non-binary sensitive attributes, and arbitrary target variables.
Following Friedler et al.
(2019), we train a linear logistic regressor on top of the sanitized attributes.
λ is optimized via grid search with increments of 0 .
We report results on the hyperparameter conﬁguration that produces the smallest accuracy disparity between sensitive subgroups.
Zafar.
Zafar et al.
Addresses disparate mistreatment via a convex relaxation.
Speciﬁcally, in the implementation provided in Friedler et al.
(2019), they train a logistic regression classiﬁer with a fairness constraint that minimizes the covariance between the sensitive attribute and the classiﬁer decision boundary.
This algorithm can handle categorical sensitive attributes and binary target variables, and numerical observations.
The maximum admissible covariance is handled by a hyperparameter c , tuned by logarithmic gridsearch with values between 0 .
001 and 1 .
We report results on the hyperparameter conﬁguration that produces the smallest accuracy disparity between sensitive subgroups.
Hardt.
Hardt et al.
(2016) proposes a post-processing algorithm that takes in an arbitrary predictor and the sensitive attribute as input, and produces a new, fair predictor that satisﬁes equalized odds.
This algorithm can handle binary target variables, an arbitrary number of sensitive attributes, and any baseline predictor, but requires test-time access to sensitive attributes.
it does not contain any tuning parameter.
We apply this method on top of both the Naive Classiﬁer and our Pareto Fair classiﬁer.
Naive Classiﬁer (Naive).
Standard classiﬁer, trained to minimize an expected risk h = arg min h ∈H E X,A,Y [ L ( h ( X ) , Y )] .
The baseline classiﬁer class H is implemented as a neural network and varies by experiment as described in Section A.6, the loss function also varies by experiment and is also described in Section A.6.
Optimization is done via stochastic gradient descent.
Naive
Balanced (NaiveB).
Baseline classiﬁer designed to address undersampling of minority classes, trained to mimimize a class-rebalanced expected risk h =
[Equation]
Like the Naive classiﬁer, it is implemented as a neural network and optimized via stochastic gradient descent.
The sole difference with the Naive classiﬁer is that, during training, training samples are drawn from the new input distribution A ∼ U [1 , . . .
, |A| ]; X, Y | A ∼ P ( X, Y | A ) , which is achieved by re-weighted sampling of the original training dataset.
Pareto Fair.
Our proposed methodology, trained to minimize an adaptive loss function using the procedure described in Algorithm 1.
Addresses risk disparity minimization without introducing unnecessary harm to any subgroup.
The baseline classiﬁer class H is implemented as a neural network and varies by experiment as described in Section A.6, the loss function also varies by experiment and is also described in Section A.6.
A. E VALUATION M ETRICS
Here we describe the metrics used to evaluate the performance of all tested methods.
We are given a set of test samples D t = { ( x i , y i ) } Ni =1 where x
i ∈
X is a realization of our model input and
y i ∈ Y the corresponding objective.
We assume that Y is a ﬁnite alphabet, as in a classiﬁcation problem, and we will represent the one-hot encoding of y i as (cid:126)e i .
Given a trained model h : X → [0 , 1] |Y| the predicted output for a an input x
i is a vector h ( x i )
=
(cid:126)p i such that ( (cid:126)p i ) j ∈ [0 , 1] , ∀ j ∈ { 1 , ..., |Y|} and | (cid:80) |Y| j =1 ( (cid:126)p i ) j = 1 (e.g.: output of a softmax layer).
The predicted class is ˆ y i
= arg max j ( (cid:126)p i ) j and its associated conﬁdence is ˆ p i = max j ( (cid:126)p i ) j .
Ideally ˆ y i should be the same as y i .
Using these deﬁnitions, we compute the following metrics.
Accuracy (AC): 1 N (cid:80) Ni
=1 1 ( y i
=
ˆ y i ) .
Fraction of correct classiﬁcations in dataset.
Conﬁdence (CO): 1 N (cid:80) Ni
=
ˆ p i .
Average magnitude of the predicted class probability.
Brier Score (BS): 1 N (cid:80) Ni
=1 || (cid:126)e i − (cid:126)p i
|| 2 where (cid:126)e i is the one-hot representation of the categorical ground truth value y i .
This quantity is also known as Mean square error (MSE).
Cross-Entropy (CE): − 1 N (cid:80) Ni =1 (cid:80) |Y| j =1 ( (cid:126)e i )
j log ( (cid:126)p i )
j also known as negative log-likelihood (NLL) of the multinomial distribution.
Expected Calibration Error (ECE): 1 N (cid:80) Mm =1 (cid:12)(cid:12) (cid:80) i ∈ B m [ 1 ( y i
=
ˆ y i )
− ˆ p i ] (cid:12)(cid:12) where M is the number of bins to divide the interval [0 , 1] such that B m =
{ i ∈ { 1 , .., N } : ˆ p i ∈ ( m − 1 M , mM ] } are the group of samples that our model assigns a conﬁdence ( ˆ p i ) in the interval ( m − 1 M , mM ] .
Measures how closely the predicted probabilities match the true base rates.
Maximum Calibration Error (MCE): max m ∈{ 1 ,...,M } (cid:12)(cid:12) 1 | B m |
[Table]
Measures worst-case miscalibration errors.
These metrics are computed independently for each sensitive subgroup on the test set and reported in Section A.7.
A. E XPERIMENTS ON REAL DATA
The following is a description of the data and experiments for each of the real datasets.
The information present here is summarized in Table 7.
[Table]
MIMIC-III.
This dataset consist of clinical records collected from adult ICU patients at the Beth
Israel Deaconess Medical Center (MIMIC-III dataset) Johnson et al.
The goal is predicting patient mortality from clinical notes.
We follow the pre-processing methodology outlined in Chen et al.
(2018), where we analyze clinical notes acquired during the ﬁrst 48 hours of ICU admission; discharge notes where excluded, as where ICU stays under 48 hours.
Tf-idf statistics on the 10 ,most frequent words in clinical notes are taken as input features.
We identify 8 sensitive groups as the combination of age (under/over 55 years old), ethnicity as determined by the majority group (white/nonwhite) and outcome (alive/deceased).
Here we will use the term adult to refer to people under 55 years old and senior otherwise.
This dataset shows large sample disparities since 56.7% of corresponds to the overall majority group (alive-senior-white) and only 0.4% to the overall minority group (deceased-adult-nonwhite).
We used a fully connected neural network as described in table 8 as the baseline classiﬁer for our proposed Pareto Fair algorithm.
We compare our results against both the Naive and Naive Balanced algorithms using the same neural network architecture, and crossentropy (CE) as our training loss.
We also evaluate the performance of Zafar applied on the feature embeddings learned by the Naive
Balanced classiﬁer (Results for Zafar over the original input features were not promising).
We report the performance across a 5-fold split of the data, we used a 60/20/20 train-validation-test partition as described on Table 7 and report results over the test set.
We denote the overall sensitive attribute as the combination of outcome (A:alive/D:deceased), age (A:adult/S: senior) and ethnicity (W:white, NW:nonwhite) with shorthand notation of the form D/A/W to denote, for example deceased, white adult.
We also note that results on both Zafar and Hardt were done over only the sensitive attributes Adult/Senior and White/Nonwhite, outcome was not considered as a sensitive attribute for both methods since Hardt requires test-time access to sensitive attributes, which would not be possible in this case, and Zafar attempts to decorrelate sensitive attributes and classiﬁcation decision boundaries, which is counterproductive when the sensitive attribute includes the correct decision outcome.
HAM10000.
This dataset contains over 10 , 000 dermatoscopic images of skin lesions over a diverse population Tschandl et al.
Lesions are classiﬁed in 7 diagnostic categories, and the goal is to learn a model capable of identifying the category from the lesion image.
The dataset is highly unbalanced since 81% of the samples correspond to a melanocytic nevi lesion (nv), and 0.5% to dermatoﬁbroma (df).
Here we chose to use the diagnosis class as both the target and sensitive variable, casting balanced risk minimization as a particular use-case for Pareto fairness.
We load a pre-trained DenseNet121 network Huang et al.
and train it to classify skin lesions from dermatoscopic images using our Pareto fairness algorithm.
We compared against the Naive and the Naive balanced training setup.
Note that in the naive balance approach we use a batch sampler where images from each class have the same probability, this can be seen as a naive oversampling technique.
Table 8 shows the details of the implementation.
We used the original train-validation-test (80/10/10) split, and report results on the test set.
Notation for each group follows the original notation: Actinic keratoses and intraepithelial carcinoma /
Bowen’s disease (akiec), basal cell carcinoma (bcc), benign keratosis-like lesions (bkl), dermatoﬁbroma (df), melanoma (mel), melanocytic nevi (nv) and vascular lesions (vasc).
Adult.
The Adult UCI dataset Dua & Graff (2017a) is based on the 1994 U.S. Census and contains data on 32 , 561 adults.
The data contains 105 binarized observations representing education status, age, ethnicity, gender, and marital status and a target variable indicating income status (binary attribute representing over or under $50 , 000 ).
Following Friedler et al.
(2019), we take ethnicity and gender as our target sensitive attributes, deﬁning two subgroups (White Males and Other).
We also present results taking just the gender as sensitive attribute (Male/Female).
To compare our Pareto
Fair algorithm evenly against the other methods, we limit our hypothesis class to linear logistic regression.
German.
The German credit dataset Dua & Graff (2017a) contains 20 observations collected across 1000 individuals, and a binary target variable assessing the individual’s credit score as good or bad.
We take gender (Male/Female) as the sensitive attribute, which is not included in the data but can be inferred.
As in the Adult dataset, we limit our hypothesis class to linear logistic regression to compare evenly across methodologies.
A. N EURAL A RCHITECTURES AND P ARAMETERS
Table 8 summarizes network architectures and loss functions for all experiments in Section 5.
Note that all networks have a standard dense softmax as their ﬁnal layer.
The training optimizer is standard
ADAM Kingma & Ba (2014), loss functions were either crossentropy (CE) or brier score (BS), also known as categorical mean square error (MSE).
[Table]
A. S UPPLEMENTARY R ESULTS
The following tables show a quantitative performance comparisons between the various fairness methods described in Section A.3 over the datasets described in Section A.5.
All metrics presented are described in Section A.4 and computed per sensitive attribute.
We additionally provide results for the overall mean across the dataset (sample mean), as well as results on the group-balanced mean across groups (group mean).
We use group max and group min to denote the maximum and minimum metric values attained over all sensitive groups.
We denote group max - group min as
Discrimination, which is the worst case difference in performance across groups.
When the dataset contains multiple splits, we report both split mean and split standard deviation.
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
A. A NALYSIS OF P ARETO - OPTIMAL CLASSIFIERS IN THE INFINITE DATA AND MODEL CAPACITY REGIME
In this section we analyze the form of Pareto-optimal solutions to classiﬁcation (and regression) tasks in the asymptotically ideal case where we have inﬁnite capacity hypothesis classes, that is, our hypothesis class H contains every function mapping points from the observation space X to the classiﬁcation or regression space R |Y| .
We additionally assume that the joint distributions between target variables Y and observation variables X given every sensitive attribute are known (i.e., P ( Y, X | A ) is known), and that the loss function L ( h ( x ) , y ) is convex with respect to h ( x ) .
Any joint loss of the form
[Equation]
produces solutions which are already in the Pareto front of { R a ( h ) } for any distribution of A ∼ P A .
Since the loss function is convex with respect to h ( X ) and the hypothesis class is complete, it can also be shown that for any point in the Pareto front, there exists a value of P A such that that point is reached by minimizing φ P A ( { R a ( h ) } )
Geoffrion (1968); Koski (1985); Miettinen (2012)
We can therefore analyze the Bayes-optimal classiﬁer h P A which minimizes the Naive risk φ P A ( { R a ( h ) } ) , and analytically compute the sub-population risks R a ( h P A ) induced.
In general, we can write
[Equation]
and in the particular case where target variable Y is categorical
[Equation]
and the classiﬁer loss is an L2 loss against the one-hot encoding of variable Y the equations reduce to
[Equation]