E QUAL I MPROVABILITY : A N EW F AIRNESS N OTION C ONSIDERING THE L ONG - TERM I MPACT
Anonymous authors
Paper under double-blind review
A BSTRACT
Devising a fair classiﬁer that does not discriminate against different groups is an important problem in machine learning.
Although researchers have proposed various ways of deﬁning group fairness, most of them only focused on the immediate fairness, ignoring the long-term impact of a fair classiﬁer under the dynamic scenario where each individual can improve its feature over time.
Such dynamic scenarios happen in real world, e.g. , college admission and credit loaning, where each rejected sample makes effort to change its features to get accepted afterwards.
In this dynamic setting, the long-term fairness should equalize the samples’ feature distribution across different groups after the rejected samples make some effort to improve.
In order to promote long-term fairness, we propose a new fairness notion called Equal Improvability (EI), which equalizes the potential acceptance rate of the rejected samples across different groups assuming a bounded level of effort will be spent by each rejected sample.
We analyze the properties of EI and its connections with existing fairness notions.
To ﬁnd a classiﬁer that satisﬁes
the
EI requirement, we propose and study three different approaches that solve EIregularized optimization problems.
Through experiments on both synthetic and real datasets, we demonstrate that the proposed EI-regularized algorithms encourage us to ﬁnd a fair classiﬁer in terms of EI.
Finally, we provide experimental results on dynamic scenarios which highlight the advantages of our EI metric in achieving the long-term fairness.
Codes are available in anonymous GitHub repository 1 .
I NTRODUCTION
Over the past decade, machine learning has been used in a wide variety of applications.
However, these machine learning approaches are observed to be unfair to individuals having different ethnicity, race, and gender.
As the implicit bias in artiﬁcial intelligence tools raised concerns over potential discrimination and equity issues, various researchers suggested deﬁning fairness notions and developing classiﬁers that achieve fairness.
One popular fairness notion is demographic parity (DP), which requires the decision-making system to provide output such that the groups are equally likely to be assigned to the desired prediction classes, e.g. , acceptance in the admission procedure.
DP and related fairness notions are largely employed to mitigate the bias in many realistic problems such as recruitment, credit lending, and university admissions (Zafar et al., 2017b;
Hardt et al., 2016; Dwork et al., 2012; Zafar et al., 2017a).
However, most of the existing fairness notions are zero-order , i.e. , they only focus on immediate fairness, without taking potential follow-up inequity risk into consideration.
In Fig.
1, we provide an example scenario when using a conventional zero-order fairness notion (DP in this example) has a long-term fairness issue, in a simple loan approval problem setting.
Consider two groups (group 0 and group 1) with different distributions, where each individual has one label (approve the loan or not) and two features (credit score, income) that can be improved over time.
Suppose each group consists of two clusters (with three samples each), and the distance between the clusters is different for two groups.
Fig.
1 visualizes the distributions of two groups and the decision boundary of a classiﬁer f which achieves DP among the groups.
We observe that the rejected samples (left-hand-side of the decision boundary) in group 1 are located further away from the decision boundary than the rejected samples in group 0.
As a result, the rejected applicants in group 1 need to make more effort to cross the decision boundary and get approval.
This improvability gap between the two groups can make the rejected applicants in group 1 less motivated to improve their features, which may increase the gap between different groups in the future.
[Figure]
We consider the binary classiﬁcation (accept/reject) on 12 samples (dots), where x is the feature of the sample and the color of the dot represents the group.
The given classiﬁer f is fair in terms of a popular zero-order notion called demographic parity (DP), but does not have equal improvability of rejected samples ( f ( x ) < 0 .
5 ) in two groups; the rejected samples in group 1 needs more effort  x to be accepted, i.e. , f ( x +  x )  0 .
5 , compared with the rejected samples in group 0.
This motivated the advent of ﬁrst-order fairness notions that consider the improvability of the rejected samples and thus deﬁning the group fairness after such improvement is made.
There are a few ﬁrst-order fairness notions that have been proposed by Gupta et al.
(2019); Heidari et al.
; Von Kügelgen et al.
However, as shown in Table 1, they have some limitations e.g. , vulnerable to imbalanced group negative rates or outliers.
In this paper, we introduce another ﬁrst-order fairness notion dubbed as Equal Improvability (EI), which does not suffer from these limitations.
Let x be the feature of a sample and f be a score-based classiﬁer, e.g. , predicting a sample as accepted if f ( x )  0 . 5 holds and as rejected otherwise.
We assume each rejected individual wants to get accepted in the future, thus improving its feature within a certain effort budget towards the direction that maximizes its score f ( x ) .
Under this setting, we deﬁne EI fairness as the equity of the potential acceptance rate of the different rejected groups, once each individual makes the best effort within the predeﬁned budget.
This prevents the risk of exacerbating the gap between different groups in the long run.
Our key contributions are as follows:
• We propose a new group fairness notion called Equal Improvability (EI), which aims to equalize the probability of rejected samples being qualiﬁed after a certain amount of feature improvement, for different groups.
EI encourages rejected individuals in different groups to have an equal amount of motivation to improve their feature to get accepted in the future.
We analyze the properties of EI and the connections of EI with other existing fairness notions.
• We provide three methods to ﬁnd a classiﬁer that is fair in terms of EI, each of which uses a unique way of measuring the inequity in the improvability.
Each method is solving a min-max problem where the inner maximization problem is ﬁnding the best effort to measure the EI unfairness, and the outer minimization problem is ﬁnding the classiﬁer that has the smallest fairness-regularized loss.
Experiments on synthetic/real datasets demonstrate that our algorithms ﬁnd classiﬁers having low EI unfairness.
• We run experiments on dynamic scenarios where the data and the classiﬁer evolve over multiple rounds, and show that training a classiﬁer with EI constraints is beneﬁcial for making the feature distributions of different groups identical in the long run,
i
e. , achieving the long-term fairness.
E QUAL I MPROVABILITY
Before deﬁning our new fairness notion called Equal Improvability (EI), we ﬁrst introduce necessary notations.
For an integer n , let [ n ] = { 0 , . .
. , n  1 } .
We consider a binary classiﬁcation setting where each data sample has an input feature vector x 2 X ✓
R d and a label y 2 Y = { 0 , 1 } .
In particular, we have a sensitive attribute z 2 Z
=
[ Z ] , where Z is the number of sensitive groups.
As suggested by Chen et al.
(2021), we sort d features x 2 R d into three categories: improvable features x
I 2 R d I , mutable features x M 2 R d M , and immutable features x IM 2 R d IM , where d I + d M + d IM = d holds.
Here, improvable features x I refer to the features that can be improved and can directly affect the outcome, e.g. , salary in the credit lending problem, and GPA in the school’s admission problem.
In contrast, mutable features x M can be altered, but are not directly related to the outcome, e.g. , marital status in the admission problem, and communication type in the credit lending problem.
Although individuals may manipulate these mutable features to get the desired outcome, we do not consider it as a way to make efforts as it does not affect the individual’s true qualiﬁcation status.
Immutable features x IM are features that cannot be altered, such as race, age, or date of birth.
Note that if sensitive attribute z is included in the feature vector, then it belongs to [Figure] [Figure] immutable features.
For ease of notation, we write x =
( x I , x M , x IM ) .
Let F = { f : X !
[0 , 1] } be the set of classiﬁers, where each classiﬁer is parameterized by w , i.e. , f = f w .
Given f 2 F , we consider the following deterministic prediction: ˆ y | x = 1 { f ( x )  0 .
5 } where 1 { A } = 1 if condition A holds, and{ A } =
otherwise.
We now introduce our new fairness notion.
Deﬁnition 2.1 (Equal Improvability) .
Deﬁne a norm µ : R d I !
For a given constant  > 0 , a classiﬁer f is said to achieve equal improvability with  -effort if
[Equation]
holds for all z 2 Z ,
where  x
I is the effort for improvable features and  x = (  x I , 0 , 0 ) .
Note that the condition f ( x ) < 0 .
5 represents that an individual is unqualiﬁed, and f ( x +  x )  0 .implies that the effort  x allows the individual to become qualiﬁed.
The above deﬁnition of fairness in equal improvability requires that unqualiﬁed individuals from different groups z
[ Z ] are equally likely to become qualiﬁed if appropriate effort is made.
Note that µ can be deﬁned on a case-by-case basis.
For example, we can use k x I k
= p x I > C x I , where C 2 R d I ⇥
d I is a cost matrix that is diagonal and positive deﬁnite.
Here, the diagonal terms of C characterize how difﬁcult to improve each feature.
For instance, consider the graduate school admission problem where x I contains features “number of publications” and “GPA”.
Since publishing more papers is harder than raising the GPA, the corresponding diagonal term for the number of publications feature in C should be greater than that for the GPA feature.
The constant  in Deﬁnition 2.1 can be selected depending on the classiﬁcation task and the features.
Appendix B.1 contains the interpretation of each term in Def. 2.1.
We introduce an alternative equivalent deﬁnition of EI fairness below.
Proposition 2.2.
The EI fairness notion deﬁned in Def. 2.1 has an equivalent format: a classiﬁer f achieves equal improvability with  -effort if and only if
[Equation]
holds for all z 2 Z where X  = { x : f ( x ) < 0 .
5 } is the set of features x for unqualiﬁed samples, and X imp  = { x : f ( x ) < 0 .
5 , max µ (  x I )   f ( x +  x )  0 .
5 } is the set of features x for unqualiﬁed samples that can be improved to qualiﬁed samples by adding  x satisfying µ (  x I )   .
The proof of this proposition is trivial from the deﬁnition of X  and X imp  .
Note that P ( x 2 X imp  | x 2 X  ) in the above equation indicates the probability that unqualiﬁed samples can be improved to qualiﬁed samples by changing the features within budget  .
This is how we deﬁne the “improvability” of unqualiﬁed samples, and the EI fairness notion is equalizing this improvability for all groups.
Visualization of EI.
Fig.
2 shows the geometric interpretation of EI fairness notion in Def. 2.1 and
Prop. 2.2, for a simple two-dimensional dataset having 12 samples in two groups z 2 { red , blue } .
Consider a linear classiﬁer f EI shown in the ﬁgure, where the samples at the right-hand-side of the decision boundary is classiﬁed as qualiﬁed samples ( f EI ( x )  0 . 5 ).
In Fig.
2a, we have L 1 ball at each unqualiﬁed sample, representing that these samples have a chance to improve their feature in a way that the improved feature x +  x allows the sample to be classiﬁed as qualiﬁed, i.e. , f EI ( x +  x )  0 .
One can conﬁrm that P
[Equation]
holds for each group z 2 { red , blue } , thus satisfying equal improvability according to Def. 2.1.
In
Fig.
2b, we check this in an alternative way by using the EI fairness deﬁnition in Prop. 2.2.
Here, instead of making a set of improved features at each sample, we partition the feature domain X into
[Table]
three parts: (i) the features for qualiﬁed samples X
+ = { x : f EI ( x )  0 .
5 } , (ii) the features for unqualiﬁed samples that can be improved X imp
[Equation]
5 } and (iii) the features for unqualiﬁed samples that cannot be improved X imp  = { x : f EI ( x ) < 0 .
5 , max µ (  x I )   f ( x +  x ) < 0 . 5 } .
In the ﬁgure, (ii) is represented as the green region and (iii) is shown as the yellow region.
From Prop. 2.2, EI fairness means that [Table] identical at each group z 2 { red , blue } , which is true for the example in Fig. 2b.
Comparison of EI with other fairness notions.
The suggested fairness notion equal improvability (EI) is in stark difference with existing popular zero-order fairness notions, e.g. , demographic parity, which can be “myopic” and focus only on achieving classiﬁcation fairness in the current status.
Our notion instead, uses classiﬁcation fairness as a tool to equalize the true qualiﬁcation status of different groups in the long run, thereby promoting social fairness.
We here also note that EI has differences with existing ﬁrst-order fairness notions that capture the dynamics of samples (Heidari et al., 2019; Huang et al., 2019; Gupta et al., 2019).
Table 1 compares our fairness notion with the related existing notions.
In particular, Bounded Effort (BE) fairness proposed by Heidari et al.
(2019) equalizes ‘the available reward after each individual making a bounded effort’ for different groups, which is very similar to EI when we set a proper reward function.
To be more speciﬁc, the BE fairness can be represented as in the Table 1.
Comparing this BE expression with EI in Deﬁnition 2.1, one can conﬁrm the difference: the inequality f ( x ) < 0 .
5 is located at the conditional part for EI, which is not true for BE.
EI and BE are identical if the negative prediction rates are equal across the groups, but in general they are different.
The condition f ( x ) < 0 .
5 here is very important since only looking into the unqualiﬁed members makes more sense when we consider improvability .
More importantly, the BE deﬁnition is based on reward functions and we are presenting BE in a form that is closest to our EI fairness expression.
Besides, Equal Recourse (ER) fairness proposed by Gupta et al.
(2019) suggests to equalize the average effort of different groups without limiting the amount of effort that each sample can make.
Note that ER is vulnerable to outliers.
For example, when we have an unqualiﬁed outlier sample that is located far way from the decision boundary, ER disparity will be dominated by this outlier and fail to reﬂect the overall unfairness.
Huang et al.
(2019) proposed a causal-based fairness notion to equalize the minimum level of effort such that the expected prediction score of the groups is equal to each other.
Note that, their deﬁnition is speciﬁc to causal settings and it considers the sensitive groups not the rejected samples of the sensitive groups.
The comparison with (Von Kügelgen et al., 2022) is given in Section B.4 of Appendix.
In addition to fairness notions, we also discuss other related works such as fairness-aware algorithms in Sec. 5.
Compatibility of EI with other fairness notions.
Here we prove the compatibility of three fairness notions (EI, DP, and BE), under two mild assumptions.
Assumption 2.3 ensures that EI is well-deﬁned, while Assumption 2.4 implies that the norm µ and the effort budget  are chosen such that we have nonzero probability that unqualiﬁed individuals can become qualiﬁed after making efforts.
Assumption 2.3.
For any classiﬁer f , the probability of unqualiﬁed samples for each demographic group is not equal to 0, i.e. , P ( f ( x ) < 0 .
5 , z [Equation]
Assumption 2.4.
For any classiﬁer f , the probability of being qualiﬁed after the effort for unqualiﬁed samples is not equal to 0, i.e. , P
[Equation]
Under these assumptions, the following theorem reveals the relationship between DP, EI and BE.
Theorem 2.5.
If a classiﬁer f achieves two of the following three fairness notions, DP, EI, and BE; then it has to achieve the remaining fairness notion as well.
The proof of the Theorem 2.5 is provided in Appendix A.
This theorem immediately implies the following corollary, which provides a condition such that EI and BE conﬂict with each other.
Corollary 2.6.
The above theorem says that if a classiﬁer f achieves EI and BE, it has to achieve DP.
Thus, by contraposition, if f does not achieve DP, then it cannot achieve EI and BE simultaneously.
A CHIEVING E QUAL I MPROVABILITY
In this section, we discuss methods for ﬁnding a classiﬁer that achieves EI fairness.
Following existing in-processing techniques (Zafar et al., 2017c; Donini et al., 2018; Zafar et al., 2017a; Cho et al., 2020), we focus on ﬁnding a fair classiﬁer by solving a fairness-regularized optimization problem.
To be speciﬁc, we ﬁrst derive a differentiable penalty term U  that approximates the unfairness with respect to EI, and then solve a regularized empirical minimization problem having the unfairness as the regularization term.
This optimization problem can be represented as
[Equation]
where { ( x i , y i ) } Ni =1 is the given dataset, ` : { 0 , 1 } ⇥ [0 , 1] !
R is the loss function, F is the set of classiﬁers we are searching over, and  2 [0 , 1) is a hyperparameter that balances fairness and prediction loss.
Here we consider three different ways of deﬁning the penalty term U  , which are (a) covariance-based, (b) kernel density estimator (KDE)-based, and (c) loss-based methods.
We ﬁrst introduce how we deﬁne U  in each method, and then discuss how we solve (1).
Covariance-based EI Penalty.
Our ﬁrst method is inspired by Zafar et al.
(2017c), which measures the unfairness of a score-based classiﬁer f by the covariance of the sensitive attribute z and the score f ( x ) , when the demographic parity (DP) fairness condition ( P ( f ( x ) > 0 .
5 | z = z ) =
P ( f ( x ) > 0 . 5) holds for all z ) is considered.
The intuition behind this idea of measuring the covariance is that a perfect fair DP classiﬁer should have zero correlation between z and f ( x ) .
By applying similar approach to our fairness notion in Def. 2.1, the EI unfairness is measured by the covariance between the sensitive attribute z and the maximally improved score of rejected samples within the effort budget.
In other words, (Cov( z , max k  x I k  f ( x +  x ) | f ( x ) < 0 . 5)) 2 represents the EI unfairness of a classiﬁer f where we took the square to penalize negative correlation case as well.
Let I
= { i : f ( x i ) < 0 .
5 } be the set of indices of unqualiﬁed samples, and ¯ z =
P
i 2 I  z
i / | I  | .
Then, EI unfairness can be approximated by the square of the empirical covariance, i.e. ,
U
[Equation]
Since
P i[-
-]
[Equation]
⌘ = 0 from
P i
[Equation]
z
[Equation]
0 , we have U  =
[Equation]
KDE-based EI Penalty.
The second method is inspired by Cho et al.
(2020), which suggests to ﬁrst approximate the probability density function of the score f ( x ) via kernel density estimator (KDE)
and then put the estimated density formula into the probability term in the unfairness penalty.
Recall that given m samples y 1 , . . .
, y m , the true density g y on y is estimated by KDE as ˆ g y
(ˆ y ) , 1 mh P
[Equation]
⇣
[Equation] y  [Equation] ⌘ , where g k is a kernel function and h is a smoothing parameter.
Here we apply this KDE-based method for estimating the EI penalty term in Def. 2.1.
Let y max i = max k
x I i k  f ( x i +  x i ) be the maximum score achievable by improving feature
x i within budget  , and I  ,z = { i : f ( x i ) < 0 .
5 , z i =
z } be the set of indices of unqualiﬁed samples of group z .
Then, the density of y max i for the unqualiﬁed samples in group z can be approximated as
[Equation]
Then, the estimate on the left-hand-side (LHS) probability term in Def. 2.1 is represented as ˆ
P
[Equation]
⌘ where G k ( ⌧ ) ,
R 1 ⌧ g k ( y )d y .
Similarly, we can estimate the righthand-side (RHS) probability term in Def. 2.1, and the EI-penalty U  is computed as the summation of the absolute difference of the two probability values (LHS and RHS) among all groups z .
Loss-based EI Penalty.
Another common way of approximating the fairness violation as a differentiable term is to compute the absolute difference of group-speciﬁc losses (Roh et al., 2021;
Shen et al., 2022).
Following the spirit of EI notion in Def. 2.1, we deﬁne EI loss of group z as ˜
L z ,
[Equation]
[Equation]
.
Here, ˜ L z measures how far the rejected samples in group z are away from being accepted after the feature improvement within budget  .
Similarly, EI loss for all groups is written as ˜ L ,
[Equation]
˜
L z .
Finally, the EI penalty term is deﬁned as U  ,
[Equation]
[Equation]
Solving (1).
For each approach deﬁned above (covariance-based, KDE-based and loss-based), the penalty term U  is deﬁned uniquely.
Note that in all cases, we need to solve a maximization problem max k  x I k  f ( x +  x ) in order to get U  .
Since (1) is a minimization problem containing U  in the cost function, it is essentially a minimax problem.
We leverage adversarial training techniques to solve (1).
The inner maximization problem is solved using one of two methods: (i) derive the closed-form solution for generalized linear models, (ii) use projected gradient descent for general settings.
The details can be found in Appendix B.2.
E XPERIMENTS
In this section, we provide empirical results on our EI fairness notion.
To measure the fairness violation, we use EI disparity = max z 2 [ Z ] | P (max µ (  x I ) <
f ( x +  x )  0 .
5 | f ( x ) < 0 .
5 , z = z )  P (max µ (  x I ) <
f ( x +  x )  0 .
5 | f ( x ) < 0 . 5) | . First, we show that our methods suggested in
Sec. 3 achieve EI fairness in various real/synthetic datasets.
Second, focusing on the dynamic scenario where each individual can make effort to improve its outcome, we demonstrate that training an EI classiﬁer at each time step promotes achieving the long-term fairness, i.e. , the feature distribution of two groups become identical in the long run.
Codes are available in anonymous GitHub repository 3 .
4. S UGGESTED METHODS ACHIEVE EI FAIRNESS
Recall that Sec. 3 provided three approaches for achieving EI fairness.
Here we check whether such methods successfully ﬁnd a classiﬁer with small EI disparity, compared with ERM which does not have fairness constraints.
Due to the space limitation, the main body contains the results for logistic regression (LR) only, and in Appendix C contains the performance for multi-layer perceptron (MLP) as well as the effect of over-parameterization.
Experiment setting.
For all experiments, we use the Adam optimizer and cross entropy loss.
We perform cross-validation on the training set to ﬁnd the best hyperparameter.
We provide statistics for ﬁve trials having different random seeds.
For KDE-based approach, we use Gaussian kernel.
Datasets.
We perform the experiments on one synthetic dataset, and two real datasets: German
Statlog Credit (Dua & Graff, 2017), and ACSIncome-CA (Ding et al., 2021).
The synthetic dataset has two non-sensitive attributes x
= ( x 1 , x 2 ) , one binary sensitive attribute z, and a binary label y.
Both features x 1 and x 2 are assumed to be improvable.
We generate 20,000 samples where ( x , y , z ) pair of each sample is generated independently as below.
We deﬁne z and ( y | z = z ) as Bernoulli random variables for all z 2 { 0 , 1 } , and deﬁne ( x | y
=
y, z = z ) as multivariate Gaussian random variables for all y, z 2 { 0 , 1 } .
The numerical details are in Appendix C.
The maximum effort  for this dataset is set to 0.5.
The ratio of the training versus test data is 4:1.
German Statlog Credit Data contains 1,000 samples and the ratio of the training versus test data is 4:1.
The task is to predict the credit risk of an individual given its ﬁnancial status.
Following Jiang &
Nachum (2020), we divide the samples into two groups using the age of thirty as the boundary, i.e. , z = 1 for samples with age above thirty.
Four features x are considered as improvable: checking account , saving account , housing and occupation , all of which are ordered categorical features.
For example, the occupation feature has four levels: (1) unemployed, (2) unskilled, (3) skilled, and (4) highly qualiﬁed.
We set the maximum effort  = 1 , meaning that an unskilled man (with level 2) can become a skilled man, but cannot be a highly qualiﬁed man.
ACSIncome-CA dataset consists of data for 195,665 people and is split into training/test set in the ratio of 4:1.
The task is predicting whether a person’s income would exceed 50K USD per year.
We use sex as the sensitive attribute; we have two sensitive groups, male and female.
We select education level (ordered categorical feature) as the improvable feature.
We set the maximum effort  = 3 .
[Figure]
We run three EI-regularized methods suggested in Sec.for different regularizer coefﬁcient  and plot the frontier lines.
For the synthetic dataset, the tradeoff curve for the ideal classiﬁer is located at the bottom left corner, which is similar to the curves of proposed EI-regularized methods.
This shows that our methods successfully ﬁnd classiﬁers balancing EI disparity and error rate.
Results.
Table 2 shows the test error rate and test EI disparity (disp.) for ERM and our three EI-regularized methods (covariance-based, KDEbased and loss-based) suggested in
Sec. 3.
For all three datasets, our EIregularized methods successfully reduce the EI disparity without increasing the error rate too much, compared with ERM.
Figure 3 shows the tradeoff between error rate and EI disparity of our EI-regularized methods.
We marked the dots after running each method multiple times with different penalty coefﬁcient  , and plotted the frontier line.
For the synthetic dataset with Gaussian feature, we numerically obtained the performance of the optimal EI classiﬁer, which is added in the yellow line at the bottom left corner of the ﬁrst column plot.
The details of ﬁnding the optimal EI classiﬁer is in
Appendix B.3.
One can conﬁrm that our methods regularizing EI are having similar tradeoff curves for the synthetic dataset.
Especially, for synthetic dataset, the tradeoff curve of our methods nearly achieves that of the optimal EI classiﬁer.
For German and ACSIncome-CA datasets, the loss-based method is having a slightly better tradeoff curve than other methods.
4. EI PROMOTES LONG - TERM FAIRNESS IN DYNAMIC SCENARIOS
Recall that the motivation for proposing EI is to achieve long-term fairness , which is equalizing the feature distribution of samples in different groups in the long run, under the dynamic scenario where each individual can improve its feature.
In this section, we provide simulation results on dynamic setting, which show that training a classiﬁer with EI constraint encourages the long-term fairness.
4.2. D YNAMIC S YSTEM D ESCRIPTION
We consider a binary classiﬁcation problem under the dynamic scenario with T rounds, where the improvable feature x t 2 R and the label y t 2 { 0 , 1 } of each sample as well as the classiﬁer f t evolve at each round t 2 { 0 , · · · , T  1 } .
We denote the sensitive attribute as z 2 { 0 , 1 } , and the estimated label as ˆ y t .
We assume z ⇠ Bern(0 . 5) and ( x t | z = z ) ⇠ P ( z ) t = N ( µ ( z ) t , {  ( z ) t } 2 ) .
To mimic the admission problem, we only accept a fraction ↵ 2 (0 , 1) of the population, i.e. , the true label is modeled as y t = 1 x t   ( t ) ↵ , where  ( t ) ↵ is the (1  ↵ ) percentile of the feature distribution at round t .
We consider z-aware linear classiﬁer outputting ˆ y t = 1 x t  ⌧ ( z ) t , which is parameterized by the thresholds ( ⌧ (0) t , ⌧ (1) t ) for two groups.
Note that this classiﬁcation rule is equivalent to deﬁning score function f t ( x, z )
= 1 / (exp( ⌧ ( z ) t  x ) + 1) and ˆ y t = 1 f ( x t , z )  0 .
Updating data parameters ( µ ( z ) t ,  ( z ) t ) .
At each round t , we allow each sample can improve its feature from x to x + ✏ ( x ) .
Here we model ✏ ( x ) = ⌫ ( x ; z )
= 1 ( ⌧ ( z ) t  x +  ) 2 1 x< ⌧ ( z ) t for a constant  > 0 .
In this model, the rejected samples with larger gap ( ⌧ ( z ) t  x ) with the decision boundary are making less effort  x , which is inspired by the intuition that a rejected sample is less motivated to improve its feature if it needs to take a large amount of effort to get accepted in one scoop.
After such effort is made, we compute the mean and standard deviation of each group: µ ( z ) t +1 =
R 1 1 ( x + ⌫ ( x ; z ))
( x ; µ ( z ) t ,  ( z ) t ) d x and  ( z ) t +1 =
[Equation]
0 .0 .(i)  0 .
 .(ii)  0 .0 .
.(iii)  0 .0 .(iv)
ERM DP
BE ER
EI (Ours)
Round L o n g - t e r m U n f a i r n e ss where  ( · ; µ,  ) is the pdf of N ( µ,  2 ) .
We assume that the feature x t +1 in the next round follows
a
Gaussian distribution parameterized by ( µ ( z ) t +1 ,  ( z ) t +1 ) for ease of simulation.
Updating classiﬁer parameters ( ⌧ (0) t , ⌧ (1) t ) .
At each round t , we update the classiﬁer depending on the current feature distribution x t .
The EI classiﬁer considered in this paper updates ( ⌧ (0) t , ⌧ (1) t ) as below.
Note that the maximized score max k  x I k  f ( x +  x ) in Def.
2.1 can be written as f t ( x t +  t , z ) , and the equation max k  x I k  f ( x +  x )  0 .
5 is equivalent to x t +  t > ⌧ ( z ) t .
Consequently, EI classiﬁer obtains ( ⌧ (0) t , ⌧ (1) t ) by solving min ⌧ (0) t , ⌧ (1) t
[Equation]
where c 2 [0 , 1) is the maximum classiﬁcation error rate we allow, and  t is the effort level at iteration t .
In our experiments,  t is chosen as the mean efforts the population makes, i.e. ,  t = 0 .
P 1 z =
R 1 1 ⌫ ( x ; z )  ( x ; µ ( z ) t ,  ( z ) t ) d x .
We can similarly obtain the classiﬁer for DP, BE and ER constraints, details of which are in Appendix C.4.
In the experiments, we numerically obtain the solution of this optimization problem.
4.2. E XPERIMENTS ON LONG - TERM FAIRNESS
We ﬁrst initialize the feature distribution in a way that both sensitive groups have either different mean ( i.e. , µ (0)0 6 = µ (1)0 ) or different variance ( i.e. ,  (0)0 6 =
(1)0 ).
At each round t 2 { 1 , · · · , T } , we update the data parameter ( µ ( z ) t ,  ( z ) t ) for group z 2 { 0 , 1 } and the classiﬁer parameter ( ⌧ (0) t , ⌧ (1) t ) , following the rule described in Sec. 4.2.1.
At each round t 2 { 1 , · · · , T } , we measure the long-term unfairness deﬁned as the total variation distance between the two group distributions: d TV ( P (0) , P (1) ) =
R 1 1 |  ( x ; µ (0) [Equation] ,  ( [Equation]
[Equation]
)   ( x ; µ (1) [Equation] ,  [Equation]
[Equation]
) | d x
[Equation] We run experiments on four different initial feature distributions: (i) ( µ (0)0 ,  (0)0 , µ (1)0 ,  (1)0 ) =
(0 , 1 , 1 , 0 . 5) , (ii) ( µ (0)0 ,  (0)0 , µ (1)0 ,  (1)0 ) =
5 , 1 , 1) , (iii) ( µ (0)0 ,  (0)0 , µ (1)0 ,  (1)0 ) =
(0 , 2 , 0 , 1) , and (iv) ( µ (0)0 ,  (0)0 , µ (1)0 ,  (1)0 ) =
5 , 1 , 0 . 5) , respectively.
We set ↵ = 0 .
2 , c = 0 .
1 ,  = 0 .
Baselines.
We compare our EI classiﬁer with multiple baselines, including the empirical risk minimization (ERM) and algorithms with fairness constraints: demographic parity (DP), bounded effort (BE) (Heidari et al., 2019), and equal recourse (ER) (Gupta et al., 2019).
Results.
Fig.
4 shows how the long-term unfairness d TV ( P (0) t , P (1) t ) changes as a function of round t , for cases (i) – (iv) having different initial feature distribution.
Note that fairness-constrained algorithms (DP, BE, ER and EI) enjoys lower long-term unfairness compared with ERM, for case (i), (ii) and (iv).
More importantly, EI accelerates the process of mitigating long-term unfairness, compared to other fairness notions.
This observation highlights the beneﬁt of EI in promoting true equity of groups in the long-term.
Fig.
5 visualizes the initial distribution (at the leftmost column) and the evolved distribution at round t = 3 for multiple algorithms (at the rest of the columns).
Each row represents different initial feature distribution, for cases (i) – (iv).
One can conﬁrm that EI brings the distribution of the two groups closer, compared with ERM, DP, BE, and ER.
R ELATED W ORKS
Fairness metric.
Group fairness is a class of fairness notions requiring the classiﬁer to treat different groups similarly.
Most of the existing group fairness notions are myopic, measuring fairness by only comparing the positive rate of certain groups or subgroups at the current snapshot (Hardt et al., 2016; Zafar et al., 2017a).
In contrast, EI suggested in this paper is more farsighted by taking improvability
[Figure]
Initial distribution
[Figure]
d TV
[Figure]
31 ERM d TV =0 .
[Figure]
DP d TV =0 .
20 BE d TV =0 .
22 ER d TV
[Figure]
At each row, the leftmost column shows the initial distribution and the rest of the columns show the evolved distribution for each algorithm, under the dynamic setting.
Compared with existing fairness notions (DP, BE, and ER), our EI fairness achieves smaller feature distribution gap between groups.
into consideration.
There exist a few works that propose effort-based fairness notions (Heidari et al., 2019; Huang et al., 2019; Gupta et al., 2019; Von Kügelgen et al., 2022) that capture the potential follow-up impact.
EI is closely related to, but also clearly distinct from these effort-based fairness notions, particularly in terms of motivation.
Note that Heidari et al.
(2019) aims to propose fairness notions that capture the dynamics, while Gupta et al.
and
Huang et al.
(2019) aim to propose reasonable fairness notions that take effort into consideration.
Our work takes one-step further, focusing on promoting the long-term fairness.
Besides, Huang et al.
(2019) only consider the causal setting and are restricted to certain models.
Without the restricted assumptions introduced by Huang et al.
(2019), our work applies to more general settings, and provides a fairness metric that is easy to measure.
We provide a more systematic comparison between EI and related work in Sec. 2.
Fairness-aware algorithms.
Most of the existing fair learning techniques fall into three categories: i) pre-processing approaches (Kamiran & Calders, 2012; 2010; Gordaliza et al., 2019; Jiang &
Nachum, 2020), which primarily involves massaging the dataset to remove the bias; ii) in-processing approaches (Fukuchi et al., 2013; Kamishima et al., 2012; Calders & Verwer, 2010; Zafar et al., 2017c;a; Zhang et al., 2018; Cho et al., 2020; Roh et al., 2020; 2021; Shen et al., 2022), adjusting the model training for fairness; iii) post-processing approaches (Calders & Verwer, 2010; Alghamdi et al., 2020; Wei et al., 2020; Hardt et al., 2016) which achieve fairness by modifying a given unfair classiﬁer.
Prior work (Woodworth et al., 2017) showed that the in-processing approach generally outperforms other approaches due to its ﬂexibility.
Hence, we focus on the in-processing approach and propose three methods to achieve EI.
These methods achieve EI by solving fairness-regularized optimization problems.
In particular, our proposed fairness regularization terms are inspired by Zafar et al.
(2017c);
Cho et al.
Roh et al.
; Shen et al.
Fairness dynamics.
There are also a few attempts to study the long-term impact of different fairness policies (Zhang et al., 2020; Heidari et al., 2019;
Hashimoto et al., 2018).
In particular, Hashimoto et al.
(2018) studies how ERM ampliﬁes the unfairness of a classiﬁer in the long run.
The key idea is that if the classiﬁer of the previous iteration favors a certain candidate group, then the candidate groups will be more unbalanced since fewer individuals from the unfavored group will apply for this position.
Thus, the negative feedback leads to a more unfair classiﬁer.
In contrast, Heidari et al.
(2019) and Zhang et al.
(2020) focus more on long-term impact instead of classiﬁcation fairness.
To be speciﬁc, Heidari et al.
(2019) studies how fairness intervention affects the different groups in terms of evenness, centralization, and clustering by simulating the population’s response through effort.
Zhang et al.
(2020) investigates how different fairness policies affect the gap between the qualiﬁcation rates of different groups under a partially observed Markov decision process.
Besides, there are a few works which study how individuals may take strategic actions to improve their outcomes given a classiﬁer (Chen et al., 2021).
However, Chen et al.
(2021) aims to address this problem by designing an optimization problem that is robust to strategic manipulation, which is orthogonal to our focus.
C ONCLUSION
In this paper, we proposed a group fairness notion called Equal Improvability (EI), which equalizes the potential acceptance of rejected samples in different groups, when appropriate effort is made by the rejected samples.
We analyzed the properties of EI fairness, and provided three approaches to ﬁnd a classiﬁer that achieves EI.
Experimental results showed that the proposed approaches reduce the EI disparity.
Lastly, we formulated a dynamic model to showcase the beneﬁt of EI in promoting the equity of feature distribution of different groups.
Extending our work to settings with multiple sensitive attributes and high-dimensional features is remained as a future direction.