C ONDITIONAL L EARNING OF F AIR R EPRESENTATIONS
A BSTRACT
We propose a novel algorithm for learning fair representations that can simultaneously mitigate two notions of disparity among different demographic subgroups in the classiﬁcation setting.
Two key components underpinning the design of our algorithm are balanced error rate and conditional alignment of representations.
We show how these two components contribute to ensuring accuracy parity and equalized false-positive and false-negative rates across groups without impacting demographic parity.
Furthermore, we also demonstrate both in theory and on two real-world experiments that the proposed algorithm leads to a better utility-fairness trade-off on balanced datasets compared with existing algorithms on learning fair representations for classiﬁcation.
I NTRODUCTION
High-stakes settings, such as loan approvals, criminal justice, and hiring processes, use machine learning tools to help make decisions.
A key question in these settings is whether the algorithm makes fair decisions.
In settings that have historically had discrimination, we are interested in deﬁning fairness with respect to a protected group, the group which has historically been disadvantaged.
The rapidly growing ﬁeld of algorithmic fairness has a vast literature that proposes various fairness metrics, characterizes the relationship between fairness metrics, and describes methods to build classiﬁers that satisfy these metrics (Chouldechova & Roth, 2018; Corbett-Davies & Goel, 2018).
Among many recent attempts to achieve algorithmic fairness (Dwork et al., 2012; Hardt et al., 2016;
Zemel et al., 2013; Zafar et al., 2015), learning fair representations has attracted increasing attention due to its ﬂexibility in learning rich representations based on advances in deep learning (Edwards & Storkey, 2015; Louizos et al., 2015; Beutel et al., 2017; Madras et al., 2018).
The backbone idea underpinning this line of work is very intuitive: if the representations of data from different groups are similar to each other, then any classiﬁer acting on such representations will also be agnostic to the group membership.
However, it has long been empirically observed (Calders et al., 2009) and recently been proved (Zhao & Gordon, 2019) that fairness is often at odds with utility.
For example, consider demographic parity, which requires the classiﬁer to be independent of the group membership attribute.
It is clear that demographic parity will cripple the utility if the demographic group membership and the target variable are indeed correlated.
To escape such inherent trade-off, other notions of fairness, such as equalized odds (Hardt et al., 2016), which asks for equal false positive and negative rates across groups, and accuracy parity (Zafar et al., 2017), which seeks equalized error rates across groups, have been proposed.
It is a well-known result that equalized odds is incompatible with demographic parity (Barocas et al., 2017) except in degenerate cases where group membership is independent of the target variable.
Accuracy parity and the so-called predictive rate parity (c.f. Deﬁnition 2.4) could be simultaneously achieved, e.g., the COMPAS tool (Dieterich et al., 2016).
Furthermore, under some conditions, it is also known that demographic parity can lead to accuracy parity (Zhao & Gordon, 2019).
However, whether it is possible to simultaneously guarantee equalized odds and accuracy parity remains an open question.
In this work, we provide an afﬁrmative answer to the above question by proposing an algorithm to align the conditional distributions (on the target variable) of representations across different demographic subgroups.
The proposed formulation is a minimax problem that admits a simple reduction to cost-sensitive learning.
The key component underpinning the design of our algorithm is the balanced error rate (BER, c.f. Section 2) (Feldman et al., 2015; Menon & Williamson, 2018), over the target variable and protected attributes.
We demonstrate both in theory and on two real-world experiments that together with the conditional alignment, BER helps our algorithm to simultaneously ensure accuracy parity and equalized odds across groups.
Our key contributions are summarized as follows:
• We prove that BER plays a fundamental role in ensuring accuracy parity and a small joint error across groups.
Together with the conditional alignment of representations, this implies that we can simultaneously achieve equalized odds and accuracy parity.
Furthermore, we also show that when equalized odds is satisﬁed, BER serves as an upper bound on the error of each subgroup.
These results help to justify the design of our algorithm in using BER instead of the marginal error as our loss functions.
• We provide theoretical results that our method achieves equalized odds without impacting demographic parity.
This result shows that we can preserve the demographic parity gap for free while simultaneously achieving equalized odds.
• Empirically, among existing fair representation learning methods, we demonstrate that our algorithm is able to achieve a better utility on balanced datasets.
On an imbalanced dataset, our algorithm is the only method that achieves accuracy parity; however it does so at the cost of decreased utility.
We believe our theoretical results contribute to the understanding on the relationship between equalized odds and accuracy parity, and the proposed algorithm provides an alternative in real-world scenarios where accuracy parity and equalized odds are desired.
P RELIMINARY
We ﬁrst introduce the notations used throughout the paper and formally describe the problem setup and various deﬁnitions of fairness explored in the literature.
Notation We use X ⊆ R d and Y = { 0 , 1 } to denote the input and output space.
Accordingly, we use X and Y to denote the random variables which take values in X and Y , respectively.
Lower case letters x and y are used to denote the instantiation of X and Y .
To simplify the presentation, we use A ∈ { 0 , 1 } as the sensitive attribute, e.g., race, gender, etc.
1 Let H be the hypothesis class of classiﬁers.
In other words, for h ∈ H , h : X → Y is the predictor that outputs a prediction.
Note that even if the predictor does not explicitly take the sensitive attribute A as input, this fairness through blindness mechanism can still be biased due to the potential correlations between X and
A .
In this work we study the stochastic setting where there is a joint distribution D over X, Y and
A from which the data are sampled.
To keep the notation consistent, for a, y ∈ { 0 , 1 } , we use D a to mean the conditional distribution of D given A = a and D y to mean the conditional distribution of D given Y =
y .
For an event E , D ( E ) denotes the probability of E under D .
In particular, in the literature of fair machine learning, we call D
( Y = 1) the base rate of distribution D
and we use ∆ BR ( D , D (cid:48) ) :
= |D ( Y = 1) − D (cid:48)
( Y = 1) | to denote the difference of the base rates between two distributions D and D (cid:48) over the same sample space.
Given a feature transformation function g : X → Z that maps instances from the input space X to feature space Z , we deﬁne g (cid:93) D :
= D ◦ g − 1 to be the induced (pushforward) distribution of D under g , i.e., for any event E (cid:48) ⊆
Z , g (cid:93) D ( E (cid:48) ) :
= D ( g − 1 ( E (cid:48) ))
=
D ( { x ∈ X | g ( x ) ∈ E (cid:48) } ) .
To measure the discrepancy between distributions, we use d TV ( D , D (cid:48) ) to denote the total variation between them: d TV ( D , D (cid:48) ) :
= sup E |D ( E ) − D (cid:48)
( E ) | .
In particular, for binary random variable Y , it can be readily veriﬁed that the total variation between the marginal distributions D ( Y ) and D (cid:48)
( Y ) reduces to the difference of their base rates, ∆ BR ( D , D (cid:48) ) .
To see this, realize that d TV ( D ( Y ) , D (cid:48)
( Y ))
=
max {|D ( Y = 1) − D (cid:48)
( Y = 1) | , |D ( Y = 0)
− D (cid:48)
( Y = 0) |} =
|D ( Y = 1) − D (cid:48)
( Y = 1) | =
∆ BR ( D , D (cid:48) ) by deﬁnition.
Given a joint distribution D , the error of a predictor h under D is deﬁned as Err D ( h ) :
= E D [ | Y − h ( X ) | ] .
Note that for binary classiﬁcation problems, when h ( X ) ∈ { 0 , 1 } , Err D ( h ) reduces to the true error rate of binary classiﬁcation.
To make the notation more compact, we may drop the subscript D when it is clear from the context.
Furthermore, we use CE D ( (cid:98) Y || Y ) to denote the crossentropy loss function between the predicted variable (cid:98) Y and the true label distribution Y over the joint distribution D .
For binary random variables Y , we deﬁne BER D ( (cid:98) Y || Y ) to be the balanced error rate of predicting Y using (cid:98) Y , e.g., BER D ( (cid:98) Y || Y ) :
= D ( (cid:98) Y =
| Y = 1)
+ D ( (cid:98) Y = 1 | Y = 0) .
Realize that D ( (cid:98) Y =
| Y = 1) is the false negative rate (FNR) of (cid:98) Y and D ( (cid:98) Y = 1 | Y = 0) corresponds to the false positive rate (FPR).
So the balanced error rate can also be understood as the sum of FPR and FNR using the predictor (cid:98) Y .
We can similarly deﬁne BER D ( (cid:98)
A || A ) as well.
Problem Setup
In this work we focus on group fairness where the group membership is given by the sensitive attribute A .
We assume that the sensitive attribute A is available to the learner during training phase, but not inference phase.
As a result, post-processing techniques to ensure fairness are not feasible under our setting.
In the literature, there are many possible deﬁnitions of fairness (Narayanan, 2018), and in what follows we provide a brief review of the ones that are mostly relevant to this work.
Deﬁnition 2.1 (Demographic Parity (DP)) .
Given a joint distribution D , a classiﬁer (cid:98) Y satisﬁes demographic parity if (cid:98) Y is independent of A .
When (cid:98) Y is a deterministic binary classiﬁer, demographic parity reduces to the requirement that
D 0 ( (cid:98) Y = 1)
=
D 1 ( (cid:98) Y = 1) , i.e., positive outcome is given to the two groups at the same rate.
Demographic parity is also known as statistical parity , and it has been adopted as the default deﬁnition of fairness in a series of work (Calders & Verwer, 2010; Calders et al., 2009; Edwards &
Storkey, 2015; Johndrow et al., 2019; Kamiran & Calders, 2009; Kamishima et al., 2011;
Louizos et al., 2015; Zemel et al., 2013; Madras et al., 2018; Adel et al., 2019).
It is not surprising that demographic parity may cripple the utility that we hope to achieve, especially in the common scenario where the base rates differ between two groups (Hardt et al., 2016).
Formally, the following theorem characterizes the trade-off in terms of the joint error across different groups:
Theorem 2.1.
(Zhao & Gordon, 2019) Let (
cid:98) Y = h ( g ( X )) be the classiﬁer.
Then Err D 0 ( h ◦ g ) +
Err D 1 ( h ◦ g ) ≥ ∆ BR ( D 0 , D 1 ) − d TV
( g (cid:93) D 0 , g (cid:93) D 1 ) .
In this case of representations that are independent of the sensitive attribute A , then the second term d
TV ( g (cid:93) D 0 , g (cid:93)
D 1 ) becomes 0, and this implies:
[Equation]
At a colloquial level, the above inequality could be understood as an uncertainty principle which says:
For fair representations, it is not possible to construct a predictor that simultaneously minimizes the errors on both demographic subgroups.
More precisely, by the pigeonhole principle, the following corollary holds:
Corollary 2.1.
If d TV ( g (cid:93) D 0 , g (cid:93) D 1 )
= 0 , then for any hypothesis h , max { Err D 0 ( h ◦ g ) , Err D 1 ( h ◦ g ) } ≥ ∆ BR ( D 0 , D 1 ) / 2 .
In words, this means that for fair representations in the demographic parity sense, at least one of the subgroups has to incur a prediction error of at least ∆ BR ( D 0 , D 1 ) / 2 which could be large in settings like criminal justice where ∆ BR ( D 0 , D 1 ) is large.
In light of such inherent trade-off, an alternative deﬁnition is accuracy parity , which asks for equalized error rates across different groups:
Deﬁnition 2.2 (Accuracy Parity) .
Given a joint distribution D , a classiﬁer (cid:98) Y satisﬁes accuracy parity if D 0 ( (cid:98) Y (cid:54) =
Y )
=
D 1 ( (cid:98) Y (cid:54)
=
Y ) .
A violation of accuracy parity is also known as disparate mistreatment (Zafar et al., 2017).
Different from the deﬁnition of demographic parity, the deﬁnition of accuracy parity does not eliminate the perfect predictor even when the base rates differ across groups.
Of course, other more reﬁned deﬁnitions of fairness also exist in the literature, such as equalized odds (Hardt et al., 2016).
Deﬁnition 2.3 (Equalized Odds, a.k.a.
Positive Rate Parity) .
Given a joint distribution D , a classiﬁer (cid:98) Y satisﬁes equalized odds if (cid:98) Y is independent of A conditioned on Y .
The deﬁnition of equalized odds essentially requires equal true positive and false positive rates between different groups, hence it is also known as positive rate parity .
Analogous to accuracy parity, equalized odds does not eliminate the perfect classiﬁer (Hardt et al., 2016), and we will also justify this observation by formal analysis shortly.
Last but not least, we have the following deﬁnition for predictive rate parity:
Deﬁnition 2.4 (Predictive Rate Parity) .
Given a joint distribution D , a classiﬁer (cid:98) Y satisﬁes predictive rate parity if D 0 ( Y = 1 | (cid:98) Y = c ) =
D
( Y = 1 | (cid:98) Y = c ) , ∀ c ∈ [0 , 1] .
Note that in the above deﬁnition we allow the classiﬁer (cid:98) Y to be probabilistic, meaning that the output of (cid:98) Y could be any value between 0 and 1.
For the case where (cid:98) Y is deterministic, Chouldechova (2017) showed that no deterministic classiﬁer can simultaneously satisfy equalized odds and predictive rate parity when the base rates differ across subgroups and the classiﬁer is not perfect.
A LGORITHM AND A NALYSIS
In this section we ﬁrst give the proposed optimization formulation and then discuss through formal analysis the motivation of our algorithmic design.
Speciﬁcally, we show in Section 3.1 why our formulation helps to escape the utility-fairness trade-off.
We then in Section 3.2 formally prove that the BERs in the objective function could be used to guarantee a small joint error across different demographic subgroups.
In Section 3.3 we establish the relationship between equalized odds and accuracy parity by providing an upper bound of the error gap in terms of both BER and the equalized odds gap.
We conclude this section with a brief discussion on the practical implementation of the proposed optimization formulation in Section 3.4.
Due to the space limit, we defer all the proofs to the appendix and focus on explaining the high-level intuition and implications of our results.
As brieﬂy discussed in Section 5, a dominant approach in learning fair representations is via adversarial training.
Speciﬁcally, the following objective function is optimized:
[Equation]
In the above formulation, the ﬁrst term corresponds to minimization of prediction loss of the target variable and the second term represents the loss incurred by the adversary h (cid:48) .
Overall this minimax optimization problem expresses a trade-off (controlled by the hyperparameter λ > 0 ) between utility and fairness through the representation learning function g : on one hand g needs to preserve sufﬁcient information related to Y in order to minimize the ﬁrst term, but on the other hand g also needs to ﬁlter out information related to A in order to maximize the second term.
3. C ONDITIONAL L EARNING OF F AIR R EPRESENTATIONS
However, as we introduced in Section 2, the above framework is still subjective to the inherent trade-off between utility and fairness.
To escape such a trade-off, we advocate for the following optimization formulation instead:
[Equation]
Note that here we optimize over two distinct adversaries, one for each conditional distribution
D y , y ∈ { 0 , 1 } .
Intuitively, the main difference between (2) and (1) is that we use BER as our objective function in both terms.
By deﬁnition, since BER corresponds to the sum of Type-I and
Type-II errors in classiﬁcation, the proposed objective function essentially minimizes the conditional errors instead of the original marginal error:
[Equation]
which means that the loss function gives equal importance to the classiﬁcation error from both groups.
Note that the BERs in the second term of (2) are over D y , y ∈ { 0 , 1 } .
Roughly speaking, the second term encourages alignment of the the conditional distributions g (cid:93)
D y 0 and g (cid:93)
D y 1 for y ∈ { 0 , 1 } .
The following proposition shows that a perfect conditional alignment of the representations also implies that any classiﬁer based on the representations naturally satisﬁes the equalized odds criterion:
Proposition 3.1.
For g : X → Z , if d TV ( g (cid:93) D y 0 , g (cid:93) D y 1 ) = 0 , ∀ y ∈ { 0 , 1 } , then for any classiﬁer h :
Z → { 0 , 1 } , h ◦ g satisﬁes equalized odds.
To understand why we aim for conditional alignment of distributions instead of aligning the marginal feature distributions, the following proposition characterizes why such alignment will help us to escape the previous trade-off:
Proposition 3.2.
For g : X → Z , if d TV ( g
[Equation]
} , then for any classiﬁer
[Equation]
As a corollary, this implies that the lower bound given in Theorem 2.1 now vanishes if we instead align the conditional distributions of representations:
[Equation]
where the ﬁrst inequality is due to Lemma 3.1 (Zhao & Gordon, 2019) and the triangle inequality by the d TV ( · , · ) distance.
Of course, the above lower bound can only serve as a necessary condition but not sufﬁcient to ensure a small joint error across groups.
Later (c.f. Theorem 3.2) we will show that together with a small BER on the target variable, it becomes a sufﬁcient condition as well.
3. T HE P RESERVATION OF D EMOGRAPHIC P ARITY G AP AND S MALL J OINT E RROR
In this section we show that learning representations by aligning the conditional distributions across groups cannot increase the DP gap as compared to the DP gap of Y .
Before we proceed, we ﬁrst introduce a metric to measure the deviation of a predictor from satisfying demographic parity:
Deﬁnition 3.1 (DP Gap) .
Given a joint distribution D , the demographic parity gap of a classiﬁer (cid:98) Y is ∆ DP (
[Equation] [Equation]
=
[Equation]
) | .
Clearly, if ∆ DP ( (cid:98) Y ) = 0 , then (cid:98) Y satisﬁes demographic parity.
To simplify the exposition, let γ a :
= D a ( Y = 0) , ∀ a ∈ { 0 , 1 } .
We ﬁrst prove the following lemma:
Lemma 3.1.
Assume the conditions in Proposition 3.1 hold and let
(cid:98) Y =
h ( g ( X )) be the classiﬁer, then |D
( (cid:98) Y = y ) − D 1 ( (cid:98) Y = y ) | ≤ | γ 0 − γ 1 | · (cid:0) D
( (cid:98) Y =
y )
+ D 1 ( (cid:98) Y =
y ) (cid:1) , ∀ y ∈ { 0 , 1 } .
Lemma 3.1 gives an upper bound on the difference of the prediction probabilities across different subgroups.
Applying Lemma 3.1 twice for y =
and y
= 1 , we can prove the following theorem:
Theorem 3.1.
Assume the conditions in Proposition 3.1 hold and let
(cid:98) Y =
h ( g ( X )) be the classiﬁer, then ∆ DP ( (cid:98) Y ) ≤ ∆ BR ( D 0 , D 1 ) =
∆ DP ( Y ) .
Remark Theorem 3.1 shows that aligning the conditional distributions of representations between groups will not add more bias in terms of the demographic parity gap.
In particular, the DP gap of any classiﬁer that satisﬁes equalized odds will be at most the DP gap of the perfect classiﬁer.
This is particularly interesting as it is well-known in the literature (Barocas et al., 2017) that demographic parity is not compatible with equalized odds except in degenerate cases.
Despite this result, Theorem 3. says that we can still achieve equalized odds and simultaneously preserve the DP gap.
In Section 3.1 we show that aligning the conditional distributions of representations between groups helps reduce the lower bound of the joint error, but nevertheless that is only a necessary condition.
In the next theorem we show that together with a small Type-I and Type-II error in inferring the target variable Y , these two properties are also sufﬁcient to ensure a small joint error across different demographic subgroups.
Theorem 3.2.
Assume the conditions in Proposition 3.1 hold and let (
cid:98) Y = h ( g ( X )) be the classiﬁer, then Err D 0 ( (cid:98) Y )
+ Err D 1 ( (cid:98) Y ) ≤ 2BER D ( (cid:98) Y || Y ) .
The above bound means that in order to achieve small joint error across groups, it sufﬁces for us to minimize the BER if a classiﬁer satisﬁes equalized odds.
Note that by deﬁnition, the BER in the bound equals to the sum of Type-I and Type-II classiﬁcation errors using (cid:98) Y as a classiﬁer.
Theorem 3. gives an upper bound of the joint error across groups and it also serves as a motivation for us to design the optimization formulation (2) that simultaneously minimizes the BER and aligns the conditional distributions.
3. C ONDITIONAL A LIGNMENT AND B ALANCED E RROR R ATES L EAD TO S MALL E RROR
In this section we will see that a small BER and equalized odds together not only serve as a guarantee of a small joint error, but they also lead to a small error gap between different demographic subgroups.
Recall that we deﬁne γ a :=
D a ( Y = 0) , ∀ a ∈ { 0 , 1 } .
Before we proceed, we ﬁrst formally deﬁne the accuracy gap and equalized odds gap of a classiﬁer (cid:98) Y :
Deﬁnition 3.2 (Error Gap) .
Given a joint distribution D , the error gap of a classiﬁer (cid:98) Y is ∆ Err ( (cid:98) Y ) :
=
|D 0 ( (cid:98) Y (cid:54)
= Y ) − D 1 ( (cid:98) Y (cid:54) =
Y ) | .
Deﬁnition 3.3 (Equalized Odds Gap) .
Given a joint distribution D , the equalized odds gap of a classiﬁer (cid:98) Y is ∆ EO ( (cid:98) Y ) := max y ∈{ 0 , 1 } |D y
( (cid:98) Y = 1) − D
y 1 ( (cid:98) Y = 1) | .
By deﬁnition the error gap could also be understood as the accuracy parity gap between different subgroups.
The following theorem characterizes the relationship between error gap, equalized odds gap and the difference of base rates across subgroups:
Theorem 3.3.
For any classiﬁer (cid:98) Y , ∆ Err ( (cid:98) Y ) ≤ ∆ BR ( D 0 , D 1 ) · BER D ( (cid:98) Y || Y ) + 2∆ EO ( (cid:98) Y ) .
As a direct corollary of Theorem 3.3, if the classiﬁer (cid:98) Y satisﬁes equalized odds, then ∆ EO ( (cid:98) Y ) = 0 .
In this case since ∆ BR ( D 0 , D 1 ) is a constant, minimizing the balanced error rate BER D ( (cid:98) Y || Y ) also leads to minimizing the error gap.
Furthermore, if we combine Theorem 3.2 and Theorem 3. together, we can guarantee that each of the errors cannot be too large:
Corollary 3.1.
For any joint distribution D and classiﬁer (cid:98) Y , if (cid:98) Y satisﬁes equalized odds, then max
[Equation]
Remark It is a well-known fact that out of the three fairness criteria, i.e., demographic parity, equalized odds, and predictive rate parity, any two of them cannot hold simultaneously (Barocas et al., 2017) except in degenerate cases where the sensitive attribute A is independent of the target variable
Y .
As an implication of Theorem 3.3, we can see that the accuracy parity is not incompatible with equalized odds: for a classiﬁer (cid:98) Y that satisﬁes equalized odds, it sufﬁces to minimize the sum of
Type-I and Type-II error BER D ( (cid:98) Y || Y ) in order to achieve accuracy parity.
It is also worth pointing out that Theorem 3.3 provides only an upper bound, but not necessarily the tightest one.
In particular, the error gap could still be 0 while BER D ( (cid:98) Y || Y ) is greater than 0.
To see this, we have (cid:40)
Err
[Equation]
[Equation] [Equation]
Y
[Equation] [Equation]
0 ( Y
[Equation] [Equation]
[Equation]
0 ( (cid:98)
[Equation]
=
| Y = 1)
Err D 1 ( (cid:98)
[Equation] [Equation]
D 1 ( Y = 0) · D 1 ( (cid:98) Y = 1 | Y = 0)
+ D 1 ( Y = 1) ·
D 1 ( (cid:98) Y =
| Y = 1) .
Now if the predictor (cid:98) Y satisﬁes equalized odds, then
[Equation]
D
[Equation]
Y =
| Y = 1) .
Hence the error gap ∆ Err ( (cid:98) Y ) admits the following identity: ∆ Err ( (cid:98) Y ) =
(cid:12)(cid:12)(cid:12) D ( (cid:98) Y
[Equation]
1 | Y = 0) (cid:0) D
( Y
[Equation] [Equation]
D
[Equation]
( Y = 0) (cid:1) + D ( (cid:98) Y =
| Y = 1) (cid:0) D
( Y = 1) − D
( Y = 1) (cid:1)(cid:12)(cid:12)(cid:12) =
∆ BR ( D 0 , D 1 ) · (cid:12)(cid:12) (cid:12) D ( (cid:98) Y = 1 | Y = 0) − D ( (cid:98) Y =
| Y = 1) (cid:12)(cid:12) (cid:12) =
∆ BR ( D 0 ,
[Table]
In other words, if the predictor (cid:98) Y satisﬁes equalized odds, then in order to have equalized accuracy, (cid:98) Y only needs to have equalized FPR and FNR globally when the base rates differ across groups.
This is a much weaker condition to ask for than the one asking BER D ( (cid:98) Y || Y ) = 0 .
3. P RACTICAL I MPLEMENTATION
We cannot directly optimize the proposed optimization formulation (2) since the binary 0 / 1 loss is NPhard to optimize, or even approximately optimize over a wide range of hypothesis classes (Ben-David et al., 2003).
However, observe that for any classiﬁer (cid:98) Y and y ∈ { 0 , 1 } , the log-loss (cross-entropy loss) CE D y ( (cid:98) Y || Y ) is a convex relaxation of the binary loss:
[Equation]
Hence in practice we can relax the optimization problem (2) to a cost-sensitive cross-entropy loss minimization problem, where the weight for each class is given by the inverse marginal probability of the corresponding class.
This allows us to equivalently optimize the objective function without explicitly computing the conditional distributions.
E MPIRICAL S TUDIES
In light of our theoretic ﬁndings, in this section we verify the effectiveness of the proposed algorithm in simultaneously ensuring equalized odds and accuracy parity using real-world datasets.
We also analyze the impact of imposing such parity constraints on the utility of the target classiﬁer, as well as its relationship to the intrinsic structure of the binary classiﬁcation problem, e.g., the difference of base rates across groups, the global imbalance of the target variable, etc.
We analyze how this imbalance affects the utility-fairness trade-off.
As we shall see shortly, we will empirically demonstrate that, in many cases, especially the ones where the dataset is imbalanced in terms of the target variable, this will inevitably compromise the target utility.
While for balanced datasets, this trend is less obvious: the proposed algorithm achieves a better fairness-utility trade-off when compared with existing fair representation learning methods and we can hope to achieve fairness without sacriﬁcing too much on utility.
[Table]
4. E XPERIMENTAL S ETUP
To this end, we perform experiments on two popular real-world datasets in the literature of algorithmic fairness, including an income-prediction dataset, known as the Adult dataset, from the UCI Machine
Learning Repository (Dua & Graff, 2017), and the Propublica COMPAS dataset (Dieterich et al., 2016).
The basic statistics of these two datasets are listed in Table 1.
Adult Each instance in the Adult dataset describes an adult, e.g., gender, education level, age, etc, from the 1994 US Census.
In this dataset we use gender as the sensitive attribute, and the processed data contains 114 attributes.
The target variable (income) is also binary:
1 if ≥ 50K/year otherwise 0.
For the sensitive attribute A , A = 0 means male otherwise female.
From Table 1 we can see that the base rates are quite different ( 0 . 310 vs. 0 . 113 ) across groups in the Adult dataset.
The dataset is also imbalanced in the sense that only around 24 .
6% of the instances have target label 1 .
Furthermore, the group ratio is also imbalanced: roughly 67 .
3% of the data are male.
COMPAS The goal of the COMPAS dataset is binary classiﬁcation on whether a criminal defendant will recidivate within two years or not.
Each instance contains 12 attributes, including age, race, gender, number of prior crimes, etc.
For this dataset, we use the race (white A = 0 vs. black
A = 1 ) as our sensitive attribute and target variable is 1 iff recidivism.
The base rates are different across groups, but the COMPAS
To validate the effect of ensuring equalized odds and accuracy parity, for each dataset, we perform controlled experiments by ﬁxing the baseline network architecture so that it is shared among all the fair representation learning methods.
We term the proposed method CF AIR (for conditional fair representations) that minimizes conditional errors both the target variable loss function and adversary loss function.
To demonstrate the importance of using BER in the loss function of target variable, we compare with a variant of CF AIR that only uses BER in the loss of adversaries, denoted as CF AIR -EO.
To see the relative effect of using cross-entropy loss vs L 1 loss, we also show the results of L AFTR (Madras et al., 2018), a state-of-the-art method for learning fair representations.
Note that L AFTR is closely related to CF AIR -EO but slightly different: L AFTR uses global crossentropy loss for target variable, but conditional L 1 loss for the adversary.
Also, there is only one adversary in L AFTR , while there are two adversaries, one for D 0 and one for D 1 , in both CF AIR and
CF AIR -EO.
Lastly, we also present baseline results of F AIR (Edwards & Storkey, 2015), which aims for demographic parity representations, and N O D EBIAS , the baseline network without any fairness constraint.
For all the fair representation learning methods, we use the gradient reversal layer (Ganin et al., 2016) to implement the gradient descent ascent (GDA) algorithm to optimize the minimax problem.
All the experimental details, including network architectures, learning rates, batch sizes, etc. are provided in the appendix.
4. R ESULTS AND A NALYSIS
In Figure 1 and Figure 2 we show the error gap ∆ Err , equalized odds gap ∆ EO , demographic parity gap ∆ DP and the joint error across groups Err
+ Err 1 of the aforementioned fair representation learning algorithms on both the Adult and the COMPAS datasets.
For each algorithm and dataset, we also gradually increase the value of the trade-off parameter λ and compute the corresponding metrics.
Adult Due to the imbalance of A in the Adult dataset, in the ﬁrst plot of Figure 1 we can see that all the algorithms except CF AIR have a large error gap of around 0.12.
As a comparison, observe that the error gap of CF AIR when λ = 1 e 3 almost reduces to 0, conﬁrming the effectiveness of our algorithm in ensuring accuracy parity.
From the second plot, we can verify that all the three methods, including CF AIR , CF AIR -EO and L AFTR successfully ensure a small equalized odds gap, and they also decrease demographic parity gaps (the third plot).
F AIR is the most effective one in mitigating ∆ DP since its objective function directly optimizes for that goal.
Note that from the second plot we can also conﬁrm that CF AIR -EO is more effective than L AFTR in reducing ∆ EO .
The reason is two-fold.
First, CF AIR -EO uses two distinct adversaries and hence it effectively competes with stronger adversaries than L AFTR .
Second, CF AIR -EO uses the cross-entropy loss instead of the Lloss for the adversary, and it is well-known that the maximum-likelihood estimator (equivalent to using the cross-entropy loss) is asymptotically efﬁcient and optimal.
On the other hand, since the
Adult dataset is imbalanced (in terms of Y ), using BER in the loss function of the target variable can thus to a large extent hurt the utility, and this is also conﬁrmed from the last plot, where we show the joint error.
COMPAS The ﬁrst three plots of Figure 2 once again verify that CF AIR successfully leads to reduced error gap, equalized odds gap and also demographic parity gap.
These experimental results are consistent with our theoretical ﬁndings where we show that if the representations satisfy equalized odds, then its ∆ DP cannot exceed that of the optimal classiﬁer, as shown by the horizontal dashed line in the third plot.
In the fourth plot of Figure 2, we can see that as we increase λ , all the fair representation learning algorithms sacriﬁce utility.
However, in contrast to Figure 1, here the proposed algorithm CF AIR has the smallest trade-off: this shows that CF AIR is particularly suited in the cases when the dataset is balanced and we would like to simultaneously ensure accuracy parity and equalized odds.
As a comparison, while CF AIR -EO is still effective, it is slightly worse than
CF AIR in terms of both ensuring parity and achieving small joint error.
R ELATED W ORK
Algorithmic Fairness In the literature of algorithmic fairness, two key notions of fairness have been extensively proposed and explored, i.e., group fairness , including various variants deﬁned in Section 2, and individual fairness , which means that similar individuals should be treated similarly.
Due to the complexity in deﬁning a distance metric to measure the similarity between individuals (Dwork et al., 2012), most recent research focuses on designing efﬁcient algorithms to achieve group fairness (Zemel et al., 2013; Zafar et al., 2015;
Hardt et al., 2016; Zafar et al., 2017; Madras et al., 2018; Creager et al., 2019; Madras et al., 2019).
In particular, Hardt et al.
(2016) proposed a post-processing technique to achieve equalized odds by taking as input the model’s prediction and the sensitive attribute.
Concurrently, Kleinberg et al.
(2016) also offered a calibration technique to achieve predictive rate parity.
However, both of these two approaches require access to the sensitive attribute during the inference phase, which is often not available in many real-world scenarios.
Another line of work uses causal inference to deﬁne notions of causal fairness and to formulate procedures for achieving these notions (Zhang et al., 2018;
Wang et al., 2019; Madras et al., 2019; Kilbertus et al., 2017; Kusner et al., 2017; Nabi & Shpitser, 2018).
These approaches require making untestable assumptions.
Of particular note is the observation in Coston et al.
(2019) that fairness-adjustment procedures based on Y in settings with treatment effects may lead to adverse outcomes.
To apply our method in such settings, we would need to match conditional counterfactual distributions, which could be a direction of future research.
Theoretical Results on Fairness Theoretical work studying the relationship between different kinds of fairness notions are abundant.
Motivated by the controversy of the potential discriminatory bias in recidivism prediction instruments, Chouldechova (2017) showed an intrinsic incompatibility between equalized odds and predictive rate parity.
In the seminal work of Kleinberg et al.
, the authors demonstrated that when the base rates differ between different groups, then a non-perfect classiﬁer cannot simultaneously be statistically calibrated and satisfy equalized odds.
In the context of cost-sensitive learning, Menon & Williamson (2018) show that if the optimal decision function is dissimilar to a fair decision, then the fairness constraint will not signiﬁcantly harm the target utility.
The idea of reducing fair classiﬁcation to cost-sensitive learning is not new.
Agarwal et al.
(2018) explored the connection between fair classiﬁcation and a sequence of cost-sensitive learning problems where each stage corresponds to solving a linear minimax saddle point problem.
In a recent work (Zhao & Gordon, 2019), the authors proved a lower bound on the joint error across different groups when a classiﬁer satisﬁes demographic parity.
They also showed that when the decision functions are close between groups, demographic parity also implies accuracy parity.
The theoretical results in this work establish a relationship between accuracy parity and equalized odds: these two fairness notions are fundamentally related by the base rate gap and the balanced error rate.
Furthermore, we also show that for any predictor that satisﬁes equalized odds, the balanced error rate also serves as an upper bound on the joint error across demographic subgroups.
Fair Representations Through the lens of representation learning, recent advances in building fair algorithmic decision making systems focus on using adversarial methods to learn fair representations that also preserve sufﬁcient information for the prediction task (Edwards & Storkey, 2015; Beutel et al., 2017;
Zhang et al., 2018; Madras et al., 2018; Adel et al., 2019).
In a nutshell, the key idea is to frame the problem of learning fair representations as a two-player game, where the data owner is competing against an adversary.
The goal of the adversary is to infer the group attribute as much as possible while the goal of the data owner is to remove information related to the group attribute and simultaneously to preserve utility-related information for accurate prediction.
Apart from using adversarial classiﬁers to enforce group fairness, other distance metrics have also been used to learn fair representations, e.g., the maximum mean discrepancy (Louizos et al., 2015), and the Wassersteindistance (Jiang et al., 2019).
In contrast to these methods, in this paper we advocate for optimizing
BER on both the target loss and adversary loss in order to simultaneously achieve accuracy parity and equalized odds.
We also show that this leads to better utility-fairness trade-off for balanced datasets.
C ONCLUSION AND F UTURE W ORK
In this paper we propose a novel representation learning algorithm that aims to simultaneously ensure accuracy parity and equalized odds.
The main idea underlying the design of our algorithm is to align the conditional distributions of representations (rather than marginal distributions) and use balanced error rate (i.e., the conditional error) on both the target variable and the sensitive attribute.
Theoretically, we prove how these two concepts together help to ensure accuracy parity and equalized odds without impacting demographic parity, and we also show how these two can be used to give a guarantee on the joint error across different demographic subgroups.
Empirically, we demonstrate on two real-world experiments that the proposed algorithm effectively leads to the desired notions of fairness, and it also leads to better utility-fairness trade-off on balanced datasets.
Calibration and Utility
Our work takes a step towards better understanding the relationships between different notions of fairness and their corresponding trade-off with utility.
In some scenarios, e.g., the COMPAS tool, it is desired to have a decision making system that is also well calibrated.
While it is well-known that statistical calibration is not compatible with demographic parity or equalized odds, from a theoretical standpoint it is still not clear whether calibration will harm utility and if so, what is the fundamental limit of a calibrated tool on utility.
Fairness and Privacy Future work could also investigate how to make use of the close relationship between privacy and group fairness.
At a colloquial level, fairness constraints require a predictor to be (to some extent) agnostic about the group membership attribute.
The membership query attack in privacy asks the same question – is it possible to guarantee that even an optimal adversary cannot steal personal information through inference attacks.
Prior work (Dwork et al., 2012) has described the connection between the notion of individual fairness and differential privacy.
Hence it would be interesting to exploit techniques developed in the literature of privacy to develop more efﬁcient fairness-aware learning algorithms.
On the other hand, results obtained in the algorithmic fairness literature could also potentially lead to better privacy-preserving machine learning algorithms (Zhao et al., 2019).
A CKNOWLEDGMENTS
HZ and GG would like to acknowledge support from the DARPA XAI project, contract #FA87501720152 and an Nvidia GPU grant.
HZ would also like to thank Richard Zemel, Toniann Pitassi, David Madras and Elliot Creager for helpful discussions during HZ’s visit to the Vector
Institute.
A P ROOFS A. P ROOF OF P ROPOSITION 3.
Proposition 3.1.
For g : X → Z , if d TV ( g (cid:93) D y 0 , g (cid:93) D y 1 ) = 0 , ∀ y ∈ { 0 , 1 } , then for any classiﬁer h :
Z → { 0 , 1 } , h ◦ g satisﬁes equalized odds.
Proof.
To prove this proposition, we ﬁrst show that for any pair of distributions D , D (cid:48) over Z and any hypothesis h : Z → { 0 , 1 } ,
[Equation]
TV (
[Equation]
(cid:93)
[Equation]
, h (cid:93)
[Equation]
(cid:48) )
≤ d TV ( D , D (cid:48) ) .
Note that since h is a hypothesis, there are only two events in the induced probability space, i.e., h ( · ) = 0 or h ( · ) = 1 .
Hence by deﬁnition of the induced (pushforward) distribution, we have:
[Equation]
Apply the above inequality twice for y ∈ { 0 , 1 } :
[Equation]
meaning
[Equation]
which further implies that h ( g ( X )) is independent of A given Y
=
y since h ( g ( X )) is binary.
(cid:4)
A. P ROOF OF P ROPOSITION
Proposition 3.2.
For g : X → Z , if d TV ( g (cid:93) D y 0 , g (cid:93) D y 1 ) = 0 , ∀ y ∈ { 0 , 1 } , then for any classiﬁer h :
Z → { 0 , 1 } , d TV (( h ◦ g ) (cid:93) D 0 , ( h ◦ g ) (cid:93) D 1 ) ≤ ∆ BR ( D 0 , D 1 ) .
Proof.
Let (cid:98) Y
=
( h ◦ g )
( X ) and note that (cid:98) Y is binary, we have d
[Equation]
h
[Equation]
Now, by Proposition 3.1, if d TV ( g (cid:93) D y 0 , g (cid:93) D y 1 )
= 0 , ∀ y ∈ { 0 , 1 } , it follows that d TV (( h ◦ g ) (cid:93) D y 0 , ( h ◦ g ) (cid:93)
D y 1 )
= 0 , ∀ y ∈ { 0 , 1 } as well.
Applying Lemma 3.1, we know that ∀ y ∈ { 0 , 1 } ,
[Equation]
Hence, d TV (
[Equation]
0 ( (cid:98) Y = 1) − D 1 ( (cid:98) Y = 1) | (cid:17) ≤ |D
( Y = 0) − D
( Y = 0) | (cid:16)(cid:0) D
( (cid:98) Y = 0)
+ D 1 ( (cid:98) Y = 0) (cid:1) + (cid:0) D
( (cid:98) Y = 1)
+ D 1 ( (cid:98) Y = 1) (cid:1)(cid:17) =
|D
( Y = 0)
− D
( Y = 0) | (cid:16)(cid:0) D
( (cid:98) Y = 0)
+ D 0 ( (cid:98) Y = 1) (cid:1) + (cid:0) D 1 ( (cid:98) Y = 0)
+ D 1 ( (cid:98) Y = 1) (cid:1)(cid:17) =
|D
( Y = 0)
− D
( Y = 0)
| ·= |D 0 ( Y = 0) − D 1 ( Y = 0) | =
∆ BR ( D 0 , D 1 ) .
(cid:4)
A. P ROOF OF L EMMA 3.
Recall that we deﬁne γ a :=
D a ( Y = 0) , ∀ a ∈ { 0 , 1 } .
Lemma 3.1.
Assume the conditions in Proposition 3.1 hold and let
(cid:98) Y =
h ( g ( X )) be the classiﬁer, then |D
( (cid:98) Y = y ) − D 1 ( (cid:98) Y = y ) | ≤ | γ 0 − γ 1 | · (cid:0) D
( (cid:98) Y =
y )
+ D 1 ( (cid:98) Y =
y ) (cid:1) , ∀ y ∈ { 0 , 1 } .
Proof.
To bound |D 0 ( (cid:98) Y = y ) − D 1 (
[Equation]
Y =
y ) | , for y ∈ { 0 , 1 } , by the law of total probability, we have:
[Equation]
where the above inequality is due to the triangular inequality.
Now apply Proposition 3.1, we know that (cid:98) Y satisﬁes equalized odds, so we have D
( (cid:98) Y = y )
=
D
( (cid:98) Y =
y )
=
D 0 ( (cid:98) Y =
y ) and
[Equation]
=
[Equation]
=
[Equation] [Equation]
Y =
y )
=
D 1 ( (cid:98) Y = y ) , leading to:
[Equation]
which completes the proof.
[Equation]
A. P ROOF OF T HEOREM 3.
Theorem 3.1.
Assume the conditions in Proposition 3.1 hold and let
(cid:98) Y =
h ( g ( X )) be the classiﬁer, then ∆ DP ( (cid:98) Y ) ≤ ∆ BR ( D 0 , D 1 ) =
∆ DP ( Y ) .
Proof.
To bound ∆ DP ( (cid:98) Y ) , realize that |D
( (cid:98) Y = 0) − D 1 ( (cid:98) Y = 0)
| =
|D 0 ( (cid:98) Y = 1) − D 1 ( (cid:98) Y = 1) | , so we can rewrite the DP gap as follows:
[Equation]
Now apply Lemma 3.1 twice for y =
and y
= 1 , we have:
[Equation]
(cid:1) .
Taking sum of the above two inequalities yields
[Equation]
Combining all the inequalities above
[Equation]
we know that
[Equation]
completing the proof.
[Equation]
A. P ROOF OF T HEOREM 3.
Theorem 3.2.
Assume the conditions in Proposition 3.1 hold and let (
cid:98) Y = h ( g ( X )) be the classiﬁer, then Err D 0 ( (cid:98) Y )
+ Err D 1 ( (cid:98) Y ) ≤ 2BER D ( (cid:98) Y || Y ) .
Proof.
First, by the law of total probability, we have:
Err
[Equation] [Equation]
Err
[Equation]
( (cid:98)
[Equation] [Equation]
(cid:54)
[Equation] [Equation]
[Equation]
(cid:54) = (cid:98) Y ) =
[Equation]
10 ( (cid:98)
[Equation]
= 0) D
( Y = 1) + D 00 ( (cid:98) Y = 1) D
( Y = 0)
+ D 11 ( (cid:98) Y = 0) D
( Y = 1) + D 01 ( (cid:98) Y = 1) D
( Y = 0)
Again, by Proposition 3.1, the classiﬁer (cid:98) Y =
( h ◦ g )
( X ) satisﬁes equalized odds, so we have
D 10 ( (cid:98) Y = 0)
=
D 1 ( (cid:98) Y = 0) , D 00 ( (cid:98) Y = 1) =
D 0 ( (cid:98) Y = 1) , D 11 ( (cid:98) Y = 0)
=
D 1 ( (cid:98) Y = 0) and
D 01 ( (cid:98) Y = 1) =
D 0 ( (cid:98) Y = 1) :
= D 1 ( (cid:98) Y = 0) D 
 ( Y = 1) + D 0 ( (cid:98) Y = 1) D
( Y = 0)
+ D 1 ( (cid:98) Y = 0) D 
 ( Y = 1) + D 0 ( (cid:98) Y = 1) D
( Y = 0)
=
D 1 ( (cid:98) Y = 0) · (cid:0) D
( Y = 1) + D
( Y = 1) (cid:1) + D 0 ( (cid:98) Y = 1) · (cid:0) D
( Y = 0)
+ D 1 ( Y = 0) (cid:1) ≤ 2 D 1 ( (cid:98) Y = 0)
+ 2 D 0 ( (cid:98) Y = 1)
= 2BER D ( (cid:98) Y || Y ) , which completes the proof.
(cid:4)
A. P ROOF OF T HEOREM 3.
Theorem 3.3.
For any classiﬁer (cid:98) Y , ∆ Err ( (cid:98) Y ) ≤ ∆ BR ( D 0 , D 1 ) · BER D ( (cid:98) Y || Y ) + 2∆ EO ( (cid:98) Y ) .
Before we give the proof of Theorem 3.3, we ﬁrst prove the following two lemmas that will be used in the following proof.
Lemma A.1.
Deﬁne γ a :=
D a ( Y = 0) , ∀ a ∈ { 0 , 1 } , then | γ 0 D 00 ( (cid:98) Y = 1) − γ 1 D 01 ( (cid:98) Y = 1) | ≤ | γ 0 − γ 1 | · D 0 ( (cid:98) Y = 1) + γ 0 D
( A = 1)∆ EO ( (cid:98) Y ) + γ 1 D
( A = 0)∆ EO ( (cid:98) Y ) .
Proof.
In order to prove the upper bound in the lemma, it sufﬁces if we could give the desired upper bound for the following term
[Equation]
following which we will have:
[Equation]
and an application of the Bayes formula could ﬁnish the proof.
To do so
[Equation]
let us ﬁrst simplify
D 00 ( (cid:98) Y = 1) − D
( (cid:98) Y = 1) .
Applying the Bayes’s formula, we know that:
[Equation]
Similarly, for the second term D 01 ( (cid:98) Y = 1) − D
( (cid:98) Y = 1) , we can show that:
[Equation]
Plug these two identities into above, we can continue the analysis with
[Equation]
The ﬁrst inequality holds by triangular inequality and the second one holds by the deﬁnition of equalized odds gap.
[Equation]
Lemma A.2.
Deﬁne γ a :=
D a ( Y = 0) , ∀ a ∈ { 0 , 1 } , then | (1 − γ 0 ) D 10 ( (cid:98) Y = 0) − (1 − γ 1 ) D 11 ( (cid:98) Y = 0) | ≤ | γ 0 − γ 1 | · D 1 ( (cid:98) Y = 0)
+ (1 − γ 0 ) D 1 ( A = 1)∆ EO ( (cid:98) Y )
+ (1 − γ 1 ) D 1 ( A = 0)∆
EO
( (cid:98) Y ) .
Proof.
The proof of this lemma is symmetric to the previous one, so we omit it here.
(cid:4)
Now we are ready to prove Theorem 3.3:
Proof of Theorem 3.3.
First, by the law of total probability, it is easy to verify that following identity holds for a ∈ { 0 , 1 } :
[Equation]
Using this identity, to bound the error gap, we have:
[Equation]
Invoke Lemma A.1 and Lemma A.2 to bound the above two terms:
[Equation]
Realize that both γ 0 , γ 1 ∈ [0 , 1]
[Equation]
we have:
[Equation]
which completes the proof.
[Equation]
We also provide the proof of Corollary 3.1:
Corollary 3.1.
For any joint distribution D and classiﬁer (cid:98) Y , if (cid:98) Y satisﬁes equalized odds, then max { Err D 0 ( (cid:98) Y ) , Err D 1 ( (cid:98) Y ) } ≤ ∆ BR ( D 0 , D 1 ) · BER D ( (cid:98) Y || Y ) / 2 + BER D ( (cid:98) Y || Y ) .
Proof.
We ﬁrst invoke Theorem 3.3, if (cid:98) Y satisﬁes equalized odds, then ∆ EO ( (cid:98) Y ) = 0 , which implies:
[Equation]
On the other hand, by Theorem 3.2, we know that
[Equation]
Combine the above two inequalities and recall that max { a, b } = ( | a + b | + | a − b | ) / 2 , ∀ a, b ∈ R , yielding:
[Equation]
completing the proof.
[Equation]
B E XPERIMENTAL D ETAILS B. T HE A DULT E XPERIMENT
For the baseline network N O D EBIAS , we implement a three-layer neural network with ReLU as the hidden activation function and logistic regression as the target output function.
The input layer contains 114 units, and the hidden layer contains 60 hidden units.
The output layer only contain one unit, whose output is interpreted as the probability of D ( (cid:98) Y = 1 | X = x ) .
For the adversary in F AIR and L AFTR , again, we use a three-layer feed-forward network.
Speciﬁcally, the input layer of the adversary is the hidden representations of the baseline network that containsunits.
The hidden layer of the adversary network contains 50 units, with ReLU activation.
Finally, the output of the adversary also contains one unit, representing the adversary’s inference probability
D ( (cid:98)
A = 1 | Z = z ) .
The network structure of the adversaries in both CF AIR and CF AIR -EO are exactly the same as the one used in F AIR and L AFTR , except that there are two adversaries, one for
D 0 ( (cid:98)
A = 1 | Z = z ) and one for D 1 ( (cid:98)
A = 1 | Z = z ) .
The hyperparameters used in the experiment are listed in Table 2.
[Table]
T HE COMPAS E XPERIMENT
Again, for the baseline network N O D EBIAS , we implement a three-layer neural network with ReLU as the hidden activation function and logistic regression as the target output function.
The input layer contains 11 units, and the hidden layer contains 10 hidden units.
The output layer only contain one unit, whose output is interpreted as the probability of D ( (cid:98) Y = 1 | X = x ) .
For the adversary in F AIR and L AFTR , again, we use a three-layer feed-forward network.
Speciﬁcally, the input layer of the adversary is the hidden representations of the baseline network that containsunits.
The hidden layer of the adversary network contains 10 units, with ReLU activation.
Finally, the output of the adversary also contains one unit, representing the adversary’s inference probability
D ( (cid:98)
A = 1 | Z = z ) .
The network structure of the adversaries in both CF AIR and CF AIR -EO are exactly the same as the one used in F AIR and L AFTR , except that there are two adversaries, one for
D 0 ( (cid:98)
A = 1 | Z = z ) and one for D 1 ( (cid:98)
A = 1 | Z = z ) .
The hyperparameters used in the experiment are listed in Table 3.
[Table]