FERMI: F AIR E MPIRICAL R ISK M INIMIZATION V IA E XPONENTIAL R ´ ENYI M UTUAL I NFORMATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Several notions of fairness, such as demographic parity and equal opportunity, are deﬁned based on statistical independence between a predicted target and a sensitive attribute.
In machine learning applications, however, the data distribution is unknown to the learner and statistical independence is not veriﬁable.
Hence, the learner could only resort to empirical evaluation of the degree of fairness violation.
Many fairness violation notions are deﬁned as a divergence/distance between the joint distribution of the target and sensitive attributes and the Kronecker product of their marginals, such as R ´ enyi correlation, mutual information, L ∞ distance, to name a few.
In this paper, we propose another notion of fairness violation, called Exponential R´enyi Mutual Information (ERMI) between sensitive attributes and the predicted target.
We show that ERMI is a strong fairness violation notion in the sense that it provides an upper bound guarantee on all of the aforementioned notions of fairness violation.
We also propose the Fair Empirical Risk Minimization via ERMI regularization framework, called FERMI.
Whereas existing in-processing fairness algorithms are deterministic, we provide a stochastic optimization method for solving FERMI that is amenable to large-scale problems. In addition, we provide a batch (deterministic) method to solve FERMI.
Both of our proposed algorithms come with theoretical convergence guarantees. Our experiments show that FERMI achieves the most favorable tradeoffs between fairness violation and accuracy on test data across different problem setups, even when fairness violation is measured in notions other than ERMI.
I NTRODUCTION
Ensuring that decisions made using machine learning algorithms are fair to different subgroups is of utmost importance.
Without any mitigation strategy, machine learning algorithms may result in discrimination against certain subgroups based on sensitive attributes, such as gender or race, even if such discrimination is absent in the training data (Datta et al., 2015; Sweeney, 2013; Bolukbasi et al., 2016; Angwin et al., 2016; du Pin Calmon et al., 2017b;
Feldman et al., 2015; Hardt et al., 2016;
Fish et al., 2016;
Woodworth et al., 2017; Zafar et al., 2017; Bechavod & Ligett, 2017; Kearns et al., 2018).
To remedy such discrimination issues, several notions for imposing algorithmic fairness have been proposed in the literature.
A learning machine satisﬁes the demographic parity notion, if the predicted target is independent of the sensitive attributes (Dwork et al., 2012).
Promoting demographic parity can lead to poor performance, especially if the true outcome is not independent of the sensitive attributes.
To remedy this, (Hardt et al., 2016) proposed equalized odds to ensure that the predicted target is conditionally independent of the sensitive attributes given the true label.
A further relaxed version of this notion is equal opportunity which is satisﬁed if predicted target is conditionally independent of sensitive attributes given that the true label is in an advantaged class (Hardt et al., 2016).
Note that the inherent assumption in such conditional notions is that the true labels are unbiased.
These notions suffer from a potential ampliﬁcation of the inherent biases that may exist in the targets/labels in the training data (e.g., data collection bias).
Tackling such bias is beyond the scope of this work.
In practice, the learner cannot empirically verify independence of random variables, and hence cannot verify demographic parity, equalized odds, or equal opportunity.
This has led the machine learning community to deﬁne several notions of fairness violation that quantify the degree of independence between random variables, e.g., demographic parity/equalized odds, L ∞ distance (Dwork et al., 2012; Hardt et al., 2016), mutual information (Kamishima et al., 2011; Rezaei et al., 2020;
Steinberg et al., 2020; Zhang et al., 2018; Cho et al., 2020), Pearson correlation (Zafar et al., 2017), false positive/negative rates (Bechavod & Ligett, 2017), Hilbert Schmidt independence criterion (HSIC) (P´erez-Suay et al., 2017), and R´enyi correlation (Baharlouei et al., 2020;
Grari et al., 2020; 2019), to name a few.
In this paper, we deﬁne yet another notion of fairness violation, called exponential R´enyi mutual information (ERMI).
We show that ERMI is easy to compute empirically and prove that ERMI provides an upper bound on the existing notions of fairness violation such as demographic parity, equalized odds, and equal opportunity.
Given a notion of fairness violation, it is still not straightforward to train an algorithm that satisﬁes a fairness violation constraint (Cotter et al., 2019).
The fairness-promoting machine learning algorithms can be categorized in three main classes: pre-processing , post-processing , and in-processing methods.
Pre-processing algorithms (Feldman et al., 2015; Zemel et al., 2013; du Pin Calmon et al., 2017b) transform the biased data features to a new space in which the labels and sensitive attributes are statistically independent.
This transform is oblivious to the training procedure.
Post-processing approaches (Hardt et al., 2016; Pleiss et al., 2017) mitigate the discrimination of the classiﬁer by altering the the ﬁnal decision, e.g., by changing the thresholds on soft labels, or reassigning the labels to impose notions of fairness.
In-processing approaches focus on the training procedure and impose the notions of fairness as constraints or regularization terms in the optimization procedure.
Several regularization-based methods are proposed in the literature to impose measures of fairness to decision-trees (Kamiran et al., 2010;
Raff et al., 2018;
Aghaei et al., 2019), support vector machines (Donini et al., 2018), neural networks (Grari et al., 2020), or (logistic) regression models (Zafar et al., 2017;
Berk et al., 2017; Taskesen et al., 2020; Chzhen & Schreuder, 2020;
Baharlouei et al., 2020; Jiang et al., 2020;
Grari et al., 2019).
To the best of our knowledge, existing in-processing methods are all deterministic, making them impractical for large-scale problems.
Furthermore, most in-processing methods (with the exception of (Baharlouei et al., 2020)) are designed for problems in which the sensitive attribute and/or the target is binary.
In this paper, we introduce a new fair empirical risk minimization framework via ERMI regularization, and call it FERMI.
We provide novel batch and stochastic gradient-based methods with guarantees for solving FERMI and demonstrate their effectiveness on multiple numerical experiments, which include a large-scale problem and problem with both non-binary sensitive attributes and targets.
We show that FERMI can be used to achieve the most favorable tradeoffs between performance and fairness, even if fairness violation is measured in notions other than ERMI.
( Z, Z ) - FAIRNESS : A GENERAL NOTION OF FAIRNESS
We consider a learner who trains a model to predict a target, (cid:98) Y , e.g., whether or not to extend a loan, supported on Y which can be discrete or continuous.
The prediction is made using a set of features,
X , e.g., ﬁnancial history features, length of credit, and amount of debt.
We also assume that there is a set of discrete sensitive attributes, S, e.g., race and sex, supported on S , associated with each sample.
Further, let A ⊆
Y denote an advantaged outcome class, e.g., the outcome where a loan is extended.
Next, we will present the main fairness notion considered in this paper, which generalizes several existing ones.
Deﬁnition 1 ( ( Z, Z ) -fairness) .
Given a random variable Z , let Z be a subset of values that Z can take.
We say that a learning machine satisﬁes ( Z, Z ) -fairness if for every z ∈ Z , (cid:98) Y is conditionally independent of S given Z = z .
More precisely,
[Equation]
Notice that ( Z, Z ) -fairness recovers several important existing notions of fairness as special cases:
( Z, Z ) -fairness recovers demographic parity
(Dwork et al., 2012) if Z = 0 and Z = { 0 } .
In this case, conditioning on Z has no effect, and hence (0 , { 0 } ) fairness is equivalent to the independence between (cid:98) Y and S ,
i
e., demographic parity (see Deﬁnition 7, Appendix A).
( Z, Z ) -fairness recovers equalized odds (Hardt et al., 2016)
if Z = Y and Z = Y .
In the case, Z ∈ Z is trivially satisﬁed and could be dropped. Hence, conditioning on Z is equivalent to conditioning on Y, which recovers the equalized odds notion of fairness, i.e., conditional independence of (cid:98) Y and S given Y (see Deﬁnition 8, Appendix A).
( Z, Z ) -fairness recovers equal opportunity (Hardt et al., 2016) if Z = Y and Z = A .
This is also similar to the previous case with Y replaced with A (see Deﬁnition 9, Appendix A).
Demographic parity amounts to requiring equality of outcomes across sensitive groups.
However, it can result in poor performance of a learned model, particularly if the true outcome Y is not independent of S. Equalized odds and equal opportunity remedy this issue by relaxing the independece constraint (Hardt et al., 2016).
In the binary classiﬁcation setting with binary sensitive attributes (e.g., male or female), equalized odds ensures equality of false negatives and false positives of a classiﬁer across sensitive groups.
Equal opportunity is a further relaxation of the equalized odds criterion, which, in the binary setting, requires just equality of false negatives across sensitive groups.
For example, this could be used to enforce that a face recognition software does not falsely classify people of one race as criminals more often than people of other races.
Note that verifying ( Z, Z ) -fairness requires having access to the joint distribution of random variables ( Z, (cid:98) Y , S ) .
This joint distribution is unavailable to the learner in the context of machine learning, and hence the learner would resort to empirical estimation of the amount of violation of independence.
In the next section, we propose exponential R´enyi mutual information as a notion of the violation of ( Z, Z ) -fairness, and show that it is a stronger notion compared to several existing fairness violation notions, including demographic parity L ∞ distance (Kearns et al., 2018), equalized odds L ∞ distance (Hardt et al., 2016), and equal opportunity L ∞ distance (Hardt et al., 2016).
E XPONENTIAL R ´ ENYI MUTUAL INFORMATION
In this section, we deﬁne ERMI and show that several existing fairness violation notions (which are mostly instances of f -divergences or distance metrics between the joint probability distribution and the Kronecker product of the marginals) are also upper bounded by ERMI, implying that ERMI is a stronger notion of fairness violation.
In particular, this means that if ERMI is small, then we automatically obtain guarantees that all other notions of fairness violation must be small as well.
We present all deﬁnitions and results for general ( Z, Z ) fairness notion, which requires careful extension of several existing notions to the conditional case.
These deﬁnitions and results will simplify signiﬁcantly when Z = 0 and Z = { 0 } , which will eliminate all conditional expectations.
Deﬁnition 2 (ERMI – exponential R ´ enyi mutual information) .
We deﬁne the exponential R´enyi mutual information between (cid:98) Y and S given Z ∈ Z as
[Equation]
In Appendix B, we unravel the deﬁnition for the special cases of interest corresponding to the existing notions of fairness. We also discuss that ERMI is the χ 2 -divergence (which is an f -divergence) between the joint distribution, p (cid:98) Y ,S | Z , and the Kronecker product of marginals, p (cid:98) Y | Z ⊗ p S | Z .
In particular, ERMI is non-negative, and zero if and only if ( Z, Z ) -fairness is satisﬁed.
Hence, ERMI is a valid notion of fairness violation.
Deﬁnition 3 (R ´ enyi mutual information (R ´ enyi, 1961)) .
Let the R´enyi mutual information of order α > 1 between random variables (cid:98) Y and S given Z ∈ Z be deﬁned as: [Equation] Z    
 [Equation]
3) which generalizes Shannon mutual information
[Equation]
and recovers it as lim α → 1 + I α ( (cid:98) Y ; S | Z ∈ Z )
=
I 1 ( (cid:98) Y ; S | Z ∈ Z ) . Note that I α ( (cid:98) Y ; S | Z ∈ Z ) ≥ 0 with equality if and only if ( Z, Z ) -fairness is satisﬁed.
Theorem 1 (ERMI is stronger than Shannon mutual information) .
We have
[Equation] [Equation]
All proofs are relegated to the appendix.
Theorem 1 establishes that ERMI is a stronger notion of fairness in the sense that driving it to zero would also bound the Shannon mutual information.
It also shows that ERMI is exponentially related to the R´enyi mutual information of order 2 .
Deﬁnition 4 (R´enyi correlation (Hirschfeld, 1935; Gebelein, 1941; R´enyi, 1959)) .
Let F and G be the set of measurable functions such that for random variables (cid:98) Y and S ,
E (cid:98) Y { f ( (cid:98) Y ; z ) } =
E S { g ( S ; z ) } = 0 , E (cid:98) Y { f ( (cid:98) Y ; z ) 2 } = E S (cid:8) g ( S ; z ) 2 (cid:9) = 1 , for all z ∈ Z .
R´enyi correlation is:
[Equation] [Equation]
R´enyi correlation generalizes Pearson correlation coefﬁcient
[Equation]
to capture nonlinear dependencies between the random variables by ﬁnding functions of random variables that maximize the Pearson correlation coefﬁcient between the random variables.
In fact, it is true that ρ R ( (cid:98) Y , S | Z ∈ Z ) ≥ 0 with equality if and only if ( Z, Z ) -fairness is satisﬁed.
Due to these favorable properties, R´enyi correlation has gained popularity as a measure of fairness violation (Baharlouei et al., 2020; Grari et al., 2020).
Theorem 2 (ERMI is stronger than R ´ enyi correlation.) .
We have
[Equation]
S [Equation] and if |S| = 2 , D R ( (cid:98) Y ; S | Z ∈ Z )
=
ρ R ( (cid:98) Y , S | Z ∈ Z ) .
Next, we turn to another popular notion of fairness violation and establish similar relationships.
Deﬁnition 5 ( L q fairness violation) .
We deﬁne the L q fairness violation for q ≥ 1 by:
[Equation]
Note that L q ( (cid:98) Y , S | Z ∈ Z )
=
if and only if ( Z, Z ) -fairness is satisﬁed.
In particular, L ∞ fairness violation recovers the demographic parity violation (Kearns et al., 2018, Deﬁnition 2.1) if we let
Z = { 0 } and Z = 0 .
It also recovers equal opportunity violation (Hardt et al., 2016) if we let
Z = A and Z = Y .
L q fairness violation generalizes this notion by considering the L q norm of the difference between the joint distribution p (cid:98) Y ,S and the Kronecker product of the marginal distributions p (cid:98) Y ⊗ p S .
Theorem 3 (ERMI is stronger than L ∞ fairness violation) .
Let (cid:98) Y be a discrete or continuous random variable, and S be a discrete random variable supported on a ﬁnite set.
Then for any q ≥ 1 ,
[Equation]
The above theorem says that if a method controls ERMI value for imposing fairness, L ∞ demographic parity violation (Kearns et al., 2018), L ∞ equal opportunity violation (Hardt et al., 2016), or L ∞ equalized odds violation (Hardt et al., 2016) is also guaranteed to be bounded.
F AIR R ISK M INIMIZATION V IA ERMI
Our goal is to train a model that balances fairness and accuracy objectives.
To this end, we introduce fair risk minimization through exponential R ´ enyi mutual information framework deﬁned below:
Deﬁnition 6 (FRMI – fair risk minimization through exponential R´enyi mutual information) .
To balance fairness and accuracy, we consider the learning objective function:
[Equation]
where (cid:96) denotes the loss function, such as L 2 loss or cross entropy loss; λ > 0 is a scalar balancing the accuracy versus fairness objectives; D R (cid:0)(cid:98) Y θ ( X ); S (cid:1) is the notion of ERMI given in Eq.
(17) ; and (cid:98) Y θ ( X ) is the output of the learned model (e.g., the output of a classiﬁcation or a regression task, or the cluster number in a clustering task).
While (cid:98) Y θ ( X ) inherently depends on X and θ , in the rest of this paper, we sometimes leave the dependence of (cid:98) Y on X and/or θ implicit for brevity of notation.
Notice that we have also left the dependence of the loss on the predicted outcome (cid:98) Y implicit.
FRMI is the objective we should solve if demographic parity is the desired fairness notion; if instead we are interested in equalized odds or equal opportunity, then D R ( (cid:98) Y , S ) should be replaced by D R ( (cid:98) Y , S | Z ∈ Z ) for an appropriate ( Z, Z ) pair, per the discussion in Section 3.
Since the theory is very similar in both cases, we stick with FRMI as deﬁned in Deﬁnition 6.
In practice, the true joint distribution of ( X , S, Y, (cid:98) Y ) is unknown and we only have N samples at our disposal, making it impossible to solve FRMI.
Hence, we turn into fair empirical risk minimization via exponential R´enyi mutual information (FERMI) approach.
While it is natural to estimate E X ,Y,S (cid:8) (cid:96) (cid:0) X , Y ; θ (cid:1)(cid:9) through the empirical risk, the estimation of D R ( (cid:98) Y , S ) in the objective function in Eq.
(11) is not as straightforward.
In what follows, we propose two approaches for estimating D R ( (cid:98) Y , S ) .
These two approaches result in two different algorithms for balancing fairness and accuracy, where we discuss the beneﬁts and shortcomings of each.
4. FERMI VIA EMPIRICAL ESTIMATION OF THE PROBABILITY DISTRIBUTIONS
Let { x i , s i , y i , (cid:98)
y i } i ∈
[ N ] denote the features, sensitive attributes, targets, and the predictions of the model parameterized by θ for samples i ∈
[ N ] .
A natural approach to estimate the objective function in Eq.
(11) and learning the parameter θ is through solving the problem
[Equation]
Here (cid:98) D R
[Equation]
Y
[Equation]
:=
[Equation]
s
[Equation]
d (cid:98) y − 1 is an empirical estimate of D R ( (cid:98) Y θ , S ) where (cid:98) p S ( s ) , (cid:98) p (cid:98) Y ( (cid:98) y ) , and (cid:98) p (cid:98) Y ,
S
( (cid:98) y, s ) are the empirical estimation of the corresponding probability functions.
To make the objective function differentiable with respect to θ , we make the following assumption:
Assumption 1.
Assume the sensitive attribute is a deterministic function of the features, i.e., S =
f s ( X ) .
This trivially holds if the sensitive attribute is available as part of the features. Further, assume the following soft conditional density:
[Equation]
where τ > 0 controls the softness of the decision and τ → ∞ would recover the hard decision made by choosing y that minimizes the loss function.
Assumption 1 is a generalization of the assumption in RFI (Baharlouei et al., 2020), and is natural in problem instances where the decisions made by the learning algorithm are soft decisions. In particular, logistic regression or neural networks with soft-max layer and cross entropy loss satisfy
Assumption 1.
We can further ﬁnd p (cid:98) Y | S , p (cid:98) Y , and p S from Eq.
Under this assumption, we will have the following empirical estimate of the joint distribution of the predicted target and the sensitive attribute:
[Equation]
which could be marginalized to deﬁne (cid:98) p S ( s ) and (cid:98) p (cid:98) Y ( · ) as well.
These empirical probabilities make (cid:98) D R ( (cid:98) Y θ ; S ) a differentiable function of θ .
Notice that these empirical quantities converge to the true distributions for ﬁnite S and Y , however, the sample complexity required for their estimation scales linearly with |S| and |Y| , which implies an exponential scaling with the number of sensitive attributes.
Our stochastic algorithm that will be presented in the next section will aim to remedy this potential problem.
To solve Eq.
(12), one can apply the gradient descent algorithm and use the dynamics
[Equation]
where η > 0 is the learning rate/step-size.
Note that when the sensitive attribute is binary, our algorithm is the same as the one in (Baharlouei et al., 2020) since then
(cid:98) D R (cid:0)(cid:98) Y θ t ; S (cid:1)
=
ρ R ( (cid:98) Y , S ) , the R´enyi correlation, by Theorem 8.
However, in the non-binary case, our algorithm is different from (Baharlouei et al., 2020) in general, and we show in Sec.
5 that it achieves a more favorable fairness-accuracy tradeoff curve. Under standard assumptions on the loss function and the learning rate, one can show that the dynamics in Eq.
(15) ﬁnd an (cid:15) − stationary point, (i.e., a point with the norm of gradient being smaller than (cid:15) ) in O ( 1 (cid:15) 2 ) iterations (Nesterov, 2013).
Theorem 4.
(Informal statement) Gradient descent (i.e. Eq.
(15) ) converges to the set of (cid:15) -ﬁrst order stationary points of the FERMI objective (Eq.
(12) ) in O ( 1 (cid:15) 2 ) iterations (gradient evaluations).
While this algorithm achieves the optimal rate of ﬁrst-order methods for general smooth non-convex optimization problems, the empirical ERMI term in the objective in Eq.
(12) is a biased estimator of ERMI in Eq.
This bias makes the optimization problem in Eq.
(12) not suitable for using stochastic methods.
For example, in the extreme case, where only one sample is available to the learner for updating the objective at each turn, (cid:98) D R ( · ; · ) can be severely biased due to the nonlinearities in how it is deﬁned.
In the next subsection, we propose another approach for estimation of
D R ( · ; · ) that results in an unbiased estimator, which is amenable to stochastic optimization. [Table] − η [Table]
(cid:104) [Table] (cid:80) [Table]
∇
[Table]
(cid:96) ( x i ,
[Table]
i , θ
[Table]
− 2 λ ∇
[Table]
vec( (cid:98) y
i ( θ ) (cid:98) y
i ( θ ) T ) T vec( W T W )
+
[Table]
[Table]
[Table]
[Table]
− 1 /
[Table]
) (cid:105)
[Table]
[Table]
(cid:16) W
[Table]
+ η
[Table]
(cid:80)
[Table]
∈
[Table]
(cid:104) − 2 λW (cid:98)
y i ( θ ) (cid:98)
y
i ( θ ) T + 2 λP − 1 / 2 s s i (cid:98) y i ( θ ) (cid:105)(cid:17) random from { 1 , . . , T } 8: Return: θ ˆ t .
4. S TOCHASTIC FERMI
In order to solve the population level objective in Eq.
(11) using stochastic methods (such as stochastic gradient descent), one needs to obtain an unbiased estimate of the objective in equation 11, i.e., E X ,Y,S (cid:8) (cid:96) (cid:0) X , Y ; θ (cid:1)(cid:9) + λD R (cid:0)(cid:98) Y θ
( X ); S (cid:1) .
Clearly, the empirical average 1 | B | (cid:80) i ∈ B (cid:96) (cid:0) x i , y i ; θ (cid:1) is an unbiased estimator of E X ,Y,S (cid:8) (cid:96) (cid:0) X , Y ; θ (cid:1)(cid:9) , where B ⊆ [ N ] is a batch of data points. Thus, to develop a stochastic algorithm, we need to have an unbiased estimator of D R (cid:0)(cid:98) Y θ ( X ); S (cid:1) given a batch of data points B . The following Theorem will help us obtain such an estimator.
Theorem 5.
For discrete random variables (cid:98) Y and S where (cid:98) Y ∈ [ m ] , S ∈ [ k ] , we have
[Equation]
[Equation] where P (cid:98)
[Equation]
Let (cid:98) Y ∈ {
[Equation]
1 } m and (cid:98) S ∈ { 0 , 1 } k be the one hot encoded version of (cid:98) Y and (cid:98) S , respectively.
Then, the above theorem implies that Eq.
(11) can be re-written as [Equation] Hence
[Equation]
given a batch of data points B , we can obtain an unbiased estimator of the above objective function by the empirical average
[Equation]
This observation leads to the stochastic algorithm presented in Algorithm 1.
Notice that this algorithm is based on the assumption that P s is known.
This assumption is practical since the distribution of sensitive attributes (such as male vs female) is known in many applications (or it can be estimated accurately using the training data).
The convergence rate of Algorithm 1 is analyzed in Theorem 6.
Theorem 6.
(Informal statement) Algorithm 1 converges to the set of (cid:15) -ﬁrst order stationary points of the FERMI objective (c.f. Eq.
12) in O ( 1 (cid:15) 4 ) iterations (stochastic gradient evaluations).
The formal statement of this theorem can be found in Theorem 11 in Appendix D. Notice that while this algorithm has a slower rate of convergence than the batch algorithm, it is stochastic (each iteration is computationally cheap) and amenable to large-scale problems.
Note also that a faster convergence rate of O ( 1 (cid:15) 3 ) could be obtained by using the (more complicated) SREDA method of (Luo et al., 2020) instead of SGDA to solve FERMI objective.
We omit the details here.
In the next section, we numerically evaluate the performance of the algorithms described in this section.
N UMERICAL E XPERIMENTS 5. B INARY FAIR CLASSIFICATION WITH A BINARY SENSITIVE ATTRIBUTE
We start the experimental setup for binary classiﬁcation problem with a binary sensitive attribute.
This is a common setup among most existing baseline methods.
Per Theorem 2, in this binary classiﬁcation case ERMI is equivalent to R´enyi correlation (Baharlouei et al., 2020), and per the discussion in Sec. 4.1, our batch algorithm is exactly the same as RFI (Baharlouei et al., 2020).
While we solve FERMI to impose an ERMI regularizer, we still measure fairness violation via popular fairness violation notions, such as conditional demographic parity L ∞ violation (Deﬁnition 10), conditional equal opportunity L ∞ violation (Deﬁnition 11), and conditional equalized odds violation.
In Fig.
1, we report the fairness violation vs error for German Credit and Adult datasets.
As can be seen, for all three popular notions of fairness, FERMI achieves the best trade-off between fairness and error probability on the test data.
This could be partly due to smoothness of FERMI optimization problem, and partly due to the fact that ERMI upper bounds other fairness notions as discussed in Sec. 3. We will have a more detailed discussion on this in Section 6.
5. N ON - BINARY FAIR CLASSIFICATION WITH A NON - BINARY SENSITIVE ATTRIBUTE
Next, we consider a general classiﬁcation problem with |S| > 2 .
In this case, we consider the
Communities and Crime dataset, which has 18 binary sensitive attributes in total, and we pick { 7 , 10 , 14 , 18 } sensitive attributes out of those for different experiments, which corresponds to |S| ∈ { 2 7 , 2 10 , 2 14 , 2 18 } .
We discretize the target into three classes { high , medium , low } (ternary classiﬁcation).
The only baseline that we are aware of that can handle non-binary classiﬁcation with non-binary is sensitive attributes is RFI (Baharlouei et al., 2020).
The results are presented in Fig.
2, where we use conditional demographic parity L ∞ violation (Deﬁnition 10) and conditional equal opportunity L ∞ violation (Deﬁnition 11) as the fairness violation notion.
As can be seen, FERMI achieves a better tradeoff curve as compared with RFI.
It is noteworthy that the per-iteration complexity of FERMI is far less than that of RFI, which requires solving a singular value decomposition at each iteration.
Finally, the convergence rate for FERMI given in Theorem 4 guarantees an O ( 1 (cid:15) 2 ) convergence vs O ( 1 (cid:15) 4 ) for RFI (Baharlouei et al., 2020, Theorem 4.1).
Empirically, we observed that FERMI converges ∼ 10x faster on this problem instance.
Finally, we also show that conditional demographic parity L ∞ violation (Deﬁnition 10) and square root of ERMI are approximately linearly related, which further justiﬁes the use of ERMI regularizer in the FERMI framework.
5. L ARGE - SCALE CLASSIFICATION WITH FERMI
In this experiment, we consider the color MNIST dataset (Li &
Vasconcelos, 2019) where MNIST digits are colored with different colors drawn from a Gaussian distribution with variance σ around a certain average color.
It is shown in (Li & Vasconcelos, 2019) that as σ → 0 , a convolutional network model overﬁts signiﬁcantly to the color on this dataset, and hence will not be able to generalize on a regular black and white test set.
Our goal in this experiment is to show that FERMI can promote independence between the predicted target and color (which we use as the sensitive attribute within FERMI) to improve generalization in this setup.
This also allows us to examine the scaling of stochastic FERMI when used in convolutional neural networks. We consider σ = 0 , where the test performance is the lowest due to overﬁtting.
The result of the experiment is presented in Fig.
As expected, FERMI results in learning representations that have less dependence on the color, hence leading to better generalization.
It is noteworthy that the test error achieved by FERMI when σ = 0 is 22.6%, as compared to 23.3% obtained using REPAIR (Li & Vasconcelos, 2019) for σ = 0 .
Further decreasing σ ≤ 0 .
the test error using REPAIR sharply goes above 50%.
We were unable to run REPAIR for σ = 0 .
D ISCUSSION & C ONCLUDING R EMARKS
In this paper we proposed a new notion of fairness, called exponential R ´ enyi mutual information (ERMI).
We showed that ERMI is a strong notion of fairness violation providing guarantees on several other popular notions, namely Pearson correlation, R´enyi correlation, Shannon mutual information, R´enyi mutual information and L q distance violation.
We proposed a Fair Empirical Risk
Minimization framework with an ERMI regularizer to balance performance and fairness, and called it FERMI.
Additionally, we showed that FERMI could be efﬁciently solved for non-binary sensitive attributes and non-binary target variables.
We proposed batch and stochastic algorithms for solving FERMI with convergence guarantees for smooth losses.
In particular, Algorithm 1 is unique among existing fair algorithms as it is stochastic, making it much more practical for large-scale problems (as demonstrated in Sec. 5.3).
This is made possible by Theorem 5, which leads to an unbiased estimator of the gradient of ERMI.
It is not at all clear if replacing ERMI by another regularizer, such as R ´ enyi correlation or Shannon mutual information, in Eq.
(12) would be amenable to stochastic optimization.
From an experimental perspective, we showed that FERMI leads to better fairness-accuracy tradeoffs than the existing baselines. There are several possible explanations for the superior empirical performance of FERMI compared to existing methods.
One possible reason is that the objective function Eq.
(12) is easier to optimize than the objectives of competing in-processing methods: ERMI is smooth; and in the discrete case, is equal to the trace of a matrix (see Theorem 8), which is easy to compute.
Contrast this with the larger computational overhead of R´enyi correlation, for example, which requires ﬁnding the second singular value of a matrix.
Furthermore, the sample complexity of estimating R´enyi mutual information of order 2 (and consequently that of ERMI) scales as Θ( (cid:112) |S| ) as compared to Shannon mutual information which scales as Θ( |S| / log |S| ) (Acharya et al., 2014).
Another possible explanation is that ERMI is a stronger notion of fairness than all of the most widely used fairness notions, as shown in Sec. 3, which might lead to better generalization. Together, these facts suggest that ERMI serves as an efﬁcient and easily optimizable proxy for these other notions, leading to better practical performance regardless of which fairness measure is used. We leave it as future work to rigorously understand which of these (or other) factors are responsible for the favorable performance tradeoffs observed from FERMI. Finally, on the Color MNIST experiment with neural network function approximation, we observed that stochastic FERMI outperforms batch
FERMI.
In this case, we suspect that the randomness in stochastic FERMI supposedly contributes to its convergence to a local minimum with superior generalization performance compared to batch
FERMI (see (Kleinberg et al., 2018) and the references therein).
A E XISTING NOTIONS OF FAIRNESS
Let ( Y, (cid:98) Y , A , S ) denote the true target, predicted target, the advantaged outcome class, and the sensitive attribute, respectively.
We review three major notions of fairness.
Deﬁnition 7 (demographic parity (Dwork et al., 2012)) .
We say that a learning machine satisﬁes demographic parity if (cid:98) Y is independent of S.
Deﬁnition 8 (equalized odds (Hardt et al., 2016)) .
We say that a learning machine satisﬁes equalized odds, if (cid:98) Y is conditionally independent of S given Y .
Deﬁnition 9 (equal opportunity (Hardt et al., 2016)) .
We say that a learning machine satisﬁes equal opportunity with respect to A , if (cid:98) Y is conditionally independent of S given Y =
y for all y ∈ A .
Notice that the equal opportunity as deﬁned here generalizes the deﬁnition in (Hardt et al., 2016).
It recovers equalized odds if A = Y , and it recovers equal opportunity of (Hardt et al., 2016) for
A = { 1 } in binary classiﬁcation.
B P ROPERTIES AND SPECIAL CASES OF ERMI
Notice that ERMI is in fact the χ 2 -divergence between the conditional joint distribution, p (cid:98) Y ,S , and the Kronecker product of conditional marginals, p (cid:98) Y ⊗ p S , where the conditioning is on Z ∈ Z .
Further, χ 2 -divergence is an f -divergence with f ( t ) =
( t − 1) 2 .
See (Csisz ´ ar & Shields, 2004, Section 4) for a discussion.
As an immediate result of this observation and well-known properties of f -divergences, we can state the following property of ERMI:
Remark 7.
D R ( (cid:98) Y ; S | Z ∈ Z ) ≥ 0 with equality if and only if for all z ∈ Z , (cid:98) Y and S are conditionally independent given Z = z .
To further clarify the deﬁnition of ERMI, especially as it relates to demographic parity, equalized odds, and equal opportunity, we will unravel the deﬁnition explicitly in a few special cases. First, let Z = and Z = { 0 } .
In this case, Z ∈ Z trivially holds, and conditioning on Z has no effect, resulting in:
[Equation]
D R ( (cid:98) Y ; S ) is the notion of ERMI that should be used when the desired notion of fairness is demographic parity.
In particular, D R ( (cid:98) Y ; S ) = 0 implies that χ 2 divergence between p (cid:98) Y ,S , and the
Kronecker product of marginals, p (cid:98) Y ⊗ p S is zero.
This in turn implies that (cid:98) Y and S are independent, which is the deﬁnition of demographic parity.
We note that when (cid:98) Y and S are discrete, this special case ( Z = 0 and Z = { 0 } ) of ERMI is referred to as χ 2 -information in (du Pin Calmon et al., 2017a).
Next, we consider Z = Y and Z
= Y .
In this case
[Equation]
Z ∈ Z is trivially satisﬁed
[Equation]
and hence,
[Equation]
D R ( (cid:98) Y ; S | Y ) should be used when the desired notion of fairness is equalized odds.
In particular,
D R ( (cid:98) Y ; S | Y ) = 0 directly implies the conditional independence of (cid:98) Y and S given Y.
Finally
[Equation]
we consider Z = Y and Z
= A .
In this case, we have
D
[Equation]
where
[Equation]
[Equation]
This notion is what should be used when the desired notion of fairness is equal opportunity.
This can be further simpliﬁed when the advantaged class is a singleton (which is the case in binary classiﬁcation).
If Z = Y and Z = { y } , then
[Equation]
[Equation]
Finally
[Equation]
we note that we use the notation D R ( (cid:98) Y ; S | Y ) and D R ( (cid:98) Y ; S
| Y =
y ) to be consistent with the deﬁnition of conditional mutual information in (Cover & Thomas, 1991)
[Equation]
C R ELATIONS BETWEEN ERMI AND OTHER FAIRNESS VIOLATION NOTIONS
Proof of Theorem 1.
We proceed to prove all the (in)equalities one by one:
• 0 ≤ I S ( (cid:98) Y ; S | Z ∈ Z ) .
This is well known and the proof can be found in any information theory textbook (Cover & Thomas, 1991).
• I 1 ( (cid:98) Y ; S | Z ∈ Z ) ≤
I 2 ( (cid:98) Y ; S | Z ∈ Z ) .
This is a known property of R´enyi mutual information, but we provide a proof for completeness in Lemma 1.
• I 2 ( (cid:98) Y ; S | Z ∈ Z )
≤ e I 2 ( (cid:98) Y ; S | Z ∈Z ) − 1 .
This follows from the fact that x ≤ e x − 1 .
• e I 2 ( (cid:98) Y ; S ) | Z ∈Z − 1 =
D R ( (cid:98) Y ; S | Z ∈ Z ) .
This follows from simple algebraic manipulation.
Lemma 1.
Let (cid:98) Y , S, Z be discrete or continuous random variables.
Then:
(a) For any α, β ∈ [1 , ∞ ] , I β ( (cid:98) Y ; S | Z ∈ Z ) ≥
I α ( (cid:98) Y ; S | Z ∈ Z )
if β > α .
(b) lim α → 1 + I α ( (cid:98) Y ; S | Z ∈ Z )
= I 1 ( (cid:98) Y ; S ) : = E Z (cid:110) D KL ( p (cid:98) Y , S | Z || p (cid:98) Y |
Z ⊗ p S | Z ) (cid:12)(cid:12)(cid:12) Z ∈ Z (cid:111) , where I 1 ( · ; · ) denotes the Shannon mutual information and D KL is Kullback–Leibler divergence (relative entropy).
(c) For all α ∈ [1 , ∞ ] , I α ( (cid:98) Y ; S | Z ∈ Z ) ≥ 0 with equality if and only if for all z ∈ Z , (cid:98) Y and S are conditionally independent given z .
Proof.
(a) First assume
< α < β < ∞ and that α, β (cid:54) = 1 .
Deﬁne a = α − 1 , and b = β − 1 .
Then the function φ ( t ) =
t b/a is convex for all t ≥ 0 , so by Jensen’s inequality we have:
[Equation]
[Equation]
[Equation]
Now suppose α = 1 .
Then by the monotonicity for α (cid:54) = 1 proved above, we have
I 1 ( (cid:98) Y ; S ) =
lim α → 1 − I α ( (cid:98) Y ; S ) =
sup α ∈ (0 , 1)
I α
( (cid:98) Y ; S ) ≤ inf α> 1 I α ( (cid:98) Y ; S ) .
Also, I ∞ ( (cid:98) Y ; S ) =
lim α →∞ I α
( (cid:98) Y ; S )
=
sup α>
I α ( (cid:98) Y ; S ) .
(b
This is
a standard property of the cumulant generating function (see (
Dembo & Zeitouni, 2009)).
c ) It is straightforward to observe that independence implies that R ´ enyi mutual information van- ishes
On the other hand, if R ´ enyi mutual information vanishes, then part (a) implies that
Shannon mutual
information also vanishes, which implies the desired conditional independence
Proof of Theorem 2.
The proof is completed using the following pieces.
• 0 ≤ | ρ ( (cid:98) Y , S | Z ∈ Z ) | ≤ ρ R ( (cid:98) Y , S | Z ∈ Z ) .
This is obvious from the deﬁnition of ρ R ( (cid:98) Y , S | Z ∈ Z ) .
• ρ R ( (cid:98) Y , S | Z ∈ Z ) ≤ D R ( (cid:98) Y ; S | Z ∈ Z ) .
This follows from Theorem 8.
• Notice that if |S| = 2 , Theorem 8 implies that D R ( (cid:98) Y ; S | Z ∈ Z )
=
ρ R ( (cid:98) Y , S | Z ∈ Z ) .
Theorem 8.
Suppose that S =
[ k ] .
Let the k × k matrix P be deﬁned as P = { P ij } i,j ∈ [ k ] ×
[ k ] , where
[Equation]
Let 1 = σ 1 ≥ σ 2 ≥ . .
≥ σ k ≥ 0 be the eigenvalues of P .
Then,
[Equation]
Proof.
Eq.
(24) is proved in (Witsenhausen, 1975, Section 3).
To prove Eq.
(25), notice that
[Equation]
which completes the proof
[Equation]
Proof of Theorem 3.
It sufﬁces to prove the inequality for L 1 , as L q is bounded above by L 1 for all q ≥ 1 .
The proof for the case where Z = 0 and Z = { 0 } follows from the following set of inequalities:
[Equation]
where Eq.
(28) follows from Cauchy-Schwarz inequality, and Eq.
(30) follows from Lemma 2.
The extension to general Z and Z is immediate by observing that ρ ( (cid:98) Y , S | Z ∈ Z ) =
E Z (cid:104) ρ ( (cid:98) Y , S | Z ) (cid:12)(cid:12)(cid:12) Z ∈ Z (cid:105) , ρ R ( (cid:98) Y , S | Z ∈ Z )
=
E Z (cid:104) ρ R ( (cid:98) Y , S | Z ) (cid:12)(cid:12)(cid:12) Z ∈ Z (cid:105) , and D R ( (cid:98) Y , S | Z ∈
Z )
=
E Z (cid:104) D R ( [Equation]
Lemma 2.
We have
[Equation]
Proof.
The proof follows from the following set of identities:
[Equation]
Next [Equation] we present some alternative fairness deﬁnitions and show that they are also upper bounded by
ERMI
[Equation] Deﬁnition 10 (conditional demographic parity L ∞ violation) .
Given a predictor (cid:98) Y supported on
Y and a discrete sensitive attribute S supported on a ﬁnite set S , we deﬁne the conditional demographic parity violation by:
[Equation]
First, we show that (cid:101) dp ( (cid:98) Y | S ) is a reasonable notion of fairness violation.
Lemma 3.
(cid:101) dp ( (cid:98) Y | S ) =
iff (if and only if) (cid:98) Y and S are independent.
Proof.
By deﬁnition, (cid:101) dp ( (cid:98) Y | S )
=
iff for all (cid:98) y ∈ Y , s ∈ S , p (cid:98) Y ,S
( (cid:98) y | s )
=
p (cid:98) Y ( (cid:98) y )
iff (cid:98) Y and S are independent (since we always assume p ( s ) > 0 for all s ∈ S ).
Theorem 9 (ERMI is stronger than conditional demographic parity L ∞ violation) .
Let (cid:98) Y be a discrete or continuous random variable supported on Y , and S be a discrete random variable supported on a ﬁnite set S .
Denote p min S
[Equation]
Then,
[Equation]
Proof.
The proof follows from the following set of (in)equalities:
[Equation]
where Eq.
(40) follows from Theorem 3.
Deﬁnition 11 (conditional equal opportunity L ∞ violation (Hardt et al., 2016)) .
Let Y, (cid:98) Y take values in Y and let A ⊆
Y be a compact subset denoting the advantaged outcomes (For example, the decision “to interview” an individual or classify an individual as a “low risk” for ﬁnancial purposes).
We deﬁne the conditional equal opportunity L ∞ violation of (cid:98) Y with respect to the sensitive attribute S and the advantaged outcome A by
[Equation]
Theorem 10 (ERMI is stronger than generalized equal opportunity Kolmogorov violation - alternative deﬁnition) .
Let (cid:98) Y , Y, be discrete or continuous random variables supported on Y , and let S be a discrete random variable supported on a ﬁnite set S .
Let A ⊆
Y be a compact subset of Y .
Denote p min S |A
=
min s ∈S ,y ∈A p S | Y ( s | y ) .
Then,
[Equation]
Proof.
Notice that the same proof for Theorem 9 would give that for all y ∈ A : [Equation] s, y ) [Equation]
p
[Equation]
≤ p min S |
[Equation]
(cid:113) D R
(cid:98) Y ; S
| Y = y ) .
Hence,
[Equation]
where the last inequality follows from Jensen’s inequality.
This completes the proof.
D S TOCHASTIC FERMI
Proof of Theorem 5.
Let W ∗ ∈ arg max W ∈ R k × m − Tr
( WP (cid:98) y W T )
+ 2 Tr( WP (cid:98) y,s P − 1 / 2 s ) . We will compute W ∗ and plug it in the RHS of Eq. (16) to show the equality in Eq.
Setting the derivative of the expression on the RHS equal to zero leads to:
[Equation]
Plugging this expression for W ∗ [Equation] we have
[Equation]
Writing out the matrix multiplication explicitly in the last expression, we have
[Equation]
where U i,j = (cid:98) p S ( i ) − 1 (cid:98) p (cid:98) Y ,S ( j, i ) and V i,j = ˆ p (cid:98) Y ( j ) − 1 ˆ p (cid:98) Y ,S ( j, i ) , for i ∈
[ k ] , j ∈ [ m ] .
Hence
[Equation]
which completes the proof.
Next, we move to the statement and proof of the precise version of Theorem 6.
We ﬁrst recall some basic deﬁnitions:
Deﬁnition 12.
A function f is β -smooth if for all u , u (cid:48) , we have (cid:107)∇ f ( u ) − ∇ f ( u ) (cid:107) ≤ β (cid:107) u − u (cid:48) (cid:107) .
Deﬁnition 13.
A point θ is an (cid:15) -stationary point of a differentiable function Φ if (cid:107)∇ Φ( θ ) (cid:107) ≤ (cid:15) .
Assumption 2.
• (cid:96) is twice differentiable, L (cid:96) -Lipscthiz, and β (cid:96) -smooth in θ .
• (cid:107)∇ θ P ˆ y (cid:107) 2 :=
(cid:107)∇ θ vec( P (cid:98) y ) (cid:107) 2 ≤ L y and max l ∈ [ m ] (cid:107)∇ θ (( P (cid:98) y ) l,l ) (cid:107) 2 ≤ (cid:101)
L y • max l ∈ [ m ] (cid:107)∇ 2 θθ ( P (cid:98) y ) l,l (cid:107) 2 ≤ β y .
• (cid:107)∇ θ P T (cid:98) y,s (cid:107) 2 :=
(cid:107)∇ θ vec( P T (cid:98) y,s ) (cid:107) 2 ≤ L ys and max l ∈ [ m ] ,j ∈ [ k ] (cid:107)∇ θ (( P (cid:98) y,s ) l,m ) (cid:107) 2 ≤ (cid:101)
L ys • max l ∈ [ m ] ,j ∈ [ k ] (cid:107)∇ 2 θθ ( P (cid:98) y,s ) l,j (cid:107) 2 ≤ β y,s .
Theorem 11 (Precise version of Theorem 6) .
Denote f
( θ , W )
=
1 N (cid:88) i ∈ [ N ] (cid:96)
[Equation]
y
[Equation] θ (cid:1) [Equation] (cid:98)
[Equation]
) −
[Equation]
(cid:17) . Set W := B F (0 , 2 D ) ⊂ R k × m (Frobenius norm ball of radius 2 D ), D := √ mk ˆ p min (cid:98) y √
ˆ p min s .
Denote ∆ Φ :=
Φ( θ 0 ) − min θ Φ( θ ) , where Φ( θ ) :=
max W ∈W f ( θ , W ) .
In Algorithm 1, choose the stepsizes as η θ = Θ(1 /κ 2 β ) and η W = Θ(1 /β ) and mini-batch size as M = Θ (cid:0) max (cid:8) 1 , κσ 2 (cid:15) − 2 (cid:9)(cid:1) .
Then under Assumption 2, the iteration complexity of Algorithm 1 to return an (cid:15) -stationary point of f is bounded by
O
[Equation]
which gives the total stochastic gradient complexity of
[Equation]
where
[Equation]
The theorem follows from (Lin et al., 2020, Theorem 4.5) combined with the following technical lemmas.
We assume Assumption 2 holds for the remainder of the proof of Theorem 11:
Lemma 4.
Let
[Equation]
Then
f is β -smooth, where β =
[Equation]
[Equation]
f ( θ , · ) is 2 λ ˆ p min (cid:98)
y -strongly concave for all θ .
(cid:107) W ∗ (cid:107) F ≤ D, where D is as deﬁned in Theorem 11 and W ∗ denotes any maximizer of f ( θ , W ) .
Proof.
By Assumption 2, g is twice continuously differentiable.
Hence for part 1, it sufﬁces to upper bound the spectral norm of the second derivative of g ( · , · , z ) by β for all z = ( x , y ) , where we vectorize and then differentiate with respect to w :=
vec W and/or θ , so that the resulting ﬁrst and second derivatives are always vectors or a matrices (not tensors). Notice that g ( θ , w, z ) = (cid:96)
[Equation] z , θ [Equation] − λ and
[Equation]
Further
[Equation]
by the deﬁnition of operator norm
[Equation]
we have
[Equation]
Now we vectorize all matrices and then compute derivatives of g with respect to θ and vec( W ) :
[Equation]
Differentiating again yields:
[Equation]
Then to establish part 1, use Assumption 2, Clairaut’s theorem, the deﬁnitions of the matrices and fact that their entries are in [0 , 1] , the relations (cid:107) AB (cid:107) 2 ≤ (cid:107) A (cid:107) 2 (cid:107) B (cid:107) 2 and (cid:107) vec W (cid:107) 1 ≤ √ mk (cid:107) vec W (cid:107) 2 = √ mk (cid:107) W (cid:107) F , and the fact that (cid:107) A ⊗ B (cid:107) 2 = (cid:107) A (cid:107) 2 (cid:107) B (cid:107) 2 to bound the spectral norm of each second derivative above.
The strong concavity statement follows by noticing ∇ 2 ww g ( θ , W ) (cid:52) − µ I [Equation] P (cid:98) y (cid:60) µ 2 λ
[Equation]
iff min i ∈
[ m ] p (cid:98) y
( i ) ≥ µ 2 λ .
Part 3 follows from the expression for W ∗ in the proof of Theorem 5.
Lemma 5. Consider f and g as deﬁned above.
Then we have
[Equation]
where both expectations are with respect to the empirical distribution on { z i } i ∈ [ N ] .
Proof.
The ﬁrst statement is obvious.
The second follows from Eq.
(44) in the proof of Lemma 4, since
[Equation]
Then use Assumption 2 and basic norm inequalities to bound the norm of each term.
E E XPERIMENT DETAILS E. M ODEL D ESCRIPTION
For all the experiments, the model’s output is of the form O = softmax( Wx + b ) .
The model outputs are treated as conditional probabilities p ( (cid:98) y = i | x ) = O i which are then used to estimate the ERMI regularizer.
We encode the true class label Y and sensitive attribute S using one-hot encoding.
We deﬁne (cid:96) () as the cross-entropy measure between the one-hot encoded class label Y and the predicted output vector O .
We perform the experiments in sections 5.1 and 5.2 with a linear model (with softmax activation).
The model parameters are estimated using the algorithm described in section 4.1.
In section 5.2, the data set is cleaned and processed as described in (Kearns et al., 2018).
The trade-off curves for FERMI are generated by sweeping across different values for λ in [0, 100], learning rate η in [0.0005, 0.01], and number of iterations T in [50, 200].
For the experiments in section 5.3, we create the synthetic color MNIST as described in (Li & Vasconcelos, 2019). We set the value σ = 0 . In ﬁgure 4, we compare the performance of stochastic solver (section 4.2) against the GD algorithm (section 4.1).
We use a mini-batch of size 512 when using the stochastic solver.
The color MNIST data has 60000 training samples, so using the stochastic solver gives a speedup of around 100x for each iteration, and an overall speedup of around 40x.
We present our results on two neural network architectures; namely, LeNet-5 (Lecun et al., 1998) and a Multi-layer perceptron (MLP).
We set the MLP with two hidden layers (with 300 andnodes) and an output layer with ten nodes.
A ReLU activation follows each hidden layer, and a softmax activation follows the output layer.
Some general advice for tuning λ : Larger value for λ generally translates to better fairness, but one must be careful to not use a very large value for λ as it could lead to poor generalization performance of the model.
The optimal values for λ , η , and T largely depend on the data and intended application.
We recommend starting with λ ≈ 10 .
In Appendix E.2, we can observe the effect of changing λ on the model accuracy and fairness for the COMPAS dataset.
We run ERMI algorithm for the binary case to COMPAS dataset to investigate the effect of hyperparameter tuning on the accuracy-fairness trade-off of the algorithm. As it can be observed in Fig. 5, by increasing λ from 0 to 1000 , test error (left axis, red curves) is slightly increased.
On the other hand, the fairness violation (right axis, green curves) is decreased as we increase λ to 1000 .
Moreover, for both notions of fairness (demographic parity with the solid curves and equality of opportunity with the dashed curves) the trade-off between test error and fairness follows the similar pattern.
To measure the fairness violation, we use demographic parity violation and equality of opportunity violation deﬁned in Section equation 5 for the solid and dashed curves respectively.
E. D ATASETS D ESCRIPTION
All of the following datasets are publicly available at UCI repository. German Credit Dataset. 3 German Credit dataset consists of 20 features (13 categorical andnumerical) regarding to social, and economic status of 1000 customers.
The assigned task is to classify customers as good or bad credit risks.
Without imposing fairness, the DP violation of the trained model is larger than 20% .
We chose ﬁrst 800 customers as the training data, and lastcustomers as the test data.
The sensitive attributes are gender, and marital-status.
Adult Dataset.
4 Adult dataset contains the census information of individuals including education, gender, and capital gain.
The assigned classiﬁcation task is to predict whether a person earns over 50k annually.
The train and test sets are two separated ﬁles consisting of 32,000 and 16,000 samples respectively.
We consider gender and race as the sensitive attributes (For the experiments involving one sensitive attribute, we have chosen gender).
Learning a logistic regression model on the training dataset (without imposing fairness) shows that only 3 features out of 14 have larger weights than the gender attribute.
Note that removing the sensitive attribute (gender), and retraining the model does not eliminate the bias of the classiﬁer.
the optimal logistic regression classiﬁer in this case is still highly biased.
For the clustering task, we have chosen 5 continuous features (Capital-gain, age, fnlwgt, capital-loss, hours-per-week), and 10,000 samples to cluster.
The sensitive attribute of each individual is gender.
Communities and Crime Dataset .
The dataset is cleaned and processed as described in (Kearns et al., 2018).
Brieﬂy, each record in this dataset summarizes aggregate socioeconomic information about both the citizens and police force in a particular U.S. community, and the problem is to predict whether the community has a high rate of violent crime. COMPAS Dataset .
6 Correctional Offender Management Proﬁling for Alternative Sanctions (COMPAS) is a famous algorithm which is widely used by judges for the estimation of likelihood of reoffending crimes.
It is observed that the algorithm is highly biased against the black defendants.
The dataset contains features used by COMPAS algorithm alongside with the assigned score by the algorithm within two years of the decision.