R ´ ENYI F AIR I NFERENCE
Anonymous authors
Paper under double-blind review
A BSTRACT
Machine learning algorithms have been increasingly deployed in critical automated decision-making systems that directly affect human lives.
When these algorithms are solely trained to minimize the training/test error, they could suffer from systematic discrimination against individuals based on their sensitive attributes, such as gender or race.
Recently, there has been a surge in machine learning society to develop algorithms for fair machine learning.
In particular, several adversarial learning procedures have been proposed to impose fairness.
Unfortunately, these algorithms either can only impose fairness up to linear dependence between the variables, or they lack computational convergence guarantees.
In this paper, we use R ´ enyi correlation as a measure of fairness of machine learning models and develop a general training framework to impose fairness.
In particular, we propose a min-max formulation which balances the accuracy and fairness when solved to optimality.
For the case of discrete sensitive attributes, we suggest an iterative algorithm with theoretical convergence guarantee for solving the proposed min-max problem.
Our algorithm and analysis are then specialized to fair classiﬁcation and fair clustering problems.
To demonstrate the performance of the proposed R´enyi fair inference framework in practice, we compare it with wellknown existing methods on several benchmark datasets.
Experiments indicate that the proposed method has favorable empirical performance against state-of-the-art approaches.
I NTRODUCTION
As we experience the widespread adoption of machine learning models in automated decisionmaking, we have witnessed increased reports of instances in which the employed model results in discrimination against certain groups of individuals – see Datta et al.
; Sweeney (2013);
Bolukbasi et al.
; Angwin et al.
In this context, discrimination is deﬁned as the unwanted distinction against individuals based on their membership to a speciﬁc group.
For instance,
Angwin et al.
(2016) present an example of a computer-based risk assessment model for recidivism, which is biased against certain ethnicities.
In another example, Datta et al.
(2015) demonstrate gender discrimination in online advertisements for web pages associated with employment.
These observations motivated researchers to pay special attention to fairness in machine learning in recent years; see Calmon et al.
; Feldman et al.
; Hardt et al.
; Zhang et al.
Xu et al.
; Dwork et al.
; Fish et al. (2016); Woodworth et al.
(2017); Zafar et al.
(2017; 2015); P´erez-Suay et al.
; Bechavod & Ligett (2017);
Liao et al. In addition to its ethical standpoint, equal treatment of different groups is legally required by many countries Act.
Anti-discrimination laws imposed by many countries evaluate fairness by notions such as disparate treatment and disparate impact.
We say a decision-making process suffers from disparate treatment if its decisions discriminate against individuals of a certain protected group based on their sensitive/protected attribute information.
On the other hand, we say it suffers from disparate impact if the decisions adversely affect a protected group of individuals with certain sensitive attribute – see Zafar et al. (2015).
In simpler words, disparate treatment is intentional discrimination against a protected group, while the disparate impact is an unintentional disproportionate outcome that hurts a protected group.
To quantify fairness, several notions of fairness have been proposed in the recent decade Calders et al.
; Hardt et al.
Examples of these notions include demographic parity , equalized odds , and equalized opportunity .
Demographic parity condition requires that the model output (e.g., assigned label) be independent of sensitive attributes.
This deﬁnition might not be desirable when the base ground-truth outcome of the two groups are completely different.
This shortcoming motivated the use of equalized odds notion
Hardt et al.
(2016) which requires that the model output is conditionally independent of sensitive attributes given the ground-truth label.
Finally, equalized opportunity requires having equal false positive or false negative rates across protected groups.
Machine learning approaches for imposing fairness can be broadly classiﬁed into three main categories: pre-processing methods, post-processing methods, and in-processing methods.
Preprocessing methods modify the training data to remove discriminatory information before passing data to the decision-making process Calders et al.
; Feldman et al.
(2015); Kamiran & Calders (2010; 2009; 2012)
; Dwork et al.
; Calmon et al.
; Ruggieri (2014).
These methods map the training data to a transformed space in which the dependencies between the class label and the sensitive attributes are removed Edwards & Storkey (2015);
Hardt et al.
; Xu et al.
Sattigeri et al.
; Raff & Sylvester (2018);
Madras et al.
; Zemel et al.
; Louizos et al.
On the other hand, post-processing methods adjust the output of a trained classiﬁer to remove discrimination while maintaining high classiﬁcation accuracy Fish et al.
; Dwork et al.
; Woodworth et al.
The third category is the in-process approach that enforces fairness by either introducing constraints or adding a regularization term to the training procedure
Zafar et al.
; P ´ erez-Suay et al.
; Bechavod & Ligett (2017);
Berk et al.
Agarwal et al.
; Celis et al.
; Donini et al.
(2018); Rezaei et al.
Kamishima et al.
(2011); Zhang et al.
(2018); Bechavod & Ligett (2017);
Kearns et al.
; Menon & Williamson (2018); Alabi et al.
The R ´ enyi fair inference framework proposed in this paper also belongs to this in-process category.
Among in-processing methods, many add a regularization term or constraints to promote statistical independence between the classiﬁer output and the sensitive attributes.
To do that, various independence proxies such as mutual information Kamishima et al.
(2011), false positive/negative rates
Bechavod & Ligett (2017), equalized odds Donini et al.
(2018), Pearson correlation coefﬁcient Zafar et al.
(2015; 2017), Hilbert Schmidt independence criterion (HSIC) P ´ erez-Suay et al.
(2017) were used.
As will be discussed in Section 2, many of these methods cannot capture nonlinear dependence between random variables and/or lead to computationally expensive algorithms.
Motivated by these limitations, we propose to use R ´ enyi correlation to impose several known group fairness measures.
R ´ enyi correlation captures nonlinear dependence between random variables.
Moreover, R´enyi correlation is a normalized measure and can be computed efﬁciently in certain instances.
Using R ´ enyi correlation coefﬁcient as a regularization term, we propose a min-max optimization framework for fair statistical inference.
In particular, we specialize our framework to both classiﬁcation and clustering tasks.
We show that when the sensitive attribute(s) is discrete (e.g., gender and/or race), the learning task can be efﬁciently solved to optimality, using a simple gradient ascentdescent approach.
We summarize our contributions next:
• We introduce R ´ enyi correlation as a tool to impose several notions of group fairness.
Unlike Pearson correlation and HSIC, which only capture linear dependence, R´enyi correlation captures any statistical dependence between random variables as zero R´enyi correlation implies independence.
Moreover, it is more computationally efﬁcient than the mutual information regularizers approximated by neural networks.
• Using R´enyi correlation as a regularization term in training, we propose a min-max formulation for fair statistical inference.
Unlike methods that use an adversarial neural network to impose fairness, we show that in particular instances such as binary classiﬁcation, or discrete sensitive variable(s), it sufﬁces to use a simple quadratic function as the adversarial objective.
This observation helped us to develop a simple multi-step gradient ascent descent algorithm for fair inference and guarantee its theoretical convergence to ﬁrst-order stationarity.
• Our R ´ enyi correlation framework leads to a natural fair classiﬁcation method and a novel fair K -means clustering algorithm.
For K -means clustering problem, we show that sufﬁciently large regularization coefﬁcient yields perfect fairness under disparate impact doctrine.
Unlike the two-phase methods proposed in Chierichetti et al.
; Backurs et al.
(2019); R¨osner & Schmidt (2018); Bercea et al.
; Schmidt et al.
(2018), our method does not require any preprocessing step, is scalable, and allows for regulating the trade-off between the clustering quality and fairness.
R ´ ENYI C ORRELATION
The most widely used notions for group fairness in machine learning are demographic parity, equalized odds, and equalized opportunities.
These notions require (conditional) independence between a certain model output and a sensitive attribute.
This independence is typically imposed by adding fairness constraints or regularization terms to the training objective function.
For instance, Kamishima et al.
(2011) added a regularization term based on mutual information.
Since estimating mutual information between the model output and sensitive variables during training is not computationally tractable, Kamishima et al.
(2011) approximates the probability density functions using a logistic regression model.
To have a tighter estimation, Song et al.
(2019) used an adversarial approach that estimates the joint probability density function using a parameterized neural network.
Although these works start from a well-justiﬁed objective function, they end up solving approximations of the objective function due to computational barriers.
Thus, no fairness guarantee is provided even when the resulting optimization problems are solved to global optimality in the large sample size limit.
A more tractable measure of dependence between two random variables is the Pearson correlation.
The Pearson correlation coefﬁcient between the two random variables A and B is deﬁned as ρ P ( A, B ) = [Equation]
( A, [Equation] √
Var( A ) √ Var( B ) , where Cov( · , · ) denotes the covariance and Var( · ) denotes the variance.
The Pearson correlation coefﬁcient is used in Zafar et al.
(2015) to decorrelate the binary sensitive attribute and the decision boundary of the classiﬁer.
A major drawback of Pearson correlation is that it only captures linear dependencies between random variables.
In fact, two random variables A and B may have strong dependence but have zero Pearson correlation.
This property raises concerns about the use of the Pearson correlation for imposing fairness.
Similar to the Pearson correlation, the HSIC measure proposed in P´erez-Suay et al.
(2017) may be zero even if the two variables have strong dependencies.
While universal Kernels can be used to alleviate this issue, they could arrive at the expense of computational intractability.
In addition, HSIC is not a normalized dependence measure Gretton et al.
(2005b;a) which raises concerns about the appropriateness of using it as a measure of dependence.
In this paper, we suggest to use Hirschfeld-Gebelein-R´enyi correlation R´enyi (1959); Hirschfeld (1935); Gebelein (1941) as a dependence measure between random variables to impose fairness.
R ´ enyi correlation, which is also known as maximal correlation, between two random variables A and B is deﬁned as
[Equation]
where the supremum is over the set of measurable functions f ( · ) and g ( · ) satisfying the constraints.
Unlike HSIC and Pearson correlation, R´enyi correlation captures higher-order dependencies between random variables.
R ´ enyi correlation between two random variables is zero if and only if the random variables are independent, and it is one if there is a strict dependence between the variables R´enyi (1959).
These favorable statistical properties of ρ R do not come at the price of computational intractability as opposed to other measures such as mutual information.
In fact, as we will discuss in Section 3, ρ R can be used in a computationally tractable framework to impose several group fairness notions.
A G ENERAL M IN -M AX F RAMEWORK FOR R ´ ENYI F AIR I NFERENCE
Consider a learning task over a given random variable Z .
Our goal is to minimize the average inference loss L ( · ) where our loss function is parameterized with parameter θ .
To ﬁnd the optimal value of parameter θ with the smallest average loss, we solve the following optimization problem
[Equation]
where the expectation is taken over Z and possible regularization terms are absorbed in the loss function L ( · ) .
Notice that this formulation is quite general and can include regression, classiﬁcation, clustering, or dimensionality reduction tasks as special cases.
As an example, in the case of linear regression Z =
( X , Y ) and L ( θ , Z ) =
( Y − θ T X ) 2 where X is a random vector and Y is the random target variable.
Assume that, in addition to minimizing the average loss, we are interested in bringing fairness to our learning task.
Let S be a sensitive attribute (such as age or gender) and (cid:98) Y θ ( Z ) be a certain output of our inference task using parameter θ .
Assume we are interested in removing/reducing the dependence between the random variable (cid:98) Y θ ( Z ) and the sensitive attribute S .
To balance the goodness-of-ﬁt and fairness, one can solve the following optimization problem
[Equation]
where λ is a positive scalar balancing fairness and goodness-of-ﬁt.
Notice that the above framework is quite general.
For example, ˆ Y θ may be the assigned label in a classiﬁcation task, the assigned cluster in a clustering task, or the output of a regressor in a regression task.
Using the deﬁnition of R´enyi correlation, we can rewrite optimization problem in equation 2 as
[Equation]
where the supremum is taken over the set of measurable functions.
The next natural question to ask is whether this optimization problem can be efﬁciently solved in practice.
This question motivates the discussions of the following subsection.
3. C OMPUTING R´ ENYI C ORRELATION
The objective function in equation 3 may be non-convex in θ in general.
Several algorithms have been recently proposed for solving such non-convex min-max optimization problems Sanjabi et al.
(2018); Nouiehed et al.
; Jin et al.
Most of these methods require solving the inner maximization problem to (approximate) global optimality.
More precisely, we need to be able to solve the optimization problem described in equation 1.
While popular heuristic approaches such as parameterizing the functions f and g with neural networks can be used to solve equation 1, we focus on solving this problem in a more rigorous manner.
In particular, we narrow down our focus to the discrete random variable case.
This case holds for many practical sensitive attributes among which are the gender and race.
In what follows, we show that in this case, equation 1 can be solved “efﬁciently” to global optimality.
Theorem 3.1 (Witsenhausen (1975)) .
Let a ∈ { a 1 , . .
. , a c } and b ∈ { b 1 , . . .
, b d } be two discrete random variables.
Then the R´enyi coefﬁcient ρ R ( a, b ) is equal to the second largest singular value of the matrix Q =
[ q ij ] i,j ∈ R c × d , where q ij =
[Equation]
The above theorem provides a computationally tractable approach for computing the R ´ enyi coefﬁcient.
This computation could be further simpliﬁed when one of the random variables is binary.
Theorem 3.2.
Suppose that a ∈ { 1 , . . .
, c } is a discrete random variable and b ∈ { 0 , 1 } is a binary random variable.
Let ˜ a be a one-hot encoding of a , i.e., ˜ a
= e i if a = i , where e i = (0 , . . .
. , 0) is the i -th standard unit vector.
Let ˜ b
= b − 1 / 2 .
Then,
[Equation]
= 1) P ( b = 0) , where γ (cid:44) min w ∈ R c
E (cid:104) ( w T (cid:101) a − ˜ b ) 2 (cid:105) .
Equivalently, γ (cid:44) min w ∈ R c c (cid:88) i =
w 2 i P ( a = i ) − c (cid:88) i = w i (cid:0) P ( a = i, b = 1) − P ( a = i, b = 0) (cid:1) + 1 / 4 .
Proof.
The proof is relegated to the appendix.
Let us specialize our framework to classiﬁcation and clustering problems in the next two sections.
R ´ ENYI F AIR C LASSIFICATION
In a typical (multi-class) classiﬁcation problem, we are given samples from a random variable Z (cid:44) ( X , Y ) and the goal is to predict Y from X .
Here
X ∈ R
d is the input feature vector, and Y ∈ Y (cid:44) { 1 , . .
. , c } is the class label.
Let (cid:98) Y
θ be the output of our classiﬁer taking different values in the set { 1 , . . . , c } . Assume further that
[Equation]
Here θ is that parameter of the classiﬁer that needs to be tuned.
For example, F ( θ , X ) =
( F 1 ( θ , X ) , . . .
, F c ( θ , X )) could represent the output of a neural network after softmax layer; the soft probability label assigned by a logistic regression model; or the 0-1 probability values obtained by a deterministic classiﬁer.
In order to ﬁnd the optimal parameter θ , we need to solve the optimization problem
[Equation]
where L is the loss function and the expectation is taken over the random variable Z
=
( X , Y ) .
Let
S be the sensitive attribute.
We say a model satisﬁes demographic parity if the assigned label (cid:98) Y is independent of the sensitive attribute S , see Dwork et al.
Using our regularization framework, to ﬁnd the optimal parameter θ balancing classiﬁcation accuracy and fairness objective, we need to solve
[Equation]
4. G ENERAL D ISCRETE
C ASE
When S ∈ { s 1 , . . .
, s d } is discrete, Theorem 3.1 implies that equation 5 can be rewritten as
[Equation]
Here v 1 = (cid:104)(cid:112) P ( S = s 1 ) , . . .
, (cid:112) P ( S = s d )
(cid:105) ∈
R
d is the right singular vector corresponding to the largest singular value of Q θ =
[ q ij ] i,j ∈ R c × d , with q ij (cid:44) P ( (cid:98) Y θ
= i | S
= s j ) P ( S = s j ) (cid:113)
P ( (cid:98) Y θ = i ) P ( S = s j ) .
Given training data ( x n , y n ) Nn =1 sampled from the random variable Z =
( X , Y ) , we can estimate the entries of the matrix Q θ using P (cid:0)(cid:98)
Y θ
=
i (cid:1) =
E [ P ( (cid:98) Y θ
=
i | X )]
≈ 1 N (cid:80)
Nn =1 F i ( θ , x n ) , and P (cid:0)(cid:98)
Y θ
= i | S =
s j (cid:1)
≈ |X
j | (cid:80) x ∈X j F i ( θ , x ) , where X j is the set of samples with sensitive attribute s j .
Motivated by the algorithm proposed in Jin et al.
(2019), we present Algorithm 1 for solving equation 6.
1 ← max v ∈⊥ v 1 , (cid:107) v (cid:107)≤ 1 f D ( θ t , v ) by ﬁnding the second singular vector of Q θ t 4:
Set θ t +1 ← θ t − η ∇ θ f D ( θ t , v t +1 ) 5: end for
To understand the convergence behavior of Algorithm 1 for the nonconvex optimization problem equation 6, we need to ﬁrst deﬁne an approximate stationary solution.
Let us deﬁne g ( θ ) = max v ∈⊥ v 1 , (cid:107) v (cid:107)≤ 1 f ( θ , v ) .
Assume further that f ( · , v ) has L 1 -Lipschitz gradient, then g ( · ) is L 1weakly convex; for more details check Raﬁque et al.
For such weakly convex function, we say θ ∗ is a (cid:15) -stationary solution if the gradient of its Moreau envelop is smaller than epsilon, i.e., (cid:107)∇ g β ( · ) (cid:107) ≤ ε with g β ( θ ) (cid:44) min θ (cid:48) g
( θ (cid:48) )
+ 1 2 β (cid:107) θ − θ (cid:48) (cid:107) and β < 1 2 L 1 is a given constant.
The following theorem demonstrates the convergence of Algorithm 1.
This theorem is a direct consequence of Theorem 27 in Jin et al.
Theorem 4.1.
Suppose that f is L 0 -Lipschitz and L 1 -gradient Lipschitz.
Then Algorithm 1 computes an ε -stationary solution of the objective function in equation 6 in O ( ε − 4 ) iterations.
4. B INARY C ASE
When S is binary, we can obtain a more efﬁcient algorithm compared to Algorithm 1 by exploiting
Theorem 3.2.
Particularly, by a simple scaling of λ and ignoring the constant terms, the optimization problem in equation 5 can be written as
[Equation]
Deﬁning ˜ S
= 2 S − 1 , the above problem can be rewritten as
[Equation]
Thus, given training data ( x n , y n ) Nn =1 sampled from the random variable Z =
( X , Y ) , we solve min θ max w [Equation]
[Equation]
Notice that the maximization problem in equation 8 is concave, separable, and has a closed-form solution.
We propose Algorithm 2 for solving equation 8.
Algorithm 2 R ´ do 3:
Set w t +1 ← arg max w f B ( θ t , w ) , i.e., set w t[-+
-]
[Equation]
1 i
[Equation]
Set θ t +1 ← θ t − η ∇ θ f B ( θ t , w t +1 ) 5: end for
While the result in Theorem 4.1 can be applied to Algorithm 2, under the following assumption, we can show a superior convergence rate.
Assumption 4.1.
We assume that there exists a constant scalar µ > 0 such that (cid:80) Nn =1 F i ( θ , x n ) ≥ µ, ∀ i = 1 , . .
. , C.
This assumption is reasonable when soft-max is used.
This is because we can always assume θ lies in a compact set in practice, and hence the output of the softmax layer cannot be arbitrarily small.
Theorem 4.2.
Suppose that f is L 1 -gradient Lipschitz.
Then Algorithm 2 computes an ε -stationary solution of the objective function in equation 8 in O ( ε − 2 ) iterations.
Proof.
The proof is relegated to the appendix.
Notice that this convergence rate is clearly a faster rate than the one obtained in Theorem 4.1.
Moreover, this rate of convergence matches the oracle lower bound for general non-convex optimization; see Carmon et al.
This observation shows that the computational overhead of imposing fairness is negligible as compared to solving the original non-convex training problem without imposing fairness.
Remark 4.3 (Extension to multiple sensitive attributes) .
Our discrete R´enyi classiﬁcation framework can naturally be extended to the case of multiple discrete sensitivity attributes by concatenating all attributes into one.
For instance, when we have two sensitivity attribute S 1 ∈ { 0 , 1 } and
S 2 ∈ { 0 , 1 } , we can consider them as a single attribute S ∈ { 0 , 1 , 2 , 3 } corresponding to the four combinations of { ( S 1 = 0 , S 2 = 0) , ( S 1 = 0 , S 2 = 1) , ( S 1 = 0 , S 2 = 0) , ( S 1 = 1 , S 2 = 1) } .
Remark 4.4 (Extension to other notions of fairness) .
Our proposed framework imposes the demographic parity notion of group fairness.
However, other notions of group fairness may be represented by (conditional) independence conditions.
For such cases, we can again apply our framework.
For example, we say a predictor ˆ Y θ satisﬁes equalized odds condition if the predictor ˆ Y θ is conditionally independent of the sensitive attribute S given the true label Y .
Similar to formulation equation 5, the equalized odds fairness notion can be achieved by the following min-max problem
[Equation]
R ´ ENYI F AIR C LUSTERING
In this section, we apply the proposed fair R´enyi framework to the widespread K -means clustering problem.
Given a set of data points x 1 , . . .
, x N ∈ R N × d , in the K -means problem, we seek to partition them into K clusters such that the following objective function is minimized:
[Equation]
where c k is the centroid of cluster k ; the variable a kn = 1 if data point x n belongs to cluster k and it is zero otherwise;
A =
[ a kn ] k,n and C =
[ c 1 , . .
c K ] represent the association matrix and the cluster centroids respectively.
Now, suppose we have an additional sensitive attribute S for each one of the given data points.
In order to have a fair clustering under disparate impact doctrine, we need to make the random variable a n =
[ a 1 n , . .
, a Kn ] independent of S .
In other words, we need to make the clustering assignment independent of the sensitive attribute S .
Using our framework in equation 2, we can easily add a regularizer to this problem to impose fairness under disparate impact doctrine.
In particular, for binary sensitive attribute S , using Theorem 3.2, and absorbing the constants into the hyper-parameter λ , we need to solve
[Equation]
where a n = ( a 1 n , . . .
, a Kn ) T encodes the clustering information of data point x n and s n is the sensitive attribute for data point n .
Fixing the assignment matrix A , and cluster centers C , the vector w can be updated in closed-form.
More speciﬁcally, w k at each iteration equals to the current proportion of the privileged group in the k -th cluster.
Combining this idea with the update rules of assignments and cluster centers in the standard K-means algorithm, we propose Algorithm 3, which is a fair K -means algorithm under disparate impact doctrine.
To illustrate the behavior of the algorithm, a toy example is presented in
Appendix C.
The main difference between this algorithm and the popular K -means algorithm is in Step 6 of
Algorithm 3.
This step is a result of optimizing equation 11 over A when both C and w are ﬁxed.
When λ = 0 , this step would be identical to the update of cluster assignment variables in Kmeans.
However, when λ > 0 , Step 6 considers fairness when computing the distance considered in updating the cluster assignments.
Remark 5.1.
Note that in Algorithm 3, the parameter w is being updated after each assignment of a point to a cluster.
More speciﬁcally, for every iteration of the algorithm, w is updated N times.
If we otherwise update w after completely updating the matrix A , then with a simple counterexample we can show that the algorithm can get stuck; see more details in Appendix C.1.
N UMERICAL E XPERIMENTS
In this section, we evaluate the performance of the proposed R´enyi fair classiﬁer and R´enyi fair kmeans algorithm on three standard datasets: Bank , German Credit , and Adult datasets.
The detailed description of these datasets is available in the supplementary material. do 4: Set do ∗ =
min k (cid:107) x n − c k (cid:107) 2 − λ k −
Set 11: end while We evaluate the performance of our proposed R ´ enyi classiﬁer under both demographic parity, and equality of opportunity notions.
We have implemented a logistic regression classiﬁer regularized by R ´ enyi correlation on Adult dataset considering gender as the sensitive feature.
To measure the equality of opportunity we use the Equality of Opportunity (EO) violation, deﬁned as
EO Violation = (cid:12)(cid:12) P ( ˆ Y =
| S = 1 , Y = 1) − P ( ˆ Y =
| S = 0 , Y = 1) (cid:12)(cid:12) , where ˆ Y and Y represent the predicted, and true labels respectively.
Smaller EO violation corresponds to a more fair solution.
Figure 1, parts (a) and (b) demonstrate that by increasing λ , the R ´ enyi regularizer coefﬁcient decreases implying a more fair classiﬁer at the price of a higher training and testing errors.
Figure 1, part (c) compares the fair R ´ enyi logistic regression model to several existing methods in the literature Hardt et al.
(2016); Zafar et al.
(2015); Rezaei et al.
; Donini et al.
As we can see in plot (c), the R´enyi classiﬁer outperforms other methods in terms of accuracy for a given level of EO violation.
The better performance of the R ´ enyi fair classiﬁer compared to the baselines could be attributed to the following.
Hardt et al.
(2016) is a post-processing approach where the output of the classiﬁcation is modiﬁed to promote a fair prediction.
This modiﬁcation is done without changing the classiﬁcation process.
Clearly, this approach limits the design space and cannot explore the possibilities that can reach with “in-processing” methods where both fairness and classiﬁcation objectives are optimized jointly. Zafar et al. (2015) imposes fairness by using linear covariance and thus can only capture linear dependence between the predictor and the sensitive attribute. Consequently, there might exist nonlinear dependencies which are revealed in fairness measures such as DP or EO violation.
Rezaei et al.
; Donini et al.
(2018), on the other hand, propose to use nonlinear measures of dependence as regularizers.
However, due to computational barriers, they approximate the regularizer and solve the approximate problem.
The approximation step could potentially have an adverse effect on the performance of the resulting classiﬁer.
Notice that while these methods are different in terms of the way they impose fairness, they are all implemented for logistic regression model (with the exception of SVM model used in Donini et al.
Thus, the difference in the performance is not due to the classiﬁcation model used in the experiments.
To show the practical beneﬁts of R´enyi correlation over Pearson correlation and HSIC regularizers under the demographic parity notion, we evaluate the logistic regression classiﬁer regularized by these three measures on Adult, Bank, and German Credit datasets.
For the ﬁrst two plots, we use p %
= min( P (ˆ Y =
| S =1) P (ˆ Y =
| S =0) , P (ˆ Y =1 | S =0) P (ˆ Y =
| S =1) )
as a measure of fairness.
Since p % is deﬁned only for binary sensitive variables, for the last two plots in Figure 2 (German dataset with gender and marital status, and Adult dataset with gender and race as the sensitive features), we use the inverse of demographic parity (DP) violation as the fairness measure.
We deﬁne DP violation as DP Violation = max a,b (cid:12)(cid:12) P ( ˆ Y = 1 | S = a ) − P ( ˆ Y =
| S = b ) (cid:12)(cid:12) .
As it is evident from the ﬁgure, R´enyi classiﬁer outperforms both HSIC and Pearson classiﬁers, especially when targeting high levels of fairness.
For the last two experiments in Figure 2, we could not further increase fairness by increasing the regularization coefﬁcient for Pearson and HSIC regularizers (see green and red curves cannot go beyond a certain point on the fairness axis).
This can be explained by the nonlinear corre- lation between the predictor and the sensitive variables in these two scenarios which cannot be fully captured using linear or quadratic independence measures.
Interestingly, our experiments indicate that minimizing R´enyi correlation eventually minimizes the Normalized Mutual Information (NMI) between the variables (See Supplementary Figure 6).
Recall that similar to R´enyi correlation, NMI can capture any dependence between two given random variables.
Finally, to evaluate the performance of our fair k-means algorithm, we implement Algorithm 3 to ﬁnd clusters of Adult and Bank datasets.
We use the deviation of the elements of the vector w as a measure of fairness.
The element w k of w represents the ratio of the number of data points that belong to the privileged group ( S = 1 ) in cluster k over the number of data points in that cluster.
This notion of fairness is closely related to minimum balance introduced by Chierichetti et al.
The deviation of these elements is a measure for the deviation of these ratios across different clusters.
A clustering solution is exactly fair if all entries of w are the same.
For K = 14 , we plot in Figurethe minimum, maximum, average, and average ± standard deviation of the entries of w vector for different values of λ .
For an exactly fair clustering solution, these values should be the same.
As we can see in Figure 3, increasing λ yields exact fair clustering at the price of a higher clustering loss.  C ONCLUSION
In this paper, we proposed R´enyi fair inference as an in-process method to impose fairness in empirical risk minimization.
Fairness is deﬁned as (conditional) independence between a sensitive attribute and the inference output from the learning machine.
As statistical independence is only measurable when the data distributions are fully known, we can only hope to promote independence through empirical surrogates in this framework.
Our method imposes a regularizer in the form of the R´enyi correlation (maximal correlation) between a sensitive attribute(s) and the inference output. R´enyi correlation between two random variables is zero if and only if they are independent, which is a desirable property for an independence surrogate. We pose R´enyi fair correlation as a minimax optimization problem. In the case where the sensitive attributes are discrete (e.g., race), we present an algorithm that ﬁnds a ﬁrst-order optimal solution to the problem with convergence guarantees.
In the special case where the sensitive attribute is binary (e.g., gender), we show an algorithm with optimal convergence guarantees.
Our numerical experiments show that R´enyi fair inference captures nonlinear correlations better than Pearson correlation or HSIC.
We also show that increasing the regularization hyperparameter results in near statistical independence between the sensitive attribute and the inference output.
Future work would naturally consider extension to continuous sensitive attributes and problems with missing or non-explicit sensitive labels such as fair word embedding problem.
A A PPENDIX : P ROOF OF T HEOREM 3.
Proof.
First, notice that since ˜ a is a one-hot encoding of a , any function f : { 1 , . .
, c } (cid:55)→ R can be equivalently represented as f (˜ a ) =
u T ˜ a for some u ∈ R c .
Therefore, following the deﬁnition of R´enyi correlation, we can write
[Equation]
Notice that since b is binary, there is a unique function g ( b ) =
[Equation]
satisfying the constraints where q (cid:44) P ( b = 1) .
Therefore, the above optimization problem can be written as
[Equation]
The last constraint simply implies that u should be orthogonal to p (cid:44) E
[˜ a ] , which is a stochastic vector capturing the distribution of a .
Equivalently, we can write u = (cid:16) I − pp T (cid:107) p (cid:107) 2 (cid:17) v for some v ∈ R c .
Thus, we can simplify the above optimization problem as
[Equation]
where in the constraint, we used the equality E (cid:2)
˜ a ˜ a T (cid:3) =
diag ( p ) , which follows the deﬁnition.
Let us do the change of variable ˆ v = diag ( √ p )( I − pp T (cid:107) p (cid:107) 2 ) v .
Then the above optimization can be simpliﬁed to
[Equation]
Clearly, this leads to
[Equation]
where in the last equality we use the fact that g (1) = (cid:113) 1 − qq and g (0)
= − (cid:113) q 1 − q .
Deﬁne p i 0 (cid:44)
P ( a = i, b = 0) and p i 1 (cid:44) P ( a = i, b = 0) , p i (cid:44) P ( a = i ) = p i 0 + p i 1 .
Then, using simple algebraic manipulations, we have that
[Equation]
where in the last equality we used the deﬁnition of γ and the optimal value of equation
[Equation]
B P ROOF OF T HEOREM 4.
Proof.
Deﬁne g B ( θ ) = max w f B ( θ , w ) .
Since the optimization problem max w f B ( θ , w ) is strongly concave in w , using Danskin’s theorem (see Danskin (1967) and Bertsekas (1971)), we conclude that the function g B ( · ) is differentiable.
Moreover,
[Equation]
where ¯ w = arg max w f B (¯ θ , w ) .
Thus Algorithm 2 is in fact equivalent to the gradient descent algorithm applied to g B ( θ ) .
Thus according to (Nesterov, 2018, Chapter 1), the algorithm ﬁnds a point with (cid:107)∇ g B ( θ ) (cid:107) ≤ (cid:15) in O ( (cid:15) − 2 ) iterations.
C R ´ ENYI F AIR K- MEANS
To illustrate the behavior of Algorithm 3, we deployed a simple two-dimensional toy example.
In this example we generated data by randomly selecting 5 center points and then randomly generating 500 data points around each center according to a normal distribution with small enough variance.
The data is shown in Figure 4 with different colors corresponding to different clusters.
Moreover, we assigned for each data point x i a binary value s i ∈ { 0 , 1 } that corresponds to its sensitive attribute.
This assignment was also performed randomly except for points generated around center 2 (green points in Figure 4) which were assigned a value of 1 and points generated around center 4 (blue points in Figure 4) which were assigned a value of 0 .
Without imposing fairness, traditional Kmeans algorithm would group points generated around center 2 in one cluster regardless of the fact that they all belong to the same protected group.
Similarly, points generated around center 4 will belong to the same cluster.
Hence, according to traditional K-means clustering shown in Figure 4, the proportion of the protected group in clusters 2 and 4 are 1 and 0 respectively.
However, when imposing our fairness scheme, we expect these points to be distributed among various clusters to achieve balanced clustering.
This is illustrated in Figure 5.
It is evident from Figure 5 that increasing lambda, data points corresponding to centers 2 and 4 are now distributed among different clusters.
C. U PDATING w AFTER UPDATING THE ASSIGNMENT OF EACH DATA POINT IN A LGORITHM
To understand the reasoning behind updating the vector of proportions w after updating each a i which is the assignment of data point i , we discuss a simple one-dimensional counterexample.
Consider the following four data points X 1 = − 5 , X 2 = − 4 , X 3 = 4 , and X 4 = 5 with their corresponding sensitive attributes S 1 = S 2 = 1 and S 3 = S 4 = 0 .
Moreover, assume the following initial A 0 and C
[Equation]
Hence, X 1 and X 2 which both have a sensitive attribute of 1 are assigned to cluster 1 with center
C
= − 4 .
5 and X 3 and X 4 which both have a sensitive attribute of 0 are assigned to cluster 2 with center C 02 = 4 . 5 .
Then w which is the current proportion of the privileged group in the clusters will be w 0 =
Now, for sufﬁciently large λ if we update A according to Step 6 of Algorithm 3, we get the following new assignment
[Equation]
Hence, the points just switch their clusters.
Then, performing another iteration will get us back to the initial setup and the algorithm will get stuck between these two states that are both not fair.
To overcome this issue we update the proportions w after updating the assignment of each data point.
D D ATASETS D ESCRIPTION
In this section we introduce the datasets used in numerical experiment discussed in Section 6.
All of these datasets are publicly available at UCI repository.
• German Credit Dataset : 1 German Credit dataset consists of 20 features (13 categorical and 7 numerical) regarding to social, and economic status of 1000 customers.
The assigned task is to classify customers as good or bad credit risks.
Without imposing fairness, the DP violation of the trained model is larger than 20% .
We chose ﬁrst 800 customers as the training data, and last 200 customers as the test data.
The sensitive attributes are gender, and marital-status.
• Bank Dataset : 2 Bank dataset contains the information of individuals contacted by a Portuguese bank institution.
The assigned classiﬁcation task is to predict whether the client will subscribe a term deposit.
For the classiﬁcation task we consider all 17 attributes (except martial status as the sensitive attribute).
Removing the sensitive attribute, and train a logistic regression model on the dataset, yields to a solution that is biased under the demographic parity notion ( p % = 70 .
To evaluate the performance of the classiﬁer, we split data into the training (32000 data points), and test set (13211 data points).
For the clustering task, we sampled 3 continuous features: Age, balance, and duration.
The sensitive attribute is the marital status of the individuals.
• Adult Dataset : 3 Adult dataset contains the census information of individuals including education, gender, and capital gain. The assigned classiﬁcation task is to predict whether a person earns over 50K annually.
The train and test sets are two separated ﬁles consisting of 32000 and 16000 samples respectively. We consider gender and race as the sensitive attributes (For the experiments involving one sensitive attribute, we have chosen gender). Learning a logistic regression model on the training dataset (without imposing fairness) shows that only 3 features out of 14 have larger weights than the gender attribute. Note that removing the sensitive attribute (gender), and retraining the model does not eliminate the bias of the classiﬁer.
the optimal logistic regression classiﬁer in this case is still highly biased ( p % = 31 . For the clustering task, we have chosen 5 continuous features (Capital-gain, age, fnlwgt, capital-loss, hours-per-week), and 10000 samples to cluster.
The sensitive attribute of each individual is gender.
E S UPPLEMENTARY F IGURES
[Figure]
F F AIR N EURAL N ETWORK
In this section, we train a 2-layers neural network on the adult dataset regularized by the R´enyi correlation.
In this experiment, the sensitive attribute is gender.
We set the number of nodes in the hidden layer, the batch-size, and the number of epochs to 12, 128, and 50, respectively.
The following table depicts the performance of the trained model.
[Table]