A
N EAR -O PTIMAL
A LGORITHM
FOR
D EBIASING T
RAINED M
ACHINE
L EARNING M ODELS
Anonymous authors
Paper under double-blind review
A BSTRACT
We present an efﬁcient and scalable algorithm for debiasing trained models, including deep neural networks (DNNs), which we prove to be near-optimal by bounding its excess Bayes risk.
Unlike previous black-box reduction methods to cost-sensitive classiﬁcation rules, the proposed algorithm operates on models that have been trained without having to retrain the model.
Furthermore, as the algorithm is based on projected stochastic gradient descent (SGD), it is particularly attractive for deep learning applications.
We empirically validate the proposed algorithm on standard benchmark datasets across both classical algorithms and modern DNN architectures and demonstrate that it outperforms previous postprocessing approaches for unbiased classiﬁcation
I NTRODUCTION
Machine learning is increasingly applied to critical decisions which can have a lasting impact on individual lives, such as for credit lending (Bruckner, 2018), medical applications (Deo, 2015), and criminal justice (Brennan et al., 2009).
Consequently, it is imperative to understand and improve the degree of bias of such automated decision-making.
Unfortunately, despite the fact that bias (or “fairness”) is a central concept in our society today, it is difﬁcult to deﬁne it in precise terms.
In fact, as people perceive ethical matters differently depending on a plethora of factors including geographical location or culture (Awad et al., 2018), no universally-agreed upon deﬁnition for bias exists.
Moreover, the deﬁnition of bias may depend on the application and might even be ignored in favor of accuracy when the stakes are high, such as in medical diagnosis (Kleinberg et al., 2017; Ingold and Soper, 2016).
As such, it is not surprising that several deﬁnitions of “unbiased classiﬁcation” have been introduced.
These include statistical parity (Dwork et al., 2012; Zafar et al., 2017a), equality of opportunity (Hardt et al., 2016), and equalized odds (Hardt et al., 2016; Kleinberg et al., 2017).
Unfortunately, such deﬁnitions are not generally compatible (Chouldechova, 2017) and some might even be in conﬂict with calibration (Kleinberg et al., 2017).
In addition, because fairness is a societal concept, it does not necessarily translate into a statistical criteria (Chouldechova, 2017; Dixon et al., 2018).
Conditional statistical parity Let X be an instance space and let Y = { 0 , 1 } be the target set in a standard binary classiﬁcation problem.
In the fair classiﬁcation setting, we may further assume the existence of a (possibly randomized) sensitive attribute 1 S :
X → { 0 , 1 } , where 1 S ( x ) indicates if an instance x ∈ X belongs to some sensitive class S ⊆ X .
For example, X might correspond to the set of job applicants while S are the female applicants.
Here, the sensitive attribute can be randomized if, for instance, the gender of an applicant is not a deterministic function of the full instance x ∈ X (e.g. number of publications, years of experience, ...etc).
Then, a general theme among common deﬁnitions of bias is to ﬁx a partitioning of the instance space X
= ∪ k X k
where
X k 1 ∩ X k 2 = ∅ for k 1 (cid:54) =
k 2 and require a balanced representation of the sensitive class within some or all of the subsets X k (i.e., the groups ).
We note that the groups X k are different from the sensitive class S .
If the sensitive attribute is e.g. gender, the groups can be based on race, age, etc., but not on gender.
In the latter case, the goal is to make sure that females and males are treated equally within each group (e.g. males and females are treated the same within every racial group, not just overall).
This property is well-captured through the notion of statistical parity (Corbett-Davies et al., 2017;
Dwork et al., 2012; Zafar et al., 2017a; Mehrabi et al., 2019):
[Figure]
Deﬁnition 1 (Conditional Statistical Parity) .
Let a , b , c ∈ { 0 , 1 } be random variables.
Let C ( a , b ) .
=
E [ a · b ] − E [ a ] · E [ b ] be their covariance, and C ( a , b | c ) their covariance conditioned on c :
C ( a , b | c = c ) =
E [ a · b | c = c ] − E [ a | c = c ] · E [ b | c = c ] .
A classiﬁer f : X → { 0 , 1 } satisﬁes (cid:15) conditional statistical parity in some group X k ⊆ X with respect to a sensitive class S if | C (cid:0) f ( x ) , 1 S ( x ) | x ∈ X k (cid:1) | ≤ (cid:15) .
A classiﬁer f satisﬁes (cid:15) conditional statistical parity with respect to S across all groups { X k } k =1 , 2 ,...,K if it satisﬁes (cid:15) conditional statistical parity in each X k .
To motivate and further clarify the deﬁnition, we showcase the empirical results on the Adult benchmark dataset (Blake and Merz, 1998) in Figure 1.
When tasked with predicting whether the income of individuals is above $50K per year, all considered classiﬁers exhibit gender-related bias.
One way of removing such bias is to enforce statistical parity across genders.
Crucially, however, without conditional statistical parity, different demographic groups may experience different outcomes.
In fact, gender bias can actually increase in some minority groups after enforcing statistical parity.
Applying our algorithm which enforces conditional statistical parity removes this bias (green bars).
Our main contribution is to present a near-optimal recipe for debiasing models, including deep neural networks, according to Deﬁnition 1.
Speciﬁcally, we formulate the task of debiasing learned models as a regularized optimization problem that is solved efﬁciently using the projected SGD method.
We show how the algorithm produces thresholding rules with randomization near the thresholds, where the width of randomization is controlled by the regularization parameter.
We also show that randomization near the threshold is necessary for Bayes risk consistency.
While we focus on binary sensitive attributes, our results can be readily extended to non-binary attributes (see Appendix F).
Statement of Contribution
We derive a near-optimal post-processing algorithm for debiasing learned models.
We motivate the algorithm via an impossibility result, which quantiﬁes the tradeoff between accuracy and fairness across multiple demographic groups (Section 3).
We prove theoretical guarantees for the proposed algorithm, including an explicit bound on its Bayes excess risk (Section 4).
We empirically validate the proposed algorithm on benchmark datasets and across both classical algorithms and modern DNN architectures.
Our experiments demonstrate that the proposed algorithm signiﬁcantly outperforms previous post-processing methods (Section 5).
R ELATED W ORK
Algorithms for fair machine learning can be broadly classiﬁed into three groups: (1) pre-processing methods, (2) in-processing methods, and (3) post-processing methods (Zafar et al., 2019).
Preprocessing algorithms transform the data into a different representation such that any classiﬁer trained on it will not exhibit bias.
This includes methods for learning a fair representation (Zemel et al., 2013; Lum and Johndrow, 2016; Bolukbasi et al., 2016; Calmon et al., 2017; Madras et al., 2018; Kamiran and Calders, 2012), label manipulation (Kamiran and Calders, 2009), data augmentation (Dixon et al., 2018), or disentanglement (Locatello et al., 2019).
On the other hand, in-processing methods constrain the behavior of learning algorithms in order to control bias.
This includes methods based on adversarial learning (Zhang et al., 2018) and constraint-based classiﬁcation, such as by incorporating constrains on the decision margin (Zafar et al., 2019) or features (Grgi´c-Hlaˇca et al., 2018).
Agarwal et al.
(2018) showed that the task of learning an unbiased classiﬁer could be reduced to a sequence of cost-sensitive classiﬁcation problems, which could be applied to any black-box classiﬁer.
One caveat of the latter approach is that it requires solving a linear program (LP) and retraining classiﬁers, such as neural networks, many times before convergence.
The algorithm we propose in this paper is a post-processing method, which can be justiﬁed theoretically (Corbett-Davies et al., 2017; Hardt et al., 2016; Menon and Williamson, 2018; Celis et al., 2019).
Fish et al.
(2016) and Woodworth et al.
(2017) fall under this category.
However, the former only provides generalization guarantees without consistency results and does not describe how the thresholds can be learned, while the latter proposes a two-stage approach that requires changes to the original training algorithm.
In contrast, our algorithm is Bayes consistent and does not alter the original classiﬁcation method.
In Celis et al.
and Menon and Williamson (2018), instancedependent thresholding rules are also learned.
However, our algorithm also learns to randomize around the threshold (Figure 2(a)) and this randomization is key to our algorithm both theoretically as well as experimentally (Appendix D and Section 5).
Hardt et al.
(2016) learns a randomized post-processing rule but our proposed algorithm outperforms it in all of our experiments (Section 5).
Woodworth et al.
(2017) showed that the post-processing approach can, sometimes, be highly suboptimal.
Nevertheless, the latter result does not contradict the statement that our post-processing rule is near-optimal because we assume that the original classiﬁer outputs a monotone transformation of some approximation to the posterior probability p ( y = 1 | x ) (e.g. margin or softmax output) whereas Woodworth et al.
(2017) assumed in their construction that the post-processing rule had access to the binary predictions only.
We argue that the proposed algorithm has distinct advantages, particularly for deep neural networks (DNNs).
First, stochastic convex optimization methods are well-understood and can scale well to massive amounts of data (Bottou, 2010), which is often the case in deep learning today.
Second, the guarantees provided by our algorithm hold w.r.t.
the binary predictions instead of using a proxy, such as the margin as in some previous works (Zafar et al., 2017b; 2019).
Third, unlike previous reduction methods to cost-sensitive classiﬁcation rules that would require retraining a deep neural network several times until convergence (Agarwal et al., 2018), which can be prohibitively expensive, our algorithm operates on learned models that are trained once and does not require retraining.
Besides developing algorithms for fair classiﬁcation, several recent works focused on other related aspects, such as proposing new deﬁnitions for fairness; e.g. demographic parity (Dwork et al., 2012; Mehrabi et al., 2019), equalized odds (Hardt et al., 2016), equality of opportunity/disparate mistreatment (Zafar et al., 2017a; Hardt et al., 2016), and individual fairness (Dwork et al., 2012).
Recent works have also established several impossibility results related to fair classiﬁcation, such as Kleinberg et al.
(2017); Chouldechova (2017).
In our case, we derive a new impossibility result that holds for any deterministic binary classiﬁer, which highlights that the groups X
k (see Section 1) when deﬁning fairness have to be known in advance .
N EAR - OPTIMAL ALGORITHM FOR C ONDITIONAL S TATISTICAL P ARITY
Notation We reserve boldface letters for random variables (e.g. x ), small letters for instances (e.g. x ), capital letters for sets (e.g. X ), and calligraphic typeface for universal sets (e.g. the instance space X ).
Given a set S , 1 S ( x ) ∈ { 0 , 1 } is the characteristic function indicating whether x ∈ S .
We denote by [ n ] the set of integers { 1 , . . . , n } and [ x ] + = max { 0 , x } .
Algorithm Given a classiﬁer f : X → [ − 1 , +1] our goal is to post-process the predictions made by f 1 in order to control the bias with respect to a sensitive attribute 1 S :
X → { 0 , 1 } and across a ﬁxed set of pairwise disjoint groups X 1 , . . .
, X K ⊆ X as in Deﬁnition 1.
To this end, instead of outputing the result of f directly, we consider randomized prediction rules of the form
[Equation]
where ˜ h ( x ) represents the probability of predicting the positive class given (i) instance x ∈ X , (ii) group X k , (iii) sensitive attribute 1 S ( x ) , and (iv) classiﬁer’s output f ( x ) .
As shown in Appendix C, for post-processing rule ˜ h ( x ) , and for each group X k ⊆ X , the fairness constraint in Deﬁnition 1 can be written as | E x
[(1 S ( x ) − ρ k ) ˜
h
( x ) | x ∈ X k ] | ≤ (cid:15) , where ρ k = E x
[1 S ( x ) | x ∈ X k ] .
On the other hand, minimizing the probability of altering the predictions of the original classiﬁer can be achieved by maximizing the inner product E x [˜ h ( x ) · f ( x )] .
Instead of optimizing this quantity directly, which would lead to a pure thresholding rule, we minimize the regularized objective: ( γ/ 2)
E x [˜ h ( x ) 2 ] − E x
[˜ h ( x ) · f ( x )] .
This regularization leads to randomization around the threshold, which we show to be critical, both theoretically (Section 4) and experimentally (Section 5).
Using Lagrange duality we show that the solution reduces to the update rules in Equation 2 with optimization variables { λ k , µ k } k ∈
[ K ] and the corresponding predictor which outputs + for group X k with probability ˜ h γ ( x ) is given by
[Equation]
where ξ γ is given by Eq.
(3) and z k ( x ) = 1 S ( x ) − ρ k .
Here, z k ( x ) assigns different values inside and outside of the sensitive class, which allows the model to produce different thresholding rules.
Update rules To learn these parameters, one can apply the following update rules (Appendix C):
[Equation]
Here, the function ξ γ : R → R + is given by:
[Equation]
Note that ξ γ is convex and its derivative ξ (cid:48)
γ is (1 /γ ) -Lipschitz continuous; it can be interpreted as differentiable approximation to the ReLU unit (Nair and Hinton, 2010).
A full pseudocode of the proposed algorithm is presented in Appendix A.
T HEORETICAL A NALYSIS
We will ﬁrst show (Appendix B) that attaining conditional statistical parity as speciﬁed in Deﬁnition 1 is, generally, impossible unless the groups are known in advance.
Proposition 1 (Impossibility result) .
Let X be the instance space and Y = { 0 , 1 } be a target set.
Let 1 S :
X → { 0 , 1 } be an arbitrary (possibly randomized) binary-valued function on X and deﬁne γ : X → [0 , 1] by γ ( x )
=
p (1 S ( x ) = 1 | x = x ) , where the probability is evaluated over the randomness of 1 S : X → { 0 , 1 } .
Write ¯ γ =
E x
[ γ ( x )] .
Then, for any binary predictor f : X → { 0 , 1 } it holds that
[Equation]
[Equation]
where C (cid:0) f ( x ) , γ ( x ) | π ( x ) (cid:1) is deﬁned in Deﬁnition 1.
[Figure]
Consequently, a deterministic classiﬁer f : X → { 0 , 1 } cannot be universally unbiased with respect to a sensitive class S across all possible known and unknown groups unless the representation x has zero mutual information with the sensitive attribute or if f is constant almost everywhere.
As a corollary, the groups X k have to be known in advance .
Next, we show that the prediction rule in Equation 1 learned through the update rules presented in
Section 3 satisﬁes the desired fairness guarantees on the training sample.
Theorem 1 (Correctness) .
Let ˜ h γ : X → [0 , 1] be the randomized predictor in Equation 1 learned by applying the update rules in Equation 2 starting with µ k = 0 , λ k = 0 ,
∀ k ∈
[ K ] until convergence.
Then, ˜ h
γ satisﬁes (cid:15) conditional statistical parity w.r.t.
{ X k } k ∈ [ K ] in the training sample.
The proof of Theorem 1 is presented in Appendix C. The following guarantee, which holds w.r.t. the underlying data distribution, shows that the randomized prediction rule converges to the Bayes optimal unbiased classiﬁer if the original classiﬁer f is Bayes consistent.
The proof of the following theorem (Appendix D) is based on the Lipschitz continuity of the decision rule when γ > 0 and the robustness framework of Xu and Mannor (2010).
Theorem 2.
Let h (cid:63) =
arg min h ∈H E [ h ( x ) (cid:54) =
y ] , where H is the set of binary predictors on X that satisfy fairness according to Deﬁnition 1 for some (cid:15) > 0 .
Let ˜ h γ : X → [0 , 1] be the randomized learning rule in Equation 1.
If ˜ h γ is trained on a freshly sampled data of size N , then with a probability of at least 1 − δ :
[Equation]
where η ( x ) = p ( y = 1 | x = x ) is the Bayes regressor.
Consequently, if the original classiﬁer is Bayes consistent and we have: N → ∞ , γ → 0 + and γN − 13 → ∞ , then E [˜ h γ ( x ) (cid:54) =
y ] P −→ E [ h (cid:63) ( x ) (cid:54) =
y ] .
Hence, the updates converge to the optimal prediction rule subject to the chosen fairness constraint.
Running time As shown in Appendix C, the update rules in Equation 2 perform a projected stochastic gradient descent on the following optimization problem:
[Equation]
We assume with no loss of generality that f ( x ) ∈ [ − 1 , 1] since f ( x ) is assumed to be an estimator to 2 η ( x ) − 1 (see Section 3 and Appendix C) and any thresholding rule over f ( x ) can be transformed into an equivalent rule over a monotone increasing function of f (i.e. using the hyperbolic tangent).
[Figure]
The curves correspond to different original classiﬁers.
In general, having a small value γ (cid:28) 1 yields better results but for some classiﬁers, such as k -NN or neural networks, in which randomization at the threshold is often necessary, it is best to tune the value of γ with a validation dataset.
Proposition 2.
Let µ (0) =
λ (0) =
and write µ ( t ) , λ ( t ) ∈ R K for the value of the optimization variables after t updates deﬁned in Equation 2 for some ﬁxed learning rate α t
= α .
Let ¯ µ = (1 /T )
(cid:80) Tt =1 µ ( t ) ( x ) and ¯ λ
= (1 /T )
(cid:80) Tt =
1 λ ( t ) ( x ) .
Then,
[Equation]
where ¯ F : R K × R K → R is the objective function in (5) using the averaged solution ¯ µ and ¯ λ while
F (cid:63) is its optimal value.
In particular, E [ ¯ F ] − F ( µ (cid:63) ) (cid:1) =
O ( (cid:112) K/T )
when α =
O
( (cid:112) K/T ) .
The proof is in Appendix E. Hence, the post-processing rule can be efﬁciently computed.
In practice, we observe fast convergence as shown in Figure 2(b).
As shown in Figure 2(a), the hyperparameter γ controls the width of randomization around the thresholds.
A large value of γ may reduce the accuracy of the classiﬁer.
On the other hand, γ cannot be zero because randomization around the threshold is, in general, necessary for Bayes risk consistency as illustrated in the following example:
Example 1 (Randomization is necessary) .
Suppose that X = {− 1 , 0 , 1 } where p ( x = − 1)
= 1 / 2 , p ( x = 0)
= 1 / 3 and p ( x = 1) = 1 / 6 .
Let η ( − 1) = 0 , η (0) = 1 / 2 and η (1) = 1 .
In addition, let s be a sensitive attribute, where p ( s = 1 | x = − 1) = 1 / 2 , p ( s = 1 | x = 0)
= 1 , and p ( s = 1 | x = 1) = 0 .
Then, the Bayes optimal prediction rule f (cid:63) ( x ) subject to statistical parity w.r.t.
s satisﬁes: p ( f (cid:63) ( x ) = 1 | x = − 1) = 0 , p ( f (cid:63) ( x ) =
| x = 0) = 1 / 2 and p ( f (cid:63) ( x ) = 1 | x = 1) = 1 .
Note that the Bayes excess risk bound in Theorem 2 is vacuous when γ = 0 .
Therefore, γ controls a trade-off depending on how crucial randomization is around the thresholds (e.g. in k -NN where the classiﬁer’s scores come from a ﬁnite set) which is demonstrated in Figure 3.
In our experiments, γ is always chosen using a validation set.
E MPIRICAL EVALUATION
Tabular Data We empirically evaluate the performance of the proposed algorithm and the baselines on two real-world datasets, namely the Adult income dataset and the Default of Credit Card
Clients (DCCC) dataset, both taken from the UCI Machine Learning Repository (Blake and Merz, 1998).
The Adult dataset contains 48,842 records with 14 attributes each and the goal is to predict if the income of an individual exceeds $50K per year.
The DCCC dataset contains 30,000 records with 24 attributes, and the goal is to predict if a client will default on their credit card payment.
Both datasets include sensitive attributes, such as sex and age.
In Figure 1 we showcased why conditional statistical parity is critical to ensure that females and males are treated equally across all demographic groups.
However, since the demographic groups are disjoint, it sufﬁces to focus on (unconditional) statistical parity to fairly compare the proposed algorithm to the existing baselines.
In our experiments, we deﬁne the sensitive class S to be the class of females.
In the DCCC dataset, we additionally introduce bias in the training set for the purpose of the experiment: if 1 S ( x ) =
y ( x ) we keep the instance and otherwise drop it with probability 0.5.
[Table]
We train four classiﬁers on each dataset: (1) random forests with maximum depth 10, (2) k -NN with k = 10 , (3) a two-layer fully connected neural network with 128 hidden nodes, and (4) logistic regression.
For the latter, we ﬁne-tune the parameter C in a grid of values chosen in a logarithmic scale between 10 − 4 and 10 4 using 10-fold cross validation.
The rate in is ﬁxed to 10 − 1 ( K/T ) 1 / 2 , where T is the number of
, and (cid:15)
We compare against two standard post-processing methods: (1) the post-processing algorithm of
Hardt et al.
(2016) and (2) the shift inference method, ﬁrst introduced in (Saerens et al., 2002)
and used more recently in (Royer and Lampert, 2015; Wang et al., 2020).
We use the implementation of the algorithm of Hardt et al.
(2016) in the FairLearn software package (Dudik et al., 2020).
Table 1 shows the bias and accuracy on test data after applying each post-processing method.
The ﬁrst column, marked as “original”, corresponds to the original classiﬁer without any alteration.
As shown in the table, both our proposed post-processing algorithm and the algorithm of Hardt et al.
(2016) eliminate bias in all classiﬁers.
By contrast, the shift-inference method does not always eliminate bias.
However, the proposed algorithm has a much lower impact on the test accuracy and can even improve it in certain cases.
The fact that fairness can sometimes improve the test accuracy was recently noted by Blum and Stangl (2020).
CelebA Dataset
Our second set of experiments builds on the task of predicting the “attractiveness” attribute in the CelebA dataset (Liu et al., 2015).
CelebA contains 202,599 images of celebrities annotated with 40 binary attributes, such as gender, attractiveness, and smiling.
We use two standard deep neural network architectures: ResNet50 (He et al., 2016) and MobileNet (Howard et al., 2017), either trained from scratch or pretrained on ImageNet.
We present the results in Table 2.
We observe that the proposed algorithm outperforms others signiﬁcantly in terms of both bias and test accuracy.
We investigated this strong performance and found that it is due to the speciﬁc form of randomization used by the proposed algorithm.
As shown in Figure 4, the post-processing algorithm of Hardt et al.
(2016) uses a ﬁxed probability when randomizing between two thresholds.
For
[Table]
[Figure]
CelebA trained from scratch, for example, the post-processing rule of Hardt et al.
(2016) predicts nearly uniformly at random when ResNet50 predicts the negative class for males.
In contrast, our algorithm uses a ramp function that takes the conﬁdence of the scores into account so that most of the instances whose labels are altered by the post-processing rule are close to the decision boundary of the original classiﬁer.
The best accuracy of the proposed algorithm is achieved with γ = 0 .(Figure 3).
We remark that the value of γ is always chosen based on a separate validation dataset.
Lastly, one important observation we note in Table 2 is the impact of pre-training – pretraining in our experiments helps in achieving a lower test error rate even after eliminating bias.
In other words, pretraining reduces the cost of debiasing trained models.
C ONCLUDING R EMARKS
In this paper, we propose a near-optimal post-processing algorithm for debiasing trained machine learning models.
The proposed algorithm is scalable, does not require retraining the classiﬁers, and has a limited impact on the test accuracy.
In addition to providing strong theoretical guarantees, we show that it outperforms previous post-processing methods for unbiased classiﬁcation on standard benchmarks across classical and modern machine learning models.
A F ULL ALGORITHM
[Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Equation]
B P ROOF OF P ROPOSITION
Proof.
Fix
< β < 1 and consider the subset:
[Equation]
and its complement ¯ W = X \ W .
Since f ( x ) ∈ { 0 , 1 } , the sets W and ¯ W are independent of β as long as it remains in the open interval (0 , 1) .
More precisely:
[Equation]
Now
[Equation]
for any set X ⊆ X
[Equation]
let p X be the projection of the probability measure p ( x ) on the set X
(i
[Equation]
e. p X ( x ) = p ( x ) /p
( X ) )
[Equation]
Then, with a simple algebraic manipulation
[Equation]
one has the identity:
E
[Equation]
By deﬁnition of W
[Equation]
we have:
[Equation]
Combining this with Eq.
[Equation]
we have:
[Equation]
Since the set W does not change when β is varied in the open interval (0 , 1) , the lower bound holds for any value of β ∈ (0 , 1) .
W set:
[Equation] [Equation]
Substituting the last equation into Eq.
(8) gives the lower bound:
[Equation]
Repeating the same analysis for the subset ¯ W , we arrive at the inequality:
[Equation]
Writing π ( x ) = 1 W ( x ) , we have by the reverse triangle inequality:
[Equation]
Finally:
[Equation]
Similarly, we have 2(1 − ¯ f ) ≥ 1 − E [ f ] .
Therefore:
[Equation]
Combining this with Eq.
(12) establishes the statement of the proposition.
C P ROOF OF T HEOREMC. C ONSTRAINED C ONVEX F ORMULATION
Suppose we have a binary classiﬁer on the instance space X .
We would like to construct an algorithm for post-processing the predictions made by that classiﬁer such that we control the bias with respect to a sensitive attribute 1 S :
X → { 0 , 1 } across a ﬁxed set of pairwise disjoint groups X 1 , . . .
, X K ⊆
X .
We assume that the output of the classiﬁer f : X → [ − 1 , +1] is an estimate to 2 η ( x ) − 1 , where η ( x )
= p ( y = 1 | x = x ) is the Bayes regressor.
This is not a strong assumption because many algorithms can be calibrated to provide probability scores (Platt et al., 1999; Guo et al., 2017) so the assumption is valid.
We consider randomized rules of the form:
[Equation]
whose arguments are: (1) the instance x ∈ X , (2) the group X k , (3) the sensitive attribute 1 S ( x ) , and (4) the original classiﬁer’s score f ( x ) .
Because randomization is sometimes necessary as proved in
Section 4, ˜ h
( x ) is the probability of predicting the positive class when the instance is x ∈ X .
Suppose we have a training sample of size N , which we will denote by S .
Let q i = ˜
h
( x i ) ∈ [0 , 1] for the i -th instance in S .
For each group X k ⊆ S , the fairness constraint in Deﬁnition 1 over the training sample can be written as:
[Equation]
This is because:
[Equation]
where the expectation is over the training sample.
To learn ˜ f , we propose solving the following regularized optimization problem:
[Equation]
where γ > 0 is a regularization parameter and (cid:15) k
= | X k | (cid:15) .
C. R EDUCTION TO U NCONSTRAINED O PTIMIZATION
Because the groups X k are pairwise disjoint
[Equation]
the optimization problems in (13) decomposes into K separate suboptimization problems, one for each group X k .
Each sub-optimization problem can be written in the following general form:
[Equation]
To recall
[Equation]
(cid:15) (cid:48) =
M(cid:15) .
The Lagrangian is:
[Equation]
q [Equation] α, β, λ, µ ) = (cid:88)
[Equation]
Taking the derivative w
[Equation]
r
[Equation]
t. q i gives us:
[Equation]
Plugging this back, the dual problem becomes:
[Equation]
Next, we eliminate variables.
By eliminating β i , we have:
[Equation]
Equivalently:
[Equation]
Next, we eliminate α i to obtain:
[Equation]
Finally
[Equation]
let’s eliminate the q i variables.
For a given optimal µ and λ , it is straightforward to observe that the minimizer q (cid:63) to γ/ 2 q 2 +[ w − γq ] + must lie in the set { 0 , w/γ, 1 } .
In particular, if w/γ ≤ 0 , then q (cid:63) = 0 .
If w/γ ≥ 1 , then q (cid:63) = 1 .
Note here that we make use of the fact that γ > 0 .
So, the optimal value of q (cid:63) to γ/ 2 q 2 + [ w − γq ]
+ is:
[Equation]
From this, the optimization problem reduces to:
[Equation]
This is a differentiable objective function and can be solved quickly using the projected gradient descent method (Boyd and Mutapcic, 2008).
The projection step here is is taking the positive parts of λ and µ .
This leads to the update rules in Algorithm 2.
What about the solution?
Given λ and µ , the solution of q i is a minimizer to;
[Equation]
As stated earlier, the solution is:
[Equation]
So, we have a ramp function.
This proves Theorem 1.
D P ROOF OF T HEOREMD. O PTIMAL U NBIASED P REDICTORS
We begin by proving the following result, which can be of independent interest.
Theorem 3.
Let f (cid:63) =
arg min f : X→{ 0 , 1 } E
[ I { f ( x ) (cid:54) = y } ] be the Bayes optimal decision rule subject to group-wise afﬁne constraints of the form E [ w k ( x ) · f ( x ) | x ∈ X k ]
=
b k for some ﬁxed partition
X = ∪ k X k .
If w k : X → R and b k ∈ R are such that there exists a constant c ∈ (0 , 1) in which p ( f ( x ) = 1) =
c will satisfy all the afﬁne constraints, then f (cid:63) satisﬁes p ( f (cid:63) ( x ) = 1)
=
I { η ( x ) > t k } + τ k I { η ( x ) =
t k } , where η ( x ) =
p ( y = 1 | x = x ) is the Bayes regressor, t k ∈ [0 , 1] is a threshold speciﬁc to the group
X k ⊆ X , and τ k ∈ [0 , 1] .
Proof.
Minimizing the expected misclassiﬁcation error rate of a classiﬁer f is equivalent to maximizing:
[Equation]
Hence, selecting f that minimizes the misclassiﬁcation error rate is equivalent to maximizing:
[Equation]
Instead of maximizing this directly, we consider the regularized form ﬁrst.
Writing g ( x ) = 2 η ( x ) − 1 , the optimization problem is:
[Equation]
Here, we focused on one subset X k because the optimization problem decomposes into K separate optimization problems, one for each X k .
If there exists a constant c ∈ (0 , 1) such that f ( x ) = c satisﬁes all the equality constraints, then Slater’s condition holds so strong duality holds (Boyd and
Vandenberghe, 2004).
The Lagrangian is:
[Equation]
where α ( x ) , β ( x ) ≥ 0 and µ ∈ R are the dual variables.
Taking the derivative w.r.t.
the optimization variable f ( x ) yields:
[Equation]
[Equation]
Therefore, the dual problem becomes:
[Equation]
We use the substitution in Eq.
(17) to rewrite it as:
[Equation]
Next, we eliminate the multiplier β ( x ) by replacing the equality constraint with an inequality:
[Equation]
Finally
[Equation]
since α ( x ) ≥
and α ( x ) ≥ g ( x ) − γf ( x ) − µw ( x )
[Equation]
the optimal solution is the minimizer to:
[Equation]
Next
[Equation]
let µ (cid:63) be the optimal solution of the dual variable µ .
Then, the optimization problem over f decomposes into separate problems, one for each x ∈ X .
We have:
[Equation]
Using the same argument in Appendix C
[Equation]
we deduce that f ( x ) is of the form:
[Equation]
otherwise Finally, the statement of the theorem holds by taking the limit as γ → 0 + .
D. E XCESS R ISK B OUND
Observe that the decision rule depends on x only via f ( x ) ∈ [ − 1 , +1] .
Hence, we write z = f ( x ) .
Since the thresholds are learned based on a fresh sample of data, the random variables z i are i.i.d.
In light of Eq.
16, we would like to minimize the expectation of the loss l (˜ h γ , x ) = − f ( x ) · ˜ h γ ( x ) = − z · q ( z ) .
=
ζ ( z ) for some function q : [ − 1 , +1] → [0 , 1] of the form shown in 2(a).
Note that ζ is 2(1 + 1 /γ ) -Lipschitz continuous within the same group and sensitive class.
This is because the thresholds are always in the interval [ − 1 − γ, 1 + γ ] ; otherwise moving beyond this interval would not change the decision rule. Let ˜ h
γ be the decision rule learned by the algorithm.
Using Corollary 5 in (Xu and Mannor, 2010), we conclude that with a probability of at least 1 − δ :
[Equation]
Here, we used the fact that the observations f ( x ) are bounded in the domain [ − 1 , 1] and that we can ﬁrst partition the domain into groups X k with/without the sensitive class S ( 2 K subsets) in addition to partitioning the interval [ − 1 , 1] into R smaller sub-intervals and using the Lipschitz constant.
Choosing R = N 13 and simplifying gives us with a probability of at least 1 − δ :
[Equation]
N The same bound also applies to the decision rule h
(cid:63)γ that results from applying optimal threshold with width γ >
(here, “optimal” is with respect to the underlying distribution) because the (cid:15) -cover
(Deﬁnition 1 in (Xu and Mannor, 2010)) is independent of the choice of the thresholds.
By the union bound, we have with a probability of at least 1 − δ , both of the following inequalities hold:
[Equation]
In particular:
[Equation]
N
The ﬁrst inequality follows from Eq.
The second inequality follows from the fact that ˜ h γ is an empirical risk minimizer to the regularized loss, where E [ ˜ f
( x ) 2 ] ≤ 1 since ˜ f
( x ) ∈ [0 , 1] .
The last inequality follows from Eq.
Finally, we know that the thresholding rule h
(cid:63)γ with width γ > 0 is, by deﬁnition, a minimizer to:
[Equation]
among all possible bounded functions h : X → [0 , 1] subject to the desired fairness constraints.
Therefore, we have:
[Equation]
Hence:
[Equation]
This implies the desired bound:
[Equation]
Therefore, we have consistency if N → ∞ , γ → 0 + and γN 13 → ∞ .
For example, this holds if γ = O ( N − 16 ) .
So far, we have assumed that the output of the original classiﬁer coincides with the Bayes regressor.
If the original classiﬁer is Bayes consistent, i.e. E [ | 2 η ( x ) − 1 − f ( x ) | ] → 0 as N → ∞ , then we have Bayes consistency of the post-processing rule by the triangle inequality.
E P ROOF OF P ROPOSITION
Proof.
Since | z ( x ) | ≤ 1 and | ξ (cid:48) γ ( w ) | ≤ 1 , the gradient at a point x is always bounded by ||∇ t || 22 ≤ (1 + (cid:15)[-
 + b-] ) 2 at all rounds.
Following the proof steps of (Boyd and Mutapcic, 2008) and using the fact that projections are contraction mappings, one obtains:
[Equation]
By Jensen’s inequality, we have 1 T (cid:80)
Tt =1 E [ F ( t ) ] ≤ E [ F (¯ µ )] .
Plugging this into the earlier results yields:
[Equation]
F E XTENSION TO N ON - BINARY S ENSITIVE A TTRIBUTES
To extend the deﬁnition of statistical parity to non-binary sensitive attributes, suppose we have groups S 1 , . .
, S K ⊆ X , which form a partition of the instance space.
The sensitive attribute is a group membership indicator s ( x ) :
X → { 1 , . . .
, K } , where s ( x ) =
k if and only if x ∈ S k .
Again, we would like the predictions of the algorithm to be independent of the sensitive attribute.
This can be achieved by requiring that:
[Equation]
Let ρ ∈ [0 , 1] be some hyperparameter (to be discussed shortly).
One can achieve the above desired fairness constraint by solving, again, the regularized optimization problem:
[Equation]
[Equation]
which decomposes into K separate suboptimization problems, one for each group S k .
Note that the optimization problem in (22) is of the same general form analyzed earlier in Appendix
C.2, where z i = 1 and b = ρ .
Therefore, the same algorithm can be applied by minimizing the objective function in Equation 14 using the projected SGD.
This produces the solutions { ( λ k , µ k ) } k ∈ [ K ] .
Then, the prediction (post-processing) rule is given by Equation 15.
Here, the hyperparameter ρ can be tuned using a validation dataset.
However, since the constraint forces the average of the post-processed predictions within each group S k to be close to ρ , one reasonable choice of ρ is to set it equal to ρ =
(1 / 2)(1 + E
[ f ( x )]) .