A
N EAR -O PTIMAL
A LGORITHM
FOR
D EBIASING T RAINED M
ACHINE L
EARNING M ODELS
Anonymous authors
Paper under double-blind review
A BSTRACT
We present an efﬁcient and scalable algorithm for debiasing trained models, including deep neural networks (DNNs), which we prove to be near-optimal by bounding its excess Bayes risk.
Unlike previous black-box reduction methods to cost-sensitive classiﬁcation rules, the proposed algorithm operates on models that have been trained without having to retrain the model.
Furthermore, as the algorithm is based on projected stochastic gradient descent (SGD), it is particularly attractive for deep learning applications.
We empirically validate the proposed algorithm on standard benchmark datasets across both classical algorithms and modern DNN architectures and demonstrate that it outperforms previous postprocessing approaches for unbiased classiﬁcation
I NTRODUCTION
Machine learning is increasingly applied to critical decisions which can have a lasting impact on individual lives, such as for credit lending (Bruckner, 2018), medical applications (Deo, 2015), and criminal justice (Brennan et al., 2009).
Consequently, it is imperative to understand and improve the degree of bias of such automated decision-making.
Unfortunately, despite the fact that bias (or “fairness”) is a central concept in our society today, it is difﬁcult to deﬁne it in precise terms.
In fact, as people perceive ethical matters differently depending on a plethora of factors including geographical location or culture (Awad et al., 2018), no universally-agreed upon deﬁnition for bias exists.
Moreover, the deﬁnition of bias may depend on the application and might even be ignored in favor of accuracy when the stakes are high, such as in medical diagnosis (Kleinberg et al., 2017; Ingold and Soper, 2016).
As such, it is not surprising that several deﬁnitions of “unbiased classiﬁcation” have been introduced.
These include statistical parity (Dwork et al., 2012; Zafar et al., 2017a), equality of opportunity (Hardt et al., 2016), and equalized odds (Hardt et al., 2016; Kleinberg et al., 2017).
Unfortunately, such deﬁnitions are not generally compatible (Chouldechova, 2017) and some might even be in conﬂict with calibration (Kleinberg et al., 2017).
In addition, because fairness is a societal concept, it does not necessarily translate into a statistical criteria (Chouldechova, 2017; Dixon et al., 2018).
Statistical parity Let X be an instance space and let Y = { 0 , 1 } be the target set in a standard binary classiﬁcation problem.
In the fair classiﬁcation setting, we may further assume the existence of a (possibly randomized) sensitive attribute s : X → { 0 , 1 , . .
, K } , where s ( x ) = k if and only if x ∈ X k for some total partition X = ∪ k X k .
For example, X might correspond to the set of job applicants while s indicates their gender.
Here, the sensitive attribute can be randomized if, for instance, the gender of an applicant is not a deterministic function of the full instance x ∈ X (e.g. number of publications, years of experience, ...etc).
Then, a commonly used criterion for fairness is to require similar mean outcomes across the sensitive attribute.
This property is well-captured through the notion of statistical parity (a.k.a. demographic parity) (Corbett-Davies et al., 2017;
Dwork et al., 2012; Zafar et al., 2017a; Mehrabi et al., 2019):
Deﬁnition 1 (Statistical Parity) .
Let X be an instance space and X = ∪ k X k be a total partition of
X .
A classiﬁer f : X → { 0 , 1 } satisﬁes (cid:15) statistical parity across all groups X 1 , . . . , X K if: [Equation] [Figure]
To motivate and further clarify the deﬁnition [Figure] we showcase the empirical results on the Adult benchmark dataset (Blake and Merz, 1998) in Figure 1.
When tasked with predicting whether the income of individuals is above $50K per year, all considered classiﬁers exhibit gender-related bias.
One way of removing such bias is to enforce statistical parity across genders.
Crucially, however, without taking ethnicity into account, different demographic groups may experience different outcomes.
In fact, gender bias can actually increase in some minority groups after enforcing statistical parity.
This can be ﬁxed by redeﬁning the sensitive attribute to be the cross product of both gender and ethnicity (green bars).
Our main contribution is to present a near-optimal recipe for debiasing models, including deep neural networks, according to Deﬁnition 1.
Speciﬁcally, we formulate the task of debiasing learned models as a regularized optimization problem that is solved efﬁciently using the projected SGD method.
We show how the algorithm produces thresholding rules with randomization near the thresholds, where the width of randomization is controlled by the regularization parameter.
We also show that randomization near the threshold is necessary for Bayes risk consistency.
While we focus on binary sensitive attributes in our experiments in Section 5, our algorithm and its theoretical guarantees continue to hold for non-binary sensitive attributes as well.
Statement of Contribution
We derive a near-optimal post-processing algorithm for debiasing learned models (Section 3).
We prove theoretical guarantees for the proposed algorithm, including a proof of correctness and an explicit bound on the Bayes excess risk (Section 4).
We empirically validate the proposed algorithm on benchmark datasets across both classical algorithms and modern DNN architectures.
Our experiments demonstrate that the proposed algorithm signiﬁcantly outperforms previous post-processing methods (Section 5).
In Appendix E, we also show how the proposed algorithm can be modiﬁed to handle other criteria of bias as well.
R ELATED W ORK
Algorithms for fair machine learning can be broadly classiﬁed into three groups: (1) pre-processing methods, (2) in-processing methods, and (3) post-processing methods (Zafar et al., 2019).
Preprocessing algorithms transform the data into a different representation such that any classiﬁer trained on it will not exhibit bias.
This includes methods for learning a fair representation (Zemel et al., 2013; Lum and Johndrow, 2016; Bolukbasi et al., 2016; Calmon et al., 2017; Madras et al., 2018; Kamiran and Calders, 2012), label manipulation (Kamiran and Calders, 2009), data augmentation (Dixon et al., 2018), or disentanglement (Locatello et al., 2019).
On the other hand, in-processing methods constrain the behavior of learning algorithms in order to control bias.
This includes methods based on adversarial learning (Zhang et al., 2018) and constraint-based classiﬁcation, such as by incorporating constrains on the decision margin (Zafar et al., 2019) or features (Grgi´c-Hlaˇca et al., 2018).
Agarwal et al.
(2018) showed that the task of learning an unbiased classiﬁer could be reduced to a sequence of cost-sensitive classiﬁcation problems, which could be applied to any black-box classiﬁer.
One caveat of the latter approach is that it requires solving a linear program (LP) and retraining classiﬁers, such as neural networks, many times before convergence.
The algorithm we propose in this paper is a post-processing method, which can be justiﬁed theoretically (Corbett-Davies et al., 2017; Hardt et al., 2016; Menon and Williamson, 2018; Celis et al., 2019).
Fish et al.
(2016) and Woodworth et al.
(2017) fall under this category.
However, the former only provides generalization guarantees without consistency results while the latter proposes a twostage approach that requires changes to the original training algorithm. Kamiran et al.
(2012) also proposes a post-processing algorithm, called Reject Option Classiﬁer (ROC), without providing any theoretical guarantees.
In contrast, our algorithm is Bayes consistent and does not alter the original classiﬁcation method.
In Celis et al.
and Menon and Williamson (2018), instance-dependent thresholding rules are also learned.
However, our algorithm also learns to randomize around the threshold (Figure 2(a)) and this randomization is key to our algorithm both theoretically as well as experimentally (Appendix C and Section 5).
Hardt et al.
(2016) learns a randomized post-processing rule but our proposed algorithm outperforms it in all of our experiments (Section 5).
Woodworth et al.
(2017) showed that the post-processing approach can, sometimes, be highly suboptimal.
Nevertheless, the latter result does not contradict the statement that our post-processing rule is near-optimal because we assume that the original classiﬁer outputs a monotone transformation of some approximation to the posterior probability p ( y = 1 | x ) (e.g. margin or softmax output) whereas Woodworth et al.
(2017) assumed in their construction that the post-processing rule had access to the binary predictions only.
We argue that the proposed algorithm has distinct advantages, particularly for deep neural networks (DNNs).
First, stochastic convex optimization methods are well-understood and can scale well to massive amounts of data (Bottou, 2010), which is often the case in deep learning today.
Second, the guarantees provided by our algorithm hold w.r.t.
the binary predictions instead of using a proxy, such as the margin as in some previous works (Zafar et al., 2017b; 2019).
Third, unlike previous reduction methods that would require retraining a deep neural network several times until convergence (Agarwal et al., 2018), which can be prohibitively expensive, our algorithm operates on learned models that are trained once and does not require retraining.
Besides developing algorithms for fair classiﬁcation, several recent works focused on other related aspects, such as proposing new deﬁnitions for fairness; e.g. demographic parity (Dwork et al., 2012; Mehrabi et al., 2019), equalized odds (Hardt et al., 2016), equality of opportunity/disparate mistreatment (Zafar et al., 2017a; Hardt et al., 2016), and individual fairness (Dwork et al., 2012).
Recent works have also established several impossibility results related to fair classiﬁcation, such as Kleinberg et al.
(2017); Chouldechova (2017).
In our case, we derive a new impossibility result that holds for any deterministic binary classiﬁer and relate it to the task of controlling the covariance between the classiﬁer’s predictions and the sensitive attribute (Appendix E).
N EAR - OPTIMAL ALGORITHM FOR S TATISTICAL P ARITY
Notation We reserve boldface letters for random variables (e.g. x ), small letters for instances (e.g. x ), capital letters for sets (e.g. X ), and calligraphic typeface for universal sets (e.g. the instance space X ).
Given a set S , 1 S ( x ) ∈ { 0 , 1 } is the characteristic function indicating whether x ∈ S .
We denote by [ n ] the set of integers { 1 , . . . , n } and [ x ] + = max { 0 , x } .
Algorithm Given a classiﬁer f : X → [ − 1 , +1] our goal is to post-process the predictions made by f 1 in order to control the bias with respect to a sensitive attribute s : X → [ K ] as in Deﬁnition 1.
To this end, instead of learning a deterministic classiﬁer, we consider randomized prediction rules of the form ˜ h :
[Equation]
×
[Equation]
[Equation]
[Equation]
× [ − [Equation] [Equation] → [0 , 1] , where ˜ h ( x ) represents the probability of predicting the positive class given (i) instance x ∈ X , (ii) sensitive attribute s ( x ) , and (iii) classiﬁer’s output f ( x ) .
As discussed in Appendix B, for post-processing rule ˜ h ( x ) , and for each group X k ⊆ X , the fairness constraint in Deﬁnition 1 can be written as | E x [˜ h ( x ) | x ∈ X k ] − ρ | ≤ (cid:15) , where ρ ∈ [0 , 1] is a hyper-parameter tuned via a validation dataset.
On the other hand, minimizing the probability of altering the predictions of the original classiﬁer can be achieved by maximizing the inner product E x [˜ h ( x ) · f ( x )] .
Instead of optimizing this quantity directly, which would lead to a pure thresholding rule, we minimize the regularized objective: ( γ/ 2)
E x [˜ h ( x ) 2 ] − E x
[˜ h ( x ) · f ( x )] for some regularization parameter γ > 0 .
This regularization leads to randomization around the threshold, which we show to be critical, both theoretically (Section 4 and Appendix C) and experimentally (Section 5).
Using Lagrange duality we show that the solution reduces to the update rules in Equation 2 with optimization variables { λ k , µ k } k ∈
[ K ] and the corresponding predictor which outputs +1 for group X k with probability ˜
h γ ( x ) is given by
[Equation]
where ξ γ is given by Eq.
Update rules To learn these parameters, one can apply the following update rules (Appendix B):
[Equation]
where, again, ρ ∈ [0 , 1] is a hyperparameter tuned via a validation dataset, s : X
→ [ K ] is the sensitive attribute, and γ > 0 is a regularization parameter that controls the level of randomization.
In addition, the function ξ γ : R → R + is given by:
[Equation]
Note that ξ γ is convex and its derivative ξ (cid:48)
γ is (1 /γ ) -Lipschitz continuous; it can be interpreted as differentiable approximation to the ReLU unit (Nair and Hinton, 2010).
A full pseudocode of the proposed algorithm is presented in Appendix A.
T HEORETICAL A NALYSIS
Next, we analyze the algorithm.
Our ﬁrst theoretical result is to show that the prediction rule in
Equation 1 learned through the update rules presented in Section 3 satisﬁes the desired fairness guarantees on the training sample.
Theorem 1 (Correctness) .
Let ˜ h γ : X → [0 , 1] be the randomized predictor in Equation 1 learned by applying the update rules in Equation 2 starting with µ k = 0 , λ k = 0 ,
∀ k ∈
[ K ] until convergence.
Then, ˜ h
γ satisﬁes (cid:15) statistical parity w.r.t.
{ X k } k ∈ [ K ] in the training sample.
The proof of Theorem 1 is presented in Appendix B.
The following guarantee, which holds w.r.t.
the underlying data distribution, shows that the randomized prediction rule converges to the Bayes -| 1f ( x ) γ ˜ h ( x ) | | --| [Figure]
[Figure]
[Figure]
[Figure]
Epoch λ 0 − u
[Figure]
optimal unbiased classiﬁer if the original classiﬁer f is Bayes consistent.
The proof of the following theorem (Appendix C) is based on the Lipschitz continuity of the decision rule when γ > 0 and the robustness framework of Xu and Mannor (2010).
Theorem 2.
Let h (cid:63) =
arg min h ∈H (cid:15) E [ h ( x ) (cid:54) =
y ] , where H (cid:15) is the set of binary predictors on X that satisfy fairness according to Deﬁnition 1 for (cid:15) > 0 .
Let ˜ h γ : X → [0 , 1] be the randomized learning rule in Equation 1.
If ˜ h γ is trained on a freshly sampled data of size N , then there exists a value of ρ ∈ [0 , 1] such that the following holds with a probability of at least 1 − δ :
[Equation]
2 log 2 δ N , where η ( x ) = p ( y = {+
+} | x = x ) is the Bayes regressor and K is the number of groups X k .
Consequently, if the original classiﬁer is Bayes consistent and we have: N → ∞ , γ → 0 + and γN − 13 → ∞ , then E [˜ h γ ( x ) (cid:54) =
y ] P −→ E [ h (cid:63) ( x ) (cid:54) =
y ] .
Hence, the updates converge to the optimal prediction rule subject to the chosen fairness constraint.
Running time As shown in Appendix B, the update rules in Equation 2 perform a projected stochastic gradient descent on the following optimization problem:
[Equation]
We assume with no loss of generality that f ( x ) ∈ [ − 1 , 1] since f ( x ) is assumed to be an estimator to 2 η ( x ) − 1 (see Section 3 and Appendix B) and any thresholding rule over f ( x ) can be transformed into an equivalent rule over a monotone increasing function of f (i.e. using the hyperbolic tangent).
Proposition 1.
Let µ (0) =
λ (0) =
and write µ ( t ) , λ ( t ) ∈ R K for the value of the optimization variables after t updates deﬁned in Equation 2 for some ﬁxed learning rate α t
= α .
Let ¯ µ = (1 /T )
(cid:80) Tt =1 µ ( t ) ( x ) and ¯ λ
= (1 /T )
(cid:80) Tt =
1 λ ( t ) ( x ) .
Then,
[Equation]
where ¯ F : R K × R K → R is the objective function in (4) using the averaged solution ¯ µ and ¯ λ while
F (cid:63) is its optimal value.
In particular, E [ ¯ F ] − F ( µ (cid:63) ) (cid:1) =
O ( (cid:112) K/T )
when α =
O
( (cid:112) K/T ) .
The proof is in Appendix D. Hence, the post-processing rule can be efﬁciently computed.
In practice, we observe fast convergence as shown in Figure 2(b).
As shown in Figure 2(a), the hyperparameter γ controls the width of randomization around the thresholds.
A large value of γ may reduce the accuracy of the classiﬁer.
On the other hand, γ [Table] [Table] cannot be zero because randomization around the threshold is, in general, necessary for Bayes risk consistency as illustrated in the following example:
Example 1 (Randomization is necessary) .
Suppose that X = {− 1 , 0 , 1 } where p ( x = − 1)
= 1 / 2 , p ( x = 0)
= 1 / 3 and p ( x = 1) = 1 / 6 .
Let η ( − 1) = 0 , η (0) = 1 / 2 and η (1) = 1 .
In addition, let s ∈ { 0 , 1 } be a sensitive attribute, where p ( s = 1 | x = − 1) = 1 / 2 , p ( s = 1 | x = 0)
= 1 , and p ( s = 1 | x = 1) = 0 .
Then, the Bayes optimal prediction rule f (cid:63) ( x ) subject to statistical parity ( (cid:15) = 0 )
[Table]
: p ( f (cid:63) ( x ) = 1 | x = − 1) = 0 , p ( f (cid:63) ( x ) = [-
-]1 | x = 0) = 7 / 10 and p ( f (cid:63) ( x ) = 1 | x = 1) = 1 .
Note that the Bayes excess risk bound in Theorem 2 is vacuous when γ = 0 .
Therefore, γ controls a trade-off depending on how crucial randomization is around the thresholds (e.g. in k -NN where the classiﬁer’s scores come from a ﬁnite set or in deep neural networks that tend to produce scores concentrated around {− 1 , +1 } ).
In our experiments, γ is always chosen using a validation set.
E MPIRICAL EVALUATION
Experiment Setup We compare against three post-processing methods: (1) the post-processing algorithm of Hardt et al.
(2016) (2) the shift inference method, ﬁrst introduced in (Saerens et al., 2002)
and used more recently in (Wang et al., 2020), and (3) the Reject Option Classiﬁer (ROC) (Kamiran et al., 2012).
We use the implementation of the algorithm of Hardt et al. (2016) in the FairLearn software package (Dudik et al., 2020). The training data used for the post-processing methods is always a fresh sample, i.e. different from the data used to train the original classiﬁers. The value of the hyper-parameter θ of the ROC algorithm is chosen in the grid { 0 . 01 , 0 .
When ROC fails, its solution with the minimum bias is reported.
In the proposed algorithm, the parameter γ is chosen in the grid { 0 . 01 , 0 . 02 , 0 . 05 , 0 . 1 , 0 . 2 , . . .
, 1 . 0 } while ρ is chosen in the gird E [ y ] ± { 0 , 0 . 05 , 0 . 1 } .
All hyper-parameters are selected based on a separate validation dataset.
Tabular Data We empirically evaluate the performance of the proposed algorithm and the baselines on two real-world datasets, namely the Adult income dataset and the Default of Credit Card
[Figure] Clients (DCCC) dataset, both taken from the UCI Machine Learning Repository (Blake and Merz, 1998).
The Adult dataset contains 48,842 records with 14 attributes each and the goal is to predict if the income of an individual exceeds $50K per year.
The DCCC dataset contains 30,000 records with 24 attributes [Figure] and the goal is to predict if a client will default on their credit card payment.
Both datasets include sensitive attributes, such as sex and age.
In Figure 1 we showcased why, in some cases, the sensitive attribute can be the cross product of multiple features (e [Figure]
g. religion, gender, and race).
In our experiments in this section, we deﬁne the sensitive class to be the class of females.
In the DCCC dataset, we additionally introduce bias in the training set for the purpose of the experiment: if s ( x ) =
y ( x ) we keep the instance and otherwise drop it with probability 0.5.
We train four classiﬁers on each dataset: (1) random forests with maximum depth 10, (2) k -NN with k = 10 , (3) a two-layer fully connected neural network with 128 hidden nodes, and (4) logistic regression.
For the latter, we ﬁne-tune the parameter C in a grid of values chosen in a logarithmic scale between 10 − 4 and 10 4 using 10-fold cross validation.
The learning rate in our algorithm is ﬁxed to 10 − 1 ( K/T ) 1 / 2 , where T is the number of steps, and (cid:15) = 0 .
Table 1 shows the bias and accuracy on test data after applying each post-processing method.
The column marked as “original” corresponds to the original classiﬁer without any alteration.
As shown in the table, both our proposed algorithm and the algorithm of Hardt et al.
(2016) eliminate bias in all classiﬁers.
By contrast, the shift-inference method does not succeed at controlling statistical parity while the ROC method can fail when the output of the original classiﬁer is discrete, such as in k NN, because it does not learn to randomize.
However, the proposed algorithm has a much lower impact on the test accuracy compared to Hardt et al. and can even improve it in certain cases.
The fact that fairness can sometimes improve accuracy was recently noted by Blum and Stangl (2020).
The full tradeoff curves between bias and performance are provided in Figure 3.
CelebA Dataset
Our second set of experiments builds on the task of predicting the “attractiveness” attribute in the CelebA dataset (Liu et al., 2015).
CelebA contains 202,599 images of celebrities annotated with 40 binary attributes, such as gender, attractiveness, and smiling.
We use two standard deep neural network architectures: ResNet50 (He et al., 2016) and MobileNet (Howard et al., 2017), either trained from scratch or pretrained on ImageNet.
We present the results in Table 2.
We observe that the proposed algorithm signiﬁcantly outperforms the post-processing algorithm of Hardt et al. (2016) and performs, at least, as well as the ROC algorithm whenever the latter algorithm succeeds. [Table]
[Figure] both subpopulations. The curves correspond to the randomized post-processing rules, i.e. p ( y = 1 | x ) , of Hardt al. and the proposed algorithm with γ = 0 .
1 and ρ = E [ y ] .
Often, however, ROC fails at debiasing the deep neural networks because it does not learn to randomize when most of the scores produced by deep neural networks are concentrated around the set {− 1 , +1 } (see for instance (Guo et al., 2017; Jiang et al., 2018)).
We investigated the strong performance compared to that of Hardt et al.
and found that it is due to the speciﬁc form of randomization used by the proposed algorithm.
As shown in Figure 4, the post-processing algorithm of Hardt et al.
(2016) uses a ﬁxed probability when randomizing between two thresholds.
For CelebA trained from scratch, for example, the post-processing rule of Hardt et al.
(2016) predicts nearly uniformly at random when ResNet50 predicts the negative class for males.
In contrast, our algorithm uses a ramp function that takes the conﬁdence of the scores into account.
In Figure 4, in particular, the male instances with scores close to -1 are ﬂipped with probability ≈ 0 .
15 , as opposed to ≈ 0 . 5 in Hardt et al. and this difference is compensated for by ﬂipping all examples with scores larger than ≈ − 0 . 9 and all female instances with scores less than ≈ 0 . Hence, less randomization is applied when the original classiﬁer is more conﬁdent.
Lastly, one important observation we note in Table 2 is the impact of pre-training – pretraining in our experiments helps in achieving a lower test error rate even after eliminating bias.
In other words, pretraining seems to reduce the cost of debiasing trained models.
C ONCLUDING R EMARKS
In this paper, we propose a near-optimal post-processing algorithm for debiasing trained machine learning models.
The proposed algorithm is scalable, does not require retraining the classiﬁers, and has a limited impact on the test accuracy.
In addition to providing strong theoretical guarantees, we show that it outperforms previous post-processing methods for unbiased classiﬁcation on standard benchmarks across classical and modern machine learning models.
A F ULL ALGORITHM
[Table]
[-Fix 
-]
[Equation]
B P ROOF OF T HEOREMB. C ONSTRAINED C ONVEX F ORMULATION
Suppose we have a binary classiﬁer on the instance space X .
We would like to construct an algorithm for post-processing the predictions made by that classiﬁer such that we control the bias with respect to a set of pairwise disjoint groups X 1 , . . .
, X K ⊆ X according to Deﬁnition 1.
We assume that the output of the classiﬁer f : X → [ − 1 , +1] is an estimate to 2 η ( x ) − 1 , where η ( x )
= p ( y = 1 | x = x ) is the Bayes regressor.
This is not a strong assumption because many algorithms can be calibrated to provide probability scores (Platt et al., 1999; Guo et al., 2017) so the assumption is valid.
We consider randomized rules of the form:
[Equation]
whose arguments are: (1) the instance x ∈ X , (2) the sensitive attribute s ( x ) ∈ [ K ] , and (3) the original classiﬁer’s score f ( x ) .
Because randomization is sometimes necessary as proved in Section 4, ˜ h
( x ) is the probability of predicting the positive class when the instance is x ∈ X .
Suppose we have a training sample of size N , which we will denote by S .
Let q i =
˜
h
( x i ) ∈ [0 , 1] for the i -th instance in S .
For each group X k ⊆ S , the fairness constraint in Deﬁnition 1 over the training sample can be written as:
[Equation]
for some hyper-parameter ρ > 0 . This holds by the triangle inequality.
To learn ˜ f , we propose solving the following regularized optimization problem:
[Equation]
k (6) where γ > 0 is a regularization parameter and (cid:15) k
= | X k | (cid:15)/ 2 .
B. R EDUCTION TO U NCONSTRAINED O PTIMIZATION
Because the groups X k are pairwise disjoint, the optimization problems in (6) decomposes into K separate suboptimization problems, one for each group X k .
Each sub-optimization problem can be written in the following general form:
[Equation]
To recall, (cid:15) (cid:48) =
M(cid:15)/ 2 .
The Lagrangian is:
L
[Equation]
q, α, β, λ, µ ) =
[Equation]
i
[Equation]
Taking the derivative w.r.t.
q i gives us:
[Equation]
Plugging this back, the dual problem becomes:
[Equation]
Next, we eliminate variables.
By eliminating β i , we have:
[Equation]
Equivalently:
[Equation]
Next [Equation] we eliminate α i to obtain:
[Equation]
Finally
[Equation]
let’s eliminate the q i variables.
For a given optimal µ and λ [Equation] it is straightforward to observe that the minimizer q (cid:63) to γ/ 2 q 2 +[ w − γq ] + must lie in the set { 0 , w/γ, 1 } .
In particular, if w/γ ≤ 0 , then q (cid:63) = 0 .
If w/γ ≥ 1 , then q (cid:63) = 1 .
Note here that we make use of the fact that γ > 0 .
So, the optimal value of q (cid:63) to γ/ 2 q 2 + [ w − γq ]
+ is:
[Equation]
From this [Equation] the optimization problem reduces to:
[Equation]
This is a differentiable objective function and can be solved quickly using the projected gradient descent method (Boyd and Mutapcic, 2008).
The projection step here is taking the positive parts of λ and µ .
This leads to the update rules in Algorithm 1.
What about the solution?
Given λ and µ , the solution of q i is a minimizer to;
[Equation]
As stated earlier, the solution is:
[Equation]
So, we have a ramp function.
In the proposed algorithm, we have z i
= 1 and b = ρ for all examples.
This proves Theorem 1.
C P ROOF OF T HEOREMC. O PTIMAL U NBIASED P REDICTORS
We begin by proving the following result, which can be of independent interest.
Theorem 3.
Let f (cid:63) =
arg min f : X→{ 0 , 1 } E
[ I { f ( x ) (cid:54) = y } ] be the Bayes optimal decision rule subject to group-wise afﬁne constraints of the form E [ w k ( x ) · f ( x ) | x ∈ X k ]
=
b k for some ﬁxed partition
X = ∪ k X k .
If w k : X → R and b k ∈ R are such that there exists a constant c ∈ (0 , 1) in which p ( f ( x ) = 1) =
c will satisfy all the afﬁne constraints, then f (cid:63) satisﬁes p ( f (cid:63) ( x ) = 1)
=
I { η ( x ) > t k } + τ k I { η ( x ) =
t k } , where η ( x ) =
p ( y = 1 | x = x ) is the Bayes regressor, t k ∈ [0 , 1] is a threshold speciﬁc to the group
X k ⊆ X , and τ k ∈ [0 , 1] .
Proof.
Minimizing the expected misclassiﬁcation error rate of a classiﬁer f is equivalent to maximizing:
[Equation]
Hence, selecting f that minimizes the misclassiﬁcation error rate is equivalent to maximizing:
[Equation]
Instead of maximizing this directly, we consider the regularized form ﬁrst.
Writing g ( x ) = 2 η ( x ) − 1 , the optimization problem is:
[Equation]
Here, we focused on one subset X k because the optimization problem decomposes into K separate optimization problems, one for each X k .
If there exists a constant c ∈ (0 , 1) such that f ( x ) = c satisﬁes all the equality constraints, then Slater’s condition holds so strong duality holds (Boyd and
Vandenberghe, 2004).
The Lagrangian is:
[Equation]
where α ( x ) , β ( x ) ≥ 0 and µ ∈ R are the dual variables.
Taking the derivative w.r.t.
the optimization variable f ( x ) yields:
[Equation]
Therefore, the dual problem becomes:
[Equation]
We use the substitution in Eq.
(10) to rewrite it as:
[Equation]
Next, we eliminate the multiplier β ( x ) by replacing the equality constraint with an inequality:
[Equation]
Finally, since α ( x ) ≥ [-
-]0 and α ( x ) ≥ g ( x ) − γf ( x ) − µw ( x ) , the optimal solution is the minimizer to:
[Equation]
Next, let µ (cid:63) be the optimal solution of the dual variable µ .
Then, the optimization problem over f decomposes into separate problems, one for each x ∈ X .
We have:
[Equation]
Using the same argument in Appendix B
[Equation]
we deduce that f ( x ) is of the form:
[Equation]
Finally, the statement of the theorem holds by taking the limit as γ → 0 + .
C. E XCESS R ISK B OUND
In this section, we write D to denote the underlying probability distribution and write S to denote the uniform distribution over the training sample (a.k.a. empirical distribution).
The parameter ρ stated in the theorem is given by:
[Equation]
Note that, by deﬁnition, the optimal classiﬁer h (cid:63) that satisﬁes (cid:15) statistical parity also satisﬁes the constraint in (6) with this choice of ρ .
Hence, with this choice of ρ , h (cid:63) remains optimal among all possible classiﬁers.
Observe that the decision rule depends on x only via f ( x ) ∈ [ − 1 , +1] .
Hence, we write z = f ( x ) .
Since the thresholds are learned based on a fresh sample of data, the random variables z i are i.i.d.
In light of Eq.
9, we would like to minimize the expectation of the loss l (˜ h γ , x ) =
− f ( x ) · ˜ h
γ ( x ) = − z · q ( z ) .
=
ζ ( z ) for some function q : [ − 1 , +1] → [0 , 1] of the form shown in 2(a).
Note that ζ is 2(1 + 1 /γ ) -Lipschitz continuous within the same group and sensitive class.
This is because the thresholds are always in the interval [ − 1 − γ, 1 + γ ] ; otherwise moving beyond this interval would not change the decision rule.
Let ˜ h
γ be the decision rule learned by the algorithm.
Using Corollary 5 in (Xu and Mannor, 2010), we conclude that with a probability of at least 1 − δ :
[Equation]
Here, we used the fact that the observations f ( x ) are bounded in the domain [ − 1 , 1] and that we can ﬁrst partition the domain into groups X k ( K subsets) in addition to partitioning the interval [ − 1 , 1] into R smaller sub-intervals and using the Lipschitz constant.
Choosing R = N 13 and simplifying gives us with a probability of at least 1 − δ :
[Equation]
N
The same bound also applies to the decision rule h
(cid:63)γ that results from applying optimal threshold with width γ >
(here, “optimal” is with respect to the underlying distribution) because the (cid:15) -cover
(Deﬁnition 1 in (Xu and Mannor, 2010)) is independent of the choice of the thresholds.
By the union bound, we have with a probability of at least 1 − δ , both of the following inequalities hold:
[Equation]
In particular:
[Equation]
The ﬁrst inequality follows from Eq.
The second inequality follows from the fact that ˜ h γ is an empirical risk minimizer to the regularized loss, where E [ ˜ f
( x ) 2 ] ≤ 1 since ˜ f
( x ) ∈ [0 , 1] .
The last inequality follows from Eq.
Finally, we know that the thresholding rule h
(cid:63)γ with width γ > 0 is, by deﬁnition, a minimizer to:
[Equation]
among all possible bounded functions h : X → [0 , 1] subject to the desired fairness constraints.
Therefore, we have:
[Equation]
Hence:
[Equation]
This implies the desired bound:
[Equation]
N Therefore, we have consistency if N → ∞ , γ → 0 + and γN 1 3 → ∞ .
For example, this holds if γ = O ( N − 16 ) .
So far, we have assumed that the output of the original classiﬁer coincides with the Bayes regressor.
If the original classiﬁer is Bayes consistent, i.e. E [ | 2 η ( x ) − 1 − f ( x ) | ] → 0 as N → ∞ , then we have Bayes consistency of the post-processing rule by the triangle inequality.
D P ROOF OF P ROPOSITION
Proof.
Since | ξ (cid:48) γ ( w ) | ≤ 1 , the gradient at a point x during SGD has a square (cid:96) 2 -norm bounded by || (1 + ρ + (cid:15) ) 2 at all rounds.
Following the proof steps of (Boyd and Mutapcic, 2008) and using the fact that projections are contraction mappings, one obtains:
[Equation]
By Jensen’s inequality, we have 1 T (cid:80)
Tt =1 E [ F ( t ) ] ≤ E [ F (¯ µ )] .
Plugging this into the earlier results yields:
[Equation]
E E XTENSION TO O THER C RITERIA E. C ONTROLLING THE C OVARIANCE
The proposed algorithm can, sometimes, be adjusted to control bias according to other criteria as well besides statistical parity. For example, we demonstrate in this section how the proposed postprocessing algorithm can be adjusted to control the covariance between the classiﬁer’s prediction and the sensitive attribute when both are binary random variables. Let a , b , c ∈ { 0 , 1 } be random variables. Let C ( a , b ) .
= E [ a · b ] − E [ a ] · E [ b ] be their covariance, and C ( a , b | c ) their covariance conditioned on c : [Equation] Then, one possible criterion for measuring bias is to measure the conditional/unconditional covariance between the classiﬁer’s predictions and the sensitive attribute when both are binary random variables.
Because the random variables are binary, it is straightforward to show that achieving zero covariance implies independence. Suppose we have a binary classiﬁer on the instance space X . We would like to construct an algorithm for post-processing the predictions made by that classiﬁer such that we guarantee | C (cid:0) f ( x ) , 1 S ( x ) | x ∈ X k (cid:1) | ≤ (cid:15) , where X = ∪ k X k is a total partition of the instance space.
Informally, this states that the fairness guarantee with respect to the
senstiive attribute 1 S : X → { 0 , } holds within each subgroup X k . We assume, again, that the output of the classiﬁer f : X → [ − 1 , +1] is an estimate to 2 η ( x ) − 1 , where η ( x ) = p ( y = 1 | x = x ) is the Bayes regressor and consider randomized rules of the form:
[Equation] whose arguments are: (i) the instance x ∈ X , (ii) the sensitive attribute 1 S : X → { 0 , 1 } , (iii) the sub-group membership k : X → [ K ] , and (iv) the original classiﬁer’s score f ( x ) . Because randomization is sometimes necessary as proved in Section 4, ˜ h
( x ) is the probability of predicting the positive class when the instance is x ∈ X . Suppose we have a training sample of size N , which we will denote by S . Let q i =
˜ h ( x i ) ∈ [0 , 1] for the i -th instance in S . For each group X k ⊆ S , the desired fairness constraint on the covariance can be written as:
[Equation]
where ρ k = E x [1 S ( x ) | x ∈ X k ] . This is because: [Equation]
where the expectation is over the training sample. Therefore, in order to learn ˜ h , we solve the regularized optimization problem:
[Equation]
{+where γ > 0 is a regularization parameter and (cid:15) k = | X k | (cid:15) . This is of the same general form analyzed in Section B.2. Hence, the same algorithm can be applied with b = 
+}
and z i = 1 S ( i ) − ρ k . E. I MPOSSIBILITY R ESULT The previous algorithm for controlling covariance requires that the subgroups X k be known in advance.
Indeed, our next impossibility result shows that this is, in general, necessary.
In other words, a deterministic classiﬁer f : X → { 0 , 1 } cannot be universally unbiased with respect to a sensitive class S across all possible known and unknown groups unless the representation x has zero mutual information with the sensitive attribute or if f is constant almost everywhere. As a corollary, the groups X k have to be known in advance .
Proposition 2 (Impossibility result) . Let X be the instance space and Y = { 0 , 1 } be a target set. Let 1 S : X → { 0 , 1 } be an arbitrary (possibly randomized) binary-valued function on X and deﬁne γ : X → [0 , 1] by γ ( x )
= p (1 S ( x ) = 1 | x = x ) , where the probability is evaluated over the randomness of 1 S : X → { 0 , 1 } . Write ¯ γ = E x
[ γ ( x )] . Then, for any binary predictor f : X → { 0 , 1 } it holds that [Equation] where C (cid:0) f ( x ) , γ ( x ) | π ( x ) (cid:1) is deﬁned in Equation 14.
Proof.
{+Fix 
 < β < 1 and consider+} the subset: [Equation] and its complement ¯ W = X \ W .
Since f ( x ) ∈ { 0 , 1 } , the sets W and ¯ W are independent of β as long as it remains in the open interval (0 , 1) .
More precisely:
[Equation]
Now, for any set X ⊆ X , let p
X
be the projection of the probability measure p ( x ) on the set X (i.e. p X ( x ) = p ( x ) /p ( X ) ). Then, with a simple algebraic manipulation, one has the identity:
[Equation]
By deﬁnition of W , we have:
[Equation] Combining this with Eq. (17), we have: [Equation]
Since the set W does not change when β is varied in the open interval (0 , 1) , the lower bound holds for any value of β ∈ (0 , 1) .
W set:
[Equation]
Substituting the last equation into Eq.
(18) gives the lower bound:
[Equation]
Repeating the same analysis for the subset ¯ W
[Equation]
we arrive at the inequality:
[Equation]
Writing π ( x ) = 1 W ( x )
[Equation]
we have by the reverse triangle inequality:
[Equation] Finally: [Equation] Similarly, we have 2(1 − ¯ f ) ≥ 1 − E [ f ] .
Therefore:
[Equation]
Combining this with Eq.
(22) establishes the statement of the proposition.