Group-Aware Threshold Adaptation for Fair Classification
Year: 2021

Abstract:
The fairness in machine learning is getting increasing attention, as its applications 1 in different fields continue to expand and diversify. To mitigate the discriminated 2 model behaviors between different demographic groups, we introduce a novel post3 processing method to optimize over multiple fairness constraints through group4 aware threshold adaptation. We propose to learn adaptive classification thresholds 5 for each demographic group by optimizing the confusion matrix estimated from 6 the probability distribution of a classification model output. As we only need 7 an estimated probability distribution of model output instead of the classification 8 model structure, our post-processing model can be applied to a wide range of 9 classification models and improve fairness in a model-agnostic manner to ensure 10 privacy. This even allows us to post-process existing fairness methods to further 11 improve the trade-off between accuracy and fairness. Moreover, our model is 12 efficient with low computational cost by alternating optimization and flexible with 13 the optimization over multiple fairness constraints. We provide Pareto frontier to 14 characterize fairness-accuracy trade-off. Also, we provide a theoretical analysis 15 of the optimal thresholds obtained from our model in terms of both accuracy 16 and fairness in classification. Experimental results demonstrate that our method 17 outperforms state-of-the-art methods and obtains the result that is closest to the 18 theoretical accuracy-fairness trade-off boundary. 19

text: The fairness in machine learning is getting increasing attention, as its applications1 in different fields continue to expand and diversify. To mitigate the discriminated2 model behaviors between different demographic groups, we introduce a novel post-3 processing method to optimize over multiple fairness constraints through group-4 aware threshold adaptation. We propose to learn adaptive classification thresholds5 for each demographic group by optimizing the confusion matrix estimated from6 the probability distribution of a classification model output. As we only need7 an estimated probability distribution of model output instead of the classification8 model structure, our post-processing model can be applied to a wide range of9 classification models and improve fairness in a model-agnostic manner to ensure10 privacy. This even allows us to post-process existing fairness methods to further11 improve the trade-off between accuracy and fairness. Moreover, our model is12 efficient with low computational cost by alternating optimization and flexible with13 the optimization over multiple fairness constraints. We provide Pareto frontier to14 characterize fairness-accuracy trade-off. Also, we provide a theoretical analysis15 of the optimal thresholds obtained from our model in terms of both accuracy16 and fairness in classification. Experimental results demonstrate that our method17 outperforms state-of-the-art methods and obtains the result that is closest to the18 theoretical accuracy-fairness trade-off boundary.19
1 Introduction20
Machine learning is broadening its impact in various fields including autonomous driving, credit21 analysis, and job application screening. As a consequence, the role and importance of fairness in22 machine learning are emerging. However, recent models have been found to behave differently23 between demographic groups in favorable predictions. For example, it has been discovered that24 COMPAS, the criminal risk assessment software currently used to help pretrial release decisions,25 has biases between different races [4]. Specifically, blacks got higher risk scores predicted from the26 model than whites with similar profiles. Therefore, discrimination truly exists and resolving it in27 machine learning is very important and urgent because its direct and potential impact is growing28 tremendously.29
However, obtaining fairness is not a trivial problem, because the data set itself will be biased when it30 is accumulated artificially. Simply removing or manipulating sensitive features (such as race, gender)31 from the data does not solve the bias, because there is indirect discrimination [19] or disparate32 treatment [1] due to the feature redundancy and relevance, which means sensitive information can be33 inferred from other features.34
In order to alleviate discrimination from different perspectives, various quantitative measurements35 of group equity [7, 11, 2, 13] have been proposed. It has been proven that the pursuit of fairness is36
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.
subject to a trade-off between fairness and accuracy [14, 10], i.e., if we want to improve fairness, we37 need to sacrifice accuracy.38
Moreover, Pleiss et al. [20] studied the trade-offs between fairness notions that cannot be satisfied39 at the same time. Therefore, recent works usually target at a certain fairness notion in different40 approaches such as pre-processing [6], in-processing [24], and post-processing [7] methods. However,41 these approaches suffer from the lack of flexibility, since it is difficult to adapt a fair model that42 is trained w.r.t. one certain fairness criterion so as to optimize over other fairness measures. If43 the fairness constraints change under some circumstances, traditional fairness models need to be44 re-trained from scratch, which is computationally demanding and sometimes inapplicable due to45 model settings. To overcome the limitations above, we propose a novel post-processing method to46 improve fairness in a model-agnostic manner. Our GSTAR (Group Specific Threshold Adaptation47 for faiR classification) model learns adaptive classification thresholds for each demographic group48 in classification task for improving the trade-off between fairness and accuracy. Given an existing49 classification model, GSTAR approximates the probability distribution of the model output via50 maximum likelihood estimation and utilizes confusion matrix to quantify accuracy and fairness w.r.t.51 the group-aware classification thresholds. This allows us to: 1) prevent from burdening additional52 complexity or deteriorate the stability of the training process of the classifier; 2) integrate different53 fairness notions into one unified objective function; 3) easily adapt one pre-trained model to other54 fairness constraints. We summarize our contributions of this paper as follows:55
1. We propose a novel post-processing method, GSTAR, which can learn group-aware thresh-56 olds to optimize the trade-off between fairness and accuracy in classification. We derive57 rigorous theoretical analysis on the trade-off in our model, and empirically show that GSTAR58 outperforms state-of-the-art methods.59
2. With GSTAR, we can simultaneously optimize over multiple fairness constraints with a low60 computational cost. GSTAR does not require multiple iterations over data, instead, it takes61 at most one pass of data in training for fast computation.62
3. GSTAR can be adapted to a wide range of classification models in a model-agnostic manner63 and can adapt an existing classification model from one fairness criterion to another without64 re-training the classifier.65
4. We derive Pareto frontiers of our model for the fairness-accuracy trade-offs that contextualize66 the quality of fair classification.67
2 Related Works68
In order to achieve group fairness, which quantifies the discrimination among different sensitive69 groups, a diverse notion of fairness has been introduced. Equalized odds [7] enforce equality of true70 positive rates and false positive rates between different demographic groups. Pleiss et al. [20] relaxed71 equalized odds to satisfy the calibration. Demographic parity or disparate impact [1] suggests that a72 model is unbiased if the model prediction is independent of the protected attribute.73
Among different fairness methods, post-processing techniques propose to improve fairness by mod-74 ifying the output of a black-box classifier. Hardt et al. [7] propose to ensure equalized odds by75 constraining the model output. Kamiran et al. [9] propose to give a favorable outcome to unprivileged76 and an unfavorable outcome to the privileged group when the confidence of the prediction is beyond a77 certain range. However, such static confidence window keeps the same regardless of the demographic78 group and is determined by grid search, so it is less efficient.79
Threshold adjustment (a.k.a. thresholding) was introduced to improve the performance of static80 thresholds. In the literature, Menon et al. [18] prove that instance-dependent thresholding of the81 predictive probability function is the optimal classifier in cost-sensitive fairness measures. Also,82 when considering immediate utility, Corbett-Davies et al. [3] show that optimal algorithm is achieved83 from group-specific threshold which is determined by group statistics. However, to the best of our84 knowledge, the threshold adjustment approach has not been deeply studied that neither encompasses85 broad group fairness metrics nor describes an explicit method to achieve the threshold.86
Trade-off between fairness and accuracy exists when we impose fairness constraint to a model. Recent87 studies [2, 25] prove that models targeting at such fairness notions conform to an information theoretic88
lower bound on the joint error across different sensitive groups. Therefore, our work presents a89 practical upper bound of the best achievable accuracy given the fairness constraints.90
Moreover, trade-offs between different fairness notions also exist if one has to consider multiple91 fairness criteria. Some of them are theoretically proven to be incompatible [6, 18, 14]. To express92 and formulate fairness, recent work [10] utilize confusion matrix and propose least-square accuracy-93 fairness optimization problem on multiple fairness notions, and categorize the trade-offs between the94 fairness notions.95
Here, our work is the most related to the post-processing methods [7, 10]. Hardt et al. [7] propose96 a post-processing method that utilizes the mixing rate to meet the equalized odds. Ours is similar97 to Hardt et al. [7] in the manner that achieving group-wise threshold from the feasible region that98 is geometrically generated by the intersection between the receiver operating characteristic (ROC)99 curves conditioned on sensitive feature. Ours differ from [7] by generalizing the concept beyond100 equalized odds to other multiple fairness constraints into consideration. FACT [10] utilizes a single101 point (static) from the classifier to be post-processed as a reference which does not fully utilize the102 classifier for the post-processing. In contrast, by approximating the distribution of the continuous103 predicted logits, our GSTAR model enables a larger feasible region than [10] with a better fairness-104 accuracy trade-off. We validate the improvement in trade-off via both theoretical and empical results.105 It is notable that these related methods [7, 10] can be considered as a special case of GSTAR.106
3 GSTAR for Fair Classification107
3.1 Motivation108
Consider a binary classification problem with a binary sensitive feature, such that the sensitive feature109 A ∈ {0, 1} and label Y ∈ {0, 1}. In general, for a given data X , a binary classification model110 outputs an unnormalized logit h(X) ∈ R with the class label probability p(X) = σ(h(X)) ∈ [0, 1],111 where σ is an activation function (sigmoid function in logistic regression and neural network). It is112 not necessary to calculate p in a classification model, e.g. support vector machines directly use the113 positiveness/negativeness of logit h(X) to determine classification outcome. For traditional models,114 we use a cut-off threshold θh = 0 for h(X) (i.e., θp = σ(0) = 0.5 for p(X)) in classification, such115 that the predicted label is determined by Ŷ = I{h(X) ≥ θh}. In the following context, unless116 otherwise mentioned, we use θ to refer to the threshold θh on logit h since it is applicable to a117 wider range of classification models, and the corresponding threshold on label probability θp can be118 easily inferred from the threshold on logit h. Traditional models use the same cut-off threshold θ for119 different demographic groups. However, since the distribution of logits h in different demographic120 groups can be different, using the same threshold θ brings biased classification.121
In Figure 1, we show a real-world example of image classification on CelebA dataset with122 ResNet50 [8] to show that the default setting of classification thresholds affects both accuracy123 and fairness in classification. The goal of this classification example is to predict the image of a124 person is whether attractive or not, and consider sensitive attribute as gender. This can be generalized125 to different sensitive attributes such as age or race [22, 16]. We can observe an obvious difference in126 the distribution of logit h between two gender groups. In this case, if we use a unified classification127 threshold θ1 = θ0 = 0, it naturally brings a difference in the true positive rate and true negative128 rate between two gender groups, thus renders bias in classification. Instead, we observe that the129 optimal group-specific threshold obtained from GSTAR (θ∗1 > θ1, and θ ∗ 0 < θ0) can adapt to such130 discrepancy in distribution between two demographic groups to improve both fairness and accuracy.131
3.2 Group-Aware Classification Thresholds132
Given an existing classification model and a sensitive attribute a, we can denote true positive rate133 (TPa), false positive rate (FPa), true negative rate (TNa), and false negative rate (FNa) in the confusion134 matrix. Most fairness notions can be represented with entries in the confusion matrix. For instance,135 Equal Opportunity (EOp) [7] requires TP0 = TP1, and Demographic Parity (DP) [1] requires136
TP1n11 + FP1n01 N1 = TP0n10 + FP0n00 N0 ,
where nya denotes the number of samples in the subset {Y = y,A = a}, Na = ∑ y nya denotes the137
number of samples in {Y = y}, and N = ∑ y,a nya is the total number of samples.138
Consider the group-aware classification threshold θ = (θ1, θ0)T, where θa is the classification139 threshold for sensitive group A = a. We can formulate the entries in the confusion matrix w.r.t. θ as140 below:141
TPa(θa) ≈ 1− ∫ θa −∞ f1a(x)dx, FNa(θa) ≈ ∫ θa −∞ f1a(x)dx,
FPa(θa) ≈ 1− ∫ θa −∞ f0a(x)dx, TNa(θa) ≈ ∫ θa −∞ f0a(x)dx,
(1)
where fya(x) is an estimated parametric probability density function of the distribution of output142 logit h in the subset {Y = y,A = a}. Here, we consider gamma, Student’s t, and normal distribution143 as the candidates for the estimated distribution, and select the one that has the maximum likelihood144 with the output distribution. Without loss of generality, this can be generalized with other parametric145 probability density function based on the needs or prior knowledge.146
Then, we formulate the fairness-constrained classification problem with the objective of minimizing147 classification error into a least-squared optimization problem. We denote our objective function148 as L(θ) which consists of the performance loss Lper(θ) and fairness loss Lfair(θ). Lper(θ) and149 Lfair(θ) measures the error in performance and fairness respectively that are represented with the150 entries of the confusion matrix. In other words, our goal is to minimize the objective function L(θ)151 as below:152
L(θ) = Lper(θ) + λLfair(θ), (2) where λ is a hyperparameter that determines how much fairness is enforced in the optimization.153
The performance error Lper(θ) can be written as154 Lper(θ) = (n01 N FP1(θ1) + n11 N FN1(θ1) + n00 N FP0(θ0) + n10 N FN0(θ0) )2 . (3)
As forLfair(θ), it can be formulated to any fairness metrics that are expressible with confusion matrix.155 For instance, when we impose EOp (TP1 = TP0) and predictive equality (PE) (FP1 = FP0) [2], we156 can get the corresponding Lfair(θ) by summing over the least squared form of each constraint. Also,157 satisfying EOp and PP is equivalent to satisfying Equalized Odds (EOd) [7], This can be formulated158 in our Lfair as159
LEOdfair (θ) = L EOp fair (θ) + L PP fair(θ) = ( TP1(θ1)− TP0(θ0) )2 + ( FP1(θ1)− FP0(θ0) )2 .
(4)
Note that a lower Lfair value indicates a fairer threshold. When LEODfair (θ) = 0, we can interpret as160 the θ satisfies the perfect EOd fairness.161
Similar to (4), we can enforce multiple fairness constraints by summing over the least squared of162 each metric with different weight constant λ to each fairness constraints if needed.163
Also, it is notable that compared to the recent paper [10] that enforces fairness through confusion164 tensor, our formulation of fairness in Lfair(θ) represents a direct notion of fairness metrics and165 improves the measures that allows us to achieve better performance and Pareto frontiers that is shown166 in Section 4.2 and Figure 2. For example,AEOd in the paper is calculated as M1EOp +M0PE, where167 My = ny0 +ny1, such that EOd is a weighted sum of EOp and PE with weights being the number of168 samples in each class. In this expression, the imbalance between the two fairness criteria will grow as169 the degree of imbalance in the data increases. In contrast, our formulation expresses the constraints170 as the exact notion of each metric that is not biased by the statistics of the datset and we observe171 improved Pareto frontier as in Figure 2.172
We propose to optimize our threshold θ with alternating optimization method. Here we take EOp173 constraint as an example to show the alternating optimization steps, then Lfair(θ) can be written as174
LEOpfair (θ) = (TP1(θ1)− TP0(θ0)) 2 . (5)
The first step is to fix θ0 and update θ1. We can approximate the terms that are related to θ1 (e.g.,175 TP1,FP1,TN1,FN1) in (1) with first-order Taylor expansion at θτ−11 . For example,176
TP1(θ1) ≈ TP1(θτ−11 ) + ∂TP1 ∂θ1 ∣∣∣ θ1=θ τ−1 1 (θ1 − θτ−11 ) (6)
From (1), we can easily derive that177 TP1(θτ−11 ) = 1− ∫ θτ−11 −∞ f11(x)dx,
∂TP1 ∂θ1
= − f11(θτ−11 ). (7)
Similarly, we can find the first order Taylor expansion of FP1,FN1, and TN1. Then, the update of θ1178 w.r.t. (2) can be approximated with the following minimization problem w.r.t. ∆1179
∆τ1 := argmin ∆1 (ητ + ατ∆1) 2 + λ( τ + βτ∆1) 2, (8)
where ∆1 = θ1 − θτ−11 and180
ατ1 = n11 N f11(θ τ−1 1 )− n01 N f01(θ τ−1 1 ), βτ1 =− f11(θτ−11 ),
ητ1 = ∫ θτ−11 −∞ (n11 N f11(x) + n01 N (1− f01(x) ) dx+ ∫ θτ−10 −∞ (n10 N f10(x) + n00 N (1− f00(x) ) dx,
τ1 = ∫ θτ−11 ∞ f11(x)dx− ∫ θτ−10 ∞ f01(x)dx.
(9)
Taking the derivative of (8) w.r.t. ∆1 and setting it to 0, we can easily obtain the closed-form solution181 of ∆τ1 as182
∆τ1 = − ατητ + λβτ τ
(ατ )2 + λ(βτ )2 . (10)
Algorithm 1 Optimization Algorithm of GSTAR Model Input dataset X ×A× Y = {(xi, ai, yi)}ni=1, classification model h(X), hyperparameter λ. Output Group-specific threshold θ = (θ1, θ0). Initialize θ = (θ1, θ0) = (0, 0). 1. Given a classifier H(x), estimate probability density function fya, y, a ∈ {0, 1} by maximum likelihood estimation. while not converge do
2. Calculate the optimal step ∆1 as ∆1 = −α1η1+λβ1 1α21+λβ21 , with α1, β1, η1, 1 values shown in (9); 3. Update the threshold: θ1 ← θ1 + ∆1; 4. Calculate the optimal step ∆0 as ∆0 = −α0η0+λβ0 0α20+λβ20 with α0, β0, η0, 0 values calculated in a similar way as in (9): 5. Update the threshold:θ0 ← θ0 + ∆0.
end while
The second step is to fix θ1 and update θ0, and this can be achieved in a similar way of updating θ1.183 Then we can finalize the alternating optimization as:184
θτ0 = θ τ−1 0 + ∆ τ 0 , θτ1 = θ τ−1 1 + ∆ τ 1 .
(11)
It is notable that in each iteration we derive the optimal update step θ, which eliminates the burden185 of tuning hyperparameter (such as learning rate) in iterative algorithm. The optimization step is186 summarized in Algorithm 1. The above algorithm can easily extend to multiple fairness constraints187 by adding corresponding squared-loss fairness terms to (2).188
Time Complexity: The alternating optimization of GSTAR model is of low computational cost. We189 take at most one pass of the data for learning the estimated probability density functions fya in (1)190 (we do not even need to traverse the data if the parameters (such mean and variance in Gaussian191 distribution) for the estimated probability density functions fya can be provided). The optimization192 of θ with alternating optimization is efficient since we only need fya as we have seen in (9) and (10).193 θ ∈ R2 is a vector with fixed small size. Therefore, we need a constant time for each update. Overall,194 the time complexity of GSTAR is O(n+ T ), where n is the number of samples, and T is the number195 of iterations in alternating optimization.196
We further derive the theoretical analysis of our GSTAR model on the balance between fairness197 and accuracy, which indicates that the optimal solution provides guarantees on model accuracy198 under the optimal fairness constraint. Details of the theoretical analysis is in the Supplementary199 material. Besides, if a unified threshold is necessary [3], i.e., θ1 = θ0, the optimization algorithm200 also applies and we only have one scalar variable in (2). When we have a unified threshold, we do not201 require sensitive information in the testing phase that we can conform more strict privacy regulations202 than group-aware thresholding. However, we have to sacrifice both fairness and accuracy as the203 thresholding is less flexible.204
4 Experiments205
In this section, we validate GSTAR model on four well-known fairness datasets and compare with206 other state-of-the-art methods. First, we plot Pareto frontiers of ours and FACT (MS) [10] to207 demonstrate the trade-offs between fairness and accuracy. Second, we evaluate the models with208 different fairness metrics and validate that our model is highly adaptive to any fairness metrics that209 are expressible with confusion matrix [7, 11, 2, 1]. Third, we use our model as a post-processing210 method to existing fair models and show that our model further improves existing fair models in an211 efficient and model-agnostic manner.212
4.1 Experimental Setup213
We compare with multiple fairness approaches in the experiments. For clear demonstration of results,214 we use different shapes of marker for each comparing methods in Figure 2 and Figure 4. The compar-215 ing methods include: Learning fair representations for kernel models (abbreviated as FGP) [23],216
Fairness confusion tensor (abbreviated as FACT) [10], Disparate impact remover (abbreviated217 as DIR) [6], Adversarial de-biasing (abbreviated as AdvDeb) [24], Calibrated equalized odds218 post-processing (abbreviated as CEOPost) [20], Equality of opportunity in supervised learning219 (abbreviated as Odds) [7], Learning adversarially fair and transferable representations (abbrevi-220 ated as LAFTR) [17], and Baseline: For CelebA dataset, we use ResNet50 [8] as a reference, and221 logistic regression for all other datasets. Our method is optimized with λ in the range of [10−1, 104]222 with alternating optimization method. All experiments are implemented with Pytorch framework on223 i9-9960X CPU and a Quadro RTX 6000 GPU.224
We choose broadly used fairness metrics in evaluation including: equal opportunity difference and225 equalized odds difference (abbreviated as EOp, and EOd respectively) [7] ; 1-disparate impact226 (abbreviated as 1-DIMP) [1]; balanced accuracy difference (abbreviated as BD).227
We evaluate the methods on four fairness datasets: CelebA image dataset1 [15], Adult dataset from228 the UCI repository [12], COMPAS2 (Correctional Offender Management Profiling for Alternative229 Sanctions) dataset, and German credit dataset from the UCI repository [5]. All data is split as 70%230 for training and 30% for testing. More details of the comparing methods, evaluation metrics, and231 datasets are provided in the Supplementary material.232
4.2 Performance and Fairness-Accuracy Trade-Offs233
In this subsection, we look into the performance evaluation of GSTAR comparing with other state-of-234 the-art methods. We consider Pareto frontier to visualize the trade-offs between fairness and accuracy235 to demonstrate the measure of performance.236
In Figure 2, we plot Pareto frontier, which is the upper bound for the accuracy-fairness trade-offs,237 desired output locates at the upper right region under the boundary which corresponds to higher238 values in accuracy and lower values in fairness discrepancy. With the same fairness constraints239 are given, we achieve a better frontier than the FACT [10] as we equally weigh on demographic240 statistics and have a better feasible region. To obtain our results (star points), we first estimate the241 logit distribution from the output of the baseline model, and then we get optimal adaptive thresholds242 with corresponding fairness metric by updating w.r.t. the objective function in (2). Here we have three243 combinations of fairness imposed to GSTAR: demographic parity (DP), equalized odds (EOd), and244
1http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html 2https://github.com/propublica/compas-analysis
Basel ine AdvD eb Eq.Od ds CEOP ost LAFTR FAC T GSTA R(DP) GSTA R(EO d)
GSTA R(DP+
EOd)
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
Fa irn
es s
Fair
Bias
0.65
0.68
0.70
0.72
0.75
0.78
0.80
0.82
0.85

heading: Perform ance
text: Bad
Good EOp EOd BD 1-DIMP BA ACC
Basel ine AdvD eb Eq.Od ds CEOP ost LAFTR FAC T GSTA R(DP) GSTA R(EO d)
GSTA R(DP+
EOd)
0.10
0.20
0.30
0.40
0.50
0.60
0.70
Fa irn
es s
Fair
Bias
0.72
0.74
0.76
0.78
0.80
0.82
0.84
0.86
Perform ance
Bad
Good EOp EOd BD 1-DIMP BA ACC

heading: Perform ance
text: with both constraints (DP+EOd). By post-processing on a simple baseline, we achieved significantly245 better fairness with small or no sacrifice in accuracy. In all datasets, GSATR got competitive or better246 results than other state-of-the-art methods on both fairness and accuracy.247
For example, we got θ∗EOd = (0.640,−0.627)T for the CelebA dataset. This shows that we have248 a higher threshold for the privileged group and a lower threshold for the unprivileged group. This249 optimal thresholding from GSTAR allows more samples from the privileged group to be correctly250 predicted as unattractive that would compensate for the discrimination of the original model. In other251 words, this improves predictive equality [2] with a huge amount from 0.235 to 0.014. Also, true252 positive rate difference (also known as equality of opportunity [7]) got reduced from 0.282 to 0.018.253 It is notable that GSTAR only sacrificed 2.2% of accuracy to bring the big improvement in fairness.254
Since the objective function of our model is independent to data dimensionality, our model is much255 more efficient especially for high dimensional data. We mostly outperform the computational cost256 comparing to the other methods. The comparison of computational time on the datasets can be found257 in the Supplementary material.258
4.3 Flexibility and Multiple Fairness Constraints259
Since each fairness metric has different interests, it has been theoretically proven that they cannot be260 perfectly satisfied all together [20, 2, 11]. Because of this inherent trade-offs between fairness metrics,261 most of the recent works focus on a single metric at a time to achieve fairness. However with GSTAR,262 we have the flexibility to optimize on multiple fairness constraints that can be represented in the263 confusion matrix format. Moreover, given the estimated distribution fya of a black-box classification264 model, we can adjust the optimal θ based on the needs by accommodating different fairness criteria.265
Figure 3 demonstrates the result of the methods with fairness metrics and accuracy trade-off eval-266 uations. Overall, the variations of GSTAR achieve the best fairness on each target fairness while267 preserving the performance. For example in Figure 3(a), GSTAR with EOd constraint has outstanding268 performance in most fairness metrics with comparable accuracy (80.3%). Comparing with GSTAR269 (EOd), when we introduce EOd and DP together (DP+EOd), we achieve significantly better w.r.t. DP270 fairness with sacrificing a small amount of accuracy and EOd.271
In general, by sacrificing individual fairness performance, we could introduce multiple constraints.272 Also, we implicitly observe that the more fairness constraints are introduced, the more accuracy is273 sacrificed. We empirically found that in some cases (e.g. Figure 3(c)), introducing multiple fairness274 is complementary to each other that improves both conditions.275
4.4 Post-Processing on an Existing Fair Model276
For a binary classifier that has a single fixed classification threshold (0 for out logit, and 0.5 for277 label probability), we can improve the trade-off between fairness and accuracy via GSTAR post-278 processing. Given the logit/probability of the dataset from a black-box model, we can improve the279 fairness as illustrated in Figure 4. In most cases, we observe improvement in fairness after GSTAR280 post-processing. It is also interesting to note that by optimizing the different thresholds for each281 protected group, we even obtain better performance on both fairness and accuracy, which indicates282 that the threshold optimization can not only improve fairness but also accuracy.283
However, when the distribution of the logits/probability is highly extreme (such as the results of using284 GSTAR to post-process CEOPost), it is difficult to estimate the distribution and thus causes erroneous285 optimization in GSTAR. We empirically found that when the dataset is extremely imbalanced such286 that we do not have enough samples to estimate the logit/probability distribution, or black-box model287 is too certain to the prediction that samples are concentrated to certain output, this problem arises.288
5 Conclusion and Discussion289
In this paper, we propose a group-aware threshold adaptation method (GSTAR) to post-process a black-290 box model and optimize over multiple fairness constraints.We directly optimize the classification291 threshold for each demographic group w.r.t. the classification error and multiple fairness constraints292 in a unified objective function, such that we can practically achieve an optimal trade-off between293 accuracy and fairness in fair classification. Our method is applicable to diverse notions of group294 fairness as the majority of fairness notions can be expressed as a linear or quadratic equation through295 confusion matrix. We empirically show that GSTAR is flexible with fairness regularization, efficient296 with low computational cost. We also notice that the adaptive thresholds benefit accuracy in some297 cases. GSTAR agrees to protect privacy such as article 17 of EU’s GDPR [21] with model-agnostic298 post-processing. We only require the estimated distribution of the output from a black-box model299 i.e., our post-processing method is oblivious to features. Thus training data is no longer needed and300 allowed to be discarded after training the black-box model.301
Further, we empirically find that GSTAR is not applicable to post-process some classification models302 in the following situations: 1) the model does not provide logit/probability as the outcome; 2) The303 model provides an extreme distribution of the output logit/probability. For example, when the model304 is too certain about its prediction, it will be difficult to perform probability density estimation. In our305 future work, we will study possible strategies to solve the above limitations, and extend GSTAR to306 multi-class, multi-sensitive group problems and improve the fairness-accuracy trade-off in a more307 general scheme.308
References309 [1] Solon Barocas and Andrew D Selbst. Big data’s disparate impact. Calif. L. Rev., 104:671, 2016.310
[2] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism311 prediction instruments. Big data, 5(2):153–163, 2017.312
[3] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic313 decision making and the cost of fairness. In KDD, pages 797–806, 2017.314
[4] Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism. Sci.315 Adv, 4(1):eaao5580, 2018.316
[5] Dheeru Dua and Casey Graff. UCI machine learning repository, 2019.317
[6] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata-318 subramanian. Certifying and removing disparate impact. In KDD, pages 259–268, 2015.319
[7] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In320 NeurIPS, pages 3315–3323, 2016.321
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image322 recognition. In CVPR, pages 770–778, 2016.323
[9] Faisal Kamiran, Asim Karim, and Xiangliang Zhang. Decision theory for discrimination-aware324 classification. In ICDM, pages 924–929. IEEE, 2012.325
[10] Joon Sik Kim, Jiahao Chen, and Ameet Talwalkar. Model-agnostic characterization of fairness326 trade-offs. arXiv preprint arXiv:2004.03424, 2020.327
[11] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair328 determination of risk scores. arXiv preprint arXiv:1609.05807, 2016.329
[12] Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In330 KDD, volume 96, pages 202–207, 1996.331
[13] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of332 fair machine learning. arXiv preprint arXiv:1803.04383, 2018.333
[14] Lydia T Liu, Max Simchowitz, and Moritz Hardt. The implicit fairness criterion of unconstrained334 learning. In ICML, pages 4051–4060, 2019.335
[15] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the336 wild. In ICCV, pages 3730–3738, 2015.337
[16] Vishnu Suresh Lokhande, Aditya Kumar Akash, Sathya N Ravi, and Vikas Singh. Fairalm:338 Augmented lagrangian method for training fair models with little regret. In ECCV, pages339 365–381. Springer, 2020.340
[17] David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair341 and transferable representations. arXiv preprint arXiv:1802.06309, 2018.342
[18] Aditya Krishna Menon and Robert C Williamson. The cost of fairness in binary classification.343 In ACM FAccT, pages 107–118, 2018.344
[19] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In345 KDD, pages 560–568, 2008.346
[20] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness347 and calibration. In NeurIPS, pages 5680–5689, 2017.348
[21] General Data Protection Regulation. Regulation eu 2016/679 of the european parliament and of349 the council of 27 april 2016. Official Journal of the European Union. Available at: http://ec.350 europa. eu/justice/data-protection/reform/files/regulation_oj_en. pdf (accessed 20 September351 2017), 2016.352
[22] Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. Inclusivefacenet: Improving face353 attribute detection with race and gender diversity. arXiv preprint arXiv:1712.00193, 2017.354
[23] Zilong Tan, Samuel Yeom, Matt Fredrikson, and Ameet Talwalkar. Learning fair representations355 for kernel models. In AISTATS, pages 155–166, 2020.356
[24] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with357 adversarial learning. In AIES, pages 335–340, 2018.358
[25] Han Zhao and Geoff Gordon. Inherent tradeoffs in learning fair representations. In NeurIPS,359 pages 15675–15685, 2019.360
Checklist361 1. For all authors...362
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s363 contributions and scope? [Yes]364
(b) Did you describe the limitations of your work? [Yes] See Section 5.365 (c) Did you discuss any potential negative societal impacts of your work? [Yes] See366 Section 5.367 (d) Have you read the ethics review guidelines and ensured that your paper conforms to368 them? [Yes]369 2. If you are including theoretical results...370
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 1371 in the Supplementary material.372
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 1 in the373 Supplementary material.374
3. If you ran experiments...375 (a) Did you include the code, data, and instructions needed to reproduce the main ex-376 perimental results (either in the supplemental material or as a URL)? [Yes] We will377 provide the code and instructions at request. The data used in the experiments are378 public available. See Section 2.3 in the Supplementary material.379
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they380 were chosen)? [Yes] See Section 4.1.381
(c) Did you report error bars (e.g., with respect to the random seed after running experi-382 ments multiple times)? [Yes] See Section 4.3.383
(d) Did you include the total amount of compute and the type of resources used (e.g., type384 of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4.1.385
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...386 (a) If your work uses existing assets, did you cite the creators? [N/A]387 (b) Did you mention the license of the assets? [N/A]388 (c) Did you include any new assets either in the supplemental material or as a URL? [N/A]389
390
(d) Did you discuss whether and how consent was obtained from people whose data you’re391 using/curating? [N/A]392
(e) Did you discuss whether the data you are using/curating contains personally identifiable393 information or offensive content? [N/A]394
5. If you used crowdsourcing or conducted research with human subjects...395 (a) Did you include the full text of instructions given to participants and screenshots, if396 applicable? [N/A]397 (b) Did you describe any potential participant risks, with links to Institutional Review398 Board (IRB) approvals, if applicable? [N/A]399 (c) Did you include the estimated hourly wage paid to participants and the total amount400 spent on participant compensation? [N/A]401

Review summaries:
This paper proposes a threshold modification method to post-process the trained classifiers. The threshold learning is formulated as an optimization task and an efficient technique is proposed to learn the thresholds for groups. The proposed adaptive threshold learning is able to work with multiple fairness constraints.

The paper proposes a post-processing approach to achieve different notions of fairness. To do this, the paper first approximates the probability distribution of the output model. Group-specific confusion matrices can be derived by thresholding this distribution. The entries of such a matrix can be used to compute accuracy and various fairness metrics. The authors experimentally evaluate their approach on several datasets.

This paper introduces a method for finding fair and optimal thresholds for binary classification. It considers the mean squared error as the objective function. Instead of solving a constrained optimization problem, they add a  fairness loss to the objective function.   Then using Taylor approximation,  the optimization problem is reduced to an optimization function with a quadratic utility. The proposed method has low complexity ($\mathcal{O}(n+T)$). Moreover, it improves the fairness accuracy tradeoff as compared to existing works. 

The authors propose a new post-processing group-aware threshold search procedure for fair classification. The procedure is efficient, amenable to different types of underlying classification models, and is empirically shown to be working with multiple fairness criteria. The authors also derive  Pareto frontier of their model for the fairness-accuracy trade-off. Lastly, the authors conclude with experiments on four datasets showing the efficacy of their method.