Group-Aware Threshold Adaptation for Fair Classification

The fairness in machine learning is getting increasing attention , as its applications1 in different fields continue to expand and diversify . To mitigate the discriminated2 model behaviors between different demographic groups , we introduce a novel post-3 processing method to optimize over multiple fairness constraints through group-4 aware threshold adaptation . We propose to learn adaptive classification thresholds5 for each demographic group by optimizing the confusion matrix estimated from6 the probability distribution of a classification model output . As we only need7 an estimated probability distribution of model output instead of the classification8 model structure , our post-processing model can be applied to a wide range of9 classification models and improve fairness in a model-agnostic manner to ensure10 privacy . This even allows us to post-process existing fairness methods to further11 improve the trade-off between accuracy and fairness . Moreover , our model is12 efficient with low computational cost by alternating optimization and flexible with13 the optimization over multiple fairness constraints . We provide Pareto frontier to14 characterize fairness-accuracy trade-off . Also , we provide a theoretical analysis15 of the optimal thresholds obtained from our model in terms of both accuracy16 and fairness in classification . Experimental results demonstrate that our method17 outperforms state-of-the-art methods and obtains the result that is closest to the18 theoretical accuracy-fairness trade-off boundary.19 1 Introduction20 Machine learning is broadening its impact in various fields including autonomous driving , credit21 analysis , and job application screening . As a consequence , the role and importance of fairness in22 machine learning are emerging . However , recent models have been found to behave differently23 between demographic groups in favorable predictions . For example , it has been discovered that24 COMPAS , the criminal risk assessment software currently used to help pretrial release decisions,25 has biases between different races [ 4 ] . Specifically , blacks got higher risk scores predicted from the26 model than whites with similar profiles . Therefore , discrimination truly exists and resolving it in27 machine learning is very important and urgent because its direct and potential impact is growing28 tremendously.29 However , obtaining fairness is not a trivial problem , because the data set itself will be biased when it30 is accumulated artificially . Simply removing or manipulating sensitive features ( such as race , gender ) 31 from the data does not solve the bias , because there is indirect discrimination [ 19 ] or disparate32 treatment [ 1 ] due to the feature redundancy and relevance , which means sensitive information can be33 inferred from other features.34 In order to alleviate discrimination from different perspectives , various quantitative measurements35 of group equity [ 7 , 11 , 2 , 13 ] have been proposed . It has been proven that the pursuit of fairness is36 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . subject to a trade-off between fairness and accuracy [ 14 , 10 ] , i.e. , if we want to improve fairness , we37 need to sacrifice accuracy.38 Moreover , Pleiss et al . [ 20 ] studied the trade-offs between fairness notions that can not be satisfied39 at the same time . Therefore , recent works usually target at a certain fairness notion in different40 approaches such as pre-processing [ 6 ] , in-processing [ 24 ] , and post-processing [ 7 ] methods . However,41 these approaches suffer from the lack of flexibility , since it is difficult to adapt a fair model that42 is trained w.r.t . one certain fairness criterion so as to optimize over other fairness measures . If43 the fairness constraints change under some circumstances , traditional fairness models need to be44 re-trained from scratch , which is computationally demanding and sometimes inapplicable due to45 model settings . To overcome the limitations above , we propose a novel post-processing method to46 improve fairness in a model-agnostic manner . Our GSTAR ( Group Specific Threshold Adaptation47 for faiR classification ) model learns adaptive classification thresholds for each demographic group48 in classification task for improving the trade-off between fairness and accuracy . Given an existing49 classification model , GSTAR approximates the probability distribution of the model output via50 maximum likelihood estimation and utilizes confusion matrix to quantify accuracy and fairness w.r.t.51 the group-aware classification thresholds . This allows us to : 1 ) prevent from burdening additional52 complexity or deteriorate the stability of the training process of the classifier ; 2 ) integrate different53 fairness notions into one unified objective function ; 3 ) easily adapt one pre-trained model to other54 fairness constraints . We summarize our contributions of this paper as follows:55 1 . We propose a novel post-processing method , GSTAR , which can learn group-aware thresh-56 olds to optimize the trade-off between fairness and accuracy in classification . We derive57 rigorous theoretical analysis on the trade-off in our model , and empirically show that GSTAR58 outperforms state-of-the-art methods.59 2 . With GSTAR , we can simultaneously optimize over multiple fairness constraints with a low60 computational cost . GSTAR does not require multiple iterations over data , instead , it takes61 at most one pass of data in training for fast computation.62 3 . GSTAR can be adapted to a wide range of classification models in a model-agnostic manner63 and can adapt an existing classification model from one fairness criterion to another without64 re-training the classifier.65 4 . We derive Pareto frontiers of our model for the fairness-accuracy trade-offs that contextualize66 the quality of fair classification.67 2 Related Works68 In order to achieve group fairness , which quantifies the discrimination among different sensitive69 groups , a diverse notion of fairness has been introduced . Equalized odds [ 7 ] enforce equality of true70 positive rates and false positive rates between different demographic groups . Pleiss et al . [ 20 ] relaxed71 equalized odds to satisfy the calibration . Demographic parity or disparate impact [ 1 ] suggests that a72 model is unbiased if the model prediction is independent of the protected attribute.73 Among different fairness methods , post-processing techniques propose to improve fairness by mod-74 ifying the output of a black-box classifier . Hardt et al . [ 7 ] propose to ensure equalized odds by75 constraining the model output . Kamiran et al . [ 9 ] propose to give a favorable outcome to unprivileged76 and an unfavorable outcome to the privileged group when the confidence of the prediction is beyond a77 certain range . However , such static confidence window keeps the same regardless of the demographic78 group and is determined by grid search , so it is less efficient.79 Threshold adjustment ( a.k.a . thresholding ) was introduced to improve the performance of static80 thresholds . In the literature , Menon et al . [ 18 ] prove that instance-dependent thresholding of the81 predictive probability function is the optimal classifier in cost-sensitive fairness measures . Also,82 when considering immediate utility , Corbett-Davies et al . [ 3 ] show that optimal algorithm is achieved83 from group-specific threshold which is determined by group statistics . However , to the best of our84 knowledge , the threshold adjustment approach has not been deeply studied that neither encompasses85 broad group fairness metrics nor describes an explicit method to achieve the threshold.86 Trade-off between fairness and accuracy exists when we impose fairness constraint to a model . Recent87 studies [ 2 , 25 ] prove that models targeting at such fairness notions conform to an information theoretic88 lower bound on the joint error across different sensitive groups . Therefore , our work presents a89 practical upper bound of the best achievable accuracy given the fairness constraints.90 Moreover , trade-offs between different fairness notions also exist if one has to consider multiple91 fairness criteria . Some of them are theoretically proven to be incompatible [ 6 , 18 , 14 ] . To express92 and formulate fairness , recent work [ 10 ] utilize confusion matrix and propose least-square accuracy-93 fairness optimization problem on multiple fairness notions , and categorize the trade-offs between the94 fairness notions.95 Here , our work is the most related to the post-processing methods [ 7 , 10 ] . Hardt et al . [ 7 ] propose96 a post-processing method that utilizes the mixing rate to meet the equalized odds . Ours is similar97 to Hardt et al . [ 7 ] in the manner that achieving group-wise threshold from the feasible region that98 is geometrically generated by the intersection between the receiver operating characteristic ( ROC ) 99 curves conditioned on sensitive feature . Ours differ from [ 7 ] by generalizing the concept beyond100 equalized odds to other multiple fairness constraints into consideration . FACT [ 10 ] utilizes a single101 point ( static ) from the classifier to be post-processed as a reference which does not fully utilize the102 classifier for the post-processing . In contrast , by approximating the distribution of the continuous103 predicted logits , our GSTAR model enables a larger feasible region than [ 10 ] with a better fairness-104 accuracy trade-off . We validate the improvement in trade-off via both theoretical and empical results.105 It is notable that these related methods [ 7 , 10 ] can be considered as a special case of GSTAR.106 3 GSTAR for Fair Classification107 3.1 Motivation108 Consider a binary classification problem with a binary sensitive feature , such that the sensitive feature109 A ∈ { 0 , 1 } and label Y ∈ { 0 , 1 } . In general , for a given data X , a binary classification model110 outputs an unnormalized logit h ( X ) ∈ R with the class label probability p ( X ) = σ ( h ( X ) ) ∈ [ 0 , 1 ] ,111 where σ is an activation function ( sigmoid function in logistic regression and neural network ) . It is112 not necessary to calculate p in a classification model , e.g . support vector machines directly use the113 positiveness/negativeness of logit h ( X ) to determine classification outcome . For traditional models,114 we use a cut-off threshold θh = 0 for h ( X ) ( i.e. , θp = σ ( 0 ) = 0.5 for p ( X ) ) in classification , such115 that the predicted label is determined by Ŷ = I { h ( X ) ≥ θh } . In the following context , unless116 otherwise mentioned , we use θ to refer to the threshold θh on logit h since it is applicable to a117 wider range of classification models , and the corresponding threshold on label probability θp can be118 easily inferred from the threshold on logit h. Traditional models use the same cut-off threshold θ for119 different demographic groups . However , since the distribution of logits h in different demographic120 groups can be different , using the same threshold θ brings biased classification.121 In Figure 1 , we show a real-world example of image classification on CelebA dataset with122 ResNet50 [ 8 ] to show that the default setting of classification thresholds affects both accuracy123 and fairness in classification . The goal of this classification example is to predict the image of a124 person is whether attractive or not , and consider sensitive attribute as gender . This can be generalized125 to different sensitive attributes such as age or race [ 22 , 16 ] . We can observe an obvious difference in126 the distribution of logit h between two gender groups . In this case , if we use a unified classification127 threshold θ1 = θ0 = 0 , it naturally brings a difference in the true positive rate and true negative128 rate between two gender groups , thus renders bias in classification . Instead , we observe that the129 optimal group-specific threshold obtained from GSTAR ( θ∗1 > θ1 , and θ ∗ 0 < θ0 ) can adapt to such130 discrepancy in distribution between two demographic groups to improve both fairness and accuracy.131 3.2 Group-Aware Classification Thresholds132 Given an existing classification model and a sensitive attribute a , we can denote true positive rate133 ( TPa ) , false positive rate ( FPa ) , true negative rate ( TNa ) , and false negative rate ( FNa ) in the confusion134 matrix . Most fairness notions can be represented with entries in the confusion matrix . For instance,135 Equal Opportunity ( EOp ) [ 7 ] requires TP0 = TP1 , and Demographic Parity ( DP ) [ 1 ] requires136 TP1n11 + FP1n01 N1 = TP0n10 + FP0n00 N0 , where nya denotes the number of samples in the subset { Y = y , A = a } , Na = ∑ y nya denotes the137 number of samples in { Y = y } , and N = ∑ y , a nya is the total number of samples.138 Consider the group-aware classification threshold θ = ( θ1 , θ0 ) T , where θa is the classification139 threshold for sensitive group A = a . We can formulate the entries in the confusion matrix w.r.t . θ as140 below:141 TPa ( θa ) ≈ 1− ∫ θa −∞ f1a ( x ) dx , FNa ( θa ) ≈ ∫ θa −∞ f1a ( x ) dx , FPa ( θa ) ≈ 1− ∫ θa −∞ f0a ( x ) dx , TNa ( θa ) ≈ ∫ θa −∞ f0a ( x ) dx , ( 1 ) where fya ( x ) is an estimated parametric probability density function of the distribution of output142 logit h in the subset { Y = y , A = a } . Here , we consider gamma , Student ’ s t , and normal distribution143 as the candidates for the estimated distribution , and select the one that has the maximum likelihood144 with the output distribution . Without loss of generality , this can be generalized with other parametric145 probability density function based on the needs or prior knowledge.146 Then , we formulate the fairness-constrained classification problem with the objective of minimizing147 classification error into a least-squared optimization problem . We denote our objective function148 as L ( θ ) which consists of the performance loss Lper ( θ ) and fairness loss Lfair ( θ ) . Lper ( θ ) and149 Lfair ( θ ) measures the error in performance and fairness respectively that are represented with the150 entries of the confusion matrix . In other words , our goal is to minimize the objective function L ( θ ) 151 as below:152 L ( θ ) = Lper ( θ ) + λLfair ( θ ) , ( 2 ) where λ is a hyperparameter that determines how much fairness is enforced in the optimization.153 The performance error Lper ( θ ) can be written as154 Lper ( θ ) = ( n01 N FP1 ( θ1 ) + n11 N FN1 ( θ1 ) + n00 N FP0 ( θ0 ) + n10 N FN0 ( θ0 ) ) 2 . ( 3 ) As forLfair ( θ ) , it can be formulated to any fairness metrics that are expressible with confusion matrix.155 For instance , when we impose EOp ( TP1 = TP0 ) and predictive equality ( PE ) ( FP1 = FP0 ) [ 2 ] , we156 can get the corresponding Lfair ( θ ) by summing over the least squared form of each constraint . Also,157 satisfying EOp and PP is equivalent to satisfying Equalized Odds ( EOd ) [ 7 ] , This can be formulated158 in our Lfair as159 LEOdfair ( θ ) = L EOp fair ( θ ) + L PP fair ( θ ) = ( TP1 ( θ1 ) − TP0 ( θ0 ) ) 2 + ( FP1 ( θ1 ) − FP0 ( θ0 ) ) 2 . ( 4 ) Note that a lower Lfair value indicates a fairer threshold . When LEODfair ( θ ) = 0 , we can interpret as160 the θ satisfies the perfect EOd fairness.161 Similar to ( 4 ) , we can enforce multiple fairness constraints by summing over the least squared of162 each metric with different weight constant λ to each fairness constraints if needed.163 Also , it is notable that compared to the recent paper [ 10 ] that enforces fairness through confusion164 tensor , our formulation of fairness in Lfair ( θ ) represents a direct notion of fairness metrics and165 improves the measures that allows us to achieve better performance and Pareto frontiers that is shown166 in Section 4.2 and Figure 2 . For example , AEOd in the paper is calculated as M1EOp +M0PE , where167 My = ny0 +ny1 , such that EOd is a weighted sum of EOp and PE with weights being the number of168 samples in each class . In this expression , the imbalance between the two fairness criteria will grow as169 the degree of imbalance in the data increases . In contrast , our formulation expresses the constraints170 as the exact notion of each metric that is not biased by the statistics of the datset and we observe171 improved Pareto frontier as in Figure 2.172 We propose to optimize our threshold θ with alternating optimization method . Here we take EOp173 constraint as an example to show the alternating optimization steps , then Lfair ( θ ) can be written as174 LEOpfair ( θ ) = ( TP1 ( θ1 ) − TP0 ( θ0 ) ) 2 . ( 5 ) The first step is to fix θ0 and update θ1 . We can approximate the terms that are related to θ1 ( e.g.,175 TP1 , FP1 , TN1 , FN1 ) in ( 1 ) with first-order Taylor expansion at θτ−11 . For example,176 TP1 ( θ1 ) ≈ TP1 ( θτ−11 ) + ∂TP1 ∂θ1 ∣∣∣ θ1=θ τ−1 1 ( θ1 − θτ−11 ) ( 6 ) From ( 1 ) , we can easily derive that177 TP1 ( θτ−11 ) = 1− ∫ θτ−11 −∞ f11 ( x ) dx , ∂TP1 ∂θ1 = − f11 ( θτ−11 ) . ( 7 ) Similarly , we can find the first order Taylor expansion of FP1 , FN1 , and TN1 . Then , the update of θ1178 w.r.t . ( 2 ) can be approximated with the following minimization problem w.r.t . ∆1179 ∆τ1 : = argmin ∆1 ( ητ + ατ∆1 ) 2 + λ ( τ + βτ∆1 ) 2 , ( 8 ) where ∆1 = θ1 − θτ−11 and180 ατ1 = n11 N f11 ( θ τ−1 1 ) − n01 N f01 ( θ τ−1 1 ) , βτ1 =− f11 ( θτ−11 ) , ητ1 = ∫ θτ−11 −∞ ( n11 N f11 ( x ) + n01 N ( 1− f01 ( x ) ) dx+ ∫ θτ−10 −∞ ( n10 N f10 ( x ) + n00 N ( 1− f00 ( x ) ) dx , τ1 = ∫ θτ−11 ∞ f11 ( x ) dx− ∫ θτ−10 ∞ f01 ( x ) dx . ( 9 ) Taking the derivative of ( 8 ) w.r.t . ∆1 and setting it to 0 , we can easily obtain the closed-form solution181 of ∆τ1 as182 ∆τ1 = − ατητ + λβτ τ ( ατ ) 2 + λ ( βτ ) 2 . ( 10 ) Algorithm 1 Optimization Algorithm of GSTAR Model Input dataset X ×A× Y = { ( xi , ai , yi ) } ni=1 , classification model h ( X ) , hyperparameter λ . Output Group-specific threshold θ = ( θ1 , θ0 ) . Initialize θ = ( θ1 , θ0 ) = ( 0 , 0 ) . 1 . Given a classifier H ( x ) , estimate probability density function fya , y , a ∈ { 0 , 1 } by maximum likelihood estimation . while not converge do 2 . Calculate the optimal step ∆1 as ∆1 = −α1η1+λβ1 1α21+λβ21 , with α1 , β1 , η1 , 1 values shown in ( 9 ) ; 3 . Update the threshold : θ1 ← θ1 + ∆1 ; 4 . Calculate the optimal step ∆0 as ∆0 = −α0η0+λβ0 0α20+λβ20 with α0 , β0 , η0 , 0 values calculated in a similar way as in ( 9 ) : 5 . Update the threshold : θ0 ← θ0 + ∆0 . end while The second step is to fix θ1 and update θ0 , and this can be achieved in a similar way of updating θ1.183 Then we can finalize the alternating optimization as:184 θτ0 = θ τ−1 0 + ∆ τ 0 , θτ1 = θ τ−1 1 + ∆ τ 1 . ( 11 ) It is notable that in each iteration we derive the optimal update step θ , which eliminates the burden185 of tuning hyperparameter ( such as learning rate ) in iterative algorithm . The optimization step is186 summarized in Algorithm 1 . The above algorithm can easily extend to multiple fairness constraints187 by adding corresponding squared-loss fairness terms to ( 2 ) .188 Time Complexity : The alternating optimization of GSTAR model is of low computational cost . We189 take at most one pass of the data for learning the estimated probability density functions fya in ( 1 ) 190 ( we do not even need to traverse the data if the parameters ( such mean and variance in Gaussian191 distribution ) for the estimated probability density functions fya can be provided ) . The optimization192 of θ with alternating optimization is efficient since we only need fya as we have seen in ( 9 ) and ( 10 ) .193 θ ∈ R2 is a vector with fixed small size . Therefore , we need a constant time for each update . Overall,194 the time complexity of GSTAR is O ( n+ T ) , where n is the number of samples , and T is the number195 of iterations in alternating optimization.196 We further derive the theoretical analysis of our GSTAR model on the balance between fairness197 and accuracy , which indicates that the optimal solution provides guarantees on model accuracy198 under the optimal fairness constraint . Details of the theoretical analysis is in the Supplementary199 material . Besides , if a unified threshold is necessary [ 3 ] , i.e. , θ1 = θ0 , the optimization algorithm200 also applies and we only have one scalar variable in ( 2 ) . When we have a unified threshold , we do not201 require sensitive information in the testing phase that we can conform more strict privacy regulations202 than group-aware thresholding . However , we have to sacrifice both fairness and accuracy as the203 thresholding is less flexible.204 4 Experiments205 In this section , we validate GSTAR model on four well-known fairness datasets and compare with206 other state-of-the-art methods . First , we plot Pareto frontiers of ours and FACT ( MS ) [ 10 ] to207 demonstrate the trade-offs between fairness and accuracy . Second , we evaluate the models with208 different fairness metrics and validate that our model is highly adaptive to any fairness metrics that209 are expressible with confusion matrix [ 7 , 11 , 2 , 1 ] . Third , we use our model as a post-processing210 method to existing fair models and show that our model further improves existing fair models in an211 efficient and model-agnostic manner.212 4.1 Experimental Setup213 We compare with multiple fairness approaches in the experiments . For clear demonstration of results,214 we use different shapes of marker for each comparing methods in Figure 2 and Figure 4 . The compar-215 ing methods include : Learning fair representations for kernel models ( abbreviated as FGP ) [ 23 ] ,216 Fairness confusion tensor ( abbreviated as FACT ) [ 10 ] , Disparate impact remover ( abbreviated217 as DIR ) [ 6 ] , Adversarial de-biasing ( abbreviated as AdvDeb ) [ 24 ] , Calibrated equalized odds218 post-processing ( abbreviated as CEOPost ) [ 20 ] , Equality of opportunity in supervised learning219 ( abbreviated as Odds ) [ 7 ] , Learning adversarially fair and transferable representations ( abbrevi-220 ated as LAFTR ) [ 17 ] , and Baseline : For CelebA dataset , we use ResNet50 [ 8 ] as a reference , and221 logistic regression for all other datasets . Our method is optimized with λ in the range of [ 10−1 , 104 ] 222 with alternating optimization method . All experiments are implemented with Pytorch framework on223 i9-9960X CPU and a Quadro RTX 6000 GPU.224 We choose broadly used fairness metrics in evaluation including : equal opportunity difference and225 equalized odds difference ( abbreviated as EOp , and EOd respectively ) [ 7 ] ; 1-disparate impact226 ( abbreviated as 1-DIMP ) [ 1 ] ; balanced accuracy difference ( abbreviated as BD ) .227 We evaluate the methods on four fairness datasets : CelebA image dataset1 [ 15 ] , Adult dataset from228 the UCI repository [ 12 ] , COMPAS2 ( Correctional Offender Management Profiling for Alternative229 Sanctions ) dataset , and German credit dataset from the UCI repository [ 5 ] . All data is split as 70 % 230 for training and 30 % for testing . More details of the comparing methods , evaluation metrics , and231 datasets are provided in the Supplementary material.232 4.2 Performance and Fairness-Accuracy Trade-Offs233 In this subsection , we look into the performance evaluation of GSTAR comparing with other state-of-234 the-art methods . We consider Pareto frontier to visualize the trade-offs between fairness and accuracy235 to demonstrate the measure of performance.236 In Figure 2 , we plot Pareto frontier , which is the upper bound for the accuracy-fairness trade-offs,237 desired output locates at the upper right region under the boundary which corresponds to higher238 values in accuracy and lower values in fairness discrepancy . With the same fairness constraints239 are given , we achieve a better frontier than the FACT [ 10 ] as we equally weigh on demographic240 statistics and have a better feasible region . To obtain our results ( star points ) , we first estimate the241 logit distribution from the output of the baseline model , and then we get optimal adaptive thresholds242 with corresponding fairness metric by updating w.r.t . the objective function in ( 2 ) . Here we have three243 combinations of fairness imposed to GSTAR : demographic parity ( DP ) , equalized odds ( EOd ) , and244 1http : //mmlab.ie.cuhk.edu.hk/projects/CelebA.html 2https : //github.com/propublica/compas-analysis Basel ine AdvD eb Eq.Od ds CEOP ost LAFTR FAC T GSTA R ( DP ) GSTA R ( EO d ) GSTA R ( DP+ EOd ) 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 Fa irn es s Fair Bias 0.65 0.68 0.70 0.72 0.75 0.78 0.80 0.82 0.85

Summary: The paper proposes a post-processing approach to achieve different notions of fairness. To do this, the paper first approximates the probability distribution of the output model. Group-specific confusion matrices can be derived by thresholding this distribution. The entries of such a matrix can be used to compute accuracy and various fairness metrics. The authors experimentally evaluate their approach on several datasets.