Pushing the limits of fairness impossibility: Who’s the fairest of them all?
Anonymous Author(s)
AffiliationAddress email
Abstract
The impossibility theorem of fairness is a foundational result in the algorithmicfairness literature.
It states that outside of special cases, one cannot exactly andsimultaneously satisfy all three common and intuitive definitions of fairnessdemographic parity, equalized odds, and predictive rate parity.
This result hasdriven most works to focus on solutions for one or two of the metrics.
Ratherthan follow suit, in this paper we present a framework that pushes the limits of theimpossibility theorem in order to satisfy all three metrics to the best extent possible.
We develop an integer-programming based approach that can yield a certifiablyoptimal post-processing method for simultaneously satisfying multiple fairnesscriteria under small violations.
We show experiments demonstrating that ourpost-processor can improve fairness across the different definitions simultaneouslywith minimal model performance reduction.
We also discuss applications of ourframework for model selection and fairness explainability, thereby attempting toanswer the question: who’s the fairest of them all?
Introduction
While fairness in machine learning has received significant attention in recent years, most existingworks focus on one of the many fairness criteria [3].
Consequently, practitioners are perhaps leftwith no choice but to use their best judgment to apply a single fairness criterion.
We suspect that theconflicting nature of existing mathematical definitions of fairness might have led to this undesirablepractice of narrowing down fairness-related measurement and mitigation to one chosen definition.
This is somewhat analogous to the trade-off between precision and recall while evaluating modelperformance [7].
Instead of choosing one of precision or recall to evaluate the performance of aclassification model, practitioners often evaluate the trade-off and choose models that can maintaina certain level of precision while optimizing recall or vice-versa [19].
In this paper, we providea framework for explicitly addressing such trade-offs among multiple fairness criteria and modelperformance toward optimal model selection.
One of the most prolific examples of fairness in machine learning arose from the ProPublica recidivismstudy ([2]), in which a risk assessment tool called COMPAS was found to be biased against blackdefendants.
But beyond the immediate implications in criminal justice, the study also promptedmore general studies in algorithmic fairness and, in particular, led to a key result highlighted in [13],[21], and [25] which some colloquially refer to as the "impossibility theorem" in fairness [30, 24].
This theorem essentially states that three common definitions of algorithmic fairness - demographicparity [16], equalized odds [21], and predictive parity [3], cannot be simultaneously satisfied outsideof pathological situations.
Several works following these initial results have therefore focused onsatisfying one metric ([21, 32]) or proposing adaptable methods for different metrics ([11, 37]).
Yet,while we do not deny the conclusions of the impossibility theorem, we also believe there have not been sufficient efforts to reconcile the conflicting fairness definitions to the best extent possible
We fill this gap in the literature by translating the trade-offs among multiple fairness criteria and modelperformance into a constrained optimization problem and propose a post-processing methodology forsimultaneously achieving approximate fairness in the conflicting definitions simultaneously.
We believe our framework would alleviate the practitioner from making the hard choice in choosing aparticular metric.
Instead, if they have a partial ordering of importance amongst the metrics (whichmany possess), our framework would explicitly allow them evaluate such trade-offs.
The applicationwhich we highlight in Section 4 discusses these in detail.
Overall, we make three main contributionsin this work:
We design a flexible optimization framework that returns a post-processing score transformation
function that can make scores group-wise ϵ -fair along three definitions (demographic parity,equalized odds, and predictive rate parity) simultaneously.
This framework can be applied toany binary classifier that produces a continuous score, can be configured for singular or multiplemetrics of fairness, and also can account for fairness vs. performance trade-offs in terms ofROC-AUC.2.
We present a novel reformulation of this non-convex optimization problem as a Mixed IntegerLinear Program (MILP) [41].
This reformulation allows us to find provably globally optimalsolutions.
We further show that in practice, we can consistently find better solutions through ourglobal optimization method compared to local optimization methods in a reasonable time.3.
We discuss and extend our framework from a post-processing mechanism to a tool that can aidpractitioners in better understanding their data and models’ empirical fairness characteristics andtrade-offs and compare these traits across models.
The rest of the paper is organized as follows.
In Section 2 we mathematically define fairness metricsand the multiple fairness optimization problem.
We discuss the optimal solution via MILP in Section3.
We try to answer the question of “who’s the fairest of them all?”
through our applications andexperiments in Section 4.
Finally, we conclude with a discussion in Section 5.
We wrap up thissection with a discussion of the related literature.
Related Work: Early works [13] exploring the conflicts between fairness definitions prove that for abinary predictor, predictive rate parity conflicts with equalized odds ([21]) unless base rates are equalor the model is perfectly predictive.
Chouldechova [13] also considers trade-offs between the fairnessdefinitions in the binary prediction case.
Kleinberg et al.
[25] generalizes this study by showing thatstatistical (i.e., demographic) parity is also inconsistent with predictive rate parity and equalized odds.
The same paper studies these inconsistencies in a more general bin-wise prediction setting and showsthat approximate fairness definitions (predictive rate parity, equalized odds) can simultaneously holdbut only under ϵ -approximate equal base rates or ϵ -approximate perfect performance.
This workfurther proves that there is an inherent trade-off between fairness and loss.
Beyond impossibility theorem results, several works have focused on trade-offs between fairness andmodel performance [21, 17, 37, 42].
They develop in-processing solutions aimed at reducing onemetric while maintaining accuracy.
A few authors have analyzed trade-offs or attempted to achievemultiple fairness.
One example is Pleiss et al.
[36], which shows that predictive rate parity and arelaxed form of equalized odds are reconcilable under a randomized prediction scheme.
Anothernotable example is Celis et al.
[11], which develops a flexible in-processing approach to achievemultiple types of fairness (potentially at the same time).
Like Celis et al.
[11], the framework wedevelop also aims for a flexible approach to focus on one or many fairness metrics simultaneously.
However, our method is distinct in that it is a post-processing based solution and also more general asit works for continuous scores (rather than binary classification).
Our work is most closely relatedto Nandy et al.
[32] in terms of the underlying score transformation mechanism, and we leveragesome of their methods.
However, whereas [32] only targets equalized odds, we go further and includedemographic parity and predictive rate parity in our framework—this posits computational challenges,which we address by proposing novel methodology based on integer programming.
As noted at the end of [25], some open questions are how to optimally assign scores to satisfymultiple criteria when base rates are equal and additionally, how to satisfy predictive rate parity andeither equal TPR or TNR when one cost outweighs the other.
To our knowledge, no prior workhas attempted to reconcile all three fairness conditions (demographic parity, equalized odds, andpredictive rate parity) simultaneously with model performance through a post-processing framework.
Although the post-processing methods tend to be less flexible for fairness-performance trade-offs than their in-processing counterparts, they can be much more easily added on top of any existingmodel training pipeline.
This makes a post-processing approach modular, and in particular, moreappealing in complex web-scale recommender systems that use (a combination of) certain predictionscores to rank a list of items.
Multiple Fairness Optimization
We consider a binary classification problem where the i -th observation is characterized by theirlabel y i ∈ { 0 , 1 } , their group membership g i ∈ G , and model predicted probability (also known asa risk score [25]) s i ∈ [0 , 1] for i
= 1 , . .
N .
The corresponding random variables are denotedas Y , G and S respectively.
To set up the problem, we discretize the scores into nonempty binsb ∈ B := { 1 , . .
. , |B|} by using, for example, a quantile transformation (we will denote |B| as B ).
Additionally, let N [ g ] b + denote the number of group g positive class ( y i = 1 ) instances in bin b , N [ g ] bdenote the total number of instances of group g instances in bin b , N [ g ]+ ( N [ g ] − ) be the total numberof group g positive (negative) instances, and N [ g ] be the total number of group g instances.
Lastly,our approach seeks to achieve fairness by moving instances from one bin to another bin: hence, wedefine variable x
[ g ] bb ′ as the probability of moving an instance of group attribute g and score in bin binto a new bin b ′ 1 .
In other words, for every group g , the collection { x [ g ] bb ′ } b,b ′ can be represented asa B × B transition matrix (with additional constraints, as discussed next).
For the optimization framework and the remainder of this paper, we translate a single fairnessdefinition (e.g. equal true positive rate) as a constraint that controls for the worst-case violationsacross all bins.
Below, we discuss different fairness constraints that we consider in our framework,and describe how they can be represented in terms of the optimization variables { x [ g ] bb ′ } .2.
1 Fairness Constraints Under Binning Framework
Demographic Parity (DP): For simplicity, we assume that there are exactly two groups g ∈ { 1 , 2 } as we formulate the fairness metrics, although everything discussed below can be applied for nonbinary groups by using pairwise constraints, which we elaborate on in Appendix A.
Starting with demographic or statistical parity from [16], this condition states that the model’s predicted score is independent of group membership.
This is equivalent to
[Equation]
Our version of this constraint uses bins B to empirically approximate the probability P ( S = s |
G = g )
and we also relax the equality to an ϵ -approximate equality (for some pre-specified ϵ > 0 ).
Therefore, after transforming the scores using { x [ g ] bb ′ } b,b ′ the ϵ DP -approximate DP can be expressedas the following as a linear constraint:
[Equation]
where N [ g ] b denote the number of observations from group g in bin b (before transformation), and
N [ g ] = (cid:80) b ∈B N [ g ] b .
For reproducibility, ϵ DP should be chosen to be larger than the approximationerror O (1 / √ N [ g ] ) for replacing P ( S = s | G = g ) with its empirical counterpart.
Equalized Odds (EOdds): The equalized odds condition for binary predictors, given in Hardt et al.
21, is a balance condition where the groups must have equal true positive and false positive rates.
For continuous scores, it translates to having equal score distributions for each group conditional on their true labels [32]:
[Equation]
Like demographic parity, our empirical score bin version requires that the distribution of positive ornegative instances be ϵ EOdds -approximately equal between groups in the new bins b ′ .
Both equaltrue positive rate and false positive rate can be expressed as linear constraints, respectively:
[Equation]
Predictive Rate Parity (PRP): Lastly, we examine the predictive rate parity condition popularized in [13].
This condition states that the probability of being a positive instance is independent of group membership when we condition on the score.
Formally:
[Equation]
Using the empirical score bin framework
[Equation]
an approximate version of the above implies that theproportion of positive instances in each bin must be ϵ PRP -approximately equal among groups:
[Equation]
Unlike constraints (1) and (2), which can be expressed as a linear function of the optimizationvariables { x [ g ] bb ′ } , condition (3) yields bilinear terms and is in general a non-convex constraint.
Amain technical difficulty of our framework arises from this non-convex fairness constraint—Section3 presents an integer programming framework to handle this non-convexity, ensuring we can obtain aglobally optimal solution to the resulting optimization problem.
Remark.
Our definition of fairness as the worst-case violation across all bins aims to resembleapproximations of the respective probabilistic definitions but we have not found identical definitionsin other works.
We comment on the differences and discuss why it does not contradict the traditionalimpossibility theorem of [25] in Appendix F.
2.2 MFOpt: Multiple Fairness Optimization Framework
We use constraints developed in Section 2.1 to state the multiple fairness optimization (MFOpt)problem:minimize [Equation] 4d
[Equation]
Above, the optimization variables are the group-specific movement probability x
[ g ] bb ′ terms and allremaining terms are problem data and/or configurable hyperparameters.
Starting with the objective(4a), we define ¯ s b as the midpoint score in the bin and hence the objective is the product of themovement distance ¯ s b − ¯ s b ′ weighed by the fraction of total samples moved N [ g ] b / N and the amountof movement x
[ g ] bb ′ .
(4b) states that the total movement out of bin b , including the movement back toitself, must sum up to 1 and along with (4g) ensures that x [ g ] bb ′ represent probabilities in a transition matrix.
(4c) states that the total movement from bin b back to itself must be lower bounded byhyperparameter ξ .
This parameter controls how far we allow the new scores to stray from the originaland is necessary to prevent zero denominators in 3 and 4f.
Constraint (4d) represents windowconstraints to restrict extreme movements of scores beyond w bins.
Constraint (4e) are the fairnessconstraints in Section 2.1.
(4f) ensures that we preserve the rank-ordering of the scores and expectedvalues, which is desirable for comparing bins against each other. While constraints (4d), (4f) are notfairness-related, we add them to retain the utility of the solution.
The predictive rate parity constraint(3) and (4f) both introduce non-convexities into Problem (4).
These two constraints also require us toassume overlap, or that each bin contains at least one member from each group.
Without overlap,predictive rate parity is undefined since it is not possible to compare expectations across groups for agiven bin and demographic parity is likely violated as it means one group has 0 probability of landingin a given bin.
We close this section with two major benefits of this framework compared to inprocessing solutionssuch as adding fairness regularization ([42, 17, 37]) or using an entirely different fairness-basedmodel ([11], [43]).
First, the number of optimization variables scales in the order of O ( |G||B| 2 ) whilethe number of constraints scale in the order of O ( |B||G| 2 ) .
This is significant as it entails that ourframework can be applied in arbitrarily large data settings as long as score-based binning is possible.
Applying the transformation is equally tractable, as it only requires binning observations and makingindependent draws from a multinomial distribution with B possible outcomes.
The second benefit of our framework is that it returns a highly interpretable solution as it returns oneoptimized B × B transition matrix per group.
Hence given a newly scored instance, several facts canbe read from the corresponding row of the matrix such as the likelihood of moving to a specific bin b ,moving into any higher or lower bin, etc.
These probabilities can also be controlled via constraints asshown with the window constraints (4d) and max movement constraints (4c).
This interpretability isan advantage over model regularization frameworks, where it is difficult to know how an individual’sscore might change when switching from the base model to a fair model.
Finding Optimal Solutions via Mixed Integer Programming (MIP)
The primary difficulty of the above optimization problem are the predictive rate parity constraint(3) and rank-order constraint (4f) which turn the problem non-convex.
Non-convex constrainedoptimization is generally NP-hard and traditional methods that seek locally optimal solutions includegradient-based interior point optimization ([40]), sequential quadratic programming ([18]), or algorithms specific to quadratically-constrainted-quadratic-programs (QCQPs) such as operator splittingmethods, semidefinite relaxations, etc.
(see [33] for an overview).
Rather than pursuing a locally optimal solution, we propose a novel reformulation of the problem intoa tractable mixed-integer-linear-program (MILP) which can be solved to global optimality ([10], [9]).
Our reformulation grants two benefits over traditional locally optimal solvers.
First, global strategiestheoretically enable us to find the best possible solution.
Second, as our problem primarily scaleswith the number of bins, it is practically tractable when utilizing the power of modern MIP solvers.
3.1 Tractable reformulations for computational efficiency
We first observe that a direct reformulation of the fractional terms into bilinear terms in constraints (3)and (4f) will lead to bilinear terms in the order of O ( B 3 ) .
We show that we can reduce this to O ( B )bilinear terms through a substitution that exploits the problem structure.
Next, we take advantage ofthe vastly reduced number of bilinear terms to apply the normalized multiparametric disaggregationtechnique (NMDT, [1]) which we explain in Section 3.1.2.
This allows us to approximate products ofcontinuous variables as products of binary variables, which can be easily linearized and handled by
MIP solvers.
Importantly, this transformation of xy (i.e, product) terms requires upper and lowerbounds for x and y and we propose a method in Section 3.1.2 for generating and tightening thesebounds by solving fractional linear program subproblems ([12]).
Taken together, the reduction ofbilinear terms combined with the bound-tightening procedure enable us to effectively apply the NMDT methodology and transform the problem from a non-convex QCQP to an MILP that can besolved to global optimality.
3.1.1 Step 1: Reducing number of bilinear variables
We reduce the number of bilinear variables in our problem by making a substitution for the fractionterm by introducing new optimization variables v [ g ] b ≥ 0 (to represent a sum) and t [ g ] b ≥ 0 (torepresent the fractional quantity) as additional variables (see Appendix D for more details).
We canthen use them to write equivalent constraints with only O ( B ) bilinear terms.
Let,
[Equation]
and
[Equation]
Then we have the following:
[Equation]
3.1.2 Step 2: NMDT and bound tightening through fractional LP subproblems
Linearizing bilinear terms (NMDT): We show how we can model each bilinear term t [ g ] b v [ g ] b by using a binary expansion for the continuous variables t
[ g ] b , v [ g ] b , and by observing that the product of binary variables can be modeled via integer programming (see [27, 39] for reference).
To this end, we make use of the NMDT transformation [1].
We recap this method below as formulated in [1]
Given any bounded optimization variable x ∈ [ x L , x U ] , and precision factor p , a negative integer, we can represent this variable exactly as x = ( x U − x L ) λ + x L where
[Equation]
where 0 ≤ ∆ λ ≤ 2 p is a remainder term and z l ∈ { 0 , 1 } are binary optimization variables.
Droppingthe remainder term ∆ λ gives us the approximate form and product forms of xy become dot productsof several integer variables, which can be effectively handled via modern MIP solvers, such as [20].
Although we are solving an approximation (e.g. precision of 1 e − 4 ) this is not a practical problemsince it is precise enough for reasonable choices of ϵ and we do not expect the constraints to holdexactly when we apply the post-processor on the testing data.
Bounds on v [ g ] b , t [ g ] b via Fractional LPs: A key requirement to apply NMDT is that all optimizationvariables in the bilinear terms ( t [ g ] b and v [ g ] b in our case) must be bounded and we need to be ableto accurately estimate these bounds (i.e., tighter bounds leads to faster runtimes [1]).
We proposeobtaining these lower and upper by minimizing and maximizing respectively, the sub-problemswhile keeping all fairness constraints except the quadratic constraints.
Firstly, note that bounds onv [ g ] b can be solved as a simple LP (details in Appendix D).
Meanwhile, obtaining a bound on t [ g ] brequires solving a nonlinear problem due to fact that t [ g ] b represents a fractional objective (ratio of twoaffine terms of optimization variables).
However, we observe that we can apply the Charnes-Coopertransformation [12] to reformulate the nonlinear problem into a simple LP (details in Appendix D).
3.2 Choice of algorithm: QCQP (heuristic) vs MIP (optimal solution)
In this section we show the results and benefits of our reformulation from a non-convex QCQP to an MILP.
In Table 1, we take each dataset, create a 60/40 train-test split, train a grid-searched randomforest model, and score the training data.
Next, we discretize the scores into bins, parameterize theproblem (# bins, ϵ , max movement, window size, solve time) on the scored training data, and compareour MILP solution solved by Gurobi ([20]) against the QCQP problem solved by IPOPT ([28]), whichis a generic interior-point log-barrier penalty method for nonlinear constrained optimization.
Wediscuss the datasets and problem parameters for all experiments in the Appendix E. For each metricsuch as AUC , we use AUC INT and AUC IP to denote the average result of applying the interiorpoint (INT, for short) or integer programming (IP) method, respectively.
The metrics used are the which we utilize objective value, optimality gap ( %∆ ), 3 and AUC 4 .
We also report the statistical significance of theimprovement based the p -value from the Wilcoxon signed-rank test to determine if %∆ IP ≤ %∆ INTis a consistent result.
Bold figures indicate statistical significance w.r.t.
1 standard deviation 5 6 .
[Table]
232 As the results show [Table] the MIP reformulation consistently beats the interior point solver applied on theraw optimization problem, even when solving for only 10 minutes.
We also observe that in mostcases, regardless of the method we choose, we can quickly find near optimal solutions that are highperforming in the AUC sense.
This is a significant result as it means that even a locally optimalsolution to our optimization problem can yield a practically useful post-processing result. Additionalexperiments on these datasets showing the effectiveness of MFOpt with respect to performance andfairness in a training/testing data scenario can be found in Appendix H.
We conclude this section by reiterating two benefits of the reformulation.
First, solving a MIP methodyields lower bounds that can be used to prove optimality or otherwise gauge the quality of a feasiblesolution.
Second, by framing the problem as a MIP, we can always theoretically continue improvingthe solution to optimality based on the acceptable time limits. There are other applicable globaloptimization methods, such as spatial branch and bound.
We considered these solutions but ultimatelyopted for a MIP approach due to the maturity and availability of solvers (see Appendix G for details).
Applications
In this section, we illustrate a few methods of applying our framework and how it can be used tohelp model developers select and understand models from a fairness perspective.
To apply theseprocedures, we first require developing an efficient frontier of fairness solutions to understand whichϵ configurations are feasible for a given model type and dataset.
To generate this frontier, we solvethe problem over a grid of parameters ϵ DP , ϵ EOdds , ϵ PRP .
Each feasible solution will yield a points ∈ { ( AUC, ϵ DP , ϵ EOdds , ϵ PRP ) } and the collection of non-dominated points from the solutionset yields a efficient frontier.
We show the 2-d profile shots of our 4-d fairness surface in Figure 1as an example, where the axes represent one of three fairness metrics and the point color gradientrepresents AUC.
In theory, we could obtain a true Pareto-optimal frontier since we have devised amethod of obtaining globally optimal solutions.
However, we generate this frontier using IPOPT dueto practical limitations as we are solving a 7 × 7 × 7 grid of ϵ parameters.
4.1 Understanding fairness tradeoffs
After the Pareto frontier is generated, the modeler can pick an operating point s based on thedesired AUC and tolerable fairness violations ϵ .
However, when communicating fairness propertiesto stakeholders and accounting for potential changes in strategy, it can be helpful to additionallyunderstand the cost of further increasing fairness in terms of ϵ DP , ϵ EOdds , ϵ PRP . [Table]
This tradeoff can be understood by looking at the characteristics of points on the frontier near theoperating point.
Suppose we are at an operating point s .
If we want to trade AUC for ϵ EOdds , thenwe would find a point s ′ with at least as good ϵ DP , ϵ PRP but worse AUC and better ϵ EOdds .
Moregenerally, if we pick trade performance/fairness characteristic c (cost) for characteristic b (benefit),then we hold all other factors constant and find a point with better b and worse c .
We illustrate thisin the Table 2.
The first row shows a hypothetical operating point while the following rows showother points on the efficient frontier that we could move to when we make a certain trade.
Blank rowsindicate that no such point was found that permits the desired trade-off.
This trade-off perspective canenable developers to better understand and communicate the costs to performance or other fairnessmetrics when trying to close the disparity in one fairness metric.
4.2 Performance Comparison
Lastly, we compare our framework against two methods and show that we can satisfy fairnessconstraint(s) just as well or better, while obtaining significantly stronger performance.
Our comparisonis done as follows (details in E), in each iteration we randomly split and pre-process the data, tune andfit a random forest model, and score the training and testing data to get the base scores ˆ y 0 .
Next, werun the methodology that we are comparing against (i.e. build a model or apply the postprocessor) toget method scores ˆ y m .
We then bin the outputs of the base model and compared method and computethe AUC along with the fairness metrics ( ϵ 0 , ϵ m ).
Next, we solve our constrained optimizationproblem on the training data, where we set the parameters ϵ to 12 min ( ϵ 0 , ϵ m ) .
After optimizing, wecan apply the optimal solutions x
[ g ] bb ′ to assign new bins in the testing data based on the original scorebins.
One method of assigning a group g instance with score s ∈ b (denote as s [ g ] b ) would be torandomly draw from a multinomial distribution parameterized by probabilities ( x [ g ] b 1 , x [ g ] b 2 , . . .
, x [ g ] b |B| ) .
We propose alternative methods to this stochastic assignment method in the Appendix B. Lastly, wecompute the resulting AUC and fairness metrics on remapped bins for the testing data (and do thesame for ˆ y 0 and ˆ y m on the testing data).
These figures are shown in Table 3.
We only show the resultson the test set due to space constraints and have placed the results for the training set in Appendix E. Bold figures indicate that a metric is statistically significant to 1 standard deviation.
First, we compare our framework against the in-processing framework in Rezaei et al.
[37], which is arobust optimization-based logistic regression model for reducing equality of opportunity violation.
Wefound that this method works well compared to standard logistic regression and managed to decrease [Table] fairness violations while maintaining the similar performance.
It improves in ϵ Eodds compared to arandom forest as well.
However, its weakness is that the underlying model is still logistic regressionand therefore has limited expressiveness.
In comparison, MFOpt can be applied on top of any modelclass and thus the performance advantages of more flexible models are better maintained.
Next, we compare our framework against the post-processing framework in Pleiss et al.
In thismethod, the authors use randomization to maintain the model’s calibration while simultaneouslysatisfying a relaxed equalized odds condition (whereby a linear combination of TPR and TNR aresatisfied).
Again, we see that this method maintains close performance as the base model, successfullyshrinks equalized odds violations, and even decreases demographic parity violations too.
However, itresults in large violations of predictive rate parity based on our definition in contrast to our method.
4.3 Who’s the fairest of them all?
In industry, machine learning model selection is guided by many factors including performance,speed, interpretability, among others.
Yet, the fairness dimension is commonly overlooked unlessthe developers specifically induce it in their model.
Even then, picking a specific fairness metric tooptimize for can be a nebulous task.
Rather than focus on a single metric, we propose a simple andintuitive method of gauging a model’s efficiency in trading between different fairness definitions.
To do so, we first construct the frontier for the two models in question and then filter all points onthe efficient frontier with tolerable performance AUC ≥ AUC min .
Next, we find the point on therespective frontiers with minimum Euclidean distance to the origin.
The model with the shorterdistance to their frontier can then be declared as the model that has better tradeoff properties.
We donot show an example due to lack of space, but remark that this procedure and the trade-off analysiscan be useful when a developer is iterating between models that were not designed for fairness, butwants a model that can be flexibly made more fair through the postprocessing framework that wepropose.
As fairness requirements may change over time, the model that can yield the best tradeoffsbetween different definitions can offer the most overall utility.
Discussion
In our study, we have devised a flexible, tractable, and interpretable post-processing method whichwe apply to push the limits of the impossibility theorem.
We show that while theoretical limitationsremain undisputed, there is a path forward to practically reconciling the conflicting fairness definitions.
These results extend the findings of [38], which state that the trade-off of fairness and accuracy arenegligible in practice.
Our work reinforces this claim but also adds on that trade-offs between fairnessdefinitions can be negligible as well.
For further research, one area is to improve the consistency of the
PRP violation reduction, as we observed the largest standard error in reducing this metric.
This couldbe due to us using random forests for all experiments, which is known to be an uncalibrated model[6].
One method to address this is therefore first calibrating the model through other methodologiessuch as Platt scaling ([35]) or binning-based calibration ([26]) before applying MFOpt.
We couldalso consider incorporating uncertainty in the training data through principled approaches such asstochastic or robust optimization.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A] [Table]
Please do not modify the questions and only use the provided macros for your answers.
Note that theChecklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
In the discussion
[Table]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] In SectionE(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes] [Table]
If you are including theoretical results...
[Table]
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] We do not
derive any theoretical results and all claims (e.g. NP-hardness of non-convex programs)
are either well-known or a source is referenced.
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Yes, all code isincluded in a zip file uploaded as the supplementary materials.
Instructions and data [Table]
used in all experiments are clearly outlined in E and
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] Referenced in E
[Table]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Referenced in E4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] License and references
can be found in the Appendix Section E
(b) Did you mention the license of the assets?
[Yes] License are mentioned alongside
open-source package references
[Table] (c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes] Under the Experimental Data Section E
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Mapping from bins to scores
Extending the MFOpt framework to non-binary groups requires creating pairwise constraints for allgroups in each bin to achieve the same fairness effect.
This means that the number of constraintsscaled in the order of O ( |B||G| 2 ) where in practice, we expect | G | to be in the range of 3-4.
Thispairwise procedure does not affect the scaling of the number of optimization variables, whichremains at O ( |B| 2 |G| ) .
However, we note that the scaling factor of the constraints is not a majorbottleneck due to our problem reformulation as presented in Section 3.
Without the reformulation,the pairwise constraints would generate O ( |B| 3 |G| 2 ) bilinear terms from the cross-multiplication ofthe numerator/denominator sums and return a much more computationally intensive problem.
B Mapping from bins to scores
As mentioned, the most straightforward method of applying the score transformation after solving the optimization problem is to sample from a multinomial distribution.
However, this is a less granular approach as we are assuming that all observations in the bin are indistinguishable.
To overcome this, we recommend the idea proposed in ([32]) which is a linear projection.
This strategy proposes that if an observation with score s falling into a bin a with upper and lower bounds [ a l , a u ] gets mapped from the random draw into a new bin b 1 with bounds [ b 1 l , b 1 u ] , then we assign it a linearly interpolated score given by:
[Equation]
This allows us to maintain rank-ordering of scores that receive the same assignment from a to b .
A more deterministic manner of mapping from bin to score would be to take the expected scoremapping.
After solving the optimization problem, we know the transitions probabilities a to { b 1 , b 2 ,..., b B } (denoted as P ( a −→ b i ) ) based on the optimization variables and from the previous method,we also know the score assignment if a were moved into b i (denote as s i ).
Hence, a deterministicmap would transform score s to s ′ =
(cid:80) i ∈
B s i P ( a −→ b i ) .
C Computing AUC from bins and using AUC as an objective
Another idea we leverage from ([32]) is the Riemann approximation of AUC from the bins.
Essentially,
ROC AUC be approximated by the FPR at bin k and TPR of the cumulative bins b ∈ { k, . . .
, B } .
Another consideration is that we could have changed the objective to maximizing AUC rather thanminimizing score movement.
However, in our experience, maximizing AUC (quadratic objective)as the objective led to a harder time finding better feasible solutions compared to minimizing scoremovement (linear objective).
D Details on the fractional LP subproblem for bound tightening
We elaborate on the methodology in Section 3.1.2.
Recall that our goal is to find bounds for:
[Equation]
and
[Equation]
Where t [ g ] b ′ is meant to represent the fractional quantity:
[Equation]
We will do this by fixing ¯ g and ¯ b such that we first tighten bounds for v ¯ g ¯ b and then use the optimalsolution to tighten bounds for t ¯ g ¯ b .
First, it is easy to see that maximizing/minimizing v ¯ g ¯ b is an LP aswe have dropped the quadratic constraints, leaving us with a linear objective and linear constraintset.
Now let v ¯ g ∗ ¯ b,min/max represent the optimal values of the min/max objective for v ¯ g ¯ b .
We now turn to bounding t ¯ g ¯ b which has the same linear constraints but a fractional (nonlinear) objective.
To dealwith this, we utilize the Charnes-Cooper transformation ([12]).
Essentially, this reformulation trickhandles the denominator by removing it from the objective and passing it to all constraints whilemaintaining linearity.
To illustrate this in detail, we first define new optimization variables:
[Equation]
Using (5), we can express the min/max problem for t [¯ g ] ¯ b as problem (6).
Min or Max
[Equation]
subject to (cid:88)
[Equation]
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
N [1]
[Equation]
(cid:88) b ∈B
[Equation]
[Equation]
N [2]+ (cid:88) b ∈B
[Table]
By solving these subproblems and taking the objective value as bounds for t [ g ] b , we can reduce thefeasible region of the problem and enhance our solutions.
The sub-problems are bounded and if anyof them are infeasible, then it also implies that the MFOpt problem is also infeasible as we drop the
PRP constraints in these sub-problems:
E Experiment data descriptions and problem parameters
We use three primary data sources for our experiments, the more recently developed American
Community Survey (ACS) data as well as two more classical datasets, Heart Disease and COMPAS.
We elaborate on each dataset in this section.
The ACS data is a dataset made publicly available bythe US Census Bureau.
Specifically, Ding et.
al [14] have created an excellent Python package 7that enables users to pull model-ready data (for a requested year and geographic region) for a set ofpre-defined binary classification tasks, such as predicting high income, health insurance coverage,whether they move or not, among others.
The tasks are detailed in the paper and we use all of thepre-defined tasks without any additional modification except for Employment.
We do not use the
Employment task because of the assumption detailed in Section 2.2 regarding overlap.
Experimentswith this task occasionally yielded models that did not have overlap which made this task unsuitablefor demonstrating our methodology.
We reiterate that this is not a practical issue if one just ignoredthe non-overlapping bins, but requires a lengthy and technical fairness interpretation that we feltwere beyond the purpose of our study.
In terms of time and geography, we use 2020 data for allexperiments while the geography varies.
In the experiments shown on Table 1, we use the West
Coast US states (California, Oregon, Washington).
In 3 we wanted a larger dataset as we required asufficiently large testing split, hence we used the West Coast States (’CA’, ’OR’, ’WA’, ’NV’, ’AZ’)with the "ACS Mobility" dataset and a 60/40 train-test split for the inprocessing comparison and East
[Table]
[Table]
) andsame with the "ACS Poverty" dataset for the postprocessing comparison.
There was no particularreason for selecting these geographies aside from obtaining a large enough sample that we canfeasibly run multiple trials on.
Though we are using census data there is no PII information nor anyendangerment to the subjects in the data.
However, we note that in practice, it is important to exercisecaution and equity in picking groups to mitigate for, as selective mitigation of favored groups by amalicious practitioner can result in underperformance for deserving groups.
The Heart Disease Dataset ([23]) is a publicly available dataset where the task is to predict whetheror not an individual has heart disease.
Most applications of this data use the standard processed"Cleveland" data and we use sex as the group variable.
We could not find a standard and preprocessedversion of this data and did it ourselves by one-hot-encoding categorical variables.
The COMPAS dataset is based on the recidivism study noted in ([2]).
We use the preprocessed versionmade available in the publicly available AIF360 package ([4]) 8 without any additional modification.
In this dataset, the task is to predict whether or not an individual will recidivate and we use ethnicityas the group variable.
We list the problem parameters used to create the results in Table 1 in Table 4.
We use the sameparameters for all tasks.
A particular note is that in Table 1, the IPOPT method performs very poorlycompared on the COMPAS and ACS Coverage data.
This is because across the 10 trials we ran, the
IPOPT algorithm had frequently failed to converge within the 10-minute time limit for these twodatasets.
From the outputs, we saw that convergence failure is accompanied by heavy violation ofthe predictive rate parity constraint (demographic parity and equalized odds are still satisfied) and ahigh objective value.
The exact reason for frequent failure in these two datasets is unclear, however,we hypothesize that it is due to a relatively high predictive rate parity gap in the data which ledto numeric issues.
Such failures were not observed in the IP formulation while both solvers wereprovided the exact same problem parameters.
{+All experiments were run on a MacBook Pro with a 2.4GHz 8-Core Intel Core i9 processor with 
+}
GB RAM.
We did not use the GPU for solving.
Data, preprocessing steps, and the random forestmodels utilize Python’s scikit-learn ([34], BDS License) package.
The optimization model is codedthrough Julia’s JuMP package ([15] MPL License).
We use the Gurobi ([20] Academic License) and
IPOPT ([28] Eclipse Public License) solvers for all problems.
Due to lack of space, we only showed the method comparison Table 3 for the testing data.
We showthe results on the training data in Table 5.
F Comparison to other fairness definitions
We compare our bin-wise worst-case fairness definition with other fairness definitions seen inliterature and explain why it does not contradict previous impossibility theorem results.
First, sincewe are considering score bins, our definition is a generalization of the definitions in ([13]), ([21]) and([11]), which consider fairness metrics for binary { 0 , 1 } classifiers or assume that there is a thresholdfor mapping probabilities to 0,1 outcomes.
In these cases, the overall FPR/FNR can be computed and
EOdds refers to the equality of those rates.
Our framework is a generalization since the same fairnessmetrics for binary classification can be achieved by specifying that all scores be moved into exactlyone of two bins (representing {0,1} predictions) under our framework.
Since we are dealing with binned scores, our fairness definitions more resemble those seen in [25],which also has a notion of binned "risk assignments".
The critical difference in fairness definitions isthat Kleinberg’s paper utilize the sum of scores in each bin compared against the number of positiveor negative instances.
Under this scheme, predictive rate parity refers to having the sum of scores beequal to the number of positive instances and true positive rate refers to the expected score of thepositive instances in each bin (and analogously with FPR).
The key difference is that rather thanthe sum of scores, our definitions are based on the expected number of {0,1} instances moved intoeach bin, irrespective of the instance’s original scores.
As such, we are not faced with the same strictfairness trade-offs.
G Commentary on other solution methods and solvers
While investigating a solution to our nonconvex problem, we considered another global integer programming based approach known as spatial branch and branch (SBB), which relies on a combinationof spatial partitioning and solving local partitions using McCormick relaxations ([29], [9]) and otherouter approximation variants ([8]).
In our testing, Gurobi’s nonconvex QCQP solver, which appliesthese SBB heuristics, worked remarkably well despite being relatively new and was sometimes ableto beat both the interior point solution and the MIP solution.
Open-source solver SCIP ([5]) alsofeatures a gender nonconvex SBB solver that works reasonably well.
However, our main goal was toprovide a widely accessible method of solving the problem to global optimality and as of writing,there are significantly more developed open-source MILP solvers, such as SCIP, HiGHS ([22]), and
CBC ([28]), than SBB solvers.
Another reason we opted for the MILP approach is that we saw morepotential in the NMDT reformulation for taking advantage of our reformulation and bound tighteningprocedure.
Nonetheless, our bound tightening procedure is theoretically beneficial for both methodsand as other open-source algorithms/solvers for SBB become more developed, such as Couenne([28]) and Alpine ([31]), we encourage a future re-evaluation of solution methods and comparisons.
Finally, we note that we chose Gurobi in our experiments for its speed and effectiveness since weare repeatedly solving many problems.
We acknowledge that Gurobi is a very powerful commercialsolver and the results solved over 10 minutes may be worse with open source solvers such as HiGHS([22]).
Nonetheless, the important fact is that all MIP solvers target the global optimum and henceeven less powerful solvers can yield strong solutions given more time.
H Additional Experiments Using Expected Assignment on Testing Data
We list additional experiments focusing on the performance of our method on the testing data in Tables6 to 11.
All results are based on 20 trials that are run with a similar procedure in the comparisonsection 4 except that we compute the testing metrics based on expected assignment rather thanstochastic assignment (explained below), which we feel better reflects the average performance of
MFOpt.
In each trial, we tune a random forest via grid-search, find the base fairness violations, set upthe parameters of MFOpt to reduce the violations by a half, and optimize.
Then, we compute theresults (AUC and fairness violations) on the testing data (baseline with no modifications vs. MFOpt)and show the 1-Standard deviation error margins as well as the p -value corresponds to a one-sided
Wilcoxon signed rank test which evaluates if the distribution of differences of the Base - MFOpt stats
(higher AUC, lower fairness violations) is symmetric around zero (null) or instead favors the base(alternative).
The major difference in this evaluation compared to the results shown in section 4 is that in section4, we assign observations to bins in the testing data based on random draws from a multinomialdistribution (explained in Appendix B).
However, a single draw per train-test split may not properlyreflect the expected performance of MFOpt, even if we average over 20 trials.
Instead, we feel amore accurate representation of the expected performance is if we apply the expected bin assignmentto obtain the post-movement number of { 0 , 1 } and total samples for each bin in the testing set.
Under this procedure, we do not move individual observations across bins, but rather move all ofthem together.
Concretely, suppose that we have optimized parameters (ˆ x [ g ] 1 b ′ , ˆ x
[ g ] 2 b ′ , . .
, ˆ x [ g ] Bb ′ ) , whichrepresented the probabilities of observations from each bin moving into bin b ′ .
Then we propose thatthe number of {1} outcomes at bin b ′ after the expected bin assignment procedure is:
[Equation]
The same computation applies for the expected number of negative samples ˆ N [ g ] b
′ − and total samplesˆ N [ g ] b ′ , which we use to compute the AUC and fairness violations and report in the tables.
We find thatacross different datasets, the decrease in AUC is miniscule in terms of both absolute amount andvariance (less than 1%).
We obviously do not expect better AUC from the MFOpt solution comparedto the unconstrained model and thus this result is remarkable as it indicates that some degree offairness can be afforded practically for free under our framework.
The second observation is that wecan reduce all three fairness metrics simultaneously and consistently across all datasets, as we findp -values below 0.05 in all cases.
We do observe variance of the PRP violation is relatively higherthan that of DP or EOdds.
We noted this in our conclusion Section 5 as an area for future work andprovide some hypotheses for methods that can address this inconsistency.
Nonetheless, improvingconflicting definitions of fairness simultaneously is another significant result as it provides empiricalevidence that there is a path forward towards multiple fairness.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
I Role of Bins Parameter and Ablation Study
In this section, we first comment on the choice of the number of bins, |B| , as a hyperparameter andthen show results of additional testing with respect to the choice of the bin parameter.
In practice,the choice of |B| should reflect the required bin granularity of the outputs for usage in, for example,rank-ordering or threshold-based classification.
Our work therefore approaches choosing |B| purelyfrom a computational complexity perspective as the number of optimization variables scales in theorder of O ( |G||B| 2 ) .
We used |B| = 50 across our experiments as we found that this parameter gavea reasonable degree of granularity in the resulting bins while also being computationally tractableenough to solve (our MIP solver could frequently find solutions with <10% optimality gap within 10minutes).
From a more theoretical perspective, the choice of |B| determines how well we can estimatethe score transformation function, with higher |B| giving us better estimates in the training data.
However, having too few samples within each bin (in the training data) results in low bias but highvariance estimates of the score transformation function and the corresponding performance/fairnessmetrics, leading to bad generalization on the testing set.
Having too few bins can, on the other hand,lead to an under-parameterized score transformation function.
While it is clear that |B| should scalewith respect to the per-bin sample size, we consider a rigorous analysis of this choice to be out ofscope as a current limitation of our work.
We hence recommend choosing |B| in practice based onthe desired granularity of results and/or treating it as a parameter to optimize via cross-validation.
To run an experiment to study the effects of changing |B| , we must first define |B| -agnostic metrics toevaluate the post-processing output in terms of performance and fairness.
The primary difficulty isthat since fairness is defined on a bin-wise basis, evaluating the worst-case violation for example
using|B| = 10 is not comparable to |B| = 50 .
Hence, we do the following: First, after optimizing for x [ g ] ∗ bb ,we apply the linear interpolation based mapping as described in Appendix B so that we have newscores s ′ .
To compare the accuracy across different |B| , we compute the ROC and precision-recall(PR
) AUC based on s ′ .
To compare the fairness, we discretize s ′ in 100 bins (regardless of which|B| we used) and compute the worst case violation for each fainess metric across all 100 bins.
Thereason we use 100 bins to assess fairness is that we specifically use a large dataset (500k+ instancesin both training and testing) to obtain more granular metrics.
To generate the results, we use the
ACS Income data as presented in previous sections.
The difference is that we now use data acrossall states to obtain a large sample for a single random 60/40 train-test split, each with over 500k samples.
Following the procedure from before, we train and tune a random forest model, score thetraining and testing data, and proceed with our evaluation methodology.
Since the linear interpolationmapping methodology utilizes stochastic draws, we sample 300 draws per bin and compute the 1standard-deviation errors shown in the tables.
Table 12 shows the performance metrics across different |B| , where we show the average metric and1-standard deviation error margins.
We observe that performance is very similar across the differentchoices, but is maximized at the higher bin counts of of 50-60.
The performance also appears toplateau beyond a certain point, suggesting that it is not necessary to select a large |B| for performancepurposes.
Interestingly, we also see that the solutions are near-optimal based on the optimality gap.
This shows that although the number of variables scales with |B| , it does not necessarily preclude usfrom high quality solutions.
On the contrary, the optimality gap for lower bin ranges such as 25 islarger with lower performance.
This could be due to the fact that since we are optimizing over thesame ϵ fairness criteria, having fewer degrees of freedom with smaller bins makes it more difficult tofind feasible solutions.
Table 13 shows the results for fairness, where we show the average worst fairness violation andthe 1-standard deviation error margins.
The story is less clear from this angle but we emphasizethat training the problem for different |B| ̸ = 100 bins using the same ϵ parameters and evaluatingit on |B| = 100 for fairness is an unintended method of using our framework which we are onlydoing to have comparable results for the ablation study.
Here, we see that the average metrics aresimilar for different parameters with |B| =
40 having the best overall result.
Demographic parityviolation is surprisingly minimized at |B| = 30 in this example, but given the other fairness andperformance metrics |B| = 40 appears to be the best choice.
Under this method of evaluation, thefairness parameters do not appear to be very sensitive to the choice of |B| .
[Table]
[Table]