Fair Wrapping for Black-box Predictions
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We introduce a new family of techniques to post-process (“wrap") a black-boxclassiﬁer in order to reduce its bias.
Our technique builds on the recent analysisof improper loss functions whose optimization can correct any twist in prediction,unfairness being treated as a twist.
In the post-processing, we learn a wrapperfunction which we deﬁne as an ↵ -tree, which modiﬁes the prediction.
We providetwo generic boosting algorithms to learn ↵ -trees.
We show that our modiﬁcationhas appealing properties in terms of composition of ↵ -trees, generalization, interpretability, and KL divergence between modiﬁed and original predictions.
Weexemplify the use of our technique in three fairness notions: conditional value atrisk, equality of opportunity, and statistical parity; and provide experiments onseveral readily available datasets
Introduction
Machine Learning has seen a dramatic increase of its impact over the past decade – enough that it hasbecome a priority to control not just the accuracy, but also the bias of models’ outputs [1, 14, 36].
Ifwe take into account the numerous fairness targets and models that have been deﬁned and / or reﬁned[20] – sometimes excluding each other or in tension with accuracy, and the factor of energy and CO2footprint of the domain [19, 29] – then the combinatorics of training accurate and fair models looknon trivial.
A suitable trend in the ﬁeld tries to “decouple” both constraints as it seeks to post-processthe outputs of pretrained (accurate) models to achieve a more fair output [36].
Post-processing maybe the only option if we have no access to the model’s training data/algorithm/hyperparameters (etc.).
In this cluster, 3 trends emerge: learning a new fair model close to the black-box, tweaking the outputsubject to fairness constraints, and exploiting sets of classiﬁers ( § 2).
If the task is class probabilityestimation [25], the estimated black-box is an accurate but potentially unfair posterior ⌘ u : X !
[0 , 1]which neither can be opened nor trained further.
The goal is then to learn a fair posterior ⌘ f from it.
A number of relevant desiderata can be considered for post-processing, including, (ﬂexibility) oftraining to substantially different fairness metrics, (proximity) of learnt ⌘ f to original ⌘ u , meaningful(composability) if e.g. ⌘ f was later treated as a new black-box to be post-processed, (explainability)in auditing the post-processing, meaningful bounds of the post-processed classiﬁer’s (complexity)for generalization, algorithmic guarantees for (convergence) .
Our contribution explores a new solution to the post-processing problem, which learns a correctionfor fairness ↵ : X !
R and deﬁnes ⌘ f as a function of ⌘ u and ↵ .
For this objective, we use arecently introduced theory which allows to fully correct a twist in posterior prediction through theminimization of speciﬁc im proper losses – unfairness being the twist.
The approach also addressesthe desiderata from three standpoints: analytical, representation, and algorithmic.
From an analyticalstandpoint, we show that the correction yields convenient divergence bounds between ⌘ f and ⌘ u ,a convenient form for the Rademacher complexity of the class of ⌘ f , and a simple composabilityproperty.
Representation-wise, the corrections we learn are easy-to-interpret tree-shaped functionsthat we deﬁne as ↵ -trees .
Algorithmically speaking, we provide two formal boosting algorithms tolearn ↵ -trees building upon seminal results [15].
We exemplify the algorithm on three fairness metrics: conditional value at risk, equality of opportunity, and statistical parity.
Intuitively, our approachcan be summarized as a method to utilize boosting to learn a ↵ -tree to parameterize corrections toa blackbox such that it will adhere to fairness constraints.
Experiments are provided against ﬁvebaselines on readily available datasets.
All proofs and many more experiments are in an Appendixdenoted SI.
Related work
Post-processing models to achieve fairness is one out of three different categories of approaches totackle
the ML + fairness challenge [36, Section 6.2].
We focus on group fairness, which concernsitself with ensuring that the statistics of sub-population are similar.
Although other notions of fairnessexist, e.g. individual fairness [12], we limit our analysis to group fairness.
We can segment thiscluster further in three subsets: (I) approaches learning a new model with two constraints: beingclose to the pretrained model and being fair [16, 23, 32, 35]; (II) approaches biasing the output ofthe pretrained model at classiﬁcation time, modifying observations to receive a more fair outcome[1, 14, 18, 21, 34, 35]; and a last one (III) consisting of exploiting sets of models to achieve fairness[11].
None of those approaches formulates substantial guarantees on all of the desiderata in theintroduction.
Some bring contributions applicable to more than two fairness notions [7, 32, 11, 35](ﬂexibility) , two of which provide the convenience of analytic conditions on new fairness notionsto ﬁt in the approach [32, 11], but for all of them the algorithmic price-tag is unclear [7, 11] orheavily depends on convex optimization routines [32].
[1, 35] provide strong guarantees regarding(proximity) , in terms of consistency and generalization .
To our knowledge, our approach thatcorrects prediction unfairness through improper losses [30] is new.
Losses for class probability estimation
We ﬁrst refresh concepts from the literature regarding class probability estimation and the ↵ -loss.
Binary experiments and supervised learning Let X be a domain of observations, Y .
= {  1 , 1 }labels and S is a sensitive attribute in X .
We assume that the modalities of S induce a partition of
X .
( X , P) and ( X , N) are measure spaces for “positive” and “negative” observations respectively,leaving implicit the  -algebra, assumed to be the same everywhere.
( X ⇥ {  1 , 1 } , D) is is the jointdistribution following the binary task ( ⇡ , P , N) [25, Section 4], ⇡ .
= P [ Y = 1] being the prior.
( X , M)is a mixture measure space deﬁned by M .
= ⇡ · P+(1  ⇡ ) · N .
This notation can simply be intuitivelythough of as a decomposition of the joint space of features and labels.
As is often assumed in ML,sampling is i.i.d.
; we make no notational distinction between empirical and true measure to simplifyexposure as most of our results would apply for both.
Distinction shall be made when discussinggeneralization.
Finally, ⌘ 2 [0 , 1] X denotes a posterior that computes (an estimate of) P
[ Y = 1 | X ] .
Inthis paper, blue-boxed text is used to single out algorithmic nuggets with lightweight description, e.g. ,given (M , ⌘ ) , we sample wrt to D by sampling an observation ( M ) and then getting the class ( ⌘ ).
Bayes posterior admits the expression ⌘ ?
= ⇡ · dP / dM [25], and is optimal for proper losses.
Losses for class probability estimation a loss for class probability estimation , ` : Y ⇥ [0 , 1] !
R ,is expressed as ` ( y, u ) .
= J y = K · ` 1 ( u ) + J y =
K · `  1 ( u ) , where
J .
K is Iverson’s bracket [17].
Functions ` 1 , `  1 are called partial losses.
A loss is symmetric when ` 1 ( u ) =
`  1 (1  u ) , 8 u 2 [0 , 1][22] and differentiable when both partial losses are differentiable.
The pointwise conditional risk ofestimator ˆ ⌘ 2 [0 , 1] with respect to ground (unknown) truth ⌘ 2 [0 , 1] is:
[Equation]
The Bayes tilted estimate of loss ` [30] is the pointwise minimizer of (1), which can be set-valued:
[Equation]
A loss ` is proper if ⌘ 2 t ` ( ⌘ ) and strictly proper if { ⌘ } = t ` ( ⌘ ) [25].
A loss is important for ourpurpose, the ↵ -loss : it is a differentiable, symmetric loss deﬁned by the following partial losses
[Equation]
for ↵  0 and ` ( ↵ ) 1 ( u ) .
= ` (  ↵ )  1 ( u ) = ` (
[Equation] (1  u ) for ↵ < 0 .
As ↵ ! 1 , the ↵ -loss convergesto log-loss ( ` LOG 1 ( u ) .
=
log( u ) , strictly proper) and as ↵ !
1 , it converges to the 0/1-loss( ` 0/1 1 ( u ) .
=
J u < 1 /
K , proper).
If ↵ 62 { 0 , 1 } and ⌘ 6 = 1 / 2 , its Bayes tilted estimate is [30]:
[Equation]
Importantly, for any ⌘ 62 { 0 , 1 / 2 , 1 } and any ⌘ 0 2 (0 , 1) , there exists ↵ 2 R in (4) such that 87 t ` ( ⌘ ) = ⌘ 0 .
This property, called twist-properness [30], allows to correct for any twist in ⌘ .
Population loss A model that ﬁts a posterior ⌘ is trained to minimize a general population version of(1), called risk , which integrates the Bayes tilted estimate, as:
[Equation]
) where ⌘ , ⌘ ? 2 [0 , 1] X (with slight notational abuse from the pointwise case); and ⌘ ? denoting thetrue posterior ( i.e. , approximated from data), but in general can be switched for any other posterior.
If the loss is proper, such as for the log-loss, we retrieve the classical expression L ( ⌘ ; M , ⌘ ? ) =
E X ⇠ M [ L ( ⌘ ( X ) , ⌘ ?
( X ))] .
Further note that taking ⌘ = ⌘ ? results in (5) simplifying to the population/ Bayes risk.
Note: evaluating the risk using ⌘ ? can simply be approximated by taking empiricalexpectations.
We now explain how we put together the elements presented here for fairness correction.
Making black-boxes fair with guarantees
The overall recipe ⌘ u is an accurate but eventually unfair black-box.
We wish to learn a fair posterior,⌘ f , which is a function of ⌘ u , but we neither “open” nor train ⌘ u further.
We thus want a mapping
[Equation]
(6) with desirable analytical, representation, and algorithmic properties ( i.e. desiderata in Section 1).
Weadd an (invertibility) condition which states that ⌘ f must be easy to be decomposed into its originalblack-box ⌘ u and its other components.
⌘ f integrates components that need to be learned from data toachieve fairness; as such, we need an algorithm, say A :
A learns ⌘ f by minimizing a risk as:
[Equation]
where loss ` , mixture M , and “target” posterior ⌘ t are designed to satisfy a (ﬂexible) selection of fairness criteria.
Assuming that the loss is ﬁxed, for fairness one goal could be to improve the risk of the worst effectedsubgroup s .
In this case, one could pick M s the mixture M conditioned on S = s ; and reﬁning overthe data we want to predict on ⌘ ? .
Further details and speciﬁcs can be found in Section 6.
Our implementation of mapping (6) and (7) We know that the Bayes tilted estimate of the ↵ -losscan correct any posterior to a desired value, so we are going to use (4) as our implementation of (6):
[Equation]
where ↵ 2 R X ⇤ thus deﬁnes what needs to be learned to wrap ⌘ u .
There remains to pick the loss to beused in L .
We choose the log -loss for two reasons: ¨ it is strictly proper and so solving (7) “pushes”⌘ f towards target ⌘ t , ≠ it is also the ↵ -loss for ↵ =
so we are guaranteed that ⌘ f !
⌘ u is equivalentto ↵ !
1 (otherwise we might have only ( ).
We now make this more quantitative.(Proximity) guarantees under bounded fairness corrections Since posteriors ⌘ u , ⌘ f have the samesupport, a good divergence measure is an f -divergence; we pick the KL divergence [2]:
[Equation]
where D u , D f are the product measures deﬁned from M and their respective posteriors (Section 3).
Tobound (9), we introduce mild assumptions to the black-box ⌘ u and corresponding ↵ ( . ) correction (SI,
Section II presents a more general result).
The former comes as a calibration of ⌘ u ( . ) .
Assumption 1.
The black-box prediction is bounded away from the extremes: 9 B > 0 such that
[Equation]
Compliance with Assumption 1 can be done by clipping the black-box’s output with user-ﬁxed B ormaking sure it is calibrated and then ﬁnding B .
We now present setting (S1) .(S1)
Assumption 1 holds for some 0 < B  3 and function ↵ satisﬁes | ↵ ( x )  1 |  1 /B (a.s.).
This setting lead to the following bound on the distortion.
Theorem 1. (S1) implies KL ( ⌘ u , ⌘ f ; M)  ⇡ 2 / (6 ·
(2 + exp( B ) + exp(  B ))) .
To get a glimpes into the quality of the bound, ﬁx B = 3 for (S1) .
In this case, we want ↵ ( . ) 2[2 / 3 , 4 / 3] (a.s.), which is a reasonable sized interval centered at 1, the clamped black-box posterior’s
[Table] [Table]
= s
[Table]
<latexit
[Table]
>
= s
[Table]
/wBF5C3uDwhldHsfpq7764Snt2l703GRLhQkgfqJ2yT6wjxbHfK/jNWyXrXyubS++AZTtqAQyb3F8ZmT9Y2g9Hm+ujH9Y3Xm6uPNmqf8y853zlfO08dEbOj84T57mz7xw6gUOdv5y/nX+WHy8fLnvL5xX17p1a5gun81mO/wNvu3ht</latexit> TopDown 0 .1 .
u ( x ) 0 .0 .
. f
( x ) interval is approximately [0 . 04 , 0 . 96] , which is quite ﬂexible, and the distortion is upperbounded as
KL ( ⌘ u , ⌘ f ; M)  7 . 5 E  2 .
This bound, relevant to (proximity) , holds for any representation of ↵ .
Overview of (7) to make the high-level process precise, we thus look after the minimization of
[Equation]
11) with ⌘ f in (8).
(11) has a simple and popular alternative expression: plugging (8) in (11) andsimplifying yields the expression based on the logistic loss (we use M , ⌘ t to craft D t ):
[Equation]
Remark The analytical form in (8) also brings the following easy-to-check (composability) property.133 Lemma 1.
The composition of any two wrapping transformations ⌘ u ↵ 7!
⌘ f ↵7! ⌘ 0 f following (8) isequivalent to the single transformation ⌘ u ↵ · ↵ 0 7! ⌘ 0 f .
This also gives us compliance with the (invertibility) condition by wrapping ⌘ f using ↵ 0 = 1 / ↵ .
Wenow investigate the functional form of ↵ and how to train it with boosting-compliant convergence.
Alpha-trees and how to grow them
We now focus on three main components with key focus on (convergence) and additional leverageon (explainability) : the models we use for function ↵ , a convenient upperbound on (11), and, aboosting-compliant algorithm to minimize this upperbound in learning.
Our algorithm T OP D OWNends up being a generalization of standard boosting for decision tree classiﬁcation [15].
At this stage,both the mixture M and ⌘ t remain unspeciﬁed as they will depend on the fairness objective tackled.
Alpha-trees
We ﬁrst deﬁne the functions we use for ↵ .
Deﬁnition 1.
An ↵ -tree is a rooted, directed binary tree, with internal nodes labeled with observationvariables.
Outgoing arcs are labeled with tests over the nodes’ variable.
Leaves are real valued.
⇤ ( ⌥ ) is the leafset of ↵ -tree ⌥ .
Figure 1 presents an example of ↵ -tree.
Just like a decision tree, an ↵ -tree recursively splits the wholedomain X , the difference being that leaf predictions are correction to the unfair posterior, not labels.
If the splits are simple, sub- ↵ -trees can be scrutinized in an (explainable) way.
General induction of an ↵ -tree Assumption 1 is instrumental for this part. Denote ◆ ( u ) . =log( u/ (1  u )) the logit of u 2 [0 , 1] and ˜ ◆ ( u ) .
=
◆ ( u ) /B a normalization which satisﬁes˜ ◆ ( I ) = [  1 , 1] (10). Also, we deﬁne the edge of the normalized logit given M and ⌘ t ,
[Equation]
which satisﬁes e (M , ⌘ t {+) 
 [  1+} , 1] when Assumption 1 is satisﬁed. The blueprint of our algorithm, T OP D OWN , is given in Algorithm 1, where H denote a function set for splits, each element of whichis a function from X to {  1 , 1 } , +1 indicating the observation follows the right arc at the split.
T OP D OWN is similar at a high level to classical top-down decision trees induction algorithms [15,
Figure 1].
A notable low-level difference is the initial ↵ -tree provided, ⌥ 0 ; using the decision treeinduction blueprint would require ⌥ 0 to be a 1-node tree.
Another difference is the loss used forselecting split(s).
We now present this criterion, letting H ( q ) .
=
q log( q )  (1  q ) log(1  q ) .
Input mixture M posterior ⌘ t , ↵ tree ⌥ 0 , B
+ do arg
[Equation]
Output
[Table]
="
[Table]
="
[Table]
+
[Table]
+
[Table]
+
[Table] [Table]
[Table] [Table]
+
[Table]
+
[Table]
==
[Table]
> S <
[Table]
=
[Table]
=
[Table]
>
[Table]
[Table] [Table]
/h55e/ynV7woYuj2LoL0xFmlZnF4SzlL1Y1ChV0td3NBICBSD6v9+5O4rwHNL9dIYEUoXG3ne6WaSnFkzEG2f7ToS1RE3qFF9yKPz+rgbmlg2D5DsPX3dKzjoG0XBfjjQ80bqee/QLjlWxXzYe3PgMMbNlAYNW8oC2zcfE3kaIbKo5W7rynvHOSI1mzDZM6/wBF5C3uDwhldHsfpq7764Snt2l703GRLhQkgfqJ2yT6wjxbHfK/jNWyXrXyubS++AZTtqAQyb3F8ZmT9Y2g9Hm+ujH9Y3Xm6uPNmqf8y853zlfO08dEbOj84T57mz7xw6gUOdv5y/nX+WHy8fLnvL5xX17p1a5gun81mO/wNvu3ht</latexit>
[Figure]
[-(13) which satisﬁes e (M , ⌘ t ) 
-]
Deﬁnition 2.
Given ↵ -tree ⌥ with leafset ⇤ , when Assumption 1 is satisﬁed, the entropy of ⌥ is:
[Equation]
(14) where H 1 (  ; M , ⌘ t ) .
=
H ((1 + e (M  , ⌘ t )) / 2) , M  is M conditioned to leaf  2 ⇤ and M ⇤ ( ⌥ ) ismeasure induced on ⇤ ( ⌥ ) by the leaves’ weights on M .
T OP D OWN is a boosting algorithm To show that T OP D OWN is a boosting algorithm, we need a
Weak Hypothesis Assumption , which postulates informally that each chosen split brings a small edgeover random splits for a tailored distribution – locally making the problem “harder”.
Deﬁnition 3.
Let  2 ⇤ ( ⌥ ) and D t  be the product measure on X ⇥ Y conditioned on  .
Thebalanced product measure D 0 t  at leaf  is deﬁned as ( z .
=
( x , y ) for short):
[Equation]
We check that
R  dD 0 t  = 1 because of the deﬁnition of e (M  , ⌘ t ) (13).
Our balanced distributionwas named after [15]’s: ours indeed generalizes theirs.
The key difference comes from the changein setting, where we not only care about the purity of a leaf but also its agreement with the unfairposterior ⌘ u (13).
The “ fairness-free case ” where ˜ ◆ ( . ) is replaced by constant 1 yields the originalbalanced distribution [15].
We now state our Weak Hypothesis Assumption (WHA).
Assumption 2.
Let h :
X ! {  1 , 1 } be the function splitting leaf  , and let 174  > 0 .
We say that h  -witnesses the Weak Hypothesis Assumption ( WHA ) at  iff(i)
[Equation]
Intuitively, (i) gives a condition on the split h ’s correlation with unfair posterior ⌘ u agreement withlabels and (ii) does the same for the conﬁdence of ⌘ u predictions.
Similarly to the balanced distribution,if we consider the fairness-free case where we only care about the purity of splits, the WHA simpliﬁedto that of [15]’s – i.e. , only (i) matters.
A further discussion on our balanced distribution and WHA isin the SI, Section X. We now state T OP D OWN ’s boosting compliant (convergence) .
Theorem 2.
Suppose (a) Assumption 1 holds, (b) we pick the heaviest leaf to split at each iterationin Step 2.1 of T OP D OWN and (c) 9  > 0 such that each split h ?
(Step 2.2) in ⌥  -witnesses the
WHA.
Then there exists a constant c > 0 such that 8 " > 0 , if the number of leaves of ⌥ satisﬁes| ⇤ ( ⌥ ) |  (1 / " ) c log ( 1 " ) /  2 , then ⌘ f crafted from (8) using T OP D OWN ’s
⌥
achieves L ( ⌘ u ; M , ⌘ t )  " .
The proof of Theorem 2 is in SI, Section V. it proceeds in two stages, the ﬁrst being the proof that
[Equation]
17) with the scoring in (15), the second being the boosting results focused on the entropy H of the ↵ -tree.
An audacious scoring scheme for ↵ -trees Let us call conservative the scoring scheme in (15).
Thereis an alternative scoring scheme, which can lead to substantially larger corrections in absolute values,hence the naming, and yields better entropic bounds for the ↵ -tree. [Table]
+y6HzOvDOB81W83m4cg2GRQNfG8d3EaqteZePcrDXYa82YrnVCJPeo4MVoIOLVZUxbz4qWv/l4Okp3BpYrrs+sugAPx2QlQXHfSpX0CJx0lqKzcWxT2Grzl7/AK3lyWDarrzMfb09k9tflcA7PY8xfbGeWploTijkhfmx6DKrta2uLGRUCwG0/+jNLhvgDCozUtjShlbbRSGh7fy/MibgVg/t1tPNCnuUNM2ydPjNt0dTVwsY6JHs294BcdjoyieHxN62Ul9NTq0awmmmA9Gbw5IMuimNGzYWlfg33xclUVK6KqWh62rHB3nqLZsyvXIvCJzfAn5GZJLXuvF5qu/++oqHdldj95kRCUrIZtQd+sxsYGUhGhU8N3/lNRcJPy86C69h1zcNQv+rIXVInjd64/HpuNhasyvb03dHzb9h0e3dqaf7Ow
[Figure]
+Gjr9l77o+cLk7cn70xuTKaTTye3J19N7k8OJvHkdPLH5M/
[Figure]
Deﬁnition 4.
For any mixture M and posteriors ⌘ u , ⌘ t , let
[Equation]
The audacious scoring schemes at the leaves of the ↵ -tree replaces (15) in Step 3 by:
[Equation]
Theorem 3.
Suppose Assumption 1 holds and let H 2 ( q ) .
=
H ( q ) / log 2 ( 2 [0 , 1] ), H being deﬁnedin Deﬁnition 2.
For any leaf  2 ⇤ ( ⌥ ) , denote for short:
[Equation]
where let e b  .
=
e b (M  , ⌘ t ) , 8 b 2 { + ,  } .
Using audacious scoring, we get instead of (17) :
[Equation]
While upperbounds in (17) and (20) may look non comparable, it takes a simple argument to showthat (20) is never worse and can me much tighter.
Lemma 2.
8 ↵ -tree ⌥ , E  ⇠ M ⇤ ( ⌥
[Equation]
[H [Equation] ⌘ [Equation]
It thus comes at no surprise that using the audacious scoring also results in a boosting result for
T OP D OWN guaranteeing the same rates as in Theorem 2.
It also takes a simple picture to show thatthe per-leaf slack in Lemma 2 can be substantial, a slack which can be represented using a simplepicture, see Figure 3 (left), following from the use of Jensen’s inequality in the Lemma’s proof.
Conservative vs audacious corrections If we were to just care about accuracy, we would barely haveany reason to use the conservative correction.
Even thinking about generalisation, the Rademachercomplexity of decision trees is a function of their depth so the faster the convergence, the better [3,
Section 4.1] (see also Section 7).
Adding fairness substantially changes the picture: some constraints,like equality of opportunity (Section 6) can antagonise accuracy to some extent.
In such a case, usingthe conservative correction can keep posteriors ⌘ u and ⌘ t close enough (Corollary 1 / Theorem A) sothat fairness can be achieved without substantial sacriﬁce on accuracy.
A convenient initial alpha-tree Since the fairness attribute partitions the dataset, there is a simpleand convenient choice for ⌥ 0 in T OP D OWN , the stump whose test is on the fairness attribute (thus, notnecessarily binary), resulting in separate sub- ↵ -trees for each modality.
We then run T OP D OWN witha speciﬁc choice of mixture and target posterior to accommodate the fairness model at hand (Fig. 2).
Handling fairness notions
At this stage, the only remaining unknowns are the fairness-model dependent M , ⌘ t ( § 5, Par. 1).
We consider three different fairness criteria: conditional value at risk, equality of opportunity, andstatistical parity; and provide corresponding M , ⌘ t for each criteria.
The choices made for our thesefairness notions are not meant to be optimal, as other choices could provide substantial leverage, butinstead provides an illustration that our approach can be adapted to a variety of criteria.
Due to the lack of space, we defer the case of statistical parity to SI (pg 28).
Conditional value at risk CV A R was introduced in optimization / ﬁnance [26] and its use in fairnessfor ML was introduced in [33].
The criterion to minimize is:
[Equation]
L  being the risk value for the  quantile among groups, which is user deﬁned; also, M S is themeasure induced on S by the groups’ weights and M s is the mixture conditioned on S
= s .
CV A Rfocuses optimization on the worst treated groups and if we denote S  the subset of modalities usedin CV A R, then a simple way to optimize CV A R is to repeatedly grow the subtree of the ↵ -tree thatmakes the correction for one of those groups.
Put simply, we iterate
T OP D OWN with M
{+M s ( s 2 S  ) and ⌘ t ⌘ ? , 
+}
and we repeat until CV A R  ( ⌘ f ) gets below a threshold or (more speciﬁcally) its worst treated groupgets a risk below a threshold (this can be used as stopping criterion).
This imposes to update S  tokeep the set accurate between runs of T OP D OWN .
The number of iteration to get CV A R  below athreshold " in our boosting framework is thus no more than the number of modalities of S times the| ⇤ ( ⌥ ) | bound in Theorem 2.
Equality of opportunity (EOO) requires to smooth discrimination within an “advantaged” group,modeled by the label y
= 1 [14].
We say that ⌘ f achieves " -equality of opportunity iff a mapping h fof ⌘ f to Y ( e.g. using the sign of its logit) satisﬁes
[Equation]
(21) where P s is the positive observations’ measure conditioned to value S
= s for the sensitive attribute.
EOO can be antagonistic with the ﬁtting of ⌘ f to ⌘ ? : if that latter one is close to zero in a subgroupand close to one in another one, then better ﬁttings on ⌘ f can arbitrarily increase the LHS in (21).
Tocope with this issue, we do not pick ⌘ t ⌘ ?
as in CV A R, but rather skew the posterior for a subsetof observations.
Fix some s  2 arg min s 2 S P X
⇠ P s [ h f ( X )
= 1] .
Our strategy consists in skewing the 242 target posterior for S
= s  so that for a subset of the subgroup, it becomes > 1 / 2 .
A convenient useof T OP D OWN then yields more positive classiﬁcations for S
= s  – thus a more fair outcome – andthus a reduction of LHS in (21) until (21) is satisﬁed 1 .
For this, we create a ( p,  ) - pushup of ⌘ ? .
Deﬁnition 5.
Fix p 2 [0 , 1] and let X p be a subset of X such that (i) inf ⌘ ?
( X p )  sup ⌘ ?
( X \ X p )and (ii)
R
X p dM = p .
For any   0 , the ( p,  ) -pushup of ⌘ ? , ⌘ ? p,  , is the posterior deﬁned as⌘ ?
p,  = ⌘ ? if inf ⌘ ?
( X p )  1 / 2 and otherwise:
[Equation]
[Equation]
249 Fig.
3 (right) presents an example of this mapping.
Notice that the transformation can introduceclassiﬁcation mistakes with respect to ⌘ ? , but only examples with (i) small “edge” | 1 / 2  ⌘ ?
|and (ii) labeled as negative on ⌘ ? are susceptible to get positive label on ⌘ ? p,  .
Notice thetradeoff: (ii) is consistent with the fairness objective while (i) limits the degradation in accuracy.
We then run T OP D OWN using as mixture the positive measure conditioned to S
= s  andp .
= P X ⇠ P s ⇤ [ h f ( X ) = 1] + " / ( K  1) ,  .
= K " / ( K  1) , with K > 1 user-ﬁxed.
Thus, we do
T OP D OWN with M
P s  and ⌘ t ⌘ ? p,  ,and we have the following guarantee: Theorem 4. If T OP D OWN is run until L ( ⌘ f ; M , ⌘ t )  ( " 4 / 2) + E X ⇠ M [ H ( ⌘ t ( X ))] , then after therun we observe P X ⇠ P s ? [ h f ( X ) = 1]  P X ⇠ P s
[ h f ( X ) = 1]  " .
For the optimization to be carried out properly in the full context of EOO, we should not wait to get thebound on L ( ⌘ u ; M , ⌘ t ) .
Rather, we should make sure (a) we update arg min s 2 S P X ⇠ P s [ h f ( X ) = 1](and thus s  ) after each split in the ↵ -tree and (b) we keep arg max s 2 S P X ⇠ P s [ h f ( X ) = 1] as is, toprevent switching targets and eventually composing pushup transformations for the same S = s  ,which would not necessarily comply with our theory. One should note that the guarantee presentedin Theorem 4 and Section 6 depends on the mapping h f
and not the direct posterior ⌘ f as typically c1 4 7 10 13 16 19 22 25 28 31 0 .0 .0 .0 .0 .0 .
C V a R
( l o w e r b e tt e r )
CVaR c 7 10 13 16 0 .0 .0 .0 .0 .
E OO ( l o w e r b e tt e r )
Equal Opportunity c1 4 7 10 13 16 19 22 25 28 31 0 .0 .0 .0 .0 .
S P ( l o w e r b e tt e r )
Statistical Parity c1 4 7 10 13 16 19 22 25 280 .0 .0 .0 .0 .
M D ( l o w e r b e tt e r )
Posterior Mean Di  erence
[Figure]
considered [14].
When taking the mapping as a threshold of the posterior (sign of the logit), h f canbe interpreted as forcing the original posterior to be extreme values of 0 or 1.
It should be notedthat (22) requires an explicit approximation of ⌘ ? .
However, if one does not want to make such anapproximation, one can adapt the statistical parity approach (details in Appendix IX).
Similarly, ifwants to consider the typical EOO deﬁnitions depending on posterior values, the target measure canbe replaced ( i.e. , replacing the measure M with the measure of the positive examples P ).
Discussion
[Figure]
Generalization As we are utilizing the logistic loss (12), a relevant capacity notion to assess theuniform convergence of this risk for the whole wrapped model is the Rademacher (complexity) ofthe following set of functions [3]:
[Equation] where we assume known the set of functions from which ⌘ u was trained. We now assume we havea m -training sample S .
= { ( x i , y i )
⇠ D } mi =1 .
The empirical Rademacher complexity of a set offunctions H from X to R , R S ( H ) .
=
E  sup h 2 H E
i
[  i h
( x i )] (sampling uniform with  i 2 {  1 , 1 } ), 278 is a capacity parameter that yields efﬁcient control of uniform convergence when the loss used is Lipschitz [3, Theorem 7], which is the case of the logistic loss.
To see how the ↵ -tree affects
the
Rademacher complexity of classiﬁcation using ⌘ f instead of ⌘ u , suppose real-valued prediction basedon
⌘ u is achieved via logit mapping, ◆  ⌘ u (23).
Such mappings are common for decision trees [27].
Lemma 3.
Suppose { ⌘ u } is the set of decision trees of depth  d and denote R S ( DT ( d ))
theempirical Rademacher complexity of decision trees of depth  d [3] and d 0 the maximum depthallowed for ↵ -trees.
Then we have for H f in (23) :
R S ( H f )  R S ( DT ( d + d 0 )) .
The proof is straightforward once we remark that elements in H f can be represented as decision trees,where we plug at each leaf of ⌘ u a copy of the ↵ -tree ⌥ .
Sensitive feature use vs proxy-based prediction Post-processing methods have been ﬂagged in thecontext of fair classiﬁcation for the fact that they require explicit access to the sensitive feature atclassiﬁcation time [36, § 6.2.3].
Our basic approach to the induction of ↵ -trees falls in the category(Fig. 2), but there is a simple way to mask the use of the sensitive attribute and the polarity ofdisparate treatment it induces: it consists in ﬁrst inducing a decision tree to predict the sensitivefeature based on the other features and use this decision tree as ⌥ 0 in T OP D OWN .
We thus alsoredeﬁne sensitive groups based on this decision tree – thus alleviating the need to use the sensitiveattribute in the ↵ -tree.
The use of proxy sensitive attributes in a similar manner has seen ample use ina various domain such as health care [6, 5] and ﬁnance [13].
We however note that its application inpost-process and ↵ -trees may not be appropriate across all domains [8].
Experiments
To evaluate T OP D OWN , we broadly considered three datasets presenting a range of different size /feature types, Bank and German Credit (preprocessed by AIF360 [4]) and the American Community
Survey (ACS) dataset preprocessed by Folktables 2 [9].
The SI (pg 31) presents all results at length(including considerations on proxy sensitive attributes, distribution shift, and interpretability), alongwith the different black-boxes considered (random forests and neural nets).
We concentrate in the com/zykls/folktables
Section on the ACS dataset for income prediction in the state of CA and evaluate T OP D OWN ’sapplication to various fairness criteria (as per Section 6 and SI pg 28) with Random Forests (RF).
For these experiments, we consider age as a binary sensitive attribute with a bin split at 25 (a trinarymodality is deferred to the SI).
For the black-box, we consider a clipped (Assumption 1 with B = 1 )random forest (RF) from scikit-learn calibrated using Platt’s method [24].
The RF consists ofan ensemble of 50 decision trees with a maximum depth of 4 and a random selection of 10% of thetraining samples per decision tree.
Data is split into 3 subsets for black-box training, post-processingtraining, and testing; consisting of 40:40:20 splits in 5 fold cross validation.
For EOO, we utilize anout-of-the-box Gaussian Naive Bayes classiﬁer from scikit-learn to approximate ⌘ ? .
Multiple fairness criteria We evaluate T OP D OWN for CV A R, equality of opportunity EOO, andstatistical parity SP.
We have pushed to SI (Sections IX, XII) the complete treatment of SP, thatwe just summarise here.
SP aims to make subgroup’s expected posteriors similar and is popularin a various post-processing methods [32, 1].
The deﬁnition can be found in SI (pg 28) alongwith the strategy used in T OP D OWN .
For SP, we present one strategy where we target the largestsubgroup posterior (denoted by " ).
A symmetric strategy of targeting the smallest subgroup posterioris presented in SI.
Conservative and audacious updates rules are also tested.
For each of these
T OP D OWN conﬁgurations
, we boost for 32 iterations.
The initial ↵ -tree is initialized as in Fig.
To evaluate T OP D OWN , we compare against 5 baseline approaches.
For CV A R we consider thein-processing approach (I N CV
A R) presented in [33].
For EOO, we consider a derived predictor(D ER EOO) [14].
Our SP baselines include an optimized score transformation approach (OST)
[32];a derived predictor modiﬁed for SP (D ER
SP)
[14]; and a
randomized
threshold
optimizer
approach(RTO) [
The clipped black-box is also displayed (BB OX ).
The experiments are summarized in Fig.
For clarity we only plot the baselines and wrapperswhich are directly associated to each fairness criterian.
In addition, we also plot the posterior meandifference between the data and debiased posteriors MD (0/1 loss) to examine the effects on accuracy.
For CV A R , both conservative and audacious approaches decreases CV A R, which results in better
CV A R values than both the original BB OX and in-processing baseline I N CV
A R– which is good newssince I
N CV
A R directly optimizes CV A R.
We note that there are cases in which the in-processingapproach is better than ours (trinary sensitive attributes in SI), but this is expected given I N CV
A R’soptimization goal.
Interestingly, the audacious update is superior in both CV A R and MD than theconservative update.
This is also consistent for trinary sensitive attributes.
Thus, the audaciousupdate is desirable when optimizing CV A R.
Another observation is that only one sensitive attributesubgroup’s ↵ -tree is initialized (only one ‘ ⇥ ’).
This indicates that after 32 iterations the worse casesubgroup does not change in the binary case.
For EOO , there is a huge difference between conservative and audacious updates as the former getsto the most fair outcomes of all baselines.
Even if we used early stopping or pruning of the ↵ -tree(taking an earlier iterations) the audacious update would fail at producing outcomes as fair as itsconservative counterpart.
Furthermore, the audacious update comes with a signiﬁcant degradation ofaccuracy MD.
Furthermore, by looking at the iterations in which subgroup ↵ -trees are initialized, theaudacious update causes large (primarily bad) jumps in performance.
This rejoins our remark on theinterest of having a conservative update in Section 5.
When compared to D ER EOO, we ﬁnd that theconservative T OP D OWN approach produces lower EOO.
However, D ER EOO tend to have betteraccuracy scores in MD.
These observations are consistent with the trinary sensitive attribute (SI).
For SP , we can observe fairness results that can be on par with contenders for the conservativeupdate, but observe a substantial degradation of MD.
This, we believe, follows from a simple plug-ininstantiation of M , ⌘ t for the fairness notion in SI Section IX, resulting in potentially harsh updates.
In SI (pg 28), we discuss an alternative approach using ties with optimal transport.
Limitations and conclusion
Given the context of fairness, it is important to highlight possible limitations of our approach andthe potential social harm from such misuse.
We highlight two of these for our T OP D OWN approach.
Firstly, our approach has shown to have failure cases in small data cases .
This can be seen in theexperiments on German Credit dataset (SI, Section XIII-XV).
This, we believe, has a formal basisin our approach.
For instances, EOO requires accurate data posterior estimation which may bedifﬁcult in small data regimes.
Secondly, T OP D OWN is of course not unilaterally better than allother post-processing fairness approaches – there is No Free Lunch .
As such, we do not claim thatour instantiation of T OP D OWN is optimal in CV A R, EOO, or SP.
However, considering that suchdifferent fairness models instantiated in the same algorithm can lead to competitive results withthe respective state of the art, the avenue for improved instantiations or accurate extensions to newfairness constraints appears promising.
We leave these for future work.