FARE: Provably Fair Representation Learning
Abstract
Fair representation learning (FRL) is a popular class of methods that can replace the original dataset with a debiased synthetic one, which is then to be used to train fair classiﬁers.
However, recent work has shown that prior methods achieve worse accuracy-fairness tradeoffs than originally suggested, dictating the need for FRL methods that provide provable bounds on unfairness of any downstream classiﬁer, a challenge yet unsolved.
In this work we address this challenge and propose
Fairness with Restricted Encoders (FARE), the ﬁrst FRL method with provable fairness guarantees.
Our key insight is that restricting the representation space of the encoder enables us to derive fairness guarantees, while allowing empirical accuracy-fairness tradeoffs comparable to prior work.
FARE instantiates this idea with a tree-based encoder, a choice motivated by advantages of decision trees when applied in our setting.
Crucially, we develop and apply a practical statistical procedure that computes a high-conﬁdence upper bound on the unfairness of any downstream classiﬁer.
In our experimental evaluation on several datasets we demonstrate that FARE produces tight upper bounds, often comparable with empirical results of prior methods, establishing the practical value of our approach.
Introduction
It has been repeatedly shown that machine learning systems deployed in real-world applications propagate training data biases, producing discriminatory predictions that can negatively affect population subgroups [1–7].
These observations have forced regulators into action, leading to directives [8, 9] which demand parties aiming to deploy such systems to ensure fairness [10] of their predictions.
Mitigation of unfairness has become a key concern for organizations, with the highest increase in perceived relevance over the last year out of all potential risks of artiﬁcial intelligence [11, 12].
Synthetic data via fair representation learning A promising approach that attempts to address this issue is fair representation learning (FRL) [13–19]—a long line of work that preprocesses the data using an encoder f , transforming each datapoint x ∈ X into a debiased representation z .
FRL can be viewed as a form of synthetic data generation that transforms input dataset into a new, debiased dataset.
The key promise of FRL is that this debiased dataset can be given to other parties, who want to solve a prediction task without being aware of fairness (or potentially even being ﬁne with discriminating), while ensuring that any downstream classiﬁer they train on these representations has favorable fairness.
However, recent work [20, 21, 16] has demonstrated that for some FRL methods it is possible to train signiﬁcantly more unfair classiﬁers than originally claimed.
This illuminates a major drawback of all existing work—their claim about fairness of the downstream classiﬁers holds only for the models they considered during the evaluation, and does not guarantee favorable fairness of other downstream classiﬁers trained on z .
This is insufﬁcient for critical applications where fairness is enforced by regulations, leading to our key question: [Figure]
Representations
Can we create an FRL method that provably bounds the unfairness of any downstream classiﬁer?
The most prominent prior attempt to tackle this question, and the work most closely related to ours, is FNF [19]; we discuss other related work in Section 2.
Assuming two groups s = 0 and s =based on the sensitive attribute s , FNF shows that knowing the input distribution for each group can lead to an upper bound on unfairness of any downstream classiﬁer.
While this work is an important step towards provable fairness, the required assumption is unrealistic for most machine learning settings, and represents an obstacle to applying the approach in practice.
Thus, the original problem of creating FRL methods that provide fairness guarantees remains largely unsolved.
This work: provably fair representation learning We propose FARE
(Fairness with Restricted
Encoders, Fig.
1)—the ﬁrst FRL method that offers provable upper bounds on the unfairness of any downstream classiﬁer g trained on its representations, without unrealistic prior assumptions.
Our key insight is that using an encoder with restricted representations , i.e., limiting possible representations to a ﬁnite set { z 1 , . . .
, z k } , allows us to derive a practical statistical procedure that computes a highconﬁdence upper bound on the unfairness of any g , detailed in Section 4.
FARE instantiates this idea with a suitable encoder based on fair decision trees, leading to a practical end-to-end FRL method which produces debiased representations augmented with strong fairness guarantees.
More concretely, FARE takes as input the set of samples { x (1) , . .
, x ( n ) } from the input distribution
X (left in Fig. 1), and partitions the input space into k cells (middle plane, k = 3 in this example) using the decision tree encoder.
Finally, all samples from the same cell i are transformed into the same representation
z i (right).
As usual in FRL, training a downstream classiﬁer on representations leads to lower empirical unfairness, while slightly sacriﬁcing accuracy on the prediction task.
However, the main advantage of FARE comes from the fact that using a restricted set of representations allows us to, using the given samples, estimate the distribution of two sensitive groups in each cell, i.e., compute an empirical estimate of the conditional probabilities P ( s =
| z i ) and
P ( s = 1 | z i ) (solid color orange bars) for all z i .
Further, we can use conﬁdence intervals to obtain upper bounds on these values that hold with high probability (transparent bars).
As noted above, this in turn leads to the key feature of our method: a tight upper bound T on the unfairness of any g ∈ G , where G is the set of all downstream classiﬁers that can be trained on the resulting representations.
As we later elaborate on, increasing the number of samples n makes the bounds tighter.
Given the current trend of rapidly growing datasets, this further illustrates the practical value of FARE.
In our experimental evaluation in Section 5 we empirically demonstrate that on real datasets FARE produces tight upper bounds, i.e., the unfairness of any downstream classiﬁer trained on FARE representations is tightly upper-bounded, which was not possible for any of the previously proposed
FRL methods.
Moreover, these downstream classiﬁers are able to achieve comparable empirical accuracy-fairness tradeoffs to methods from prior work.
We believe this work represents a major step towards solving the important problem of generating debiased synthetic data that provably prevents training of discriminatory machine learning models.
Main contributions
The key contributions of our work are:
• A practical statistical procedure that, for restricted encoders, upper-bounds the unfairness of any downstream classiﬁer trained on their representations.
• An end-to-end FRL method FARE, that instantiates this approach with a fair decision tree encoder, and applies the said statistical procedure to augment the synthetic dataset with a tight provable upper bound on unfairness of any downstream classiﬁer.
• An extensive experimental evaluation in several settings, demonstrating favorable empirical fairness results, as well as tight upper bounds on unfairness (which were out of reach for prior work), often comparable to empirical results of existing FRL methods.
Related Work
We discuss related work on FRL, and prior attempts to obtain guarantees.
See Appendix A for an additional discussion of literature on fair decision trees and provable fairness in other settings.
FRL for group fairness Following Zemel et al.
[13] which originally introduced FRL, a plethora of different methods have been proposed based on optimization [22, 18], adversarial training [23, 24, 15, 25–29, 17], variational approaches [30, 14, 31, 32], disentanglement [33], mutual information [16, 34], and normalizing ﬂows [19, 35].
No prior method restricts representations, which is a key step in our work.
While Zemel et al.
[13] map data to prototypes , this mapping is probabilistic, thus fundamentally incompatible with our bounding procedure (see Section 4).
Towards fairness guarantees The key issue is that most of these methods produce representations that have no provable guarantees of fairness.
Concretely, this means that a machine learning model trained on the representations produced by these methods could have arbitrarily bad fairness.
In fact, prior work [36, 20, 16] has shown that methods based on adversarial training often signiﬁcantly overestimate the fairness of their representations.
While some of them derive bounds on maximum possible unfairness [37, 16, 29], these are of purely theoretical nature and cannot be exactly computed in practice.
Closest to our work is FNF [19] that can compute high-conﬁdence bounds, but critically, assumes knowledge of the input probability distribution, which is rarely the case in practice.
Our work makes no such assumption, which makes it signiﬁcantly more practical.
Preliminaries
We now set up the notation and provide the background necessary to understand our contributions.
Fair representation learning Assume data (
x , s ) ∈ R d × { 0 , 1 } from a joint probability distribution X , where each datapoint belongs to a group with respect to a sensitive attribute s .
We focus on binary classiﬁcation, i.e., given y ∈ { 0 , 1 } for each x , we aim to build g : R d → { 0 , 1 } to predict y from x .
The goal is to maximize both accuracy and fairness of g with respect to s , according to some deﬁnition.
This often implies a slight accuracy loss, as these goals are generally at odds.
A large class of methods aims to directly produce g with satisfactory fairness properties.
A different group of methods, our focus here, preprocesses data by applying an encoder f : R d → R d (cid:48) to obtain a new representation z
= f ( x , s ) of each datapoint, producing a debiased synthetic dataset.
This induces a joint distribution Z of ( z , s ) .
The downstream classiﬁer g is now trained to predict y from z , i.e., now we have g : R d (cid:48)
→ { 0 , 1 } .
The main advantage of these methods is that by ensuring fairness properties of representations z , we can limit the unfairness of any g trained on data from Z .
Fairness metric Let Z 0 and Z 1 denote conditional distributions of z where s = 0 and s = 1 , respectively.
In this work, we aim to minimize the demographic parity distance of g , reﬂecting the goal of equally likely assigning positive outcomes to inputs from both sensitive groups:
[Equation]
Our choice of metric is primarily motivated by consistency with prior work—other deﬁnitions (e.g., equalized odds) may be more suitable for a particular use-case [38], and our method can be easily adapted to support them, following the corresponding results of Madras et al.
In the remainder of this work, we will use p 0 and p 1 to denote the PDFs of Z 0 and Z 1 respectively, i.e., p
( z i ) =
P ( z i | s = 0) and p 1 ( z i ) =
P ( z i | s = 1) and p to denote the PDF of the marginal distribution of z .
Similarly, we will use q for the marginal distribution of s , and q i for the conditional distribution of s for z
= z i , i.e., q i (0) =
P ( s =
| z
= z i ) and q i (1) =
P ( s = 1 | z
= z i ) .
FARE: Provable Fairness Bounds with Restricted Encoders
We present our key contributions, the derivation of provable unfairness bounds under the assumption of restricted encoders (explained shortly), and an instantiation based on decision trees.
Optimal adversary Consider the adversary h : R d (cid:48)
→ { 0 , 1 } predicting group membership s , which aims to maximize the following balanced accuracy objective:
[Equation]
Let h (cid:63) , such that for all h , BA Z 0 , Z 1 ( h (cid:63) ) ≥ BA Z 0 , Z 1 ( h ) , denote the optimal adversary .
Intuitively, the optimal adversary predicts the group s for which the likelihood of z under the corresponding distribution ( Z 0 or Z 1 ) is larger.
More formally, h (cid:63) ( z ) =
1 { p 1 ( z ) ≥ p 0 ( z ) } , where 1 { φ } = 1 if φ holds, and 0 otherwise (see Balunovi´c et al.
[19] for a proof).
As shown in Madras et al.
[Equation]
holds for any g , i.e., we can upper-bound the unfairness of any downstream classiﬁer trained on data from Z by computing the balanced accuracy of the optimal adversary h (cid:63) .
[Equation]
CI
BA Z
[Equation]
1 ( h [Equation]
Restricted encoders Prior work is unable to utilize Eq.
(2) to obtain a fairness guarantee, as using unconstrained neural network encoders generally makes it intractable to compute the densities p 0 ( z ) and p 1 ( z ) that deﬁne the optimal adversary h ∗ .
Notably, Balunovi ´ c et al.
[19] use normalizing ﬂows, allowing computation of p 0 ( z ) and p 1 ( z ) under the assumption of knowing corresponding densities in the original distribution
X .
In contrast, we propose a class of encoders for which we can derive a procedure that can upper-bound the RHS of Eq.
(2), and thus the unfairness of g , without imposing any assumption in terms of knowledge of X .
We rely only on a set of samples ( z , s ) ∼ Z , obtained by applying f to samples ( x , s ) ∼ X , readily available in the form of a given dataset.
Namely, we hypothesize that restricting the space of representations can still lead to favorable fairness-accuracy tradeoffs.
Based on this, we propose restricted encoders f : R d → { z 1 , . . .
, z k } , i.e., encoders that map each x to one of k possible values ( cells )
z i ∈ R d (cid:48) .
As now there is a ﬁnite number of possible values for a representation, we can use samples from Z to analyze the optimal adversary h ∗ on each possible z .
Moreover, we can upper-bound its balanced accuracy on the whole distribution Z with some value S with high probability, using conﬁdence intervals (CI) (as illustrated in Fig. 2).
Finally, we can apply
Eq.
(2) to obtain the bound ∆ DP Z 0 , Z 1 ( g ) ≤ 2 S − 1 = T .
A sketch of our upper-bounding procedure follows; see Appendix C for a detailed exposition.
Upper-bounding the balanced accuracy We reformulate Eq.
(1) as follows:
[Equation]
where applications of Bayes’ rule are p
( z i ) =
q i (0) p ( z i ) /q (0) and p 1 ( z i ) =
q i (1) · p ( z i ) /q
We do not know Z , but instead have access a set D of datapoints ( z ( j ) , s ( j ) ) ∼ Z .
Further, we assume a standard setting, where D is split into a training set D train , used to train f , validation set
D val , held-out for the upper-bounding procedure (and not used in training of f in any capacity), and a test set D test , used to evaluate the empirical accuracy and fairness of downstream classiﬁers.
Using these samples, we aim to obtain an upper bound BA Z 0 , Z 1 ( h ∗ ) ≤ S that holds with conﬁdence at least 1 − (cid:15) , where (cid:15) is chosen in advance (we use (cid:15) = 0 . 05 ).
We heuristically choose a split (cid:15) =
(cid:15) b + (cid:15) c
+ (cid:15) s , and perform the upper-bounding procedure in three steps.
First, we upper-bound the base rates α [-
-]0 and α 1 with conﬁdence 1 − (cid:15) b , by applying the Clopper-Pearson binomial CI [39] (Appendix B) on D train ; this is sound as estimated probabilities are independent of the encoder f .
Second, we use the obtained upper bounds to bound the per-cell balanced accuracy of h (cid:63) , i.e., the expression max( α 0 q
i (0) , α 1 q i (1)) for each cell i , with conﬁdence 1 − (cid:15) c , again using ClopperPearson CI, this time on D eval .
Finally, we use the results of the previous steps to upper-bound the ﬁnal sum with conﬁdence 1 − (cid:15) s , applying Hoeffding’s inequality [40] (Appendix B) on samples from D test .
Finally, we obtain the desired upper bound on the DP distance of any encoder g trained on the embeddings from a restricted encoder:
[Equation]
which per union bound holds with desired error probability (cid:15) , with respect to the sampling process.
This completes the bounding procedure, enabling provable fair representation learning with no restrictive assumptions.
Our procedure can be applied to representations produced by any restricted encoder—here, we use a particular instantiation based on decision trees, that we describe next.
Restricted representations with fair decision trees The restricted encoder used in FARE is based on decision trees, a choice motivated by strong results of tree-based models on tabular data [41], as well as their feature space splitting procedure, whose discrete behavior is inherently suitable for our requirement of restricted representations.
In particular, we train a classiﬁcation tree f with k leaves, and obtain a synthetic dataset by encoding all samples that end up in leaf i to the same representation z i .
We construct z i based on the set of training examples in leaf i , taking the median value for continuous, and the most common value for categorical features (thus in our case, d (cid:48) =
d ).
To be able to obtain good empirical results, and tight bounds, we modify the vanilla decision trees in two main ways.
First, similar to Kamiran et al.
[42] and others (see Appendix A), we generalize the
Gini impurity criterion to optimize each split with respect to both fairness and accuracy (instead of just accuracy), introducing a tradeoff parameter γ .
Second, we use ordinal encoding of categorical variables and generalize the usual techniques used in this case, again to make the procedure more fairness-aware.
These changes are crucial to obtain a practical restricted encoder.
We describe both changes in more detail and discuss the hyperparameters of FARE in Appendix D.
Experimental Evaluation
Here we evaluate FARE on several datasets, and show that its fairness-accuracy tradeoffs are comparable to prior work, while for the ﬁrst time offering provable fairness bounds.
We further investigate the tightness of our bounds and provide additional experiments on transfer learning in Appendix F.
[Figure] Experimental setup We consider common fairness datasets: Health [43] and two variants of ACSIncome [44], ACSIncome-CA (only California), and ACSIncome-US (US-wide, larger but more difﬁcult due to distribution shift).
The sensitive attributes are age and sex, respectively.
We compare our method with the following recent FRL baselines (described in Section 2): LAFTR
CVIB [14], FCRL [16], FNF [19], sIPM-LFR [17], and FairPath [18].
We provide all omitted details regarding datasets, baselines, and our experimental setup, in Appendix E.
Main experiments We explore the fairness-accuracy tradeoff of each method by running it with various hyperparameters.
Each run produces representations, used to train a 1-hidden-layer neural network (1-NN) for the prediction task using a standard training procedure (same for each method), and plot its demographic parity (DP) distance and prediction accuracy.
Following Kim et al.
[17], we show a test set Pareto front for each method.
Further, for FARE we independently show a Pareto front of a 95% conﬁdence provable upper bound on DP distance (following Section 4), which is a key feature of our approach and cannot be produced by any other method.
Finally, we include an
Unfair Baseline, which uses an identity encoder.
The results on ACSIncome-CA and ACSIncomeUS are shown in Fig.
3; the results on Health are given in Fig.
6 in Appendix F.
We omit FairPath and LAFTR from the main plots (see extended results in Appendix F), as LAFTR has stability and convergence issues [16, 17], and FairPath uses a different metric to us [18].
Across all datasets, FARE can achieve a better or comparable accuracy-fairness tradeoff compared to baselines.
Crucially, other methods cannot guarantee that there is no classiﬁer with a worse DP distance when trained on their representations.
This cannot happen for our method—we produce a provable upper bound on DP distance of any classiﬁer trained on our representations.
The results indicate that our provable upper bound is often comparable to empirical values of baselines.
Finally, another advantage of FARE is its efﬁciency compared to the baselines (seconds instead of hours).
[Figure]
Exploring downstream classiﬁers In Fig.
4, we show a representative point from our main experiments on Health (Fig. 6), its fairness guarantee, and 24 diverse downstream classiﬁers (see Appendix F) trained on same representations, where half are trained to maximize accuracy, and half to maximize unfairness.
The latter (left cluster) can reach higher unfairness than initially suggested, reafﬁrming a known limitation of prior work [20, 16]: evaluating representations with some model class (here, a 1-NN) does not reliably estimate unfairness, as other classiﬁers (perhaps intentionally created by a malicious actor) might be more unfair.
This highlights the value of FARE which provides a provable unfairness upper bound—all unfairness values still remain below a known upper bound.
In Appendix F we perform the same analysis on another point from Fig.
3 (right), where since k = 6 , it is possible to enumerate all 2 6 = 64 possible classiﬁers, and directly conﬁrm that each DP distance is below the upper bound.
Interpretability Finally, another advantage of FARE is that its tree-based encoder enables direct interpretation of representations.
To illustrate this, for representations with k = 6 mentioned above we can easily ﬁnd that, for example, the representation z 6 is assigned to each person older that 24 , with at least a Bachelor’s degree, and an occupation in management, business or science.
Conclusion
We introduced FARE, a method to produce provably debiased synthetic data via fair representation learning.
The key idea was that using restricted encoders enables a practical statistical procedure for computing a provable upper bound on unfairness of downstream classiﬁers trained on these representations.
We instantiated this idea with a tree-based encoder, and experimentally demonstrated that
FARE can for the ﬁrst time obtain tight fairness bounds on several datasets, while simultaneously producing empirical fairness-accuracy tradeoffs similar to prior work which offers no guarantees.
A Additional Related Work
Here we discuss additional related work omitted from Section 2.
Provable fairness in other settings Numerous related works on provable fairness provide a different kind of guarantee or assume a different setting than ours.
First, in the setting of FRL, several methods have proposed approaches for learning individually fair representations [45–47], a different notion of fairness than group fairness which we focus on.
Prior work has also examined provable fairness guarantees in various problem settings such as ranking [48], distribution shifting [49, 50], fair classiﬁcation with in-processing [51, 52], individually fair classiﬁcation with postprocessing [53], and fair meta-learning [54].
These are all different from our setting, which is FRL for group fairness.
Fair decision trees The line of work focusing on adapting decision trees to fairness concerns includes a wide range of methods which differ mainly in the branching criterion.
Common choices include variations of Gini impurity [42, 55, 56], mixed-integer programming [57, 58] or AUC [59], while some apply adversarial training [60, 61].
Further, some works operate in a different setting such as online learning [56] or post-processing
The only works in this area that offer provable fairness guarantees are Ranzato et al.
[61], which certiﬁes individual fairness for post-processing, and Meyer et al.
[63], which certiﬁes that predictions will not be affected by data changes.
This fundamentally differs from our FRL setting where the goal is to certify fairness of any downstream classiﬁer.
B Mathematical Tools
Here we formally state the Hoeffding’s inequality and the Clopper-Pearson binomial conﬁdence intervals, used in our upper-bounding procedure in Section 4.
Hoeffding’s inequality [40] : Let X (1) , . .
, X ( n ) be independent random variables such that
P ( X ( j ) ∈ [ a ( j ) , b ( j ) ]) = 1 .
Let ˆ µ = X
(1) + ...X ( n ) n and µ = E [ˆ µ ] .
It holds that:
P
[Equation]
Clopper-Pearson binomial proportion conﬁdence intervals [39] : Assume a binomial distribution with an unknown success probability θ .
Given m successes out of n experiments, it holds that:
[Equation]
with conﬁdence at least 1 − α over the sampling process, where B ( p ; v, w ) denotes the p -th quantile of a beta distribution with parameters v and w .
C Detailed Description of the Upper-bounding Procedure
In this section we expand on the overview given in Section 4 and provide a detailed presentation of our practical statistical procedure used to upper-bound the unfairness of downstream classiﬁers trained on embeddings from a restricted encoder.
Recall from Section 4 that we aim to upper bound the following quantity with high probability, using samples from Z :
[Equation]
The expression above can be interpreted as the prior-weighted (i.e., weighted by p ( z i ) ) per-cell balanced accuracy (i.e., max( α 0 q
i (0) , α 1 q i (1)) for each cell i ), where we deﬁne α 0 = 1 / 2 q (0) and α 1 = 1 / 2 q (1) .
Next, we introduce 3 lemmas, and later combine them to obtain the desired upper bound.
We use
B ( p ; v, w ) to denote the p -th quantile of a beta distribution with parameters v and w .
Note that for
Lemma 1 we do not use the values z ( j ) in the proof, but still introduce them for consistency.
Lemma 1 (Bounding base rates) .
Given n independent samples { ( z (1) , s (1) ) , . . .
, ( z ( n ) , s ( n ) ) } ∼ Z and a parameter (cid:15) b , for α
and α 1 as deﬁned above, it holds that
[Equation]
with conﬁdence 1 − (cid:15) b , where m = (cid:80) nj =1 1 { s ( j ) = 0 } .
Proof.
We deﬁne n independent Bernoulli random variables X ( j ) :
= 1 { s ( j ) = 0 } with same unknown success probability q (0) .
Using the Clopper-Pearson binomial CI [39] (Appendix B) to estimate the probability q (0) we get P ( q (0) ≤ B ( (cid:15) b 2 ; m, n − m + 1)) ≤ (cid:15) b / 2 and
P ( q (0) ≥ B (1 − (cid:15) b 2 ; m + 1 , n − m )) ≤ (cid:15) b / 2 .
Substituting q (0) = 1 − q (1) in the latter, as well as the deﬁnitions of α 0 and α 1 in both inequalities, produces the inequalities from the lemma statement, which per union bound simultaneously hold with conﬁdence 1 − (cid:15) b .
Lemma 2 (Bounding balanced accuracy for each cell) .
Given n independent samples { ( z (1) , s (1) ) , . . .
, ( z ( n ) , s ( n ) ) } ∼ Z , parameter (cid:15) c , and constants ¯ α 0 and ¯ α 1 such that α 0 < ¯ αand
α 1 < ¯ α 1 , it holds for each cell i ∈ { 1 , . . .
, k } , with total conﬁdence 1 − (cid:15) c ,
that
[Equation]
where t i = max (cid:0) ¯ α 0 B ( (cid:15) c
2 k ; m i , n i − m i + 1) , ¯ α 1 (1 − B (1 − (cid:15) c
2 k ; m i + 1 , n i − m i )) (cid:1) .
In this expression, n i = | Z i | , and m i
= (cid:80)
j ∈ Z i 1 { s ( j ) = 0 } , where we denote Z i
= |{ j | z ( j ) =
z i }| .
Proof.
As in Lemma 1, for each cell we use the Clopper-Pearson CI to estimate q i (0) with samples indexed by Z i and conﬁdence 1 − (cid:15) c /k .
As before, we apply q i (0)
=
− q i (1) to arrive at a set of k inequalities of the form Eq.
(5), which per union bound jointly hold with conﬁdence 1 − (cid:15) c .
Lemma 3 (Bounding the sum) .
Given n independent samples { ( z (1) , s (1) ) , . . .
, ( z ( n ) , s ( n ) ) } ∼ Z , where for each j ∈ { 1 , . . . , n } we deﬁne a function idx ( z ( j ) )
=
i such that z ( j )
=
z i (cell index), parameter (cid:15) s , and a set of real-valued constants { t 1 , . . .
, t k } , it holds that
[Equation] ≥ 1 − (cid:15) s , where S = 1 n n (cid:88)
j = t idx ( z ( j ) )
+ ( b − a ) (cid:114) − log (cid:15) s 2 n , (6) and we denote a = min { t 1 , . .
t k } and b = max { t 1 , . .
t k } .
Proof.
For each j let X ( j ) := t idx ( z ( j ) ) denote a random variable.
As for all j , X ( j ) ∈ [ a, b ] with probability 1 and X ( j ) are independent, we can apply Hoeffding’s inequality [40] (restated in Appendix B) to upper-bound the difference between the population mean (cid:80) ki =1 p ( z i ) t i =
E z ∼Z t idx ( z ) and its empirical estimate 1 n (cid:80) nj =1 X ( j ) .
Setting the upper bound such that the error is below (cid:15) s directly recovers S and the statement of the lemma.
Applying the lemmas Finally, we describe how we apply the lemmas in practice to upper-bound
BA Z 0 , Z 1 ( h (cid:63) ) , and in turn upper-bound ∆ DP Z 0 , Z 1 ( g ) for any downstream classiﬁer g trained on representations learned by a restricted encoder.
We assume a standard setting, where a set D of datapoints { ( x ( j ) , s ( j ) ) } from X is split into a training set D train , used to train f , validation set D val , held-out for the upper-bounding procedure (and not used in training of f in any capacity), and a test set D test , used to evaluate the empirical accuracy and fairness of downstream classiﬁers.
After training the encoder and applying it to produce representations ( z ( j ) , s ( j ) ) ∼ Z for all three data subsets, we aim to derive an upper bound on ∆ DP Z 0 , Z 1 ( g ) for any g , that holds with conﬁdence at least 1 − (cid:15) , where (cid:15) is the hyperparameter of the procedure (we use (cid:15) = 0 . 05 ).
To this end, we heuristically choose some decomposition (cid:15) =
(cid:15) b + (cid:15) c
+ (cid:15) s , and apply Lemma 1 on D train to obtain upper bounds α 0 < ¯ α 0 and α 1 < ¯ α 1 with error probability (cid:15) b .
As mentioned above, using D train in this step is sound as estimated probabilities q (0) and q (1) are independent of the encoder f .
Next, we use ¯ α 0 , ¯ α 1 and D val in Lemma 2, to obtain upper bounds t 1 , . . .
, t k on per-cell accuracy that jointly hold with error probability (cid:15) c .
Finally, we upper-bound the sum (cid:80) ki =1 p ( z i ) t i ≤ S with error probability (cid:15) s using Lemma 3 on D test with previously computed t 1 , . . .
, t k .
Combining this with
Eq.
(2) ﬁnally gives the desired upper bound, that per union bound holds with conﬁdence 1 − (cid:15) :
[Equation]
Z 0 ,
[Equation]
D Fair Decision Trees as Restricted Encoders
Here we expand on our description of a decision tree as a restricted encoder given in Section 4.
We start by recalling the background on decision trees and proceed to describe the two main additional components used in FARE.
Vanilla classiﬁcation trees Starting from the training set D root of examples (
x , y ) ∈
R
d ×{ 0 , 1 } , a binary classiﬁcation tree f repeatedly splits some leaf node P with assigned D P , i.e., picks a split feature j ∈ { 1 , . . .
, d } and a split threshold v , and adds two nodes L and R as children of P , such that D L
= { ( x , y ) ∈
D P | x j ≤ v } and D R = D P \ D L .
j and v are picked to minimize a chosen criterion, weighted by | D L | and | D R | , aiming to produce leaves where the distribution of y is highly unbalanced.
We focus on Gini impurity, computed as Gini y ( D ) =
2 p y (1 − p y ) ∈ [0 , 0 . 5] where p y =
(cid:80) (
x ,y ) ∈
D 1 { y = 1 } / | D | .
At inference, a test example x is propagated to a leaf l , and we predict the majority class of D l .
Fairness-aware criterion Using a tree-based encoder that utilizes one of the common splitting criteria focused on accuracy (such as Gini y ( D ) ) generally leads to high unfairness, making it necessary to introduce a direct way to prioritize more fair tree structures.
To this end, similar to Kamiran et al.
[42] and others (see discussion in Appendix A), we use the criterion
FairGini ( D ) =
(1 − γ )
Gini y ( D ) + γ (0 .
5 − Gini s ( D ))
∈ [0 , 0 .
5] , where Gini s is deﬁned analogously to Gini y .
The second term aims to maximize Gini s ( D ) , i.e., make the distribution of s in each leaf i as close to uniform (making it challenging for the adversary to infer the value of s based on z i ), while the hyperparameter γ controls the accuracy-fairness tradeoff.
Fairness-aware categorical splits Further, while usual splits of the form x j ≤ v are suitable for continuous , they are inefﬁcient for categorical (usually one-hot) variables, as only 1 category can be isolated.
Consequently, this increases the number of cells and makes our fairness bounds loose.
Instead, we represent n j categories for feature j as integers c ∈ { 1 , 2 , ..., n j } .
To avoid evaluating all 2 n j − 1 possible partitions, we sort the values by p y ( c ) =
(cid:80) ( x
,y ) ∈
D c 1 { y = 1 } / | D c | where
D c = { x ∈ D | x j = c } , and consider all preﬁx-sufﬁx partitions ( Breiman shortcut ).
This ordering focuses on accuracy and is provably optimal for FairGini ( D ) with γ =
However, as it ignores fairness, it is inefﬁcient for γ > 0 .
To alleviate this, we generalize the Breiman shortcut, and explore all preﬁx-sufﬁx partitions under several orderings.
Namely, for several values of the parameter q , we split the set of categories { 1 , 2 , . . . , n j } in q -quantiles with respect to p s ( c ) (deﬁned analogous to p y ( c ) ), and sort each quantile by p y ( c ) as before, interspersing q obtained arrays to obtain the ﬁnal ordering.
Note that while this offers no optimality guarantees, it is an efﬁcient way to consider both objectives, complementing our fairness-aware criterion.
Hyperparameters There are four main hyperparameters of FARE:
γ (used for the criterion, where larger γ puts more focus on fairness), ¯ k (upper bound for the number of leaves), n i (lower bound for the number of examples in a leaf), and v (the ratio of the training set to be used as a validation set).
Note that all parameters affect accuracy, empirical fairness, and the tightness of the fairness bound.
For example, larger n i is likely to improve the bound by making Lemma 2 tighter, as more samples can be used for estimation.
For the same reason, increasing v improves the tightness of the bound, but may slightly reduce the accuracy as fewer samples remain in the training set used to train the tree.
[Table]
E Details of Main Experiments
In this section we provide details of our main experimental evaluation omitted from the main text.
Datasets As mentioned in Section 5, we perform our experiments on ACSIncome [44] and
Health [43] datasets.
In Table 1 we show some general statistics about the datasets: size of the training and test set, base rate for the sensitive attribute s (percentage of the majority group out of the total population), and base rate for the label y (accuracy of the majority class predictor).
ACSIncome is a dataset recently proposed by Ding et al.
[44] as an improved version of UCI Adult, with comprehensive data from US Census collected across all states and several years (we use 2014).
The task is to predict whether an individual’s income is above $50,000, and we consider sex as a sensitive attribute.
We evaluate our method on two variants of the dataset: ACSIncome-CA, which contains only data from California, and ACSIncome-US, which merges data from all states and is thus signiﬁcantly larger but also more difﬁcult, due to distribution shift.
10% of the total dataset is used as the test set.
We also use the Health dataset [43], where the goal is to predict the Charlson
Comorbidity Index, and we consider age as a sensitive attribute (binarized by thresholding atyears).
For this dataset perform the same preprocessing as Balunovi ´ c et al.
[19], and use 20% of the total dataset as the test set.
Evaluation procedure For our main experiments, as a downstream classiﬁer we use a 1-hiddenlayer neural network with hidden layer size 50 , trained until convergence on representations normalized such that their mean is approximately 0 and standard deviation approximately 1 .
We train the classiﬁer 5 times and in our main ﬁgures report the average test set accuracy, and the maximal DP distance obtained, following the procedure of Gupta et al.
Hyperparameters For baselines, we follow the instructions in respective writeups, as well as
Gupta et al.
[16] to densely explore an appropriate parameter range for each value (linearly, or exponentially where appropriate), aiming to obtain different points on the accuracy-fairness curve.
For CVIB, we explore λ ∈ [0 .
01 , 1] and β ∈ [0 .
For FCRL on ACSIncome we explore λ = β ∈ 01 , 2] , and for Health λ ∈ [0 . 01 , 2] and β = 0 .
5 λ .
For FNF, we explore γ ∈ [0 , 1] .
For sIPM-LFR, we use λ ∈ [0 . 0001 , 1 . 0] and λ F ∈ [0 . 0001 , 100 . 0] , extending the suggested ranges.
For
FairPath we set the parameter κ ∈ [0 , 100] .
Finally, for LAFTR we use g ∈ [0 . 1 , 50] , extending the range of [0 , 4] suggested by [16].
We adjust the parameters for transfer learning whenever supported by the method.
Regarding FARE (see description of hyperparameters in Appendix D), we investigate γ ∈ [0 , 1] , k ∈ [2 , 200] , n i ∈ [50 , 1000] , v ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 5 } .
For the upper-bounding procedure, we always set (cid:15) = 0 .
05 , (cid:15) b = (cid:15) s = 0 .
005 , and thus (cid:15) c
= 0 .
Finally, when sorting categorical features as described in Appendix D, we use q ∈ { 1 , 2 , 4 } in all cases.
F Additional Experimental Results
In this section we provide additional experimental results omitted from the main text.
For main experiments, we provide results on the Health dataset and with two additional methods.
Further, we provide an additional investigation of downstream classiﬁers, results of our experiment on the relationship of data and bound tightness, and a set of experiments on transfer learning.
Main experiments In Fig.
5 and Fig.
6 we provide the extended results of our main experiments, including another dataset (Health), and two originally excluded methods, LAFTR and FairPath.
[Figure]
Exploring downstream classiﬁers We provide additional info on the experiment with downstream classiﬁers given in Fig. [Figure] and repeat a similar experiment in a different setting
[Figure]
Namely, for Fig.
4 we explored the following classiﬁers: (i) 1-hidden-layer neural network (1-NN) with hidden layer sizes 50 and 200 , (ii) 2-NN with hidden layers of size (50 , 50) , as well as (200 , 100) , (iii) logistic regression, (iv) random forest classiﬁer with 100 and 1000 estimators, (v) decision tree with 100 and an unlimited number of leaf nodes.
We trained all these classiﬁers with a standardization preprocessing step as described above.
We further trained one variant of 1-NN, 2-NN, random forest, and logistic regression, on unnormalized data.
All described models were trained both to predict the task label y , and to maximize unfairness, i.e., predict s , leading to 24 evaluated models.
Next, in a similar vein, we explore a point from Fig.
3 (right), with accuracy 75 .
1% and DP distance of 0 .
As opposed to our previous experiment, here we have k = 6 , i.e., the possible representations are { z 1 , . . .
, z 6 } , thus the previous investigation of downstream classiﬁers simpliﬁes.
Instead of choosing a model class, we can enumerate all 2 6 = 64 possible classiﬁers, and directly conﬁrm that each DP distance is below the upper bound, as shown in
Fig.
7 (left).
Note that in the original experiment, all baseline methods have DP distance ≥ 0 . 04 at similar accuracy of ≈ 75% , implying that the FARE bound is in this case very tight.
Data improves bounds As mentioned in the main text, we investigate the effect of increased dataset size on bound tightness.
Namely, we choose a representative set of FARE points from
Fig.
3 (left), and repeat the upper-bounding procedure with the dataset repeated M times, showing the resulting upper bounds for M ∈ { 2 , 4 , 8 , 16 , 32 } in Fig.
7 (right).
We can clearly observe a signiﬁcant improvement in the provable upper bound for larger dataset sizes.
Transfer learning Finally, we analyze the transferability of learned representations across tasks.
We produce a diverse set of representations on the Health dataset with each method, and following the procedure from prior work [15, 19, 17] evaluate them on ﬁve unseen tasks y , where for each the goal is to predict a certain primary condition group.
For each task and each method, we identify the highest accuracy obtained while keeping ∆ DP Z 0 , Z 1 not above a certain threshold.
Moreover, we show T , the provable DP distance upper bound of FARE.
The results are shown in Table 2.
First, we observe that some methods are unable to reduce ∆ DP Z 0 , Z 1 below the given threshold.
Our method can always reduce the ∆ DP Z 0 , Z 1 sufﬁciently, but due to our restriction on representations which enables provable upper bounds, we often lose more accuracy than other methods for high ∆ DP Z 0 , Z 1 thresholds.
Future work could focus on investigating alternative restricted encoders with better fairness-accuracy tradeoffs in the transfer learning setting.
[Figure]
y ∆ DP Z
[Figure]
T
FARE
FCRL
FNF sIPM
MIS ≤ 0.
≤ 0. ≤ 0.
≤ 0. ≤ 0.
NEUMENT ≤ 0.
≤ 0.
≤ 0. ≤ 0.
/ ≤ 0.
ARTHSPIN ≤ 0.
≤ 0. ≤ 0.
/ ≤ 0. / ≤ 0.
METAB ≤ 0.
/ ≤ 0. / ≤ 0.
/ ≤ 0. / ≤ 0.
MSC2a ≤ 0.
≤ 0.
/ ≤ 0.
/ / ≤ 0. / / ≤ 0.