Beyond Adult and COMPAS: Fair Multi-Class Prediction via Information Projection
Abstract
We consider the problem of producing fair probabilistic classifiers for multi-class classification tasks.
We formulate this problem in terms of “projecting” a pretrained (and potentially unfair) classifier onto the set of models that satisfy target group-fairness requirements.
The new, projected model is given by post-processing the outputs of the pre-trained classifier by a multiplicative factor.
We provide a parallelizable, iterative algorithm for computing the projected classifier and derive both sample complexity and convergence guarantees.
Comprehensive numerical comparisons with state-of-the-art benchmarks demonstrate that our approach maintains competitive performance in terms of accuracy-fairness trade-off curves, while achieving favorable runtime on large datasets.
We also evaluate our method at scale on an open dataset with multiple classes, multiple intersectional groups, and over 1M samples.
Introduction
Machine learning (ML) algorithms are increasingly used to automate decisions that have significant social consequences.
This trend has led to a surge of research on designing and evaluating fairness interventions that prevent discrimination in ML models.
When dealing with group fairness , fairness interventions aim to ensure that a ML model does not discriminate against different groups determined by, for example, race, sex and/or nationality.
Extensive comparisons between discrimination control methods can be found in [BDH + 18, FSV + 19, WRC21].
As these studies demonstrate, there is still no “best” fairness intervention for ML, and the majority of existing approaches are tailored to either binary classification tasks, binary population groups, or both.
Moreover, discrimination control methods are often tested on overused datasets of modest sizes collected in either the US or Europe (e.g., UCI Adult [Lic13] and COMPAS [ALMK16]).
Most fairness interventions in ML focus on binary outcomes.
In this case, the classification output is either positive or negative, and group-fairness metrics are tailored to binary decisions [HPS16].
While binary classification covers a range of ML tasks of societal importance (e.g., whether to approve a loan, whether to admit a student), there are many cases where the predicted variable is not binary.
For example, in education, grading algorithms assign one out of several grades to students.
In healthcare, predicted outcomes are frequently not binary (e.g., severity of disease).
We introduce a theoretically-grounded discrimination control method called FairProjection .
This method ensures group fairness in multi-class classification for several, potentially overlapping population groups.
We consider group fairness metrics that are natural multi-class extensions of their binary
See Related Work and Table 1 for notable exceptions.
classification counterparts, such as statistical parity [FFM + 15], equalized odds [HPS16], and error rate imbalance [PRW + 17, Cho17].
When restricted to two predicted classes, FairProjection performs competitively against state-of-the-art fairness interventions tailored to binary classification tasks.
FairProjection is model-agnostic (i.e., applicable to any model class) and scalable to datasets that are orders of magnitude larger than standard benchmarks found in the fair ML literature.
Our approach is based on an information-theoretic formulation called information projection .
We show that this formulation is particularly well-suited for ensuring fairness in probabilistic classifiers with multi-class outputs.
Given a probability distribution P and a convex set of distributions P , the goal of information projection is to find the “closest” distribution to P in P .
The study of information projection can be traced back to [Csi75], which used KL-divergence to measure “distance” between distributions.
Since then, information projection has been extended to other divergence measures, such as f -divergences [Csi95] and Rényi divergences [KS16, KS15].
Recently, [AAW + 20] studied how to project a probabilistic classifier, viewed as a conditional distribution, onto the set of classifiers that satisfy target group-fairness requirements.
Remarkably, the projected classifier is obtained by multiplying (i.e., post-processing) the predictions of the original classifier by a factor that depends on the group-fairness constraints.
Prior work on information projection relies on a critical—and limiting—information-theoretic assumption: the underlying probability distributions are known exactly .
This is infeasible in practical
ML applications, where only a set of training examples sampled from the underlying data distribution is available.
FairProjection fills this gap by using an efficient algorithm for computing the projected classifier with finite samples.
We establish theoretical guarantees for this algorithm in terms of convergence and sample complexity.
Notably, our proposed fairness intervention is parallelizable (e.g., on a GPU).
Hence,
FairProjection scales to datasets with the number of samples comparable to the population of many US states ( > 10 6 samples).
We provide a TensorFlow [AAB + 15] implementation of
FairProjection and apply it to post-process the outputs of probabilistic classifiers to ensure group fairness.
We benchmark our post-processing approach against several state-of-the-art fairness interventions selected based on the availability of reproducible code, and qualitatively compare it against many others.
Our numerical results are among the most comprehensive comparisons of fairness interventions to date.
We present performance results on the HSLS (High School Longitudinal Study, used in [JWC22]), Adult [Lic13], and COMPAS [ALMK16] datasets.
We also evaluate FairProjection on a dataset derived from open and anonymized data from Brazil’s national high school exam—the Exame Nacional do Ensino Médio (ENEM)—with over 1 million samples.
We made use of this dataset due to the need for large-scale benchmarks for evaluating fairness interventions in multi-class classification tasks.
We also answer recent calls [BZZ + 21,
DHMS21] for moving away from overused datasets such as Adult [Lic13] and COMPAS [ALMK16].
We hope that the ENEM dataset encourages researchers in the field of fair ML to test their methods within broader contexts.
In summary, our main contributions are: (i)
We introduce a post-processing fairness intervention for multi-class classification problems that can account for multiple protected groups and is scalable to large datasets; (ii) We derive finite-sample guarantees and convergence-rate results for our post-processing method.
Importantly, FairProjection makes information projection practical without requiring exact knowledge of probability distributions; (iii) We demonstrate the favourable performance of our approach through comprehensive benchmarks against state-of-the-art fairness interventions; (iv) We put forth a new large-scale dataset (ENEM) for benchmarking discrimination control methods in multi-class classification tasks; this dataset may encourage researchers in fair ML to evaluate their methods beyond Adult and COMPAS.
Related work.
We summarize key differentiating factors from prior work in Table 1 and provide a more in-depth discussion in Appendix A.2.5.
The fairness interventions that are the most similar to ours are the FairScoreTransformer [WRC20, WRC21, FST] and the pre-processing method in [JN20].
[Table]
The FST and [JN20] can be viewed as instantiations of FairProjection when restricted to the binary classification setting and to cross-entropy (for FST) or KL-divergence (for [JN20]) as the f -divergence of choice.
Thus, our approach is a generalization of both methods to multiple fdivergences.
Importantly, unlike our method, [JN20] requires retraining a classifier multiple times.
A reductions approach for fair classification was introduced in [ABD + 18].
When restricted to binary classification, the benchmarks in Section 5 indicate that the reductions approach consistently achieves the most competitive fairness-accuracy trade-off compared to ours.
FairProjection has two key differences from [ABD + 18]: it is not restricted to binary classification tasks and does not require refitting a classifier several times over the training dataset.
These are also key differentiating points from [CHKV19], which presented a meta-algorithm for fair classification that accounts for multiple constraints and groups.
The reductions approach was later significantly generalized in the
GroupFair method by [YCK20] to account for overlapping groups and multiple predicted classes.
Unlike [YCK20], we do not require retraining classifiers.
Several other recent fairness intervention methods consider optimizing accuracy under group-fairness constraints.
In [CJG + 19], a “proxy-Lagrangian” formulation was proposed for incorporating nondifferentiable rate constraints, including group fairness constraints.
We avoid non-differentiability issues by considering the probabilities (scores) at the output of the classifier instead of thresholded decisions.
In [ZVRG17], a fairness-constrained optimization was introduced that is applicable to margin-based classifiers (our approach can be used on any probabilistic classifier).
In [CDPF + 17] and [MW18], the fairness-accuracy trade-offs in binary classification tasks are characterized when the underlying distributions are known.
A non-parity-based fairness notion was proposed in [KGZ19], called “multiaccuracy,” which aims to ensure high accuracy for all subgroups even when the group information is not given in the data.
We limit our analysis to parity notions of group fairness.
To circumvent the non-differentiability of group-fairness constraints, approximate fairness constraints based on functionals found in information theory have been explored in [LPB + 21, Rényi mutual information], [BNBR19, Rényi maximal correlation], and [PQC + 19, maximum mean discrepancy].
We avoid such non-differentiability issues by casting group fairness constraints in the score domain.
[Table]
Notation.
Boldface Latin letters will always refer to vectors or matrices.
The entries of a vector z are denoted by z j , and those of a matrix G by G i,j .
The all-1 and all-0 vectors are denoted byand 0 .
We set [ N ] ≜ { 1 , · · · , N } and R + ≜ [0 , ∞ ) .
The probability simplex over [ N ] is denoted by ∆ N ≜ { p ∈ R N + ; 1 T p = 1 } , and ∆ + N is its (relative) interior.
If P is a Borel probability measure over R N , Z ∼ P is a random variable, and f : R N → R K is Borel, then the expectation of f ( Z ) is denoted by E [ f ( Z )]
= E P [ f ] = E P [ f ( Z )]
=
E Z ∼ P [ f ( Z )] .
We use the standard asymptotic notations O, Θ , and Ω .
Problem formulation and preliminaries
Classification tasks.
The essential objects in classification are the input sample space X , the predicted classes Y , and the classifiers.
We fix two random variables X and Y , taking values in sets X and Y ≜
[ C ] .
Here, ( X, Y ) is a pair comprised of an input sample and corresponding class label randomly drawn from X × Y with distribution P X,Y .
A probabilistic classifier is a function h : X → ∆ C , where h c ( x ) represents the probability of sample x ∈ X falling in class c ∈ Y .
Thus, h gives rise to a Y -valued random variable (cid:98) Y via the distribution P (cid:98) Y | X
= x ( c ) ≜ h c ( x ) .
Group-fairness constraints.
Let S be a group attribute (e.g., race and/or sex), taking values in S ≜
[ A ] .
We consider multi-class generalization of three commonly used group fairness criteria in Table 2.
As observed by existing works [see, e.g., ABD + 18, MW18, CHKV19,
WRC20, AAW + 20], each of these fairness constraints 6 can be written in the vector-inequality form E P X [ Gh ] ≤ 0 for a closed-form matrix-valued function G : X → R K × C .
For instance, for statistical parity, the G matrix evaluated at a fixed individual x ∈ X has K = 2 AC rows indexed by ( δ, a, c ′ ) ∈ { 0 , 1 } ×
[ A ] × [ C ] , where the ( δ, a, c ′ )
-th row is equal to (cid:16) ( − 1) δ P S ( a ) − 1 (cid:80) c
∈ [ C ] P S | X = x,Y = c ( a ) h base c ( x ) − ( α + ( − 1) δ ) (cid:17)
e c ′ , with e 1 , · · · , e C denoting the standard basis for R C .
The expressions for the G matrix corresponding to the other fairness metrics are given in Appendix A.1.8, with a detailed derivation of statistical parity in Appendix A.1.9.
Note that G depends on P S | X,Y .
If the group attribute S is part of the input feature X , then P S | X,Y is simply replaced with an indicator function.
Otherwise, we approximate this conditional distribution by training a probabilistic classifier.
Goal.
Our goal is to design an efficient post-processing method that takes a pre-trained classifier h base that may violate some target group-fairness criteria and finds a fair classifier that has the most similar outputs (i.e., closest utility performance) to that of h base .
Fairness through information-projection.
We formulate the fairness intervention problem as follows.
For a fixed search space H ⊂ ∆ X C ≜ { h : X → ∆ C } , a loss function err : ∆ X C × ∆ X C
→ R , and a base classifier h base ∈ ∆ X C , one seeks to solve:
[Equation]
The function err quantifies the “closeness” between the scores given by h and h base .
The constraint on h can encode any arbitrary statistical information about the joint distribution induced on the pair ( X, (cid:98) Y ) .
Specifically, any constraint E P X, (cid:98) Y
[ g ( X, (cid:98) Y )] ≤ 0 , where g : X ×
[ C ] → R K , may be recast in the form (1).
Thus, solving the optimization (1) amounts to finding the minimal necessary perturbation to the base classifier h base to make it satisfy a given on-average constraint.
Since we consider raw output scores, we measure “closeness” via f -divergences:
[Equation]
where f is a convex function over (0 , ∞ ) .
By varying different choices of f , we can obtain e.g., cross-entropy (CE, f ( t ) =
− log t ) and KL-divergence ( f ( t ) =
t log t ).
For a chosen f -divergence, the optimization problem (1) becomes a generalization of information projection [Csi75].
Preliminaries on information-projection.
In a recent work [AAW + 20], an optimal solution for the information projection formulation (1) was theoretically characterized.
We briefly describe this result next.
Let 7 H ≜ { h ∈ C ( X , ∆ C ) ; inf c,x h c ( x ) > 0 } and we introduce the following definition and assumption.
Definition 1.
For p ∈ ∆ C , let D conj f ( · , p ) denote the convex conjugate of D f ( · ∥ p ) :
[Equation]
Assumption 1.
Assume that: (i) f ∈ C 2 ( R ) , f (1) = 0 , f ′ (0 + ) =
−∞ , and f ′′ ( t ) > 0 for all t > 0 ; (ii) each G k,c is bounded, differentiable, and has bounded gradient; (iii) h base ∈ H , and each h base c has bounded partial derivatives; and (iv) there is an h ∈ H
such that E P X
[ Gh ] < 0 .
Now, the solution for (1) can be obtained by a simple “tilting” of the base classifier’s output, as stated in the next theorem.
Theorem 1 ([AAW + 20]) .
If f, h base , and G satisfy Assumption 1, and X =
R d , then there is a unique solution h opt for the optimization problem (1) for the f -divergence objective (2) .
Furthermore, h opt is given by the tilt
[Equation]
where: (i) the function ϕ denotes the inverse of f ′ ; (ii) the function v : X × R K
→ R C is defined by v ( x ; λλλ ) ≜ − G ( x ) T λλλ ; (iii) the function γ :
X × R K → R is characterized by the equation E
[Equation]
( x ) [ ϕ ( v c ( x ; λλλ ) + γ ( x ; λλλ ))]
= 1 ; and (iv) λλλ ⋆ ∈ R K is any solution to the convex problem
[Equation]
If the underlying data distribution is known, Theorem 1 yields an expression for the projected classifier as a post-processing of the base classifier.
However, in practice, we do not know the underlying distribution and have to approximate it from a finite number of i.i.d.
samples.
In Section 3, we first describe how we approximate the solution given in Theorem 1 with finite samples.
We then propose a parallelizable algorithm to solve the approximation in Section 4.
A finite-sample approximation of information projection
In practice, P X is unknown and only
data points X ≜ { X i } i ∈ [ N ] ⊂ X , drawn from P X , are available.
Thus, we propose the following fairness optimization problem.
We search for a (multi-class) classifier h : X → ∆ C that solves the following:
[Equation]
subject to
[Equation]
with (cid:98) P X being the empirical measure (e.g., obtained from a dataset), and τ 1 , τ 2 > 0 prescribed constants.
The terms a and b are added to circumvent infeasibility issues and aid convergence of our numerical procedure.
We show in the following theorem that there is a unique solution for (6), and that it is given by a tilt (i.e., multiplicative factor) of h base .
The tilting parameter is the solution of a finite-dimensional strongly convex optimization problem.
Theorem 2.
Suppose Assumption 1 holds, and set ζ ≜ τ 22 / (2 τ 1 ) .
There exists a unique solution h opt ,N to (6) , and it is given by the formula
[Equation]
with v , ϕ, γ as in Theorem 1, and λλλ ⋆ζ,N ∈ R K is the unique solution to the strongly convex problem
[Equation]
where G N ≜ (cid:16) G ( X 1 ) / √ N, · · · , G ( X N ) / √ N, I K (cid:17) ∈ R K ×
[Equation]
Proof.
See Appendix A.1.1.
Theorem 2 shows that: strong duality holds between the primal (6) and (the negative of) the dual (8); there is a unique classifier h opt ,N minimizing our fairness formulation (6); there is a unique solution λλλ ⋆ζ,N to the dual (5); and there is an explicit functional form of h opt ,N in terms of λλλ ⋆ζ,N in (7).
Moreover, Theorem 2 yields a practical two-step procedure for solving the functional optimization in equation (6): (i) compute the dual variables λλλ by solving the strongly convex optimization in (8); (ii) tilt the base classifier by using the dual variables according to (7).
This process is applied on real-world datasets using FairProjection (see Algorithm 1) in the next section.
The key distinctions between our formulation and Theorem 1 are that we use the empirical measure (cid:98) P X (e.g., produced using a dataset with i.i.d.
samples), we have a strongly convex dual problem in (8) (in contrast to the convex program in (5)), and we prove strong duality in Theorem 2 (whereas an analogous strong duality is absent from the results of [AAW + 20]).
Remark 1.
In practice, Assumption 1 is not a limiting factor for Theorem 2 and FairProjection .
This is because: we are considering here a finite-set domain so continuity is automatic; we can perturb h base by negligible noise to push it away from the simplex boundary; and the uniform classifier is strictly feasible.
Nevertheless, Assumption 1 simplifies the derivation of our theoretical results.
Fair projection and theoretical guarantees
We introduce a parallelizable algorithm, FairProjection , that solves (6) using N i.i.d.
data points.
We prove that its utility converges to D ⋆ (see (5)) in the population limit and establish both samplecomplexity and convergence rate guarantees.
Applying FairProjection to the group-fairness intervention problem in (1) yields the optimal parameters in (7) for post-processing (i.e., tilting) the output of a multi-class classifier in order to satisfy target fairness constraints.
The FairProjection algorithm uses ADMM [BPC + 11] to solve the convex program in (8).
Recall that it suffices to optimize (8) for computing (6) as proved in Theorem 2.
Algorithm 1 presents the steps of FairProjection , and its detailed derivation is given in Appendix A.1.2.
A salient feature of FairProjection is its parallelizability .
Each step that is done for i varying over [ N ] can be executed for each i separately and in parallel.
In particular, this applies to the most computationally intensive step, the v i -update step.
We discuss next how the v i -update step is carried out.
Inner iterations.
One approach to carry out the inner iteration in Algorithm 1 that updates v i is to study the vanishing of the gradient of v (cid:55)→ D conj f ( v , p i ) + ξ ∥ v ∥ 22 + a Ti v (where ξ = ( ρ + ζ ) / 2 and a i ∈ R C is some vector).
In the KL-divergence case, D conj KL is given by a log-sum-exp function, so its gradient is given by a softmax function, and equating the gradient to zero becomes a fixed-point equation.
We give an iterative routine to solve this fixed point equation in Appendix A.1.3.1, whose proof of convergence is discussed in the same section.
Beyond the KL-divergence case, setting the gradient to zero does not seem to be an analytically tractable problem.
Nevertheless, we may reduce the vector minimization in Line 6 of Algorithm 1 to a tractable 1 -dimensional root-finding problem, as the following result aids in showing.
Lemma 1.
For p ∈ ∆ + C , a ∈ R C , and ξ > 0 , if f satisfies Assumption 1, we have that
[Equation]
Proof.
See Appendix A.1.3.2.
We note that the v i -update steps for both KL and CE (provided in detail in Appendix A.1.3.3) give, as a byproduct, the implicitly defined function γ ( x ; λλλ ) (see the statements of Theorems 1–2).
Convergence guarantees.
Our proposed algorithm, FairProjection , enjoys the following convergence guarantees.
The output after the t -th iteration λλλ ( t ) ζ,N converges exponentially fast to λλλ ⋆ζ,N (see (8)).
Theorem 3.
Suppose Assumption 1 holds, and that the matrix ( G ( X i )) i ∈ N ∈ R K
× NC has full row-rank.
Let λλλ ( t ) ζ,N and h ( t ) be the t -th iteration outputs of FairProjection for the KL-divergence case.
Then, we have the exponential decay of errors ∥ λλλ ( t ) ζ,N − λλλ ⋆ζ,N ∥ 2 = e − Ω( t ) and h ( t ) ( x ) =
h opt ,N ( x ) · (cid:0) 1 ± e − Ω( t ) (cid:1) uniformly in x ∈ X as t → ∞ .
Proof.
See Appendix A.1.5.
Remark 2.
The full-rank assumption on the matrix ( G ( X i ))
i ∈ N ∈ R K
× NC can be ensured by adding negligible noise to it.
Further, although Theorem 3 is shown for the KL-divergence, the proof directly extends to general f -divergences satisfying Assumption 1 (see Appendix A.1.6 for further discussions).
Finally, we show in Theorem 3 in Appendix A.1.7 that carrying t = Ω(log N ) iterations of FairProjection , with regularizer ζ =
Θ( N − 1 / 2 ) , yields a parameter λλλ ( t ) ζ,N that works well for the population problem for information projection (5); this makes FairProjection have a computational runtime of O ( N log N ) .
Benefit of parallelization.
The parallelizability of FairProjection provides significant speedup.
In Appendix A.2.2, we provide an ablation study comparing the speedup due to parallelization.
For the ENEM dataset (discussed next section), parallelization yields a 15 -fold reduction in runtime.
In addition to the parallel advantage of FairProjection , its inherent mathematical approach is more advantageous than gradient-based solutions.
When numerically solving the dual problem (8) (or any close variant) via gradient methods, the gradient of D conj f (the convex conjugate of an fdivergence) must be computed.
However, this gradient is tractable in only a very limited number of relevant instances of f -divergences.
FairProjection tackles this intractability through having its subroutines be informed by Lemma 1 and the discussion preceding it.
Algorithm 1 : FairProjection for solving (8).
1: Input: divergence f , predictions { p i ≜ h base ( X i ) } i ∈
[ N ] , constraints { G i ≜
G ( X i ) } i ∈ [ N ] , regularizer ζ , ADMM penalty ρ , and initializers λλλ and ( w i ) i ∈
[ N ] .
2: Output: h opt ,N c ( x ) ≜ h base c ( x ) · ϕ ( γ ( x ; λλλ ) + v c ( x ; λλλ )) .
3: Q ← ζ 2 I + ρ 2 N (cid:80) i ∈
[ N ] G i G Ti
: for t = 1 , 2 , · · · ,
do
R
f
p i )
+
[Table]
∥
[Table] [Table]
Ti
[Table]
i ∈ [ N ]
[Table]
i
[Table] [Table]
[Table] [Table]
[Table] [Table]
i ∈ [ N ] 10: end for
Numerical benchmarks
We present empirical results and show that FairProjection has competitive performance both in terms of runtime and fairness-accuracy trade-off curves compared to benchmarks—most notably the reductions approach in [ABD + 18], which requires retraining.
Extensive additional benchmarks and experiment details are reported in Appendix A.2.
Setup.
We consider three base classifiers ( Base ): gradient boosting (GBM), logistic regression (LR), and random forest (RF), implemented by Scikit-learn [PVG + 11].
For FairProjection (the constrained optimization in (6)), we use cross-entropy ( FairProjection-CE ) and KL-divergence ( FairProjection-KL ) as the loss function 8 .
We consider two fairness constraints: mean equalized odds (MEO) and statistical parity (SP) (cf.
Table 2).
Particularly, to measure multi-class performance, we extend the definition of MEO as
[Equation]
where TPR i ( s ) =
P ( (cid:98) Y = i
| Y =
i, S = s ) , and FPR i ( s ) =
P ( (cid:98) Y = i
| Y ̸
= i, S = s ) .
The definition of multi-class statistical parity is provided in Appendix A.2.4.2.
All values reported in this section are from the test set with 70/30 train-test split.
When benchmarking against methods tailored to binary classification, we restrict our results to both binary Y and S since, unlike FairProjection , competing methods cannot necessarily handle multi-class predictions and multiple groups.
Datasets.
We evaluate FairProjection and all benchmarks on four datasets.
We use two datasets in the education domain: the high-school longitudinal study (HSLS) dataset
[IPH + 11, JWC22] and a novel dataset ENEM [INE20] (details in Appendix A.2.1).
The ENEM dataset contains Brazilian college entrance exam scores along with student demographic information and socio-economic questionnaire answers (e.g., if they own a computer).
After pre-processing, the dataset contains ∼ 1.
million samples with 139 features.
Race was used as the group attribute S , and Humanities exam score is used as the label Y .
The score can be quantized into an arbitrary number of classes.
For binary experiments, we quantize Y into two classes, and for multi-class, we quantize it to 5 classes.
The race feature S has 5 categories, but we binarize it into White and Asian ( S = 1 ) and others ( S = 0 ).
We call the entire ENEM dataset ENEM-1.4M. We also created smaller versions of the dataset with 50k samples: ENEM-50k-2C (binary classes) and ENEM-50k-5C (5 classes).
9 For completeness, we report results on UCI Adult [Lic13] and COMPAS [ALMK16].
Benchmarks.
For binary classification experiments, we compare our method with five existing fair learning algorithms: Reduction [ABD + 18], reject-option classifier [KKZ12, Rejection ], equalized-odds
HPS16, EqOdds ], calibrated equalized-odds [PRW + 17, CalEqOdds ], and leveraging equal opportunity [CDH + 19, LevEqOpp ].
The choice of benchmarks is based on the availability of reproducible codes.
For the first four baselines, we use IBM AIF360 library [BDH + 18].
For
Reduction and Rejection , we vary the tolerance to achieve different operation points on the fairness-accuracy trade-off curves.
As EqOdds , CalEqOdds and LevEqOpp only allow hard equality constraint on equalized odds, they each produce a single point on the plot (see Fig. 1).
We include the group attribute as a feature in the training set following the same benchmark procedure described in [ABD + 18, WRC21] for a consistent comparison.
For multi-class classification experiments, we did not find methods that can be easily compared against FairProjection and use the multi-class extensions of mean equalized odds and statistical parity.
For the sake of completeness, we modified the codes of adversarial debiasing [ZLM18, Adversarial ], and compare our method against it.
Note that Reduction [ABD + 18] and Adversarial [ZLM18] are in-processing methods, and the rest of the benchmark algorithms are post-processing methods like FairProjection .
Additional comparisons to [KCT20] are given in Appendix A.2.4.1.
There are four methods in Table 1 we did not include the experiments: FACT [KCT20], Identifying [JN20], FST [WRC21], and Overlapping [YCK20], as explained in Appendix A.2.1.3.
Binary classification results.
We compare FairProjection with benchmarks tailored to binary classification in terms of the MEO-accuracy trade-off on the ENEM-50k-2C, HSLS, Adult, and
COMPAS datasets in Fig.
Each point is obtained by averaging 10 runs with different train-test splits.
FairProjection-CE curves were obtained by varying α values (cf.
Table 2).
When α = 1 . 0 , the outputs of FairProjection-CE are equivalent to the base classifier RF.
We observe that FairProjection-CE and Reduction have the overall best and most consistent performances.
On ENEM-50k-2C and HSLS datasets, although EqOdds achieves the best fairness, that fairness comes at the cost of 4% accuracy drop (additively).
The other four methods, on the other hand, produce comparatively good fairness with an accuracy loss of < 1% .
In particular,
FairProjection-CE has the smallest accuracy drop whilst improving MEO from 0.17 to 0.
on HSLS.
CalEqOdds requires strict calibration requirements and yields inconsistent performance when these requirements are not met.
On ENEM-50k-2C and HSLS, LevEqOpp achieves comparable MEO with a slight accuracy drop, and on COMPAS, LevEqOpp performs equally well as
FairProjection-CE and Reduction .
Note that with high fairness constraints (i.e., small tolerance), the accuracy of Rejection deteriorates.
Multi-Class results.
We illustrate how FairProjection performs on multi-class prediction using
HSLS and ENEM-50k-5C.
For HSLS, we divided student math performance into quartiles and generated four classes.
In Figure 2, we plot fairness-accuracy trade-off of FairProjection-CE with logistic regression and adversarial debiasing [ZLM18, Adversarial ].
As their base classifiers are different ( Adversarial is a GAN-based method), we plot accuracy difference compared to the base classifier instead of plotting the absolute value of accuracy 11 .
FairProjection reduces MEO significantly with very small loss in accuracy.
While Adversarial is also able to reduce MEO with negligible accuracy drop, it does not reduce the MEO as much as FairProjection .
We show more extensive results with multi-group and multi-class ( |Y| = 5 , = |S| = 5 ) in Appendix A.2.4.2.
Runtime comparisons.
To demonstrate the scalability of FairProjection , in Table 3, we record the runtime of FairProjection-CE and -KL with the five benchmarks on ENEM-1.4M-2C, which is the biggest dataset we have.
These experiments were run on a machine with AMD Ryzen 2990WX 64thread 32-Core CPU and NVIDIA TITAN
Xp 12-GB GPU.
For consistency, we used the same fairness metric (MEO, α = 0 . 01 ), base classifier (GBM), and train/test split, and each number is the average of 2 repeated experiments.
EqOdds , LevEqOpp , and CalEqOdds are faster than FairProjection since they are optimized to produce one trade-off point (cf.
Fig. 1).
Compared to baselines that produce full fairness-accuracy trade-off curves (i.e., Reduction and Rejection ), FairProjection has the fastest runtime.
Also, the non-parallel implementation of FairProjection-KL takes 25.
mins—parallelization attains 15 × speedup (detailed results in Appendix A.2.2).
We further compare
[Figure]
[Table]
the runtime results for the binary HSLS, which is the second biggest dataset, with the baselines that produce full fairness-accuracy trade-off curves.
The runtimes for Reduction , Rejection and
FairProjection-CE are 81 .
1 sec, 9 .
73 sec and 4 .
50 sec respectively—again, FairProjection has the fastest runtime.
For a theoretical comparison between the runtime of FairProjection and
Reduction , see Appendix A.2.3.
Final remarks and limitations
We introduce a theoretically-grounded and versatile fairness intervention method, FairProjection , and showcase its favorable performance in extensive experiments.
We encourage the reader to peruse our theoretical result in Appendix A.1 and extensive additional numerical benchmarks in
Appendix A.2.
FairProjection is able to correct bias for multigroup/multiclass datasets and it enjoys a fast runtime thanks to its parallelizability.
We also evaluate our method on the ENEM dataset (see Appendix A.3 for a detailed description of the dataset).
Our benchmarks are a step forward in moving away from the overused COMPAS and UCI Adult datasets.
We only consider group-fairness, and it would be interesting to try to incorporate other fairness notions (e.g., individual fairness [DHP + 12]) into our formulation.
We assume that h base is a pre-trained accurate (and potentially unfair) classifier; one future research direction is understanding how the accuracy of h base influences the performance of the projected classifier.
Finally, the performance of FairProjection is inherently constrained by data availability.
Performance may degrade with intersectional increases of the number of groups, the number of labels, and the number of fairness constraints.
Acknowledgement
We thank the anonymous referees for their careful critique, which helped improve the quality of the paper considerably.
This material is based upon work supported by the National Science Foundation under grants CAREER 1845852, IIS 1926925, FAI 2040880, CIF 1900750, an HDSI Bias ˆ 2 award, a gift from Oracle Research, and Meta Ph.D. Fellowship.