Title: Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions
Venue: NeurIPS 2023
Accepted: yes
Keywords: information theory

fair machine learning
Abstract: Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions made during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy when fairness constraints are applied and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing fairness interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination on standard (overused) tabular datasets. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.

Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions

Abstract

Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination , which is inherent in the data distribution, and epistemic discrimination , which is due to decisions made during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy when fairness constraints are applied and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing fairness interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination on standard (overused) tabular datasets. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.

1 Introduction

Algorithmic discrimination may occur in different stages of the machine learning (ML) pipeline. For example, historical biases in the data-generating process can propagate to downstream tasks; human biases can influence a ML model through inductive bias; optimizing solely for accuracy can lead to disparate model performance across groups in the data (Suresh and Guttag, 2019; Mayson, 2019). The past years have seen a rapid increase in algorithmic interventions that aim to mitigate biases in ML models (see e.g., Zemel et al., 2013; Feldman et al., 2015; Calmon et al., 2017; Menon and Williamson, 2018; Zhang et al., 2018; Zafar et al., 2019; Friedler et al., 2019; Bellamy et al., 2019; Kim et al., 2019; Celis et al., 2019; Yang et al., 2020; Jiang and Nachum, 2020; Jiang et al., 2020; Martinez et al., 2020; Lowy et al., 2021; Alghamdi et al., 2022). A recent survey (Hort et al., 2022) found nearly 400 fairness-intervention algorithms, including 123 pre-processing, 212 in-processing, and 56 post-processing algorithms introduced in the past decade.

Which sources of biases are (the hundreds of) existing fairness interventions trying to control? In order to create effective strategies for reducing algorithmic discrimination, it is critical to disentangle where biases in model performance originate. For instance, if a certain population group has significantly more missing features in training data, then it is more beneficial to collect data than selecting a more complex model class or training strategy. Conversely, if the model class does not accurately represent

the underlying distribution of a specific population group, then collecting more data for that group will not resolve performance disparities.

We divide algorithmic discrimination 1 into two categories: aleatoric and epistemic discrimination. 2 Aleatoric discrimination captures inherent biases in the data distribution that can lead to unfair decisions in downstream tasks. Epistemic discrimination, in turn, is due to algorithmic choices made during model development and lack of knowledge about the optimal 'fair' predictive model.

In this paper, we provide methods for measuring aleatoric and epistemic discrimination in classification tasks for group fairness metrics. Since aleatoric discrimination only depends on properties of the data distribution and the fairness measure of choice, we quantify it by asking a fundamental question:

For a given data distribution, what is the best achievable performance (e.g., accuracy) under a set of group fairness constraints?

We refer to the answer as the fairness Pareto frontier . This frontier delineates the optimal performance achievable by a classifier when unlimited data and computing power are available. For a fixed data distribution, the fairness Pareto frontier represents the ultimate, information-theoretic limit for accuracy and group fairness beyond which no model can achieve. Characterizing this limit enables us to (i) separate sources of discrimination and create strategies to control them accordingly; (ii) evaluate the effectiveness of existing fairness interventions for reducing epistemic discrimination; and (iii) inform the development of data collection methods that promote fairness in downstream tasks.

At first, computing the fairness Pareto frontier can appear to be an intractable problem since it requires searching over all possible classifiers-even if the data distribution is known exactly. Our main technical contribution is to provide an upper bound estimate for this frontier by solving a sequence of optimization problems. The proof technique is based on Blackwell's seminal results (Blackwell, 1953), which proposed the notion of comparisons of statistical experiments and inspired a line of works introducing alternative comparison criteria (see e.g., Shannon, 1958; Cam, 1964; Torgersen, 1991; Cohen et al., 1998; Raginsky, 2011). Here, we apply these results to develop an algorithm that iteratively refines the achievable fairness Pareto frontier. We also prove convergence guarantees for our algorithm and demonstrate how it can be used to benchmark existing fairness interventions.

We quantify epistemic discrimination by comparing a classifier's performance with the informationtheoretic optimal given by the fairness Pareto frontier. Our experiments indicate that given sufficient data, state-of-the-art (SOTA) group fairness interventions are effective at reducing epistemic discrimination as their gap to the information-theoretic limit is small (see Figure 1 and 2). Consequently, there are diminishing returns in benchmarking new fairness interventions on standard (overused) tabular datasets (e.g., UCI Adult and ProPublica COMPAS datasets). However, existing interventions do not eliminate aleatoric discrimination as this type of discrimination is not caused by choice of learning algorithm or model class, and is due to the data distribution. Factors such as data missing values can significantly contribute to aleatoric discrimination. We observe that when population groups have disparate missing patterns, aleatoric discrimination escalates, leading to a sharp decline in the effectiveness of fairness intervention algorithms (see Figure 3).

Related Work

There is significant work analyzing the tension between group fairness measures and model performance metrics (see e.g., Kleinberg et al., 2016; Chouldechova, 2017; Corbett-Davies et al., 2017; Chen et al., 2018; Wick et al., 2019; Dutta et al., 2020; Wang et al., 2021). For example, there is a growing body of work on omnipredictors (Gopalan et al., 2021; Hu et al., 2023; Globus-Harris et al., 2023) discussing how, and under which conditions, the fair Bayes optimal classifier can be derived using post-processing techniques from multicalibrated regressors. While previous studies (Hardt et al., 2016; Corbett-Davies et al., 2017; Menon and Williamson, 2018; Chzhen et al., 2019; Yang et al., 2020; Zeng et al., 2022a,b) have investigated the fairness Pareto frontier and fair Bayes

optimal classifier, our approach differs from this prior work in the following aspects: our approach is applicable to multiclass classification problems with multiple protected groups; it avoids disparate treatment by not requiring the classifier to use group attributes as an input variable; and it can handle multiple fairness constraints simultaneously and produce fairness-accuracy trade-off curves (instead of a single point). Additionally, our proof techniques based on Blackwell's results on comparing statistical experiments are unique and may be of particular interest to fair ML and information theory communities. We present a detailed comparison with this line of work in Table 2 of Appendix E.

We recast the fairness Pareto frontier in terms of the conditional distribution P ˆ Y | Y , S of predicted outcome ˆ Y given true label Y and group attributes S. This conditional distribution is related to confusion matrices conditioned on each subgroup. In this regard, our work is related to Verma and Rubin (2018); Alghamdi et al. (2020); Kim et al. (2020); Yang et al. (2020); Berk et al. (2021), which observed that many group fairness metrics can be written in terms of the confusion matrices for each subgroup. Among them, the closest work to ours is Kim et al. (2020), which optimized accuracy and fairness objectives over these confusion matrices and proposed a post-processing technique for training fair classifiers. However, they only imposed marginal sum constraints for the confusion matrices. We demonstrate that the feasible region of confusion matrices can be much smaller (see Remark 2 for an example), leading to a tighter approximation of the fairness Pareto frontier.

Recently, many strategies have been proposed to reduce the tension between group fairness and model performance by investigating properties of the data distribution. For example, Blum and Stangl (2019); Suresh and Guttag (2019); Fogliato et al. (2020); Wang et al. (2020); Mehrotra and Celis (2021); Fernando et al. (2021); Wang and Singh (2021); Zhang and Long (2021); Tomasev et al. (2021); Jacobs and Wallach (2021); Kallus et al. (2022); Jeong et al. (2022) studied how noisy or missing data affect fairness and model accuracy. Dwork et al. (2018); Ustun et al. (2019); Wang et al. (2021) considered training a separate classifier for each subgroup when their data distributions are different. Another line of research introduces data pre-processing techniques that manipulate data distribution for reducing its bias (e.g., Calmon et al., 2017; Kamiran and Calders, 2012). Among all these works, the closest one to ours is Chen et al. (2018), which decomposed group fairness measures into bias, variance, and noise (see their Theorem 1) and proposed strategies for reducing each term. Compared with Chen et al. (2018), the main difference is that we characterize a fairness Pareto frontier that depends on fairness metrics and a performance measure, giving a complete picture of how the data distribution influences fairness and accuracy.

2 Preliminaries

Next, we introduce notation, overview the key results in Blackwell (1953) on comparisons of experiments, and outline the fair classification setup considered in this paper.

Notation. For a positive integer n , let [ n ] ≜ { 1 , · · · , n } . We denote all probability distributions on the set X by P ( X ) . Moreover, we define the probability simplex ∆ m ≜ P ([ m ]) . When random variables A, X, Z form a Markov chain, we write A -X -Z. We write the mutual information between A, X as I ( A ; X ) ≜ E P A , X [ log P A , X ( A , X ) P A ( A ) P X ( X ) ] . Since I ( A ; X ) is determined by the marginal distribution P A and the conditional distribution P X | A, we also write I ( A ; X ) as I ( P A ; P X | A ) . When A, X are independent, we write A | = X.

If a random variable A ∈ [ n ] has finite support, the conditional distribution P X | A : [ n ] →P ( X ) can be equivalently written as P ≜ ( P 1 , · · · , P n ) where each P i = P X | A = i ∈ P ( X ) . Additionally, if X is a finite set [ m ] , then P X | A can be fully characterized by a transition matrix. We use T ( m | n ) to denote all transition matrices from [ n ] to [ m ] : { P ∈ R n × m ∣ ∣ ∣ 0 ≤ P i,j ≤ 1 , ∑ m j =1 P i,j = 1 , ∀ i ∈ [ n ] } .

Comparisons of Experiments

Given two statistical experiments (i.e., conditional distributions) P and Q , is there a way to decide which one is more informative? Here P and Q have the common input alphabet [ n ] and potentially different output spaces. Blackwell gave an answer in his seminal work (Blackwell, 1953) from a decision-theoretic perspective. We review these results next.

Let A be a closed, bounded, convex subset of R n . A decision function f ( x ) = ( a 1 ( x ) , · · · , a n ( x )) is any mapping from X to A . It is associated with a loss vector:

The collection of all v ( f ) is denoted by B ( P , A ) . Blackwell defined that P is more informative than Q if for every A , B ( P , A ) ⊇ B ( Q , A ) . Intuitively, this result means any risk achievable with Q is also achievable with P . Moreover, Blackwell considered the standard measure P ∗ which is the probability distribution of p ( ¯ X ) where p ( x ) : X → ∆ n is a function defined as

and ¯ Xfollows the probability distribution P 1 + ··· + P n n . One of the most important findings by Blackwell in his paper is to discover the following equivalent conditions.

Lemma 1 (Blackwell (1951, 1953)) . The following three conditions are equivalent:

P is more informative than Q ;

for any continuous and convex function ϕ : ∆ n → R , ∫ ϕ ( p ) d P ∗ ( p ) ≥ ∫ ϕ ( p ) d Q ∗ ( p ) ;

there is a stochastic transformation T s.t. T P i = Q i . In other words, there exists a Markov chain A -X -Z for any distributions on A such that P = P X | A and Q = P Z | A .

If P = P X | A is more informative than Q = P Z | A, by the third condition of Lemma 1 and the data processing inequality, I ( P A ; P X | A ) ≥ I ( P A ; P Z | A ) holds for any marginal distribution P A. However, the converse does not hold in general-even if the above inequality holds for any P A, P is not necessarily more informative than Q (Rauh et al., 2017). In this regard, Blackwell's conditions are 'stronger' than the mutual information based data processing inequality.

Group Fair Classification

Consider a multi-class classification task, where the goal is to train a probabilistic classifier h : X → ∆ C that uses input features X to predict their true label Y ∈ [ C ] . Additionally, assume the classifier produces a predicted outcome ˆ Y ∈ [ C ] and let S ∈ [ A ] represent group attributes (e.g., race and sex). Depending on the domain of interest, X can either include or exclude S as an input to the classifier. Our framework can be easily extended to the setting where multiple subgroups overlap (Kearns et al., 2018). Throughout this paper, we focus on three standard group fairness measures: statistical parity ( SP ) (Feldman et al., 2015), equalized odds ( EO ) (Hardt et al., 2016; Pleiss et al., 2017), and overall accuracy equality ( OAE ) (Berk et al., 2021) (see Table 1 for their definitions) but our analysis can be extended to many other group fairness metrics, including the ones in Table 1 of Kim et al. (2020), as well as alternative performance measures beyond accuracy.

3 Fairness Pareto Frontier

In this section, we introduce our main concept-fairness Pareto frontier ( FairFront ). We use it to measure aleatoric discrimination and quantify epistemic discrimination by comparing a classifier's performance to the FairFront . We recast FairFront in terms of the conditional distribution P ˆ Y | S , Y and apply Blackwell's conditions to characterize the feasible region of this conditional distribution. This effort converts a functional optimization problem into a convex program with a small number of variables. However, this convex program may involve infinitely many constraints. Hence, we introduce a greedy improvement algorithm that iteratively refines the approximation of FairFront and tightens the feasible region of P ˆ Y | S , Y . Finally, we establish a convergence guarantee for our algorithm.

Recall that we refer to aleatoric discrimination as the inherent biases of the data distribution that can lead to an unfair or inaccurate classifier. As its definition suggests, aleatoric discrimination only relies on properties of the data distribution and fairness metric of choice-it does not depend on the hypothesis class nor optimization method. Below we introduce FairFront that delineates a curve of optimal accuracy over all probabilistic classifiers under certain fairness constraints for a given data distribution P S , X , Y. We use FairFront to quantify aleatoric discrimination.

Definition 1. For α SP , α EO , α OAE ≥ 0 and a given P S , X , Y, we define FairFront ( α SP , α EO , α OAE ) by

where I is the indicator function; ˆ Y is produced by applying the classifier h to X; the maximum is taken over all measurable h ; and the definitions of SP , EO , and OAE are in Table 1. As a special case, if α SP , α EO , α OAE ≥ 1 , then FairFront ( α SP , α EO , α OAE ) is the accuracy of the Bayes optimal classifier.

Solving this functional optimization problem is difficult since it optimizes over all measurable classifiers. There is a line of works that proposed different fairness-intervention algorithms for training group-fair classifiers (see e.g., Menon and Williamson, 2018; Zhang et al., 2018; Zafar et al., 2019; Celis et al., 2019; Yang et al., 2020; Wei et al., 2021; Alghamdi et al., 2022). They restrict the model class and vary loss functions and optimizers to find classifiers that approach FairFront as close as possible. However, these algorithms only describe a lower bound for FairFront . They do not determine what is the best achievable accuracy for a given set of fairness constraints.

We circumvent the above-mentioned challenges by rewriting FairFront in terms of the conditional distribution P ˆ Y | S , Y . The caveat is that although each classifier yields a P ˆ Y | S , Y , not every conditional distribution corresponds to a valid classifier. Hence, we introduce the following definition which characterizes all feasible P ˆ Y | S , Y .

Definition 2. Given P X | S , Y, we define C as the set of all conditional distributions P ˆ Y | S , Y where ˆ Y is produced by some probabilistic classifier h . In other words,

Throughout this paper, we write P ˆ Y | S , Y or its corresponding transition matrix P ∈ T ( C | AC ) interchangeably. Specifically, the ( C ( s -1) + y ) -th row, ˆ y -th column of P represents P ˆ Y | S , Y (ˆ y | s, y ) and we denote it by P ( s,y ) , ˆ y .

Remark 1. We demonstrate the connection between the conditional distribution P ˆ Y | S , Y and confusion matrices in the setting of binary classification with binary groups. We define ˆ C as the counterpart of C when we replace P X | S , Y with an empirical distribution ˆ P X | S , Y computed from a dataset. The confusion matrix for group s ∈ { 0 , 1 } consists of four numbers: True Positive ( TP s ), False Positive ( FP s ), False Negative ( FN s ), True Negative ( TN s ). Assume that the number of positive-label data n + s = TP s + FN s and negative-label data n -s = TN s + FP s are given-these numbers do not depend on the classifier. Then there is a one-to-one mapping from each element in ˆ C to a confusion matrix:

Hence, ˆ C essentially characterizes all feasible confusion matrices and C is the population counterpart of ˆ C . Note that C is determined by the underlying data distribution while ˆ C (and confusion matrices) are tailored to a specific dataset.

Proposition 1. FairFront ( α S P , α E O , α O AE ) in (3) is equal to the solution of the following convex optimization:

Here the constants µ s,y ≜ Pr( S = s, Y = y ) and µ s ≜ Pr( S = s ) for s ∈ [ A ] , y ∈ [ A ] and P ( s,y ) , ˆ y denotes the ( C ( s -1) + y ) -th row, ˆ y -th column of the transition matrix P , which is P ˆ Y | S , Y (ˆ y | s, y ) .

For example, in binary classification with a binary group attribute, the above optimization only has 8 variables, 14 linear constraints + a single convex constraint P ∈ C . Hence, standard convex optimization solvers can directly compute its optimal value as long as we know how to characterize C .

Remark 2. Note that Kim et al. (2020) investigated fairness Pareto frontiers via confusion matrices. The main difference is that Definition 1 in Kim et al. (2020) relaxed the constraint (5c) to P ∈ T ( C | AC ) where T ( C | AC ) represents all transition matrices from [ AC ] to [ C ] . This leads to a loose approximation of the frontier because C is often a strict subset of T ( C | AC ) . To demonstrate this point, consider the scenario where X | = ( S , Y ) . Then ˆ Y | = ( S , Y ) by data processing inequality so

Optimizing over C rather than T ( C | AC ) can significantly tighten the fairness Pareto frontier.

Before diving into the analysis, we first introduce a function g : X → ∆ AC defined as g ( x ) = ( P S , Y | X (1 , 1 | x ) , · · · , P S , Y | X ( A,C | x ) ) . To obtain this function in practice, a common strategy among various post-processing fairness interventions (see e.g., Menon and Williamson, 2018; Alghamdi et al., 2022) is to train a probabilistic classifier that uses input features X to predict ( S , Y ) . The output probability generated by this classifier is then utilized as an approximation of the function g .

The following theorem is the main theoretical result in this paper. It provides a precise characterization of the set C through a series of convex constraints.

Theorem 1. The set C is the collection of all transition matrices P ∈ T ( C | AC ) such that the following condition holds:

For any k ∈ N and any { a i | a i ∈ [ -1 , 1] AC , i ∈ [ k ] } ,

where p ˆ y is the ˆ y -th column of P and Λ µ = diag ( µ 1 , 1 , · · · , µ A,C ) .

Intuitively, (7) uses piecewise linear functions to approximate the boundary of the convex set C where k represents the number of linear pieces. Unfortunately, replacing P ∈ C with this series of constraints in (5) may result in an intractable problem since standard duality-based approaches will lead to infinitely many dual variables. To resolve this issue, we first fix k and let C k be the set of P such that (7) holds under this fixed k . Accordingly, we define FairFront k ( α SP , α EO , α OAE ) as the optimal value of (5) when replacing C with C k . Since C 1 ⊇ C 2 ⊇ ··· ⊇ C , we have FairFront 1 ( α SP , α EO , α OAE ) ≥ FairFront 2 ( α SP , α EO , α OAE ) ≥ · · · ≥ FairFront ( α SP , α EO , α OAE ) . However, computing FairFront k ( α SP , α EO , α OAE ) still involves infinitely many constraints.

Next, we introduce a greedy improvement algorithm that consists of solving a sequence of tractable optimization problems for approximating FairFront k ( α SP , α EO , α OAE ) . We use A to collect the constraints of P and set A = ∅ initially. At each iteration, our algorithm solves a convex program to find an optimal P that maximizes the accuracy while satisfying the desired group fairness constraints and the constraints in A ; then we verify if this P is within the set C k by solving a DC (difference of

Algorithm 1 Approximate the fairness Pareto frontier.

Input: D = { ( x i , y i , s i ) } N i =1 , max number of iterations T ; max pieces k ; classifier g ( x ) ; α SP , α EO , α OAE . Initialize: set A = ∅ ; µ s,y = |{ i | s i = s,y i = y }| N ; t = 1 .

Repeat:

Solve a convex program:

Let v t and P t be the optimal value and optimal solution. Solve a DC program:

If the optimal value is ≥ 0 or t = T , stop

;

otherwise ,

add the optimal ( a 1 , · · · , a k ) to A and t = t +1 . return: v t , P t , A .

convex) program (Shen et al., 2016; Horst and Thoai, 1999). If P ∈ C k , then the algorithm stops. Otherwise, the algorithm will find the constraint that is mostly violated by P and add this constraint to A . Specifically, we determine a piecewise linear function that divides the space into two distinct regions: one containing P and the other containing C k . By 'mostly violated', we mean the function is constructed to maximize the distance between P and the boundary defined by the function. We describe our algorithm in Algorithm 1 and establish a convergence guarantee below.

Theorem 2. Let T = ∞ . If Algorithm 1 stops, its output P t is an optimal solution of FairFront k ( α S P , α E O , α O AE ) . Otherwise, any convergent sub-sequence of { P t } ∞ t =1 converges to an optimal solution of FairFront k ( α S P , α E O , α O AE ) .

Note that the output v t from Algorithm 1 is always an upper bound for FairFront ( α SP , α EO , α OAE ) , assuming the estimation error is sufficiently small. The tightness of this upper bound is determined by k (i.e., how well the piecewise linear functions approximate the boundary of C ), T (i.e., the total number of iterations). On the other hand, running off-the-shelf in-processing and post-processing fairness interventions can only yield lower bounds for FairFront ( α SP , α EO , α OAE ) .

4 Numerical Experiments

In this section, we demonstrate the tightness of our upper bound approximation of FairFront , apply it to benchmark existing group fairness interventions, and show how data biases, specifically missing values, impact their effectiveness. We find that given sufficient data, SOTA fairness interventions are successful at reducing epistemic discrimination as their gap to (our upper bound estimate of) FairFront is small. However, we also discover that when different population groups have varying missing data patterns, aleatoric discrimination increases, which diminishes the performance of fairness intervention algorithms. Our numerical experiments are semi-synthetic since we apply fairness interventions to train classifiers using the entire dataset and resample from it as the test set. This setup enables us to eliminate the estimation error associated with Algorithm 1 (see Appendix E for a discussion). We provide additional experimental results and details in Appendix C.

Adult

COMPAS

84.5

77.0

84.2

84.0

76.8

83.7

76.6

83.5

83.2

76.4

FairFront

FairFront

83.0

Baseline

Baseline

FairProjection

FairProjection

82.7

Reduction

76.2

Reduction

0

2.5

5.00

7.50

10.0

12.5

0

5.0

10.0

15.0

20.0

Max equalized odds (%)

Max equalized odds (%)

4.1 Benchmark Fairness Interventions

Setup. We evaluate our results on the UCI Adult dataset (Bache and Lichman, 2013), the ProPublica COMPAS dataset (Angwin et al., 2016), the German Credit dataset (Bache and Lichman, 2013), and HSLS (High School Longitudinal Study) dataset (Ingels et al., 2011; Jeong et al., 2022). We recognize that Adult, COMPAS, and German Credit datasets are overused and acknowledge the recent calls to move away from them (see e.g., Ding et al., 2021). We adopt these datasets for benchmarking purposes only since most fairness interventions have available code for these datasets. The HSLS dataset is a new dataset that first appeared in the fair ML literature last year and captures a common use-case of ML in education (student performance prediction, see Jeong et al., 2022). It has multi-class labels and multiple protected groups. We apply existing (group) fairness interventions to these datasets and measure their fairness violations via Max equalized odds :

where the max is taken over y, ˆ y, s, s ′ . We run Algorithm 1 with k = 6 pieces, 20 iterations, and varying α EO to estimate FairFront on each dataset. We compute the expectations and the g function from the empirical distributions and solve the DC program by using the package in Shen et al. (2016). The details about how we pre-process these datasets and additional experimental results on the German Credit and HSLS datasets are deferred to Appendix C.

Group fairness interventions. We consider five existing fairness-intervention algorithms: Reduction (Agarwal et al., 2018), EqOdds (Hardt et al., 2016), CalEqOdds (Pleiss et al., 2017), LevEqOpp (Chzhen et al., 2019), and FairProjection Alghamdi et al. (2022). Among them, Reduction is an in-processing method and the rest are all post-processing methods. For the first three benchmarks, we use the implementations from IBM AIF360 library (Bellamy et al., 2018); for LevEqOpp and FairProjection , we use the Python implementations from the Github repo in Alghamdi et al. (2022). For Reduction and FairProjection , we can vary their tolerance of fairness violations to produce a fairness-accuracy curve; for EqOdds , CalEqOdds , and LevEqOpp , each of them produces a single point since they only allow hard equality constraint. We note that FairProjection is optimized for transforming probabilistic classifier outputs (see also Wei et al., 2021), but here we threshold the probabilistic outputs to generate binary predictions which may limit its performance. Finally, we train a random forest as the Baseline classifier.

Results. We observe that if we run Algorithm 1 for a single iteration, which is equivalent to solving Proposition 1 without (5c), its solution is very close to 1 for all α EO . This demonstrates the benefits of incorporating Blackwell's conditions into the fairness Pareto frontier.

Adult

COMPAS

84.5

Aleatoric

Epistemic

77.0

discrimination

discrimination

84.0

76.0

83.5

75.0

Accurac

83.0

FairFront

74.0

FairFront

82.5

Baseline

Baseline

FairProjection

FairProjection

Reduction

73.0

Reduction

82.0

LevEqOpp

LevEqOpp

CalEqOdds

CalEqOdds

EqOdds

EqOdds

81.5

72.0

0

2.5

5.0

7.50

10.0

12.5

0

5.0

10.0

15.0

20.0

Max equalized odds (%)

Max equalized odds (%)

We train a classifier that approximates the Bayes optimal and use it as a basis for both Reduction and FairProjection , which are SOTA fairness interventions. We then apply these two fairness interventions to the entire dataset and evaluate their performance on the same dataset. Figure 1 shows that in this infinite sample regime, the fairness-accuracy curves produced by Reduction and FairProjection can approach our upper bound estimate of FairFront . This result not only demonstrates the tightness of our approximation (recall that Algorithm 1 gives an upper bound of FairFront and existing fairness interventions give lower bounds) but also shows that SOTA fairness interventions have already achieved near-optimal fairness-accuracy curves.

Recall that we use FairFront to quantify aleatoric discrimination since it characterizes the highest achievable accuracy among all classifiers satisfying the desired fairness constraints. Additionally, we measure epistemic discrimination by comparing a classifier's accuracy and fairness violation with FairFront . Given that our Algorithm 1 provides a tight approximation of FairFront , we use it to benchmark existing fairness interventions. Specifically, we first train a base classifier which may not achieve Bayes optimal accuracy. Then we use it as a basis for all existing fairness interventions. The results in Figure 2 show that SOTA fairness interventions remain effective at reducing epistemic discrimination. In what follows, we demonstrate how missing values in data can increase aleatoric discrimination and dramatically reduce the effectiveness of SOTA fairness interventions.

4.2 Fairness Risks in Missing Values

Real-world data often have missing values and the missing patterns can be different across different protected groups (see Jeong et al., 2022, for some examples). There is a growing line of research (see e.g., Jeong et al., 2022; Fernando et al., 2021; Wang and Singh, 2021; Subramonian et al., 2022; Caton et al., 2022; Zhang and Long, 2021; Schelter et al., 2019) studying the fairness risks of data with missing values. In this section, we apply our result to demonstrate how disparate missing patterns influence the fairness-accuracy curves.

Setup. We choose sex (group 0: female, group 1: male) as the group attribute for the Adult dataset, and race (group 0: African-American, group 1: Caucasian) for the COMPAS dataset. To investigate the impact of disparate missing patterns on aleatoric discrimination, we artificially generate missing values in both datasets. This is necessary as the datasets do not contain sufficient missing data. The missing values are generated according to different probabilities for different population groups. For each data point from group 0, we erase each input feature with a varying probability p 0 ∈ { 10% , 50% , 70% } , while for group 1, we erase each input feature with a fixed probability p 1 = 10% . We then apply mode imputation to the missing values, replacing them with the mode of non-missing values for each feature. Finally, we apply Algorithm 1 along with Reduction and Baseline to the imputed data. The experimental results are shown in Figure 3.

Adult

COMPAS

84.5

75.0

84.0

72.5

83.5

70.0

o

83.0

米

67.5

Accurac

FairFront (10%,10%)

65.0

FairFront (10%,10%)

82.5

+

FairFront (50%,10%)

+

FairFront (50%,10%)

?

FairFront (70%, 10%)

Z

62.5

?

FairFront (70%,10%)

Baseline (10%, 10%)

Baseline (10%, 10%)

82.0

★

Baseline (50%,10%)

★

Baseline (50%,10%)

米

Baseline (70%, 10%)

60.0

米

Baseline (70%,10%)

Reduction (10%，10%)

Reduction(10%,10%)

81.5

Reduction (50%，10%)

57.5

Reduction (50%,10%)

米

Reduction (70%，10%)

米

Reduction (70%,10%)

0

10.0

20.0

30.0

40.0

50.0

0

20.0

40.0

60.0

Max equalized odds (%)

Max equalized odds (%)

Results. As we increase the missing probability of group 0, (our upper bound estimate of) FairFront decreases since it becomes more difficult to accurately predict outcomes for group 0. This in turn affects the overall model performance, since the fairness constraint requires that the model performs similarly for both groups. We also observe the fairness-accuracy curves of Reduction decrease as the missing data for group 0 become more prevalent. In other words, as the missing data for group 0 increase, it becomes more difficult to maintain both high accuracy and fairness in the model's prediction.

5 Final Remarks

The past years have witnessed a growing line of research introducing various group fairnessintervention algorithms. Most of these interventions focus on optimizing model performance subject to group fairness constraints. Though comparing and benchmarking these methods on various datasets is valuable (e.g., see benchmarks in Friedler et al., 2019; Bellamy et al., 2019; Wei et al., 2021), this does not reveal if there is still room for improvement in their fairness-accuracy curves, or if existing methods approach the information-theoretic optimal limit when infinite data is available. Our results address this gap by introducing the fairness Pareto frontier, which measures the highest possible accuracy under a set of group fairness constraints. We precisely characterize the fairness Pareto frontier using Blackwell's conditions and present a greedy improvement algorithm that approximates it from data. Our results show that the fairness-accuracy curves produced by SOTA fairness interventions are very close to the fairness Pareto frontier on standard datasets.

Additionally, we demonstrate that when data are biased due to missing values, the fairness Pareto frontier degrades. Although existing fairness interventions can still reduce performance disparities, they come at the cost of significantly lowering overall model accuracy. The methods we present for computing the fairness Pareto frontier can also be applied to analyze other sources of aleatoric discrimination, such as when individuals may misreport their data or when there are measurement errors. Overall, the fairness Pareto frontier can serve as a valuable framework for guiding data collection and cleaning. Our results indicate that existing fairness interventions can be effective in reducing epistemic discrimination, and there are diminishing returns in developing new fairness interventions focused solely on optimizing accuracy for a given group fairness constraint on pristine data. However, existing fairness interventions have yet to effectively provide both fair and accurate classification when additional sources of aleatoric discrimination are present (such as missing values in data). This suggests that there is still significant need for research on handling aleatoric sources of discrimination that appear throughout the data collection process.

We provide an in-depth discussion on future work in Appendix E.

References

Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., and Wallach, H. (2018). A reductions approach to fair classification. In International Conference on Machine Learning , pages 60-69. PMLR.

Alghamdi, W., Asoodeh, S., Wang, H., Calmon, F. P., Wei, D., and Ramamurthy, K. N. (2020). Model projection: Theory and applications to fair machine learning. In 2020 IEEE International Symposium on Information Theory (ISIT) , pages 2711-2716. IEEE.

Alghamdi, W., Hsu, H., Jeong, H., Wang, H., Michalak, P. W., Asoodeh, S., and Calmon, F. P. (2022). Beyond Adult and COMPAS: Fair multi-class prediction via information projection. In Advances in Neural Information Processing Systems .

Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). Machine bias. ProPublica .

Bache, K. and Lichman, M. (2013). UCI Machine Learning Repository.

Bellamy, R. K., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018). Ai fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01943 .

Bellamy, R. K., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovi´ c, A., et al. (2019). Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development , 63(4/5):4-1.

Berk, R., Heidari, H., Jabbari, S., Kearns, M., and Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research , 50(1):3-44.

Blackwell, D. (1951). Comparison of experiments. Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability , pages 93-102.

Blackwell, D. (1953). Equivalent comparisons of experiments. The annals of mathematical statistics , pages 265-272.

Blodgett, S. L., Barocas, S., Daumé III, H., and Wallach, H. (2020). Language (technology) is power: A critical survey of" bias" in nlp. arXiv preprint arXiv:2005.14050 .

Blum, A. and Stangl, K. (2019). Recovering from biased data: Can fairness constraints improve accuracy? arXiv preprint arXiv:1912.01094 .

Calmon, F., Wei, D., Vinzamuri, B., Natesan Ramamurthy, K., and Varshney, K. R. (2017). Optimized pre-processing for discrimination prevention. Advances in neural information processing systems , 30.

Cam, L. L. (1964). Sufficiency and approximate sufficiency. The Annals of Mathematical Statistics , pages 1419-1455.

Caton, S., Malisetty, S., and Haas, C. (2022). Impact of imputation strategies on fairness in machine learning. Journal of Artificial Intelligence Research , 74:1011-1035.

Celis, L. E., Huang, L., Keswani, V., and Vishnoi, N. K. (2019). Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency , pages 319-328.

Chen, I., Johansson, F. D., and Sontag, D. (2018). Why is my classifier discriminatory? Advances in neural information processing systems , 31.

Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data , 5(2):153-163.

Chzhen, E., Denis, C., Hebiri, M., Oneto, L., and Pontil, M. (2019). Leveraging labeled and unlabeled data for consistent fair binary classification. Advances in Neural Information Processing Systems , 32.

Cohen, J., Kempermann, J. H., and Zbaganu, G. (1998). Comparisons of stochastic matrices with applications in information theory, statistics, economics and population . Springer Science & Business Media.

Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., and Huq, A. (2017). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining , pages 797-806.

Ding, F., Hardt, M., Miller, J., and Schmidt, L. (2021). Retiring adult: New datasets for fair machine learning. Advances in neural information processing systems , 34:6478-6490.

Dutta, S., Wei, D., Yueksel, H., Chen, P.-Y., Liu, S., and Varshney, K. (2020). Is there a trade-off between fairness and accuracy? a perspective using mismatched hypothesis testing. In International Conference on Machine Learning , pages 2803-2813. PMLR.

Dwork, C., Immorlica, N., Kalai, A. T., and Leiserson, M. (2018). Decoupled classifiers for group-fair and efficient machine learning. In Conference on fairness, accountability and transparency , pages 119-133. PMLR.

Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , pages 259-268.

Fernando, M.-P., Cèsar, F., David, N., and José, H.-O. (2021). Missing the missing values: The ugly duckling of fairness in machine learning. International Journal of Intelligent Systems , 36(7):3217-3258.

Fogliato, R., Chouldechova, A., and G'Sell, M. (2020). Fairness evaluation in presence of biased noisy labels. In International Conference on Artificial Intelligence and Statistics , pages 2325-2336. PMLR.

Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., and Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the conference on fairness, accountability, and transparency , pages 329-338.

Globus-Harris, I., Gupta, V., Jung, C., Kearns, M., Morgenstern, J., and Roth, A. (2023). Multicalibrated regression for downstream fairness. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society , pages 259-286.

Gopalan, P., Kalai, A. T., Reingold, O., Sharan, V., and Wieder, U. (2021). Omnipredictors. arXiv preprint arXiv:2109.05389 .

Hardt, M., Price, E., and Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems , volume 29.

Horst, R. and Thoai, N. V. (1999). Dc programming: overview. Journal of Optimization Theory and Applications , 103(1):1-43.

Hort, M., Chen, Z., Zhang, J. M., Sarro, F., and Harman, M. (2022). Bia mitigation for machine learning classifiers: A comprehensive survey. arXiv preprint arXiv:2207.07068 .

Hu, L., Navon, I. R. L., Reingold, O., and Yang, C. (2023). Omnipredictors for constrained optimization. In International Conference on Machine Learning , pages 13497-13527. PMLR.

Hüllermeier, E. and Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning , 110(3):457-506.

Ingels, S. J., Pratt, D. J., Herget, D. R., Burns, L. J., Dever, J. A., Ottem, R., Rogers, J. E., Jin, Y ., and Leinwand, S. (2011). High school longitudinal study of 2009 (hsls: 09): Base-year data file documentation. nces 2011-328. National Center for Education Statistics .

Jacobs, A. Z. and Wallach, H. (2021). Measurement and fairness. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages 375-385.

Jeong, H., Wang, H., and Calmon, F. P. (2022). Fairness without imputation: A decision tree approach for fair prediction with missing values. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 9558-9566.

Jiang, H. and Nachum, O. (2020). Identifying and correcting label bias in machine learning. In International Conference on Artificial Intelligence and Statistics , pages 702-712. PMLR.

Jiang, R., Pacchiano, A., Stepleton, T., Jiang, H., and Chiappa, S. (2020). Wasserstein fair classification. In Uncertainty in Artificial Intelligence , pages 862-872. PMLR.

Kallus, N., Mao, X., and Zhou, A. (2022). Assessing algorithmic fairness with unobserved protected class using data combination. Management Science , 68(3):1959-1981.

Kamiran, F. and Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and information systems , 33(1):1-33.

Katzman, J., Wang, A., Scheuerman, M., Blodgett, S. L., Laird, K., Wallach, H., and Barocas, S. (2023). Taxonomizing and measuring representational harms: A look at image tagging. arXiv preprint arXiv:2305.01776 .

Kearns, M., Neel, S., Roth, A., and Wu, Z. S. (2018). Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning , pages 2564-2572. PMLR.

Kim, J. S., Chen, J., and Talwalkar, A. (2020). Fact: A diagnostic for group fairness trade-offs. In International Conference on Machine Learning , pages 5264-5274. PMLR.

Kim, M. P., Ghorbani, A., and Zou, J. (2019). Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages 247-254.

Kleinberg, J., Mullainathan, S., and Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 .

Lowy, A., Pavan, R., Baharlouei, S., Razaviyayn, M., and Beirami, A. (2021). Fermi: Fair empirical risk minimization via exponential Rényi mutual information. arXiv preprint arXiv:2102.12586 .

Martinez, N., Bertran, M., and Sapiro, G. (2020). Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning , pages 6755-6764. PMLR.

Mayson, S. G. (2019). Bias in, bias out. The Yale Law Journal , 128(8):2218-2300.

Mehrotra, A. and Celis, L. E. (2021). Mitigating bias in set selection with noisy protected attributes. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , pages 237-248.

Menon, A. K. and Williamson, R. C. (2018). The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency , pages 107-118. PMLR.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830.

Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., and Weinberger, K. Q. (2017). On fairness and calibration. Advances in neural information processing systems , 30.

Raginsky, M. (2011). Shannon meets blackwell and le cam: Channels, codes, and statistical experiments. In 2011 IEEE International Symposium on Information Theory Proceedings , pages 1220-1224. IEEE.

Rauh, J., Banerjee, P. K., Olbrich, E., Jost, J., Bertschinger, N., and Wolpert, D. (2017). Coarsegraining and the blackwell order. Entropy , 19(10):527.

Schelter, S., He, Y ., Khilnani, J., and Stoyanovich, J. (2019). Fairprep: Promoting data to a first-class citizen in studies on fairness-enhancing interventions. arXiv preprint arXiv:1911.12587 .

Shannon, C. E. (1958). A note on a partial ordering for communication channels. Information and control , 1(4):390-397.

Shen, X., Diamond, S., Gu, Y., and Boyd, S. (2016). Disciplined convex-concave programming. In 2016 IEEE 55th Conference on Decision and Control (CDC) , pages 1009-1014. IEEE.

Subramonian, A., Chang, K.-W., and Sun, Y. (2022). On the discrimination risk of mean aggregation feature imputation in graphs. Advances in Neural Information Processing Systems .

Suresh, H. and Guttag, J. V. (2019). A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002 , 2:8.

Tomasev, N., McKee, K. R., Kay, J., and Mohamed, S. (2021). Fairness for unobserved characteristics: Insights from technological impacts on queer communities. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society , pages 254-265.

Torgersen, E. (1991). Comparison of statistical experiments , volume 36. Cambridge University Press.

Ustun, B., Liu, Y., and Parkes, D. (2019). Fairness without harm: Decoupled classifiers with preference guarantees. In International Conference on Machine Learning , pages 6373-6382. PMLR.

Varshney, K. R. (2021). Trustworthy machine learning. Chappaqua, NY .

Verma, S. and Rubin, J. (2018). Fairness definitions explained. In 2018 ieee/acm international workshop on software fairness (fairware) , pages 1-7. IEEE.

Wang, H., Hsu, H., Diaz, M., and Calmon, F. P. (2021). To split or not to split: The impact of disparate treatment in classification. IEEE Transactions on Information Theory , 67(10):6733-6757.

Wang, S., Guo, W., Narasimhan, H., Cotter, A., Gupta, M., and Jordan, M. (2020). Robust optimization for fairness with noisy protected groups. Advances in Neural Information Processing Systems , 33:5190-5203.

Wang, Y. and Singh, L. (2021). Analyzing the impact of missing values and selection bias on fairness. International Journal of Data Science and Analytics , 12(2):101-119.

Wei, D., Ramamurthy, K. N., and Calmon, F. P. (2021). Optimized score transformation for consistent fair classification. J. Mach. Learn. Res. , 22:258-1.

Wick, M., Tristan, J.-B., et al. (2019). Unlocking fairness: a trade-off revisited. Advances in neural information processing systems , 32.

Yang, F., Cisse, M., and Koyejo, S. (2020). Fairness with overlapping groups; a probabilistic perspective. In Advances in Neural Information Processing Systems , volume 33, pages 4067-4078.

Zafar, M. B., Valera, I., Gomez-Rodriguez, M., and Gummadi, K. P. (2019). Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research , 20(1):27372778.

Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. (2013). Learning fair representations. In International conference on machine learning , pages 325-333. PMLR.

Zeng, X., Dobriban, E., and Cheng, G. (2022a). Bayes-optimal classifiers under group fairness. arXiv preprint arXiv:2202.09724 .

Zeng, X., Dobriban, E., and Cheng, G. (2022b). Fair Bayes-optimal classifiers under predictive parity. In Advances in Neural Information Processing Systems .

Zhang, B. H., Lemoine, B., and Mitchell, M. (2018). Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society , pages 335-340.

Zhang, Y. and Long, Q. (2021). Assessing fairness in the presence of missing data. Advances in neural information processing systems , 34:16007-16019.

A Technical Background

In this section, we extend some results in Blackwell (1951, 1953) to our setting. For a random variable X, we denote its probability distribution by L ( X ) . A conditional distribution P X | A : [ n ] →P ( X ) can be equivalently written as P ≜ ( P 1 , · · · , P n ) where each P i = P X | A = i ∈ P ( X ) . Let A be a closed, bounded, convex subset of R n . A decision function is a mapping f : X → A , which can also be written as f ( x ) = ( a 1 ( x ) , · · · , a n ( x )) . A decision function is associated a loss vector:

The collection of all v ( f ) is denoted by B ( P X | A , A ) or B ( P , A ) .

For a vector λ ∈ ∆ n such that λ > 0 , we define a function p λ ( x ) : X → ∆ n :

Note that p λ ( X ) is a sufficient statistic for X, considering A as the parameter (it can be proved by Fisher-Neyman factorization theorem). In other words, two Markov chains hold: A -p λ ( X ) -X and A -X -p λ ( X ) for any distribution on A.

Consider a new set of probability distributions P ∗ λ ≜ ( L ( p λ ( X 1 )) , · · · , L ( p λ ( X n ))) where L ( X i ) = P i . Here P ∗ λ can be viewed as a conditional distribution from [ n ] to P (∆ n ) since each L ( p λ ( X i )) is a probability distribution over ∆ n . The following lemma follows from the sufficiency of p λ ( X ) .

Proof. Suppose that f ∗ ( p ) = ( a ∗ 1 ( p ) , · · · , a ∗ n ( p )) is a decision function for ( P ∗ λ , A ) . Accordingly, we define f ( x ) = ( a ∗ 1 ( p λ ( x )) , · · · , a n ( p λ ( x ))) where the function p λ is defined in (10). Then it is clear that f is a decision function for ( P , A ) . By the law of unconscious statistician, we have

Hence, v ( f ∗ ) = v ( f ) , which implies B ( P ∗ λ , A ) ⊆ B ( P , A ) . For the other direction, suppose f ( x ) = ( a 1 ( x ) , · · · , a n ( x )) is a decision function for ( P , A ) . Let f ∗ ( p ) = ( a ∗ 1 ( p ) , · · · , a ∗ n ( p )) where a ∗ i ( p ) ≜ E [ a i ( X i ) | p λ ( X i ) = p ] . Since p λ ( X ) is a sufficient statistics, for any i ∈ [ n ]

Therefore, f ∗ ( p ) = E [ f ( X 1 ) | p λ ( X 1 ) = p ] . Since A is a convex set, f ∗ is a decision function for ( P ∗ , A ) . By the law of total expectation, we have

Hence, v ( f ) = v ( f ∗ ) , which implies B ( P , A ) ⊆ B ( P ∗ λ , A ) .

For a vector λ ∈ ∆ n such that λ > 0 , the condition distribution P X | A induces a weighted standard measure P ∗ λ ≜ L ( p λ ( ¯ X )) where L ( ¯ X ) = λ 1 P 1 + · · · + λ n P n .

Theorem 3 (Adaptation of Theorem 4 in Blackwell (1951)) . For any two conditional distributions P X | A and Q Y | A , let P ∗ λ and Q ∗ λ be their weighted standard measures, respectively. Then B ( P X | A , A ) ⊇ B ( Q Y | A , A ) for any closed, bounded, convex set A if and only if for any continuous convex ϕ : ∆ n → R , ∫ ϕ ( p ) d P ∗ λ ( p ) ≥ ∫ ϕ ( p ) d Q ∗ λ ( p )

Proof. First, by Lemma 2, we know B ( P X | A , A ) = B ( P ∗ λ , A ) and B ( Q Y | A , A ) = B ( Q ∗ λ , A ) . We denote Λ = diag ( λ 1 , · · · , λ n ) . Consider any A = conv ( a 1 , · · · , a k ) . Let

Note that f ∗ ( p ) ∈ { a 1 , · · · , a k } since this set contains all the extreme points of A . 3 By definition, for any decision function w.r.t. ( P ∗ λ , A ) , we have

Let v = v ( f ) . By the same reason with (11), we have

where the last step is due to the law of unconscious statistician. Therefore,

The equality is achieved by v ( f ∗ ) . Hence, for any A = conv ( a 1 , · · · , a k )

Recall that Theorem 2.(3) in Blackwell (1951) states

B ( P X | A , A ) ⊇ B ( P Y | A , A ) for every closed, bounded, convex A

By approximation theory, the second condition can be relaxed to any A that is a convex hull of a finite set. By (24), this relaxed condition is equivalent to

for all ϕ ( p ) that are the maximum of finitely many linear functions. By approximation theory again, the above condition is equivalent to the one holding for any continuous convex function ϕ .

B Omitted Proofs

B.1 Proof of Lemma 3

Proof. Clearly, C is a subset of T ( C | AC ) . Let λ ∈ (0 , 1) and P ˆ Y 0 | S , Y , P ˆ Y 1 | S , Y ∈ C . Now we introduce a Bernoulli random variable B such that Pr( B = 0) = λ . Finally, we define ˆ Y λ = B ˆ Y 1 +(1 -B ) ˆ Y 0 . By definition, we have ( S , Y ) -X -ˆ Y λ so P ˆ Y λ | S , Y ∈ C . Moreover,

Hence, C is convex.

Let λ ∈ (0 , 1) . Assume P and ¯ P achieve the maximal values of Proposition 1 under ( α SP , α EO , α OAE ) and (¯ α SP , ¯ α EO , ¯ α OAE ) , respectively. We define P λ = λ P +(1 -λ ) ¯ P , which satisfies the constraints of Proposition 1 with thresholds ( λα SP +(1 -λ )¯ α SP , λα EO +(1 -λ )¯ α EO , λα OAE +(1 -λ )¯ α OAE ) . Finally, since the objective function of Proposition 1 is a linear function, it is equal to λ FairFront ( α SP , α EO , α OAE ) + (1 -λ ) FairFront (¯ α SP , ¯ α EO , ¯ α OAE ) under P λ .

B.2 Proof of Theorem 1

Proof. The proof relies on Theorem 3 and Lemma 1. For simplicity, we write the conditional P ˆ Y | S , Y as its corresponding transition matrix P . Let µ = (Pr( S = 1 , Y = 1) , · · · , Pr( S = A, Y = C )) . The function (10) in our setting can be written as:

Note that p µ ( x ) = g ( x ) due to Bayes' rule. By Lemma 1, we can rewrite C in Definition 2 as

By Lemma 1 and Theorem 3, the above set is further equivalent to all transition matrices P ∈ T ( C | AC ) satisfying

for any function ϕ : ∆ AC → R which is the maximum of finitely many linear functions. Now we can write ϕ ( p ) = max i ∈ [ k ] { a T i p } -we ignore the bias term because a T i p + b i = ( a i + b i 1 ) T p . Then the inequality in (29) can be simplified as

where p ˆ y is the ˆ y -th column of P and Λ µ = diag ( µ 1 , 1 , · · · , µ A,C ) . Finally, we can always normalize the above inequality so that each a i ∈ [ -1 , 1] AC .

B.3 Proof of Theorem 2

Proof. We denote

Let F t be the constraint set of P at the t -th iteration of our algorithm. Note that F ⊆ F t by definition. If the algorithm stops at the t -th iteration, then for any { a i | a i ∈ [ -1 , 1] AC , i ∈ [ k ] } , P t satisfies

which implies P t ∈ F . Consequently,

As a result, f ( P t ) = max P ∈F f ( P ) so P t is an optimal solution of FairFront k ( α SP , α EO , α OAE ) .

If the algorithm never stops, consider any convergent sub-sequence of P t that converges to a limit point P ∗ ∈ T ( C | AC ) . To simplify our notation, we assume P t → P ∗ as t →∞ . Since {F t } t ≥ 1 is non-increasing and they all contain F , there exists a set F ∗ such that

Therefore, we have

Since F ⊆ F ∗ , we have

Therefore,

If P ∗ ̸∈ F , then there exists a (¯ a 1 , · · · , ¯ a k ) , such that g ( P ∗ ; ¯ a 1 , · · · , ¯ a k ) > 0 . Let ( a 1 ,t , · · · , a k,t ) be the output of Step 2 at t -th iteration. Since P ∗ ∈ F t for all t , we have

By the optimality of ( a 1 ,t , · · · , a k,t ) , we have

Suppose that some sub-sequence of ( a 1 ,t , · · · , a k,t ) converges to a vector ( a ∗ 1 , · · · , a ∗ k ) . For the sake of simplicity, we assume ( a 1 ,t , · · · , a k,t ) → ( a ∗ 1 , · · · , a ∗ k ) as t →∞ . On the one hand, taking limit of t →∞ on both sides of (32) leads to

On the other hand, taking limit of t →∞ on both sides of (31) leads to

which is impossible. Therefore, P ∗ ∈ F and, as a result, we have

B.4 Additional Results

We establish basic properties of C and FairFront ( α SP , α EO , α OAE ) in the following lemma.

Lemma 3. C is a convex subset of T ( C | AC ) and FairFront ( α S P , α E O , α O AE ) is a concave function w.r.t. α S P , α E O , α O AE . Here the constants A and C denote the number of protected groups and the number of classes.

Next, we discuss a special case-X is discrete-under which C has a simple characterization.

Remark 3. If X is a discrete variable with a finite support [ D ] , we can write P X | S , Y as a transition matrix Φ ∈ T ( D | AC ) . By introducing an auxiliary variable M ∈ T ( C | D ) , we can write P ∈ C equivalently as linear constraints: P = Φ M by using the last condition of Lemma 1. Consequently, Proposition 1 boils down to a linear program. However, this characterization fails to generalize to continuous data because Φ and M will have an infinite dimension; for categorical data, this characterization suffers from the curse of dimensionality since the support size of X grows exponentially fast w.r.t. the number of features.

C Details on the Experimental Results

C.1 Additional Experiments

In this section, we present additional experimental results to further support our findings. We reproduce our experimental results on the German Credit dataset (Bache and Lichman, 2013) and HSLS (High School Longitudinal Study) dataset (Ingels et al., 2011; Jeong et al., 2022) in Figure 4 and Figure 5. In particular, the HSLS dataset experiment is a multi-group, multi-label experiment. Our observation is consistent with those on the previous two datasets-the fairness-accuracy curves given by SOTA fairness interventions, such as Reduction and FairProjection , are close to the information-theoretic limit.

German Credit

German Credit

86.0

86.0

85.5

85.9

85.0

85.9

84.5

85.8

84.0

85.8

85.7

83.5

FairFront

Baseline

83.0

FairProjection

85.7

Reduction

FairFront

82.5

LevEqOpp

85.6

Baseline

CalEqOdds

FairProjection

EqOdds

85.6

Reduction

0

5.0

10.0

15.0

20.0

0

5.0

10.0

15.0

20.0

25.0

Max equalized odds (%)

Max equalized odds (%)

HSLS

HSLS

82.0

82.0

80.0

81.0

78.0

80.0

76.0

Accurac

74.0

79.0

72.0

FairFront

FairFront with only EO constraints

78.0

Baseline

70.0

FairFront with EO, 10% OAE, and 10% SP constraints

FairProjection

FairFront with EO, 5% OAE, and 5% SP constraints

0

10.0

20.0

30.0

0

10.0

20.0

30.0

Maxequalized odds(%)

Maxequalized odds(%)

C.2 Dataset

Adult. We use sex (female or male) as the group attribute and income ( > 50 K or < = 50 K) as the target for prediction. We use sex, hours-per-week, education-num, age, marital status, relationship status (husband or wife) as the input features-we include the group attribute as an input feature. We group age into a total of 12 disjoint intervals: [0 , 20) , [20 , 25) , · · · , [65 , 70) , [70 , ∞ ) ; we group hours-per-week into a total of 14 disjoint intervals: [0 , 10) , [10 , 15) , · · · , [65 , 70) , [70 , ∞ ) .

COMPAS. We use race (African-American or Caucasian) as the group attribute and is_recid (recid. or no recid.) as the target for prediction. We use race, age, c_charge_degree, sex, priors_count, c_jail_in, c_jail_out as the input features-we include the group attribute as an input feature. We use the last two features by taking their difference to be their length_of_stay. We remove entries where COMPAS case could not be found (is_recid = -1) and entries with inconsistent arrest information. We also binarize sex and remove traffic offenses. We quantize age the same way we do in the Adult dataset and quantize length_of_stay by every 30 days and let 0 be a separate category.

German Credit. We use age (below or above 25 years old) as the group attribute and the credit column, which represents whether the loan was a good decision, as the target for prediction. We use

loan duration in month, credit amount, age, number of existing credits at this bank, sex, credit history, savings, and length of present employment as input features. We include the group attribute age as an input feature. We group credit amount into three disjoint intervals: [0, 5000), [5000, 10000), [10000, ∞ ). We group duration of loan into two categories: under 36 months and over 36 months.

HSLS. We use race as the group attribute and mathematics test score (number of questions answered correctly out of 72) as the target for prediction. This is a multi-group and multi-label dataset. The entire population is grouped by 4 categories: White, Asian, African American, and Others. We seek to predict the mathematics test performance from a set of attributes, including the scale of student's mathematical identity, scale of student's mathematics utility, scale of one's mathematics self-efficacy, parent's education, parent's income, scale of student's sense of school belonging, race, and sex. Note that we include the group attribute as an input feature. We group the target column (estimated number of questions answered correctly) into a total of 5 disjoint intervals: [0 , 30) , [30 , 40) , [40 , 50) , [50 , 60) , [60 , ∞ ) ; we group the scale of student's mathematical identity, mathematics utility, mathematics self-efficacy, and sense of school belonging into a total of 4 disjoint intervals, characterized by standard deviations away from the mean: ( -∞ , -1) , [ -1 , 0) , [0 , 1) , [1 , ∞ ) .

C.3 Benchmark

Each benchmark method's hyper-parameter values are provided below. Each point in Figure 2 for Baseline , EqOdds , CalEqOdds , Reduction , LevEqOpp , and FairProjection is obtained by applying the obtained classifier to 10 different test sets. For the Adult dataset, we use Random Forest with n_estimators=15, min_samples_leaf=3, criterion = log_loss, bootstrap = False as our baseline classifier; for the COMPAS dataset, we use Random Forest with n_estimators = 17 as our baseline classifier. For the German Credit dataset, we use Random Forest with n_estimators=100,min_samples_split =2,min_samples_leaf=1 as our baseline classifier. They are all implemented by Scikit-learn (Pedregosa et al., 2011).

EqOdds (Hardt et al., 2016). We use AIF360 implementation of EqOddsPostprocessing and the default hyper-parameter setup.

CalEqOdds (Pleiss et al., 2017). We use AIF360 implementation of CalibratedEqOddsPostprocessing and the default hyper-parameter setup.

Reduction (Agarwal et al., 2018). We use AIF360 implementation of ExponentiatedGradientReduction . We vary the allowed fairness constraint violation ϵ ∈ { 0 . 001 , 0 . 01 , 0 . 2 , 0 . 5 , 1 , 2 , 5 , 10 , 15 } for Adult dataset and ϵ ∈ { 0 . 001 , 0 . 01 , 0 . 2 , 0 . 5 , 1 , 2 , 5 , 10 , 15 } for Adult with missing values. We vary ϵ ∈ { 0 . 001 , 2 , 5 , 10 , 15 , 20 , 25 , 30 , 35 , 40 } for COMPAS to obtain a fairness-accuracy curve, and ϵ ∈ { 0 . 001 , 0 . 1 , 0 . 5 , 1 , 2 , 7 , 8 , 10 , 15 , 20 , 25 , 30 } for COMPAS with 50% missing values in the minority group. We use ϵ ∈ { 20 , 50 , 80 , 95 } for German Credit dataset and ϵ ∈ { 5 , 8 , 10 , 20 , 23 } when using Bayes Optimal classifier.

LevEqOpp (Chzhen et al., 2019). We use the Python implementation of LevEqopp from the Github repo in Alghamdi et al. (2022). We follow the same hyperparameters setup as in the original method.

FairProjection Alghamdi et al. (2022). We use the implementation from the Github repo in Alghamdi et al. (2022) and set use_protected = True. We use Random Forest with n_estimators = 17 as the baseline classifier to predict S from ( X , Y ) . We set the list of fairness violation tolerance to be { 0 . 07 , 0 . 075 , 0 . 08 , 0 . 085 , 0 . 09 , 0 . 095 , 0 . 1 , 0 . 5 , 0 . 75 , 1 . 0 } for Adult dataset and { 0 . 02 , 0 . 03 , 0 . 04 , 0 . 05 , 0 . 06 , 0 . 07 , 0 . 08 , 0 . 1 , 0 . 5 , 1 . 0 } for COMPAS dataset to obtain a fairnessaccuracy curve. We set the list of fairness violation tolerance to be { 0 . 005 , 0 . 01 , 0 . 02 , 0 . 07 , 0 . 1 , 0 . 15 } on the German Credit dataset experiment, and { 0 . 0001 , 0 . 001 , 0 . 005 , 0 . 01 , 0 . 015 , 0 . 02 , 0 . 05 } when using a Bayes optimal baseline classifier. For the HSLS dataset, the list of tolerance is { 0 . 06 , 0 . 065 , 0 . 07 , 0 . 08 , 0 . 1 , 0 . 09 , 0 . 15 , 0 . 2 , 0 . 3 } .

Multiclass

Multigroup

Avoid

disparate treatment

Multi-constraint

Curve

Hardt et al. (2016)

#

!

#

#

#

Corbett-Davies et al. (2017)

#

!

#

#

#

#

#

!

#

!

Chzhen et al. (2019)

#

#

!

#

#

Yang et al. (2020)

!

!

#

!

!

#

!

#

#

!

#

!

#

#

#

Our approach

!

!

!

!

!

D More on Related Work

We provide a detailed comparison with existing work on fairness Pareto frontier and ML uncertainty in this section.

D.1 Fairness Pareto Frontier

We present in Table 2 a detailed comparison of our approach with previous studies that have investigated the fairness Pareto frontier and fair Bayes optimal classifier. In short, our approach is different from this line of research as it simultaneously combines several important aspects: it is applicable to multiclass classification problems with multiple protected groups; it avoids disparate treatment by not requiring the classifier to use group attributes as an input variable; and it can handle multiple fairness constraints simultaneously and produce fairness-accuracy trade-off curves (instead of a single point).

D.2 Aleatoric and Epistemic Uncertainty

In this paper, we divide algorithmic discrimination into aleatoric and epistemic discrimination. We borrow this notion from ML uncertainty literature (see Hüllermeier and Waegeman, 2021, for a survey). Here we provide a detailed comparison between them.

In terms of their definitions, epistemic uncertainty arises from a lack of knowledge about the best model, such as the Bayes predictor, while epistemic discrimination results from a lack of knowledge about the optimal 'fair' predictive model. On the other hand, aleatoric uncertainty is the irreducible part of uncertainty caused by the random relationship between input features and label, while aleatoric discrimination is due to inherent biases in the data-generating distribution.

In terms of their characterization, epistemic uncertainty can in principle be reduced by including additional information (e.g., more data); epistemic discrimination can be reduced in a similar approach since a data scientist can choose a more effective fairness-intervention algorithm with access to more information.

Finally, in the infinite sample regime, a consistent learner will be able to remove all epistemic uncertainty, assuming the model class is large enough and there are no computational constraints. Analogously, we demonstrate in Figure 1 that when the underlying distribution is known, SOTA fairness interventions are able to eliminate epistemic discrimination as their fairness-accuracy curves are close to the fair front.

E More on Future Work

In this paper, we present an upper bound estimate for FairFront in Algorithm 1. It is important to note that this estimate may be subjected to errors originating from various sources. These include (i)

the approximation error of the function g , (ii) estimation errors from computing the expectation in (7) with a finite dataset, and (iii) the influence of hyperparameters, T (number of running iterations of Algorithm 1) and k (number of segments in the piece-wise linear functions). Regarding the dependence on T , our Theorem 2 ensures the algorithm's asymptotic convergence as T → ∞ . However, we have not established a proof for its behavior at a finite T . Regarding the dependence on k , we conjecture that k = A ∗ C should suffice, where A is the number of protected groups and C is the number of labels. While Blackwell proved this result for k = 2 in Theorem 10 of Blackwell (1953), an extension of this proof to a general value of k appears to remain an open problem.

We define aleatoric and epistemic discrimination with respect to the entire population. Investigating their per-instance counterparts and the relationship to individual fairness would be a compelling area of future research. Additionally, a more nuanced analysis of aleatoric and epistemic discrimination is desirable, further breaking them down into fine-grained components. For instance, epistemic discrimination may be attributed to various factors including limited training data, noisy observations of labels or sensitive attributes, and limitations of learning algorithms. Finally, investigating other criteria, such as scalability, generalization, and robustness in evaluating existing fairness interventions is a significant topic for future exploration.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript introduces FairFront i.e., an estimation for the upper bound on the Pareto Frontier for Fairness and Accuracy. The authors empirically show the tightness of this bound by showing how SOTA approaches perform close to the FairFront, while the gap may be attributed to the distributional variations.\n\nSTRENGTHS:\n- The authors tackle the problem of finding the upper bound on the Pareto frontier in contrast to prior works which focus on the lower bound.\n- The approach presented in the manuscript is theoretically grounded (by the essence of using Blackwell's results).\n- SOTA approaches are used to show the tightness of the bound empirically.\n\nWEAKNESSES:\n- The biggest weakness is that epistemic and aleatoric discrimination are terms introduced in the paper which have a link to the corresponding concepts in the uncertainty literature. However, the way these terms are used contradicts the definitions of the uncertainty counterparts e.g., Aleatoric uncertainty is linked to missing data (line 14-15, 65-66, 250-251). In prior literature, the lack of data falls in the domain of epistemic uncertainty [1][2][3].\n- In the manuscript, Aleatoric discrimination is linked to distributional differences. However, prior work either differentiates it from both aleatoric and epistemic uncertainty[4] or considers it as a part of epistemic uncertainty [5 (Section 9.2 dedicated to this topic)].\n- It is mentioned that distributional differences are largely a grey area in prior work. However, works such as FairBatch [6] and FairMixup [7], tackle such cases and are ideal for inclusion in the experiments.\n- The manuscript is a bit hard to follow. Even though it has content for describing which components are used formally, there is little intuitive background. Some links are hard to follow. e.g. it is not immediately obvious that $(S, Y) - X - \\hat{Y})$ refers to the Markov chain in Definition 2. It is not evident why this is used as the Markov chain.\n\n\n[1] Swiler, Laura P., Thomas L. Paez, and Randall L. Mayes. \"Epistemic uncertainty quantification tutorial.\" _Proceedings of the 27th International Modal Analysis Conference_. 2009.\n[2] Shaker, Mohammad Hossein, and Eyke Hüllermeier. \"Aleatoric and epistemic uncertainty with random forests.\" _Advances in Intelligent Data Analysis XVIII: 18th International Symposium on Intelligent Data Analysis, IDA 2020, Konstanz, Germany, April 27–29, 2020, Proceedings 18_. Springer International Publishing, 2020.\n[3] https://docs.aws.amazon.com/prescriptive-guidance/latest/ml-quantifying-uncertainty/epistemic-uncertainty.html\n[4] Amini, Alexander, et al. \"Deep evidential regression.\" _Advances in Neural Information Processing Systems_ 33 (2020): 14927-14937.\n[5] Varshney, Kush R. \"Trustworthy Machine Learning.\" _Chappaqua, NY_ (2021).\n[6] Roh, Yuji, et al. \"FairBatch: Batch Selection for Model Fairness.\" _International Conference on Learning Representations_.\n[7] Mroueh, Youssef. \"Fair Mixup: Fairness via Interpolation.\" _International Conference on Learning Representations_. 2021.\n\nQUESTIONS:\nIt is hard to see intuitively why FairFront works e.g., why is the markov chain $(S, Y) - X- \\hat(Y)$? How does it filter out valid classifiers?\n\nI acknowledge that I read the response and I feel overall the rebuttal was satisfying.\n\nLIMITATIONS:\nNo. \n\n- From Lemma 1, it appears that one of the limitations of this work is that the FairFront is only applicable to convex classifiers.\n- I would highly suggest renaming the terms 'epistemic' and 'aleatoric' discrimination since they do not portray the corresponding uncertainty counterparts accurately and may lead to confusion. Terms such as 'Distributional Discrimination' may be more fitting.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper splits discrimination in machine learning into aleatoric (which is that inherent to the data distribution), and epistemic (which is that due to choices in the model). They use Blackwell’s results to characterize the fairness Pareto frontier curve. Then, on 4 datasets with 5 fairness interventions, they characterize how close to their upper bound the algorithms are able to achieve.\n\nSTRENGTHS:\n- Comprehensive set of experiments on a number of datasets and fairness interventions\n- Clear writing and presentation of work\n- Claims are backed up by theories and proofs\n\nWEAKNESSES:\n- Given that the output from the algorithm (L236) is always the upper bound for FairFront, rather than exactly FairFront, I wish the paper would have been more upfront with this throughout, especially in the introduction and abstract, because I found this a bit misleading\n- Even though the names of aleatoric and epistemic discrimination are taken from prior work, I think they should be potentially rethought because for example, “epistemic discrimination” feels like a term that should be restricted to the philosophical foundations of the word “epistemology,” e.g., as it is used in epistemic injustice (https://academic.oup.com/book/32817), rather than a not-so related mathematical constraint\n- Given that a primary differentiation from prior work (L76) is that it works on multiclass classification as well as multiple protected groups, I would have liked to see empirical results on this rather than just claiming this is true\n- Given how similar the motivation of this work is to Chen, I. et al. (NeurIPS 2018)’s, I would have liked to see a deeper comparison of the findings of both works\n- “Discrimination” as used in this paper is only restricted to different forms of measurement disparities, e.g., statistical parity, equalized odds, and excludes many other forms of algorithmic discrimination that are more structural. So, sentences like L35 “We divide algorithmic discrimination into two categories” should be caveated that this is only a very specific form of algorithmic discrimination that is being divided into two categories (https://aclanthology.org/2020.acl-main.485/)\n  - As further explanation, taxonomies of algorithmic discrimination are its own genre of normative research, for example the distinction between representational vs allocational harms (https://www.youtube.com/watch?v=fMym_BKWQzk), or a taxonomy of harms within representational harms (https://arxiv.org/abs/2305.01776)\n\nQUESTIONS:\nListed above in weaknesses.\n\nLIMITATIONS:\nI think that the authors could be more upfront about the limitation that their work maps out the upper bound of the FairFront rather than the FairFront exactly, and also that they have taken a rather narrow lens of fairness, neglecting broader societal concerns.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a decomposition of discrimination (in ML classifiers) into aleatoric (irreducible) and epistemic (reducible) components. The paper surveys related work in fairness. It then introduces and discusses the Fairness Pareto Frontier, which is essentially an upper bound on the accuracy of the best possible fairness constrained classifier. The paper proposes an algorithm for estimating aleatoric discrimination (this upper bound) which utilizes Blackwell’s results on comparing statistical experiments. It then demonstrates the applicability of the algorithm through experiments on 5 datasets. Finally, it identifies missing values in the dataset as a source of aleatoric discrimination,  providing experimental evidence to support the discussion.\n\nSTRENGTHS:\nThough I was admittedly not familiar with Blackwell’s results, the paper’s technical contributions are insightful and convincing. \nThe paper makes a novel connection to statistical results that (as far as I am aware) had not been considered previously in fairness research.\nThe theoretical framework and results, as well as the estimation method are applicable to a wide range of fairness-constrained predictive modelling problems (multi-class, multi-attribute), and thus are a substantial contribution to the field.\nThe paper does a good job of surveying related work, and distinguishing its contributions. \nThe writing is concise and of high quality.\n\nWEAKNESSES:\nThe paper is dense and would benefit considerably from an illustrative example, helping to build intuition about the terms and concepts introduced. Specifically to help digest the main theoretical result, Theorem 1, and the piece-wise linear approximation. \n\nThe paper presents FairFront as an upper bound whose estimate depends on k, and T. However, for a given data distribution, it is unclear how the k-dependence is affected by the quantity of available data. As a practitioner, I would want to understand the robustness of this upper bound to data availability. Some experiments that subsample the dataset would be a helpful starting point. Perhaps a version of the experiment discussed in Appendix C.2 with varying levels of data availability (up to the infinite data regime). \n\nThe region described as epistemic discrimination (if Figure 1) is effectively a gap between an upper bound (FairFront) and a lower bound (some fairness-constrained modelling approaches). As such, the true fairness Pareto Frontier is somewhere in between. It is unclear why the reader should interpret FairFront as the more accurate bound, and thus interpret the gap as epistemic discrimination (rather than FairFront estimation error). That is to say, the paper could elaborate on the tightness of this approximate upper bound based on choices of k and T. \n\nThe use of transparency in Figure 2 makes it difficult to compare scenarios (missing probabilities), especially when printed.\n\nQUESTIONS:\nWhen the fairness constraints are trivially satisfied, for example $\\alpha_{\\text{SP}}, \\alpha_{\\text{EO}}, \\alpha_{\\text{OAE}} = \\inf$, how should one interpret the FairFront? \n\nLine 231: what is meant by “mostly violated”?\n\nLIMITATIONS:\nThe paper refers to fairness and discrimination, but makes no effort to connect these concepts to the harms or benefits experienced by real people. This is fairly common practice in the field and not unique to this paper. However, it is nonetheless a noteworthy limitation. That is to say, one cannot truly comment on the fairness (or discrimination) of a machine learning system without a broader understanding of the social impacts–beyond just its predictions. \n\nIt is unclear whether the method can be applied in the numerous important situations whereby the prediction outcome is unobserved (or partially observed), and what effects estimating these counterfactuals would have on the method, e.g., in lending–where the outcomes of the rejected cohort are counterfactuals.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper casts the issue of fairness in machine learning from the perspective of two classes of discrimination: those due to aleatoric uncertainty, or where the inherent limitations of the data distribution, and epistemic discrimination, which is due to modeling choices. It then uses this framework to analyze a common set of fairness metrics, showing that epistemic discrimination can be reduced by optimizing for these fairness measures, but not necessarily aleatoric discrimination.\n\nSTRENGTHS:\n- The choice to disentangle sources of discrimination due to modeling choices vs. data properties is sound and reasonable. \n- To the best of my knowledge, studying aleatoric discrimination via Blackwell's equations is novel (the reviewer is not familiar with this set of techniques, so can offer little here)\n- The choice to study missing data values is natural.\n\nWEAKNESSES:\nMy main concern with this paper was that its contribution feels quite limited both in terms of thoroughness of the experiments (it would have been interesting to study different sources of data properties beyond miss values, such as Khani & Liang, 2020) do with spurious features, as well as its relevance in modern machine learning (the only data studied is the well-known COMPAS dataset, and it could have been interesting to consider situations where more advanced models -- such as language or vision models -- are applied, as well as settings where the motivation to apply ML is well justified unlike risk assessment). \n\nI also think the authors could clarify quite early on they specific a particular subset of fairness interventions - group fairness with defined subgroup labels. This is not always possible when subgroups aren't clearly defined, challenging to measure, or are intractable. \n\nIn a similar vein, the writing could be more precise to properly convey the scope of FairFront. For example, sentences like \" For instance, if the training set contains few samples from a given population group, then increasing sample diversity is a more effective strategy than  selecting a more complex model class or training strategy.\" --> there are training methods that try to account for unequal group size (e.g. Hashimoto, 2018)) when group labels are inaccessible, and increasing sample diversity is hard. These sentences could just be qualified more that we are in a particular setting where we know the groups we want to be fair over, and can measure them.\n\nQUESTIONS:\nAre you able to run experiments that involve a bit more complex sources of data variability (e.g. a natural language task with different dialects, so more than 2 group labels, and using more state-of-the-art models like GPT series)?\n\nLIMITATIONS:\nI think the paper could also be more clear on the importance of the overall result. Lacking in thorough experimentation, the paper largely presents a framework but does not argue for its utility. Do the authors intend their framework to be used as a rigorous benchmark? If so, they should focus on more comprehensive set of experiments. Otherwise, what is the core message of knowing that methods that are not data-specific do not address dataset-specific sources of discrimination?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this manuscript, the authors make two main contributions to the technical study of algorithmic fairness.\n\nFirstly, on the conceptual level, they propose to distinguish between aleatoric and epistemic discrimination. By the former term, they refer to the notion that the optimal achievable performance level may differ between different (e.g., racial or gender-based) groups, corresponding to notions of differing task difficulty. No algorithmic bias mitigation approach can resolve aleatoric discrimination, which is a property of a dataset, not a model. Epistemic discrimination, on the other hand, refers to a model that performs sub-optimally on a given group, compared to the performance level it could achieve on this dataset.\n\nSecondly, the authors develop a new way to characterize an upper bound on the Fairness Pareto frontier, i.e., the frontier characterizing optimal trade-offs between overall model accuracy and various fairness constraints. The characterization is based on an old theoretical result by Blackwell and facilitates an efficient implementation by means of a greedy algorithm developed by the authors. This second contribution relates to the first in that it provides a lower bound on the level of aleatoric discrimination in a given dataset.\n\nSeveral numerical experiments in standard (low-dimensional, tabular) algorithmic fairness datasets confirm the validity of the bound and the relative tightness of some previously proposed bias mitigation techniques, emphasizing that significantly better trade-offs are unlikely to be achievable in these datasets. An experiment with artificially induced missing data shows how this increases aleatoric discrimination, as would be expected.\n\nSTRENGTHS:\nThe paper makes several highly original and important contributions to the algorithmic fairness field.\n\nThe framing of different sources of algorithmic discrimination as separated into aleatoric and epistemic discrimination is novel, and it importantly helps in understanding and characterizing the limitations of bias mitigation techniques, which can only ever aim to reduce epistemic discrimination. While similar distinctions have been made before, I find the framing and formulation presented in the manuscript particularly clear and useful.\n\nTo my knowledge, the manuscript provides the first characterization of an *upper* bound on the Pareto frontier, the latter being an essential object of study in algorithmic fairness. It also treats the general case of both multiple sensitive groups and multiple output labels, which is relatively rare in the field (which mostly focuses on the binary setting for both.)\n\nThe manuscript introduces a novel theoretical tool, based on the old results of Blackwell, into the field of algorithmic fairness, which may spark further theoretical innovations beyond the present paper. I consider this alone to be an important creative contribution.\n\nThe paper is very well written, the results are clearly presented, and prior work is comprehensively discussed and acknowledged.\n\nWEAKNESSES:\nI do not see many weaknesses in the manuscript, but there is one issue that I think deserves a more comprehensive discussion. This is related to the approximation of g, which maps inputs X to P(S,Y | X), where Y is the output label and S the sensitive group. As I understand it, the correctness of the approximated upper bound depends crucially on the correctness of this approximation. In the simple low-dimensional tabular data cases considered in the manuscript, g can be computed efficiently and precisely, but in the case of more complex (continuous, image?) domains, I would imagine this to be a serious limitation. Do I understand correctly that if g is approximated poorly, the computed bound may in fact not be an upper bound? I would appreciate a more comprehensive discussion of the impact of this approximation on the correctness of the estimated upper bound.\n\nAlso, can the authors provide some hints at important future extensions of the presented work, or important limitations of the approach? Section 5 is called \"Final Remarks and Limitations\", but it actually only summarizes the contributions of the present manuscript and lists no limitations.\n\nQUESTIONS:\n1. As outlined in detail above: what are the implications of the approximation on g? How does this impact potential other application domains, such as image analysis? Do I understand correctly that in the tabular cases considered in the paper, g is not approximated but in fact computed exactly?\n2. I was a little confused by the notation in some places. For instance, the authors write that they \"define FairFront(αSP, αEO, αOAE) as the solution of the following optimization problem\", where the following optimization problem optimizes over a classifier h. However, later on, the authors use notation such as \"FairFront1(αSP, αEO, αOAE) ≥ FairFront2(αSP, αEO, αOAE)\", prompting me to ask: what exactly *is* \"FairFront\"? Is it the model? Its achieved accuracy? Something else? This could be made clearer by a notation such as \"FairFront = arg max ...\", for instance.\n3. How does the complexity of the algorithm scale with, e.g., the number of sensitive groups to be considered? The authors emphasize that their approach \"can be easily extended to the setting where multiple subgroups overlap\"; how exactly would this work? In the case of multiple non-binary sensitive attributes, the number of subgroups to consider grows combinatorially; will that present practical problems?\n4. To give practitioners some sense of the complexity of Algorithm 1, could the authors provide run time measurements for the case studies?\n\nLIMITATIONS:\n1. Like outlined above, the limitations induced by approximating g are presently not obvious to me; I could imagine these to be quite consequential if they effectively limit the applicability of the presented work to low-dimensional tabular data.\n2. More generally, the authors do not discuss any limitations of their approach. One limitation that comes to my mind is that it has been shown repeatedly that fairness-accuracy trade-offs may be illusory in the case of group-dependent label noise or selection biases, see, e.g., Blum and Stangl (2020), Wick et al. (2019), Dutta et al. (2020). Another limitation seems to be that the presented approach relies on the availability, correctness, and validity of sensitive attributes, which is often not given; see, e.g., Jacobs and Wallach (2021) and Tomasev et al. (2021).\n\nBlum and Stangl (FORC 2020): Recovering from Biased Data: Can Fairness Constraints Improve Accuracy? https://drops.dagstuhl.de/opus/volltexte/2020/12019/pdf/LIPIcs-FORC-2020-3.pdf\n\nDutta et al. (ICML 2020): Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing. https://proceedings.mlr.press/v119/dutta20a.html\n\nJacobs and Wallach (FAccT 2021): Measurement and Fairness. https://dl.acm.org/doi/10.1145/3442188.3445901\n\nTomasev et al. (FAccT 2021): Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities. https://dl.acm.org/doi/10.1145/3461702.3462540\n\nWick et al. (NeurIPS 2019): Unlocking Fairness: a Trade-off Revisited. Unlocking Fairness: a Trade-off Revisitedhttps://papers.nips.cc/paper/2019/file/373e4c5d8edfa8b74fd4b6791d0cf6dc-Paper.pdf", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-08-01"}