Title: Achievable Fairness on Your Data With Utility Guarantees
Venue: NeurIPS 2024
Accepted: yes
Keywords: fairness

dataset

accuracy-fairness trade-off

uncertainty quantification
Abstract: In machine learning fairness, training models that minimize disparity across different sensitive groups often leads to diminished accuracy, a phenomenon known as the fairness-accuracy trade-off. The severity of this trade-off inherently depends on dataset characteristics such as dataset imbalances or biases and therefore, using a uniform fairness requirement across diverse datasets remains questionable. To address this, we present a computationally efficient approach to approximate the fairness-accuracy trade-off curve tailored to individual datasets, backed by rigorous statistical guarantees. By utilizing the You-Only-Train-Once (YOTO) framework, our approach mitigates the computational burden of having to train multiple models when approximating the trade-off curve. Crucially, we introduce a novel methodology for quantifying uncertainty in our estimates, thereby providing practitioners with a robust framework for auditing model fairness while avoiding false conclusions due to estimation errors. Our experiments spanning tabular (e.g., Adult), image (CelebA), and language (Jigsaw) datasets underscore that our approach not only reliably quantifies the optimum achievable trade-offs across various data modalities but also helps detect suboptimality in SOTA fairness methods.

Achievable Fairness on Your Data With Utility Guarantees

Abstract

In machine learning fairness, training models that minimize disparity across different sensitive groups often leads to diminished accuracy, a phenomenon known as the fairness-accuracy trade-off. The severity of this trade-off inherently depends on dataset characteristics such as dataset imbalances or biases and therefore, using a uniform fairness requirement across diverse datasets remains questionable. To address this, we present a computationally efficient approach to approximate the fairness-accuracy trade-off curve tailored to individual datasets, backed by rigorous statistical guarantees. By utilizing the You-Only-Train-Once (YOTO) framework, our approach mitigates the computational burden of having to train multiple models when approximating the trade-off curve. Crucially, we introduce a novel methodology for quantifying uncertainty in our estimates, thereby providing practitioners with a robust framework for auditing model fairness while avoiding false conclusions due to estimation errors. Our experiments spanning tabular (e.g., Adult ), image ( CelebA ), and language ( Jigsaw ) datasets underscore that our approach not only reliably quantifies the optimum achievable trade-offs across various data modalities but also helps detect suboptimality in SOTA fairness methods.

1 Introduction

A key challenge in fairness for machine learning is to train models that minimize disparity across various sensitive groups such as race or gender [9, 35, 10]. This often comes at the cost of reduced model accuracy, a phenomenon termed accuracy-fairness trade-off [36, 32]. This trade-off can differ significantly across datasets, depending on factors such as dataset biases, imbalances etc. [1, 8, 11].

To demonstrate how these trade-offs are inherently dataset-dependent, we consider a simple example involving two distinct crime datasets. Dataset A has records from a community where crime rates are uniformly distributed across all racial groups, whereas Dataset B comes from a community where historical factors have resulted in a disproportionate crime rate among a specific racial group. Intuitively, training models which are racially agnostic is more challenging for Dataset B, due to the unequal distribution of crime rates across racial groups, and will result in a greater loss in model accuracy as compared to Dataset A.

This example underscores that setting a uniform fairness requirement across diverse datasets (such as requiring the fairness violation metric to be below 10% for both datasets), while also adhering to essential accuracy benchmarks is impractical. Therefore, choosing fairness guidelines for any

Empirical trade-off using optimally trained model evaluated on data split 1

Empirical trade-off using optimally trained model evaluated on data split 2

Empirical trade-off using suboptimally trained model

Permissible trade-off region (Our method)

0.56

0.58

0.60

0.62

0.64

0.66

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Demographic Parity

Suboptimal accuracy-fairness trade-off

Unlikely to be achievable

0.62

0.63

0.64

0.65

0.04

0.05

0.06

0.07

0.08

dataset necessitates careful consideration of its individual characteristics and underlying biases. In this work, we advocate against the use of one-size-fits-all fairness mandates by proposing a nuanced, dataset-specific framework for quantifying acceptable range of accuracy-fairness trade-offs. To put it concretely, the question we consider is:

Given a dataset, what is the range of permissible fairness violations corresponding to each accuracy threshold for models in a given class H ?

This question can be addressed by considering the optimum accuracy-fairness trade-off, which shows the minimum fairness violation achievable for each level of accuracy. Unfortunately, this curve is typically unavailable and hence, various optimization techniques have been proposed to approximate this curve, ranging from regularization [8, 33] to adversarial learning [45, 40].

However, approximating the trade-off curve using these aforementioned methods has some serious limitations. Firstly, these methods require retraining hundreds if not thousands of models to obtain a good approximation of the trade-off curve, making them computationally infeasible for large datasets or models. Secondly, these works do not account for finite-sampling errors in the obtained curve. This is problematic since the empirical trade-off evaluated over a finite dataset may not match the exact trade-off over the full data distribution.

We illustrate this phenomenon in Figure 1 where the black and red trade-off curves are obtained using the same model but evaluated over two different test data draws. Here, relying solely on the estimated curves without accounting for the uncertainty could lead us to make the incorrect conclusion that the methodology used to obtain the black trade-off curve is sub-optimal (compared to the red curve) as it achieves a higher fairness violation for accuracies in the range [0 . 62 , 0 . 66] . However, this discrepancy arises solely due to finite-sampling errors.

In this paper, we address these challenges by introducing a computationally efficient method of approximating the optimal accuracy-fairness trade-off curve, supported by rigorous statistical guarantees. Our methodology not only circumvents the need to train multiple models (leading to at least a 10-fold reduction in computational cost) but is also the first to quantify the uncertainty in the estimated curve, arising from both, finite-sampling error as well as estimation error. To achieve this, our approach adopts a novel probabilistic perspective and provides guarantees that remain valid across all finite-sample draws. This also allows practitioners to distinguish if an apparent suboptimality in a baseline could be explained by finite-sampling errors (as in the black curve in Figure 1), or if it stems from genuine deficiencies in the fairness interventions applied (as in the blue curve).

The contributions of this paper are three-fold:

We present a computationally efficient methodology for approximating the accuracy-fairness trade-off curve by training only a single model. This is achieved by adapting a technique from [15] called You-Only-Train-Once (YOTO) to the fairness setting.

To account for the approximation and finite-sampling errors, we introduce a novel technical framework to construct confidence intervals (using the trained YOTO model) which contain the optimal accuracy-fairness trade-off curve with statistical guarantees. For any accuracy threshold ψ chosen at inference time , this gives us a statistically backed range of permissible fairness violations [ l ( ψ ) , u ( ψ )] , allowing us to answer our previously posed question:

Given a dataset, the permissible range of fairness violations corresponding to an accuracy threshold of ψ is [ l ( ψ ) , u ( ψ )] for models in a given class H .

Lastly, we showcase the vast applicability of our method empirically across various data modalities including tabular, image and text datasets. We evaluate our framework on a suite of SOTA fairness methods and show that our intervals are both reliable and informative.

2 Preliminaries

Notation Throughout this paper, we consider a binary classification task, where each training sample is composed of triples, ( X,A,Y ) . X ∈ X denotes a vector of features, A ∈ A indicates a discrete sensitive attribute, and Y ∈ Y := { 0 , 1 } represents a label. To make this more concrete, if we take loan default prediction as the classification task, X represents individuals' features such as their income level and loan amount; A represents their racial identity; and Y represents their loan default status. Having established the notation, for completeness, we provide some commonly used fairness violations Φ fair ( h ) ∈ [0 , 1] for a classifier model h : X → Y when A = { 0 , 1 } :

Demographic Parity (DP) The DP condition states that the selection rates for all sensitive groups are equal, i.e. P ( h ( X ) = 1 | A = a ) = P ( h ( X ) = 1) for any a ∈ A . The absolute DP violation is:

Equalized Opportunity (EOP) The EOP condition states that the true positive rates for all sensitive groups are equal, i.e. P ( h ( X ) = 1 | A = a, Y = 1) = P ( h ( X ) = 1 | Y = 1) for any a ∈ A . The absolute EOP violation is:

2.1 Problem setup

Next, we formalise the notion of accuracy-fairness trade-off , which is the main quantity of interest in our work. For a model class H (e.g., neural networks) and a given accuracy threshold ψ ∈ [0 , 1] , we define the optimal accuracy-fairness trade-off τ ∗ fair ( ψ ) as,

Here, Φ fair ( h ) and acc ( h ) denote the fairness violation and accuracy of h over the full data distribution . For an accuracy ψ ′ which is unattainable, we define τ ∗ fair ( ψ ′ ) = 1 and we focus on models H trained using gradient-based methods. Crucially, our goal is not to estimate the trade-off at a fixed accuracy level, but instead to reliably and efficiently estimate the entire trade-off curve τ ∗ fair . In contrast, previous works [1, 11] impose an apriori fairness constraint during training and therefore each trained model only recovers one point on the trade-off curve corresponding to this pre-specified constraint.

If available, this trade-off curve would allow practitioners to characterise exactly how, for a given dataset, the minimum fairness violation varies as model accuracy increases. This not only provides a principled way of selecting data-specific fairness requirements, but also serves as a tool to audit if a model meets acceptable fairness standards by checking if its accuracy-fairness trade-off lies on this curve. Nevertheless, obtaining this ground-truth trade-off curve exactly is impossible within the confines of a finite-sample regime (owing to finite-sampling errors). This means that even if baseline A's empirical trade-off evaluated on a finite dataset is suboptimal compared to the empirical trade-off of baseline B, this does not necessarily imply suboptimality on the full data distribution.

We illustrate this in Figure 1 where both red and black trade-off curves are obtained using the same model but evaluated on different test data splits. Here, even though the black curve appears suboptimal

compared to the red curve for accuracies in [0 . 62 , 0 . 66] , this apparent suboptimality is solely due to finite-sampling errors (since the discrepancy between the two curves arises only due to different evaluation datasets). If we rely only on comparing empirical trade-offs, we would incorrectly flag the methodology used to obtain the black curve as suboptimal.

To address this, we construct confidence intervals (CIs), shown as the green region in Figure 1, that account for such finite-sampling errors. In this case both trade-offs fall within our CIs which correctly indicates that this apparent suboptimality could stem from finite-sample variability. Conversely, a baseline's trade-off falling above our CIs (as in the blue curve in Figure 1) offers a confident assessment of suboptimality, as this cannot be explained away by finite-sample variability. Therefore, our CIs equip practitioners with a robust auditing tool. They can confidently identify suboptimal baselines while avoiding false conclusions caused by considering empirical trade-offs alone.

High-level road map To achieve this, our proposed methodology adopts a two-step approach:

Firstly, we propose loss-conditional fairness training , a computationally efficient methodology of estimating the entire trade-off curve τ ∗ fair by training a single model, obtained by adapting the YOTO framework [15] to the fairness setting.

Secondly, to account for the approximation and finite-sampling errors in our estimates, we introduce a novel methodology of constructing confidence intervals on the trade-off curve τ ∗ fair using the trained YOTO model. Specifically, given α ∈ (0 , 1) , we construct confidence intervals Γ α fair ⊆ [0 , 1] which satisfy guarantees of the form:

Here, Γ α fair and Ψ ∈ [0 , 1] are random variables obtained using a held-out calibration dataset D cal (see Section 3.2) and the probability is taken over different draws of D cal .

3 Methodology

First, we demonstrate how our 2-step approach offers a practical and statistically sound method for estimating τ ∗ fair ( ψ ) . Figure 1 provides an illustration of our proposed confidence intervals (CIs) Γ α fair and shows how they can be interpreted as a range of 'permissible' values of accuracy-fairness trade-offs (the green region). Specifically, if for a classifier h 0 , the accuracy-fairness pair ( acc ( h 0 ) , Φ fair ( h 0 )) lies above the CIs Γ α fair (i.e., the pink region in Figure 1), then h 0 is likely to be suboptimal in terms of the fairness violation, i.e., there likely exists h ′ ∈ H with acc ( h ′ ) ≥ acc ( h 0 ) and Φ fair ( h ′ ) ≤ Φ fair ( h 0 ) . On the other hand, it is unlikely for any model h ′ ∈ H to achieve a trade-off below the CIs Γ α fair (the blue region in Figure 1). Next, we outline how to construct such intervals.

3.1 Step 1: Efficient estimation of trade-off curve

The first step of constructing the intervals is to approximate the trade-off curve by recasting the problem into a constrained optimization objective. The optimization problem formulated in Eq. (1) is however, often too complex to solve, because the accuracy acc ( h ) and fairness violations Φ fair ( h ) are both non-smooth [1]. These constraints make it hard to use standard optimization methods that rely on gradients [25]. To get around this issue, previous works [1, 8] replace the non-smooth constrained optimisation problem with a smooth surrogate loss. Here, we consider parameterized family of classifiers H = { h θ : X → R | θ ∈ Θ } (such as neural networks) trained using the regularized loss:

where, l CE is the cross-entropy loss for the classifier h θ and L fair ( h θ ) is a smooth relaxation of the fairness violation Φ fair [8, 29]. For example, when the fairness violation is DP, [8] consider

for different choices of g ( x ) , including the identity and sigmoid functions. We include more examples of such regularizers in Appendix F.3. The parameter λ in L λ modulates the accuracy-fairness trade-off with lower values of λ favouring higher accuracy over reduced fairness violation.

Now that we defined the optimization objective, obtaining the trade-off curve becomes straightforward by simply optimizing multiple models over a grid of regularization parameters λ . However, training multiple models can be computationally expensive, especially when this involves large-scale models (e.g. neural networks). To circumvent this computational challenge, we introduce loss-conditional fairness training obtained by adapting the YOTO framework proposed by [15].

3.1.1 Loss-conditional fairness training

As we describe above, a popular approach for approximating the accuracy-fairness trade-off τ ∗ fair ( ψ ) involves training multiple models h θ ∗ λ over a discrete grid of λ hyperparameters with the regularized loss L λ . To avoid the computational overhead of training multiple models, [15] propose 'You Only Train Once' (YOTO), a methodology of training one model h θ : X × Λ → R , which takes λ ∈ Λ ⊆ R as an additional input using Feature-wise Linear Modulation (FiLM) [34] layers. YOTO is trained such that at inference time h θ ( · , λ ′ ) recovers the classifier obtained by minimising L λ ′ in Eq. (2).

Recall that we are interested in minimising the family of losses L λ , parameterized by λ ∈ Λ (Eq. (2)). Instead of fixing λ , YOTO solves an optimisation problem where the parameter λ is sampled from a distribution P λ . As a result, during training the model observes many values of λ and learns to optimise the loss L λ for all of them simultaneously. At inference time, the model can be conditioned on a chosen value λ ′ and recovers the model trained to optimise L λ ′ . Hence, once adapted to our setting, the YOTO loss becomes:

Having trained a YOTO model, the trade-off curve τ ∗ fair ( ψ ) can be approximated by simply plugging in different values of λ at inference time and thus avoiding additional training. From a theoretical point of view, [15, Proposition 1] proves that under the assumption of large enough model capacity, training the loss-conditional YOTO model performs as well as the separately trained models while only requiring a single model. Although the model capacity assumption might be hard to verify in practice, our experimental section has shown that the trade-off curves estimates ̂ τ ∗ fair ( ψ ) obtained using YOTO are consistent with the ones obtained using separately trained models.

It should be noted, as is common in optimization problems, that the estimated trade-off curve ̂ τ ∗ fair ( ψ ) may not align precisely with the true trade-off curve τ ∗ fair ( ψ ) . This discrepancy originates from two key factors. Firstly, the limited size of the training and evaluation datasets introduces errors in the estimation of ̂ τ ∗ fair ( ψ ) . Secondly, we opt for a computationally tractable loss function instead of the original optimization problem in Eq. (1). This may result in our estimation ̂ τ ∗ fair ( ψ ) yielding sub-optimal trade-offs, as can be seen from Figure 1. Therefore, to ensure that our procedure yields statistically sound inferences, we next construct confidence intervals using the YOTO model, designed to contain the true trade-off curve τ ∗ fair ( ψ ) with high probability.

3.2 Step 2: Constructing confidence intervals

As mentioned above, our goal here is to use our trained YOTO model to construct confidence intervals (CIs) for the optimal trade-off curve τ ∗ fair ( ψ ) defined in Eq. (1). Specifically, we assume access to a held-out calibration dataset D cal := { ( X i , A i , Y i ) } i which is disjoint from the training data. Given a level α ∈ [0 , 1] , we construct CIs Γ α fair ⊆ [0 , 1] using D cal, which provide guarantees of the form:

Here, it is important to note that Ψ ∈ [0 , 1] and Γ α fair are random variables obtained from the calibration data D cal, and the guarantee in Eq. (3) holds marginally over Ψ and Γ α fair . While our CIs in this section require the availability of the sensitive attributes in D cal, in Appendix D we also extend our methodology to the setting where sensitive attributes are missing. In this section, for notational convenience we use h λ ( · ) to denote the YOTO model h θ ( · , λ ) for λ ∈ Λ .

The uncertainty in our trade-off estimate arises, in part, from the uncertainty in the accuracy and fairness violations of our trained model. Therefore, our methodology of constructing CIs on τ ∗ fair , involves first constructing CIs on test accuracy acc ( h λ ) and fairness violation Φ fair ( h λ ) for a given value of λ using D cal, denoted as C α acc ( λ ) and C α fair ( λ ) respectively satisfying,

One way to construct these CIs involves using assumption-light concentration inequalities such as Hoeffding's inequality. To be more concrete, for the accuracy acc ( h λ ) :

Lemma 3.1 (Hoeffding's inequality) . Given a classifier h λ : X → Y , we have that,

Lemma 3.1 illustrates that we can use Hoeffding's inequality to construct confidence interval C α acc ( λ ) = [ ˜ acc ( h λ ) -δ, ˜ acc ( h λ ) + δ ] on acc ( h λ ) such that the true acc ( h λ ) will lie inside the CI with probability 1 -α . Analogously, we also construct CIs for fairness violations, Φ fair ( h λ ) , although this is subject to additional nuanced challenges, which we address using a novel sub-sampling based methodology in Appendix B. Once we have CIs over acc ( h λ ) and Φ fair ( h λ ) for a model h λ , we next outline how to use these to derive CIs for the minimum achievable fairness τ ∗ fair , satisfying Eq. (3). We proceed by explaining how to construct the upper and lower CIs separately, as the latter requires additional considerations regarding the trade-off achieved by YOTO.

3.2.1 Upper confidence intervals

We first outline how to obtain one-sided upper confidence intervals on the optimum accuracy-fairness trade-off τ ∗ fair (Ψ) of the form Γ α fair = [0 , U λ fair ] , which satisfies the probabilistic guarantee in Eq. (3). To this end, given a classifier h λ ∈ H , our methodology involves constructing one-sided lower CI on the accuracy acc ( h λ ) and upper CI on the fairness violation Φ fair ( h λ ) . We make this concrete below:

Proposition 3.2. Given h λ ∈ H , let L λ acc , U λ fair ∈ [0 , 1] be lower and upper CIs on acc ( h λ ) and Φ fair ( h λ ) , i.e.

Proposition 3.2 shows that for any model h λ , the upper CI on model fairness, U λ fair , provides a valid upper CI for the trade-off value at L λ acc , i.e. τ ∗ fair ( L λ acc ) . This can be used to construct upper CIs on τ ∗ fair ( ψ ) for a given accuracy level ψ . To understand how this can be achieved, we first find λ ∈ Λ such that the lower CI on the accuracy of model h λ , L λ acc , satisfies L λ acc ≥ ψ . Then, since by definition τ ∗ fair is a monotonically increasing function, we know that τ ∗ fair ( L λ acc ) ≥ τ ∗ fair ( ψ ) . Since Proposition 3.2 tells us that U λ fair is an upper CI for τ ∗ fair ( L λ acc ) , it follows that U λ fair is also a valid upper CI for τ ∗ fair ( ψ ) .

Intuitively, Proposition 3.2 provides the 'worst-case' optimal trade-off, accounting for finite-sample uncertainty. It is important to note that this result does not rely on any assumptions regarding the optimality of the trained classifiers. This means that the upper CIs will remain valid even if the YOTO classifier h λ is not trained well (and hence achieves sub-optimal accuracy-fairness trade-offs), although in such cases the CI may be conservative.

Having explained how to construct upper CIs on τ ∗ fair ( ψ ) , we next move on to the lower CIs.

3.2.2 Lower confidence intervals

Obtaining lower confidence intervals on τ ∗ fair ( ψ ) is more challenging than obtaining upper confidence intervals. We begin by explaining at an intuitive level why this is the case.

Suppose that h λ ∈ H is such that acc ( h λ ) = ψ , then since τ ∗ fair denotes the minimum attainable fairness violation (Eq. (1)), we have that τ ∗ fair ( ψ ) ≤ Φ fair ( h λ ) . Therefore, any valid upper confidence interval on Φ fair ( h λ ) will also be valid for τ ∗ fair ( ψ ) . However, a lower bound on Φ fair ( h λ ) cannot be used as a lower bound for the minimum achievable fairness τ ∗ fair ( ψ ) in general. A valid lower CI for τ ∗ fair ( ψ ) will therefore depend on the gap between fairness violation achieved by h λ , Φ fair ( h λ ) , and minimum achievable fairness violation τ ∗ fair ( ψ ) (i.e., ∆( h λ ) term in Figure 2a). We make this concrete by constructing lower CIs depending on ∆( h λ ) explicitly.

Proposition 3.3. Given h λ ∈ H , let U λ acc , L λ fair ∈ [0 , 1] be upper and lower CIs on acc ( h λ ) and Φ fair ( h λ ) , i.e.

Proposition 3.3 can be used to derive lower CIs on τ ∗ fair ( ψ ) at a specified accuracy level ψ , using a methodology analogous to that described in Section 3.2.1. Intuitively, this result provides the 'best-case' optimal trade-off, accounting for finite-sample uncertainty. However, unlike the upper CI, the lower CI includes the ∆( h λ ) term, which is typically unknown. To circumvent this, we propose a strategy for obtaining plausible approximations for ∆( h λ ) in practice in the following section.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Fairness Violation

(

h

)

Feasible trade-off region

Unachievable trade-off region

(acc(

h

),

fair

(

h

))

Trade-off achieved by model

h

(acc(

h

),

*

fair

(acc(

h

)))

Optimum achievable trade-off

Optimal tradeoff

*

fair

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Fairness Violation

1

2

3

Shift lower CI by

i

i

=

2

YOTO trade-off

CIs before sensitivity analysis

CIs after sensitivity analysis

Separately trained models

3.2.3 Sensitivity analysis for ∆( h λ )

Recall that ∆( h λ ) quantifies the difference between the fairness loss of classifier h λ and the minimum attainable fairness loss τ ∗ fair ( acc ( h λ )) , and is an unknown quantity in general (see Figure 2a). Here, we propose a practical strategy for positing values for ∆( h λ ) which encode our belief on how close the fairness loss Φ fair ( h λ ) is to τ ∗ fair ( acc ( h λ )) . This allows us to construct CIs which not only incorporate finite-sampling uncertainty from calibration data, but also account for the possible sub-optimality in the trade-offs achieved by h λ . The main idea behind our approach is to calibrate ∆( h λ ) using additional separately trained standard models without imposing significant computational overhead.

Details Our sensitivity analysis uses k additional models M := { h (1) , h (2) , . . . , h ( k ) } ⊆ H trained separately using the standard regularized loss L λ ′ (Eq. (2)) for some randomly chosen values of λ ′ . Let M 0 ⊆ M denote the models which achieve a better empirical trade-off than the YOTO model on D cal, i.e. the empirical trade-offs for models in M 0 lie below the YOTO trade-off curve (see Figure 2b). We choose ∆( h λ ) for our YOTO model to be the maximum gap between empirical trade-offs of these separately trained models in M 0 and the YOTO model. It can be seen from Proposition 3.3 that, in practice, this will result in a downward shift in the lower CI until all the separately trained models in M lie above the lower CI. As a result, our methodology yields increasingly conservative lower CIs as the number of additional models |M| increases.

Even though the procedure above requires training additional models M , it does not impose the same computational overhead as training models over the full range of λ values. We show empirically in Section 5 that in practice 2 models are usually sufficient to obtain informative and reliable intervals. Additionally, we also show that when YOTO achieves the optimal trade-off (i.e., ∆( h λ ) = 0 ), our sensitivity analysis leaves the CIs unchanged, thereby preventing unnecessary conservatism.

Asymptotic analysis of ∆( h λ ) While the procedure described above provides a practical solution for obtaining plausible approximations for ∆( h λ ) , we next present a theoretical result which provides reassurance that this gap should become negligible as the number of training data D tr increases.

Theorem 3.4. Let ̂ Φ fair ( h ′ ) , ̂ acc ( h ′ ) denote the fairness violation and accuracy for h ′ ∈ H evaluated on training data D tr and let

Then, given η ∈ (0 , 1) , under standard regularity assumptions, we have that with probability at least 1 -η , ∆( h ) ≤ ˜ O ( |D tr | -γ ) , for some γ ∈ (0 , 1 / 2] where ˜ O ( · ) suppresses dependence on log (1 /η ) .

Theorem 3.4 shows that as the training data size |D tr | increases, the error term ∆( h λ ) will become negligible with a high probability for any model h λ which minimises the empirical training loss in Eq. (4). In this case, ∆( h λ ) should not have a significant impact on the lower CIs in Proposition 3.3 and the CIs will reflect the uncertainty in τ ∗ fair arising mostly due to finite calibration data. We also verify this empirically in Appendix F.7. It is worth noting that Theorem 3.4 relies on the same assumptions as used in Theorem 2 in [1], which have been provided in Appendix A.2.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

Adult dataset

COMPAS dataset

CelebA dataset

Jigsaw dataset

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

0.50

0.55

0.60

0.65

0.70

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.5

0.6

0.7

0.8

0.9

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.55

0.60

0.65

0.70

0.75

0.00

0.01

0.02

0.03

0.04

0.05

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.88

0.89

0.90

0.91

0.92

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.5

0.6

0.7

0.8

0.9

0.00

0.02

0.04

0.06

0.08

0.10

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

4 Related works

Many previous fairness methods in the literature, termed in-processing methods, introduce constraints or regularization terms to the optimization objective. For instance, [1, 10] impose a priori uniform constraints on model fairness at training time. However, given the data-dependent nature of accuracyfairness trade-offs, setting a uniform fairness threshold may not be suitable. Other in-processing methods [37, 24] consider information-theoretic bounds on the optimal trade-off in infinite data limit, independent of a specific model class. While these works offer valuable theoretical insights, there is no guarantee that these frontiers are attainable by models within a given model class H . We verify this empirically in Appendix F.6 by showing that, for the Adult dataset, the frontiers proposed in [24] are not achieved by any SOTA method we considered. In contrast, our method provides guarantees on the achievable trade-off curve within realistic constraints of model class and data availability.

Various other regularization approaches [39, 33, 8, 14, 41, 42, 43] have also been proposed, but these often necessitate training multiple models, making them computationally intensive. Alternative strategies include learning 'fair' representations [44, 30, 31], or re-weighting data based on sensitive attributes [18, 22]. These, however, provide limited control over accuracy-fairness trade-offs.

Besides this, post-processing methods [19, 38] enforce fairness after training but can lead to other forms of unfairness such as disparate treatment of similar individuals [16]. Moreover, many post-hoc approaches such as [3, 2] still require solving different optimisation problems for different fairness thresholds. Other methods such as [46, 27] involve learning a post-hoc module in addition to the base classifier. As a result, the computational cost of training the YOTO model is similar to (and in many cases lower than) the combined cost of training a base model and subsequently applying a post-processing intervention to this pre-trained classifier. We confirm this empirically in Section 5.

5 Experiments

In this section, we empirically validate our methodology of constructing confidence intervals on the fairness trade-off curve, across diverse datasets with neural networks as model class H . These datasets range from tabular ( Adult and COMPAS ), to image-based ( CelebA ), and natural language processing datasets ( Jigsaw ). Recall that our approach involves two steps: initial estimation of the trade-off via the YOTO model, followed by the construction of CIs using calibration data D cal .

To evaluate our methodology, we implement a suite of baseline algorithms including SOTA inprocessing techniques such as regularization-based approaches [8], a SOTA kernel-density based method [12] (denoted as 'KDE-fair'), as well as the reductions method [1]. Additionally, we also compare against adversarial fairness techniques [45] and a post-processing approach (denoted as 'RTO') [2] and consider the three most prominent fairness metrics: Demographic Parity (DP), Equalized Odds (EO), and Equalized Opportunity (EOP). We provide additional details and results in Appendix F, where we also consider a synthetic setup with tractable τ ∗ fair . The code to reproduce our experiments is provided at github.com/faaizT/DatasetFairness.

5.1 Results

Figure 3 shows the results for different datasets and fairness violations, obtained using a 10% data split as calibration dataset D cal. For each dataset, we construct 4 CIs that serve as the upper and lower bounds on the optimal accuracy-fairness trade-off curve. These intervals are computed at a 95% confidence level using various methodologies, including 1) Hoeffding's, 2) Bernstein's inequalities which both offer finite sample guarantees as well as, 3) bootstrapping [17], and 4) asymptotic intervals based on the Central Limit Theorem [26] which are valid asymptotically. There are 4 key takeaways:

Takeaway 1: Trade-off curves are data dependent. The results in Figure 3 confirm that the accuracy-fairness trade-offs can vary significantly across the datasets. For example, achieving nearperfect fairness (i.e. Φ fair ( h ) ≈ 0 ) seems significantly easier for the Jigsaw dataset than the COMPAS dataset, even as the accuracy increases. Likewise, for Adult and COMPAS, the DP increases gradually with increasing accuracy, whereas for CelebA, the increase is sharp once the accuracy increases above 90%. Therefore, using a uniform fairness threshold across datasets [as in 1] may be too restrictive, and our methodology provides more dataset-specific insights about the entire trade-off curve instead.

Takeaway 2: Our CIs are both reliable and informative. Recall that, any trade-off which lies above our upper CIs is guaranteed to be sub-optimal with probability 1 -α , thereby enabling practitioners to effectively distinguish between genuine sub-optimalities and those due to finite-sample errors. Table 1 lists the proportion of sub-optimal empirical trade-offs for each baseline and provides a principled comparison of the baselines. For example, the adversarial, RTO and logsig baselines have a significantly higher proportion of sub-optimal trade-offs than the KDE-fair and separate baselines.

On the other hand, the validity of our lower CIs depends on the optimality of our YOTO model and the lower CIs may be too tight if YOTO is sub-optimal. Therefore, for the lower CIs to be reliable, it must be unlikely for any baseline to achieve a trade-off below the lower CIs. Table 1 confirms this empirically, as the proportion of models which lie below the lower CIs is negligible. In Appendix E, we also account for the uncertainty in baseline trade-offs when assessing the optimality, hence yielding more robust inferences. The results remain similar to those in Table 1.

Takeaway 3: YOTO trade-offs are consistent with SOTA. We observe that the YOTO trade-offs align well with most of the SOTA baselines considered while reducing the computational cost by approximately 40-fold (see the final column of Table 1). In some cases, YOTO even achieves a better trade-off than the baselines considered. See, e.g., the Jigsaw dataset results (especially for EOP). Moreover, we observe that the baselines yield empirical trade-offs which have a high variance as

accuracy increases (see Jigsaw results in Figure 3, for example). This behaviour starkly contrasts the smooth variations exhibited by our YOTO-generated trade-off curves along the accuracy axis.

Takeaway 4: Sensitivity analysis does not cause unnecessary conservatism. We use 2 randomly chosen separately trained models to perform our sensitivity analysis for Figure 3. We find that this only causes a shift in lower CIs for 2 out of the 12 trade-off curves presented (i.e. for DP and EO trade-offs on the Adult dataset), leaving the rest of the CIs unchanged. Therefore, in practice sensitivity analysis does not impose significant computational overhead, and only changes the CIs when YOTO achieves a suboptimal trade-off. Additional results have been included in Appendix C.

6 Discussion and Limitations

In this work, we propose a computationally efficient approach to capture the accuracy-fairness tradeoffs inherent to individual datasets, backed by sound statistical guarantees. Our proposed methodology enables a nuanced and dataset-specific understanding of the accuracy-fairness trade-offs. It does so by obtaining confidence intervals on the accuracy-fairness trade-off, leveraging the computational benefits of the You-Only-Train-Once (YOTO) framework [15]. This empowers practitioners with the ability to, at inference time, specify desired accuracy levels and promptly receive corresponding permissible fairness ranges. By eliminating the need for repetitive model training, we significantly streamline the process of obtaining accuracy-fairness trade-offs tailored to individual datasets.

Limitations Despite the evident merits of our approach, it also has some limitations. Firstly, our methodology requires distinct datasets for training and calibration, posing difficulties when data is limited. Under such constraints, the YOTO model might not capture the optimal accuracy-fairness trade-off, and moreover, the resulting confidence intervals could be overly conservative. Secondly, our lower CIs incorporate an unknown term ∆( h λ ) . While we propose sensitivity analysis for approximating this term and prove that it is asymptotically negligible under certain mild assumptions in Section 3.2.3, a more exhaustive understanding remains an open question. Exploring informative upper bounds for ∆( h λ ) under weaker conditions is a promising avenue for future investigations.

Acknowledgments

We would like to express our gratitude to Sahra Ghalebikesabi for her valuable feedback on an earlier draft of this paper. We also thank the anonymous reviewers for their thoughtful and constructive comments, which enhanced the clarity and rigor of our final submission.

References

A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, and H. Wallach. A reductions approach to fair classification. 03 2018.

I. M. Alabdulmohsin and M. Lucic. A near-optimal algorithm for debiasing trained machine learning models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 8072-8084. Curran Associates, Inc., 2021.

W. Alghamdi, H. Hsu, H. Jeong, H. Wang, P. W. Michalak, S. Asoodeh, and F. P. Calmon. Beyond adult and compas: Fairness in multi-class prediction, 2022.

A. N. Angelopoulos, S. Bates, C. Fannjiang, M. I. Jordan, and T. Zrnic. Prediction-powered inference, 2023.

J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias, 2016.

P. L. Bartlett and S. Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. In D. Helmbold and B. Williamson, editors, Computational Learning Theory , pages 224-240, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg.

B. Becker and R. Kohavi. Adult. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5XW20.

H. Bendekgey and E. B. Sudderth. Scalable and stable surrogates for flexible classifiers with fairness constraints. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems , 2021.

S. Caton and C. Haas. Fairness in machine learning: A survey. CoRR , abs/2010.04053, 2020.

L. E. Celis, L. Huang, V. Keswani, and N. K. Vishnoi. Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the Conference on Fairness, Accountability, and Transparency , FAT* '19, page 319-328, New York, NY, USA, 2019. Association for Computing Machinery.

L. E. Celis, L. Huang, V. Keswani, and N. K. Vishnoi. Fair classification with noisy protected attributes: A framework with provable guarantees. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 1349-1361. PMLR, 18-24 Jul 2021.

J. Cho, G. Hwang, and C. Suh. A fair classifier using kernel density estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 15088-15099. Curran Associates, Inc., 2020.

J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR , abs/1810.04805, 2018.

M. Donini, L. Oneto, S. Ben-David, J. S. Shawe-Taylor, and M. Pontil. Empirical risk minimization under fairness constraints. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

A. Dosovitskiy and J. Djolonga. You only train once: Loss-conditional training of deep networks. In International Conference on Learning Representations , 2020.

EEOC. Uniform guidelines on employee selection procedures, 1979.

B. Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics , 7(1):1-26, 1979.

A. Grover, J. Song, A. Agarwal, K. Tran, A. Kapoor, E. Horvitz, and S. Ermon. Bias Correction of Learned Generative Models Using Likelihood-Free Importance Weighting . Curran Associates Inc., Red Hook, NY, USA, 2019.

M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems , NIPS'16, page 3323-3331, Red Hook, NY, USA, 2016. Curran Associates Inc.

Jigsaw and Google. Unintended bias in toxicity classification, 2019.

S. M. Kakade, K. Sridharan, and A. Tewari. On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems , volume 21. Curran Associates, Inc., 2008.

F. Kamiran and T. Calders. Data pre-processing techniques for classification without discrimination. Knowledge and Information Systems , 33, 10 2011.

J. S. Kim, J. Chen, and A. Talwalkar. Fact: A diagnostic for group fairness trade-offs. CoRR , abs/2004.03424, 2020.

J. S. Kim, J. Chen, and A. Talwalkar. Model-agnostic characterization of fairness trade-offs. CoRR , abs/2004.03424, 2020.

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

L. Le Cam. The central limit theorem around 1935. Statistical Science , 1(1):78-91, 1986.

E. Z. Liu, B. Haghgoo, A. S. Chen, A. Raghunathan, P. W. Koh, S. Sagawa, P. Liang, and C. Finn. Just train twice: Improving group robustness without training group information. CoRR , abs/2107.09044, 2021.

Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , December 2015.

M. Lohaus, M. Perrot, and U. V. Luxburg. Too relaxed to be fair. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 6360-6369. PMLR, 13-18 Jul 2020.

C. Louizos, K. Swersky, Y. Li, M. Welling, and R. Zemel. The variational fair autoencoder, 2017.

K. Lum and J. Johndrow. A statistical framework for fair predictive algorithms, 2016.

N. Martinez, M. Bertran, and G. Sapiro. Minimax pareto fairness: A multi objective perspective. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 6755-6764. PMLR, 13-18 Jul 2020.

M. Olfat and Y. Mintz. Flexible regularization approaches for fairness in deep learning. In 2020 59th IEEE Conference on Decision and Control (CDC) , pages 3389-3394, 2020.

E. Perez, F. Strub, H. de Vries, V . Dumoulin, and A. C. Courville. Film: Visual reasoning with a general conditioning layer. CoRR , abs/1709.07871, 2017.

B. Ustun, Y. Liu, and D. Parkes. Fairness without harm: Decoupled classifiers with preference guarantees. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 6373-6382. PMLR, 09-15 Jun 2019.

A. Valdivia, J. Sánchez-Monedero, and J. Casillas. How fair can we go in machine learning? assessing the boundaries of accuracy and fairness. International Journal of Intelligent Systems , 36(4):1619-1643, 2021.

H. Wang, L. He, R. Gao, and F. Calmon. Aleatoric and epistemic discrimination: Fundamental limits of fairness interventions. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

D. Wei, K. N. Ramamurthy, and F. Calmon. Optimized score transformation for fair classification. In S. Chiappa and R. Calandra, editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of Machine Learning Research , pages 1673-1683. PMLR, 26-28 Aug 2020.

S. Wei and M. Niethammer. The fairness-accuracy pareto front. Statistical Analysis and Data Mining: The ASA Data Science Journal , 15(3):287-302, 2022.

J. Yang, A. A. S. Soltan, D. W. Eyre, Y. Yang, and D. A. Clifton. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. npj Digital Medicine , 6:55, 2023.

M. Zafar, I. Valera, M. Rodriguez, and K. P. Gummadi. Fairness constraints: A mechanism for fair classification. 07 2015.

M. B. Zafar, I. Valera, M. Gomez Rodriguez, and K. P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web , WWW '17, page 1171-1180, Republic and Canton of Geneva, CHE, 2017. International World Wide Web Conferences Steering Committee.

M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gummadi. Fairness constraints: A flexible approach for fair classification. Journal of Machine Learning Research , 20(75):1-42, 2019.

R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In S. Dasgupta and D. McAllester, editors, Proceedings of the 30th International Conference on Machine Learning , volume 28 of Proceedings of Machine Learning Research , pages 325-333, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR.

B. H. Zhang, B. Lemoine, and M. Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society , AIES '18, page 335-340, New York, NY, USA, 2018. Association for Computing Machinery.

A. ¸ Tifrea, P. Lahoti, B. Packer, Y . Halpern, A. Beirami, and F. Prost. Frappé: A group fairness framework for post-processing everything, 2024.

A Proofs

A.1 Confidence intervals on τ ∗ fair

Proof of Lemma 3.1. This lemma is a straightforward application of Heoffding's inequality.

Proof of Proposition 3.2. Here, we prove the result for general classifiers h ∈ H . Let L h acc , U h fair be the lower and upper CIs for acc ( h ) and Φ fair ( h ) respectively,

Then, using a straightforward application union bounds, we get that

Using the definition of the optimal fairness-accuracy trade-off τ ∗ fair , we get that the event

From this, it follows that

Proof of Proposition 3.3. We prove the result for general classifiers h ∈ H . Let U h acc , L h fair be the upper and lower CIs for acc ( h ) and Φ fair ( h ) respectively,

Then, using an application of union bounds, we get that

Then, using the fact that ∆( h ) = Φ fair ( h ) -τ ∗ fair ( acc ( h )) , we get that

where in the second last inequality above, we use the fact that τ ∗ fair : [0 , 1] → [0 , 1] is a monotonically increasing function.

A.2 Asymptotic convergence of ∆( h )

In this Section, we provide the formal statement for Theorem 3.4 along with the assumptions required for this result.

Assumption A.1. τ ∗ fair is L -Lipschitz.

Assumption A.2. Let R |D tr | ( H ) denote the Rademacher complexity of the classifier family H , where |D tr | is the number of training examples. We assume that there exists C ≥ 0 and γ ≤ 1 / 2 such that R |D tr | ( H ) ≤ C |D tr | -γ .

It is worth noting that Assumption A.2, which was also used in [1, Theorem 2], holds for many classifier families with γ = 1 / 2 , including norm-bounded linear functions, neural networks and classifier families with bounded VC dimension [21, 6].

.

Theorem A.3. Let ̂ Φ fair ( h ′ ) , ̂ acc ( h ′ ) denote the fairness violation and accuracy metrics for model h ′ evaluated on training data D tr and define

Then, under Assumptions A.1 and A.2, we have that with probability at least 1 -η , ∆( h ) ≤ ˜ O ( |D tr | -γ ) , where ˜ O ( · ) suppresses polynomial dependence on log (1 /η ) .

Proof of Theorem A.3. Let

Next, we define

Then, we have that

We know using Assumption A.1 that

Moreover,

Putting the two together, we get that

Next, we consider the term ̂ Φ fair ( h ) -̂ Φ fair ( h ∗ ) . First, we observe that

Next, using [1, Lemma 4] with g ( X,A,Y ) = ✶ ( h ( X ) = Y ) , we get that with probability at least 1 -η/ 4 , we have that

This implies that with probability at least 1 -η/ 4 ,

and hence,

Again, using [1, Lemma 4] with g ( X,A,Y ) = ✶ ( h ∗ ( X ) = Y ) , we get that with probability at least 1 -η/ 4 , we have that

Finally, putting this together using union bounds, we get that with probability at least 1 -η/ 2 ,

and hence, and therefore,

Using the definition of h , we have that ̂ acc ( h ∗ ) ≥ δ = ⇒ ̂ Φ fair ( h ) ≤ ̂ Φ fair ( h ∗ ) . Therefore, with probability at least 1 -η/ 2 ,

Next, to bound the term | Φ fair ( h ) -̂ Φ fair ( h ) | above, we consider a general formulation for the fairness violation Φ fair, also presented in [1],

where m ≥ 1 , g j are some known functions and E j are events with positive probability defined with respect to ( X,A,Y ) .

Using this, we note that for any h ′ ∈ H

Let p ∗ j := P ( E j ) . Then, from [1, Lemma 6], we have that if |D tr | p ∗ j ≥ 8 log (4 m/η ) , then with probability at least 1 -η/ 2 m , we have that

A straightforward application of the union bounds yields that if |D tr | p ∗ j ≥ 8 log (4 m/η ) for all j ∈ { 1 , . . . , m } , then with probability at least 1 -η/ 2 , we have that

Therefore, in this case for any h ′ ∈ H , we have that with probability at least 1 -η/ 2 ,

Finally, putting Eq. (5), Eq. (6) and Eq. (7) together using union bounds, we get that with probability at least 1 -η , we have that

B Constructing the confidence intervals on Φ fair ( h )

In this section, we outline methodologies of obtaining confidence intervals for a fairness violation Φ fair. Specifically, given a model h ∈ H , with h : X → Y and α ∈ (0 , 1) , we outline how to find C α fair which satisfies,

Similar to [1] we express the fairness violation Φ fair as:

where m ≥ 1 , g j are some known functions and E j are events with positive probability defined with respect to ( X,A,Y ) . For example, when considering the demographic parity (DP), i.e. Φ fair = Φ DP, we have m = 2 , with g 1 ( X,A,Y,h ( X )) = h ( X ) , E 1 = { A = 1 } , g 2 ( X,A,Y,h ( X )) = -h ( X ) and E 2 = { A = 0 } . Moreover, as shown in [1], the commonly used fairness metrics like Equalized Odds (EO) and Equalized Opportunity (EOP) can also be expressed in similar forms.

Our methodology of constructing CIs on Φ fair ( h ) involves first constructing intervals C α, ± fair on Φ ± fair ( h ) satisfying:

Once we have a C α, ± fair , the confidence interval C α fair satisfying Eq. (8) can simply be constructed as:

In what follows, we outline two different ways of constructing the confidence intervals C α, ± fair on Φ ± fair ( h ) satisfying Eq. (9).

B.1 Separately constructing CIs on Φ j

One way to obtain intervals on Φ ± fair ( h ) would be to separately construct confidence intervals on Φ j , denoted by C α j , which satisfies the joint guarantee

Given such set of confidence intervals { C α j } j m =1 which satisfy Eqn. Eq. (10), we can obtain the confidence intervals on Φ ± fair ( h ) by using the fact that

Where, the notation ∑ m i =1 C α j denotes the set { ∑ m i =1 x i : x i ∈ C α i } . One naïve way to obtain such { C α j } j m =1 which satisfy Eq. (10) is to use the union bounds, i.e., if C α j are chosen such that

then, we have that

Here, for an event E , we use E c to denote the complement of the event. This methodology therefore reduces the problem of finding confidence intervals on Φ ± fair ( h ) to finding confidence intervals on Φ j for j ∈ { 1 , . . . , m } . Now note that Φ j are all expectations and we can use standard methodologies to construct confidence intervals on an expectation. We explicitly outline how to do this in Section B.3.

Remark The methodology outlined above provides confidence intervals with valid finite sample coverage guarantees. However, this may come at the cost of more conservative confidence intervals. One way to obtain less conservative confidence intervals while retaining the coverage guarantees would be to consider alternative ways of obtaining confidence intervals which do not require constructing the CIs separately on Φ j . We outline one such methodology in the next section.

B.2 Using subsampling to construct the CIs on Φ ± fair directly

Here, we outline how we can avoid having to use union bounds when constructing the confidence intervals on Φ ± fair . Let D j denote the subset of data D cal, for which the event E j is true. In the case where the events E j are all mutually exclusive and hence D j are all disjoint subsets of data (which is true for DP, EO and EOP), we can also construct these intervals by randomly sampling without replacement datapoints ( x ( j ) i , a ( j ) i , y ( j ) i ) from D j for i ≤ l := min k ≤ m |D k | . We use the fact that

is an unbiased estimator of Φ ± fair ( h ) . Moreover, since D j are all disjoint datasets, the datapoints ( x ( j ) i , a ( j ) i , y ( j ) i ) are all independent across different values of j , and therefore, ∑ m j =1 g j ( x ( j ) i , a ( j ) i , y ( j ) i , h ( x ( j ) i )) are i.i.d.. In other words,

and ϕ i are all i.i.d. samples and unbiased estimators of Φ ± fair ( h ) . Therefore, like in the previous section, our problem reduces to constructing CIs on an expectation term (i.e. Φ ± fair ( h ) ), using i.i.d. unbiased samples (i.e. ϕ i ) and we can use standard methodologies to construct these intervals.

Benefit of this methodology This methodology no longer requires us to separately construct confidence intervals over Φ j and combine them using union bounds (for example). Therefore, intervals obtained using this methodology may be less conservative than those obtained by separately constructing confidence intervals over Φ j .

Limitation of this methodology For each subset of data D j , we can use at most l := min k ≤ m |D k | data points to construct the confidence intervals. Therefore, in cases where l is very small, we may end up discarding a big proportion of the calibration data which could in turn lead to loose intervals.

B.3 Constructing CIs on expectations

Here, we outline some standard techniques used to construct CIs on the expectation of a random variable. These techniques can then be used to construct CIs on Φ fair ( h ) (using either of the two methodologies outlined above) as well as on acc ( h ) . In this section, we restrict ourselves to constructing upper CIs. Lower CIs can be constructed analogously.

Given dataset { Z i : 1 ≤ i ≤ n } , our goal in this section is to construct upper CIs on E [ Z ] which satisfies

Hoeffding's inequality We can use Hoeffding's inequality to construct these intervals U α as formalised in the following result:

Lemma B.1 (Hoeffding's inequality) . Let Z i ∈ [0 , 1] , 1 ≤ i ≤ n be i.i.d. samples with mean E [ Z ] . Then,

Bernstein's inequality Bernstein's inequality provides a powerful tool for bounding the tail probabilities of the sum of independent, bounded random variables. Specifically, for a sum ∑ n i =1 Z i

comprised of n independent random variables Z i with Z i ∈ [0 , B ] , each with a maximum variance of σ 2 , and for any t > 0 , the inequality states that

where B denotes an upper bound on the absolute value of each random variable. Re-arranging the above, we get that

This allows us to construct upper CIs on E [ Z ] .

Central Limit Theorem The Central Limit Theorem (CLT) [26] serves as a cornerstone in statistics for constructing confidence intervals around sample means, particularly when the sample size is substantial. The theorem posits that, for a sufficiently large sample size, the distribution of the sample mean will closely resemble a normal (Gaussian) distribution, irrespective of the original population's distribution. This Gaussian nature of the sample mean empowers us to form confidence intervals for the population mean using the normal distribution's characteristics.

Given Z 1 , Z 2 , . . . , Z n as n independent and identically distributed (i.i.d.) random variables with mean µ and variance σ 2 , the sample mean ¯ Z approximates a normal distribution with mean µ and variance σ 2 /n for large n . An upper (1 -α ) confidence interval for µ is thus:

where z α/ 2 represents the critical value from the standard normal distribution corresponding to a cumulative probability of 1 -α .

Bootstrap Confidence Intervals Bootstrapping, introduced by [17], offers a non-parametric approach to estimate the sampling distribution of a statistic. The method involves repeatedly drawing samples (with replacement) from the observed data and recalculating the statistic for each resample. The resulting empirical distribution of the statistic across bootstrap samples forms the basis for confidence interval construction.

Given a dataset Z 1 , Z 2 , . . . , Z n , one can produce B bootstrap samples by selecting n observations with replacement from the original data. For each of these samples, the statistic of interest (for instance, the mean) is determined, yielding B bootstrap estimates. An upper (1 -α ) bootstrap confidence interval for E [ Z ] is given by:

with z ∗ α denoting the α -quantile of the bootstrap estimates. It's worth noting that there exist multiple methods to compute bootstrap confidence intervals, including the basic, percentile, and bias-corrected approaches, and the method described above serves as a general illustration.

C Sensitivity analysis for ∆( h )

Recall from Proposition 3.3 that the lower confidence intervals for τ ∗ fair include a ∆( h ) term which is defined as

In other words, ∆( h ) quantifies how 'far' the fairness loss of classifier h (i.e. Φ fair ( h ) ) is from the minimum attainable fairness loss for classifiers with accuracy acc ( h ) , (i.e. τ ∗ fair ( acc ( h )) ). This quantity is unknown in general and therefore, a practical strategy of obtaining lower confidence intervals on τ ∗ fair ( ψ ) may involve positing values for ∆( h ) which encode our belief on how close the fairness loss Φ fair ( h ) is to τ ∗ fair ( acc ( h )) . For example, when we assume that the classifier h achieves the optimal accuracy-fairness tradeoff, i.e. Φ fair ( h ) = τ ∗ fair ( acc ( h )) then ∆( h ) = 0 .

However, the assumption Φ fair ( h ) = τ ∗ fair ( acc ( h )) may not hold in general because we only have a finite training dataset and consequently the empirical loss minimisation may not yield the optima to the true expected loss. Moreover, the regularised loss used in training h is a surrogate loss which

approximates the solution to the constrained minimisation problem in Eq. (1). This means that optimising this regularised loss is not guaranteed to yield the optimal classifier which achieves the optimal fairness τ ∗ fair ( acc ( h )) . Therefore, to incorporate any belief on the sub-optimality of the classifier h , we may consider conducting sensitivity analyses to plausibly quantify ∆( h ) .

Let h θ : X × Λ → R be the YOTO model. Our strategy for sensitivity analysis involves training multiple standard models M := { h (1) , h (2) , . . . , h ( k ) } ⊆ H by optimising the regularised losses for few different choices of λ .

Importantly, we do not require covering the full range of λ values when training separate models M , and our methodology remains valid even when M is a single model. Next, let h ∗ λ ∈ M∪{ h θ ( · , λ ) } be such that

Here, ̂ Φ fair and ̂ acc denote the finite sample estimates of the fairness loss and model accuracy respectively. We treat the model h ∗ λ as a model which attains the optimum trade-off when estimating subject to the constraint acc ( h ) ≥ acc ( h θ ( · , λ )) . Specifically, we use the maximum empirical error max λ ̂ ∆( h θ ( · , λ )) as a plausible surrogate value for ∆( h θ ( · , λ ′ )) , where ̂ ∆( h θ ( · , λ )) := ̂ Φ fair ( h θ ( · , λ )) -̂ Φ fair ( h ∗ λ ) ≥ 0 , i.e., we posit for any λ ′ ∈ Λ

Next, we can use this posited value of ∆( h θ ( · , λ ′ )) to construct the lower confidence interval using the following corollary of Proposition 3.3:

Corollary C.1. Consider the YOTO model h θ : X × Λ → R . Given λ 0 ∈ Λ , let U h acc , L h fair ∈ [0 , 1] be such that

Then, we have that P ( τ ∗ fair ( U h acc ) ≥ L h fair -∆( h θ ( · , λ 0 ))) ≥ 1 -α.

This result shows that if the goal is to construct lower confidence intervals on τ ∗ fair ( ψ ) and we obtain that ψ ≥ U h acc , then using the monotonicity of τ ∗ fair we have that τ ∗ fair ( ψ ) ≥ τ ∗ fair ( U h acc ) . Therefore the interval [ L h fair -∆( h θ ( · , λ 0 ))) , 1] serves as a lower confidence interval for τ ∗ fair ( ψ ) .

When YOTO satisfies Pareto optimality, ∆( h θ ( · , λ )) → 0 as |D cal | → ∞ : Here, we show that in the case when YOTO achieves the optimal trade-off, then our sensitivity analysis leads to ∆( h θ ( · , λ )) = 0 as the calibration data size increases for all λ ∈ Λ . Our arguments in this section are not formal, however, this idea can be formalised without any significant difficulty.

First, the concept of Pareto optimality (defined below) formalises the idea that YOTO achieves the optimal trade-off:

Assumption C.2 (Pareto optimality) .

If for some λ ∈ Λ and h ′ ∈ H we have that, acc ( h ′ ) ≥ acc ( h θ ( · , λ )) then, Φ fair ( h ′ ) ≥ Φ fair ( h θ ( · , λ )) ,

In the case when YOTO satisfies this optimality property, then it is straightforward to see that ∆( h θ ( · , λ )) = 0 for all λ ∈ Λ . In this case, as D cal →∞ , we get that Eq. (11) roughly becomes

Here, Assumption C.2 implies that h ∗ λ = h θ ( · , λ ) , and therefore

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(a) No sensitivity analysis

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(b)

|M|

= 2

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(c)

|M|

= 5

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(a) No sensitivity analysis

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(b)

|M|

= 2

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(c)

|M|

= 5

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(a) No sensitivity analysis

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(b)

|M|

= 2

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(c)

|M|

= 5

Intuition behind our sensitivity analysis procedure Intuitively, the high-level idea behind our sensitivity analysis is that it checks if we train models separately for fixed values of λ (i.e. models in M ), how much better do these separately trained models perform in terms of the accuracy-fairness trade-offs as compared to our YOTO model. If we find that the separately trained models achieve a better trade-off than the YOTO model for specific values of λ , then the sensitivity analysis adjusts the empirical trade-off obtained using YOTO models (using the ̂ ∆( h θ ( · , λ )) term defined above). If, on the other hand, we find that the YOTO model achieves a better trade-off than the separately trained models in M , then the sensitivity analysis has no effect on the lower confidence intervals as in this case ̂ ∆( h θ ( · , λ )) = 0 .

C.1 Experimental results

Here, we include empirical results showing how the CIs constructed change as a result of our sensitivity analysis procedure. In Figures 4 and 5, we include examples of CIs where the empirical trade-off obtained using YOTO is sub-optimal. In these cases, the lower CIs obtained without sensitivity analysis (i.e. when we assume ∆( h λ ) = 0 ) do not cover the empirical trade-offs for the separately trained models. However, the figures show that the sensitivity analysis procedure adjusts the lower CIs in both cases so that they encapsulate the empirical trade-offs that were not captured without sensitivity analysis.

Recall that M represents the set of additional separately trained models used for the sensitivity analysis. It can be seen from Figures 4 and 5 that in both cases our sensitivity analysis performs well with as little as two models (i.e. |M| = 2 ), which shows that our sensitivity analysis does not come at a high computational cost.

Tables 2 and 3 contain results corresponding to these figures and show the proportion of trade-offs which lie in the three trade-off regions shown in Figure 1 with and without sensitivity analysis. It can be seen that in both tables, when |M| = 2 , the proportion of trade-offs which lie below the lower CIs (blue region in Figure 1) is negligible.

Additionally, in Figure 6 we also consider an example where YOTO achieves a better empirical trade-off than most other baselines considered, and therefore there is no need for sensitivity analysis. In this case, Figure 6 (and Table 4) show that sensitivity analysis has no effect on the CIs constructed since in this case sensitivity analysis gives us ̂ ∆( h θ ( · , λ )) = 0 for λ ∈ Λ . This shows that in cases where sensitivity analysis is not needed (for example, if YOTO achieves optimal empirical trade-off), our sensitivity analysis procedure does not make the CIs more conservative.

D Scarce sensitive attributes

Our methodology of obtaining confidence intervals on Φ fair assumes access to the sensitive attributes A for all data points in the held-out dataset D . However, in practice, we may only have access to A for a small proportion of the data in D . In this case, a naïve strategy would involve constructing confidence intervals using only the data for which A is available. However, since such data is scarce, the confidence intervals constructed are very loose.

Suppose that we additionally have access to a predictive model f A which predicts the sensitive attributes A using the features X . In this case, another simple strategy would be to simply impute the missing values of A , with the values ˆ A predicted using f A . However, this will usually lead to a biased estimate of the fairness violation Φ fair ( h ) , and hence is not very reliable unless the model f A is highly accurate. In this section, we show how to get the best of both worlds, i.e. how to utilise the data with missing sensitive attributes to obtain tighter and more accurate confidence intervals on τ ∗ fair ( ψ ) .

Formally, we consider D cal = D∪ ˜ D where D denotes a data subset of size n that contains sensitive attributes (i.e. we observe A ) and ˜ D denotes the data subset of size N for which we do not observe the sensitive attributes A , and N ≫ n . Additionally, for both datasets, we have predictions of the sensitive attributes made by a machine-learning algorithm f A : X → A , where f A ( X ) ≈ A . Concretely we have that D = { ( X i , A i , Y i , f A ( X i )) } n i =1 and ˜ D = { ( ˜ X i , ˜ Y i , f A ( ˜ X i )) } N i =1

High-level methodology Our methodology is inspired by prediction-powered inference [4] which builds confidence intervals on the expected outcome E [ Y ] using data for which the true outcome Y is only available for a small proportion of the dataset. In our setting, however, it is the sensitive attribute A that is missing for the majority of the data (and not the outcome Y ).

For h ∈ H , let Φ fair ( h ) be a fairness violation (such as DP or EO), and let ˜ Φ fair ( h ) be the corresponding fairness violation computed on the data distribution where A is replaced by the surrogate sensitive attribute f A ( X ) . For example, in the case of DP violation, Φ fair ( h ) and ˜ Φ fair ( h ) denote:

We next construct the confidence intervals on Φ fair ( h ) using the following steps:

Even though the size of D is small, we choose a methodology which yields tight intervals for ϵ ( h ) when f A ( X i ) = A i with a high probability.

Next, using the dataset ˜ D , we construct intervals ˜ C f ( α ; h ) on ˜ Φ fair ( h ) satisfying

This interval will also be tight as the size of ˜ D , N ≫ n .

Finally, using the union bound idea we combine the two confidence intervals to obtain the confidence interval for Φ fair ( h ) -˜ Φ fair ( h ) + ˜ Φ fair ( h ) = Φ fair ( h ) . We make this precise in the following result:

Lemma D.1. Let C ϵ ( α ; h ) , ˜ C f ( α ; h ) be as defined in equations 12 and 13. Then, if we define C α fair ( h ) = { x + y | x ∈ C ϵ ( α ; h ) , y ∈ ˜ C f ( α ; h ) } , we have that

When constructing the CIs over ˜ Φ fair ( h ) using imputed sensitive attributes f A ( X ) in step 2 above, the prediction error of f A introduces an error in the obtained CIs (denoted by ϵ ( h ) ). Step 1 rectifies this by constructing a CI over the incurred error ϵ ( h ) , and therefore combining the two allows us to obtain intervals which utilise all of the available data while ensuring that the constructed CIs are well-calibrated.

Example: Demographic parity Having defined our high-level methodology above, we concretely demonstrate how this can be applied to the case where the fairness loss under consideration is DP. As described above, the first step involves constructing intervals on ϵ ( h ) := Φ fair ( h ) -˜ Φ fair ( h ) using a methodology which yields tight intervals when f A ( X i ) = A i with a high probability. To this end, we use bootstrapping as described in Algorithm 1.

Even though bootstrapping does not provide us with finite sample coverage guarantees, it is asymptotically exact and satisfies the property that the confidence intervals are tight when ˆ A = A with a high

probability. On the other hand, concentration inequalities (such as Hoeffding's inequality) seek to construct confidence intervals individually on Φ fair ( h ) and ˜ Φ fair ( h ) and subsequently combine them through union bounds argument, for example. In doing so, these methods do not account for how close the values of Φ fair ( h ) and ˜ Φ fair ( h ) might be in the data.

To make this concrete, consider the example where f A ( X ) a.s. = A and hence Φ fair ( h ) = ˜ Φ fair ( h ) . When using concentration inequalities to construct the 1 -α confidence intervals on Φ fair ( h ) and ˜ Φ fair ( h ) , we obtain identical intervals for the two quantities, say [ l, u ] . Then, using union bounds we obtain that Φ fair ( h ) -˜ Φ fair ( h ) ∈ [ l -u, u -l ] with probability at least 1 -2 α . In this case even though Φ fair ( h ) -˜ Φ fair ( h ) = 0 , the width of the interval [ l -u, u -l ] does not depend on the closeness of Φ fair ( h ) and ˜ Φ fair ( h ) and therefore is not tight. Bootstrapping helps us circumvent this problem, since in this case for each resample of the data D , the finite sample estimates ̂ Φ fair ( h ) and ̂ ˜ Φ fair ( h ) will be equal. We outline the bootstrapping algorithm below.

Using Algorithm 1 we construct a confidence interval C ϵ ( α ; h ) on ϵ ( h ) of size 1 -α , which approximately satisfies Eq. (12). Next, using standard techniques we can obtain an interval ˜ C f ( α ; h ) on ˜ Φ fair ( h ) using ˜ D which satisfies Eq. (13). Like before, the interval ˜ C f ( α ; h ) is likely to be tight as we use ˜ D to construct it, which is significantly larger than D . Finally, combining the two as shown in Lemma D.1, we obtain the confidence interval on Φ fair ( h ) .

D.1 Experimental results

Here, we present experimental results in the setting where the sensitive attributes are missing for majority of the calibration data. Figures 7-12 show the results for different datasets and predictive models f A with varying accuracies. Here, the empirical fairness violation values for both YOTO and separately trained models are evaluated using the true sensitive attributes over the entire calibration data.

CIs with imputed sensitive attributes are mis-calibrated Figures 7, 9 and 11 show results for Adult, COMPAS and CelebA datasets, where the CIs are computed by imputing the missing sensitive attributes with the predicted sensitive attributes f A ( X ) ≈ A . The figures show that when the accuracy of f A is below 90%, the CIs are highly miscalibrated as they do not entirely contain the empirical trade-offs for both YOTO and separately trained models.

Our methodology corrects for the mis-calibration In contrast, Figures 8, 10 and 12 which include the corresponding results using our methodology, show that our methodology is able to correct for the mis-calibration in CIs arising from the prediction error in f A . Even though the CIs obtained using our methodology are more conservative than those obtained by imputing the missing sensitive attributes with f A ( X ) , they are more well-calibrated and contain the empirical trade-offs for both YOTO and separately trained model.

Imputing missing sensitive attributes may work when f A has high accuracy Finally, Figures 7c, 9c and 11c show that the CIs with imputed sensitive attributes are relatively better calibrated as the

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(a) acc

(

f

A

) = 50%

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(b) acc

(

f

A

) = 70%

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(c) acc

(

f

A

) = 90%

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(a) acc

(

f

A

) = 50%

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(b) acc

(

f

A

) = 70%

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(c) acc

(

f

A

) = 90%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.00

0.05

0.10

0.15

Demographic Parity

(a) acc

(

f

A

) = 50%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.00

0.05

0.10

0.15

Demographic Parity

(b) acc

(

f

A

) = 70%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.00

0.05

0.10

0.15

Demographic Parity

(c) acc

(

f

A

) = 90%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.00

0.05

0.10

0.15

Demographic Parity

(a) acc

(

f

A

) = 50%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(b) acc

(

f

A

) = 70%

0.56

0.58

0.60

0.62

0.64

0.66

0.68

0.00

0.05

0.10

0.15

Demographic Parity

(c) acc

(

f

A

) = 90%

accuracy of f A increases to 90%. In this case, the CIs with imputed sensitive attributes mostly contain empirical trade-offs. This shows that in cases where the predictive model f A has high accuracy, it may be sufficient to impute missing sensitive attributes with f A ( X ) when constructing the CIs.

E Accounting for the uncertainty in baseline trade-offs

In this section, we extend our methodology to also account for uncertainty in the baseline trade-offs when assessing the optimality of different baselines. Recall that our confidence intervals constructed

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(a) acc

(

f

A

) = 50%

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(b) acc

(

f

A

) = 70%

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(c) acc

(

f

A

) = 90%

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(a) acc

(

f

A

) = 50%

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(b) acc

(

f

A

) = 70%

0.86

0.88

0.90

0.92

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(c) acc

(

f

A

) = 90%

on τ ∗ fair satisfy the guarantee

This means that if the accuracy-fairness tradeoff for a given model h ′ ∈ H , ( acc ( h ′ ) , Φ fair ( h ′ )) , lies above the confidence intervals Γ α fair (i.e. in the pink region in Figure 13), then we can confidently infer that the model h ′ achieves a suboptimal trade-off. This is because we know from the probabilistic guarantee above that the optimal trade-off ( acc ( h ′ ) , τ ∗ fair ( acc ( h ′ ))) must lie in the intervals Γ α fair with probability at least 1 -α .

Here, acc ( h ′ ) , Φ fair ( h ′ ) denote the accuracy and fairness violations for model h ′ on the full data distribution. However, in practice, we only have access to finite data and therefore can only compute the empirical values of accuracy and fairness violations which we denote by ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ ) . This means that when checking if the accuracy-fairness trade-off ( acc ( h ′ ) , Φ fair ( h ′ )) lies inside the confidence intervals Γ α fair , we must account for the uncertainty in the empirical estimates ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ ) . This can be achieved by constructing confidence regions C α ( h ′ ) satisfying

Baseline's best-case accuracy-fairness trade-off If the confidence region C α ( h ′ ) lies entirely above the confidence intervals Γ α fair (i.e. in the pink region in Figure 13a), then using union bounds we can confidently conclude that with probability 1 -2 α we have that the model h ′ achieves suboptimal fairness violation, i.e.

This allows practitioners to confidently check if a model h ′ is suboptimal in terms of its accuracyfairness trade-off. This is different from simply checking if the empirical trade-off achieved by the model ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) lies in the permissible trade-off region (green region in Figure 1) as it also accounts for the finite-sampling uncertainty in the tradeoff achieved by model h ′ . However, this means that the criterion for flagging a baseline model h ′ as suboptimal becomes more conservative.

Next, we show how to construct confidence regions C α ( h ′ ) satisfying Eq. (14).

Lemma E.1. For a classifier h ′ ∈ H , let U h acc , L h fair be the upper and lower CIs on acc ( h ′ ) , Φ fair ( h ′ ) respectively,

where

This would indicate that the YOTO model does not achieve the optimal trade-off and can be used to further calibrate the ∆( h λ ) values when constructing the lower confidence interval using Proposition 3.3. Again, this is different from simply checking if the empirical trade-off achieved by the model ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) lies in the unlikely-to-be-achieved trade-off region (blue region in Figure 13) as it also accounts for the finite-sampling uncertainty in the tradeoff achieved by model h ′ .

Next, we show to construct such confidence region C α ( h ′ ) using an approach analogous to the one outlined above:

Lemma E.2. For a classifier h ′ ∈ H , let L h acc , U h fair be the lower and upper CIs on acc ( h ′ ) , Φ fair ( h ′ ) respectively,

Then, P (( acc ( h ′ ) , Φ fair ( h ′ )) ∈ [ L h acc , 1] × [0 , U h fair ]) ≥ 1 -α .

Supoptimal accuracy-fairness trade-off Permissible trade-off region

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Fairness Violation

(

acc

(

h

),

fair

(

h

))

(

U h

acc

,

L h

fair

)

Unlikely to be achievable trade-off Confidence region for ( acc ( h ), fair ( h ))

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Fairness Violation

(

acc

(

h

),

fair

(

h

))

(

L h

acc

,

U h

fair

)

Lemma E.1 shows that C α ( h ′ ) = [0 , U h acc ] × [ L h fair , 1] forms the 1 -α confidence region for the accuracy-fairness trade-off ( acc ( h ′ ) , Φ fair ( h ′ )) . We illustrate this in Figure 13a. If this confidence region lies entirely above the permissible region (i.e. in the pink region in Figure 13), we can confidently conclude that the model h ′ achieves suboptimal accuracy-fairness trade-off. From Figure 13a it can be seen that this will occur if ( U h acc , L h fair ) lies above the permissible region.

Intuitively, ( U h acc , L h fair ) can be seen as an optimistic best-case accuracy-fairness trade-off achieved by the model h ′ , since at this point in the confidence region C α ( h ′ ) the accuracy is maximised and fairness violation is minimised. Therefore if this best-case trade-off lies above the permissible region, then this intuitively indicates that the worst-case optimal trade-off τ ∗ fair is still better than the best-case trade-off achieved by the model h ′ , leading us to the confident conclusion that h ′ achieves suboptimal accuracy-fairness trade-off.

Baseline's worst-case accuracy-fairness trade-off Conversely, if the confidence region C α ( h ′ ) lies entirely below the confidence intervals Γ α fair (i.e. in the blue region in Figure 13), then using union bounds we can confidently conclude that with probability 1 -2 α we have that the model h ′ achieves a better trade-off than the YOTO model h θ ( · , λ ) . Formally, this means that

Lemma E.2 shows that C α ( h ′ ) = [ L h acc , 1] × [0 , U h fair ] forms the 1 -α confidence region for the accuracy-fairness trade-off ( acc ( h ′ ) , Φ fair ( h ′ )) . If this confidence region lies entirely below the permissible region (i.e. in the blue region in Figure 13), we can confidently conclude that the model h ′ achieves a better accuracy-fairness trade-off than the YOTO model. From Figure 13b it can be seen that this will occur if ( L h acc , U h fair ) lies below the permissible region.

Intuitively, ( L h acc , U h fair ) can be seen as a conservative worst-case accuracy-fairness trade-off achieved by the model h ′ . Therefore if this worst-case trade-off lies below the permissible region, then this intuitively indicates that the best-case YOTO trade-off τ YOTO is still worse than the worst-case trade-off achieved by the model h ′ , leading us to the confident conclusion that h ′ achieves a better accuracy-fairness trade-off than the YOTO model. This can subsequently be used to calibrate the suboptimality gap for the YOTO model, denoted by ∆( h λ ) in Proposition 3.3.

E.1 Experimental results

Here, we present the results with the empirical baseline trade-offs replaced by best or worst-case trade-offs as appropriate. More specifically, in Figure 14,

if the empirical trade-off for a baseline ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) lies in the suboptimal region (as in Figure 13a), then we plot the best-case trade-off ( U h acc , L h fair ) for the baseline,

if the empirical trade-off for a baseline ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) lies in the unlikely-to-beachievable trade-off region (as in Figure 13b), then we plot the worst-case trade-off ( L h acc , U h fair ) for the baseline,

if the empirical trade-off for a baseline ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) lies in the permissible region, then we simply plot the empirical trade-off ( ̂ acc ( h ′ ) , ̂ Φ fair ( h ′ )) .

Therefore, in Figure 14 if a baseline's best-case trade-off lies above the permissible trade-off region, then we can confidently conclude that the baseline achieves suboptimal accuracy-fairness tradeoff with probability 1 -2 α = 90% . Similarly, a baseline's worst-case trade-off lying below the permissible trade-off region would suggest that the YOTO trade-off achieves a suboptimal trade-off and that the value of ∆( h θ ) needs to be adjusted accordingly.

Table 5 shows the proportion of best-case trade-offs which lie above the permissible trade-off region and the proportion of worst-case trade-offs which lie below the permissible region. Firstly, the table shows that the proportion of worst-case trade-offs which lie in the 'unlikely' region is negligible, empirically confirming that our confidence intervals on optimal trade-off τ ∗ fair are indeed valid. Secondly, we can see that there are a considerable proportion of baselines whose best-case trade-off lies above the permissible region, highlighting that our methodology remains effective in flagging suboptimalities in SOTA baselines even when we account for the possible uncertainty in baseline trade-offs. This shows that our methodology yields CIs which are not only reliable but also informative.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

Adult dataset

COMPAS dataset

CelebA dataset

Jigsaw dataset

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

0.50

0.55

0.60

0.65

0.70

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.5

0.6

0.7

0.8

0.9

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.55

0.60

0.65

0.70

0.75

0.00

0.01

0.02

0.03

0.04

0.05

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.88

0.89

0.90

0.91

0.92

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.5

0.6

0.7

0.8

0.9

0.00

0.02

0.04

0.06

0.08

0.10

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

F Experimental details and additional results

In this section, we provide greater details regarding our experimental setup and models used. We first begin by defining the Equalized Odds metric which has been used in our experiments, along with DP and EOP.

Equalized Odds (EO): EO condition states that, both the true positive rates and false positive rates for all sensitive groups are equal, i.e. P ( h ( X ) = 1 | A = a, Y = y ) = P ( h ( X ) = 1 | Y = y ) for any a ∈ A and y ∈ { 0 , 1 } . The absolute EO violation is defined as:

Next, we provide additional details regarding the YOTO model.

F.1 Practical details regarding YOTO model

As described in Section 3, we consider optimising regularized losses of the form

When training YOTO models, instead of fixing λ , we sample the parameter λ from a distribution P λ . As a result, during training the model observes many different values of λ and learns to optimise the loss L λ for all of them simultaneously. At inference time, the model can be conditioned on a chosen parameter value λ ′ and recovers the model trained to optimise L λ ′ . The loss being minimised can thus be expressed as follows:

The fairness losses L fair considered for the YOTO model are:

Here, σ ( x ) := 1 / (1 + e -x ) denotes the sigmoid function.

In our experiments, we sample a new λ for every batch. Moreover, we use the log-uniform distribution as per [15] as the sampling distribution P λ , where the uniform distribution is U [10 -6 , 10] . To condition the network on λ parameters, we follow in the footsteps of [15] to use Feature-wise Linear Modulation (FiLM) [34]. For completeness, we include the description of the architecture next.

Initially, we determine which network layers should be conditioned, which can encompass all layers or just a subset. For each chosen layer, we condition it based on the weight parameters λ . Given a layer that yields a feature map f with dimensions W × H × C , where W and H denote the spatial dimensions and C stands for the channels, we introduce the parameter vector λ to two distinct multi-layer perceptrons (MLPs), denoted as M σ and M µ . These MLPs produce two vectors, σ and µ , each having a dimensionality of C . The feature map is then transformed by multiplying it channel-wise with σ and subsequently adding µ . The resultant transformed feature map f ′ is given by:

Next, we provide exact architectures we used for each dataset in our experiments.

F.1.1 YOTO Architectures

Adult and COMPAS dataset Here, we use a simple logistic regression as the main model, with only the scalar logit outputs of the logistic regression being conditioned using FiLM. The MLPs M µ , M σ both have two hidden layers, each of size 4, and ReLU activations. We train the model for a maximum of 1000 epochs, with early stopping based on validation losses. Training these simple models takes roughly 5 minutes on a Tesla-V100-SXM2-32GB GPU.

CelebA dataset For the CelebA dataset, our architecture is a convolutional neural network (ConvNet) integrated with the FiLM (Feature-wise Linear Modulation) mechanism. The network starts with two convolutional layers: the first layer has 32 filters with a kernel size of 3 × 3 , and the second layer has 64 filters, also with a 3 × 3 kernel. Both convolutional layers employ a stride of 1 and are followed by a max-pooling layer that reduces each dimension by half.

The feature maps from the convolutional layers are flattened and passed through a series of fully connected (MLP) layers. Specifically, the first layer maps the features to 64 dimensions, and the subsequent layers maintain this size until the final layer, which outputs a scalar value. The activation function used in these layers is ReLU.

To condition the network on the λ parameter using FiLM, we design two multi-layer perceptrons (MLPs), M µ and M σ . Both MLPs take the λ parameter as input and have 4 hidden layers. Each of these hidden layers is of size 256. These MLPs produce the modulation parameters µ and σ , which are used to perform feature-wise linear modulation on the outputs of the main MLP layers. The final output of the network is passed through a sigmoid activation function to produce the model's prediction. We train the model for a maximum of 1000 epochs, with early stopping based on validation losses. Training this model takes roughly 1.5 hours on a Tesla-V100-SXM2-32GB GPU.

Jigsaw dataset For the Jigsaw dataset, we employ a neural network model built upon the BERT architecture [13] integrated with the Feature-wise Linear Modulation (FiLM) mechanism. We utilize the representation corresponding to the [CLS] token, which carries aggregate information about the entire sequence. To condition the BERT's output on the λ parameter using FiLM, we design two linear layers, which map the λ parameter to modulation parameters γ and β , both of dimension equal to BERT's hidden size of 768. These modulation parameters are then used to perform feature-wise linear modulation on the [CLS] representation. The modulated representation is passed through a classification head, which consists of a linear layer mapping from BERT's hidden size (768) to a scalar output. In terms of training details, our model is trained for a maximum of 10 epochs, with early stopping based on validation losses. Training this model takes roughly 6 hours on a Tesla-V100-SXM2-32GB GPU.

F.2 Datasets

We used four real-world datasets for our experiments.

Adult dataset The Adult income dataset [7] includes employment data for 48,842 individuals where the task is to predict whether an individual earns more than $50k per year and includes demographic

attributes such as age, race and gender. In our experiments, we consider gender as the sensitive attribute.

COMPAS dataset The COMPAS recidivism data comprises collected by ProPublica [5], includes information for 6172 defendants from Broward County, Florida. This information comprises 52 features including defendants' criminal history and demographic attributes, and the task is to predict recidivism for defendants. The sensitive attribute in this dataset is the defendants' race where A = 1 represents 'African American' and A = 0 corresponds to all other races.

CelebA dataset The CelebA dataset [28] consists of 202,599 celebrity images annotated with 40 attribute labels. In our task, the objective is to predict whether an individual in the image is smiling. The dataset comprises features in the form of image pixels and additional attributes such as hairstyle, eyeglasses, and more. The sensitive attribute for our experiments is gender.

Jigsaw Toxicity Classification dataset The Jigsaw Toxicity Classification dataset [20] contains online comments from various platforms, aimed at identifying and mitigating toxic behavior online. The task is to predict whether a given comment is toxic or not. The dataset includes features such as the text of the comment and certain metadata such as the gender or race to which each comment relates. In our experiments, the sensitive attribute is the gender to which the comment refers, and we only filter the comments which refer to exactly one of 'male' or 'female' gender. This leaves us with 107,106 distinct comments.

F.3 Baselines

The baselines considered in our experiments include:

Regularization based approaches [8]: These methods seek to minimise fairness loss using regularized losses as shown in Section 3. We consider different regularization terms L fair ( h θ ) as smooth relaxations of the fairness violation Φ fair as proposed in the literature [8]. To make this concrete, when the fairness violation under consideration is DP, we consider

with g ( x ) = x denoted as 'linear' in our results and g ( x ) = log σ ( x ) where σ ( x ) := 1 / (1 + e -x ) , denoted as 'logsig' in our results. In addition to these methods, we also consider separately trained models with the same regularization term as the YOTO models, i.e.,

Here, σ ( x ) := 1 / (1 + e -x ) denotes the sigmoid function. We denote these models as 'separate' in our experimental results as they are the separately trained counterparts to the YOTO model. For each relaxation, we train models for a range of λ values uniformly chosen in [0 , 10] interval.

Reductions Approach [1]: This method transforms the fairness problem into a sequence of cost-sensitive classification problems. Like the regularization approaches this requires multiple models to be trained. Here, to try and reproduce the trade-off curves, we train the reductions approach with a range of different fairness constraints uniformly in [0 , 1] .

KDE-fair [12]: This method employs a kernel density estimation trick to quantify fairness measures, capturing the degree of irrelevancy of prediction outputs to sensitive attributes. These quantified fairness measures are expressed as differentiable functions with respect to classifier model parameters, allowing the use of gradient descent to solve optimization problems that respect fairness constraints. We focus on binary classification and well-known definitions of group fairness: Demographic Parity (DP), Equalized Odds (EO) and Equalized Opportunity (EOP).

Randomized Threshold Optimizer (RTO) [2]: This scalable post-processing algorithm debiases trained models, including deep neural networks, and is proven to be near-optimal by bounding its excess Bayes risk. RTO optimizes the trade-off curve by applying adjustments

to the trained model's predictions to meet fairness constraints. In our experiments, we first train a standard classifier which is referred to as the base classifier. Next, we use the RTO algorithm to apply adjustments to the trained model's prediction to meet fairness constraints. To reproduce the trade-off curves, we apply post-hoc adjustments to the base classifier for a range of fairness violation constraints.

Adversarial Approaches [45]: These methods utilize an adversarial training paradigm where an additional model, termed the adversary, is introduced during training. The primary objective of this adversary is to predict the sensitive attribute A using the predictions h θ ( X ) generated by the main classifier h θ . The training process involves an adversarial game between the primary classifier and the adversary, striving to achieve equilibrium. This adversarial dynamic ensures that the primary classifier's predictions are difficult to use for determining the sensitive attribute A , thereby minimizing unfair biases associated with A . Specifically, for DP constraints, the adversary takes the logit outputs of the classifier as the input and predicts A . In contrast for EO and EOP constraints, the adversary also takes the true label Y as the input. For EOP constraint, the adversary is only trained on data with Y = 1 .

The training times for each baseline on the different datasets have been listed in Table 6.

F.4 Additional results

In Figures 15-26 we include additional results for all datasets and fairness violations with an increasing number of calibration data D cal. It can be seen that as the number of calibration data increases, the CIs constructed become increasingly tighter. However, the asymptotic, Bernstein and bootstrap CIs are informative even when the calibration data is as little as 500 for COMPAS data (Figures 18-20) and 1000 for all other datasets. These results show that the larger the calibration data D cal , the tighter the constructed CIs are likely to be. However, even in cases where the calibration dataset is relatively small, we obtain informative CIs in most cases.

Tables 7 - 18 show the proportion of the baselines which lie in the three trade-off regions, 'Unlikely', 'Permissible' and 'Sub-optimal' for the Bernstein's CIs, across the different datasets and fairness metrics. Overall, it can be seen that our CIs are reliable since the proportion of trade-offs which lie below our CIs is small. On the other hand, the tables show that our CIs can detect a significant number of sub-optimalities in our baselines, showing that our intervals are informative.

F.5 Additional details for Figure 1

For Figure 1, we consider a calibration data D cal of size 2000. For the optimal model, we trained a YOTO model on the COMPAS dataset with the architecture given in F.1.1. We train the model for a maximum of 1000 epochs, with early stopping based on validation losses. Once trained, we obtained the red and black curves by first splitting D cal randomly into two subsets, and evaluating the trade-offs on each split separately. On the other hand, for the sub-optimal model, we use another YOTO model with the same architecture but stop the training after 20 epochs. This results in the trained model achieving a sub-optimal trade-off as shown in the figure. We evaluated this trade-off curve on the entire calibration data.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(a)

|D

cal

|

= 1000

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(b)

|D

cal

|

= 5000

0.76

0.78

0.80

0.82

0.84

0.86

0.00

0.05

0.10

0.15

0.20

Demographic Parity

(c)

|D

cal

|

= 10

,

000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

(a)

|D

cal

|

= 1000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

(b)

|D

cal

|

= 5000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

(c)

|D

cal

|

= 10

,

000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(a)

|D

cal

|

= 1000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(b)

|D

cal

|

= 5000

0.74

0.76

0.78

0.80

0.82

0.84

0.00

0.05

0.10

0.15

0.20

Equalized Odds

(c)

|D

cal

|

= 10

,

000

F.6 Experimental results with FACT Frontiers

MAFACT and MS FACT yield conservative Pareto Frontiers. In Figure 27 we include the results for the Adult dataset along with the model-specific (MS) and model-agnostic (MA) FACT Pareto frontiers, obtained using the methodology in [23]. It can be seen that both Pareto frontiers, MS FACT and MA FACT are overly conservative, with the MA FACT being completely non-informative on the Adult dataset. The model-agnostic FACT trade-off considers the best-case trade-off for any given dataset and does not depend on a specific model class. As a result, there is no guarantee that this Pareto frontier is achievable. This is evident from the fact that the MA FACT yields a non-informative Pareto frontier on the Adult dataset in Figure 27, which is non-achievable in practice.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

0.50

0.55

0.60

0.65

0.70

0.00

0.05

0.10

0.15

Demographic Parity

(a)

|D

cal

|

= 500

0.50

0.55

0.60

0.65

0.70

0.00

0.05

0.10

0.15

Demographic Parity

(b)

|D

cal

|

= 1000

0.50

0.55

0.60

0.65

0.70

0.00

0.05

0.10

0.15

Demographic Parity

(c)

|D

cal

|

= 5000

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Opportunity

(a)

|D

cal

|

= 500

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Opportunity

(b)

|D

cal

|

= 1000

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Opportunity

(c)

|D

cal

|

= 5000

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(a)

|D

cal

|

= 500

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(b)

|D

cal

|

= 1000

0.50

0.55

0.60

0.65

0.70

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Equalized Odds

(c)

|D

cal

|

= 5000

The model-specific FACT trade-off curve is comparatively more realistic as it is derived using a pre-trained model. However, it is important to note that the trade-off for this pre-trained model may not lie on the obtained MS FACT frontier. In fact, MS FACT may still not be achievable by any model in the given model class, as it does not provide any guarantee of being achievable either. This can be seen from Figure 27 which shows that the MS-FACT frontier is not achieved by any of the SOTA baselines under consideration. In these experiments, we used a logistic regression classifier as a pre-trained model for obtaining the MS FACT frontier.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

0.5

0.6

0.7

0.8

0.9

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(a)

|D

cal

|

= 1000

0.5

0.6

0.7

0.8

0.9

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(b)

|D

cal

|

= 5000

0.5

0.6

0.7

0.8

0.9

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Demographic Parity

(c)

|D

cal

|

= 10

,

000

0.88

0.89

0.90

0.91

0.92

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Equalized Opportunity

(a)

|D

cal

|

= 1000

0.88

0.89

0.90

0.91

0.92

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Equalized Opportunity

(b)

|D

cal

|

= 5000

0.88

0.89

0.90

0.91

0.92

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Equalized Opportunity

(c)

|D

cal

|

= 10

,

000

0.5

0.6

0.7

0.8

0.9

0.00

0.02

0.04

0.06

0.08

0.10

Equalized Odds

(a)

|D

cal

|

= 1000

0.5

0.6

0.7

0.8

0.9

0.00

0.02

0.04

0.06

0.08

0.10

Equalized Odds

(b)

|D

cal

|

= 5000

0.5

0.6

0.7

0.8

0.9

0.00

0.02

0.04

0.06

0.08

0.10

Equalized Odds

(c)

|D

cal

|

= 10

,

000

F.7 Synthetic data experiments

In real-world settings, the ground truth trade-off curve τ ∗ fair remains intractable because we only have access to a finite dataset. In this section, we consider a synthetic data setting, where the ground truth trade-off curve can be obtained, to verify that the YOTO trade-off curves are consistent with the ground truth and that the confidence intervals obtained using our methodology contain τ ∗ fair .

Dataset Here, we consider a setup with X = R , A = { 0 , 1 } and Y = { 0 , 1 } . Specifically, A ∼ Bern (0 . 5) and we define the conditional distributions X | A = a as:

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

KDE-fair

logsig

linear

RTO

reductions

adversary

0.55

0.60

0.65

0.70

0.75

0.00

0.01

0.02

0.03

0.04

0.05

Demographic Parity

(a)

|D

cal

|

= 1000

0.55

0.60

0.65

0.70

0.75

0.00

0.01

0.02

0.03

0.04

0.05

Demographic Parity

(b)

|D

cal

|

= 5000

0.55

0.60

0.65

0.70

0.75

0.00

0.01

0.02

0.03

0.04

0.05

Demographic Parity

(c)

|D

cal

|

= 10

,

000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Opportunity

(a)

|D

cal

|

= 1000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Opportunity

(b)

|D

cal

|

= 5000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Opportunity

(c)

|D

cal

|

= 10

,

000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Odds

(a)

|D

cal

|

= 1000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Odds

(b)

|D

cal

|

= 5000

0.50

0.55

0.60

0.65

0.70

0.75

0.00

0.02

0.04

0.06

Equalized Odds

(c)

|D

cal

|

= 10

,

000

Moreover, we define the labels Y as follows:

Y = Z ✶ ( X > 0 . 5) + (1 -Z ) ✶ ( X ≤ 0 . 5) , where Z ∼ Bern (0 . 9) and Z ⊥ ⊥ X . Here, Z introduces some 'noise' to the labels Y and means that perfect accuracy is not achievable by linear classifiers. If perfect accuracy was achievable, the optimal values for Equalized Odds and Equalized Opportunity would be 0 (and would be achieved by the perfect classifier), therefore our use of 'noisy' labels Y ensures that the ground truth trade-off curves will be non-trivial.

YOTO model training Using the data generating mechanism, we generate 5000 training datapoints, which we use to train the YOTO model. The YOTO model for this dataset comprises of a simple

UNLIKELY

PLAUSIBLE

SUB-OPTIMAL

KDE-FAIR

0.0

1.0

0.0

RTO

0.0

0.4

0.6

ADVERSARY

0.0

0.36

0.64

LINEAR

0.0

0.4

0.6

LOGSIG

0.0

0.65

0.35

REDUCTIONS

0.0

0.1

0.9

SEPARATE

0.0

1.0

0.0

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

separate

logsig

linear

reductions

adversary

MA FACT

RTO

MS FACT

0.70

0.75

0.80

0.85

0.90

0.95

1.00

0.00

0.05

0.10

0.15

0.20

Demographic Parity

0.70

0.75

0.80

0.85

0.90

0.95

1.00

0.00

0.05

0.10

0.15

0.20

Equalized Opportunity

0.70

0.75

0.80

0.85

0.90

0.95

1.00

0.00

0.05

0.10

0.15

0.20

Equalized Odds

logistic regression as the main model, with only the scalar logit outputs of the logistic regression being conditioned using FiLM. The MLPs M µ , M σ both have two hidden layers, each of size 4, and ReLU activations. We train the model for a maximum of 1000 epochs, with early stopping based on validation losses. Training these simple model only requires one CPU and takes roughly 2 minutes.

Ground truth trade-off curve To obtain the ground truth trade-off curve τ ∗ fair , we consider the family of classifiers h c ( X ) = ✶ ( X > c ) for c ∈ R . Next, we calculate the trade-offs achieved by this model family for a fine grid of c values between -3 and 3, using a dataset of size 500,000 obtained using the data-generating mechanism described above. The large dataset size ensures that the finite sample errors in accuracy and fairness violation values are negligible. This allows us to reliably plot the trade-off curve τ ∗ fair .

F.7.1 Results

Figure 28 shows the results for the synthetic data setup for three different fairness violations, obtained using a calibration dataset D cal of size 2000. It can be seen that for each fairness violation considered, the YOTO trade-off curve aligns very well with the ground-truth trade-off curve τ ∗ fair . Additionally, we also consider four different confidence intervals obtained using our methodology, and Figure 28

shows that each of the four confidence intervals considered contain the ground-truth trade-off curve. This empirically verifies the validity of our confidence intervals in this synthetic setting.

Additionally, in Figure 29 we plot how the worst-case values for ∆( h ) change (relative to the optimal trade-off τ ∗ fair ) as the number of training data D tr increases. Here, we use h λ as a short-hand notation for the YOTO model h θ ( · , λ ) and the quantity on the y -axis, max λ ∈ Λ ∆( h λ ) τ ∗ fair ( acc ( h λ ) , can be intuitively thought of as the worst-case error percentage between YOTO trade-off and the optimal trade-off. The figure empirically verifies our result in Theorem 3.4 by showing that as the number of training data increases, the error term declines across all fairness metrics.

hoeffdings CIs

asymptotic CIs

bootstrap CIs

YOTO (Ours)

GT tradeoff

0.5

0.6

0.7

0.8

0.9

0.00

0.20

0.40

0.60

0.80

1.00

Demographic Parity

0.5

0.6

0.7

0.8

0.9

0.00

0.20

0.40

0.60

0.80

1.00

Equalized Opportunity

0.5

0.6

0.7

0.8

0.9

0.00

0.20

0.40

0.60

0.80

1.00

Equalized Odds

0

2500

5000

7500

10000

12500

15000

17500

20000

|

tr

|

0.00

0.02

0.04

0.06

0.08

0.10

(

h

)

*

(

acc

(

h

))

DP

EOP

EO

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Section 3 includes the theoretical results and Section 5 includes the experimental results as claimed in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Section 6 includes a discussion of the limitations of our methodology.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Appendix A contains all proofs and assumptions.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Appendix F contains comprehensive practical details needed to reproduce the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Our code has been made open source, and a link to our code has been included in Section 5.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Section 5 and Appendix F include the training and test details.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Table 1 in Section 5 report the standard errors in our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Appendix F includes information on the compute resources used.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: We have tried to be transparent regarding our methodology, and theoretical and practical details to reproduce and use our method in practice.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The Introduction (Section 1) and Preliminaries (Section 2) discuss the implications of our method for fairness decisions. We contribute to better auditing of model fairness standards and are unaware of the negative social impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We use open-source data to train models. We do not release trained models (only the code used to train the models).

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We have included details of our code and models in our submission.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses a significant limitation in the fairness literature, which is the use of uniform fairness requirements across diverse datasets. It proposes the YOTO framework to approximate the fairness-accuracy trade-off and reduce computational costs while existing methods typically require multiple models for training.\n\nSTRENGTHS:\n- Originality: This paper discusses the problems associated with using uniform fairness metrics across diverse datasets and provides a framework for choosing fairness guidelines based on the characteristics of individual datasets. While this issue has been addressed in the fairness literature, this paper proposes a new framework to tackle it. Unlike previous work that recovers a single point on the trade-off curve corresponding to pre-specified fairness constraints during the training procedure, this method estimates the entire trade-off curve.\n\n- Quality and Clarity: The paper is well-constructed and is easy-to-follow. The motivation and research question are clearly stated. The theoretical analysis is solid. The experiment is conducted across three different practical applications (tabular data, image, and text), which is rigorous.\n\n- Significance: This paper proposes a novel framework to approximate the accuracy-fairness trade-off curve, which is an important and efficient evaluation paradigm for providing fairness guidelines in practice. The method has a practical advantage, allowing decision-makers to observe the minimum fairness violation that a particular dataset could have as the model accuracy increases.\n\nWEAKNESSES:\n- The content in lines 121-126 that illustrates the suboptimal problem, as shown in Figure 1, is highly similar to the content in lines 61-66. This content should be summarized to make it more concise.\n- Since this work is based on YOTO, YOTO should be introduced in the Preliminary section for better illustration.\n- Section 3.1 should also be introduced in the Preliminary section, as it is the basic in-processing framework for fairness.\n- There is a lack of dataset and experiment setup descriptions, including data size, sensitive attributes chosen, held-out set size, and train-test split ratio.\n\nQUESTIONS:\nIn Lemma 3.1, $\\delta$ specifies the relationship between the confidence interval and the data size of the held-out set. Is there any requirement for the size of the held-out calibration dataset to guarantee an effective measure for the confidence intervals?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations. Since this work is mainly about fairness, I do not see any negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nConsidering inherent accuracy-fairness trade-off in real-world scenarios with data imbalance and bias, imposing strict fairness constraint could be impractical. To address this, the paper introduces an efficient method to contextualize and accurately estimating fairness-accuracy trade-off curve for each dataset. The proposed YOTO ease redundant model training to realize the trade-off curves. Also, the paper presents a method to quantify the confidence of the estimate.\n\nSTRENGTHS:\n- Clarity of the writing: The paper is easy to follow and well-motivated.\n- Novel Approach: The paper introduces a novel method to approximate the fairness-accuracy trade-off curve efficiently, which is a significant improvement over traditional methods requiring multiple model trainings.\n- Computational Efficiency: By leveraging the YOTO framework, the proposed method reduces the computational burden, making it feasible for large datasets and complex models.\n- Practical Utility: The framework provides a practical tool for auditing model fairness, offering a principled way to select data-specific fairness requirements and assess compliance.\n\nWEAKNESSES:\n- Data Dependency: The methodology requires separate datasets for training and calibration, which may not be feasible in situations with limited data.\n- Assumptions for Statistical Guarantees: The statistical guarantees rely on assumptions that may not hold in all practical scenarios, potentially limiting the generalizability of the results.\n- Objective Formulation Dependency: The trend (scattered points) of YOTO and corresponding CIs are specifically designed for relaxed fairness metric rather than explicit fairness notion, which may make the boundary inaccurate. In other words, trade-off curve and CIs can be sensitive to fairness loss formulation.\n\nQUESTIONS:\n- Can I understand the trade-off curve estimated by proposed method similar to Pareto Frontier with CIs inherent in dataset when model design is specified? If so, I think it would be worth mentioning related thresholding post-processing methods [1,2].\n- I think we can easily interpret points reside in upper-left are suboptimal in Figure 3. However, how can we interpret the ones reside in lower-right region? Does this mean the proposed trade-off estimate and corresponding CI is not accurate?\n\n\n\n[1] Kim, Joon Sik, Jiahao Chen, and Ameet Talwalkar. \"FACT: A diagnostic for group fairness trade-offs.\" International Conference on Machine Learning. PMLR, 2020.\n\n[2] Jang, Taeuk, Pengyi Shi, and Xiaoqian Wang. \"Group-aware threshold adaptation for fair classification.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 6. 2022.\n\nLIMITATIONS:\nPlease see the questions and weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a computationally efficient method to estimate the accuracy-fairness trade-off curve with the statistical guarantee given the dataset and model class. Specifically, it first adopts an existing method, You-Only-Train-Once (YOTO), to get the trade-off curve and then proposes a systematic way to obtain the confidence intervals for fairness and accuracy.\n\nSTRENGTHS:\n1. The research problem on the fairness-accuracy tradeoff is fundamental and practical. There is rarely literature that touches on this problem.\n2. Considering the finite sample error, the confidence interval is essential for tradeoff investigation. This paper proposes a systematic way to investigate this problem.\n3. The paper is well-written and easy to follow.\n\nWEAKNESSES:\n1. The research focus, the fairness-accuracy tradeoff, is computationally expensive since the auditing process usually involves multiple model training. Even though the author adopts a computation-efficient YOTO framework to tackle this issue. However, the main contribution on the efficiency side is mainly from existing work. I suggest the authors highlight the values of statistical guarantee. For example, given the statistical guarantee, can the authors provide a more comprehensive evaluation or any insights for model training? Otherwise, the audience may have a \"so what\" question in mind.\n2. I understand the general clue for deriving confidence intervals for the fairness and accuracy side, but I am still confused about how the authors estimate confidence intervals in practice, especially for sensitivity $\\Delta(h_{\\lambda})$. (1) How do you estimate or justify $\\Delta(h_{\\lambda})$ without know Pareto optimality? It seems to be counter-intuitive. (2) Do you consider Pareto optimality only for specific model classes? I am curious about \"When YOTO satisfies Pareto optimality\" (Line 693). If the main conclusion only holds under specific model classes, it would be better to highlight such condition. (3) I suspect the analysis is rigorous. For example, for Hoeffding’s inequality in Line 215, $\\tilde{acc}$ should be the expectation of $acc$. However, $\\tilde{acc}$ is the accuracy of $h_{\\lambda}$ on $D_{cal}$, which is a realization (not mean) of ACC variable.\n3. I notice that there is a similar literature at https://openreview.net/forum?id=dSbbZwCTQI, which also trains once with flexible tradeoffs. The main idea of YODO is to ``line'' in the weight space that connects the accuracy-optimum and fairness-optimum points using a single model, which is different from YOTO manner. It would be better to highlight the difference and why the authors chose YOTO instead of YODO.\n\nQUESTIONS:\nPlease see Weakness.\n\nLIMITATIONS:\nThe method requires an in-distribution calibration dataset and only applies to in-distribution tests.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}