FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness

## Abstract

Abstract Fair predictive algorithms hinge on both equality and trust, yet inherent uncertainty in real-world data challenges our ability to make consistent, fair, and calibrated decisions. While fairly managing predictive error has been extensively explored, some recent work has begun to address the challenge of fairly accounting for irreducible prediction uncertainty . However, a clear taxonomy and well-specified objectives for integrating uncertainty into fairness remains undefined. We address this gap by introducing FairlyUncertain , an axiomatic benchmark for evaluating uncertainty estimates in fairness. Our benchmark posits that fair predictive uncertainty estimates should be consistent across learning pipelines and calibrated to observed randomness. Through extensive experiments on ten popular fairness datasets, our evaluation reveals: (1) A theoretically justified and simple method for estimating uncertainty in binary settings is more consistent and calibrated than prior work; (2) Abstaining from binary predictions, even with improved uncertainty estimates, reduces error but does not alleviate outcome imbalances between demographic groups; (3) Incorporating consistent and calibrated uncertainty estimates in regression tasks improves fairness without any explicit fairness interventions. Additionally, our benchmark package is designed to be extensible and open-source, to grow with the field. By providing a standardized framework for assessing the interplay between uncertainty and fairness, FairlyUncertain paves the way for more equitable and trustworthy machine learning practices.

## 1 Introduction

Fairness in machine learning enhances transparency and trust in algorithmic predictions, and is both a legal and moral imperative given the direct impact predictive models can have on peoples’ lives. Although extensive research has addressed reducing predictive error disparities among demographic groups – by tackling limited data, model biases, or structural inequities – achieving fairness also necessitates accurately assessing the uncertainty associated with each prediction; however, the precise interplay between algorithmic fairness and uncertainty estimates remains an open question (Bhatt et al., ; Hendrickx et al., ).

Assessing the interplay between fairness and uncertainty is challenging due to a lack of principled objectives for integrating the two notions. Much work has contended with how to define and measure fairness in predictive models (Lee, ; Caton & Haas, ; Mitchell et al., ; Barocas et al., ; Corbett-Davies et al., ).
Additionally, predictions made on real-world data have inherent uncertainty, given factors like noisy individual behavior, measurement errors, and environmental influences. In statistics and machine learning, a careful typology helps organize these sources of randomness. Uncertainty or variance that is specific to each observation is said to display heteroscedasticity (as opposed to homoscedasticity, which is not a function of the observation itself). Heteroscedastic uncertainty can either be epistemic or aleatoric in nature (Hüllermeier & Waegeman, ). Epistemic uncertainty is uncertainty that can be reduced with more data (Chen et al., ); while aleatoric uncertainty is uncertainty that arises from the inherent data distribution, and cannot be reduced with larger samples. We will focus on models that provide a single uncertainty estimate for each observation-specific prediction, but will carefully reason about the aleatoric component of that uncertainty estimate, with the goal that this quantity is consistent with and calibrated to the data.

As a concrete example, consider the task of predicting student performance on a standardized exam: one student reliably earns the same score on the test while another student, perhaps because of their home situation, earns a significantly different score depending on unobservable factors in their daily life. In Figure 1, we hypothesize about how differing unseen environmental factors might affect outcome variance. We can produce more accurate epistemic uncertainty estimates by incorporating more data and improving our predictive model. However, if we cannot have the same student take the test multiple times, we cannot precisely estimate aleatoric uncertainty because we lack repeated observations of a student’s performance measuring individual fluctuations under identical conditions.

**Table 1: The following uncertainty typology allows us to speak precisely about sources of uncertainty we can estimate directly from data, and sources of uncertainty we can only hope to approximate.**
| Uncertainty Type | Description | Example |
| --- | --- | --- |
| (A) Unmeasurable individual-level uncertainty | Uncertainty inherent to individual outcomes that cannot be measured. | A student may experience a unique, random moment of distraction during a test. |
| (B) Within-individual variability | Uncertainty captured through repeated measurements of an individual. | Scores for one student on tests throughout the year across different testing conditions. |
| (C) Across-individual variability | Uncertainty arising from differences between individuals (covariate attributable). | Two students perform differently due to factors such as access to study resources, etc. |
| (D) Sampling uncertainty | Uncertainty stemming from repeated sampling i.e. process of data collection. | Differing test scores between two random samples of students from sample variability. |
| (E) Modeling uncertainty | Uncertainty introduced by the modeling process itself (hyperparameters, etc.). | Different models give slightly different predictions for the same set of students. |

Its worth noting that estimating forms of heteroscedastic uncertainty – uncertainty at the individual level – in the context of algorithmic fairness has received substantial recent interest (Liu et al., ; Ali et al., ; Han et al., ; Tahir et al., ; Wang et al., ).
Some approaches use ensembles of models to estimate uncertainty while others train models to learn the uncertainty directly.
In both cases, the models are trained through a machine learning pipeline on a particular architecture with specific hyperparameters.
Unfortunately, the estimates can vary substantially with respect to a model’s depth, activation function, and other settings (Lakshminarayanan et al., ; Guo et al., ; Malinin & Gales, ). As methods for uncertainty estimation in prior work differ on the actual estimation objective (to varying degrees), we find it useful to provide a precise typology over sources of uncertainty (Hüllermeier & Waegeman, ) in Table 1. This typology helps us to define precise targets for each uncertainty estimation evaluation strategy in our benchmark.

As an example of how this typology helps describe reasonable uncertainty estimate objectives, consider the scenario given in Figure 1, which exemplifies both (A) and (B) uncertainty in Table 1. Producing a fair uncertainty estimate ultimately lies in understanding (A) and (B) uncertainties (Black et al., ; Cooper et al., ), but empirical methods over a fixed data sample can only directly estimate (C), (D), and (E). A goal of our work is to use estimates of (C)-(E) to assess the consistency and calibration of approximations for the unmeasurable uncertainties (A) and (B). In the next section, we specify consistency and calibration as axiomatic principles for successful and fair uncertainty estimates; these principles guide our construction of the FairlyUncertain benchmark.

Figure: Figure 1: Two distributions over observable outcomes. For example, Distribution A can represent the test scores of a student in a stable home whereas Distribution B can represent the test scores of a student in an unstable home. While both have the same mean and $80\%$ confidence interval, the distributions are substantially different as captured by the standard deviation.
Refer to caption: x1.png

## 2 Preliminaries

In Section 2.1, we state guiding axioms for consistent and calibrated uncertainty estimates, as evaluated by the FairlyUncertain benchmark. First, however, we must define a learning pipeline (given in Definition 2.1, related to definitions from (Black et al., ; Long et al., )), and what makes two learning pipelines similar (given in Definition 2.2).

###### Definition 2.1 (Learning Pipeline) .

Let $\mathcal{T}$ be a training set of $n$ observations with covariates $\mathbf{x}_{1},\ldots,\mathbf{x}_{n}\in\mathcal{X}$, protected attributes $a_{1},\ldots,a_{n}\in\mathcal{A}$, and outcomes $y_{1},\ldots,y_{n}\in\mathcal{Y}$.
Consider a set of $m$ hyperparameters $\lambda_{1},\ldots,\lambda_{m}$.
A learning pipeline $\mathcal{P}$ is a randomized training procedure that takes the training set and hyperparameters to learn a predictive function $f:\mathcal{X}\times\mathcal{A}\to\mathcal{Y}\times\mathbb{R}$.
The predictive function produces an estimate of the outcome $\hat{y}$ and uncertainty $\sigma\geq 0$ for given covariates $\mathbf{x}$ and protected attributes $a$.

###### Definition 2.2 (Similar Learning Pipelines) .

For each hyperparameter $\lambda_{j}$, let $\tau_{k}$ represent a threshold that takes on a reasonable value with respect to hyperparameter $j$.
Two learning pipelines $\mathcal{P}$ and $\mathcal{P}^{\prime}$ (Definition 2.1) are considered similar if they differ only in hyperparameter settings i.e., there is some $j\in[m]$ for which $\lambda_{k}=\lambda_{k}\textquoteright$ except when $j=k$ and $|\lambda_{j}-\lambda_{j}\textquoteright|\leq\tau_{j}$.

###### Definition 2.1 (Learning Pipeline) .

###### Definition 2.2 (Similar Learning Pipelines) .

### 2.1 Axioms

The FairlyUncertain benchmark focuses on two properties that uncertainty estimates produced by similar learning pipelines (Definitions 2.1 and 2.2) should satisfy. Let $f(P)$ be a predictive function from a learning pipeline $P$ such that $f(P):(\mathbf{x},a)\mapsto(\mu,\sigma)$ i.e. $f(P)$ maps inputs $\mathbf{x}$ and protected attributes $a$ to a point prediction $\mu$ and an estimated standard deviation of the inherent (A) and (B) types of uncertainty $\sigma$ for the predicted value of $\mu$. Note if $f(P)$ is a classification model, $\mu$ is a probability, and if $f(P)$ is a regression model, $\mu$ is a real value.

Now, we are ready to state Axiom 2.3, which formalizes the idea that uncertainty estimates should be a function of the data rather than of an arbitrary learning pipeline. In other words, similar learning pipelines applied to the same dataset should produce similar uncertainty estimates.

###### Axiom 2.3 (Consistency) .

For $f$ and $f^{\prime}$, which are produced by similar learning pipelines, $\sigma$ and $\sigma^{\prime}$ should be close. This means that if two learning pipelines are similar, the uncertainty estimates they produce should not vary much.

This idea is related to the motivation behind selective ensemble and self-consistency ensemble methods (Black et al., ; Cooper et al., ). However, Axiom 2.3 is a property of uncertainty estimates rather than outcome predictions.

Consistency by itself is not sufficient; a model that always returns 0 is consistent but not meaningful. This leads to our second property, which posits that uncertainty estimates should also be calibrated. In other words, uncertainty estimates should always explain observed heteroscedastic variance.

###### Axiom 2.4 (Calibration) .

The predicted uncertainty estimates should satisfy $\textrm{Var}(y~{}|~{}\mathbf{x},a)=\sigma^{2}$, where $\sigma$ is the estimated standard deviation of the outcome $y$ given covariates $\mathbf{x}$ and protected attributes $a$. This means the predicted uncertainty $\sigma$ should match the actual variability observed in the data, capturing the heteroscedastic nature of the uncertainty. When uncertainty estimates are biased or incorrect on average, they fail to be calibrated.

Once uncertainty estimates are *consistent* and *calibrated*(^1^11We use *consistent* and *calibrated* to denote the formal/axiomatic definitions throughout, as opposed to common usage. Additionally, we note that Axioms 2.3 and 2.4 contain specific assumptions that dictate strategies for evaluation; different formalizations would lead to different strategies. We acknowledge this limitation, but argue that any alternative strategies for evaluating uncertainty should intuitively align with each desiderata.), the next challenge is figuring out how to integrate them into fair algorithms effectively.
Integrating uncertainty in social settings is commonly done through abstention; for a sample where the level of uncertainty is too high, defer making a prediction, thereby avoiding unreliable results (Black et al., ; Cooper et al., ).
Another method (commonly adopted in regression settings) is to return the uncertainty estimates alongside the predictions, providing a clearer picture of the confidence in the results and allowing practitioners to make more informed decisions (Liu et al., ; Wang et al., ). In either setting, no consensus best approach for incorporating uncertainty estimates into fairness interventions has emerged; this motivates our benchmark, FairlyUncertain.

###### Axiom 2.3 (Consistency) .

###### Axiom 2.4 (Calibration) .

## 3 FairlyUncertain : A Benchmark for Uncertainty in Fairness

Contributions   To the best of our knowledge, FairlyUncertain is a first of its kind fairness and uncertainty benchmark. Prior work has either provided singular uncertainty evaluations (Cooper et al., ) or extensive fairness benchmarking without methods to incorporate uncertainty (Bird et al., ; Bellamy et al., ). FairlyUncertain fills the gap, introducing a variety of tests and experiments, and is designed to be extensible and grow with the literature.
FairlyUncertain supports five binary datasets and five regression datasets off-the-shelf that are standard to the fair machine learning context; Table 6 (deferred to the Appendix) provides prediction task summaries for each. All of our code for FairlyUncertain is ready for release as a Python package; the benchmark is modular, so that uncertainty methods and datasets can be easily added.
The package includes functions to generate all of our experiments with just a few lines of code.

Road Map   In Section 4, we begin by evaluating consistency and calibration on both classification and regression tasks. We find that baseline ensemble methods are the most consistent. Additionally, we find that in classification tasks, complex ensemble-based methods from prior work on uncertainty are neither consistent nor calibrated. In contrast, methods that directly learn uncertainty parameters via negative log-likelihood are both consistent and calibrated. Next, we incorporate uncertainty estimates into making fair decisions. In Section 5, we show that while abstaining from predictions can improve accuracy, this approach does not reduce the imbalance in outcomes between demographic groups. In Section 6, our results lead us to advocate for outputting both predictions and uncertainty estimates in regression tasks. To facilitate this, we introduce Uncertainty-Aware Statistical Parity (Definition 6.2), a natural generalization of statistical parity in the regression setting that includes uncertainty estimates. Remarkably, we find that consistent and calibrated uncertainty methods can reduce Uncertainty-Aware Statistical Parity without any explicit fairness interventions.

Experimental Details   We use a cluster of 24-core Intel Cascade Lake Platinum 8268 chips to run the experiments.
We use default model hyperparameters except when the experiment explicitly varies them i.e., for assessing consistency.
Our benchmark considers a privileged class and an unprivileged class for each dataset. The privileged class can be at the intersection of multiple protected features; for example, “white” (race) and “male” (gender) could become a privileged class subset of “white males” for evaluation. All fairness metrics in the submitted paper are then binary and are the absolute difference between the disadvantaged and advantaged group. The benchmark defaults to using the XGBoost model for all predictive tasks; XGBoost is a fast, state-of-the-art model that generally outperforms neural models in relatively low data, low dimensional tabular regimes, like most datasets in our benchmark (Chen & Guestrin, ; Grinsztajn et al., ; McElfresh et al., ).

## 4 Consistency and Calibration

In this section, we evaluate methods for capturing heteroscedastic uncertainty according to Axioms 2.3 and 2.4. FairlyUncertain provides an evaluation strategy for *consistency* that investigates the sensitivity of heteroscedastic uncertainty estimates to hyperparameters in the learning pipeline (Figure 2). For learning pipelines that vary only in one hyperparameter, we compute uncertainty estimates and then the standard deviation of these estimates (Table 2).

Measuring calibration is much harder since we cannot observe more samples from the same distribution (type (A) uncertainty) and would need generated data for type (B). Thus, we devise strategies which measure uncertainty types (C), (D) and (E) to approximately test calibration in a qualitative and quantitative manner. Qualitatively, FairlyUncertain measures calibration over similar groups, based on the following intuition: if estimated uncertainties are *calibrated*, observations with higher estimated uncertainties should have higher variation in the test data.
Concretely, we create groups with similar uncertainty estimates; then, within each group, we compute the empirical standard deviation of the residual difference between true and predicted outcomes (Figure 3).

Now, imagine we have access to the true probability $p_{i}$ that individual $i$ will belong to the positive class in the binary setting. Then, following the binomial distribution, the true uncertainty as measured by variance would be $\sigma^{2}=p_{i}(1-p_{i})$. With our quantitative strategy for measuring calibration, we contend that if we have a prediction for the true probability $\tilde{p}_{i}\approx p_{i}$ then a natural prediction for the uncertainty as measured by variance is $\tilde{\sigma}^{2}=\tilde{p}_{i}(1-\tilde{p}_{i})$. A canonical method to evaluate the goodness of fit given probabilistic assumptions is Negative Log Likelihood (NLL) Fisher ().

This leads to the quantitative evaluation given in Table 2, where we make this assumption on the meaning of uncertainty estimates in binary predictions to offer a clear and simple measure of *calibration* based on the NLL of observing the data according to the estimated parameters of the Binomial distribution, interpreting estimate $\mu$ as Binomial probability $p$ (with uncertainty estimate $\sigma$ being the standard variance $p(1-p)$). Because there are existing methods which produce uncertainty estimates that cannot be interpreted as standard deviations (namely Black et al. (); Cooper et al. ()), when assessing these methods, one should focus on the output of the qualitative assessment (Figure 3), which provides a more general purpose approach for approximately testing calibration.

### 4.1 Models and Classification Tasks

We subject several methods for heteroscedastic uncertainty estimation on binary classification tasks to our evaluation.
Many of the methods we consider rely on an ensemble of $k$ models, where each model in the ensemble is trained on samples taken randomly with replacement from the training set.
For predictions, these methods simply output the mode prediction of the $k$ models, while for uncertainty estimates, they differ in their calculations.
We’ll let $k^{(0)}$ be the number of models predicting 0 and $k^{(1)}$ be the number of models predicting 1.
Our experiments are run with the following methods:

- •
The Ensemble computes the standard deviation of the $k$ predictions to estimate uncertainty.
- •
The Selective Ensemble method of Black et al. () estimates uncertainty by computing the $p$-value of observing the number of negative predictions $k^{(0)}$ and the number of positive predictions $k^{(1)}$ were they sampled from a binomial distribution with probability $\frac{1}{2}$.
- •
The Self-consistency Ensemble method of Cooper et al. () estimates uncertainty by computing a so-called self-consistency metric $1-2k^{(0)}k^{(1)}/(k(k-1))$. Since the self-consistency metric increases with certainty, we will report self-(in)consistency, i.e., $2k^{(0)}k^{(1)}/(k(k-1))$ for parity with the other approaches.
- •
The Binomial NLL method contrasts with the ensemble methods by directly estimating uncertainties under the Binomial assumption e.g. by producing probabilities $p$ that minimize negative log likelihood on the training set, yielding an uncertainty estimate that is the Binomial standard deviation $\sigma=\sqrt{p(1-p)}$. We offer a brief formal justification as to why we expect Binomial NLL to produce good uncertainty estimates in the binary classification setting in Appendix Section B.

Figure: Figure 2: This boxplot shows the standard deviation of each individual’s uncertainty estimates across different max_depth hyperparameter settings. For example, if an individual has the same uncertainty estimate for each hyper-parameter setting, then their standard deviation is 0 (perfect consistency) whereas if they vary wildly, the standard deviation is high (not consistent). The Binomial NLL and Ensemble methods exhibits are the most consistent.
Refer to caption: x2.png

Consistency   Our first experiment evaluates consistency between similar learning pipelines.
We do this by varying a hyperparameter setting for our model class; ideally, we would select parameters that affect how the model makes predictions, but shouldn’t substantially affect the error rate of those predictions. We note that FairlyUncertain makes it easy to experiment with many hyperparameter settings.
In this paper, we first experiment with the XGBoost hyperparameter max_depth; depth is widely applicable to many kinds of models and there are often many equally-valid settings (FairlyUncertain ensures that all depths produce models with similar accuracy).
The second is reduction_threshold $\gamma$, which is specific to the XGBoost model, and smoothly interpolates between encouraging more or less complex models through tree splits.
Figure 2 plots heteroscedastic uncertainty estimates for all individuals in each dataset for max_depth (see Figure 7 for the reduction_threshold plot, and see Figure 10 for a more granular view where we plot the consistency ranges for each individual in the data, both in the Appendix).
For some individuals, all the methods consistently output the same uncertainty estimates but, for others, estimates produced by the Selective Ensemble and Self-(in)consistency Ensemble vary drastically.
We measure the overall *consistency* of each method by computing the empirical standard deviation of the estimates at each depth.
Table 2 reports the maximum individual empirical standard deviation for each algorithm and dataset.
The Ensemble algorithm is the most consistent, followed closely by Binomial NLL.

**Table 2: Comparison of *calibration* and *consistency* for each algorithm on each binary dataset. Here, calibration is measured by negative log-likelihood while the consistency is measured by the maximum individual empirical standard deviation. The $\pm$ indicates the standard deviation of these values over 10 iterations. Note that we adopt the Olympic medal convention in all Tables throughout our paper: gold, silver and bronze cells signify first, second and third best performance, respectively.**
|  | Calibration (Negative Log-Likelihood) | Consistency |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Approach | ACS | Adult | Bank | COMPAS | German | ACS | Adult | Bank | COMPAS | German |
| Ensemble | 1.1 $\pm$ 0.04 | 0.9 $\pm$ 0.02 | 0.5 $\pm$ 0.02 | 1.8 $\pm$ 0.07 | 0.97 $\pm$ 0.12 | 0.08 $\pm$ 0.00 | 0.08 $\pm$ 0.01 | 0.09 $\pm$ 0.01 | 0.06 $\pm$ 0.00 | 0.062 $\pm$ 0.00 |
| *Selective Ens.* | 1.1 $\pm$ 0.05 | 0.88 $\pm$ 0.02 | 0.5 $\pm$ 0.02 | 1.8 $\pm$ 0.08 | 1.0 $\pm$ 0.16 | 0.45 $\pm$ 0.01 | 0.45 $\pm$ 0.01 | 0.45 $\pm$ 0.01 | 0.44 $\pm$ 0.01 | 0.40 $\pm$ 0.02 |
| (In)cons. Ens. | 1.0 $\pm$ 0.04 | 0.82 $\pm$ 0.03 | 0.42 $\pm$ 0.02 | 1.5 $\pm$ 0.07 | 0.82 $\pm$ 0.13 | 0.26 $\pm$ 0.01 | 0.26 $\pm$ 0.01 | 0.25 $\pm$ 0.01 | 0.25 $\pm$ 0.01 | 0.21$\pm$ 0.01 |
| Binom. NLL | 0.4 $\pm$ 0.01 | 0.31 $\pm$ 0.0 | 0.2 $\pm$ 0.0 | 0.6 $\pm$ 0.01 | 0.5 $\pm$ 0.04 | 0.10 $\pm$ 0.01 | 0.13 $\pm$ 0.01 | 0.12 $\pm$ 0.01 | 0.07 $\pm$ 0.00 | 0.08 $\pm$ 0.01 |

Calibration (qualitative)   The next experiment qualitatively evaluates calibration.
For each method, we identify groups of individuals with similar uncertainty estimates and empirically evaluate the standard deviation of the residual difference between the observed and predicted outcomes.
Results in Figure 3 demonstrate that predicted uncertainty estimates from the Selective Ensemble and Self-(in)consistency algorithms do not appear related to the empirical standard deviation.
In contrast, the estimates from the Ensemble and Binomial NLL methods are clearly related to the empirical standard deviation; we note that the Binomial NLL method most closely tracks the identity line.

Figure: Figure 3: For five groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes. An algorithm is perfectly calibrated if predicted uncertainty equals the empirical standard deviation i.e., the points lie on the dashed identity line. Note that uncertainty estimates do not always represent variance, so we expect a positive but not necessarily linear correlation. Additionally, note that this calibration graph also reflects consistency; a less consistent method will have a more arbitrary grouping leading to a flatter observed slope.
Refer to caption: x3.png

Calibration (quantitative)   Finally, we quantitatively evaluate calibration by interpreting uncertainty estimates as the parameter of a fixed distribution.
In the binary setting, the distribution is completely described by a single probability and the Binomial distribution.
Table 2 gives the negative log-likelihood for each method on each dataset. To produce Table 2, we interpret estimates from models that produce a different kind of uncertainty as a standard deviation for comparison.
By design, the Binomial NLL method minimizes the negative log-likelihood on the training set, so unsurprisingly we find that the Binomial NLL method also gives the best performance on the test set. Overall, Figures 2 and 3 and Table 2 suggest that the Binomial NLL method produces heteroscedastic uncertainty estimates that are simultaneously the most *consistent* and *calibrated*.

### 4.2 Regression Tasks

In this section, we turn our attention to evaluating methods for estimating heteroscedastic uncertainty in the regression setting over continuous outcomes. FairlyUncertain contains several algorithms drawn from the heteroscedastic uncertainty literature.

- •
The Normal NLL method learns parameters $\mu$ and $\sigma$ that minimize the negative log-likelihood loss of the normal distribution, given by $\ell(y,\mu,\sigma)=-\log\sigma+\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}$.
- •
The $\beta$-NLL method (Seitzer et al., ) learns a version of the NLL loss that is multiplied by the constant $\sigma^{2\beta}$ (in our experiments we set $\beta=\frac{1}{2}$ as suggested in the original paper).
- •
The Faithful NLL method (Stirn et al., ) learns mean predictions $\mu$ with the standard mean squared error loss while the standard deviation predictions $\sigma$ are learned with the NLL loss.
- •
A natural Ensemble method serves as a point of comparison for these heteroscedastic algorithms: Ensemble learns an ensemble of models trained on bagged samples of the training set, and outputs the mean and standard deviation over model predictions.

**Table 3: Comparison of Negative Log-Likelihood (NLL) and Consistency for each algorithm on each dataset. We measure *consistency* as the maximum over individuals of the standard deviation of the predictions they receive with respect to the depth of the model.**
|  | Calibration (Negative Log-Likelihood) | Consistency |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Approach | Comm. | IHDP | Insur. | Law | Twins | Comm. | IHDP | Insur. | Law | Twins |
| Ensemble | 120.0 $\pm$ 21.0 | 3.9 $\pm$ 1.3 | 21.0 $\pm$ 15.0 | 200.0 $\pm$ 21.0 | 160.0 $\pm$ 9.3 | 0.05 $\pm$ 0.01 | 0.02 $\pm$ 0.00 | 0.03 $\pm$ 0.00 | 0.05 $\pm$ 0.01 | 0.09 $\pm$ 0.01 |
| Normal NLL | 0.35 $\pm$ 0.08 | -0.6 $\pm$ 0.07 | -0.67 $\pm$ 0.04 | -0.25 $\pm$ 0.0 | 0.41 $\pm$ 0.0 | 0.12 $\pm$ 0.01 | 0.08 $\pm$ 0.01 | 0.09 $\pm$ 0.01 | 0.11 $\pm$ 0.01 | 0.20 $\pm$ 0.01 |
| $\beta$-NLL | 0.34 $\pm$ 0.08 | -0.6 $\pm$ 0.02 | -0.65 $\pm$ 0.03 | -0.25 $\pm$ 0.0 | 0.41 $\pm$ 0.0 | 0.11 $\pm$ 0.01 | 0.06 $\pm$ 0.01 | 0.06 $\pm$ 0.01 | 0.08 $\pm$ 0.00 | 3.26 $\pm$ 7.66 |
| Faithful NLL | 0.34 $\pm$ 0.11 | -0.58 $\pm$ 0.01 | -0.64 $\pm$ 0.03 | -0.25 $\pm$ 0.0 | 0.41 $\pm$ 0.0 | 0.12 $\pm$ 0.01 | 0.08 $\pm$ 0.01 | 0.07 $\pm$ 0.01 | 0.09 $\pm$ 0.01 | 0.19 $\pm$ 0.01 |

Using FairlyUncertain, we evaluate these methods for *consistency* and *calibration*.
We evaluate *consistency* in the same manner in the regression setting as we did in the binary setting, varying two hyperparameters while checking that the maximum deviation of uncertainty estimates remains stable (see Figure 8 in the Appendix). In Table 3, we see that the Ensemble approach is the most *consistent*, although for four of the five datasets, $\beta$-NLL performs similarly.

*Calibration* is also evaluated in the same manner for regression tasks as for binary. Figure 9 in the Appendix provides the qualitative assessment, while quantitatively Table 3 gives the NLL for each algorithm on each dataset.
Our results demonstrate that though the Ensemble method is *consistent*, its predictions are very poorly *calibrated*. In contrast, all NLL approaches achieve a similar level of *calibration* across all five regression datasets. Overall, the various NLL approaches are all comparably *consistent* to the Ensemble method and are significantly more *calibrated*.

## 5 Abstaining on Classification Tasks

We now turn our attention to incorporating uncertainty estimates into fair algorithms for classification, to assess their impact on downstream fairness metrics, focusing on the abstention framework where models are allowed to abstain from making a prediction if the heteroscedastic uncertainty estimates are too large (Black et al., ; Cooper et al., ). FairlyUncertain instantiates this intervention, allowing models to abstain from predicting under high uncertainty. We find that while the abstention framework allows models to abstain from incorrect predictions, unsurprisingly reducing the overall error rate, it does not make the distribution of predictions more balanced between demographic groups. Moreover, the flexibility afforded by abstention is also its biggest limitation; if allowed to make almost no predictions, a model can easily achieve optimal performance without meaningful outputs for the majority of observations. Thus, FairlyUncertain crucially focuses on the question of abstention rate and how it affects overall error, statistical parity, equalized odds, etc.

In Figure 5, we see that allowing models to make fewer predictions based on uncertainty estimates decreases their error. In contrast, a Random baseline (real binary predictions but random uncertainty estimates) maintains the same error rate. Notice that the Selective Ensemble and Self-(in)consistency Ensemble produce the same uncertainty estimates for many observations, hence the flat lines. While abstaining invariably reduces overall error, it has a more chaotic impact on Statistical Parity. Figure 4 shows that abstaining can improve (decrease) or worsen (increase) statistical parity unpredictably across different data distributions; here, the model behavior resembles the Random uncertainty prediction baseline. We observe a similar trend in Figure 12, which plots the Equalized Odds fairness metric (Hardt et al., ) against the abstention rate (see Appendix C).

**Table 4: Various evaluation and fairness metrics for each algorithm on the ACS Income (see Appendix G for definitions). While the abstention framework allows models to reduce their Error Rate, it does not magically reduce the imbalance in outcomes between demographic groups.**
| Approach | Error Rate | Stat. Parity | Equalized Odds | Equal Opp. | Disp. Impact | Predictive Parity | FPR | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.22 $\pm$ 0.01 | 0.18 $\pm$ 0.02 | 0.17 $\pm$ 0.05 | 0.17 $\pm$ 0.05 | 1.7 $\pm$ 0.14 | 0.057 $\pm$ 0.03 | 0.083 $\pm$ 0.02 | 100 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.23 $\pm$ 0.01 | 0.029 $\pm$ 0.02 | 0.14 $\pm$ 0.05 | 0.063 $\pm$ 0.03 | 0.98 $\pm$ 0.07 | 0.24 $\pm$ 0.04 | 0.13 $\pm$ 0.05 | 100 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.23 $\pm$ 0.01 | 0.10 $\pm$ 0.03 | 0.080 $\pm$ 0.05 | 0.079 $\pm$ 0.05 | 1.3 $\pm$ 0.11 | 0.12 $\pm$ 0.04 | 0.024 $\pm$ 0.02 | 100 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.23 $\pm$ 0.01 | 0.041 $\pm$ 0.02 | 0.12 $\pm$ 0.04 | 0.065 $\pm$ 0.04 | 0.95 $\pm$ 0.08 | 0.20 $\pm$ 0.03 | 0.12 $\pm$ 0.04 | 100 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.22 $\pm$ 0.01 | 0.10 $\pm$ 0.04 | 0.092 $\pm$ 0.05 | 0.090 $\pm$ 0.05 | 1.3 $\pm$ 0.15 | 0.12 $\pm$ 0.05 | 0.031 $\pm$ 0.02 | 100 $\pm$ 0.0 |
| Grid Search SP | 0.23 $\pm$ 0.01 | 0.083 $\pm$ 0.05 | 0.14 $\pm$ 0.06 | 0.11 $\pm$ 0.07 | 1.0 $\pm$ 0.26 | 0.18 $\pm$ 0.05 | 0.11 $\pm$ 0.07 | 100 $\pm$ 0.0 |
| Grid Search EO | 0.23 $\pm$ 0.01 | 0.15 $\pm$ 0.03 | 0.13 $\pm$ 0.07 | 0.13 $\pm$ 0.07 | 1.5 $\pm$ 0.15 | 0.087 $\pm$ 0.05 | 0.056 $\pm$ 0.03 | 100 $\pm$ 0.0 |
| Random | 0.22 $\pm$ 0.01 | 0.17 $\pm$ 0.03 | 0.15 $\pm$ 0.05 | 0.15 $\pm$ 0.05 | 1.6 $\pm$ 0.16 | 0.044 $\pm$ 0.03 | 0.085 $\pm$ 0.02 | 88 $\pm$ 7.0 |
| Ensemble | 0.20 $\pm$ 0.02 | 0.19 $\pm$ 0.03 | 0.16 $\pm$ 0.06 | 0.16 $\pm$ 0.06 | 1.7 $\pm$ 0.18 | 0.032 $\pm$ 0.04 | 0.093 $\pm$ 0.03 | 89 $\pm$ 8.0 |
| Selective Ensemble | 0.20 $\pm$ 0.02 | 0.18 $\pm$ 0.03 | 0.15 $\pm$ 0.05 | 0.15 $\pm$ 0.06 | 1.7 $\pm$ 0.20 | 0.039 $\pm$ 0.04 | 0.082 $\pm$ 0.03 | 94 $\pm$ 7.3 |
| Self-(in)consistency | 0.19 $\pm$ 0.03 | 0.19 $\pm$ 0.03 | 0.16 $\pm$ 0.05 | 0.15 $\pm$ 0.06 | 1.7 $\pm$ 0.20 | 0.035 $\pm$ 0.03 | 0.081 $\pm$ 0.03 | 89 $\pm$ 8.9 |
| Binomial NLL | 0.18 $\pm$ 0.03 | 0.19 $\pm$ 0.02 | 0.15 $\pm$ 0.04 | 0.15 $\pm$ 0.05 | 1.8 $\pm$ 0.14 | 0.034 $\pm$ 0.03 | 0.080 $\pm$ 0.02 | 83 $\pm$ 9.5 |

Figure: Figure 4: Abstaining has no reliable effect on Statistical Parity (comparable to the Random baseline).
Refer to caption: x4.png

Our benchmark additionally evaluates uncertainty methods against standard baseline and state-of-the-art fairness algorithms. To construct this comparison, FairlyUncertain tests the Random, Selective Ensemble, Self-(in)consistency, and Binomial NLL methods at inclusion rates between $75\%$ and $100\%$. It selects an abstention rate for comparison by optimizing a simple objective function that is the normalized sum of the Error Rate, Statistical Parity, and Equalized Odds. Table 4 gives performances on the ACS Income task (Ding et al., ) (results for the other datasets appear in Appendix D).
Once again, methods allowed to abstain had lower error (Self-(in)consistency and Binomial NLL methods had the lowest). However, across 5 of the 6 fairness metrics reported, methods that did not abstain were the highest performing in terms of fairness. This is surprising, as it seems directly opposed to one of the main claims in Cooper et al. ().

Figure: Figure 5: For an abstention rate $r$, FairlyUncertain abstains on the $r$ fraction of observations with the highest uncertainty. For heteroscedastic uncertainty methods, predictions become more accurate as the model abstains more, while the error rate for the random baseline remains steady.
Refer to caption: x5.png

Figure: Figure 6: CDF by group of on each dataset. The solid lines indicate one protected group and the dashed lines indicate the other protected group. Normal NLL effectively smooths the CDF.
Refer to caption: x6.png

A natural question arises given the results in Table 4: for which individuals do the abstaining models choose not to predict? In Figure 14 (deferred to Appendix C), we address this by comparing the empirical distributions of the overall population and the included (non-abstained) population for each feature across methods. We compute the average Wasserstein distance between these distributions (averaged over methods) and plot the feature with the largest distance. The variables with the greatest differences tend to be protected attributes like marriage status and sex. However, except for the Binomial NLL distribution on the Adult dataset, these differences are small between the overall distribution and the one selected through the abstaining process.

## 6 Uncertainty Aware Fair Regression

So far, we have used FairlyUncertain to demonstrate that incorporating heteroscedastic uncertainty estimates into fair classification algorithms requires care; this is no different in the regression setting. By outputting both predictions and uncertainty estimates in either setting, downstream users have the flexibility to apply uncertainty estimates in domain-appropriate ways beyond abstention. For example, practitioners can flag high uncertainty estimates for manual review (Madras et al., ), combine predictions and uncertainty estimates into domain-specific metrics (Ali et al., ), or discover patterns of high uncertainty that indicate a lack of meaningful features, prompting more data collection (Chen et al., ).
Still, it remains an open question how to measure the fairness for models that output both predictions and uncertainty estimates, particularly for regression models.
To address this, we propose a new fairness metric (included in FairlyUncertain), uncertainty-aware statistical parity (UA-SP) (Definition 6.2), which naturally incorporates the available uncertainty estimates into the the standard regression-specific form of Statistical Parity (Definition 6.1).

Note that UA-SP (Definition 6.2) applies to models which produce point predictions $\mu$ and uncertainty estimates $\sigma$ for each observation. These estimates are treated as model-derived inputs to our fairness metric, not as the underlying true parameters. We model predictions as distributions to incorporate the inherent uncertainty of model outputs; this approach is particularly salient in settings with heteroscedastic variance. When the uncertainty estimates are zero, our metric reduces to the standard statistical parity definition, thus generalizing it to account for variability in predictions due to uncertainty. The generalized definition assumes that the uncertainty follows a normal distribution with mean $\mu$ and standard deviation $\sigma$. The validity of this assumption depends on the setting.

###### Definition 6.1 (Statistical Parity) .

Consider a method $f:\mathcal{X}\times\mathcal{A}\to\mathcal{Y}$ that outputs a mean prediction.
Then $f$ satisfies statistical parity if
$\Pr(f(\mathbf{X},A)\geq y|A=a)=\Pr(f(\mathbf{X},A)\geq y)$
for all protected groups $a\in\mathcal{A}$ and outcomes $y\in\mathcal{Y}$.

###### Definition 6.2 (Uncertainty-Aware Statistical Parity ( UA-SP )) .

Consider $f:\mathcal{X}\times\mathcal{A}\to\mathcal{Y}\times\mathbb{R}$ that estimates a mean $\mu$ and a standard deviation $\sigma$.
The predictions induce a randomized function $\tilde{f}:\mathcal{X}\times\mathcal{A}\to\mathcal{Y}$ that samples $y\sim\mathcal{N}(\mu,\sigma^{2})$. Then $f$ satisfies uncertainty-aware statistical parity if $~{}\Pr(\tilde{f}(\mathbf{X},A)\geq y|A=a)=\Pr(\tilde{f}(\mathbf{X},A)\geq y)$
for all protected groups $a\in\mathcal{A}$ and outcomes $y\in\mathcal{Y}$.

Definition 6.2 incorporates uncertainty estimates in a natural way: the predicted distribution is smoothed for observations with high uncertainty.
In contrast, Definition 6.1 holds the algorithms to a stringent standard even when heteroscedastic variance is large.

In regression settings, Definition 6.1 is often interpreted in terms of a Cumulative Density Function (CDF) over predictions (Agarwal et al., ; Liu et al., ); uncertainty-aware statistical parity (Definition 6.2) shares this interpretation. For methods that only produce predictions, the CDF is with respect to the randomness of the observations $\mathbf{x}$.
For methods that produce predictions and uncertainty estimates, the CDF is with respect to the randomness of the observations and the induced randomized function $\tilde{f}$. Figure 6 gives CDFs for True, Baseline and Normall NLL: here, True indicates the correct labels in the test data, Baseline indicates a standard regression model without any fairness intervention, and Normal NLL is the smoothed method as described in Section 4.2 (for visual clarity, we present comparisons only against Normal NLL, which is similar to Faithful NLL and more *consistent* and *calibrated* than Ensemble and $\beta$-NLL).

Figure 6 illustrates how incorporating uncertainty into model predictions effectively smooths the CDF. In the case of the Twins dataset, we observe that this smoothing is both strong and accurate, allowing the model to closely capture the true distribution of the labels. Table 5 presents fairness metrics computed according to either Definition 6.1 or Definition 6.2, depending on whether the model provides uncertainty estimates. The Normal NLL method significantly outperforms others in terms of fairness, even surpassing the Ensemble method. Remarkably, the Normal NLL method, which is both competitively *consistent* and *calibrated*, achieves substantial fairness improvements without any explicit fairness interventions.

**Table 5: Measuring (uncertainy-aware) statistical parity (UA-SP) via Kolmogorov-Smirnov distance: the maximum over outputs $y$ of the distance between per-group CDFs (Agarwal et al., ). Simply using *consistent* and *calibrated* uncertainty methods substantially reduces UA-SP.**
| Approach | Communities | IHDP | Insurance | Law School | Twins |
| --- | --- | --- | --- | --- | --- |
| True | 0.561 $\pm$ 0.036 | 0.153 $\pm$ 0.061 | 0.1 $\pm$ 0.022 | 0.484 $\pm$ 0.021 | 0.161 $\pm$ 0.011 |
| Baseline | 0.644 $\pm$ 0.025 | 0.144 $\pm$ 0.05 | 0.091 $\pm$ 0.019 | 0.965 $\pm$ 0.009 | 0.404 $\pm$ 0.009 |
| Exponentiated Gradient Square | 0.638 $\pm$ 0.024 | 0.16 $\pm$ 0.053 | 0.094 $\pm$ 0.025 | 0.902 $\pm$ 0.015 | 0.357 $\pm$ 0.01 |
| Exponentiated Gradient Absolute | 0.638 $\pm$ 0.024 | 0.157 $\pm$ 0.057 | 0.094 $\pm$ 0.025 | 0.902 $\pm$ 0.015 | 0.359 $\pm$ 0.009 |
| Grid Search Square | 0.649 $\pm$ 0.035 | 0.167 $\pm$ 0.056 | 0.093 $\pm$ 0.025 | 0.888 $\pm$ 0.019 | 0.357 $\pm$ 0.007 |
| Grid Search Absolute | 0.649 $\pm$ 0.035 | 0.167 $\pm$ 0.056 | 0.092 $\pm$ 0.024 | 0.888 $\pm$ 0.019 | 0.358 $\pm$ 0.007 |
| Ensemble | 0.62 $\pm$ 0.021 | 0.101 $\pm$ 0.052 | 0.076 $\pm$ 0.022 | 0.939 $\pm$ 0.008 | 0.389 $\pm$ 0.009 |
| Normal NLL | 0.37 $\pm$ 0.016 | 0.048 $\pm$ 0.028 | 0.052 $\pm$ 0.022 | 0.196 $\pm$ 0.003 | 0.096 $\pm$ 0.002 |
| $\beta$-NLL | 0.375 $\pm$ 0.019 | 0.045 $\pm$ 0.025 | 0.051 $\pm$ 0.022 | 0.195 $\pm$ 0.004 | 0.096 $\pm$ 0.003 |
| Faithful NLL | 0.299 $\pm$ 0.151 | 0.042 $\pm$ 0.021 | 0.054 $\pm$ 0.02 | 0.192 $\pm$ 0.004 | 0.095 $\pm$ 0.003 |

###### Definition 6.1 (Statistical Parity) .

###### Definition 6.2 (Uncertainty-Aware Statistical Parity ( UA-SP )) .

### 6.1 Related Work

Prior work in algorithmic fairness largely focuses on fixed fairness constraints, framing the issue as optimizing along a Pareto frontier between accuracy and fairness metrics that may conflict (Hardt et al., ; Zafar et al., ; Mitchell et al., ; Bell et al., ; Wang et al., ). More recent research has introduced uncertainty estimation as a crucial factor in model selection under fairness. Particularly relevant are Black et al. () and Cooper et al. (), who explore ensemble approaches to estimate prediction uncertainty based on the standard deviation from an ensemble of models. These methods, as we have demonstrated, tend to focus on artifacts of the learning pipeline rather than inherent heteroscedastic uncertainty.

Other studies examine uncertainty estimation through model multiplicity (Black et al., ), analyze the relationship between inadequate sample sizes and disparate epistemic uncertainty among subgroups (Chen et al., ; Zhang & Long, ), and extend abstention frameworks to regression (Shah et al., ). Additionally, tailored inference and prediction models under heteroscedastic assumptions have been extensively studied in economics and statistics, particularly with Bayesian inference (White, ; MacKinnon, ; Rigobon, ; Hayes & Cai, ; Ji et al., ). The impact of Bayesian conditioning via latent variables on fairness has also been considered as an alternative to bagging for uncertainty estimation, though *calibration* of these estimates can be challenging due to the large space of priors (McNair, ; Ji et al., ). More recently, loss functions accommodating heteroscedastic assumptions in machine learning have improved uncertainty estimation and model robustness (Collier et al., ; Abdar et al., ), with some work addressing heteroscedastic pitfalls in log-likelihood-based loss functions
(Seitzer et al., ; Stirn et al., ).

Prior work on uncertainty in fair regression has focused on estimating quantiles – confidence intervals – for predictions (Liu et al., ; Kuzucu et al., ; Wang & Wang, ).
Quantile predictions are considered fair if they are equally calibrated and accurate for different demographic groups (Wang et al., ; ).
However, quantiles are only valid for a given probability threshold and fail to adequately describe the distribution. For example, recall how Figure 1 demonstrated how the mean and quantiles could be the same for two distributions that are substantially different.

## 7 Conclusion

FairlyUncertain actualizes the *consistency* and *calibration* axioms in the form of a robust benchmark for evaluating uncertainty estimates in fair predictions. FairlyUncertain suggests the following results: In the binary setting, natural uncertainty estimates beat complex ensemble based approaches and abstaining improves error but not imbalance between demographic groups. In the regression setting, *consistent* and *calibrated* uncertainty methods can reduce distributional imbalance without any explicit fairness intervention. The current version of FairlyUncertain is not without limitations: different models (Gorishniy et al., ), parameters, and datasets could be varied to assess *consistency*, different metrics introduced for *calibration*, and additional fairness interventions explored. Because of its specialized nature, we expect a net positive social impact from FairlyUncertain. We hope that the extensible package construction allows FairlyUncertain to grow with the community.

## Acknowledgements

Both LR and RTW were supported by the National Science Foundation under Graduate Research Fellowship Grant No. DGE-2234660.

## Appendix A Datasets Included in FairlyUncertain

**Table 6: We note that even though ACS Folktables is the improved version of Adult, we include Adult for parity with prior work (Ding et al., ; Cooper et al., ); we include COMPAS and Communities & Crimes for similar parity reasons, although highlight significant concerns with automating criminal justice (Fabris et al., ; Thomas & Pontón-Núñez, ).**
| Datasets (binary) | Size | # Feat. | Goal is to predict… | Protected Att. |
| --- | --- | --- | --- | --- |
| ACS Ding et al. () | 16249 | 16 | whether an individual is employed | race |
| Adult Kohavi et al. () | 45222 | 102 | whether individual income exceeds a certain level | gender |
| Bank Marketing Moro et al. () | 30488 | 57 | whether clients will subscribe to a product | age |
| COMPAS Angwin et al. () | 6167 | 406 | whether a defendant will re-offend | gender |
| German Credit Hofmann () | 1000 | 57 | whether an individual has ‘good’ or ‘bad’ credit | age |
| Datasets (regression) |  |  |  |  |
| Law School GPA Sander () | 22342 | 4 | students’ GPA in law school | race |
| Communities & Crimes Redmond & Baveja () | 1994 | 100 | # of per-capita violent crimes in a community | race |
| Insurance Lantz () | 1338 | 8 | individual medical costs billed by insurance | gender |
| IHDP Hill () | 747 | 26 | the cognitive test scores of infants | gender |
| Twins Almond et al. () | 68995 | 19 | the number of prenatal visits | race |

## Appendix B Additional Results on Consistency and Calibration

#### Why to Use Binomial NLL for Binary Classification Uncertainty

Consider a model $\tilde{f}:\mathcal{X}\times\mathcal{A}\to\mathcal{Y}\times\mathbb{R}$ that outputs binary predictions $\tilde{y}$ and uncertainty estimates $\tilde{\sigma}$.
We can evaluate the NLL of the estimates by converting them to probabilities.
We can solve for probabilities that produce the estimates $\tilde{\sigma}$ with the quadratic formula:(^2^22The catch is that two probabilities can map to the same standard deviation. Since the model $\tilde{f}$ also makes predictions $\tilde{y}$, we will choose the probability $\tilde{p}$ closer to the prediction.)

$$ $\displaystyle\tilde{p}^{-}=\frac{1-\sqrt{1-4\tilde{\sigma}^{2}}}{2}$ $\displaystyle\tilde{p}^{+}=\frac{1+\sqrt{1-4\tilde{\sigma}^{2}}}{2}.$ $$

Implicitly, Binomial NLL argues that if we had some $(\tilde{\sigma}\textquoteright)^{2}$ that was more accurate than the estimated uncertainty $\tilde{\sigma}$,
then we could get more accurate probability predictions $\tilde{p}\textquoteright$ from $(\tilde{\sigma}^{\prime})^{2}$. Thus, we should simply do our best to estimate $\tilde{p}$ (with NLL as the natural objective), which then induces a standard deviation $\tilde{\sigma}$. This motivates Binomial NLL and likely explains its strong performance.

#### Additional experiments

In Figure 7, we present boxplots of individual uncertainty predictions for binary classification, varying the reduction threshold $\gamma$ for the XGBoost model; the Binomial NLL method demonstrates greater consistency across different $\gamma$ values. Figure 8 shows similar boxplots in the regression setting, varying max_depth (top plot) and reduction threshold $\gamma$ (bottom plot), indicating that the Ensemble method is the most consistent across max_depth values, while other methods exhibit similar consistency levels. Figure 9 illustrates the calibration of various algorithms in regression by plotting the empirical standard deviation against the predicted uncertainty using 100 bins. Finally, Figure 10 displays uncertainty plots for binary classification across datasets and methods, varying max_depth, demonstrating that the Ensemble method produces consistent uncertainty estimates, whereas the Selective Ensemble method shows large variance and inconsistency in uncertainty estimates.

Figure: Figure 7: Boxplot displaying the mean, variance, and outliers of individual uncertainty predictions for the binary classification setting, with varying the reduction_threshold hyperparameter $\gamma$ (bottom plot) for the XGBoost model. The Binomial NLL method demonstrates significantly greater consistency across varying $\gamma$ values.
Refer to caption: x7.png

Figure: Figure 8: Boxplots showing the variance of individual uncertainty predictions for the regression setting with varying max$_$depth (top plot) and reduction threshold $\gamma$ (bottom plot) parameters. The Ensemble method is the most consistent across the max$_$depth values, while all other methods exhibit similar levels of consistency.
Refer to caption: x8.png

Figure: Figure 9: The *calibration* of various algorithms. We compute *calibration* by making 100 bins on the uncertainty measure: we plot the empirical standard standard deviation in the bin against the predicted uncertainty.
Refer to caption: x10.png

Figure: Figure 10: Plots of uncertainty for the binary classification problem across datasets (rows) and methods (columns), where we vary the hyperparameter max-depth. Each individual is plotted as a line, where the line’s higher y-coordinate is the maximum uncertainty estimate over all hyperparameter settings for that, and the lower y-coordinate is the individual’s minimum uncertainty estimate. Each individual is also placed along the $x$-axis at the standard deviation of their uncertainty estimates over the hyper-parameter settings. The Ensemble method produces consistent uncertainty estimates: the standard deviation of these estimates is less than 0.1 for all datasets and the uncertainty range starts close to 0 and increases with the standard deviation of the uncertainty (this is desirable). In contrast, the Selective Ensemble method produces inconsistent uncertainty estimates: the standard deviation of the uncertainty predictions is large and the range of the uncertainty predictions is large (especially for large standard deviations).
Refer to caption: x11.png

## Appendix C Binary Abstention Results

FairlyUncertain explores how abstaining from binary predictions based on uncertainty affects error rate, equalized odds, and statistical parity. While it clearly reduces error rate (Figure 11), abstaining has an unreliable effect on equalized odds (Figure 12) and statistical parity (Figure 13), similar to the behavior of the Random baseline. Figure 14 shows that the differences in variable distributions between the overall population and the set selected through the abstaining process are relatively small, except for the Binomial NLL distribution on the Adult dataset.

Figure: Figure 11: Abstention error rate.
Refer to caption: x12.png

Figure: Figure 12: Abstention equalized odds.
Refer to caption: x13.png

Figure: Figure 13: Abstention statistical parity.
Refer to caption: x14.png

Figure: Figure 14: The variable with the largest difference (as measured by Wasserstein distance) between the distribution on the overall population and the set selected through the abstaining process. Except for the Binomial NLL distribution on the Adult dataset, the differences tend to be relatively small.
Refer to caption: x15.png

## Appendix D Binary Fairness Results

FairlyUncertain fairness benchmark on all five binary datasets.
Abstaining reduces the error rate, and often equalized odds as a result, but has no improvement on statistical parity. Only Predictive Parity (the difference in the ratio of true positives to all positive labels assigned) was relatively low for abstaining methods, and is likely explained by none of the fairness methods explicitly optimizing this parameter.

**Table 7: ACS Income. Note that this is reported in Table 4 of the main paper body to 2 significant digits of precision. This was in order to make the font more legible - all results in this section are with 3 significant digits.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.22 $\pm$ 0.009 | 0.181 $\pm$ 0.023 | 0.165 $\pm$ 0.049 | 0.165 $\pm$ 0.049 | 1.65 $\pm$ 0.136 | 0.057 $\pm$ 0.034 | 0.083 $\pm$ 0.024 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.233 $\pm$ 0.009 | 0.029 $\pm$ 0.021 | 0.135 $\pm$ 0.046 | 0.063 $\pm$ 0.033 | 0.979 $\pm$ 0.07 | 0.241 $\pm$ 0.043 | 0.134 $\pm$ 0.046 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.226 $\pm$ 0.01 | 0.101 $\pm$ 0.03 | 0.08 $\pm$ 0.049 | 0.079 $\pm$ 0.049 | 1.3 $\pm$ 0.113 | 0.119 $\pm$ 0.039 | 0.024 $\pm$ 0.021 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.229 $\pm$ 0.011 | 0.041 $\pm$ 0.022 | 0.117 $\pm$ 0.038 | 0.065 $\pm$ 0.04 | 0.946 $\pm$ 0.08 | 0.203 $\pm$ 0.033 | 0.115 $\pm$ 0.038 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.218 $\pm$ 0.008 | 0.101 $\pm$ 0.041 | 0.092 $\pm$ 0.049 | 0.09 $\pm$ 0.052 | 1.3 $\pm$ 0.15 | 0.119 $\pm$ 0.045 | 0.031 $\pm$ 0.017 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.23 $\pm$ 0.012 | 0.083 $\pm$ 0.045 | 0.142 $\pm$ 0.064 | 0.108 $\pm$ 0.067 | 1.04 $\pm$ 0.259 | 0.183 $\pm$ 0.048 | 0.106 $\pm$ 0.067 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.227 $\pm$ 0.008 | 0.148 $\pm$ 0.034 | 0.132 $\pm$ 0.07 | 0.125 $\pm$ 0.074 | 1.48 $\pm$ 0.152 | 0.087 $\pm$ 0.048 | 0.056 $\pm$ 0.031 | 100.0 $\pm$ 0.0 |
| Random | 0.221 $\pm$ 0.009 | 0.174 $\pm$ 0.028 | 0.154 $\pm$ 0.046 | 0.15 $\pm$ 0.052 | 1.61 $\pm$ 0.155 | 0.044 $\pm$ 0.034 | 0.085 $\pm$ 0.024 | 88.3 $\pm$ 6.96 |
| Ensemble | 0.2 $\pm$ 0.019 | 0.193 $\pm$ 0.032 | 0.158 $\pm$ 0.063 | 0.155 $\pm$ 0.064 | 1.72 $\pm$ 0.18 | 0.032 $\pm$ 0.038 | 0.093 $\pm$ 0.031 | 89.0 $\pm$ 8.04 |
| Selective Ensemble | 0.201 $\pm$ 0.024 | 0.183 $\pm$ 0.03 | 0.154 $\pm$ 0.049 | 0.15 $\pm$ 0.059 | 1.69 $\pm$ 0.204 | 0.039 $\pm$ 0.037 | 0.082 $\pm$ 0.029 | 94.1 $\pm$ 7.29 |
| Self-(in)consistency | 0.188 $\pm$ 0.026 | 0.188 $\pm$ 0.028 | 0.155 $\pm$ 0.05 | 0.15 $\pm$ 0.058 | 1.73 $\pm$ 0.198 | 0.035 $\pm$ 0.028 | 0.081 $\pm$ 0.026 | 89.3 $\pm$ 8.87 |
| Binomial NLL | 0.176 $\pm$ 0.028 | 0.194 $\pm$ 0.021 | 0.153 $\pm$ 0.044 | 0.149 $\pm$ 0.051 | 1.77 $\pm$ 0.138 | 0.034 $\pm$ 0.032 | 0.08 $\pm$ 0.024 | 82.9 $\pm$ 9.5 |

**Table 8: ACS Public Coverage.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.265 $\pm$ 0.011 | 0.25 $\pm$ 0.031 | 0.211 $\pm$ 0.04 | 0.168 $\pm$ 0.061 | 0.479 $\pm$ 0.037 | 0.031 $\pm$ 0.03 | 0.194 $\pm$ 0.041 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.291 $\pm$ 0.016 | 0.038 $\pm$ 0.025 | 0.14 $\pm$ 0.033 | 0.129 $\pm$ 0.046 | 0.882 $\pm$ 0.07 | 0.034 $\pm$ 0.038 | 0.077 $\pm$ 0.031 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.288 $\pm$ 0.017 | 0.168 $\pm$ 0.028 | 0.141 $\pm$ 0.037 | 0.069 $\pm$ 0.045 | 0.637 $\pm$ 0.042 | 0.064 $\pm$ 0.039 | 0.139 $\pm$ 0.04 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.279 $\pm$ 0.008 | 0.035 $\pm$ 0.021 | 0.091 $\pm$ 0.052 | 0.088 $\pm$ 0.055 | 0.961 $\pm$ 0.116 | 0.152 $\pm$ 0.039 | 0.032 $\pm$ 0.02 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.277 $\pm$ 0.014 | 0.176 $\pm$ 0.036 | 0.148 $\pm$ 0.034 | 0.073 $\pm$ 0.048 | 0.62 $\pm$ 0.053 | 0.051 $\pm$ 0.04 | 0.147 $\pm$ 0.035 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.291 $\pm$ 0.013 | 0.077 $\pm$ 0.023 | 0.124 $\pm$ 0.07 | 0.115 $\pm$ 0.079 | 1.01 $\pm$ 0.255 | 0.139 $\pm$ 0.08 | 0.072 $\pm$ 0.042 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.276 $\pm$ 0.014 | 0.221 $\pm$ 0.034 | 0.184 $\pm$ 0.037 | 0.12 $\pm$ 0.054 | 0.557 $\pm$ 0.043 | 0.041 $\pm$ 0.046 | 0.181 $\pm$ 0.041 | 100.0 $\pm$ 0.0 |
| Random | 0.263 $\pm$ 0.012 | 0.239 $\pm$ 0.041 | 0.205 $\pm$ 0.042 | 0.152 $\pm$ 0.077 | 0.495 $\pm$ 0.049 | 0.028 $\pm$ 0.042 | 0.185 $\pm$ 0.043 | 88.4 $\pm$ 8.66 |
| Ensemble | 0.248 $\pm$ 0.019 | 0.281 $\pm$ 0.044 | 0.225 $\pm$ 0.054 | 0.21 $\pm$ 0.052 | 0.42 $\pm$ 0.036 | 0.015 $\pm$ 0.027 | 0.191 $\pm$ 0.042 | 89.6 $\pm$ 7.63 |
| Selective Ensemble | 0.241 $\pm$ 0.02 | 0.271 $\pm$ 0.033 | 0.236 $\pm$ 0.05 | 0.211 $\pm$ 0.073 | 0.412 $\pm$ 0.048 | 0.024 $\pm$ 0.022 | 0.169 $\pm$ 0.044 | 89.0 $\pm$ 8.52 |
| Self-(in)consistency | 0.251 $\pm$ 0.02 | 0.249 $\pm$ 0.04 | 0.202 $\pm$ 0.055 | 0.183 $\pm$ 0.064 | 0.447 $\pm$ 0.045 | 0.029 $\pm$ 0.025 | 0.162 $\pm$ 0.051 | 93.3 $\pm$ 7.86 |
| Binomial NLL | 0.236 $\pm$ 0.024 | 0.257 $\pm$ 0.034 | 0.207 $\pm$ 0.056 | 0.175 $\pm$ 0.071 | 0.436 $\pm$ 0.037 | 0.012 $\pm$ 0.021 | 0.166 $\pm$ 0.05 | 87.0 $\pm$ 7.32 |

**Table 9: Adult.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.139 $\pm$ 0.002 | 0.168 $\pm$ 0.006 | 0.092 $\pm$ 0.022 | 0.092 $\pm$ 0.022 | 0.291 $\pm$ 0.017 | 0.037 $\pm$ 0.017 | 0.058 $\pm$ 0.005 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.185 $\pm$ 0.002 | 0.007 $\pm$ 0.007 | 0.132 $\pm$ 0.01 | 0.024 $\pm$ 0.015 | 0.999 $\pm$ 0.037 | 0.499 $\pm$ 0.013 | 0.132 $\pm$ 0.01 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.15 $\pm$ 0.002 | 0.122 $\pm$ 0.008 | 0.026 $\pm$ 0.018 | 0.024 $\pm$ 0.019 | 0.53 $\pm$ 0.028 | 0.271 $\pm$ 0.03 | 0.01 $\pm$ 0.006 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.15 $\pm$ 0.004 | 0.019 $\pm$ 0.009 | 0.288 $\pm$ 0.023 | 0.288 $\pm$ 0.023 | 0.903 $\pm$ 0.045 | 0.335 $\pm$ 0.019 | 0.054 $\pm$ 0.006 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.135 $\pm$ 0.002 | 0.134 $\pm$ 0.005 | 0.034 $\pm$ 0.008 | 0.028 $\pm$ 0.012 | 0.433 $\pm$ 0.017 | 0.11 $\pm$ 0.032 | 0.03 $\pm$ 0.006 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.162 $\pm$ 0.004 | 0.029 $\pm$ 0.009 | 0.359 $\pm$ 0.013 | 0.359 $\pm$ 0.013 | 1.17 $\pm$ 0.055 | 0.412 $\pm$ 0.021 | 0.091 $\pm$ 0.009 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.132 $\pm$ 0.001 | 0.18 $\pm$ 0.005 | 0.075 $\pm$ 0.007 | 0.074 $\pm$ 0.009 | 0.318 $\pm$ 0.008 | 0.018 $\pm$ 0.018 | 0.062 $\pm$ 0.005 | 100.0 $\pm$ 0.0 |
| Random | 0.139 $\pm$ 0.002 | 0.168 $\pm$ 0.006 | 0.087 $\pm$ 0.018 | 0.087 $\pm$ 0.018 | 0.293 $\pm$ 0.017 | 0.032 $\pm$ 0.019 | 0.058 $\pm$ 0.005 | 87.1 $\pm$ 6.8 |
| Ensemble | 0.115 $\pm$ 0.023 | 0.171 $\pm$ 0.01 | 0.09 $\pm$ 0.028 | 0.09 $\pm$ 0.029 | 0.24 $\pm$ 0.047 | 0.03 $\pm$ 0.019 | 0.046 $\pm$ 0.009 | 86.6 $\pm$ 11.6 |
| Selective Ensemble | 0.129 $\pm$ 0.008 | 0.165 $\pm$ 0.008 | 0.105 $\pm$ 0.026 | 0.105 $\pm$ 0.026 | 0.271 $\pm$ 0.022 | 0.032 $\pm$ 0.021 | 0.049 $\pm$ 0.008 | 96.4 $\pm$ 2.54 |
| Self-(in)consistency | 0.134 $\pm$ 0.008 | 0.166 $\pm$ 0.007 | 0.101 $\pm$ 0.029 | 0.101 $\pm$ 0.029 | 0.276 $\pm$ 0.025 | 0.037 $\pm$ 0.021 | 0.052 $\pm$ 0.006 | 97.9 $\pm$ 2.21 |
| Binomial NLL | 0.106 $\pm$ 0.025 | 0.153 $\pm$ 0.007 | 0.122 $\pm$ 0.053 | 0.122 $\pm$ 0.053 | 0.263 $\pm$ 0.042 | 0.016 $\pm$ 0.01 | 0.033 $\pm$ 0.015 | 89.6 $\pm$ 7.89 |

**Table 10: Bank.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.092 $\pm$ 0.002 | 0.012 $\pm$ 0.006 | 0.053 $\pm$ 0.027 | 0.053 $\pm$ 0.027 | 0.925 $\pm$ 0.096 | 0.033 $\pm$ 0.016 | 0.004 $\pm$ 0.004 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.096 $\pm$ 0.002 | 0.006 $\pm$ 0.003 | 0.023 $\pm$ 0.014 | 0.023 $\pm$ 0.015 | 1.01 $\pm$ 0.062 | 0.038 $\pm$ 0.022 | 0.005 $\pm$ 0.003 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.097 $\pm$ 0.002 | 0.006 $\pm$ 0.004 | 0.025 $\pm$ 0.01 | 0.025 $\pm$ 0.011 | 0.975 $\pm$ 0.06 | 0.02 $\pm$ 0.018 | 0.003 $\pm$ 0.002 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.096 $\pm$ 0.002 | 0.006 $\pm$ 0.004 | 0.032 $\pm$ 0.017 | 0.032 $\pm$ 0.017 | 0.986 $\pm$ 0.067 | 0.035 $\pm$ 0.025 | 0.004 $\pm$ 0.003 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.096 $\pm$ 0.002 | 0.005 $\pm$ 0.003 | 0.026 $\pm$ 0.015 | 0.026 $\pm$ 0.016 | 0.99 $\pm$ 0.05 | 0.039 $\pm$ 0.025 | 0.005 $\pm$ 0.002 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.097 $\pm$ 0.002 | 0.009 $\pm$ 0.006 | 0.038 $\pm$ 0.021 | 0.038 $\pm$ 0.021 | 0.93 $\pm$ 0.056 | 0.031 $\pm$ 0.031 | 0.004 $\pm$ 0.004 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.097 $\pm$ 0.002 | 0.009 $\pm$ 0.006 | 0.038 $\pm$ 0.021 | 0.038 $\pm$ 0.021 | 0.93 $\pm$ 0.056 | 0.031 $\pm$ 0.031 | 0.004 $\pm$ 0.004 | 100.0 $\pm$ 0.0 |
| Random | 0.092 $\pm$ 0.002 | 0.01 $\pm$ 0.005 | 0.049 $\pm$ 0.028 | 0.048 $\pm$ 0.029 | 0.934 $\pm$ 0.084 | 0.031 $\pm$ 0.017 | 0.003 $\pm$ 0.003 | 91.0 $\pm$ 7.38 |
| Ensemble | 0.045 $\pm$ 0.026 | 0.005 $\pm$ 0.004 | 0.067 $\pm$ 0.05 | 0.067 $\pm$ 0.05 | 0.985 $\pm$ 0.135 | 0.021 $\pm$ 0.019 | 0.001 $\pm$ 0.0 | 84.3 $\pm$ 8.45 |
| Selective Ensemble | 0.077 $\pm$ 0.013 | 0.007 $\pm$ 0.004 | 0.043 $\pm$ 0.035 | 0.042 $\pm$ 0.036 | 0.922 $\pm$ 0.071 | 0.033 $\pm$ 0.021 | 0.002 $\pm$ 0.002 | 95.7 $\pm$ 3.8 |
| Self-(in)consistency | 0.076 $\pm$ 0.013 | 0.008 $\pm$ 0.007 | 0.053 $\pm$ 0.044 | 0.053 $\pm$ 0.045 | 0.902 $\pm$ 0.109 | 0.038 $\pm$ 0.032 | 0.002 $\pm$ 0.002 | 95.8 $\pm$ 3.82 |
| Binomial NLL | 0.067 $\pm$ 0.028 | 0.007 $\pm$ 0.005 | 0.043 $\pm$ 0.031 | 0.043 $\pm$ 0.032 | 0.908 $\pm$ 0.074 | 0.033 $\pm$ 0.029 | 0.001 $\pm$ 0.001 | 92.2 $\pm$ 8.74 |

**Table 11: COMPAS.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.316 $\pm$ 0.012 | 0.194 $\pm$ 0.027 | 0.188 $\pm$ 0.05 | 0.183 $\pm$ 0.054 | 0.542 $\pm$ 0.055 | 0.075 $\pm$ 0.051 | 0.125 $\pm$ 0.039 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.344 $\pm$ 0.015 | 0.042 $\pm$ 0.025 | 0.112 $\pm$ 0.047 | 0.074 $\pm$ 0.052 | 0.963 $\pm$ 0.107 | 0.227 $\pm$ 0.067 | 0.09 $\pm$ 0.053 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.35 $\pm$ 0.016 | 0.065 $\pm$ 0.031 | 0.074 $\pm$ 0.048 | 0.066 $\pm$ 0.052 | 0.82 $\pm$ 0.084 | 0.163 $\pm$ 0.067 | 0.04 $\pm$ 0.028 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.329 $\pm$ 0.013 | 0.041 $\pm$ 0.025 | 0.051 $\pm$ 0.027 | 0.038 $\pm$ 0.033 | 0.902 $\pm$ 0.069 | 0.161 $\pm$ 0.05 | 0.028 $\pm$ 0.018 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.332 $\pm$ 0.014 | 0.083 $\pm$ 0.028 | 0.085 $\pm$ 0.046 | 0.084 $\pm$ 0.047 | 0.792 $\pm$ 0.071 | 0.164 $\pm$ 0.066 | 0.038 $\pm$ 0.025 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.327 $\pm$ 0.015 | 0.047 $\pm$ 0.058 | 0.112 $\pm$ 0.06 | 0.097 $\pm$ 0.063 | 0.991 $\pm$ 0.182 | 0.176 $\pm$ 0.043 | 0.07 $\pm$ 0.042 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.323 $\pm$ 0.012 | 0.213 $\pm$ 0.03 | 0.224 $\pm$ 0.059 | 0.223 $\pm$ 0.06 | 0.517 $\pm$ 0.054 | 0.088 $\pm$ 0.063 | 0.135 $\pm$ 0.034 | 100.0 $\pm$ 0.0 |
| Random | 0.317 $\pm$ 0.014 | 0.181 $\pm$ 0.025 | 0.168 $\pm$ 0.04 | 0.154 $\pm$ 0.052 | 0.573 $\pm$ 0.051 | 0.063 $\pm$ 0.067 | 0.12 $\pm$ 0.043 | 86.3 $\pm$ 6.72 |
| Ensemble | 0.305 $\pm$ 0.011 | 0.201 $\pm$ 0.033 | 0.206 $\pm$ 0.06 | 0.197 $\pm$ 0.076 | 0.5 $\pm$ 0.09 | 0.056 $\pm$ 0.048 | 0.127 $\pm$ 0.022 | 87.8 $\pm$ 8.68 |
| Selective Ensemble | 0.288 $\pm$ 0.015 | 0.211 $\pm$ 0.047 | 0.213 $\pm$ 0.07 | 0.202 $\pm$ 0.088 | 0.498 $\pm$ 0.107 | 0.054 $\pm$ 0.046 | 0.126 $\pm$ 0.032 | 88.0 $\pm$ 7.52 |
| Self-(in)consistency | 0.287 $\pm$ 0.019 | 0.213 $\pm$ 0.033 | 0.219 $\pm$ 0.071 | 0.216 $\pm$ 0.074 | 0.485 $\pm$ 0.067 | 0.07 $\pm$ 0.06 | 0.121 $\pm$ 0.034 | 86.9 $\pm$ 9.61 |
| Binomial NLL | 0.292 $\pm$ 0.018 | 0.206 $\pm$ 0.033 | 0.199 $\pm$ 0.058 | 0.196 $\pm$ 0.061 | 0.5 $\pm$ 0.074 | 0.052 $\pm$ 0.047 | 0.123 $\pm$ 0.035 | 88.7 $\pm$ 8.4 |

**Table 12: German.**
| Approach | Error Rate | Statistical Parity | Equalized Odds | Equal Opportunity | Disparate Impact | Predictive Parity | False Positive Rate | Included % |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline | 0.251 $\pm$ 0.025 | 0.129 $\pm$ 0.06 | 0.147 $\pm$ 0.064 | 0.129 $\pm$ 0.07 | 1.72 $\pm$ 0.35 | 0.082 $\pm$ 0.048 | 0.094 $\pm$ 0.067 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer SP | 0.258 $\pm$ 0.026 | 0.081 $\pm$ 0.059 | 0.175 $\pm$ 0.082 | 0.152 $\pm$ 0.097 | 1.08 $\pm$ 0.469 | 0.187 $\pm$ 0.11 | 0.078 $\pm$ 0.062 | 100.0 $\pm$ 0.0 |
| Threshold Optimizer EO | 0.255 $\pm$ 0.029 | 0.155 $\pm$ 0.081 | 0.166 $\pm$ 0.077 | 0.151 $\pm$ 0.065 | 1.75 $\pm$ 0.423 | 0.105 $\pm$ 0.105 | 0.124 $\pm$ 0.092 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient SP | 0.264 $\pm$ 0.025 | 0.065 $\pm$ 0.056 | 0.11 $\pm$ 0.066 | 0.089 $\pm$ 0.063 | 1.29 $\pm$ 0.371 | 0.112 $\pm$ 0.112 | 0.063 $\pm$ 0.068 | 100.0 $\pm$ 0.0 |
| Exponentiated Gradient EO | 0.255 $\pm$ 0.029 | 0.155 $\pm$ 0.081 | 0.166 $\pm$ 0.077 | 0.151 $\pm$ 0.065 | 1.75 $\pm$ 0.423 | 0.105 $\pm$ 0.105 | 0.124 $\pm$ 0.092 | 100.0 $\pm$ 0.0 |
| Grid Search SP | 0.269 $\pm$ 0.023 | 0.136 $\pm$ 0.079 | 0.175 $\pm$ 0.073 | 0.16 $\pm$ 0.072 | 1.46 $\pm$ 0.602 | 0.086 $\pm$ 0.068 | 0.111 $\pm$ 0.078 | 100.0 $\pm$ 0.0 |
| Grid Search EO | 0.259 $\pm$ 0.038 | 0.15 $\pm$ 0.094 | 0.197 $\pm$ 0.108 | 0.186 $\pm$ 0.109 | 1.67 $\pm$ 0.443 | 0.109 $\pm$ 0.1 | 0.108 $\pm$ 0.073 | 100.0 $\pm$ 0.0 |
| Random | 0.25 $\pm$ 0.026 | 0.116 $\pm$ 0.057 | 0.139 $\pm$ 0.075 | 0.107 $\pm$ 0.091 | 1.64 $\pm$ 0.315 | 0.068 $\pm$ 0.04 | 0.087 $\pm$ 0.06 | 92.5 $\pm$ 8.25 |
| Ensemble | 0.222 $\pm$ 0.046 | 0.125 $\pm$ 0.076 | 0.155 $\pm$ 0.088 | 0.142 $\pm$ 0.1 | 2.04 $\pm$ 0.753 | 0.069 $\pm$ 0.077 | 0.077 $\pm$ 0.054 | 85.4 $\pm$ 9.95 |
| Selective Ensemble | 0.214 $\pm$ 0.045 | 0.122 $\pm$ 0.064 | 0.176 $\pm$ 0.097 | 0.157 $\pm$ 0.113 | 1.93 $\pm$ 0.496 | 0.082 $\pm$ 0.084 | 0.073 $\pm$ 0.047 | 88.4 $\pm$ 8.57 |
| Self-(in)consistency | 0.216 $\pm$ 0.046 | 0.109 $\pm$ 0.063 | 0.148 $\pm$ 0.084 | 0.142 $\pm$ 0.092 | 1.85 $\pm$ 0.481 | 0.068 $\pm$ 0.087 | 0.062 $\pm$ 0.042 | 90.3 $\pm$ 8.45 |
| Binomial NLL | 0.232 $\pm$ 0.023 | 0.107 $\pm$ 0.049 | 0.143 $\pm$ 0.085 | 0.132 $\pm$ 0.091 | 1.68 $\pm$ 0.351 | 0.057 $\pm$ 0.034 | 0.065 $\pm$ 0.054 | 92.6 $\pm$ 7.86 |

## Appendix E Binary Calibration with Different Sized Groups

We find that bigger groups (a smaller number of groups) exhibit more identifiable patterns with a positive correlation whereas smaller groups (a bigger number of groups) exhibit slightly more noisy behavior.

Figure: Figure 15: For five groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x16.png

Figure: Figure 16: For ten groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x17.png

Figure: Figure 17: For fifteen groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x18.png

Figure: Figure 18: For twenty groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x19.png

Figure: Figure 19: For twenty five groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x20.png

Figure: Figure 20: For thirty groups assembled by predicted uncertainty, we plot the average predicted uncertainty against the empirical standard deviation of the outcomes.
Refer to caption: x21.png

## Appendix F Notes on XGBoost and Customizing the Loss

XGBoost optimizes a loss using its gradient $g_{i}$ for first-order information and its Hessian $h_{i}$ for second-order information in a relatively [standard](https://en.wikipedia.org/wiki/Hessian_matrix#Use_in_optimization), higher order optimization framework (see Section 2.2. in Chen & Guestrin ()). Intuitively, $g_{i}$ indicates the direction for improving model predictions, and $h_{i}$ is useful for determining the curvature or “rate of change” of the loss. During XGBoost tree construction, each tree is built iteratively in order to correct the “residuals” left by previous trees. Here, “residuals” refer to the differences between the observed values of the target variable and the values predicted by the model e.g. the errors in the predictions made by the model. We let

$$ $\displaystyle w^{*}_{j}=-\frac{\sum_{i\in I_{j}}g_{i}}{\sum_{i\in I_{j}}h_{i}+ \lambda},$ (1) $$

where $w^{*}_{j}$ is the optimal weight for the $j$-th leaf, $I_{j}$ represents the set of instances in leaf $j$, and $\lambda$ is a regularizer. Formulating it in this way is the classic balancing act - we want to reduce the loss, but we regularize against the complexity of the model. Then, when we want to evaluate potential splits during tree construction, the “gain” from a split is calculated as:

$$ $\displaystyle\text{Gain}=\frac{1}{2}\left[\frac{(\sum_{i\in I_{L}}g_{i})^{2}}{ \sum_{i\in I_{L}}h_{i}+\lambda}+\frac{(\sum_{i\in I_{R}}g_{i})^{2}}{\sum_{i\in I _{R}}h_{i}+\lambda}-\frac{(\sum_{i\in I}g_{i})^{2}}{\sum_{i\in I}h_{i}+\lambda }\right]-\gamma,$ (2) $$

where $I_{L}$ and $I_{R}$ are the instance sets of the left and right child nodes post-split. Thus, specifying a custom loss for XGBoost simply requires deriving the first order gradient and second order hessian for a given loss function with arbitrary outputs.

## Appendix G Fairness Metrics

Below we provide formulas for the fairness metrics included in our benchmark. Let $X$ denote the set of features, $Y$ the true label, $\hat{Y}$ the predicted label, and $A$ the protected attribute (e.g., race, gender). Note that we generally report these as metrics, which is to say we quantify the degree of fairness by calculating the absolute difference between the two sides of the equality in each definition. For disparate impact, we report $\delta$ directly.

###### Definition G.1 (Statistical Parity) .

$$ $\displaystyle P(\hat{Y}=1\mid A=a)=P(\hat{Y}=1\mid A=a^{\prime})\quad\forall a ,a^{\prime}$ (3) $$

i.e. the probability of receiving a positive outcome should be the same across all $A$.

###### Definition G.2 (Equalized Odds) .

$$ $\displaystyle P(\hat{Y}=1\mid A=a,Y=y)=P(\hat{Y}=1\mid A=a^{\prime},Y=y)\quad \forall a,a^{\prime},y$ (4) $$

i.e. the prediction outcome is conditionally independent of the protected attribute $A$ given $Y$.

###### Definition G.3 (Equal Opportunity) .

$$ $\displaystyle P(\hat{Y}=1\mid A=a,Y=1)=P(\hat{Y}=1\mid A=a^{\prime},Y=1)\quad \forall a,a^{\prime}$ (5) $$

i.e. the true positive rate is the same across all $A$.

###### Definition G.4 (Disparate Impact) .

$$ $\displaystyle\frac{P(\hat{Y}=1\mid A=a)}{P(\hat{Y}=1\mid A=a^{\prime})}\geq \delta\quad\text{for some threshold }\delta$ (6) $$

i.e. the ratio of positive outcomes between groups remains close to 1 (typically $\delta=0.8$).

###### Definition G.5 (Predictive Parity) .

$$ $\displaystyle P(Y=1\mid\hat{Y}=1,A=a)=P(Y=1\mid\hat{Y}=1,A=a^{\prime})\quad \forall a,a^{\prime}$ (7) $$

i.e. precision (the probability of a true positive given a positive prediction) is the same across all $A$.

###### Definition G.6 (False Positive Rate Equality) .

$$ $\displaystyle P(\hat{Y}=1\mid A=a,Y=0)=P(\hat{Y}=1\mid A=a^{\prime},Y=0)\quad \forall a,a^{\prime}$ (8) $$

i.e. the false positive rate is the same across all $A$.

###### Definition G.1 (Statistical Parity) .

###### Definition G.2 (Equalized Odds) .

###### Definition G.3 (Equal Opportunity) .

###### Definition G.4 (Disparate Impact) .

###### Definition G.5 (Predictive Parity) .

###### Definition G.6 (False Positive Rate Equality) .

## Peer Reviews

confidence: 5
contribution: 3
initial_rating: 5
presentation: 3
questions: None
soundness: 3
strengths: 1. The paper identifies and addresses the underexplored intersection of fairness and uncertainty in predictive modeling.
2. The study spans multiple datasets and predictive tasks, enhancing the benchmark's relevance and applicability.
3. Designing the benchmark as an open-source tool encourages ongoing development and community contributions.
summary: The paper introduces FairlyUncertain, a Python package designed to evaluate uncertainty in fairness for machine learning algorithms. The package proposes methods and standards to incorporate uncertainty estimation into fairness evaluations in predictive models.
weaknesses: 1. The paper targets uncertainty as a critical factor in predictive fairness, given the real-world impact of machine learning decisions. However, it does not adequately justify the importance of the consistency and calibration axioms themselves within this context. Specifically, while uncertainty is broadly recognized as significant for transparent AI, the paper could strengthen its justification by explaining why consistency and calibration specifically matter for fair uncertainty estimation. 

2. Although the paper targets "fair uncertainty," the fairness component feels less integrated with uncertainty until Section 5, where it discusses the abstention framework and Uncertainty-Aware Statistical Parity. The paper could benefit from introducing the importance of fairness alongside uncertainty earlier, so readers understand the relationship between these two aims from the outset. 

3. This paper is largely an implementation paper, presenting a framework and accompanying package for uncertainty-fairness evaluation rather than introducing fundamentally new theoretical insights into either fairness or uncertainty estimation. Its main contribution lies in providing a package that can support standardized evaluations of uncertainty estimation methods within fairness contexts. Therefore, it may be better suited for a benchmark track or tools/package track than a general research track.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: 1- How does this work compare with existing research on fairness evaluation (beyond Section 6.1)? i.e., to what extent does the authors' focus on uncertainty help the practical selection of fairness measures and mitigation strategies?

2- Could this framework be adapted for more complex models or architectures and extended to additional fairness notions, such as intrinsic fairness? If so, how?

3- What are the societal implications of incorporating uncertainty in the selection of fair models? Could the authors expand on the strengths of their approach relative to existing methods, providing examples to illustrate the impact beyond simple numerical metrics? Existing literature on model multiplicity highlights the uncertainty involved and explores its societal, legal, and philosophical implications (there are many papers out there on this topic). I am wondering, what new insights does this paper contribute to these ongoing discussions?
soundness: 3
strengths: 1- FairlyUncertain introduces a standardized benchmark that is both theoretically grounded and practical, which allows researchers to evaluate how uncertainty affects fairness and vice versa. This is timely, given the increasing need for fair AI applications under uncertainty.

2- The paper defines fairness-focused axioms: Consistency (Axiom 2.3) and Calibration (Axiom 2.4), which set criteria for reliable uncertainty estimates. These axioms are clear and can be used for the practical goal of having fair predictive models that remain robust even under model variations.

3- The authors provide extensive evaluations of consistency and calibration across datasets and methods. They cover both abstention and confidence thresholding. The benchmark’s focus on these uncertainty handling methods, without endorsing one as definitive, offers a structured way to assess their impact on fairness. 

4- The benchmark is open-source which could be used in future research.

5- The benchmark’s use of consistency as a measure for similar pipelines indirectly addresses concerns about model multiplicity. FairlyUncertain's evaluation across slight hyperparameter changes serves as a practical check for this multiplicity effect.
summary: The paper introduces FairlyUncertain, a benchmark designed to assess the integration of uncertainty into fairness contexts within machine learning. Addressing the inherent prediction uncertainty challenge for fairness, the paper emphasizes that uncertainty estimates should ideally be consistent across similar models and calibrated to observed randomness. Through experiments across 10 datasets, including binary classification and regression tasks, the authors evaluate various methods for uncertainty estimation and introduce a new fairness metric, Uncertainty-Aware Statistical Parity (UA-SP), tailored for regression tasks. FairlyUncertain provides a structured benchmark to explore the nuanced relationship between uncertainty and fairness, specifically examining the effects of abstention and confidence thresholds.
weaknesses: 1- Axiom 2.3 suggests that small changes in hyperparameters should not significantly impact uncertainty estimates. While this is a reasonable assumption, it would benefit from explicit clarification on the stability limits of uncertainty under various hyperparameter changes to avoid overgeneralization.

2- Although the benchmark evaluates abstention as a fairness strategy, abstention does not reduce demographic disparities in binary classification tasks. Addressing why abstention fails to impact fairness in this context could inform potential adjustments to the benchmark. Also, given that abstention was shown not to reduce demographic disparities, the paper could address situations where abstention might inadvertently introduce biases. For example, explaining conditions under which abstention could be counterproductive would deepen the understanding of its impact on fairness.

3- While tables like Table 2 and Table 4 capture calibration and consistency scores, the paper could strengthen its analysis by discussing why certain methods perform better/worse across different datasets. Additionally, the appendix includes valuable demographic-specific breakdowns that could be better integrated into the main analysis to improve understanding of method reliability and demographic-specific impacts.

4- While XGBoost is effective for tabular data, expanding to deep learning models like MLPs, or transformers would align with recent trends in fairness research and test FairlyUncertain’s utility on more complex architectures. Reproducibility might vary across model architectures, especially complex ones, due to the implicit assumption of consistency under slight hyperparameter variations. While the benchmark is robust as presented, extending it to a broader model set might require tuning and validation.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Are there plans to include additional fairness interventions?
- Which new calibration metrics could enhance FairlyUncertain?
- Why does regression improve fairness without explicit interventions?
- How generalizable are the results beyond the ten datasets used?
soundness: 3
strengths: - Originality: Introduces FairlyUncertain, a novel benchmark for integrating uncertainty into fairness.

- Quality: Validated through extensive experiments across ten datasets; open-source and extensible.

- Clarity: Clear motivation, methodology, and presentation.

- Significance: Provides a foundational, scalable framework for enhancing fairness in machine learning.
summary: The paper introduces "FairlyUncertain", a benchmark to evaluate predictive uncertainty in fairness contexts. It emphasizes the need for consistency and calibration of uncertainty estimates across learning pipelines. Experiments on ten datasets reveal that a simple uncertainty estimation method outperforms prior work, that abstaining from uncertain binary predictions reduces errors but not demographic imbalances, and that incorporating calibrated uncertainty in regression improves fairness. The benchmark is extensible and aims to standardize fairness assessments involving uncertainty, promoting equitable and trustworthy machine learning practices.
weaknesses: - Findings on reducing errors but not outcome imbalances could be expanded with deeper insights or additional benchmarks.

- The paper does not fully explore variations in models, parameters, or datasets, which limits its generalizability and insights into broader use cases.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Is there more discussion on the connection between fairness and consistency/calibration? For example, how does the standard deviation across different hyper-parameters (which don't take sensitive attributes into account) help mitigate bias towards certain groups (which need information on sensitive attributes)?

2. Can this definition of consistency/calibration be generalized to other ML methods other than ensembles?

I would like to raise my score if these weaknesses/questions can be addressed.
soundness: 3
strengths: 1. The organization and writing are fairly clear, and the structure of the paper is sound.

2. The motivation is clear, and the background as well as the related works are well explained.

3. The paper shows enough originality and novelty, by providing a novel framework for fairness and uncertainty.
 
4. The experiments are extensive, comparing the performances on various datasets and different prediction tasks.
summary: The authors introduce FairlyUncertain, an axiomatic benchmark for evaluating uncertainty estimates in fairness. The benchmark posits that fair predictive uncertainty estimates should be consistent across learning pipelines and calibrated to observed randomness. FairlyUncertain suggests that: In the binary setting, natural uncertainty estimates beat complex ensemble-based approaches and abstaining improves error but not imbalance between demographic groups. In the regression setting, consistent and calibrated uncertainty methods can reduce distributional imbalance without any explicit fairness intervention.
weaknesses: 1. The connection between "Fairness" and "Consistency/Calibration" is not so clear: in the description of consistency/calibration metrics, the sensitive attributes don't seem to be included and discussed. For example, for consistency, all the metrics (SD or p-values) are computed over different ensembles, but how is it related to fairness? For calibration, it says "We identify groups of individuals with similar uncertainty estimates and empirically evaluate the standard deviation of the residual difference between the observed and predicted outcomes", but in the previous definition of calibration (Axiom 2.4), the group should be defined by sensitive attributes. The relationships between fairness and consistency/calibration should be further explained.

2. The methodology is limited to ensemble methods: the consistency/calibration metrics discussed in the paper all focus on ensemble methods (i.e., XGBoost), how can this be generalized to a wider class of ML methods (e.g., neural networks with stochastic optimizers)?

3. The experiments on fairness interventions are not very convincing: in section 5, the authors show that "abstaining from binary predictions, even with improved uncertainty estimates, reduces error but does not alleviate outcome imbalances between demographic groups", can the authors provide more insights into this phenomenon? This point is linked to my previous request for more explanation on the connection between fairness and uncertainty estimates.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: - I’m not sure the framework in Table 1 does a ton for me - doesn’t really connect to the rest of the paper at the moment so tightly and I’m not sure what the takeaways are. I would be interested to see a clearer explication of how these types of uncertainty connect to epistemic/aleatoric
- would like to see a more fleshed out argument for the consistency axiom
- would be interested to know more about connections to current calibration literature
- how much do results change if each fairness metric is optimized for individually (in combination with error)
- is there a more general version of the UA-SP definition?
soundness: 2
strengths: - this is an important topic for which there are no currently existing benchmarks that I’m aware of, I think this could be useful for people
- good level of thoroughness here exploring the various different metrics across datasets
- abstention experiments are interesting - surprising to me that abstention doesn’t improve most fairness metrics
- uncertainty-aware Statistical Parity is a good idea - incorporating the uncertainty estimate into a CDF-based metric makes sense
summary: This paper presents the FairlyUncertain benchmark, which is aimed at measuring the fairness properties of model uncertainty estimates. They focus on two metrics which are called consistency and calibration, and explore the results on these metrics across several algorithms and several datasets, concluding that probabilistically-grounded methods tend to return the best results. They explore the ramifications of abstention on fairness metrics, and propose a new uncertainty-aware form of statistical parity.
weaknesses: - I’m not sure I agree with the characterization (around L90) of A/B types as unmeasurable and C/D/E types as estimatable from a fixed sample. I think both of these statements are conditional on various distributional assumptions - for instance, you can measure individual variance if you assume you know the mean (or its parametric form and inputs). I think explicating the underlying data generating process would make this whole contribution more substantial
- A couple issues with formalisms in Section 2: 1. I don’t understand Def 2.2 - “differ only in hyperparameter settings” and “there is some j…” seem to say different things: the first implies P and P’ are in some shared “pipeline class” parametrized by different hyperparameters, the second implies that just one hyperparameter differs. 2. On Line 127, f(P) is used - this seems wrong to me, as on L117, f is named as the *output* of P, not a function of it
- I’m not sure I agree with the given definition of consistency as a useful fairness axiom - it seems like more of a Lipschitz condition over what a well-parametrized pipeline might look like. My impression of a fairness axiom of this type might be of the flavor that uncertainty estimates should be as invariant as possible to hyperparameter choices (I’m not sure this is a good axiom either but it seems more fairness-related).
- there is a rich literature on calibration metrics which is not connected to at all in this paper - it would be good to get a better understanding of how those relate to the chosen metrics
- L287 and elsewhere: for consistency, it seems like a non-robust choice to output a maximum std. Dev over individuals - I’m wondering if like a 90th percentile might be a better choice
- L391 - it seems odd to optimize an objective function that mixes statistical parity and equalized odds. Usually, one or the other is picked, since they are very different objectives
- Def 6.2 - it seems overly specific to me to assume that the model outputs are parametrizing a normal distribution. In particular, I don’t think this makes sense in the binary Y case - and the end of the definition (CDF comparison for all outcomes y \in Y) I think doesn’t make sense in the binary y case either (rather you want this to be true at all points in the range)

Small points:
- L142: “a model that always returns 0 is consistent” - by the previous definition this is not true, as the definition of consistency applies to a pipeline class, rather than the model which is outputted. Such a pipeline may output a constant-0 model for some set of hyperparameters, but not others.
- how is clustering done for calibration? I don’t think this is discussed
- Table 4: not sure how Disp. Impact is defined

Meta-review: This paper introduces FairlyUncertain, a tool that evaluate uncertainty in fairness for machine learning algorithms. This framework emphasizes consistent uncertainty estimation and be calibrated to observed randomness. 

The paper falls a bit short in its contribution. The main observations and reported findings suffer from generalizability without further exploring variations in models, datasets and fairness measures. In addition, the paper can benefit from better justifying the choice of computing the consistency and calibration of models.

Decision: Reject