Equal Opportunity of Coverage in Fair Regression
Authors: Fangxin Wang
Lu Cheng
Ruocheng Guo
Kay Liu
Philip S. Yu
Venue: NeurIPS 2023 poster (2023)
Keywords: Equal Opportunity; Fair Machine Learning; Conformal Prediction; Uncertainty Quantification
TL;DR: We propose Equal Opportunity of Coverage (EOC) for fair ML under uncertainty and develop methods to measure and improve it.

Abstract: We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy decision-making. The seminal work of 'equalized coverage' proposed an uncertainty-aware fairness notion. However, it does not guarantee equal coverage rates across more fine-grained groups (e.g., low-income females) conditioning on the true label and is biased in the assessment of uncertainty. To tackle these limitations, we propose a new uncertainty-aware fairness -- Equal Opportunity of Coverage (EOC) -- that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level. Further, the prediction intervals should be narrow to be informative. We propose Binned Fair Quantile Regression (BFQR), a distribution-free post-processing method to improve EOC with reasonable width for any trained ML models. It first calibrates a hold-out set to bound deviation from EOC, then leverages conformal prediction to maintain EOC on a test set, meanwhile optimizing prediction interval width. Experimental results demonstrate the effectiveness of our method in improving EOC.

Equal Opportunity of Coverage in Fair Regression
Fangxin Wang University of Illinois Chicago Chicago, USA fwang51@uic.edu
Lu Cheng
Ruocheng Guo ByteDance Research London, UK rguo asu@aaailmmm.ccommmmmm com
University of Illinois Chicago Chicago, USA
luchengQuic.ed.
Kay Liu
Philip S. Yu University of Illinois Chicago
University of Illinois Chicago Chicago, USA z1iu2340uic.edu
Chicago, USA payuεuic.
Abstract
We study fair machine learning (ML) under predictive uncertainty to enable reliable and trustworthy makii... The seminal work of "cqualized coverage" proposed an uncertainty-awe awarre fairness notion. However, it does not guarantee equal coverage rates across more fine-grained groups (e.g.. low-income females) conditioning on the true label and is biased in the assessment of uncertainty. To fairness Equal
tackle these limitations, we propose a new uncertainty-awart Opportunity of Coverage (EOC) that aims to achieve two properties: (1) coverage rates for different groups with similar outcomes are close, and (2) the coverage rate for the entire population remains at a predetermined level. Further, the prediction intervals should be narrow to be informative. We propose Binned Fair Quantile Regression (BFQR), distribution-fre post-processing method to improve EOC with reasonable width for any trained ML models. first calibrates a hold-out set to bound deviation from EOC, then leverages conformal prediction to maintain EOC on a test set, meanwhile optimizing prediction interval width. Experimental results demonstrate the effectiveness of our method in improving EOC. Our code is publicly available at httttps: htt/tgtttuuuu// com/ fangxin Waaggin-wwng/ballee wang/bfqr
Introduction
Machine Learning (ML) can bring bias and discrimination even with good intentions || Fair ML has been developed to counteract unfairness, but the practical use of fair ML models is limited by predictive uncertainty. Predictive uncertainty is the extent to which ML can confidently predict the future. Over- or under-confidence can cause an ML model to be unaware of its own knowledge gaps and make inaccurate predictions 7 This can lead unfairness in dreision-making To address this, we can produce predicted intervals for each sample and incorporate uncertainty into fairness make decisions more reliable and trustworthy.
2
The idea of "equalized coverage" an uncertainty-awar notion of demographic parity 10 was introduced in study [[]] as way ensure that every group receives the same level of prediction certainty. It works by generating prediction intervals that cover the true label Y with a specified probability (c.g.. 90%), while also reflecting uncertainty through interval width. However, even with this approach, there are still disparities in coverage rates across groups when conditioning on Y. For example, observe from the empirical results (Fig. M for Adult dataset 4 that low-income women are less likely to be covered than men in the same income bracket, and high-income men are less likely to be predicted earn as much nigh-income women. Consequently. the widths of prediction intervals for different groups, as an indicator of the uncertainty, are not comparable under
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

different coverage rates. These disparities can lead to unfair risk assessment for domains like bank loans and taxation. Further, equalized coverage may sacrifice the efficiency of uncertainty estimation for ensuring coverage rate as it produces wider prediction intervals for the group it intends to protect (See Section 5
Coverage rates for men and women with salaries in different bins
1.0
Net
0.9
Womon
Ouverage Rale
0.8
6.7-
6.4
Salary
Figure Evaluating equalized coverage on the Adult dataset. The protected attribute is gender. The test data is partitioned into 10 equal-mass bins based on the ascending order of salaries. Significant variations in the coverage rates can be observed among different groups within the head and tail bins.
To address the limitations of equalized coverage, propose novel uncertainty- based fairness notion Equal Opportunity of Coverage (EOC), extending from the standard fairness notion of equal opportunity [12]. EOC aims achieve two properties: (1) similar coverage rates for different groups (e.g., female and male) with similar outcomes (e.g.. salaries), (2) achieve a desired level of coverage rate (e.g.. 90%) for the entire population. Ideally, provide informative predictions, intervals should be as narrow as possible while still satisfying EOC.
We consider the regression task as it is more general fair ML setting assumptions about the underlying data distribution.
13
with minimal
Achieving EOC confronts various challenges. Firstly, the majority of prior fair ML approaches are developed for classification problems 12 [3]. with only few for regression 14 16] This necessitates developing effective techniques to measure and improve EOC in regression setting. Secondly, ensuring EOC and marginal coverage rate for test data difficult when true labels are unknown. Finally, prior works [I] I7 satisfying coverage rate guarantees, but often neglect the width of prediction intervals during the optimization process. This often results in generating wider intervals that limit the amount of decision- making information available. However, optimizing interval width with fairness constraints is a non-convex problem, and coverage rates difficult to guarantce noisy data, making it challenging to consider all these factors simultancously.
18. [9 have primarily focused on improving faireess and
To address these challenges, we propose Binned Fair Quantile Regression (BFQR), distribution free post poot-processing method that improves EOC while maintaining a desired marginal coverage rate and narrow prediction interval. It consists of three major steps: First, a hold-out (calibration) dataset calibrated to improve EOC based on true label Y within discretized bins; second, we leverage conformal prediction (20 |2 to achieve EOC for test data from the calibration results in the first step: and finally, an efficient and robust optimization technique is developed to minimize the mean width of prediction intervals. Experiments on both synthetic and real-world data show that BFQR is more effective in improving EOC fairness than state-of-the methods.
Related Work. In regression settings, equal opportunity 12 has been studied mostly by adversarial training 2 23 For quantile regression, models with equal opportunity constraint 24 25 are
proposed. However, all mentioned are in-processing methods and have trade-off's between accuracy and fairness 22 22] There are several works on uncertainty-
fairness in regression. The pioneering work equalized coverage [[I] is built on the validity of conditional coverage [28] and has several follow-ups [1. 19] also proposes conformalized fair regression, imposing demographic parity fairness on prediction interval bounds. However, neither of these methods considers the fine- grained group (e-g., low-income females) fairness conditional on true labels as well as the increased width of prediction intervals that provide little information.

---

2 Preliminary
Conformal Prediction extends traditional ML by providing set of prediction intervals around the predicted value, which can be used to assess the level of confidence or uncertainty in the prediction [20]] It is distribution-free and has rigorous statistical guarantee. A commonly used approach is split conformal prediction 20
21 To start, the training data is divided into two
sets: the training set Dtr and the calibration set De {(X,, .i.,... (X,, Y,)}. prediction model f is trained on the training set Dr. The key ingredient is the conformity score function S(z, v) I used to evaluate the model's prediction performance. Given desired enror rate a, it then caleulates the quantile Q1-a (D),, denoting the (1 a1 1/|De|)-th quantile of the empirical distribution of S on DD. Finally, for sample X in the test set DD, its prediction set is (X++1) {v S(Xn+1,, v) S Q1-a(S, (S, (.))- This set contains all possible values of y for which the conformity score S(x++1.9) y) less than or equal to the calculated quantile. Given a mild assumption that the test and calibration set are exchangeable, the coverage rate in conformal prediction is guaranteed with a probability of P(Yn+1 E C (Xa+1)) 2 1 a.
Equalized Coverage. Let V E V be the indicator of whether Y is covered in the prediction set Ć(X). ie., V = Y E C( J. V = {0, 1). Given a desired error rate a E [0. 1], equalized coverage is Equalized coverage guarantees equal conditional coverage, i.e., V is independent of A (denoted as V 11. A), but fails to ensure equal coverage conditional on Y, especially for extreme values of Y (Fig. D.. This is problematic as it can perpetuate discrimination against marginalized groups (e.g., females with low income) in risk assessment.
satisfied M when Va E A, Pr(vIÁ = a} = Pr(V) 2 - and Pr(V) 2
3 Equal Opportunity of Coverage
Equal Opportunity of Coverage addresses the limitation of equalized coverage and is defined based on the equal opportunity 11:
Definition 3.1 (Equal Opportunity of Coverage (EOC)). EOC is satisfied when Pr(VI4 = a, Y y) = Pr(V = y}, i..., V 1 A|Y and Pr(V)
a, Vy Y and Va A.
The definition of EOC requires that (I) conditioned on the target variable Y, whether a sample is covered in its prediction interval should be independent of its sensitive attribute A; (2) the marginal coverage rate is above the desired level. Interestingly, the difference between equalized coverage and EOC in their mathematical forms share similarities with the difference between demographic parity and equalized odds. Note that our focus here is whether the true label is covered in the prediction interval since even though the prediction interval contains false labels, V = II[ E Č(X) is still valid. Therefore, while EOC has a similar formation to equalized odds, in fact describes equal opportunity which focuses on Y == 1.
Preferably, both EOC and equalized coverage should be guaranteed. However, the mutual exclusivity theorem below suggests that there is an inherent trade-off between EOC and equalized coverage:
Theorem 3.1 (Mutual Exclusivity). A V. Y and V M.. Y, then either equalized coverage or equal opportunity of coverage holds but not both.
Proof. If V TT A and V IL AYY, then either ALL Y or J.Y.
Unfairness often arises from the fact that features predictive of Y are also correlated to the protected attribute due to e.g., historical bias in the data 29 This indicates V II A. For the second condition, though V J. Y is possible, V needs to depend on Y to ensure prediction intervals with reasonable width. Predicting certain values (c.g., extreme values) of Y can be challenging due representative bias. If we enforce the predictor to provide high coverage for marginalized groups with these values, it is highly likely to result in extremely wide intervals that offer little guidance in decision-male making.
Meauring FOC. Given the underlying distribution p (VVxAxX) where V. A. y the domain of U a, y respectively. We can determine how likely p satisfies EOC by measuring its distance to p the closest distribution that perfectly achieves BOC. Formally, we denote Proc as the property BOC, the set of all distributions in defined space that satisfy EOC, i.e., Proc == (v E A(V x A x y) (V, A, ~ P.V. V IL AlY) p' E PEoC is the distribution with minimum total variation (TV) distance P, i.c., Vq E PEOC. drv(p.v') drv q). The TV distance between p and p' is
3

---

formally defined as drv(p.p) = E a)) p'(,,, 1pp-r'l
where ||-1|, denotes the 1 norm of a distribution. Instead of the commonly-used Kolmogorov-Smirnoy (KS) distance [12 30I. TV distance is chosen as the measure due to the significant drawback of the KS distance its insensitivity to the deviations between p and p' at the tails [31] Whereas in real-world data, we often confront such significant deviation (see e.g., Fig. D
It is challenging to directly measure the distance between p and p' from observed data when the target variable Y is continuous due to data sparsity issues for each value of Y. However, drawing from previous works 32 E&l
we can instead use casy-to-compute statistic T as surrogate of (pv T is the measure of independence, assessing the weighted summed violation of independence within each discretized bin. The expectation of T, E[TT", has fixed upper bound when p satisfies EOC, and a lower bound that increases with the TV distance between p and p'. The intuition here is that when Y is divided into sufficient bins and py Pr(V, A|Y), ), the conditional distribution of p on Y, is Lipschitz continuous, enforcing independence within each bin generates a distribution that does deviate far from Peog in terms of TV distance.
Lemma 3.2. 133 Let [n2/5] be the number of bins Y discretized into, with each bin having an equal size of samples. Let ' be sum of independence measures within all discretized bins. Given the Lipschitz continuity of py Assumption
2 1. is the Lipschitz constant, we have:
1) When satisfies EOC, [E[T] < IDL." d=
2) When (p.p') = 6, there exists a constant Z such that [B[2] 2 Z(E
34)². 224)2
Lemma 3.2 indicates that we can approximately evaluate EOC for p by measuring the independence within each bin. If V IL A almost holds in each bin, such that E[T] is small enough, then it highly possible that V IL A|Y, i.e., satisfies EOC. Moreover, an increasing function of e is upper bounded by [[T] As E[T) increases, it is highly likely that p is further away from any p' that satisfies EOC. This lemma serves as theoretical foundation for the proposed method below to improve EOC and for adopting T' an evaluation metric in experiments.
In order to calculate E[T] from data, we introduce unbiased estimators of T in Appendix 8.1.1 According the central limit theorem, random samplings. Whereas for the sake of efficiency, we prefer to construct T with bounded variance such that E[T] could converge through limited repeated samplings.
could estimate E[T] through sufficient number of
4 Improving Equal Opportunity of Coverage
In this section, we introduce post-processing approach where we have a trained ML model, calibration dataset, and test dataset for which we aim improve EOC. consists of three steps. First, we enforce the independence of A and V within each interval of Y (i.e., EOC) for the calibration data, and then leverage conformal prediction to achieve EOC for test data. Lastly, we describe an efficient and robust optimization approach that optimizes both EOC and widths of prediction intervals.
4.1 Improving EOC on calibration data
The first step aims to improve EOC on the calibration data where we have ground labels. As the target variable Y is continuous, the number of samples with certain values of Y can be extremely small, thus calibrating for each distinct value Y is almost impossible. Meanwhile, commonly used
methods designed for continuous variables such as adversarial learning [24] 22 which intend to
learn a ncar-optimal p. are computationally inefficient for post- post-processing approaches
34
According to Lemma 3.2 [E[] is an upper bound of an increasing function of drv (p, Thus, this indicates that if T decreases, the maximum distance between p and p' is reduced, resulting in an improvement in EOC. Informed by this idea, we first focus on enhancing EOC on the calibration data De. For simplicity, we employ the framework introduced in (35) as the conformal prediction model, though our prot-processing method is applicable to any base model. Specifically, let qa. denotes the a-th conditional quantile regression function, i.e., for i-th sample (X,, Y.). do (X,) == inf (y E Ay: Pr < x X.) > a). Fix the lower and upper quantiles as ato = a/2 and αra = a/2, then Go.. (X,) and lons (X.) denote lower and upper quantile regression functions, respectively.
The base model is trained on the training data and used for inference on calibration and test data.

---

Following the discretization idea in Lemma 3.2 we divide the continuous variable Y E [Y [mmi:: Ymas) in D. into M bins with equal sample sizes, and the m-th bin is denoted as B,n [m y
To enhance EOC on ee. we can minimize [T] by enforcing the independence of A and V within all discretized bins. In partieular, fix the coverage rates within each bin 3. E [0, 11, therefore V TT A|Y E Bm. Note that, although the coverage rates are equal across groups within the same bin, the quantile value at fm are computed separately for each group, as illustrated in Figure 2 We first calculate the vanilla prediction intervals Č(X,) = Qen (X,), 9as. (X,)] obtained from the trained model, and get the conformity score SXX,, Y:) = max (9au (X,) - Y,, Y 9a.. (X,)) for all samples in De. Then for different combinations of bins and protect groups, we calculate the quantile value of conformity scores at coverage rate 3m. (5, (a m)), for data in (a m) {ili E DD., A, = a, Y E }. To simplify notations, we substitute 2 la m)) by Ga. (3m) Since each bin has an equal sample size and coverage rate 3m. the average coverage rate 2m, 9m/ /M is then set to 1 a to keep coverage rate of a on the calibration data.
1.00
853
0.800
880
0.000
9866
0.40
0.403
0.400
0.20
0.200
0.200
0.00
-3000-109039
U000 20000 3009 40300
0.003 -30090000000pp
Mikee 2006 2000 30300
0.000
100000 200600 30000 400000 500000
Contamiry
(a) Bin 1
(b) Bin 2
(c) Bin 10
Figure 2: Empirical Cumulative Distribution Functions (ECDF) of conformity scores of men (blue) and women (red) within bins 1,2 and 10 on the Adult dataset. The desired coverage rates within different bins could be set at different levels but should be equal across groups within the same bin. as indicated by the overlapping horizontal dashed lines. Due to the distinct disparity between the ECDF of men and women, the same coverage rate is mapped to different quantile values the x-axis.
Through the reconstruction of prediction intervals C(X,) with Ga.m (8m), i.e., Vi E De(a, m). C(X,) [âc.. (X,) - Go.m (3m). qa.. (X+) + Go,m (.m)), EOC enhanced and the marginal coverage rate is guaranteed on the calibration data.
4.2 For Coverage and Independence Guarantees
This subsection seeks to preserve EOC and marginal coverage rate for test data based on results for calibration data through conformal prediction. The key is to find out, for a test sample with
Ar+1 Ox+1, which bin B,, it belongs to. We can then calibrate the test sample with the quantile (B)) However, direct calibration based on the predicted bin would not improve EOC since the prediction result can be biased due to, e.g., skewed distributions for different groups
29
To address this issue, we propose Binned Fair Quantile Regression (BFQR) (see Algorithm I in Appendix 8. Our method could be treated variant of Mondrian conformal prediction Rd 17 where the confidence in each bin evaluated independently. Suppose that a new data point with feature X; and protected attribute falls into certain bin Bm. we calibrate it with the corresponding quantile value Go. (Bm).. Then, we obtain a sub-interval of prediction within bin Bm i.e., C.. (X,) B., n [Âmn, (X,) Ga.m (Bm). Gon, (X.) + Ga. After computing Cm(X,) (X.). a union of all subsets C(X,)= = Um Cm (X,) is then the prediction interval of X,, Under a mild exchangeability assumption similar to [I]. our algorithm provides both the marginal coverage guarantee and fair coverage guarantees within each bin.
Assumption 1 (Exchangeabilltty) All calibration data (X,, YY), i= L,..., n and a sample of test data and Y.+1 E Bm, and conformity scores
(Xn+1, Yn41) are exchangeable conditioned An+1 {S(X,, Y,, á E DD(a, m) U (n + 1}} are almost surely distinct.
Theorem 4.1 (Bin Coverage and Independence Guarantee). Under Assumption ! 3m S Pr(Yn+1 e C(Xn+1) = a, Yn+1 E Bm3 S 3m + 1/([DD, m)l + -1). The expectation of max coverage gap inside m-th bin is upper bounded by max marxa1/((BBaa,, m)] + 1)].
Theorem 4.2 (Marginal Coverage Guarantee). If we have Em 3m/ = a. under Assumption
then Pr (Y,+1 C(Xn+1)) > 1 - a.
5

---

Here we sketch the proofs. Take (X,, A, Yi) new sample from test data, i E Dr. Sup- pose Y Bm then the p-value of the null hypothesis Y, E Bn is given by A.A. (t)
which is the proportion of conformity scores that are less than the score of the new sample among all calibration data that have the same protected attribute and fall into the same bin. The prediction interval Čm (X:) in Bm the intersection of Bn and the prediction interval calibrated by Ga.m (3m),. which includes the part of Bm in the prediction interval where the p-value A. is greater than 8m. This guarantees the bin coverage of B., at level 3m in Theorem 4.1
I+I..
The complete proof' can be found in Appendix 8.2.2
4.3 Constrained Optimization
With improved EOC and coverage guarantee, we now need to identify S.. for each bin so that the mean width of prediction intervals for ID, is the smallest. As such, goal is to solve constrained optimization problem, where the decision variables are the coverage rates in each bin 3m. m l,...; .M, and the objective function is the mean width of prediction intervals in test data:
min Σ
Icm (X,)))//DD
m,iElP
Σn 8 M = - 02 Bm [0, 1].
s.t.
m
M.
An easy solution to the optimization problem Eq. 1 can be obtained by adjusting Č(X,), X₁ E Do with split conformal prediction described in Section 2 However, it is not optimal since 3m is determined by Q (S, D2) calculated on all calibration data, but different bins have varying costs of width associated with changes in their coverage rates. For instance, if a bin has coverage rate of 0.98, increasing it to 0.99 would lead to a significant increase in its width, whereas increasing the coverage rate of bin from 0.50 0.51 would result in only minor increase in width. Therefore, we use the solution of split conformal prediction as the initialization and then optimize it.
Solving Eq. iss challenging in that it is computationally expensive or even infeasible to compute the value and the gradient of the objective function. First, the computation of prediction interval C(X.) involves multiple intersection and union operations, which is complex step function of 3m. Second, prediction interval C(X)) is related to quantile value Ga.m. which is estimated from data with noise. Directly using slopes of G..m the gradient methods could result in ever-fitting to noise, as shown in Section 5.2.4] Third, We cannot assume the objective function's 's convexity or differentiabilityy
the data may come from any possible distribution, and sorting is involved in calculating Go m. [8)) To address those obstacles, we propose an efficient and robust optimization algorithm (detailed in Algorithm in Appendix 8.3.2) that utilizes a relaxed upper bound and optimizes through approximated subgradients
The steps of our approach are described below. First, to accelerate computing, we use dummy continuous prediction interval (X,) = Conver(U,, (X.)), i... the convex hull of all sub- intervals, as an upper bound to substitute the original prediction interval C(X,) in the objective function. However, (x,) ) is still related to noisy Ga.m. To address this, in each iteration, we compute the slope for each bin in increasing and decreasing directions, denoted as i and i- respectively. Since the objective function aims to decrease without changing the marginal coverage rate, we take greedy strategy by moving up step 1) in the bin with the steepest descendent direction max,, (f, meanwhile taking a step down the slowest ascendant direction min, ( }. We stop until max (ftt) 2 min,
ff.m + 2s, where is an appropriate estimation eтror bound related with More details can found in Appendix 8.3.1
[Del 40
The proposed method can be viewed as a subgradient method, incorporating considerations for quantile value estimation errors and maintaining constant mean coverage rate. Combining the three steps, we could get prediction intervals with improved EOC, guaranteed marginal coverage, and decreased average width of prediction intervals.
5 Experiments
In this section, we conduct three sets of experiments on both synthetic and real-world data to evaluate (L) the effectiveness of the proposed approach for achieving EOC (Section 5.2) (2) the impact of the

---

key parameter M, the number of bins (Section 5.2.3 and (3) the effectiveness and efficiency of the proposed optimization framework (Section 5.2.4).
5.1 Experimental Setup
We propose two metrics to evaluate the first property of EOC, i.e., whether coverage rates for different groups with similar outcomes are close. With the M discretized bins in Section 4. the first metric the average of maximum difference in coverage rates between groups for all bins, similar to the definition of conditional KS distance in 12 In main experiments, M is set as 20. The second metric introduced in Lemma 3.2 where T is calculated on d = [[D;] 12/07 bins. Specifically, o M for fairness of evaluation, e.g., d = 100 for synthetic data. A smaller T implies closer distance to the ideal EOC distribution and, therefore better EOC. Furthermore, for the second property of EOC, we need to ensure the desired marginal coverage rates. The efficiency of uncertainty estimation is measured by the width of the prediction interval. In addition, we check the conditional coverage rates on groups to measure equalized coverage. All metrics related to coverage rates are multiplied by 100 in tables to exhibit significant differences. Every experiment is repeated 100 times on random divisions of data with different seeds, with [Dtrl] [De] [B, = 3: 1: 1.
We compare our method with the following tateeoff-hhe-ar methods: 1) Split Conformalized Quantile Regression (CQR) B5 21 with only marginal coverage guarantee; 2) Group- crouu-condiiional Confor- malized Quantile Regression (GCQR) [ with both marginal and conditional coverage guarantee; 3) MultiValid Predictor (MVP) DS with both marginal and conditional coverage guarantee. Note that the base model for this algorithm is trained on the union of training and calibration data as MVP does not require any calibration data; 4) Conformal Fair Quantile Prediction (CFQR) [19], which guarantees marginal coverage and demographic parity on both upper and lower bounds of the predicted intervals; 5) Label-conditional Conformalized Quantile Regression (LCQR) 42 which is designed for classification problems to provide marginal coverage and equalized coverage for each class. We adapt it to regression tasks. The base model for all compared conformal prediction methods is set as the QR model at the level of 0.05 and 0.95, and the desired marginal coverage is set to 0.9. Considering that for some real-world applications like scoring [3) disjoint prediction intervals make little sense, we evaluate prediction intervals C(X) along with their dummy prediction
intervals Cdd (X) in Section 4.3 represented as BFQR and BFQR* In the comparison tables, the best results and the second-bes -bestdddbtttudee results are highlighted in bold and underlined, and undercovered groups who fail to reject the null hypothesis at 0.05 level in one-tailed t-tests are emphasized in Italian.
5.2 Results
5.2.1 Synthetic data
We generate ten independent and exponentially distributed features with the scale of 1. X = (X),)..;Xoo) protected attribute A is randomly selected from {0, 1,2] with a probability of 0.1, 0.2. 0.7, respectively. The labels Y for A = =1 follow a random distribution, thus impossible to predict; labels Y for the other two groups are the linear summations of X and A, plus noises that increase with Y. A size of 100,000 samples are generated from this distribution and the data generating process is detailed in Appendix 8.4.1
We have the following observations from the results in Table I 1) All methods have marginal coverage guarantee, which is attributed to statistical guarantee from conformal prediction. 2) Our proposed methods, BFQR with disjoint intervals and BFQR* with joint intervals achieve the best trade-off between EOC and equalized coverage. In particular, our method achieves the second-best EOC (i.c.. Mean Max Coverage Gap and 2), meanwhile, the conditional coverage rates are almost equal across different groups. While CFQR has significantly better performance w.r.t. EOC, the conditional coverage for A = 1 (the most challenging case) is extremely low compared to A = 0 and = This result aligns with the mutual exclusivity between EOC and equalized coverage formulated in Theorem 3.1 3) Among all methods, the average interval width of our method is the smallest, validating the effectiveness of the optimization process in Section
One of the main advantages of BFQR and BFQR* * is optimizing through bin coverages. In this process, bins that sacrifice interval width for an over-coverage rate are adjusted to a lower but satisfactory coverage rate. Therefore, we are able to guarantee a smaller average interval width while improving the EOC.
7

---

Table 1: Experiments results for synthetic data.
FOC
Equalized Coverage
Method
Width +
Mcan Max
Marginal Coverage
Coverage
Coverage (A = 1)
Coverage (A = 2)
Coverage Gap +
(4 = 0)
CQR
20.11=1.05
306.33+18.JJ
89.99=0.32
16. 16.02+0.04 04
82.84+0.90
90.03+0.76 76
91.00=0.22
GCQR
1..4
386.81+:35.66 60
90.00+0.33
16.21+0.06
90.08+0.99
89.92+0.64
=0.37
MVP
9.03+1.43
71.27=6.70 70
89. 99.9820.32
16.31+0.13
89.35
90.11
89.96+0.22
CFQR
0.19=0.09
0.09=0.24
89.93=1.220
17.00+0.06
93.8940.53
71.38+0.87
94.74+0.24
LCQR
6.38+0.26
18 6..4466.71
90.41=0.29
17.32+0.09
91.37+0.84 84
90.55+0.68
90.23=0.37
BFQR*
3.15+0..8
2.78=3.79
91.74+1.25
16.24+0.21
93.81
91.19+3.13
91 01.000.900
BFQR
3.82+0.45
3.65a4.75
90.03+0.32
15.96+0.13
91.99+1.16
89.08+3661
90.03+1 004
5.2.2 Real-world Data
We further evaluate our method on two benchmark datasets: Adult 45 protected attribute and the outcome is salary; MEPS (Medical Expenditure Panel Survey) data 46 where race is the protected attribute and the outcome is the health care system utilization score.
where gender is the
We observe similar results: For Adult data (shown in Table 2 all methods achieve marginal coverage. Our methods achieve the best EOC and smallest mean width of prediction interval while maintaining competitive conditional coverage rates. CFQR with the best EOC on synthetic data does not have consistently good performance. and the mean interval width is greatly larger compared with other methods. The increased width of LCQR implies that some bins difficult to predict, enforcing all bins to reach the same high coverage rates generates prediction results with little useful information. For MEPS data (shown in Table B the results are similar those in synthetic data: method is the second-best w.r.t. EOC, with marginal coverage rates guaranteed and a significantly smaller average prediction width.
Table 2: Experiments results for Adult data.
EOC
Equalized Coverage
Method
Width
Mean Max
Marginal Coverage
Coverage (Men)
Coverage
T
Coverage Gap L
(Women)
CQR
5.01+0.40
10.36
89.9820.29
93,951. 10 .25
90.01
89.9420.37
GCQR
4.97=0.50
67.30=10.63
89.99=0.29
89.98+0.41
09.99=0.46
MVP
5.50+0.34
211.01+19.44
90.05+0.28
29,229.811+1,223
90.0740.1
90.08=0.27
CFQR
3.57=0.53
18.17+6.02
90.08#0.11
111259.77:499..4
90.02=0.44 42
90.08=0.40
LCQR
3.91=0.45
5.03+4.42
90.50+0.30
160,107 107. 96*8929.07
90.46+0.37
90.54=0.46
BFQR*
2.90+0.37
3,60:3.29
91.08+0.46
93.689.33 33-976.76
91 01.990.74
90.11=0.43
BFQR
3.05+0.35
3.55+3.14
90.32+0.28
91.969.66299666
90.97+0.42
89.5620.47
Table 3: Experiments results for MEPS data.
BOC
Equalized Coverage
Method
Width +
Mean Max
Marginal Coverage
Coverage (White)
Coverage
T'
Coverage Gap
(Non-white)
CQR
6.47+1.22
7.6244.28
89.85=0.77
32.06+1.85
89.81
89. 89.90+0.88
GCQR
6.54+1.79
9.50+8.35
89.91+0.74
32.08+1.90
89.91=0.95
89.93+1.11
MVP
8.27+2.37
8.78+3.47
89.95+0.84
41.08+6.87
90.97+0.71
90.75+0.97
CFQR
0.94+0.39
0.28+0.55
90.88+0.51
36.55+1.85
87.93+1.06
93.18+0.77
LCQR
5.29+0.97
3.29+2.52
91.97+0.69
160.65+30.71
91.59+0.94
92.57+1.02
BFQR*
3.0440.64
1.20+1.70
92.33+0.75
23.9542.53
93.8340.84
19.94+1.16
BFQR 3.99+0.76
2.27+2.08
91.05+0.80
23.11+2.41
92.66+0.85
88.46+1.34
5.2.3
The Impact M
The number of bins, M, is the only primary parameter our method. Using synthetic data that ideally satisfies Assumption I evaluate how M influences EOC. In our experiment, M varies

---

among 11, 5, 10. 20, 50}. When bin size is 1, our method degenerates to GCQR. The results are shown in Fig. [ where the best EOC for BFQR and BFQR* achieved with 10 bins. When the number of bins is small, the sample size of Da,m increases. According to Theorem 4.1 the upper bound of the expectation of max group coverage rate gap decreased. However, this only suggests a decrease in the expectation, not for every Y within bins. When the quantile value is calculated on large samples, fails to characterize the conformity scores individuals. BFQR with continuous intervals exhibits better EOC compared to BFQR as its coverage gaps are primarily influenced by the quantile values the first and last bins, involving less randomness. As M greatly affects the performance of our methods, it should be cautiously chosen for various problem settings.
5.2.4 Comparisons of Optimization Methods
To demonstrate the efficiency of the proposed optimiza- tion method, we compare it with three recent optimiza- tion methods: 1) BFGS 47 48 a constrained non- smooth optimization method, 2) Augmented Lagrange (AL) 49.. another method to solve constrained non- smooth optimization problems, and 3) Bayesian optimiza- tion 50 511, a global optimization method for solving problems with noisy objective funetions. We evaluate these methods based on their performance terms of EOC, width, and running times. The results on synthetic data are in Table 4 The Bayesian method has extremely slow computational speed and cannot guarantee marginal cov- erage. In comparison to BFGS-SQP SQP and AL, our method achieves similar EOC performance but with slightly lower prediction interval widths, which confirms the effectiveness of our subgradient approximation. Additionally, our method is more efficient as it significantly reduces running times.
BFOR"
BFQR
30
Figure 3: Impact of M on EOC.
Table 4: Optimization results on synthetic data.
Optimzation Method
EOC
Running Time (seconds)
Width J
Mean Max
Marginal Coverage
T
Coverage Gap |
BFQS-SQP
4.86+0.75 4.81+0.73
3.89+3.71
90.08+0.16
06.15:00.06
78.11 1.8
AL
3.83+3.67
90.18+0.18
16. 16.066
77.57 .0
Bayesian BFQR
8.41+1.96
4.47+5.63
18.008
18.80+2.51
805.815152.05 33.62#14.78
3.82+0.45
3.6524.75
90.03+0.32
15.96#0.13
6 Discussion
EOC or equalized coverage? The results in Section 5.2 clearly demonstrate that perfect EOC and equalized coverage are mutually exclusive, empirically verifying Theorem 3.1 In real-world applications, becomes crucial trade off between EOC and equalized coverage. We recommend placing a higher emphasis on EOC when certain labels are considered more favorable, e.g., when individuals labeled as low-salary are the most important subpopulation for decision-make making. Under EOC, each group with the same label is treated equally, not only in terms of equal coverage rates (i.e., equal probability of being included in the prediction result) but also in terms of comparable prediction interval widths as a measure of uncertainty. For example, under equalized coverage, the coverage rate for men is around 0.65 but 0.55 for women in the first bin, low-income population. We cannot conclude that the model is more confident in the prediction results for low-income men solely based on the larger prediction intervals of men: this discrepancy might arise from the over-coverage of men. In this sense, our designed metric EOC, not only contributes to evaluating and enhancing fairness under uncertainty but also serves for fair uncertainty quantification, discovering model bias in uncertainty quantification. A possible application of EOC is for guiding sample selection in active learning (52 53 allowing the applied model to label the most uncertain samples in a fair manner.
Discretizition into equal-mass bins is favorable strategy to avoid excessively small sizes of Di m). While this guarantees tighter upper bound for the expectation of the maximum coverage gap within bins according to Theorem
method can be extended beyond equal-mass bins. When the

---

number of samples in each bin is unequal, we adjust the constraints in Eq. L by incorporating Bin M]| = 1 a. Consequently, in Algorithm the gradient in
weighted sum of 8m, i.e., _me[d]] the direction of each bin also should be weighted by [D(m)]. These adaptations allow our method to maintain its effectiveness when the sample sizes in each class are uneven, e.g., in imbalanced classification problems. Moreover, our method is able to address certain coverage rate concerns within particular bins, such as low-income females, by assigning specific coverage rates to the corresponding bins while allowing flexibility in adjusting the coverage rates of other bins.
Why Better EOC? Our methods have been empirically shown to have better EOC than methods designed for equalized coverage, and here we elucidate the reasons behind this improvement. Given the underlying distribution of the a-quantile conformity scores in group a conditioned on as Qa.(aa) For every y and a, calibrating with Qo.y(ar) would generate perfect EOC. However, due to the limitations of available calibration samples, the estimation of Qo.y(a) is subject to bias. Since the base model cannot perfectly capture the underlying distribution, the conformity score, as a measure of disagreement, must correlate with Y. While equalized coverage methods generally ignore this correlation, our method tries handle In Section 4.1 we used Ga.m (8m) to estimate Qa.y(8m) for all Bm, which is the fm -quantite value of conformity scores whose y belong to bin B,.- The localized estimation within discretized bins is upper-bounded by the largest fun-quantile value of score conditioned on any y and woundd by the smallest one, regardless of the unknown distribution of Y. Consequently, we could better characterize the conditional distribution within each bin. This enhancement is especially evident when the Lipschitiz constant L in Lemma 3.2 is small, such that the deviation from perfect EOC to the distribution generated by bins negligible.
Possible Extension. For future work, we aim to further improve our method by seeking a tighter bound achieve optimal EOC. Possible direction may follow [4 54). by discretizing Y into finite values and bounding deviation in EOC of the discretized predictor. Additionally, our conformalized method leverages statistical properties to ensure coverage rates in each bin and optimize prediction interval widths, making it particularly appropriate for large datasets. With better discretization strategy that better utilizes the calibration samples, e.g., ensemble sampling, our method may maintain its effectiveness on small calibration set.
Potential Exploration beyond the Current Scope. The context of this paper is situated in post- processing fairness. Though post-procent -sooperccssing methods typically underperform in-processing methods, in- in-processing methods are not applicable in many situations, e.g., the prediction model is a pre-trained black-box regression model, or flexible and computationally efficient method is required. Whereas, adapting the proposed method an in-processing setting to obtain better EOC is a possible extension. It would also be interesting to extend our method to the context that sensitive attributes are not available in the test data, possibly through missing data augmentation for the calibration set (55) or prdiction-powered inference [56].
Conclusion
In this paper, we introduce a new uncertainty-aware fairness notion, equal opportunity of coverage (EOC), which addresses the limitations of the seminal work of equalized coverage [T]] EOC has several desired properties: It guarantees equal coverage rates for groups with the same labels and marginal coverage rate at pre-determined level. It also ensures a small prediction interval. The theoretical analyses and empirical findings indicate that EOC and equalized coverage are generally incompatible. We suggest using EOC an alternative to equalized coverage when equal coverage rates and assessment of uncertainty are needed for more fine- ine-grained demographic groups To improve EOC, we propose distribution-fre free post-processing method, BFQR, based on discretization. Experimental results on synthetic and real-world datasets show that BFQR achieves competitive EOC and ensures guaranteed marginal coverage rates with small mean prediction interval widths compared to the state-of-thee-rt. Moreover, BFQR is adaptable to various settings, such as classification and other dacision-making tasks.
Acknowledgments and Disclosure of Funding
This work is supported part by NSF under grant 111-2106758. Lu Cheng part supported by the Cisco Research Gift Grant. We thank Xinhua Zhang for the helpful discussion on the optimization
10

---

method. We are grateful to the anonymous reviewers at NeurIPS 2023 for providing valuable feedback and suggestions.
References
[1] Osonde A Osoba and William Welser IV. An intelligence in our image: The risks of bias and errors in artificial intelligence. Rand Corporation, 2017.
[2] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of data and analytics, pages 254-264. Auerbach Publications, 2016.
[3] Ayanna Howard and Jason Borenstein. The ugly truth about ourselves and our rohot creations: the problem of bias and social inequity. Science and engineering ethics, 24:1521-1536, 2018.
[4] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163, 2017.
[5] Lu Cheng, Kush R Varshney, and Huan Liu. Socially responsible algorithms: Issues, purposes. and challenges. Journal of Artificial Intelligence Research, 71:1137-1181, 2021.
[6] Henry Peng Zou, Yue Zhou, Weizhi Zhang, and Cornelia Caragea. Decrisismb: Debiased semi-supervised learning for crisis tweet classification via memory bank. In Findings of the Association for Computational Linguisties: EMNLP 2023, 2023.
[7] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[8] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference machine learning, pages 1321-1330. PMLR, 2017.
[9] Henry Peng Zou and Comelia Caragea. Jointmatch: A unified approach for diverse and collaborative pseudo-labeling to semi-supervised text classification. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
[10] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
[11] Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel Candès. With malice toward none: Assessing uncertainty via equalized coverage. Harvard Data Science Review, 2(2):4, 2020.
[12] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
[13] Junpei Komiyama, Akiko Takeda, Junya Honda, and Hajime Shimao. Nonconvex optimization for regression with fairness constraints. Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2737-2746. PMLR, 10-15 Jul 2018.
[14] Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. Fair regression: Quantitative defini- tions and reduction- based algorithms. International Conference on Machine Learning, pages 120-129, PMLR, 2019.
[15] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning, pages 60-69. PMLR, 2018.
[16] Jianfeng Chi, Yuan Tian, Geoffrey Gordon, and Han Zhao. Understanding and mitigating accuracy disparity in regression. In International conference on machine learning, pages 1866-1876. PMLR, 2021.
[17] Shai Feldman, Stephen Bates, and Yaniv Romano. Improving conditional coverage via orthogo- nal quantile regression. Advances in Neural Information Processing Systems, 34:2060-2011 2021.
[18] Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Practical adversarial multivalid conformal prediction. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances Neural Information Processing Systems, volume 35, pages 29362-29373. Curran Associates, Inc., 2022.

---

[19] Meichen Liu, Lei Ding, Dengdeng Yu, Wulong Liu, Linglong Kong, and Bei Jiang. Confor- malized fairness via quantile regression. In S. Koyejo, S. Mohamed, Agarwal. D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 11561-11572. Curran Associates, Inc., 2022.
[20] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in random
world, volume 29. Springer, 2005.
[21] Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021.
[22] Pieter Delobelle, Paul Temple, Gilles Perrouin, Benoît Frénay, Patrick Heymans, and Bettina Berendt. Ethical adversaries: Towards mitigating unfairness with adversarial machine learning. ACM SIGKDD Explorations Newsletter, 23(1):32-44 2021.
[23] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340, 2018.
[24] Dana Yang. John Lafferty, and David Pollard. Fair quantile regression. arXiv preprint arXiv: 006666 2019.
[25] Robert Williamson and Aditya Menon. Faimess risk measures. In International Conference on Machine Learning, pages 9786-6797, PMLR, 2019.
[26] Carlos Pinzón, Catuscia Palamidessi, Pablo Piantanida, and Frank Valencia. On the incompati- bility of accuracy and equal opportunity. Machine Learning, pages 1-30, 2023,
[27]
Han Zhao and Geoffrey J Gordon. Inherent tradeoff's in learning fair representations. The Journal of Machine Learning Research, 23(1):2527-2552, 2022.
[28] Vladimir Vovk. Conditional validity of inductive conformal predictors. In Steven C. H. Hoi and Wray Buntine, editors, Proceedings of the Asian Conference on Machine Learning, volume 25 of Proceedings of Machine Learning Research, pages 475-490, Singapore Management University, Singapore, 04-06 Nov 2012. PMLR.
[29] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairess in machine learning. ACM Computing Surveys (CSUR), 54(6):1-35, 2021.
[30] Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv:/202.3775.. 2012.
[31] Veeranjaneyulu Sadhanala, Yu-X fu-Xiang Wang, Aaditya Ramdas, and Ryan Tibshirani. A higher- order kolmogorov-smirme test. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2621-2630. PMLR, 2019.
[32] Clément L Canonne, Ilias Diakonikolas, Daniel M Kane, and Alistair Stewart. Testing condi- tional independence of discrete distributions. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, pages 735-748, 2018.
[33] Matey Neykov, Sivaraman Balakrishnan, and Larry Wasserman. Minimax optimal conditional independence testing. The Anmals of Statistics, 40(44)2151-2177, 2021.
[34] Eric Wong, Leslie Rice, and J Zico Kolter. Fast better than free: Revisiting adversarial training. In International Conference Learning Representations, 2020.
[35] Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
[36] Henrik Boström and Ulf Johansson. Mondrian conformal regressors. In Conformal and Probabilistic Prediction and Applications, pages 1 14-133. PMLR, 2020.
[37] Mattco Fontana, Gianluca Zeni. and Simone Vantini. Conformal prediction: unified review of theory and new challenges. Bernoulli, 29(1):1-23, 2023.
[38] Bat-Sheva Einbinder, Yaniv Romano, Matteo Sesia, and Yanfei Zhou. Training uncertainty- aware classifiers with conformalized deep learning. In Advances in Neural Information Process- ing Systems, 2022.
[39] Torbjörn Larsson, Michael Patriksson, and Ann-Brith Strömberg. Conditional subgradient optimization- and applications. European Journal of Operational Research, 88(2)382- 403, 1996.
12

---

[40] Patrizia Berti and Pietro Rigo. A glivenko- theorem for exchangeable random variables. Statistics & Probability Letters, 32(():385-33) 1997.
[41] Stephen Boyd, Lin Xiao, and Almir Mutapeic. Subgradient methods. lecture noles of EE392o. Stanford University, Autumn Quarter, 2004:2004-2055, 2003.
[42] Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 33:3581-3591, 2020.
[43] Julia Angwin, Jeff Larson, Lauren Kirchner, and Surya Mattu, Machine bias, May 2016.
[44] Drago Plečko, Nicolas Bennett, and Nicolai Meinshausen. fairadapt: Causal reasoning for fair data pre-processing. arXiv preprint arXiv:2 I0. 10200, 2021.
[45] Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in neural information processing systems, 2021.
[46] Agency for Healthcare Research and Quality. Medical expenditure panel survey. https 1996.
[47] Buyun Liang, Tim Mitchell, and Ju Sun. Nevx: A general-purpose optimization solver for constrained machine and deep learning. arXiv preprini arXiv:2210.00977. 2022.
[48] Frank E Curtis, Tim Mitchell, and Michael L Overton. A bfgs-sqp method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles.
Optimization Methods and Software, (2(1):148-1) 2017.
[49] Yu Wang, Wotao Yin, and Jinshan Zeng. Global convergence of admm in nonconvex nonsmooth optimization. Journal of Scientific Computing, 78:29-63, 2019.
[50] Benjamin Letham, Brian Karrer, Guilherme Ottoni, and Eytan Bakshy. Constrained bayesian optimization with noisy experiments. Bayesian Analysis, 14(2):495-519, 2019.
[51] Yang Li, Yu Shen, Wentao Zhang, Yuanwei Chen, Huaijun Jiang, Mingchao Liu, Jiawei Jiang, Jinyang Gao, Wentao Wu, Zhi Yang, et al. Openbox: A generalized black-box optimization service. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 3209-3219, 2021.
[52] Yifan Fu, Xingquan Zhu, and Bin Li. A survey on instance selection for active learning. Knowledge and information systems, 35:249-288 2013.
[53] Hadis Anahideh, Abolfazl Asudeh, and Saravanan Thirumuruganathan. Fair active learning. Expert Systems with Applications, 199:116981, 2022.
[54] Exgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Fair regression via plug-in estimator and recalibration with statistical guarantees. Advances in Neural Information Processing Systems, 33:19137-19148 2020.
[55] Margaux Zaffran, Aymeric Dieuleveut, Julie Josse, and Yaniv Romano. Conformal prediction with missing values. arXiv preprint arXiv:2306.027333 020.0,,222222222322.2 2023.
[56] Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael Jordan, and Tijana inference. arXiv preprint arXiv:2301.06633 2023.
Zrnic. Prediction-
[57] Gábor J Székely, Maria I. Rizzo, and Nail Bakirov. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 35(6):2769, 2007.
[58] Gábor J Székely and Maria L Rizzo. Energy statistics: A class of statistics based on distances. Journal of statistical planning and inference, 1249-12229912 2013.
[59] Jing Lei. Max G'Sell, Alessandro Rinaldo, Ryan Tibshirani. and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Associ- ation, (195523):0094111 2018.
13


Reviews:
[{"review_id": "E3kVSixj5f", "reviewer": "Reviewer_219P", "summary": "This paper studies the problem of algorithmic fairness using uncertainty quantification methods in regression settings. The main novelty of this work over previous work is taking the notion of “equalized coverage” and strengthening it by considering finer groups by additionally conditioning on the target variable. The result is a conformal prediction based post-processing method that provably gives conditional coverage guarantees over sets defined by the demographic groups and target variable, where most prior work just considers sets on the group membership. Another upshot of this method is that the conformal-based post-processing method proposed (BFQR) explicitly focuses on also optimizing interval widths of predictions to improve “the amount of decision-making information available” to downstream decision-makers. \n\nThe paper is organized by presenting the main fairness notion studied - equal opportunity of coverage (EOC), describing the conformal prediction based post-processing approach to improve EOC and optimize for prediction interval widths. The main theoretical results are marginal and conditional coverage guarantees of BFQR. Finally, there is experimental evaluation on one synthetic and two benchmark datasets with comparisons to a few other conformal prediction based regression procedures. \n", "questions": "There are a few things that would be helpful to clarify in your experimental setup. What are the groups / conditioning events for the comparison experiments run with GCQR and MVP? My understanding is that the sets they give conditional guarantees for do not need need to condition on the true label - but can use them if given the information. If tested in this way, could it take some of the bite out of the improvement we see in EOC with BFQR / is that already how the experiments were run?\n", "limitations": "-", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper studies an important problem - giving conditional coverage guarantees, while conditioning on the true label of the data, using a general post-processing conformal method, with particular emphasis on minimizing interval width to give more utility to downstream decision makers. The presentation is clear and precise. ", "weaknesses": "A main upshot of BFQR as presented is the improvement on interval width, which seems to be modest on 2/3 (synthetic and Adult) datasets. It seems hard to compare this statistic with the other methods and evaluate how it compares to BFQR’s slight degradation in tightness of coverage rates.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation for your valuable feedback and suggestions. Regarding your concerns, we would like to offer further clarification.\n\n**Q1**: Modest improvement in interval width and degradation in coverage rates.\n\n**A1**: The research problem of the trade-off between interval width and coverage rate is significant. That is, with the same samples and calibration method, when we set a high coverage rate, the width of the prediction interval tends to increase. Therefore, our goal is to achieve EOC (the desired coverage rate for different demographic groups) meanwhile reduce the prediction intervals, not to maximize coverage rate. The width of the prediction interval can be also influenced by other factors: \n\ni) uncertainty of the sample: the hardness of predicting the distribution of outcomes for the base model significantly contributes to the lower boundary of the prediction interval width.\n\nii) proper quantification and calibration: if the uncertainty is not properly quantified and adjusted, we may not achieve the desired coverage rate or at a cost of wider intervals.\n\nFor *Adult* and *MEPS* data, our proposed BFQR shows a significantly lower interval width than baselines with similar coverage rates of around 0.9. BFQR* is a variation of BFQR that fills the noncontinuous space between prediction intervals of BFQR (Lines 270-272). For example, if BFQR gets intervals of [0,1], [4,5], and [10,15], then the output of BFQR* would be [0,15]. Enforcing continuity in BFQR* inevitably results in larger widths and a higher coverage rate than BFQR.\n\nFor synthetic data, the improvement of BFQR in width is modest compared with CQR, while significant compared with other methods. We believe it is because we utilize the result of CQR as the starting point, and the greedy strategy in optimization may lead to early stopping. However, as presented in Table 4, other optimization methods exhibit inferior results w.r.t. the width, running time, and other metrics. In future work, to get narrower widths, we will advance our method by improving the calibration process (factor ii).\n\n**Q2**: Unclear experimental settings.\n\n**A2**: We would like to further clarify our experimental settings. (1) No true label is given in the test stage and we only use true labels for evaluation. (2) GCQR, MVP, and CFQR are only conditioned on sensitive attributes, i.e., they focus on equalized coverage. BFQR aims to achieve EOC which is conditioned on both sensitive attributes and true labels."}, {"review_id": "TXldcH4rt3", "reviewer": "Reviewer_RXWV", "summary": "The authors propose a method to achieve same coverage rates between different groups conditioned on similar targets in addition to satisfying a marginal coverage rate on the entire population. The proposed objective is denoted as Equal Opportunity of Coverage (EOC) and the authors propose a two step approach denoted as Binned Fair Quantile Regression (BFQR) to address it. This method partitions the target space into bins and for each bin finds a coverage rate such that EOC is satisfied (i.e., equal coverage across sensitive groups inside the bin and global coverage rate level is achieved.). At test time the prediction intervals are built as the union of the intersections of each bin and its associated conformal prediction interval taking into account the group membership of the test sample. ", "questions": "If the sensitive attribute needs to be disclosed for each test sample, I think a reasonable comparison would be against a model that performs a group conditional split conformal approach that conditions on the sensitive attribute and a binning based on the outcomes of the quantile regression model. Is any of the competing approaches doing this?\n\nCFQR seem to be the method achieving the best Max coverage gap in Table 1 and Table 3 by a large margin. In particular, in Table 1 the max coverage gap is 0.19 but group A=1 has an overall coverage of 71.38 versus the other two groups whose coverage are larger than 93. I wonder if this is because across the majority of bins group A=1 is the one with the worse coverage even though by a small amount per bin (i.e., less than 0.19). How many bins where considered to the synthetic example?  I think to achieve that large difference between group coverages with small coverage gaps there has to be many bins.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "I think the problem presented in this work is well motivated. Figure 1 is a nice graphical example to motivate the importance of considering EOC over Equalized coverage.\n\nThe definition of EOC is clear and as far as I know it was not discussed in other works. Its associated theory is well linked to previous works on Equal Opportunity.\n\nEmpirically, the proposed BFQR method is improving the maximum coverage gap between groups across the bins when compared to almost all the competing approaches. It also seems to achieve similar group coverage (average) across the groups and maintaining smaller interval widths.", "weaknesses": "I think the clarity of the presentation should be improved. In particular, I found that the description of the proposed procedure in Section 4 needs better clarification. For instance, L161 to L164 mentions that to enforce EOC the coverage rate within each bin $\\beta_m$ is the same across groups. The method to achieve this in practice is not clear to me when reading the main manuscript. Paragraph in L116 discusses that TV distance w.r.t. the closest EOC distribution (p’) is measured. However, it is not clear enough how p’ is obtained and how this distance is obtained. I understand the authors are citing the works from [39,40] where an ‘easy-to'-compute statistic T’ is being used but this is an important point in the proposed method and should be clarified here for reproducibility purposes.\n\nI found that Algorithm 1 in Appendix is a better summary of the description provided in last part of Section 4.1 and 4.2. However, this Algorithm builds on top of the previously estimated bin coverage rate $\\beta_m$.\n\nI think the organization of the paper is a bit convoluted since Section 4.3 describes how to obtain $\\beta_m$ but this was already discussed in Section 4.1 and Section 3. Moreover, Section 4.1 and Section 4.2 were relying on this. I think that the description on how $\\beta_m$ is obtained could be improved, it could be made more concise and informative.\n\nFigure 1 provides a nice and intuitive illustration of the problem, it would be nice to see the same plot after the proposed post processing procedure is applied in comparison with the other competing methods.\n\nFrom Algorithm 1 in the Appendix and L172-173 it seems that during test time the sensitive attribute needs to be available. I can think of this as a weak point in the fairness setting since the sensitive attribute needs to be available at test time. However, this is also the case for other existing fairness post-processing approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable and informative suggestions and would clarify the questions and concerns in the following.\n\n**Q1**: Unclear writing in Section 4.\n\n**A1**: We will rephrase this paragraph and add a new Figure 3 illustrating how $\\beta_m$ is enforced to be equal within the same bin (see attached pdf in the global response). While our method relies on the standard exchangeability assumption (Assumption 1) in conformal prediction, the distribution of conformity scores can be different across different groups within the same bin $m$.\nTherefore, to guarantee coverage rates for all groups in bin $m$ to be the same ($\\beta_m$), we calibrate the test data with the $\\beta_m$-th quantile of conformity scores calculated separately within each group. \n\nHere is a formal definition of $p'$ in L116: We denote $P_{EOC}$ as the property of EOC, the set of all distributions in defined space that satisfy EOC, i.e., $P_{EOC}:= $\\{ $p \\in \\Delta(\\mathcal{V} \\times \\mathcal{A} \\times \\mathcal{Y}): \\text{if}  (V, A, Y) \\in p, V⫫A | Y $ \\}. $p' \\in P_{EOC}$ is the distribution with minimum TV distance to $p$, i.e., for all distributions $\\forall q \\in P_{EOC}$, $d_{TV}(p, p') \\leq d_{TV}(p, q)$. We will add the definition to the next version of this work.\n\nTo avoid introducing complex definitions and notations, we intuitively introduce $T$ as the sum of the independence measure with bins in the paper. The details of the computation of $T$ can be found in Appendix 8.1.1, where we follow Section 4.1 in the referred paper [30]. Detailed implementation of codes is also available in Line 61. We will also add a more elaborate description to the next version of this work.\n\n**Q2**: Section 4 can be better organized.\n\n**A2**: Section 4 is organized as follows: we first enforce the independence of $A$ and $V$ within each interval of $Y$ (i.e., EOC) for the calibration data in Section 4.1, and then leverage conformal prediction to achieve EOC for test data in Section 4.2. Lastly in Section 4.3, we describe an efficient and robust optimization approach that optimizes both EOC and widths of prediction intervals. We organize Section 4.3 behind Sections 4.1 and 4.2 because, in Section 4, the most important aspect of the proposed method is its ability to improve EOC (Sections 4.1 and 4.2). On this premise, we then expect tighter intervals (Section 4.3). We concur that it is natural to elaborate on how to improve $\\beta_m$ right after introducing it in Section 4.1, but it may arise other difficulties, e.g., the objective function in optimization is derived from Section 4.2. \n\n**Q3**: Add plots for competing methods.\n\n**A3**: Thanks for the suggestion. We added Figure 4 to clearly present the comparison results in the pdf attached to the global response. We will add this figure to the next version of this draft.\n\n**Q4**: Missing sensitive attributes.\n\n**A4**: As one of the first few works studying equalized coverage, this work as well as the baselines adopts conventional settings in fair machine learning, which assumes the availability of sensitive attributes in the test phase.\nFor practical concerns, it is surely a promising future direction to adapt our method to handle missing sensitive attributes. We will discuss this limitation in the Discussion section.\n\n**Q5**: Comparison with the conditional split conformal inference approach.\n\n**A5**: We compared our method with Group-conditional Conformalized Quantile Regression (GCQR) in the paper, which is a split conformal prediction method that aims to reach equal coverage rates across groups. Experimental results show that our method leads to better EOC than GCQR (see Tables 1-3).\n\n**Q6**: Small max coverage gap of CFQR.\n\n**A6**: We believe that CFQR has better performance related to data distribution. The Column *Max Coverage Gap* in Tables 1 to 4 is defined as the average maximum difference in coverage rates between groups for all bins in L251. For Adult data, the challenging group (A=1) mainly falls in the first few bins. CFQR only has a few bins showing large coverage gaps and the gaps in other bins are extremely small, leading to a small average. For example, when we set random seed as $1$ and run CFQR, each group has a coverage rate of 0, 0.6, 1.2 in bin 1 and 98.28, 99.15, 99.45 in bin 2, while the other 8 bins either are fully covered or have no samples. Therefore, the mean of max coverage gaps is 0.24 while group A=1 has a conditional coverage rate of  74.97, and groups A=0 and A=2 have 92.52 and 94.86, respectively. We observe a similar phenomenon for *MEPS* data in Table 3, where CFQR has a large gap in the first bin and small gaps for other bins. We will explore more advanced binning strategies in future works.\n"}, {"review_id": "qhuySelUL5", "reviewer": "Reviewer_1ZNj", "summary": "This paper studies fair machine learning under predictive uncertainty. \nThey propose a new notion of fairness named Equal Opportunity of Coverage (EOC), building upon \"equalized coverage.\" While EC ensures that every demographic group receives the same level of prediction certainty, EOC ensures that every demographic group with similar true labels receives the same level of prediction certainty. \nThis extension is similar to the extension of equal opportunity from statistical parity. \nWith EOC, they also ensure that the entire population's coverage can be maintained at a predetermined level. \nTo achieve EOC, they propose a post-processing method called Binned Fair Quantile Regression (BFQR), a distribution-free post-processing method for a given trained model. \n\nThey show the power of their method both on the simulated data and real-world datasets.\n", "questions": "### Major Comments:\n\n1) Line 83: Why are the calibration and test sets exchangeable?\n2) Line 111: How does this indicate \"V \\perp A\"?\n3) TV definition in Line 122 needs to be more rigorous. Please define what $p$ and $p'$, which space they live, etc. \n4) Up to Line 135, I need clarification on $p$. Is the true data-generating distribution, or is it the empirical distribution? \n5) In Lemma 3.2, what is the sum of independence measures? What is the exact reference of Lemma 3.2 in [30]?\n6) Why do you choose to use TV? (I understand why not KS distance, but I do not understand why you choose TV over other metrics. An example could be Wasserstein distance, MMD, KL divergence etc.)\n7) Line 164: The coverage number $\\beta_m$ is the same for each group, right? It is different across subgroups. Then, I do not understand why $\\beta$ is computed separately for each group. Isn't it computed separately for each bin? I do not understand that part.\n8) The difference between BFQR and BFQR* is not discussed in the main text. What are the differences between the two? \n\n9) What are the conditions that Assumption 1 is violated? How strong is this assumption?\n\n10) Is it possible to include EOC constraints somehow during training?\nDo you think it would be a stronger method if this method could be an in-processing method?\n\n\n\n### Minor Comments:\n\n1) Typo Line 85: \"Pr(\" instead of \"P(\".\n2) In (1), $min$ and $s.t.$ should be aligned, and \"s.t.\" should be a math operator. \n3) Please introduce each object rigorously. It is unclear to me what sort of mathematical objects you are referring to until I see them used. For example, quantile regression. What is $\\hat q$? Is this a function? Is it a vector? What is $\\alpha_{lo}$? This would help the readers a lot if you were to introduce each object appearing in the paper rigorously. \n4) Line 252: \"l\" should be capitalized in lemma\n5) Line 312:  \"t\" should be capitalized in theorem\n6) Please indicate the best results in the experiment with bold font. \n7) Line 224: \"W\" should not be capitalized.\n8) Line 360: \"\\beta_m\" instead of $\\beta m$\n9) Line 272: \"Section\" instead of \"Sec.\" (also 290). \n10) Line 311: \"Figure\" instead of \"Fig.\" \n11) What is the probability distribution $\\mathbb P$ appeared suddenly in Theorem 4.1 and Theorem 4.2? \nI do not think the notation of this paper is unified. Please unify the notation during the rebuttal. Defining notation rigorously and sticking with it throughout the paper is important. Otherwise, the paper becomes unnecessarily hard to follow.\n12) I think it would be safe to drop the \"under no label distribution shift on test data\" assumption from Theorem 4.2. In the beginning, if you were to define the true data-generating distribution and explain how the test data and the calibration data are created from iid samples from this true data-generating distribution, then there is no need to mention this shift. If you could cover the shifted distribution case, then I would suggest mentioning it. \n", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This is a very interesting research direction. \n\nFair regression is also a challenging problem, and little research extends equal opportunity (and its variants) to regression settings. \nThis is a promising research direction. \n", "weaknesses": "I have a few apprehensions regarding the notation and the inadequate introduction of certain mathematical objects. Please see my comments below. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful and constructive suggestions which help enhance the clarity and accessibility of this paper. Below are our replies to your questions/comments, and we will revise our ambiguous expressions to avoid confusion and misunderstanding.\n\n1. The exchangeability of calibration and test sets is the fundamental assumption of classic conformal prediction to guarantee its coverage.\n\n2. Symbol '⫫' means independent of in our paper.\n\n3. \nDefine distribution $p, p' \\in \\Delta(\\mathcal{V} \\times \\mathcal{A} \\times \\mathcal{Y})$, and $\\mathcal{V}$, $\\mathcal{A}$, $\\mathcal{Y}$ is the domain of $V$, $A$, $Y$ respectively, as shown in the preliminary. \nThe total variation distance between $p$ and $p'$ is defined as $d_{TV}(p,p') := \\sup_{S \\subseteq \\Delta(\\mathcal{V} \\times \\mathcal{A} \\times \\mathcal{Y})} $\\{$ p(S) - p'(S)\\$\\}$ = \\frac{1}{2} \\sum_{x \\in \\Delta(\\mathcal{V} \\times \\mathcal{A} \\times \\mathcal{Y})} |p(x) - p'(x)|$.\nWe further denote $P_{EOC}$ as the property of EOC, the set of all distributions in defined space that satisfy EOC, i.e., $P_{EOC}:= $\\{$p \\in \\Delta(\\mathcal{V} \\times \\mathcal{A} \\times \\mathcal{Y}): if (V, A, Y) \\in p, V ⫫ A | Y $\\}. Fix distribution $p' \\in P_{EOC}$ as of minimum TV distance to $p$, i.e., for all distributions $\\forall q \\in P_{EOC}$, $d_{TV}(p, p') \\leq d_{TV}(p, q)$. \n\n4. $p$ is the true data-generating distribution.\n\n5. The details of the independence measure can be found in Appendix 8.1.1. We are inspired by Theorem 5.2 in the referred paper [30] and simplify its notation.\n\n6. We follow referred works [29,30] in the paper using TV distance as it is a commonly used metric for measuring how far two variables are from being conditionally independent. TV distance is a more appropriate metric in this work as it is symmetric compared with KL divergence and easier to compute than Wasserstein distance. Since MMD is also widely used for conditional independence tests [1], we conjecture that it would produce similar evaluation results.\n\n7. For better understanding, we illustrate the process described in Lines 161 -171 with Figure 3 in the attached pdf file for the global response. We will include it in the next version of this work.\n\n8. We apologize for the confusion. The difference between BFQR and BFQR* is described in the experiment setup section (Lines 270-272). BFQR* is the convex hull of intervals in BFQR (Line 230), disabling disjoint intervals. This variant of BFQR is meaningful in some real-world settings. For example, for MEPS data, predicting a healthcare system utilization score to be in [0,1], [4,5], or [10,15] is confusing. Although the total interval width is small, survey researchers cannot decide whether the score is high or low. In this case, a continuous [0,15] with a wider interval makes more sense.\n\n9. Assumption 1 assumes the exchangeability of calibration and test data with the same bin and is a relaxation of the i.i.d. assumption. It does not impose assumptions on data distribution and base models. It is violated when there is a distribution shift of conformity scores within a bin from calibration to test data. Relaxing the exchangeability assumption is important, however, it needs more advanced techniques which are beyond the scope of this work. We will explore potential solutions (e.g., sample reweighting) in future works.\n\n10. We agree that post-processing methods typically underperform in-processing methods. However, there are many situations where in-processing methods are not applicable, e.g., the prediction model is a pretrained black-box regression model or a flexible and computationally efficient method is required. Moreover, conformal prediction itself is a post-hoc uncertainty quantification approach. Therefore, it is a natural application to post-processing methods. Adapting our methods to an in-processing setting will be considered in future works. \n\nWe have carefully reviewed your minor comments and will revise all the typos and format problems. We clarify the other ambiguities that require further explanations (numbered in correspondence with your comments):\n\n3. Let $\\\\hat{q}_{\\\\alpha}$ denote the $\\\\alpha$-th conditional quantile regression function, i.e., for $i$-th sample $(X_i, Y_i)$, (here are some unknown errors with Markdown)\n\n$\\hat{q}_{\\alpha}(X_i) := \\inf $\\{$ y\\in \\Delta \\mathcal{Y}: Pr$\\{$Y_i \\leq y | X = X_i $\\}$ \\geq \\alpha $\\}. \n\nFix the lower and upper quantiles as $\\alpha_{lo} = \\alpha/2$ and $\\alpha_{hi} = 1 - \\alpha/2$, then $\\\\hat{q}_{\\\\alpha_{lo}}(X_i)$ \nand $\\\\hat{q}_{\\\\alpha_{hi}}(X_i)$ denote lower and upper quantile regression functions, respectively.\n\n11. Thank you for the reminder about the notion inconsistency problem. We will use $Pr$ in this paper.\n\n12. We remove the \"under no label distribution shift on test data\" assumption. Thanks for pointing it out!\n\n[1] Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., & Smola, A. (2012). A kernel two-sample test. The Journal of Machine Learning Research, 13(1), 723-773.\n"}, {"review_id": "92IfQfAtnN", "reviewer": "Reviewer_FNuJ", "summary": "Authors propose a new uncertainty-aware fairness – Equal Opportunity of Coverage (EOC) since current definition do not guarantee equal coverage rates across more fine-grained groups (e.g., low-income females) conditioning on the true label and is biased in the assessment of uncertainty.", "questions": "The paper is, in my opinion, technically sound with promisin gresults.\nNevertheless I think that the approach is not completely novel.\nMany works in the literature of fairness focus on distribution matching (see also NeurIPS papers when they use MMD, Sinkhorn distance, or first order matching) and there in no discussion on the relation with these works and a comparison with the idea of this work.\nAuthors should clarify and elaborate on this.", "limitations": "Authors should better clarify the novelty of the work and its impact.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- paper is technically sound \n- experimental results are promising", "weaknesses": "- presentation is mostly verbose and unclear\n- novelty is limited ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestions. We study fair regression under uncertainty in this paper. Our major contributions are summarized as follows: \n\nWe identified several important limitations in prior works: equal coverage is not guaranteed when conditioned on different values of $Y$ and; interval width is largely increased (i.e., low prediction confidence) when ensuring equal coverage. \n\nTo tackle these limitations, (1) we proposed a novel uncertainty-aware fairness notion, EOC. As mentioned by Reviewer RXWV, the definition of EOC (a variant of equal opportunity) was not discussed in any other works, and it introduced a fresh perspective to the field. (2) We also proposed a novel effective and efficient method that can improve EOC meanwhile reduce interval width. According to Reviewer 1ZNj, achieving equal opportunity in regression is less researched but is an important problem. Our work reflects our efforts to tackle challenges in this setting.\n\nRegarding the concerns about using distribution matching techniques, we need to clarify that distribution matching is only used for evaluation, and should not be considered as a contribution of this work.\nOur proposed post-processing method is distinct from in-processing fairness methods, where distribution matching is commonly employed. While adapting our method to in-processing settings is very interesting, it is required more advanced techniques which is beyond the scope of this work. Therefore, we did not compare our method with models using a distribution matching framework."}]

Author rebuttal:
We sincerely appreciate all the recognition and suggestions from all reviewers, which greatly contribute to enhancing both the quality and clarity of our paper. We have thoughtfully addressed several common concerns raised by the reviewers and would incorporate them into the next version of our paper:

1. To illustrate the computation of $\beta_m$ and the corresponding quantile values in Lines 161-171, we have added Figure 3 to the attached PDF, along with a detailed explanation.

2. In an effort to provide a clear understanding of our work, we have included rigorous definitions for the key concepts, such as $p$, $p'$, and $d_{TV}(p,p')$ in Lines 116-122.

3. Recognizing the importance of clarity in our presentation of experiments, we have further elaborated on the experimental settings and results in our responses. Additionally, we have introduced Figure 4 in the attached PDF to offer enhanced visual comparisons between the different methods.

Meta-review: The paper proposes an equal opportunity analogue of "equalized coverage", which they call "equal opportunity of coverage". It is a two-fold objective that tries to have parity for coverage rates for different groups with similar outcomes and the overall coverage rate for the entire population is lower bounded. The paper proposes Binned Fair Quantile Regression, a post-processing method to improve EOC which is agnostic to the model based on calibrating on a held-out set and uses conformal prediction and constrained optimization to optimize for the bin width.

The reviewers agreed the paper presents some nice and well-motivated ideas with promising results. One reviewer raised concerns around the novelty referencing previous works related to distribution matching post-processing techniques in fair regression; however it appears they are not directly applicable here since the present work involves conformal interval prediction. During discussions, the only concern raised was that the proposed method requires some trade-offs in some metrics such as the original equalized coverage. I believe this is reasonable as EOC is a different goal from equalized coverage and other metrics; moreover, the paper highlights this and provides practical insights around this. Overall, this paper is a worthwhile contribution.
Decision: Accept (poster)