SlowGuess/NeurIPS_2023_OCR huggingface dataset:
An easy solution to the optimization problem Eq. 1 can be obtained by adjusting  $\hat{C}(X_i)$ ,  $X_i \in \mathbb{D}_c$  with split conformal prediction described in Section 2. However, it is not optimal since  $\beta_m$  is determined by  $\hat{Q}_{1 - \alpha}(S, \mathbb{D}_c)$  calculated on all calibration data, but different bins have varying costs of width associated with changes in their coverage rates. For instance, if a bin has a coverage rate of 0.98, increasing it to 0.99 would lead to a significant increase in its width, whereas increasing the coverage rate of a bin from 0.50 to 0.51 would result in only a minor increase in width. Therefore, we use the solution of split conformal prediction as the initialization and then optimize it.
Solving Eq.  $\boxed{1}$  is challenging in that it is computationally expensive or even infeasible to compute the value and the gradient of the objective function. First, the computation of prediction interval  $C(X_{i})$  involves multiple intersection and union operations, which is a complex step function of  $\beta_{m}$ . Second, prediction interval  $C(X_{i})$  is related to quantile value  $G_{a,m}$ , which is estimated from data with noise. Directly using slopes of  $G_{a,m}$  as the gradient methods could result in over-fitting to noise, as shown in Section 5.2.4. Third, We cannot assume the objective function's convexity or differentiability as the data may come from any possible distribution, and sorting is involved in calculating  $G_{a,m}$  [38]. To address those obstacles, we propose an efficient and robust optimization algorithm (detailed in Algorithm 2 in Appendix 8.3.2) that utilizes a relaxed upper bound and optimizes through approximated subgradients [39].
The steps of our approach are described below. First, to accelerate computing, we use a dummy continuous prediction interval  $C^d(X_i) = \text{Convex}(\bigcup_{m} C_m(X_i))$ , i.e., the convex hull of all sub-intervals, as an upper bound to substitute the original prediction interval  $C(X_i)$  in the objective function. However,  $C^d(X_i)$  is still related to noisy  $G_{a,m}$ . To address this, in each iteration, we compute the slope for each bin  $m$  in increasing and decreasing directions, denoted as  $\hat{t}_m^+$  and  $\hat{t}_m^-$ , respectively. Since the objective function aims to decrease without changing the marginal coverage rate, we take a greedy strategy by moving up a step  $\eta$  in the bin with the steepest descendant direction  $\max_m \{\hat{t}_m^-\}$ , meanwhile taking a step  $\eta$  down the slowest descendant direction  $\min_m \{\hat{t}_m^+\}$ . We stop until  $\max_m \{\hat{t}_m^+\} \geq \min_m \{\hat{t}_m^-\} + 2\varepsilon$ , where  $\varepsilon$  is an appropriate estimation error bound related with  $|\mathbb{D}_c|$  [40, 41]. More details can be found in Appendix 8.3.1.
The proposed method can be viewed as a subgradient method, incorporating considerations for quantile value estimation errors and maintaining a constant mean coverage rate. Combining the three steps, we could get prediction intervals with improved EOC, guaranteed marginal coverage, and decreased average width of prediction intervals.
In this section, we conduct three sets of experiments on both synthetic and real-world data to evaluate (1) the effectiveness of the proposed approach for achieving EOC (Section 5.2); (2) the impact of the
key parameter  $M$  ,the number of bins (Section 5.2.3); and (3) the effectiveness and efficiency of the proposed optimization framework (Section 5.2.4).
We propose two metrics to evaluate the first property of EOC, i.e., whether coverage rates for different groups with similar outcomes are close. With the  $M$  discretized bins in Section 4.1, the first metric is the average of maximum difference in coverage rates between groups for all bins, similar to the definition of conditional KS distance in [12]. In main experiments,  $M$  is set as 20. The second metric  $T$  is introduced in Lemma 3.2 where  $T$  is calculated on  $d = \lceil |\mathbb{D}_t|^{2/5}\rceil$  bins. Specifically,  $d \neq M$  for fairness of evaluation, e.g.,  $d = 100$  for synthetic data. A smaller  $T$  implies a closer distance to the ideal EOC distribution and, therefore a better EOC. Furthermore, for the second property of EOC, we need to ensure the desired marginal coverage rates. The efficiency of uncertainty estimation is measured by the width of the prediction interval. In addition, we check the conditional coverage rates on groups to measure equalized coverage. All metrics related to coverage rates are multiplied by 100 in tables to exhibit significant differences. Every experiment is repeated 100 times on random divisions of data with different seeds, with  $|\mathbb{D}_{tr}|:|\mathbb{D}_c|:|\mathbb{D}_t| = 3:1:1$ .
We compare our method with the following state-of-the-art methods: 1) Split Conformalized Quantile Regression (CQR) [35, 21] with only marginal coverage guarantee; 2) Group-conditional Conformalized Quantile Regression (GCQR) [11] with both marginal and conditional coverage guarantee; 3) MultiValid Predictor (MVP) [18] with both marginal and conditional coverage guarantee. Note that the base model for this algorithm is trained on the union of training and calibration data as MVP does not require any calibration data; 4) Conformal Fair Quantile Prediction (CFQR) [19], which guarantees marginal coverage and demographic parity on both upper and lower bounds of the predicted intervals; 5) Label-conditional Conformalized Quantile Regression (LCQR) [42], which is designed for classification problems to provide marginal coverage and equalized coverage for each class. We adapt it to regression tasks. The base model for all compared conformal prediction methods is set as the QR model at the level of 0.05 and 0.95, and the desired marginal coverage is set to 0.9. Considering that for some real-world applications like scoring [43], disjoint prediction intervals make little sense, we evaluate prediction intervals  $C(X)$  along with their dummy prediction intervals  $C^d(X)$  in Section 4.3, represented as BFQR and BFQR*. In the comparison tables, the best results and the second-best results are highlighted in bold and underlined, and uncovered groups who fail to reject the null hypothesis at 0.05 level in one-tailed t-tests are emphasized in Italian.
5.2.1 Synthetic data
We generate ten independent and exponentially distributed features with the scale of 1,  $X = (X_{1}, \ldots, X_{10})$ ; protected attribute  $A$  is randomly selected from  $\{0, 1, 2\}$  with a probability of 0.1, 0.2, 0.7, respectively. The labels  $Y$  for  $A = 1$  follow a random distribution, thus impossible to predict; labels  $Y$  for the other two groups are the linear summations of  $X$  and  $A$ , plus noises that increase with  $Y$ . A size of 100,000 samples are generated from this distribution and the data generating process is detailed in Appendix [8.4.1].
We have the following observations from the results in Table I: 1) All methods have marginal coverage guarantee, which is attributed to statistical guarantee from conformal prediction. 2) Our proposed methods, BFQR with disjoint intervals and BFQR* with joint intervals achieve the best trade-off between EOC and equalized coverage. In particular, our method achieves the second-best EOC (i.e., Mean Max Coverage Gap and  $T$ ), meanwhile, the conditional coverage rates are almost equal across different groups. While CFQR has significantly better performance w.r.t. EOC, the conditional coverage for  $A = 1$  (the most challenging case) is extremely low compared to  $A = 0$  and  $A = 2$ . This result aligns with the mutual exclusivity between EOC and equalized coverage formulated in Theorem 3.1. 3) Among all methods, the average interval width of our method is the smallest, validating the effectiveness of the optimization process in Section 4.3. One of the main advantages of BFQR and BFQR* is optimizing through bin coverages. In this process, bins that sacrifice interval width for an over-coverage rate are adjusted to a lower but satisfactory coverage rate. Therefore, we are able to guarantee a smaller average interval width while improving the EOC.
Table 1: Experiments results for synthetic data.
Method	EOC	Width ↓	Equalized Coverage
Mean Max Coverage Gap ↓	T ↓	Marginal Coverage	Coverage (A = 0)	Coverage (A = 1)	Coverage (A = 2)
CQR	20.11±1.05	306.33±18.78	89.99±0.32	16.02±0.04	82.84±0.90	90.03±0.76	91.00±0.29
GCQR	14.48±1.36	386.81±35.60	90.00±0.33	16.21±0.06	90.08±0.99	89.92±0.64	90.01±0.37
MVP	9.03±1.43	71.27±6.70	89.98±0.32	16.31±0.13	89.35±0.94	90.11±0.54	89.96±0.24
CFQR	0.19±0.09	0.09±0.24	89.93±0.20	17.00±0.06	93.89±0.53	71.38±0.87	94.74±0.24
LCQR	6.38±0.26	18.64±6.71	90.41±0.29	17.32±0.09	91.37±0.84	90.55±0.68	90.23±0.37
BFQR*	3.15±0.58	2.78±3.79	91.74±1.28	16.24±0.21	93.81±1.03	91.19±3.13	91.60±0.90
BFQR	3.82±0.45	3.65±4.75	90.03±0.32	15.96±0.13	91.99±1.16	89.08±3.61	90.03±1.04
5.2.2 Real-world Data
We further evaluate our method on two benchmark datasets: Adult [44, 45] where gender is the protected attribute and the outcome is salary; MEPS (Medical Expenditure Panel Survey) data [46, 11] where race is the protected attribute and the outcome is the health care system utilization score.
We observe similar results: For Adult data (shown in Table2), all methods achieve marginal coverage. Our methods achieve the best EOC and smallest mean width of prediction interval while maintaining competitive conditional coverage rates. CFQR with the best EOC on synthetic data does not have consistently good performance, and the mean interval width is greatly larger compared with other methods. The increased width of LCQR implies that as some bins are difficult to predict, enforcing all bins to reach the same high coverage rates generates prediction results with little useful information. For MEPS data (shown in Table3), the results are similar to those in synthetic data: our method is the second-best w.r.t. EOC, with marginal coverage rates guaranteed and a significantly smaller average prediction width.
Table 2: Experiments results for Adult data.
Method	EOC	Width ↓	Equalized Coverage
Mean Max Coverage Gap ↓	T ↓	Marginal Coverage	Coverage (Men)	Coverage (Women)
CQR	5.01±0.40	67.31±10.56	89.98±0.29	93,951.11±409.25	90.01±0.42	89.94±0.37
GCQR	4.97±0.50	67.30±10.63	89.99±0.29	93,994.10±435.35	89.98±0.41	89.99±0.46
MVP	5.50±0.34	211.01±19.44	90.05±0.28	98,229.81±1,723.82	90.07±0.13	90.08±0.27
CFQR	3.57±0.53	18.17±6.02	90.08±0.14	147,253.27±499.47	90.02±0.42	90.08±0.40
LCQR	3.91±0.45	5.03±4.42	90.50±0.30	160,107.96±8929.07	90.46±0.37	90.54±0.46
BFQR*	2.90±0.37	3.60±3.29	91.08±0.46	93,689.33±976.76	91.89±0.74	90.11±0.43
BFQR	3.05±0.35	3.55±3.14	90.32±0.28	91,969.66±996.63	90.97±0.42	89.56±0.47
Table 3: Experiments results for MEPS data.
Method	EOC	Width ↓	Equalized Coverage
Mean Max Coverage Gap ↓	T ↓	Marginal Coverage	Coverage (White)	Coverage (Non-white)
CQR	6.47±1.22	7.62±4.28	89.85±0.77	32.06±1.85	89.81±0.99	89.90±0.88
GCQR	6.54±1.79	9.50±8.35	89.91±0.74	32.08±1.90	89.91±0.95	89.93±1.11
MVP	8.27±2.37	8.78±3.47	89.95±0.84	41.08±6.87	90.97±0.71	90.75±0.97
CFQR	0.94±0.39	0.28±0.55	90.88±0.51	36.55±1.85	87.93±1.06	93.18±0.78
LCQR	5.29±0.97	3.29±2.52	91.97±0.69	160.65±30.71	91.59±0.94	92.57±1.02
BFQR*	3.04±0.64	1.20±1.70	92.33±0.75	23.95±2.53	93.83±0.84	89.94±1.16
BFQR	3.99±0.76	2.27±2.08	91.05±0.80	23.11±2.41	92.66±0.85	88.46±1.34
5.2.3 The Impact of  $M$
The number of bins,  $M$ , is the only primary parameter in our method. Using synthetic data that ideally satisfies Assumption  $\square$ , we evaluate how  $M$  influences EOC. In our experiment,  $M$  varies
among  $\{1, 5, 10, 20, 50\}$ . When bin size is 1, our method degenerates to GCQR. The results are shown in Fig. 3 where the best EOC for BFQR and BFQR* is achieved with 10 bins. When the number of bins is small, the sample size of  $\mathbb{D}_{a,m}$  increases. According to Theorem 4.1 the upper bound of the expectation of max group coverage rate gap is decreased. However, this only suggests a decrease in the expectation, not for every  $Y$  within bins. When the quantile value is calculated on large samples, it fails to characterize the conformity scores of individuals. BFQR* with continuous intervals exhibits better EOC compared to BFQR as its coverage gaps are primarily influenced by the quantile values of the first and last bins, involving less randomness. As  $M$  greatly affects the performance of our methods, it should be cautiously chosen for various problem settings.
5.2.4 Comparisons of Optimization Methods
To demonstrate the efficiency of the proposed optimization method, we compare it with three recent optimization methods: 1) BFGS-SQP [47, 48], a constrained nonsmooth optimization method, 2) Augmented Lagrange (AL) [49], another method to solve constrained nonsmooth optimization problems, and 3) Bayesian optimization [50, 51], a global optimization method for solving problems with noisy objective functions. We evaluate these methods based on their performance in terms of EOC, width, and running times. The results on synthetic data are in Table 4. The Bayesian method has extremely slow computational speed and cannot guarantee marginal coverage. In comparison to BFGS-SQP and AL, our method
Figure 3: Impact of  $M$  on EOC.
achieves similar EOC performance but with slightly lower prediction interval widths, which confirms the effectiveness of our subgradient approximation. Additionally, our method is more efficient as it significantly reduces running times.
Table 4: Optimization results on synthetic data.
Optimization Method	EOC	Width ↓	Running Time (seconds)↓
Mean Max Coverage Gap ↓	T ↓	Marginal Coverage
BFQS-SQP	4.86±0.75	3.89±3.71	90.08±0.16	16.15±0.06	78.11±41.98
AL	4.81±0.73	3.83±3.67	90.18±0.18	16.16±0.06	77.57±1.50
Bayesian	8.41±1.96	4.47±5.63	88.18±10.08	18.80±2.51	805.81±152.05
BFQR	3.82±0.45	3.65±4.75	90.03±0.32	15.96±0.13	33.62±14.78
EOC or equalized coverage? The results in Section 5.2 clearly demonstrate that perfect EOC and equalized coverage are mutually exclusive, empirically verifying Theorem 3.1. In real-world applications, it becomes crucial to trade off between EOC and equalized coverage. We recommend placing a higher emphasis on EOC when certain labels are considered more favorable, e.g., when individuals labeled as low-salary are the most important subpopulation for decision-making. Under EOC, each group with the same label is treated equally, not only in terms of equal coverage rates (i.e., equal probability of being included in the prediction result) but also in terms of comparable prediction interval widths as a measure of uncertainty. For example, under equalized coverage, the coverage rate for men is around 0.65 but 0.55 for women in the first bin, low-income population. We cannot conclude that the model is more confident in the prediction results for low-income men solely based on the larger prediction intervals of men: this discrepancy might arise from the over-coverage of men. In this sense, our designed metric EOC, not only contributes to evaluating and enhancing fairness under uncertainty but also serves for fair uncertainty quantification, discovering model bias in uncertainty quantification. A possible application of EOC is for guiding sample selection in active learning [52, 53], allowing the applied model to label the most uncertain samples in a fair manner.
Discretization into equal-mass bins is a favorable strategy to avoid excessively small sizes of  $\mathbb{D}(a, m)$ . While this guarantees a tighter upper bound for the expectation of the maximum coverage gap within bins according to Theorem 4.1, our method can be extended beyond equal-mass bins. When the
number of samples in each bin is unequal, we adjust the constraints in Eq. [1] by incorporating a weighted sum of  $\beta_{m}$ , i.e.,  $\sum_{m\in [M]}\frac{\beta_m}{M[\mathbb{D}(m)]} = 1 - \alpha$ . Consequently, in Algorithm [2] the gradient in the direction of each bin also should be weighted by  $|\mathbb{D}(m)|$ . These adaptations allow our method to maintain its effectiveness when the sample sizes in each class are uneven, e.g., in imbalanced classification problems. Moreover, our method is able to address certain coverage rate concerns within particular bins, such as low-income females, by assigning specific coverage rates to the corresponding bins while allowing flexibility in adjusting the coverage rates of other bins.
Why Better EOC? Our methods have been empirically shown to have better EOC than methods designed for equalized coverage, and here we elucidate the reasons behind this improvement. Given the underlying distribution of the  $\alpha$ -quantile conformity scores in group  $a$  conditioned on  $y$  as  $Q_{a,y}(\alpha)$ . For every  $y$  and  $a$ , calibrating with  $Q_{a,y}(\alpha)$  would generate perfect EOC. However, due to the limitations of available calibration samples, the estimation of  $Q_{a,y}(\alpha)$  is subject to bias. Since the base model cannot perfectly capture the underlying distribution, the conformity score, as a measure of disagreement, must correlate with  $Y$ . While equalized coverage methods generally ignore this correlation, our method tries to handle it. In Section 4.1, we used  $G_{a,m}(\beta_m)$  to estimate  $Q_{a,y}(\beta_m)$  for all  $y \in B_m$ , which is the  $\beta_m$ -quantile value of conformity scores whose  $y$  belong to bin  $B_m$ . The localized estimation within discretized bins is upper-bounded by the largest  $\beta_m$ -quantile value of score conditioned on any  $y$  and lower-bounded by the smallest one, regardless of the unknown distribution of  $Y$ . Consequently, we could better characterize the conditional distribution within each bin. This enhancement is especially evident when the Lipschitz constant  $L$  in Lemma 3.2 is small, such that the deviation from perfect EOC to the distribution generated by bins is negligible.
Possible Extension. For future work, we aim to further improve our method by seeking a tighter bound to achieve optimal EOC. Possible direction may follow [14, 54], by discretizing  $Y$  into finite values and bounding deviation in EOC of the discretized predictor. Additionally, our conformalized method leverages statistical properties to ensure coverage rates in each bin and optimize prediction interval widths, making it particularly appropriate for large datasets. With a better discretization strategy that better utilizes the calibration samples, e.g., ensemble sampling, our method may maintain its effectiveness on a small calibration set.
Potential Exploration beyond the Current Scope. The context of this paper is situated in post-processing fairness. Though post-processing methods typically underperform in-processing methods, in-processing methods are not applicable in many situations, e.g., the prediction model is a pre-trained black-box regression model, or a flexible and computationally efficient method is required. Whereas, adapting the proposed method to an in-processing setting to obtain better EOC is a possible extension. It would also be interesting to extend our method to the context that sensitive attributes are not available in the test data, possibly through missing data augmentation for the calibration set [55] or prediction-powered inference [56].
In this paper, we introduce a new uncertainty-aware fairness notion, equal opportunity of coverage (EOC), which addresses the limitations of the seminal work of equalized coverage [11]. EOC has several desired properties: It guarantees equal coverage rates for groups with the same labels and marginal coverage rate at a pre-determined level. It also ensures a small prediction interval. The theoretical analyses and empirical findings indicate that EOC and equalized coverage are generally incompatible. We suggest using EOC as an alternative to equalized coverage when equal coverage rates and assessment of uncertainty are needed for more fine-grained demographic groups. To improve EOC, we propose a distribution-free post-processing method, BFQR, based on discretization. Experimental results on synthetic and real-world datasets show that BFQR achieves competitive EOC and ensures guaranteed marginal coverage rates with small mean prediction interval widths compared to the state-of-the-art. Moreover, BFQR is adaptable to various settings, such as classification and other decision-making tasks.
This work is supported in part by NSF under grant III-2106758. Lu Cheng is in part supported by the Cisco Research Gift Grant. We thank Xinhua Zhang for the helpful discussion on the optimization
method. We are grateful to the anonymous reviewers at NeurIPS 2023 for providing valuable feedback and suggestions.