SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 1. For  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$ , if  $\xi$  is a function with bounded total variation on the range of  $X$ , there exists increasing functions  $f_{1},f_{2}$  with explicit and calculable forms, such that with probability at least  $1 - \delta$ ,  $\xi (F^{-})\preceq f_{1}(\hat{F}_{n,L}^{\delta , - }) - f_{2}(\hat{F}_{n,U}^{\delta , - })$ .
As an example, recall that [4] studies forms like
$$
\int_ {0} ^ {1} \xi (F ^ {-} (\alpha)) \psi (\alpha) d \alpha
$$
to quantify how sensitive the  $\alpha$ -VaR value is w.r.t its parameter  $\alpha$ . For nonlinear functions beyond polynomials, consider the example where  $\xi = e^{x} + e^{-x}$ . This can be readily bounded since it is a mixture of monotone functions.
This can be easily achieved by taking truncation over 0. Also, the construction of  $\hat{F}_{n,L}^{\delta}$  in Section A.2 always satisfies this constraint.
4.2.2 Control of nonlinear functionals of CDFs and beyond
Finally, we show how our techniques from the previous section can be applied to handle the general form  $\int_0^1\xi (T(F^- (\alpha)))\psi (\alpha)d\alpha$ , where  $\psi$  can be a general function (not necessarily non-negative) and  $\xi$  can be a functional of the inverse CDF. To control  $\xi (T(F^{-}))$ , we can first obtain two-sided bounds for  $T(F^{-})$  if  $T(F^{-})$  is in the class of QBRMs or in the form of  $\int \psi (p)\xi_2(F^{-}(p))dp$  for some nonlinear function  $\xi_{2}$  (as in [4]). We can also generalize the weight functions in QBRMs from non-negative to general weight functions once we notice that  $\psi$  can be decomposed into two non-negative functions, i.e.  $\psi = \max \{\psi ,0\} -\max \{-\psi ,0\}$ . Then, we can provide upper bounds for terms like  $\int \max \{\psi ,0\} \xi (F^{-}(p))dp$  by adopting an upper bound for  $\xi (F^{-})$ .
4.3 Numerical optimization towards tighter bounds for statistical functionals
Having described our framework for obtaining CDF bounds and controlling rich families of risk measures, we return to the question of how to produce the CDF bounds. One drawback of directly using the bound returned by Berk-Jones is that it is not weight function aware, i.e., it does not leverage knowledge of the target risk measures. This motivates the following numerical optimization method, which shows significant improvement over previous bounds including DKW and Berk-Jones bounds (as well as the truncated version proposed in [30]).
Our key observation is that for any  $0 \leq L_1 \leq \dots \leq L_n \leq 1$ , we have  $\mathbb{P}\big(\forall i, F(X_{(i)}) \geq L_i\big) \geq n! \int_{L_n}^1 dx_n \int_{L_{n-1}}^{x_n} dx_{n-1} \cdots \int_{L_1}^{x_2} dx_1$ , where the right-hand side integral is a function of  $\{L_i\}_{i=1}^n$  and its partial derivatives can be calculated exactly by the package in [21]. Consider controlling  $\int_0^1 \psi(p) F^-(p) dp$  as an example. For any  $\{L_i\}_{i=1}^n$  satisfying  $\mathbb{P}\big(\forall i, F(X_{(i)}) \geq L_i\big) \geq 1 - \delta$ , one can use conservative CDF completion to obtain  $\hat{F}_{n,L}^\delta$ , i.e.
$$
\int_ {0} ^ {1} \psi (p) \xi (\hat {F} _ {n, L} ^ {\delta , -} (p)) d p = \sum_ {i = 1} ^ {n + 1} \xi \left(X _ {(i)}\right) \int_ {L _ {i - 1}} ^ {L _ {i}} \psi (p) d p,
$$
where  $L_{n + 1}$  is 1,  $L_0 = 0$ , and  $X_{(n + 1)} = \infty$  or a known upper bound for  $X$ . Then, we can formulate tightening the upper bound as an optimization problem:
$$
\min  _ {\{L _ {i} \} _ {i = 1} ^ {n}} \sum_ {i = 1} ^ {n + 1} \xi (X _ {(i)}) \int_ {L _ {i - 1}} ^ {L _ {i}} \psi (p) d p
$$
such that  $\mathbb{P}\big(\forall i, F(X_{(i)})\geq L_i\big)\geq 1 - \delta$ , and  $0\leq L_1\leq \dots \leq L_n\leq 1$ . We optimize the above problem with gradient descent and a simple post-processing procedure to make sure the obtained  $\{\hat{L}_i\}_{i = 1}^n$  strictly satisfy the above constraints. In practice, we re-parameterize  $\{L_i\}_{i = 1}^n$  with a network  $\phi_{\theta}$  that maps  $n$  random seeds to a function of the  $L_{i}$ 's, and transform the optimization objective from  $\{L_i\}_{i = 1}^n$  to  $\theta$ . We find that a simple parameterized neural network model with 3 fully-connected hidden layers of dimension 64 is enough for good performance and robust to hyper-parameter settings.
$$
\gamma^ {*} = \inf \{\gamma : n! v (L _ {1} (\hat {\theta}) - \gamma , \dots , L _ {n} (\hat {\theta}) - \gamma , 1) \geq 1 - \delta , \gamma \geq 0 \}.
$$
Notice that there is always a feasible solution. We can use binary search to efficiently find (a good approximate of)  $\gamma^{*}$ .
With our experiments, we aim to examine the contributions of our methodology in two areas: bound formation and responsible model selection.
5.1 Learn then calibrate for detecting toxic comments
Using the CivilComments dataset [6], we study the application of our approach to toxic comment detection under group-based fairness measures. CivilComments is a large dataset of online comments
labeled for toxicity as well as the mention of protected sensitive attributes such as gender, race, and religion. Our loss function is the Brier Score, a proper scoring rule that measures the accuracy of probabilistic predictions, and we work in the common setting where a trained model is calibrated post-hoc to produce confidence estimates that are more faithful to ground-truth label probabilities. We use a pre-trained toxicity model and apply a Platt scaling model controlled by a single parameter to optimize confidence calibration. Our approach is then used to select from a set of hypotheses, determined by varying the scaling parameter in the range [0.25, 2] (where scaling parameter 1 recovers the original model). See the Appendix for more details on the experimental settings and our bound optimization technique.
5.1.1 Bounding complex expressions of group dispersion
First, we investigate the full power of our framework by applying it to a complex statistical dispersion objective. Our overall loss objective considers both expected mean across groups as well as the maximum difference between group medians, and can be expressed as:  $\mathcal{L} = \mathbb{E}_g[T_1(F_g)] + \lambda \sup_{g,g'}|T_2(F_g) - T_2(F_{g'})|$ , where  $T_{1}$  is expected loss and  $T_{2}$  is a smoothed version of a median (centered around  $\beta = 0.5$  with spread parameter  $a = 0.1$ ). Groups are defined by intersectional attributes:  $g\in G = \{\mathrm{black~female},\mathrm{white~female},\mathrm{black~male},\mathrm{white~male}\}$ . We use 100 and 200 samples from each group, and select among 50 predictors. For each group, we use our numerical optimization framework to optimize a bound on  $\mathcal{O} = T_1(F_g) + T_2(F_g)$  using the predictor (and accompanying loss distribution) chosen under the Berk-Jones method. Results are shown in Table 1. We compare our numerically-optimized bound (NN-Opt.) to the bound given by Berk-Jones as well as an application of the DKW inequality to lower-bounding a CDF.
Our framework enables us to choose a predictor that fits our specified fairness criterion, and produces reasonably tight bounds given the small sample size and the convergence rate of  $\frac{1}{\sqrt{n}}$ . Moreover, there is a large gain in tightness from numerical optimization in the case where  $n = 100$ , especially with respect to the bound on the maximum difference in median losses (0.076 vs. 0.016). These results show that a single bound can be flexibly optimized to improve on multiple objectives at once via our numerical method, a key innovation point for optimizing bounds reflecting complex societal concerns like differences in group medians [5].
Table 1: Applying our full framework to control an objective considering expected group loss as well as a maximum difference in group medians for  $n = 100$  and  $n = 200$  samples.
Method	n = 100	n = 200
Exp. Grp.	Max Diff.	Total	Exp. Grp.	Max Diff.	Total
DKW	0.36795	0.90850	1.27645	0.32236	0.96956	1.29193
BJ	0.34532	0.07549	0.42081	0.31165	0.00666	0.31831
NN-Opt. (ours)	0.32669	0.01612	0.34281	0.30619	0.00292	0.30911
Empirical	0.20395	0.00004	0.20399	0.20148	0.00010	0.20158
5.1.2 Optimizing bounds on measures of group dispersion
Having studied the effects of applying the full framework, we further investigate whether our method for numerical optimization can be used to get tight and flexible bounds on functionals of interest. First,  $\beta$ -CVaR is a canonical tail measure, and we bound the loss for the worst-off  $1 - \beta$  proportion of predictions (with  $\beta = 0.75$ ). Next, we bound a specified interval of the VaR ([0.5, 0.9]), which is useful when a range of quantiles of interest is known but flexibility to answer different queries within the range is important. Finally, we consider a worst-quantile weighting function  $\psi(p) = p$ , which penalizes higher loss values on higher quantiles, and study a smooth delta function around  $\beta = 0.5$ , a more robust version of a median measure. We focus on producing bounds using only 100 samples from a particular intersectionally-defined protected group, in this case black females, and all measures are optimized with the same hyperparameters. The bounds produced via numerical optimization (NN-Opt.) are compared to the bounds in [30] (as DKW has been previously shown to produce weak CDF bounds), including the typical Berk-Jones bound as well as a truncated version tailored to particular quantile ranges. See Table 2 and the Appendix for results.
The numerical optimization method induces much tighter bounds than Berk-Jones on all measures, and also improves over the truncated Berk-Jones where it is applicable. Further, whereas the truncated Berk-Jones bound will give trivial control outside of  $[\beta_{min},\beta_{max}]$ , the numerically-optimized bound not only retains a reasonable bound on the entire CDF, but even improves on Berk-Jones with respect to the bound on expected loss in all cases. For example, after adapting to CVaR, the numerically-optimized bound gives a bound on the expected loss of 0.23, versus 0.25 for Berk-Jones and 0.50 for Truncated Berk-Jones. Thus numerical optimization produces both the best bound in the range of interest as well as across the rest of the distribution, showing the value of adapting the bound to the particular functional and loss distribution while still retaining the distribution-free guarantee.
Table 2: Optimizing bounds on measures for protected groups.
Method	CVaR	VaR-Interval	Quantile-Weighted	Smoothed-Median
Berk-Jones	0.91166	0.38057	0.19152	0.00038
Truncated Berk-Jones	0.86379	0.34257	-	-
NN-Opt. (ours)	0.85549	0.32656	0.17922	0.00021
5.2 Investigating bounds on standard measures of dispersion
Next, we aim to explore the application of our approach to responsible model selection under non-group-based fairness measures, and show how using our framework leads to a more balanced distribution of loss across the population. Further details for both experiments can be found in the Appendix.
5.2.1 Controlling balanced accuracy in detection of genetic mutation
RxRx1 [31] is a task where the input is a 3-channel image of cells obtained by fluorescent microscopy, the label indicates which of 1,139 genetic treatments the cells received, and there is a batch effect that creates a challenging distribution shift across domains. Using a model trained on the train split of the RxRx1 dataset, we evaluate our method with an out-of-distribution validation set to highlight the distribution-free nature of the bounds. We apply a threshold to model output in order to produce prediction sets, or sets of candidate labels for a particular task instance. Prediction sets are scored with a balanced accuracy metric that equally weights sensitivity and specificity, and our overall objective is:  $\mathcal{L} = T_1(F) + \lambda T_2(F)$ , where  $T_{1}$  is expected loss,  $T_{2}$  is Gini coefficient, and  $\lambda = 0.2$ . We choose among 50 predictors (i.e. model plus threshold) and use 2500 population samples to produce our bounds. Results are shown in Figure 2.
Figure 2: Left: Bounds on the expected loss, scaled Gini coefficient, and total objective across different hypotheses. Right: Lorenz curves induced by choosing a hypothesis based on the expected loss bound versus the bound on the total objective. The y-axis shows the cumulative share of the loss that is incurred by the best-off  $\beta$  proportion of the population, where a perfectly fair predictor would produce a distribution along the line  $y = x$ .
The plot on the left shows how the bounds on the expected loss  $T_{1}$ , scaled Gini coefficient  $\lambda T_{2}$ , and total objective  $\mathcal{L}$  vary across the different hypotheses (i.e. model and threshold combination for producing prediction sets). The bold points indicate the optimal threshold choice for each quantity. On the right is shown the Lorenz curves (a typical graphical expression of Gini) of the loss distributions induced by choosing a hypothesis based on the expected loss bound versus the bound on the total objective. Incorporating the bound on Gini coefficient in hypothesis selection leads to a more equal loss distribution. Taken together, these figures illustrate how the ability to bound a non-group based
dispersion measure like the Gini coefficient can lead to less skewed outcomes across a population, a key goal in societal applications.
5.2.2 Producing recommendation sets for the whole population
Using the MovieLens dataset [12], we test whether better control on another important non-group based dispersion measure, the Atkinson index (with  $\epsilon = 0.5$ ), leads to a more even distribution of loss across the population. We train a user/item embedding model, and compute a loss that balances precision and recall for each set of user recommendations. Results are shown in Figure 3. Tighter control of the Atkinson index leads to a more dispersed distribution of loss across the population, even for subgroups defined by protected attributes like age and gender that are unidentified for privacy or security reasons.
Figure 3: We select two hypotheses  $h_0$  and  $h_1$  with different bounds on Atkinson index produced using 2000 validation samples, and once again visualize the Lorenz curves induced by each. Tighter control on the Atkinson index leads to a more equal distribution of the loss (especially across the middle of the distribution, which aligns with the choice of  $\epsilon$ ), highlighting the utility of being able to target such a metric in conservative model selection.
The field of distribution-free uncertainty quantification has its roots in conformal prediction [27]. The coverage guarantees of conformal prediction have recently been extended and generalized to controlling the expected loss of loss functions beyond coverage [1, 3]. The framework proposed by [30] offers the ability to select predictors beyond expected loss, to include a rich class of quantile-based risk measures (QBRMs) like CVaR and intervals of the VaR; they also introduce a method for achieving tighter bounds on certain QBRMs by focusing the statistical power of the Berk-Jones bound on a certain quantile range. Note that these measures cannot cover the range of dispersion measures studied in this work.
There is a rich literature studying both standard and group-based statistical dispersion measures, and their use in producing fairer outcomes in machine learning systems. Some work in fairness has aimed at achieving coverage guarantees across groups [24, 25], but to our knowledge there has not been prior work exploring controlling loss functions beyond coverage, such as the plethora of loss functions aimed at characterizing fairness, which can be expressed as group-based measures (cf. Section 3.2). Other recent fairness work has adapted some of the inequality measures found in economics. [32] aims to enforce that outcomes are not too different across groups defined by protected attributes, and introduces a convex notion of group CVaR, and [24] propose a DFUQ method of equalizing coverage between groups. [19] studies distributional inequality measures like Gini and Atkinson index since demographic group information is often unavailable, while [7] use the notion of Lorenz efficiency to generate rankings that increase the utility of both the worst-off users and producers.
In this work, we focus on a rich class of statistical dispersion measures, both standard group-based, and show how these measures can be controlled. In addition, we offer a novel numerical optimization method for achieving tighter bounds on these quantities. We investigate the effects of applying our framework via several experiments and show that our methods lead to more fair model selection and tighter bounds. We believe our study offers a significant step towards the sort of thorough and transparent validation that is critical for applying machine learning algorithms to applications with societal implications.
We thank the Google Cyber Research Program and ONR (Award N00014-23-1-2436) for their generous support. J. Snell gratefully acknowledges financial support from the Schmidt DataX Fund at Princeton University made possible through a major gift from the Schmidt Futures Foundation.
[1] Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control. arXiv:2110.01052, November 2021.
[2] Anthony B Atkinson et al. On the Measurement of Inequality. Journal of Economic Theory, 2(3):244-263, 1970.
[3] Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael I. Jordan. Distribution-Free, Risk-Controlling Prediction Sets. arXiv:2101.02703, August 2021.
[4] Mohammed Berkhouch, Ghizlane Lakhnati, and Marcelo Brutti Righi. Spectral Risk Measures and Uncertainty. arXiv preprint arXiv:1905.07716, 2019.
[5] Neil Bhutta, Andrew Chang, Lisa Dettling, and Joanne Hsu. Disparities in Wealth by Race and Ethnicity in the 2019 Survey of Consumer Finances. FEDS Notes, 2020, 09 2020.
[6] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491-500, 2019.
[7] Virginie Do, Sam Corbett-Davies, Jamal Atif, and Nicolas Usunier. Two-Sided Fairness in Rankings via Lorenz Dominance. Advances in Neural Information Processing Systems, 34:8596-8608, 2021.
[8] Kevin Dowd and David Blake. After VaR: The Theory, Estimation, and Insurance Applications of Quantile-Based Risk Measures. Journal of Risk & Insurance, 73(2):193-229, June 2006.
[9] Chengyue Gong and Xingchao Liu. Bi-Objective Trade-off with Dynamic Barrier Gradient Descent. Advances in Neural Information Processing Systems, 2021.
[10] Laura Hanu and Unitary team. Detoxify. Github. https://github.com/unitaryai/detoxify, 2020.
[11] Moritz Hardt, Eric Price, and Nati Srebro. Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems, 29, 2016.
[12] Maxwell F Harper and Joseph A Konstan. The Movielens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5(4):1-19, 2015.
[13] HO Hartley and HA David. Universal Bounds for Mean Range and Extreme Observation. The Annals of Mathematical Statistics, pages 85-99, 1954.
[14] Ursula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (Computationally-Identifiable) Masses. In International Conference on Machine Learning, pages 1939–1948. PMLR, 2018.
[15] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing Fairness Gerry-mandering: Auditing and Learning for Subgroup Fairness. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2564-2572. PMLR, 2018.
[16] Bruce P Kennedy, Ichiro Kawachi, and Deborah Prothrow-Stith. Income Distribution and Mortality: Cross-Sectional Ecological Study of the Robin Hood Index in the United States. British Medical Journal, 312(7037):1004–1007, 1996.
Broader impact. The broader impact of the proposed framework is significant, as it extends the ability to gain trust in machine learning systems. However there are important concerns and limitations.
- Focus on performance metrics In this paper we propose a range of performance metrics, which extend well beyond standard metrics concerning expected loss. However, in many situations these metrics are not sufficient to capture the effects of the machine learning system. Often a number of different metrics are required to provide a clearer picture of model performance, while some effects are difficult to capture in any metric. Also, while the measures studied offer the ability to more evenly distribute a quantity across a population, they do not offer guarantees to individuals. Finally, achieving a more equal distribution of the relevant quantity (e.g., loss or income) may have negative impacts on some segments of the population.
- Limitations These are summarized in the Conclusion but are expanded upon here. An important assumption in this work, and in distribution-free uncertainty quantification more generally, is that the examples seen in deployment are drawn from the same distribution as those in the validation set that are used to construct the bounds. Although this is an active area of research, here we make this assumption, and the quality of the bounds produced may degrade if the assumption is violated. A second limitation is that the scope of hypotheses and predictors we can select from is limited, due to theoretical constraints: a correction must be performed based on the size of the hypothesis set. Finally, the generated bounds may not be tight, depending on the amount of available validation data and unavoidable limits of the techniques used to produce the bounds. We did some comparisons to Empirical values of the measures we obtained bounds for in the experiments; more extensive studies would be useful to elucidate the value of the bounds in practice.
Organization of the Appendix. (1) In Appendix A, we provide detailed statements and derivations of our methodology presented Section 4.2.1, including how to obtain bounds for those measures mentioned in Section 3; (2) in Appendix B, we introduce further societal dispersion measures, beyond those presented in Section 3 and corresponding bounds; (3) in Appendix C, we investigate the extension of our results to multi-dimensional settings; (4) lastly, in Appendix D and E, we provide more complete details and results from our experiments (Section 5).
A Derivations and proofs for bounding methods
Section A.1, we first consider how to control, or provide upper bounds on, various quantities when we are given  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$ , which are constructed by  $\{X_{i}\}_{i = 1}^{n}$ , such that
$$
\mathbb {P} (\hat {F} _ {n, L} ^ {\delta , -} \preceq F \preceq \hat {F} _ {n, U} ^ {\delta , -}) \geq 1 - \delta
$$
where the randomness is taken over  $\{X_i\}_{i=1}^n$ .
Then, in Section A.2, we will show how we obtain  $(\hat{F}_{n,L}^{\delta , - },\hat{F}_{n,U}^{\delta , - })$  by extending the arguments in [30]. In addition, we show details in Section A.2.2 on how we go beyond the methods in [30] and provide a numerical optimization method for tighter bounds.
Proof of Proposition 1. We briefly describe the proof for Proposition 1. The proof is mainly based on [30], but we include it here for completeness. Notice for any non-decreasing function  $G: \mathbb{R} \to \mathbb{R}$  (not just a CDF), there exists the (general) inverse of  $G$  as  $G^{-}(p) = \inf \{x: G(x) \geq p\}$  for any  $p \in \mathbb{R}$ .
Proposition 2 (Restatement of Proposition 1). For the CDF  $F$  of  $X$ , if there exists two increasing functions  $F_U, F_L$  such that  $F_U \succeq F \succeq F_L$ , then we have  $F_L^- \succeq F^- \succeq F_U^-$ .