SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. For any two non-decreasing function  $G(p)$  and  $C(p)$ , by the definition of the general inverse function,  $G(G^{-}(p)) \geq p$ . If  $C \succeq G$ , we therefore have  $C(G^{-}(p)) \geq G(G^{-}(p)) \geq p$ . Applying  $C^{-}$  to both sides yields  $C^{-}(C(G^{-}(p))) \geq C^{-}(p)$ . But  $x \geq C^{-} \circ C(x)$  (see e.g. Proposition 3 on p. 6
of [28]) and thus  $G^{-}(p) \geq C^{-}(p)$ . Plugging in  $F$  and  $F_{U}$  as  $G$  and  $C$ , this can yield  $F^{-} \succeq F_{U}^{-}$ . The other direction is similar.
A.1 Control of nonlinear functions of CDFs (Section 4.2.1)
A.1.1 Control for monotonic functions
Recall that we start with the simplest case where  $\xi$  is a monotonic function in the range of  $X$ . It is straightforward to have the following claim.
Claim 1. If we have  $\hat{F}_{n,L}^{\delta, -} \preceq F \preceq \hat{F}_{n,U}^{\delta, -}$  with probability at least  $1 - \delta$  for some  $\delta \in (0,1)$ , if  $\xi$  is an increasing function, then
$$
\xi (\hat {F} _ {n, L} ^ {\delta , -}) \succeq \xi (\hat {F} ^ {-}) \succeq \xi (\hat {F} _ {n, U} ^ {\delta , -})
$$
with probability at least  $1 - \delta$ . Similarly, if  $\xi$  is a decreasing function, then  $\xi(\hat{F}_{n,L}^{\delta, -}) \preceq \xi(\hat{F}^{-}) \preceq \xi(\hat{F}_{n,U}^{\delta, -})$  with probability at least  $1 - \delta$ .
We show how this could be applied to provide bounds for Gini coefficient and Atkinson index by controlling the numerator and denominator separately as integrals of monotonic functions of  $F^{-}$ .
Example 3 (Gini coefficient). If given a  $(1 - \delta)$ -CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$  and  $\hat{F}_{n,L}^{\delta}\succeq 0^{4}$ , we can provide the following bound for the Gini coefficient. Notice that
Example 4 (Atkinson index). First, we present the complete version of Atkinson index. Namely,
$$
\mathcal {A} (\varepsilon , X) := \left\{ \begin{array}{l l} 1 - \frac {\left(\int_ {0} ^ {1} (F ^ {-} (p)) ^ {1 - \varepsilon} d p\right) ^ {\frac {1}{1 - \varepsilon}}}{\int_ {0} ^ {1} F ^ {-} (p) d p}, & i f   \varepsilon \geq 0,   \varepsilon \neq 1; \\ 1 - \frac {\exp (\int_ {0} ^ {1} \ln (F ^ {-} (p)) d p)}{\int_ {0} ^ {1} F ^ {-} (p) d p}, & i f   \varepsilon = 1. \end{array} \right.
$$
Notice that for  $\varepsilon \geq 0$ ,  $(\cdot)^{1 - \varepsilon}$  and  $\ln (\cdot)$  are increasing functions, thus, for Atkinson index and  $a(1 - \delta)$ -
CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$  if  $\hat{F}_{n,L}^{\delta}\succeq 0$  let us define  $\mathcal{A}_U^\delta (\varepsilon ,X)\coloneqq 1 - \frac{\left(\int_0^1(\hat{F}_{n,U}^\delta, - (p))^{1 - \varepsilon}dp\right)^{\frac{1}{1 - \varepsilon}}}{\int_0^1\hat{F}_{n,L}^{\delta, - }(p)dp}$ $i\in \varepsilon \geq 0,\varepsilon \neq$ $1;1 - \frac{\exp(\int_0^1\ln(\hat{F}_{n,U}^{\delta, - }(p))dp)}{\int_0^1\hat{F}_{n,L}^{\delta, - }(p)dp}$ $i\in \varepsilon = 1$  . Then, with probability at least  $1 - \delta$ $\mathcal{A}_U^\delta (\varepsilon ,X)$  is an upper bound for  $\mathcal{A}(\varepsilon ,X)$  for all  $\varepsilon \in [0,1)$
As mentioned in Remark 1, instead of calculating bounds separately for each  $\varepsilon$ , simple post-processing enables us to efficiently issue a family of bounds.
Example 5 (CVaR fairness-risk measures and beyond). Recall that for  $\alpha \in (0,1)$
$$
\mathcal {D} _ {C V, \alpha} (T (F _ {g})) = \min  _ {\rho \in \mathbb {R}} \left\{\rho + \frac {1}{1 - \alpha} \cdot \mathbb {E} _ {g \sim \mathcal {P} _ {l d x}} [ T (F _ {g}) - \rho ] _ {+} \right\} - \mathbb {E} _ {g \sim \mathcal {P} _ {l d x}} [ T (F _ {g}) ].
$$
The function  $[T(F_g) - \rho]_+$  is an increasing function when  $\rho$  is fixed and its further composition with the expectation operation is still increasing. If we have  $(T_L^\delta(F_g), T_U^\delta(F_g))$  such that  $T_L^\delta(F_g) \leq$
$T(F_{g})\leq T_{U}^{\delta}(F_{g})^{5}$  for all  $g$  with probability at least  $1 - \delta$ , then we have
$$
\mathcal {D} _ {C V, \alpha} (T (F _ {g})) \leq \min _ {\rho \in \mathbb {R}} \left\{\rho + \frac {1}{1 - \alpha} \cdot \mathbb {E} _ {g \sim \mathcal {P} _ {I d x}} [ T _ {U} ^ {\delta} (F _ {g}) - \rho ] _ {+} \right\} - \mathbb {E} _ {g \sim \mathcal {P} _ {I d x}} [ T _ {L} ^ {\delta} (F _ {g}) ],
$$
and the first term of RHS can be minimized easily since it is a convex function of  $\rho$ .
A.1.2 Control for absolute and polynomial functions
Recall that if  $s_L \leq s \leq s_U$ , then
$$
s _ {L} \mathbf {1} \left\{s _ {L} \geq 0 \right\} - s _ {U} \mathbf {1} \left\{s _ {U} \leq 0 \right\} \leq | s | \leq \max  \left\{\left| s _ {U} \right|, \left| s _ {L} \right| \right\}.
$$
More generally, for any polynomial function  $\phi(s) = \sum_{k=0} \alpha_k s^k$ . Notice if  $k$  is odd,  $s^k$  is monotonic w.r.t.  $s$  and we can bound
$$
\begin{array}{l} \phi (s)\leq \sum_{\{k\text{is odd,}\alpha_{k}\geq 0\}}\alpha_{k}s_{U}^{k} + \sum_{\{k\text{is odd,}\alpha_{k} <   0\}}\alpha_{k}s_{L}^{k} \\ + \sum_ {\{k \text {i s e v e n}, \alpha_ {k} \geq 0 \}} \alpha_ {k} \max  \left\{\left| s _ {L} \right| ^ {k}, \left| s _ {U} \right| ^ {k} \right\} + \sum_ {\{k \text {i s e v e n}, \alpha_ {k} <   0 \}} \alpha_ {k} \left(s _ {L} \mathbf {1} \left\{s _ {L} \geq 0 \right\} - s _ {U} \mathbf {1} \left\{s _ {U} \leq 0 \right\}\right) ^ {k}. \\ \end{array}
$$
So, for  $\phi(F^{-})$ , we can plug in  $\hat{F}_{n,L}^{\delta,-}$  and  $\hat{F}_{n,U}^{\delta,-}$  to replace  $s_U$  and  $s_L$  to obtain an upper bound with probability at least  $(1-\delta)$ . The derivation for the lower bound is similar. We summarize our results as the following proposition.
Proposition 3. If given a  $(1 - \delta)$ -CBP, then with probability at least  $1 - \delta$ ,  $(\hat{F}_{n,L}^{\delta}, \hat{F}_{n,U}^{\delta})$
$$
\hat {F} _ {n, U} ^ {\delta , -} \mathbf {1} \{\hat {F} _ {n, U} ^ {\delta , -} \geq 0 \} - \hat {F} _ {n, L} ^ {\delta , -} \mathbf {1} \{\hat {F} _ {n, L} ^ {\delta , -} \leq 0 \} \preceq | F ^ {-} | \preceq \max  \{| \hat {F} _ {n, L} ^ {\delta , -} |, | \hat {F} _ {n, -} ^ {\delta , -} | \}.
$$
Moreover, for any polynomial function  $\phi(s) = \sum_{k=0}^{\infty} \alpha_k s^k$ , we have
$$
\begin{array}{l} \phi (F ^ {-}) \preceq \sum_ {\{k i s o d d, \alpha_ {k} \geq 0 \}} \alpha_ {k} (\hat {F} _ {n, L} ^ {\delta , -}) ^ {k} + \sum_ {\{k i s o d d, \alpha_ {k} <   0 \}} \alpha_ {k} (\hat {F} _ {n, U} ^ {\delta , -}) ^ {k} \\ + \sum_ {\{k \text {i s e v e n}, \alpha_ {k} \geq 0 \}} \alpha_ {k} \max  \left\{\left| \hat {F} _ {n, U} ^ {\delta , -} \right| ^ {k}, \left| \hat {F} _ {n, L} ^ {\delta , -} \right| ^ {k} \right\} \\ + \sum_ {\{k \text {i s e v e n}, \alpha_ {k} <   0 \}} \alpha_ {k} \left(\hat {F} _ {n, U} ^ {\delta , -} \mathbf {1} \{\hat {F} _ {n, U} ^ {\delta , -} \geq 0 \} - \hat {F} _ {n, L} ^ {\delta , -} \mathbf {1} \{\hat {F} _ {n, L} ^ {\delta , -} \leq 0 \}\right) ^ {k}. \\ \end{array}
$$
Example 6. If we have  $(T_L^\delta (F_g),T_U^\delta (F_g))$  such that  $T_{L}^{\delta}(F_{g})\leq T(F_{g})\leq T_{U}^{\delta}(F_{g})$  holds for all  $g$  we consider, then we can provide high probability upper bounds for
We will further show in Appendix B how our results are applied to specific examples.
A.1.3 Control for a general function
To handle general non-linearity, we need to introduce the class of functions of bounded variation on a certain interval, which is a very rich class that includes all the functions that are continuously differentiable or Lipschitz continuous on that interval.
Definition 4 (Functions of bounded total variation [26]). Define the set of partitions on  $[a,b]$  as
$$
\Pi = \left\{\pi = \left(x _ {0}, x _ {1}, \dots , x _ {n _ {\pi}}\right) \mid \pi i s a p a r t i t i o n \text {o f} [ a, b ] s a t i s f y i n g x _ {i} \leq x _ {i + 1} f o r a l l 0 \leq i \leq n _ {\pi} - 1 \right\}.
$$
${}^{5}T\left( {F}_{g}\right)$  here is one of the functionals in the form we studied,so that we can provide upper and lower bounds for it.
Then, the total variation of a continuous real-valued function  $\xi$ , defined on  $[a,b] \subset \mathbb{R}$  is defined as
$$
V _ {a} ^ {b} (\xi) := \sup  _ {\pi \in \Pi} \sum_ {i = 0} ^ {n _ {\pi}} | \xi (x _ {i + 1}) - \xi (x _ {i}) |
$$
where  $\Pi$  is the set of all partitions, and we say a function  $\xi$  is of bounded variation, i.e.  $\xi \in BV([a,b])$  iff  $V_{a}^{b}(\xi) < \infty$ .
Recall that  $X \geq 0$  in our cases, then, for  $\xi(F^{-})$ , we can have the following bound.
Theorem 2 (A restatement & formal version of Theorem 1). For  $a(1 - \delta)$ -CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$ , for any  $p\in [0,1]$  such that the total variation of  $\xi$  is finite on  $[0,\hat{F}_{n,L}^{\delta , - }(p)]$ , then
$$
\xi (F ^ {-} (p)) \leq V _ {0} ^ {\hat {F} _ {n, L} ^ {\delta , -} (p)} (\xi) - V _ {0} ^ {\hat {F} _ {n, U} ^ {\delta , -} (p)} (\xi) + \xi (\hat {F} _ {n, U} ^ {\delta , -} (p)).
$$
Moreover, if  $\xi$  is continuously differentiable on  $[0,\hat{F}_{n,L}^{\delta , - }(p)]$ , we can express  $V_0^s (\xi)$  as  $\int_0^x |\frac{d\xi}{ds} (s)|ds$  for any  $x\in [0,\hat{F}_{n,L}^{\delta , - }(p)]$ .
Proof. By the property of functions of bounded total variation [26], if  $\xi$  is of bounded total variation on  $[0, \hat{F}_{n,L}^{\delta, -}(p)]$ , then, we have that: for any  $x \in [0, \hat{F}_{n,L}^{\delta, -}(p)]$
$$
\xi (x) = V _ {0} ^ {x} (\xi) - \left(V _ {0} ^ {x} (\xi) - \xi (x)\right)
$$
where both  $f_{1}(x) \coloneqq V_{0}^{x}(\xi)$  and  $f_{2}(x) \coloneqq V_{0}^{x}(\xi) - \xi(x)$  are increasing functions. Moreover,
$$
V _ {0} ^ {x} (\xi) = \int_ {0} ^ {x} \left| \frac {d \xi}{d s} (s) \right| d s
$$
if  $\xi$  is continuously differentiable.
Thus, by taking advantage of the monotonicity, we have
So, if  $\xi$  is of bounded variation on the range of  $X$ , then
$$
\xi (F ^ {-}) \preceq V _ {0} ^ {\hat {F} _ {n, L} ^ {\delta , -}} (\xi) - V _ {0} ^ {\hat {F} _ {n, U} ^ {\delta , -}} (\xi) + \xi (\hat {F} _ {n, U} ^ {\delta , -}) = f _ {1} (\hat {F} _ {n, L} ^ {\delta , -}) - f _ {2} (\hat {F} _ {n, U} ^ {\delta , -}).
$$
A.2 Methods to obtain confidence two-sided bounds for CDFs (Section 4.1)
We provide details for two-sided bounds and our numerical methods in the following.
A.2.1 The reduction approach to constructing upper bounds of CDFs (Section 4.1.1)
We here provide the proof of Lemma 1.
Lemma 2 (A restatement & formal version of Lemma 1). For  $0 \leq L_1 \leq L_2 \dots \leq L_n \leq 1$ , since  $\mathbb{P}(\forall i : F(X_{(i)}) \geq L_i) \geq \mathbb{P}(\forall i : U_{(i)} \geq L_i)$  by [30], if we further have  $\mathbb{P}(\forall i : U_{(i)} \geq L_i) \geq 1 - \delta$ , then we have
Furthermore, let  $R(x)$  be defined as
$$
R (x) = \left\{ \begin{array}{l l} 1 - L _ {n}, & \text {f o r} x <   X _ {(1)} \\ 1 - L _ {n - 1}, & \text {f o r} X _ {(1)} \leq x <   X _ {(2)} \\ \ldots \\ 1 - L _ {1}, & \text {f o r} X _ {(n - 1)} \leq x <   X _ {(n)} \\ 1, & \text {f o r} X _ {(n)} \leq x. \end{array} \right.
$$
Then,  $F \preceq R$ .
Proof. Notice that for given order statistics  $\{X_{(i)}\}_{i = 1}^{n}$ , let  $\mathbb{P}_{\{X_{(i)}\}_{i = 1}^n}$  denote the probability taken over the randomness of  $\{X_{(i)}\}_{i = 1}^{n}$ , and  $\mathbb{P}_X$  denote the probability taken over the randomness of  $X$ , which is an independent random variable drawn from  $F$ . Let us denote  $B = -X$ , and  $B_{(i)}$  as the  $i$ -th order statistic for samples  $\{-X_i\}_{i = 1}^n$ . It is easy to see that  $B_{(n - i + 1)} = -X_{(i)}$ . We also denote  $\mathbb{P}_B$  as the probability taken over the randomness of  $B$ , and  $F_B$  as the CDF of  $B$ .
$$
\begin{array}{l} \mathbb {P} _ {\{X _ {(i)} \} _ {i = 1} ^ {n}} (\forall i: \lim  _ {\epsilon \rightarrow 0 ^ {+}} F (X _ {(i)} - \epsilon) \leq 1 - L _ {n - i + 1}) = \mathbb {P} _ {\{X _ {(i)} \} _ {i = 1} ^ {n}} (\forall i: \mathbb {P} _ {X} (X \geq X _ {(i)}) > L _ {n - i + 1}) \\ = \mathbb {P} _ {\left\{X _ {(i)} \right\} _ {i = 1} ^ {n}} (\forall i: \mathbb {P} _ {X} (- X \leq - X _ {(i)}) > L _ {n - i + 1}) \\ = \mathbb {P} _ {\left\{X _ {(i)} \right\} _ {i = 1} ^ {n}} (\forall i: \mathbb {P} _ {B} (B \leq B _ {(n - i + 1)}) > L _ {n - i + 1}) \\ = \mathbb {P} (\forall i: F _ {B} \circ F _ {B} ^ {-} (U _ {(n - i + 1)}) > L _ {n - i + 1}) \\ \geq \mathbb {P} (\forall i: U _ {(n - i + 1)} > L _ {n - i + 1}). \\ \end{array}
$$
where we use the fact that  $F_B^- (U_{(n - i + 1)})$  is of the same distribution as  $B_{(n - i + 1)}$  and the last inequality follows from Proposition 1, eq. 24 on p.5 of [28].
Notice that  $\mathbb{P}(\forall i:U_{(n - i + 1)} > L_{n - i + 1}) = \mathbb{P}(\forall i:U_{(n - i + 1)}\geq L_{n - i + 1})$  , and according to [30] and our assumption,  $\mathbb{P}(\forall i:F(X_{(i)})\geq L_i)\geq \mathbb{P}(\forall i:U_{(i)}\geq L_i)\geq 1 - \delta$
The conservative construction of  $R$  satisfies  $R \succeq F$  straightforwardly if  $\forall i: \lim_{\epsilon \to 0^+} F(X_{(i)} - \epsilon) \leq (1 - L_{n - i + 1})$  holds. Thus, we know  $R \succeq F$  with probability at least  $1 - \delta$ . Our proof is complete.
Figure 4: Example illustrating the construction of distribution-free CDF lower and upper bounds by bounding order statistics. On the left, order statistics are drawn from a uniform distribution. On the right, samples are drawn from a real loss distribution, and the corresponding Berk-Jones CDF lower and upper bound are shown in black. Our distribution-free method gives bound  $b_{i}^{(l)}$  and  $b_{i}^{(u)}$  on each sorted order statistic such that the bound depends only on  $i$ , as illustrated in the plots for  $i = 5$  (shown in blue). On the left, 1000 realizations of  $x_{(1)}, \ldots, x_{(n)}$  are shown in yellow. On the right, 1000 empirical CDFs are shown in yellow, and the true CDF  $F$  is shown in red.
A.2.2 Details of numerical optimization method (Section 4.3)
Now, we introduce the details of our numerical optimization method. Recall that one drawback of the QBRM bounding approach is that it is not weight function aware: when controlling  $\int_0^1\psi (p)F^{-}(p)dp$  for a non-negative weight function  $\psi$ , the procedure ignores the structure of  $\psi$ , as it first obtains  $\hat{F}_{n,L}^{\delta}$ , then provides an upper bound  $\int_0^1\psi (p)\hat{F}_{n,L}^{\delta , - }(p)dp$ .
Our numerical approach can overcome that drawback and can also easily be applied to handle mixtures of multiple functionals. The bounds obtained by our method are significantly tighter than those provided by methods in [30] in the regime of small data size. Notice that the small data size regime is the one people care about because when the data size is large, all the bounds we discussed will converge to the same value, and the gap between different bounds will shrink to 0 as the data size grows.
First, by [?] and Proposition 1, eq. 24 on p.5 of [28], we have for any  $0 \leq L_1 \leq \dots \leq L_n \leq 1$ ,
$$
\begin{array}{l} \mathbb {P} \left(\forall i, F (X _ {(i)}) \geq L _ {i}\right) \geq \mathbb {P} \left(\forall i, U _ {(i)} \geq L _ {i}\right) \\ \geq n! \int_ {L _ {n}} ^ {1} d x _ {n} \int_ {L _ {n - 1}} ^ {x _ {n}} d x _ {n - 1} \dots \int_ {L _ {1}} ^ {x _ {2}} d x _ {1}, \\ \end{array}
$$
where the right-hand side integral is a function of  $\{L_i\}_{i=1}^n$  and its partial derivatives can be exactly calculated by the package in [21]. Specifically, the package in [21] enables us to calculate
$$
v \left(L _ {1}, L _ {2}, \dots , L _ {n}, 1\right) := \int_ {L _ {n}} ^ {1} d x _ {n} \int_ {L _ {n - 1}} ^ {x _ {n}} d x _ {n - 1} \dots \int_ {L _ {1}} ^ {x _ {2}} d x _ {1}
$$
for any positive integer  $n$ . Notice that the partial derivative of  $v(L_1, L_2, \dots, L_n, 1)$  with respect to  $L_i$  is:
$$
\begin{array}{l} \partial_ {L _ {i}} v \left(L _ {1}, L _ {2}, \dots , L _ {n}, 1\right) = - \int_ {L _ {n}} ^ {1} d x _ {n} \int_ {L _ {n - 1}} ^ {x _ {n}} d x _ {n - 1} \dots \int_ {L _ {i + 1}} ^ {x _ {i + 2}} d x _ {i + 1} \\ \cdot \int_ {L _ {i - 1}} ^ {L _ {i}} d x _ {i - 1} \dots \int_ {L _ {1}} ^ {x _ {2}} d x _ {1}, \\ = - v \left(L _ {i + 1}, \dots , L _ {n}, 1\right) \cdot v \left(L _ {1}, \dots , L _ {i - 1}, L _ {i}\right), \\ \end{array}
$$
which we can also use the package in [21] to calculate the partial derivatives.
Consider providing upper or lower bounds for  $\int_0^1\psi (p)F^{-}(p)dp$  for non-negative weight function  $\psi$  as an example. For any  $\{L_i\}_{i = 1}^n$  satisfying  $\mathbb{P}\big(\forall i,F(X_{(i)})\geq L_i\big)\geq 1 - \delta$ , one can use conservative CDF completion in [30] to obtain  $\hat{F}_{n,L}^{\delta}$ , i.e.  $\int_0^1\psi (p)\xi (\hat{F}_{n,L}^{\delta , - }(p))dp = \sum_{i = 1}^{n + 1}\xi (X_{(i)})\int_{L_{i - 1}}^{L_i}\psi (p)dp,$  where  $L_{n + 1}$  is  $1, L_0 = 0$ , and  $X_{(n + 1)} = \infty$  or a known upper bound for  $X$ . Then, we can formulate tightening the upper bound as an optimization problem:
$$
\mathbb {P} \big (\forall i, F (X _ {(i)}) \geq L _ {i} \big) \geq 1 - \delta , \text {a n d} 0 \leq L _ {1} \leq \dots \leq L _ {n} \leq 1.
$$
Similarly, for the lower bound, we can use the CDF completion mentioned in Theorem 1, and construct  $\hat{F}_{n,U}^{\delta}$ , then, we can study the following lower bound for  $\int_0^1\psi (p)F^{-}(p)dp$
$$
\sum_ {i = 1} ^ {n} \xi (X _ {(i)}) \int_ {L _ {n - i}} ^ {L _ {n - i + 1}} \psi (p) d p
$$
where  $X_{(0)} = 0$
Parameterized model approach. Notice the above optimization problem formulation has a drawback: if more samples are drawn, i.e.  $n$  increases, then the number of parameters we need to optimize also increases. In practice, we re-parameterize  $\{L_i\}_{i=1}^n$  as the following:
$$
L _ {i} (\theta) = \frac {\sum_ {j = 1} ^ {i} \exp (\phi_ {\theta} (g _ {j}))}{1 + \sum_ {j = 1} ^ {n} \exp (\phi_ {\theta} (g _ {j}))}
$$
where  $g_{i}$  are random Gaussian seeds. This is of the same spirit as using random seeds in generative models. We find that a simple parameterized neural network model with 3 fully-connected hidden layers of dimension 64 is enough for good performance and robust to hyper-parameter settings. Take the upper bound optimization problem as an example; using the new parameterized model, we have
$$
\min  _ {\{\theta \} _ {i = 1} ^ {n}} \sum_ {i = 1} ^ {n + 1} \xi \left(X _ {(i)}\right) \int_ {L _ {i - 1} (\theta)} ^ {L _ {i} (\theta)} \psi (p) d p \tag {1}
$$
$$
n! \int_ {L _ {n} (\theta)} ^ {1} d x _ {n} \int_ {L _ {n - 1} (\theta)} ^ {x _ {n}} d x _ {n - 1} \dots \int_ {L _ {1} (\theta)} ^ {x _ {2}} d x _ {1} \geq 1 - \delta ,
$$
where  $L_0 = 0$ ,  $L_{n + 1} = 1$ ,  $X_{(n + 1)} = \infty$  or a known upper bound for  $X$ . We can solve the above optimization problem using heuristic methods such as [9].
Post-processing for a rigorous guarantee for constraints. Notice that we may not ensure the constraint  $n! \int_{L_n(\theta)}^{1} dx_n \int_{L_{n-1}(\theta)}^{x_n} dx_{n-1} \cdots \int_{L_1(\theta)}^{x_2} dx_1 \geq 1 - \delta$  is satisfied in the above optimization because we may use surrogates like Langrange forms in our optimization processes. To make sure the constraint is strictly satisfied, we can do the following post-processing: let us denote the obtained  $L_i$ 's by optimizing (1) as  $L_i(\hat{\theta})$ . Then, we look for  $\gamma^* \in [0, L_n(\hat{\theta})]$  such that
$$
\gamma^ {*} = \inf  \left\{\gamma : n! v \left(L _ {1} (\hat {\theta}) - \gamma , \dots , L _ {n} (\hat {\theta}) - \gamma , 1\right) \geq 1 - \delta , \gamma \geq 0 \right\}.
$$
Notice there is always a feasible solution as when  $\gamma = L_{n}(\hat{\theta})$
$$
n! v \left(L _ {1} (\hat {\theta}) - \gamma , \dots , L _ {n} (\hat {\theta}) - \gamma , 1\right) \geq \mathbb {P} \left(\forall i, U _ {(i)} \geq 0\right) = 1
$$
and  $\upsilon(L_1(\hat{\theta}) - \gamma, \dots, L_n(\hat{\theta}) - \gamma, 1)$  is a decreasing function of  $\gamma$ . We can use binary search to efficiently find (a good approximate of)  $\gamma^*$ .
B Other dispersion measures and calculation
B.1 Lorenz curve & the extended Gini family
Lorenz curve. In the main context, Lorenz curve has been mentioned in reference to Gini coefficient and Atkinson index. To be more complete, we further demonstrate the definition of Lorenz curve in its mathematical form.
Definition 5 (Lorenz curve). The definition of Lorenz curve is a function: for  $t \in [0,1]$ ,
$$
\mathcal {L} (t) = \frac {\int_ {0} ^ {t} F ^ {- 1} (p)   d p}{\int_ {0} ^ {1} F ^ {- 1} (p)   d p}.
$$
We can obtain a lower bound and an upper bound function for the Lorenz curve. Given a  $(1 - \delta)$ -CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$  and  $\hat{F}_{n,L}^{\delta}\succeq 0$ , we can construct a lower bound function  $\mathcal{L}_L^\delta (t)$ :
$$
\mathcal {L} _ {L} ^ {\delta} (t) = \frac {\int_ {0} ^ {t} \hat {F} _ {n , U} ^ {\delta , -} (p) d p}{\int_ {0} ^ {1} \hat {F} _ {n , L} ^ {\delta , -} (p) d p},
$$
and an upper bound can be obtained by
$$
\mathcal {L} _ {U} ^ {\delta} (x) = \frac {\int_ {0} ^ {t} \hat {F} _ {n , L} ^ {\delta , -} (p)   d p}{\int_ {0} ^ {1} \hat {F} _ {n , U} ^ {\delta , -} (p)   d p}.
$$
With probability at least  $1 - \delta$ , the true Lorenz curve sits between the upper bound function and the lower bound function for all  $t \in [0, 1]$ .
The extended Gini family. The Gini coefficient can further give rise to the extended Gini family, which is a family of variability and inequality measures that depends on one parameter – the extended Gini parameter. The definition is as follows.
Definition 6 (The extended Gini family[34]). The extended Gini coefficient is given by
$$
\begin{array}{l} \mathcal {G} (\nu , X): = \frac {- \nu \operatorname {C o v} (X , [ 1 - F (X) ] ^ {\nu - 1})}{\mathbb {E} [ X ]} \\ = 1 - \frac {\nu \int_ {0} ^ {1} (1 - p) ^ {\nu - 1} F ^ {-} (p) d p}{\int_ {0} ^ {1} F ^ {-} (p) d p}, \\ \end{array}
$$
where  $\nu > 0$  is the extended Gini parameter and  $Cov(\cdot, \cdot)$  is the covariance.
For the extended Gini coefficient, choosing different  $\nu$ ’s corresponds to different weighting schemes applied to the vertical distance between the egalitarian line and the Lorenz curve; and if  $\nu = 2$ , it is the standard Gini coefficient.
Given a  $(1 - \delta)$ -CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$  and  $\hat{F}_{n,L}^{\delta}\succeq 0$ , we can construct upper bound for  $\mathcal{G}$ . Let
$$
\mathcal {G} _ {U} ^ {\delta} (\nu , X) := 1 - \frac {\nu \int_ {0} ^ {1} (1 - p) ^ {\nu - 1} \hat {F} _ {n , U} ^ {\delta , -} (p) d p}{\int_ {0} ^ {1} \hat {F} _ {n , L} ^ {\delta , -} (p) d p},
$$
then  $\mathcal{G}_U^\delta (\nu ,X)\succeq \mathcal{G}(\nu ,X)$  with probability at least  $1 - \delta$
B.2 Generalized entropy index
The generalized entropy index [29] is another measure of inequality in a population. Specifically, the definition is: for real number  $\alpha$
$$
G E (\alpha , X) := \left\{ \begin{array}{l} \frac {1}{\alpha (\alpha - 1)} \mathbb {E} \left[ \left(\frac {X}{\mathbb {E} X}\right) ^ {\alpha} - 1 \right], \alpha \neq 0, 1 \\ \mathbb {E} \left[ \frac {X}{\mathbb {E} X} \ln (\frac {X}{\mathbb {E} X}) \right], \quad \text {i f} \alpha = 1 \\ - \mathbb {E} \left[ \ln (\frac {X}{\mathbb {E} X}) \right], \quad \text {i f} \alpha = 0. \end{array} \right.
$$
It is not hard to further expand the expressions and write the generalized entropy index as:
$$
G E (\alpha , X) := \left\{ \begin{array}{l} \frac {1}{\alpha (\alpha - 1)} \int_ {0} ^ {1} \left[ \left(\frac {F ^ {-} (p)}{\int_ {0} ^ {1} F ^ {-} (p) d p}\right) ^ {\alpha} - 1 \right] d p, \alpha \neq 0, 1 \\ \int_ {0} ^ {1} \left[ \frac {F ^ {-} (p)}{\int_ {0} ^ {1} F ^ {-} (p) d p} \ln \left(\frac {F ^ {-} (p)}{\int_ {0} ^ {1} F ^ {-} (p) d p}\right) \right] d p, \text {i f} \alpha = 1 \\ - \int_ {0} ^ {1} \left[ \ln \left(\frac {F ^ {-} (p)}{\int_ {0} ^ {1} F ^ {-} (p) d p}\right) \right] d p, \text {i f} \alpha = 0. \end{array} \right.
$$
Notice that  $(\cdot)^{\alpha}$  is a monotonic function for the case  $\alpha \neq 0,1$ , and  $\ln (\cdot)$  is also a monotonic function, so the bound can be obtained similarly as in the case of Atkinson index. For instance, for  $\alpha >1$  given a  $(1 - \delta)$ -CBP  $(\hat{F}_{n,L}^{\delta},\hat{F}_{n,U}^{\delta})$