SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \tilde {\rho} _ {\mathrm {R S - D e l}} (\boldsymbol {x}; \mu_ {y}) = \mu_ {y} - 1 + p ^ {r} \\ = \mu_ {y} - 1 + \left(\frac {| \boldsymbol {x} | - (1 - p) | \boldsymbol {x} |}{| \boldsymbol {x} |}\right) ^ {r} \\ \geq \mu_ {y} - 1 + \left(\frac {| \boldsymbol {x} | - \lceil (1 - p) | \boldsymbol {x} | \rceil}{| \boldsymbol {x} |}\right) ^ {r} \\ = \mu_ {y} - 1 + \prod_ {j = 1} ^ {r} \frac {| \boldsymbol {x} | - k (| \boldsymbol {x} |)}{| \boldsymbol {x} |} \\ \geq \mu_ {y} - 1 + \prod_ {j = 1} ^ {r} \frac {\left(\left| \boldsymbol {x} \right| - k \left(\left| \boldsymbol {x} \right|\right) - j + 1\right)}{\left(\left| \boldsymbol {x} \right| - j + 1\right)} \\ = \mu_ {y} - 1 + \frac {\binom {| \boldsymbol {x} | - r}{k (| \boldsymbol {x} |)}}{\binom {| \boldsymbol {x} |}{k (| \boldsymbol {x} |)}} \\ = \tilde {\rho} _ {\mathrm {R S - A b n}} (\boldsymbol {x}; \mu_ {y}) \\ \end{array}
$$
Remark 12. Jia et al. [25] extend randomized ablation to top- $k$  prediction, while at the same time proposing enhancements to the Hamming distance certificate. These enhancements also apply to regular classification and involve: (1) discretizing lower/upper bounds on the confidence scores; and (2) using a better statistical method to estimate lower/upper bounds on the confidence scores. However, both of these enhancements would have a negligible impact in our experiments, as we shall now explain. The first enhancement tightens lower/upper bounds on the confidence scores by rounding up/down to the nearest integer multiple of  $q \coloneqq 1 / \binom{|x|}{k(|x|)}$ . This improves the lower/upper bound by at most
$$
q \leq \min  \left\{\left(\frac {| \boldsymbol {x} | - k}{| \boldsymbol {x} |}\right) ^ {| \boldsymbol {x} | - k}, \left(\frac {k}{| \boldsymbol {x} |}\right) ^ {k} \right\} \leq \frac {1}{| \boldsymbol {x} |}
$$
if the number of retained elements satisfies  $1 \leq k \leq |\pmb{x}|$ , as is the case in our experiments. However, since we consider inputs of length  $|x| \geq 10^3$ , this means the improvement in the bound due to discretization is at most  $q \leq 10^{-3}$ . This improvement is comparable to the resolution of our estimator  $(1 / n_{\mathrm{bnd}} \sim 10^{-4})$ , and consequently, discretization will not have a discernible impact. The second enhancement involves simultaneously estimating lower/upper bounds using a statistical method called SimuEM [97]. SimuEM has been demonstrated to improve tightness when there are multiple classes [25], however it has no impact when there are two classes (as is the case in our experiments).