SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathcal {L} _ {h} = \frac {1}{M} \sum_ {m = 1} ^ {M} \ell \left(h \circ r \left(\boldsymbol {x} _ {m}\right), y _ {m}\right) \tag {10}
$$
The selection score for a particular point  $x$  is then given by:
$$
g _ {\mathrm {S N}} (\boldsymbol {x}, f) = \sigma (g \circ r (\boldsymbol {x})) = \frac {1}{1 + \exp (g \circ r (\boldsymbol {x}))} \tag {11}
$$
Self-Adaptive Training (SAT) Alternatively, prior works like Deep Gamblers [Liu et al., 2019] and Self-Adaptive Training [Huang et al., 2020] have also considered explicitly modeling the abstention
class  $\perp$  and adapting the optimization process to provide a learning signal for this class. For instance, Self-Adaptive Training (SAT) incorporates information obtained during the training process into the optimization itself by computing and monitoring an exponential moving average of training point predictions over the training process. Samples with high prediction uncertainty are then used for training the abstention class. To ensure that the exponential moving average captures the true prediction uncertainty, an initial burn-in phase is added to the training procedure. This delay allows the model to first optimize the non-augmented, i.e., original  $C$ -class prediction task and optimize for selective classification during the remainder of the training process. The updated loss is defined as:
$$
\mathcal {L} = - \frac {1}{M} \sum_ {m = 1} ^ {M} \left(t _ {i, y _ {i}} \log p _ {i, y _ {i}} + \left(1 - t _ {i, y _ {i}}\right) \log p _ {i, C + 1}\right) \tag {12}
$$
The rejection/abstention decision is then determined by the degree of confidence in the rejection class:
$$
g _ {\mathrm {S A T}} (\boldsymbol {x}, f) = f (\boldsymbol {x}) _ {C + 1} \tag {13}
$$
Deep Ensembles (DE) Finally, ensemble methods combine the information content of  $M$  models into a single final model. Since these models approximate the variance of the underlying prediction problem, they are often used for the purpose of uncertainty quantification and, by extension, selective prediction. The canonical instance of this approach for deep learning based models, Deep Ensembles (DE) [Lakshminarayanan et al., 2017], trains multiple models from scratch with varying initializations using a proper scoring rule and adversarial training. Then, after averaging the predictions made by the model, the softmax response (SR) mechanism is applied:
$$
g _ {\mathrm {D E}} (\boldsymbol {x}, f) = \max  _ {c \in C} \frac {1}{M} \sum_ {m = 1} ^ {M} f _ {\boldsymbol {\theta} _ {m, T}} (\boldsymbol {x}). \tag {14}
$$
Monte-Carlo Dropout (MCD0) To overcome the computational cost of estimating multiple models from scratch, Monte-Carlo Dropout (MCD0) [Gal and Ghahramani, 2016] allows for bootstrapping of model uncertainty of a dropout-equipped model at test time. While dropout is predominantly used during training to enable regularization of deep neural nets, it can also be used at test time to yield a random sub-network of the full neural network. Concretely, given a model  $f$  with dropout-probability  $o$ , we can generate  $M$  random sub-networks at test-time by deactivating a fraction  $o$  of nodes. For a given test input  $x$  we can then average the outputs over all models and apply softmax response (SR):
$$
g _ {\mathbf {M C D O}} (\boldsymbol {x}, f) = \max  _ {c \in C} \frac {1}{Q} \sum_ {q = 1} ^ {Q} f _ {o \left(\boldsymbol {\theta} _ {T}\right)} (\boldsymbol {x}) \tag {15}
$$
Selective Classification Training Dynamics (SCTD) Another ensembling approach that has recently demonstrated state-of-the-art selective classification performance is based on monitoring the model evolution during the training process. Selective Classification Training Dynamics (SCTD) [Rabanser et al., 2022] records intermediate models produced during training and computes a disagreement score of the intermediate predictions with the final prediction for any test-time input  $x$ :
$$
g _ {\mathrm {S C T D}} (\boldsymbol {x}, f) = \sum_ {t = 1} ^ {T} v _ {t} a _ {t} (\boldsymbol {x}, f) \quad \text {w i t h} \quad a _ {t} (\boldsymbol {x}, f) = \left\{ \begin{array}{l l} 1 & f _ {\boldsymbol {\theta} _ {t}} (\boldsymbol {x}) \neq f _ {\boldsymbol {\theta} _ {T}} (\boldsymbol {x}) \\ 0 & \text {o t h e r w i s e} \end{array} \right. \quad v _ {t} = \left(\frac {t}{T}\right) ^ {k} \tag {16}
$$
SCTD only observes the training process and does not modify the model architecture/training objective.
In this section, we document additional hyper-parameter choices. For Self-Adaptive Training (SAT), we set the pre-training epochs to 100 and momentum parameter 0.9. For Selective Classification Training Dynamics (SCTD), we set the weighting parameter  $k = 3$  and consider checkpoints at a 50 batch resolution. For Monte-Carlo Dropout, we set the dropout probability to 0.1. Entropy regularization as suggested in Feng et al. [2023] is employed with  $\beta = 0.01$ .
Figure 6: Inducing a class imbalance on CIFAR-10. We train multiple CIFAR-10 models across privacy levels and sampling probabilities for class 0 given by  $p_0$ . We plot the accuracy/coverage trade-off as well at the exact coverage level at which any point from the minority class is accepted. The accuracy-coverage trade-off for the full dataset is given by the dashed line while the trade-off for the minority group only is given by the solid line. Blue vertical lines show correctly classified points, orange points show incorrectly classified points. Non-private models accept correct points first and do so at the end of the coverage spectrum (i.e., majority points are accepted first). As we increase privacy (i.e., decrease  $\varepsilon$ ), the model is increasingly unable to rank minority examples based on prediction correctness and even accepts incorrect points first. Moreover, the accuracy of the model on the minority class decreases with stronger DP. These effects are especially strong for small sampling probabilities.
B.2 Class Imbalance Experiments
We provide additional experiments on the effect of class imbalance to extend our intuition from Section 3.3. To that end, we take two data sets from our main results, namely CIFAR-10 and FashionMNIST, and produce four alternate datasets from each dataset. These datasets feature various degrees of class imbalance with  $p_0 \in \{0.5, 0.25, 0.1, 0.01\}$  specifying the sampling probability for class 0. All other classes maintain a sampling probability of 1. We then train the same model as described in Section 4 and apply the softmax response SC algorithm.
We document these results in Figures 6 and 7. For  $\varepsilon = \infty$ , we observe the expected gains from SC: minority points are accepted towards the end of the coverage spectrum [Jones et al., 2020] and correct points are accepted first. This effect is independent of the sampling probability  $p_0$ . As we decrease the  $\varepsilon$  budget we observe that (i) the acceptance of minority groups starts to spread over the full coverage spectrum; (ii) the accuracy on the subgroup increasingly deteriorates with smaller  $\varepsilon$ , and (iii) wrongful overconfidence on the minority reverses the acceptance order at low sampling probabilities  $p_0$  (i.e., incorrect points are often accepted first). These results indicate that employing selective classification on private data can have unwanted negative effects in the presence of subgroups. Future work should investigate this connection more thoroughly.
B.3 Upper Bound Reachability
In Equation 5, we have introduced an upper bound on the selective classification performance on a model with full-coverage accuracy  $a_{\mathrm{full}}$ . We now present a simple experimental panel across varying
Figure 7: Inducing a class imbalance on FashionMNIST. Same insights as in Figure 7.
Figure 8: Upper bound matching experiment. The experiment as described in Section B.3 matches the optimal bound exactly across multiple full-coverage accuracy levels.
full-coverage accuracy levels showing that this bound is in fact reachable by a perfect selective classifier.
We assume a binary classification setting for which we generate a true label vector  $\pmb{y} \in \{0,1\}^{n_0 + n_1}$  with balanced classes, i.e.,  $n_0 = n_1$  where  $n_0$  corresponds to the number points labeled as 0 and  $n_1$  corresponds to the number points labeled as 1. Then, based on a desired accuracy level  $a_{\mathrm{full}}$ , we generate a prediction vector  $\pmb{p}$  which overlaps with  $\pmb{y}$  for a fraction of  $a_{\mathrm{full}}$ . Finally, we sample a scoring vector  $\pmb{s}$  where each correct prediction is assigned a score  $s_i \sim \mathcal{U}_{0,0.5}$  and each incorrect prediction is assigned a score  $s_i \sim \mathcal{U}_{0.5,1}$ . Here,  $\mathcal{U}_{a,b}$  corresponds to the uniform distribution on the interval  $[a,b)$ . This score is clearly optimal since thresholding the scoring vector  $\pmb{s}$  at 0.5 perfectly captures correct/incorrect predictions: all  $s_i < 0.5$  correspond to a correct prediction, while all  $s_i \geq 0.5$  correspond to an incorrect prediction. Computing the accuracy/coverage trade-off of this experiment, across various utility levels, matches the bound exactly.