SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Qiang Ding $^{1,2}$ , Yixuan Cao $^{1,2}$ , Ping Luo $^{1,2,3}$
$^{1}$ Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, China  
 $^{2}$ University of Chinese Academy of Sciences, Beijing 100049, China  
 $^{3}$ Peng Cheng Laboratory, Shenzhen 518066, China {dingqiang22z, caoyixuan, luop}@ict.ac.cn
Selective classification allows a machine learning model to reject some hard inputs and thus improve the reliability of its predictions. In this area, the ensemble method is powerful in practice, but there has been no solid analysis on why the ensemble method works. Inspired by an interesting empirical result that the improvement of the ensemble largely comes from top-ambiguity samples where its member models diverge, we prove that, based on some assumptions, the ensemble has a lower selective risk than the member model for any coverage within a range. The proof is nontrivial since the selective risk is a non-convex function of the model prediction. The assumptions and the theoretical results are supported by systematic experiments on both computer vision and natural language processing tasks.
Although recent years have witnessed the broad applications of deep learning models, their reliability has not been guaranteed, which gives rise to the study of selective classification. For any given deep learning classifier, there might be inputs that the model is not able to classify correctly in practical applications. One approach to overcoming this problem is accurately delimiting the deep learning classifier's application scope so that the classifier rejects to predict over those hard inputs.
To this end, selective classification aims to learn a pair of models consisting of a standard classifier and a selective function that decides whether the system should reject the input. Since the classifier is well studied, the study of selective classification focuses on the design of the selective function. A selective function is usually a confidence estimator with a threshold [1], rejecting the input if and only if the estimated confidence is below the threshold. There are four categories of confidence estimation methods: post-hoc methods [2, 3], Bayesian methods [4, 5, 6], learning-based methods [7, 8, 9, 10], and ensemble methods [11]. Details about these methods is provided in Section 3.
Previous work has shown that Deep Ensemble has a good performance in selective classification  $[11]^1$ , but no solid analysis explains why the ensemble method works. This paper first proposes a theoretical foundation of Deep Ensemble, i.e., with some assumptions, the ensemble has a lower selective risk than the member model for any target coverage within a range. The proof is nontrivial since the selective risk (with the  $0/1$  loss) are non-convex. We then verify our assumptions and
<sup>1</sup>We conducted more comprehensive experiments, showing that Deep Ensemble is still one of the SOTA in Appendix C.
theoretical result on both computer vision and natural language processing tasks. In summary, the contributions of this paper are summarized as follows.
- We show an interesting empirical result that the performance improvement of Deep Ensemble in terms of selective classification dominantly comes from top-ambiguity samples.
- We are the first to prove that the ensemble has a lower selective risk than the member model for any target coverage within a range based on several reasonable assumptions. Our assumptions and analysis results are verified by systematic experiments on the tasks of image classification and text classification.
A selective classifier comprises a standard classifier and a selective function. Considering a standard classification problem,  $\mathbb{X}$  is a feature space,  $\mathbb{Y} = \{1,2,\dots,K\}$  is a finite label set, and a classifier is a function the maps  $\mathbb{X}$  to  $\mathbb{Y}$ . A labeled dataset  $\mathbb{D} = \{(x_i,y_i)\}_{i=1}^N \subseteq \mathbb{X} \times \mathbb{Y}$  is independently sampled from an identical distribution  $\mathcal{D}$ . Our goal is to learn a selective classifier  $(f,g)$ , where  $f$  is a standard classifier, and  $g: \mathbb{X} \to \{0,1\}$  is a selective function that estimates the correctness of  $f$ 's prediction. Given input  $x$ , the output of selective classifier  $(f,g)$  is
$$
(f, g) (x) = \left\{ \begin{array}{l l} f (x), & \text {i f} g (x) = 1 \\ \operatorname {r e j e c t}, & \text {i f} g (x) = 0 \end{array} . \right. \tag {1}
$$
The goal of selective classifiers is to minimize the selective risk subject to a given target coverage. The coverage of  $(f,g)$  is defined to be the probability of  $(f,g)$  not rejecting the input, i.e.,  $\phi (f,g)\coloneqq \mathbb{E}_{(x,y)\sim \mathcal{D}}[g(x)]$ . The selective risk of  $(f,g)$  is the risk of  $f$  conditioned on  $g(x) = 1$ , i.e.,  $R(f,g)\coloneqq \mathbb{E}_{(x,y)\sim \mathcal{D}}[\ell (f(x),y)g(x)] / \mathbb{E}_{(x,y)\sim \mathcal{D}}(g(x))$ , where  $\ell :\mathbb{Y}\times \mathbb{Y}\to \mathbb{R}$  is a given loss function. Typically,  $\ell$  is the 0/1 loss [1, 7, 8, 9]. Based on these definitions, the objective of selective classifiers is formalized as
$$
\min  _ {f, g} R (f, g), \text {s . t .} \phi (f, g) \geq c _ {\text {t a r g e t}},
$$
where  $c_{\mathrm{target}}$  is the target coverage. Note that the above objective is evaluated only on in-distribution samples, i.e., samples independently drawn the same distribution of the training data, excluding samples with distribution shifts — also known as OOD samples [12].
The selective function is usually realized by a confidence estimator  $\kappa : \mathbb{X} \to \mathbb{R}$  with a confidence threshold  $t$ :  $g(x) = \mathbf{1}_{\kappa(x) \geq t}$ , where  $\mathbf{1}$  is the indicator function [1]. In this case, we denote the selective classifier as  $(f, \kappa; t)$ . As the threshold  $t$  controls the coverage, a fixed  $(f, \kappa)$  can be fitted to different target coverages (with different confidence thresholds). Thus,  $(f, \kappa)$ 's performance can be evaluated under multiple different coverages, raising metrics such as risk-coverage curve (RC curve) [1] and the Area Under the Risk-Coverage curve (AURC) [10]. The RC curve of  $(f, \kappa)$  is a plot of  $(f, \kappa)$ 's selective risks against its coverages, which depicts the entire performance profile of  $(f, \kappa)$ . A lower risk-coverage curve suggests a better selective classifier. The AURC is a more concise and comprehensive metric. Similar to the RC curve, a lower AURC suggests a better selective classifier.
This paper focuses on analyzing the selective risk of Deep Ensemble [11], which is defined as follows. To obtain a Deep Ensemble  $(f_{E},\kappa_{E})$ ,  $M$  member models are trained in parallel with random initialization of their parameters and random shuffling of the data points. Assume that each member classifier provides a predictive probability distribution  $\pi_{m} = (\pi_{m}^{1},\dots,\pi_{m}^{K})$ ,  $1\leq m\leq M$  and predicts  $f_{m}(x) = \arg \max_{1\leq k\leq K}\pi_{m}^{k}$ . Then Deep Ensemble is a uniformly-weighted soft voting of member models:  $\pi_E(x)\coloneqq \frac{1}{M}\sum_{m = 1}^{M}\pi_{m}(x)$ , and its confidence estimator is  $\kappa_{E}(x) = \max_{k}\pi_{E}^{k}(x)$ .
Selective Classification. A selective function usually relies on a confidence estimator. To realize the confidence estimator, four approaches has been investigated by previous work. 1. Post-hoc methods, including Softmax Response (SR) [2] and Trust Score [3]. SR leverages the classifier's maximum class probability (MCP) as the confidence score. Trust Score considers both the distance from the sample to the predicted class and the distance from the sample to another nearest class,
using their ratio as the confidence score. 2. Bayesian methods. MC-Dropout is an easy-to-use Bayesian methods [5], which samples the classifier's parameters through Dropout [13]. Specifically, MC-Dropout enables the dropout layer of the classifier at test time, then runs multiple forwards passes, and finally uses the minus variance of the maximum activation of the classifier's softmax layer as the confidence score. A shortcoming of MC-Dropout is its high time cost at test time. 3. Learning-based methods. There are two categories of learning-based methods: 1. using MCP as the confidence score and only modifying the standard classifier's loss function, including OSP [14], and Reg-curr [10]; 2. modifying both the architecture and the loss function of the classifier, including SelectiveNet [7], Gambler [8], ConfidNet [15], and SAT [9]. SAT is the SOTA selective classifier in terms of non-ensemble methods, which adds an extra class to the classifier as a reject option and proposes a loss function to train the classifier and the confidence estimator simultaneously. 4. Ensemble Methods. Deep Ensemble is a vanilla classifiers' ensemble combined by soft voting and equipped with an MCP confidence estimator [11], which has achieved the SOTA in uncertainty estimation. Considering the heavy computational overhead of the ensemble, [16, 17] reduced the ensemble's time consumption by ensemble distillation techniques that distill the ensemble's knowledge into an individual model with the ensemble's diversity retained.
Analyses of Ensemble Methods. Ensemble methods combine multiple individual models to improve machine learning models' predictive performances (see [18] for a review). The error-ambiguity decomposition has been proposed to explain the better performance of the ensemble in regression tasks [19]. However, for classification tasks, there is no such simple and elegant analysis, since the evaluation metrics are non-convex [20]. Thus, the corresponding analysis for classification tasks needs additional assumptions, e.g., unbiased, uncorrelated, and identically distributed estimation errors for the posterior probability distribution [21, 22]. Nevertheless, these assumptions are impractical [22]. In terms of metrics other than 0-1 loss, [23] uses the entropy of the predictive distribution as uncertainty, deriving the decomposition of data uncertainty and knowledge uncertainty of ensembles. Several other works [24, 25, 26] also discuss different forms of ensemble performance decomposition for metrics other than 0-1 loss. Despite these advances in analyzing ensembles, the analysis of the selective risk of the ensemble remains under-explored.
Other related topic of selective classification include Out-Of-Distribution Detection (OOD Detection) and Calibration, which along with selective classification are subdomains of uncertainty estimation [27]. OOD detection (also known as open set recognition) aims to detect test samples with semantic distribution shift (e.g., occurrence of new classes) without losing the in-distribution classification accuracy [12]. OOD detection and selective classification share the requirement for in-distribution classification, and their difference lies in their mutually exclusive detection targets: the former detects misclassified in-distribution samples; while the latter detects samples with semantic distribution shifts. Calibration tackles the problem of predicting confidence scores that are representative of the true correctness probabilities [28]. Calibration and selective classification focus on different dimensions of uncertain estimation: calibration focuses on the overall confidence level; while selective classification focuses on the relative confidence ranking among the samples [10].
4 The Dominant Source of Performance Improvement of Deep Ensemble
Before we analyze the selective risk of Deep Ensemble, we present an experimental result that helps us to focus on the dominant source of performance improvement of Deep Ensemble. The experiment is to examine whether the performance improvement of the ensemble dominantly comes from high-ambiguity samples, where ambiguity is quantified as  $S = \sqrt{\frac{\sum_{i=1}^{M} \|\pi_i - \pi_E\|^2}{M-1}}$ . We conduct the experiment on datasets CIFAR-10, CIFAR-100 [29], SVHN [30], MRPC [31], MNLI [32], and QNLI [33] (the details of the datasets and the models are provided in Section 6). The procedure of this experiment is as follows. First, we train a member model and an ensemble, and feed them test data. Second, all test samples are sorted by their ambiguities, and we define samples with the top-50% ambiguities as high-ambiguity samples and the rest 50% samples as low-ambiguity samples<sup>2</sup>. Third, we construct ensembling on high-ambiguity samples, i.e., combining the member models (ensembling) on high-ambiguity samples and using a member model on low-ambiguity samples. Mathematically, denoting the median of ambiguities on the dataset as  $t$ , ensembling on
To intuitively see how large are the top-50% ambiguities, we plot the distribution of ambiguities on each dataset in Appendix F.
(f)
Figure 1: The risk-coverage curves of the member model, Deep Ensemble, ensembling on high-ambiguity samples, and ensembling on low-ambiguity samples on each dataset, where top/lowest-50%-S ens. represents ensembling on high/low-ambiguity samples.
high-ambiguity samples induces a model  $\tilde{f}_E$  that makes predictions as
$$
\tilde {f} _ {E} (x) = \left\{ \begin{array}{l} f _ {E} (x), (\text {e c h o i n g t h e p r e d i c t i o n o f t h e e n s e m b l e}) \text {i f} S (x) \geq t; \\ f _ {m} (x), (\text {e c h o i n g t h e p r e d i c t i o n o f a m e m b e r m o d e l}), \end{array} \right.
$$
We equate this model with ensembling on high-ambiguity samples in the following. Similarly, we define ensembling on low-ambiguity samples as combining the member models (ensembling) on low-ambiguity samples and using a member model on high-ambiguity samples. Finally, we plot risk-coverage curves of the member model, the ensemble, ensembling on high-ambiguity samples, and ensembling on low-ambiguity samples.
Figure 1 shows the results of these risk-coverage curves. As we can see, on each dataset, the risk-coverage curve of ensembling on high-ambiguity samples almost coincides with that of the ensemble, and risk-coverage of ensembling on low-ambiguity samples almost coincides with that of the member model. This result suggests that ensembling on high-ambiguity samples provides the dominant performance improvement of the ensemble, while ensembling on low-ambiguity samples provides a minor performance improvement.
This section aims to prove that the selective risk of Deep Ensemble is lower than its member models for any given coverage within a range. If the selective risk is a convex function of member models' predictive probability distributions, then the definition of the ensemble guarantees that the ensemble's selective risk is less than or equal to that of the member model according to Jensen's inequality. However, the selective risk is not a convex function of member models' predictive probability distributions due to the  $0/1$  loss. Thus, the definition of the ensemble is not sufficient to prove the superiority of the ensemble. The same problem exists in the analysis of error rate in standard classification tasks. To address this problem in classification, three unrealistic assumptions: unbiased estimation error, identically distributed estimation error, and uncorrelated estimation error have been proposed [22]. However, we develop other assumptions that are more realistic to make our analysis practical.
5.1 Intuition
Our first assumption is motivated by the empirical result in Section 4. In the previous experiment, the samples are divided into two categories: low-ambiguity samples and high-ambiguity samples. The result shows that low-ambiguity samples bring little performance improvement, but high-ambiguity samples bring the dominant performance improvement of the ensemble. For the convenience of theoretical analysis, we approximate these two types of samples as follows: 1. definite samples on which all member models produce the same predictive probability distribution; 2. ambiguous samples on which member models produce statistically-dependent predictive probability distributions (but their joint distribution is unknown).
Based on this assumption, Figure 2 illustrates the intuition of why ensembling works. Given a target coverage, the corresponding confidence thresholds of the member model and the ensemble are shown as the dashed green line and the solid green line, respectively. The solid blue line represents the PDF of the member's (as well as the ensemble's) confidence score over definite samples, and the dashed/solid red line represents the PDF of the member/ensemble model's confidence score over ambiguous samples. Thus, in the upper subfigure, the area under the blue/red line on the right-hand side of the dashed green line is the probability of the member model accepting (i.e., not rejecting) the sample given the sample being definite/ambiguous; and a similar result applies to the ensemble in the lower subfigure. In addition, we assume that the member model is not modest over ambiguous samples and over definite samples, i.e., the right-hand ends of the dashed red line and blue line are above the horizontal axis. We can prove that the ensemble prevents high confidence scores (termed as being modest) over ambiguous samples, i.e., the right-hand end of the solid red line falls on the horizontal axis. Thus, through ensembling, the PDF over ambiguous samples moves down from the dashed red curve to the solid red curve when the confidence score is near 1 (as the red arrows show). On the contrary, the PDF over definite samples stays the same. Thus, if the target coverage is small so that the threshold of the member model is near 1, within accepted samples (on the right-hand side of the dashed green line), the proportion of ambiguous samples drops. The threshold of the ensemble model also drops to retain the target coverage, but the net effect of these two changes is a lower proportion of ambiguous samples (or a higher proportion of definite samples) within accepted samples. In addition, we assume that the error rate of classifiers over definite samples is lower than that over ambiguous samples. Thus, the selective risk becomes smaller through ensembling given this coverage.
In the following, we first introduce the assumptions, then we show that the ensemble is modest over ambiguous samples, which leads to an upper bound of ensembles' selective risk, and finally we prove the superiority of the ensemble model.
5.2 Assumptions
This section follows the notations of Section 2. Assume that  $(f_{*},\kappa_{*})$  is any member model of  $(f_{E},\kappa_{E})$ , where both  $\kappa_{*}$  and  $\kappa_{E}$  are MCP confidence estimators, whose outputs are in  $[\frac{1}{K},1]$ . For the convenience of notation, let  $R_{*}(t)$  and  $\phi_{*}(t)$  denote  $(f_{*},\kappa_{*};t)$ 's selective risk and coverage, respectively; and  $R_{E}(t)$  and  $\phi_E(t)$  denote  $(f_{E},\kappa_{E};t)$ 's selective risk and coverage, respectively, where  $t$  is a confidence threshold.
Figure 2: An illustration of the intuition of our analysis. The upper/ lower graph shows the PDF of the member/ensemble model's confidence score over ambiguous samples and definite samples.
According to the motivation in Section 4, we assume that the whole data distribution comprises two types of samples: 1. definite samples for which all member models provide the same predictive probability distribution; 2. ambiguous samples for which member models provide dependent predictive probability distributions (but their dependency is unknown). Formally, we have
Assumption 1. (Correlation and Diversity Assumption) Let  $a(x) \in \{0,1\}$  (or  $a$  for short) be a hidden variable that denotes whether the sample  $x$  is ambiguous ( $a = 0/1$  indicates a definite/ambiguous sample). We assume: first, given  $a = 0$ ,  $\pi_1 = \pi_2 = \dots = \pi_M$ ; second, given
$a = 1, \forall k \in \{1, \dots, K\}, p(\pi_1^k, \pi_2^k, \dots, \pi_M^k | a = 1)$  is bounded, i.e., always finite, where  $p$  denotes probability density function (PDF).
Assumption 1 depicts both the correlation and the diversity of the ensemble. The existence of definite samples provides a strong correlation among ensemble members' predictions, and the existence of ambiguous samples provides both the correlation and the diversity of the member models' predictions. Furthermore, because Assumption 1 does not specify the correlation of ensemble members over ambiguous samples, this assumption is a relaxation of the uncorrelated-estimation-error assumption of [21, 22].
In addition, because ambiguous samples seem more difficult to classify than definite samples, we assume the member model's selective risk on ambiguous samples is greater than that on definite samples:
Assumption 2. (Hardness Assumption) For any member model  $(f_{*},\kappa_{*})$ , we assume  $\lim_{t\to 1^{-}}R_{*}(t|a = 1) > \lim_{t\to 1^{-}}R_{*}(t|a = 0)$ , where  $R_{*}(t|a)$  is the conditional selective risk of  $(f_{*},\kappa_{*};t)$  given  $a$ , i.e.,  $\mathbb{E}_{(x,y)\sim \mathcal{D}}[\mathbf{1}_{y\neq f_{*}(x)}|a,\kappa_{*}(x)\geq t]$ .
Finally, we assume that the member model is not modest over some ambiguous samples and some definite samples:
Assumption 3. (Confidence Assumption) For any member model  $(f_{*},\kappa_{*})$ , we have  $\lim_{t\to 1^{-}}p(\kappa_{*}(x) = t|a = 0) > 0$  and  $\lim_{t\to 1^{-}}p(\kappa_{*}(x) = t|a = 1) > 0$ .
5.3 The Ensemble is Modest over Ambiguous Samples
According to Assumption 3, the member model is not modest over ambiguous samples. However, we show that the ensemble is more modest over ambiguous samples (proof is provided in Appendix A):
Lemma 1. Let  $B := \sum_{k=1}^{K} \sup_{[0,1]^M} p(\pi_1^k, \dots, \pi_M^k | a = 1)$ . Assumption 1 implies  $\forall t \in [0,1]$ ,  $p(\kappa_E(x) = t | a = 1) \leq M^M B(1 - t)^{M-1}$ . Integrating this inequality, we derive  $P(\kappa_E(x) \geq t | a = 1) \leq M^{M-1} B(1 - t)^M$ .
The first inequality in this lemma shows that the ensemble satisfies  $\lim_{t\to 1^{-}}p(\kappa_E(x) = t|a = 1) = 0$  which is the opposite of a member model. In other words, the ensemble is less possible to provide high confidence scores over ambiguous samples than its member models. This result then leads to an upper bound of the ensemble's selective risk.
5.4 An Upper Bound of the Ensemble's Selective Risk
We derive an upper bound of the ensemble's selective risk, demonstrating how members' performance and their diversity affect the ensemble's performance. According to selective risk's definition, the ensemble's selective risk can be bounded as (see Appendix A for proof)
$$
R _ {E} (t) \leq R _ {E} (t | a = 0) + P (a = 1 | \kappa_ {E} (x) \geq t). \tag {2}
$$
The two terms in the right-hand side of (2) can be further reduced to member models' performance and their diversity as follows. First, given  $a = 0$ , we have  $f_{E}(x) = f_{*}(x)$  and  $\kappa_{E}(x) = \kappa_{*}(x)$  (due to Assumption 1), which suggests  $R_{E}(t|a = 0) = R_{*}(t|a = 0)$ . Second, combining the second inequality in Lemma 1 with Bayes' rule, we can bound the second term as follows:
$$
P (a = 1 \mid \kappa_ {E} (x) \geq t) \leq \frac {\gamma \cdot (1 - t) ^ {M}}{\gamma \cdot (1 - t) ^ {M} + P \left(\kappa_ {*} (x) \geq t , a = 0\right)}, \tag {3}
$$
where  $\gamma = BM^{M - 1} \cdot P(a = 1)$ . The right-hand side of (3) is negatively related to the diversity of member models. The reason of this statement is as follows. More diverse member models may result in a flatter  $p(\pi_1^k, \dots, \pi_M^k | a = 1)$ , leading to a lower  $B$ . A lower  $B$  further leads to a lower right-hand side of (3). Combining the results above, we have
Lemma 2. (Selective Risk Bound of The Ensemble) The ensemble's selective risk is bounded as
$$
R _ {E} (t) \leq R _ {*} (t | a = 0) + \frac {\gamma \cdot (1 - t) ^ {M}}{\gamma \cdot (1 - t) ^ {M} + P \left(\kappa_ {*} (x) \geq t , a = 0\right)}. \tag {4}
$$
(b)
Figure 3: The member model's selective risks on low-ambiguity samples (denoted as def.) and on high-ambiguity samples (denoted as amb.) given confidence thresholds near 1 on each dataset.