SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yuzhou Cao<sup>1</sup> Hussein Mozannar<sup>2</sup> Lei Feng<sup>1*</sup> Hongxin Wei<sup>3</sup> Bo An<sup>1</sup>
$^{1}$ School of Computer Science and Engineering, Nanyang Technological University, Singapore  
 $^{2}$ CSAIL and IDSS, Massachusetts Institute of Technology, Cambridge, MA  
 $^{3}$ Department of Statistics and Data Science, Southern University of Science and Technology, China yuzhou002@e.ntu.edu.sg, mozannar@mit.edu  
lfengqaq@gmail.com, weihx@sustech.edu.cn, boan@ntu.edu.sg
Enabling machine learning classifiers to defer their decision to a downstream expert when the expert is more accurate will ensure improved safety and performance. This objective can be achieved with the learning-to-defer framework which aims to jointly learn how to classify and how to defer to the expert. In recent studies, it has been theoretically shown that popular estimators for learning to defer parameterized with softmax provide unbounded estimates for the likelihood of deferring which makes them uncalibrated. However, it remains unknown whether this is due to the widely used softmax parameterization and if we can find a softmax-based estimator that is both statistically consistent and possesses a valid probability estimator. In this work, we first show that the cause of the miscalibrated and unbounded estimator in prior literature is due to the symmetric nature of the surrogate losses used and not due to softmax. We then propose a novel statistically consistent asymmetric softmax-based surrogate loss that can produce valid estimates without the issue of unboundedness. We further analyze the non-asymptotic properties of our method and empirically validate its performance and calibration on benchmark datasets.
As machine learning models get deployed in risk-critical tasks such as autonomous driving [18], content moderation [27], and medical diagnosis [20], we have a higher urgency to prevent incorrect predictions. To enable a safer and more accurate system, one solution is to allow the model to defer to a downstream human expert when necessary. The Learning to Defer (L2D) paradigm achieves this aim [28, 30, 31, 9, 44, 32, 33, 45, 3, 35, 15, 42] and enables models to defer to an expert, i.e., abstain from giving a prediction and request a downstream expert for an answer when needed. L2D aims to train an augmented classifier that can choose to defer to an expert when the expert is more accurate or make a prediction without the expert. L2D can be formulated as a risk-minimization problem that minimizes the 0-1-deferral risk [30], which incurs a cost of one when the classifier is incorrect or when we defer to an expert who errs and incurs a cost of zero otherwise.
Despite being formulated straightforwardly, the risk minimization problem is NP-hard even in simple settings [15, 32] due to the discontinuous and non-convex nature of the 0-1-deferral loss. To make the optimization problem tractable, many efforts have been made to design a continuous surrogate loss for the 0-1-deferral loss while guaranteeing statistical consistency, which means that the minimizer of the surrogate risk is that of the 0-1-deferral risk. In Mozannar and Sontag [30], a cross-entropy-like
*Corresponding author: Lei Feng.
(a) Unbounded Estimator [30]
(b) Uncorrelated Estimator [44]
(c) Proposed Estimator
Figure 1: Illustration of the proposed and previous estimators. Probability estimation in L2D aims to predict both the class probabilities and the expert's accuracy  $\pmb{\eta} \times \operatorname{Pr}(M = Y | X = \pmb{x}) \in \Delta^K \times [0,1]$ , while only our proposed estimator takes exactly the same range.
surrogate loss is proposed that is statistically consistent. Charusaie et al. [9] further generalized the results in Mozannar and Sontag [30] and showed that all the losses that are consistent for ordinary multi-class classification, i.e., classification-calibrated [38, 43, 6], are consistent for L2D with certain reformulation, which makes L2D a more comprehensive framework. By minimizing the risk w.r.t. consistent surrogate losses [30, 9], we can obtain a model that defers to who between the classifier and expert is more accurate on an instance basis. However, we also need assessments of the uncertainty of the classifier when it predicts and the uncertainty of the expert prediction when we defer. This allows us to perform better triaging of samples and provide reliable estimates of uncertainty. Verma and Nalisnick [44] pointed out that the previous estimator in [30] generates highly biased probability estimates for the predictions due to its unboundedness. They then proposed a surrogate based on the One-versus-All (OvA) strategy [52] to alleviate this problem, which allows both statistical consistency and improved calibration compared to the previous softmax-based method [30].
Does the difference in performance between OvA and softmax-based methods indicate that softmax parameterization is a sub-optimal choice in L2D? The cause of the unbounded probability estimation in [30] is in fact still elusive, and we cannot simply attribute it to softmax parameterization. Furthermore, notice that the softmax-based method [30] is only a specific implementation of applying softmax in L2D. Meanwhile, softmax parameterization is also a more straightforward modeling of multiclass posterior distributions: while the OvA strategy works by splitting a  $K$ -class classification problem into  $K$  independent binary classification problems and thus induces  $K$  uncorrelated probability estimators for each class, softmax parameterization works by directly estimating class posterior probabilities as a whole. Given the wide use and practical advantages of softmax parameterization in classification tasks and the resemblance between L2D and multiclass classification, it is also promising that the usage of the softmax function can result in competitive methods for L2D. Then a natural question arises: can we design a softmax-based consistent surrogate loss for L2D without triggering unbounded probability estimation enabling calibrated estimates?
In this paper, we give a positive answer to this question by providing a novel consistent surrogate loss with asymmetric softmax parameterization, which can be seen as the combination of ordinary softmax function and an additional probability estimator. This surrogate can also induce a non-trivial bounded probability estimator for prediction probabilities and expert accuracy. To defend the use of softmax parameterization, we show that it is not the cause of the failure but rather it is the symmetric structure of the surrogates that leads to the unboundedness in probability estimation. Unlike the unbounded estimator [30] that mixes up class probabilities and the expert's accuracy and the OvA estimator that treats each class and expert independently, our method models the class probabilities as a whole with softmax and the expert's accuracy independently with a normalized function, which better reflects the structure of L2D with probability estimation. The differences between our work and previous works are illustrated in Figure 1. We further analyze the limitation and broader impact of our work in Appendix G. Our contributions are four-fold and summarized below:
- In Section 3, we prove that a non-trivial bounded probability estimator does not exist as long as we are using symmetric loss functions.
- We propose an asymmetric formulation of softmax parameterization, and show that it can induce both a consistent surrogate loss and a non-trivial bounded probability estimator. We further show that the proposed asymmetric softmax-based method and OvA-based method [44] actually benefit from their asymmetric structure.
- We further study the regret transfer bounds of our proposed asymmetric surrogate and show that it is compatible with both L2D and multi-class classification.
- Experiments on datasets with both synthetic experts and real-world experts are conducted to demonstrate the usefulness of our method for both prediction and expert accuracy estimation.
In this section, we review the problem setting of L2D and briefly introduce previous works.
In this paper, we study the problem of learning to defer to an expert in the  $K$ -class classification scenario. Let us denote by  $\mathcal{X}$  and  $\mathcal{Y} = \{1,\dots ,K\}$  the feature and label space respectively. Denote by  $X\times Y\times M\in \mathcal{X}\times \mathcal{Y}\times \mathcal{Y}$  the data-label-expert random variable triplet and  $\pmb {x},y,m$  are their realizations, which obeys an underlying distribution with density  $p(\pmb {x},y,m)$ . We have access to data triplets  $\{(x_i,y_i,m_i)\}_{i = 1}^n$  that are drawn independently and identically from the distribution. The goal of L2D in the classification scenario is to obtain a classifier with deferral option  $f(\cdot):\mathcal{X}\to \mathcal{Y}^{\perp}$ , where  $\bot$  is the option of deferral to the expert and  $\mathcal{V}^{\perp}$  is the augmented decision space  $\mathcal{V}\cup \{\bot \}$ .
The problem evaluation is the following 0-1-deferral loss  $\ell_{01}^{\perp 2}$ , which is a generalized version of the zero-one loss  $\ell_{01}(f(\boldsymbol{x}),y) = [[f(\boldsymbol{x})\neq y]]$ , which takes the value of 1 if we use the output of the classifier when it predicts an incorrect label  $([[f(\boldsymbol{x})\in \mathcal{V}$  and  $f(\boldsymbol {x})\neq y]])$  or defer to the expert when they are incorrect  $([[f(\boldsymbol {x}) = \bot$  and  $m\neq y]]$ ), where  $[\cdot ]$  is the Iverson bracket notation suggested by Knuth [22]. Then we aim to minimize the risk w.r.t. to this loss:
$$
\min  _ {f} R _ {0 1} ^ {\perp} (f) = \mathbb {E} _ {p (\boldsymbol {x}, y, m)} \left[ \ell_ {0 1} ^ {\perp} (f (\boldsymbol {x}), y, m) \right]. \tag {1}
$$
Denote by  $f^{*}$  the minimizer of the risk  $R_{01}^{\perp}(f)$ , i.e., the Bayes optimal solution, and  $\boldsymbol{\eta}(\boldsymbol{x}) = \{\operatorname{Pr}(Y = y | X = \boldsymbol{x})\}_{y=1}^{K}$ . Mozannar and Sontag [30] provide a characterization of the form of the Bayes optimal solution as:
Definition 1. (Bayes optimality of L2D) A classifier with deferral option  $f^{*} \to \mathcal{V}^{\perp}$  is the minimizer of  $R_{01}^{c}(f)$  if and only if it meets the following condition almost surely:
$$
f ^ {*} (\boldsymbol {x}) = \left\{ \begin{array}{l l} \perp , & \max  _ {y} \eta_ {y} (\boldsymbol {x}) <   \operatorname * {P r} (M = Y | X = \boldsymbol {x}), \\ \operatorname * {a r g m a x} _ {y} \eta_ {y} (\boldsymbol {x}), & \text {e l s e}. \end{array} \right.
$$
The form of the optimal solution is intuitive: we should defer to the expert if they have a higher probability of being correct than the optimal classifier which follows the most likely label class given the input  $x$ . This optimal solution can also be seen as the generalized version of Chow's rule in learning to reject [10], where a fixed cost in [0, 1] serves as the known accuracy of an expert.
2.2 Consistent Surrogate Losses for L2D
Although we know the form of the Bayes optimal solution, practically the risk minimization problem above faces computational difficulties: the minimization of the discontinuous and non-convex problem (1) is NP-hard [32]. A widely used strategy for tackling this difficulty is to substitute the original loss function which is discontinuous with a continuous surrogate, which has been applied in many areas including ordinary classification [52, 6, 43, 39, 16, 37], multi-label classification [17, 23, 51, 47], AUC optimization [17, 23, 29], cost-sensitive learning [41, 11], top- $K$  classification [25, 48], adversarially robust classification [4, 2, 1], and learning to reject [12, 13, 5, 49, 34, 8, 7]. Denote by  $g: \mathcal{X} \to \mathbb{R}^{K+1}$  the learnable scoring function that induces our decision function for L2D  $f: \mathcal{X} \to \mathcal{Y}^{\perp}$  with the following transformation  $\varphi: \mathbb{R}^{K+1} \to \mathcal{Y}^{\perp}$ :
$$
\varphi (\boldsymbol {g} (\boldsymbol {x})) = \left\{ \begin{array}{l l} \perp , & g _ {K + 1} (\boldsymbol {x}) > \max  _ {y \in \mathcal {Y}} g _ {y} (\boldsymbol {x}) \\ \operatorname * {a r g m a x} _ {y \in \mathcal {Y}} g _ {y} (\boldsymbol {x}), & \text {e l s e .} \end{array} \right.
$$
2Though the formulation introduced here is not its most general version, we only study this one and its induced surrogates (4) since previous works on the design of surrogate losses for L2D [30, 9, 44] also concentrated on it.
We consider a continuous surrogate function  $\Phi : \mathbb{R}^{K + 1} \times \mathcal{Y} \times \mathcal{Y} \to \mathbb{R}^+$  and the surrogate risk below:
$$
\min  _ {\boldsymbol {g}} R _ {\Phi} ^ {\perp} (\boldsymbol {g}) = \mathbb {E} _ {p (\boldsymbol {x}, y, m)} [ \Phi (\boldsymbol {g} (\boldsymbol {x}), y, m) ]. \tag {2}
$$
Assuming we find a surrogate function that overcomes the computational optimization challenges, we need to verify the consistency of  $\Phi$  w.r.t.  $\ell_{01}^{\perp}$ , i.e., any minimizer of the surrogate risk also minimizes the original risk: (2):
$$
\varphi \circ \boldsymbol {g} ^ {*} \in \operatorname * {a r g m i n} _ {f} R _ {0 1} ^ {\perp} (f), \quad \forall \boldsymbol {g} ^ {*} \in \operatorname * {a r g m i n} _ {\boldsymbol {g}} R _ {\Phi} ^ {\perp} (\boldsymbol {g}).
$$
The first consistent surrogate [30] w.r.t.  $\ell_{01}^{\perp}$  is proposed by modifying the softmax cross-entropy loss:
$$
L _ {\mathrm {C E}} (\boldsymbol {g} (\boldsymbol {x}), y, m) = - \log \psi_ {y} ^ {\mathrm {s m}} (\boldsymbol {g} (\boldsymbol {x})) - [   [ m = y ]   ] \log \psi_ {K + 1} ^ {\mathrm {s m}} (\boldsymbol {g} (\boldsymbol {x})), \tag {3}
$$
where  $\psi^{\mathrm{sm}}$  is the softmax function  $\psi_y^{\mathrm{sm}}(\pmb {u}) = \exp (u_y) / \sum_{y' = 1}^{d}\exp (u_{y'})$ , where  $d$  is the dimensionality of the input. Inspired by the risk formulation above, Charusaie et al. [9] further generalized the family of consistent surrogate for  $\ell_{01}^{\perp}$  by considering the following consistent surrogate reformulation:
$$
L _ {\phi} (\boldsymbol {g} (\boldsymbol {x}), y, m) = \phi (\boldsymbol {g} (\boldsymbol {x}), y) + [   [ m = y ]   ] \phi (\boldsymbol {g} (\boldsymbol {x}), K + 1). \tag {4}
$$
It is known from Proposition 2 in Charusaie et al. [9] that surrogates take the form above are consistent w.r.t.  $\ell_{01}^{\perp}$  if  $\phi$  is a classification-calibrated multi-class loss [43, 6]. Using this result, we can directly make use of any statistically valid surrogate loss in ordinary classification with a simple modification.
2.3 Problems with Probability Estimation for L2D
While prior literature has established how to construct consistent surrogates for L2D, it is less well-known whether such surrogates can actually provide calibrated estimates of classifier and deferral probabilities. As mentioned before, we are also interested in the true probability of the correctness of our prediction, i.e., the value of  $\eta (\pmb {x})$  (label being  $Y$ ) and  $\operatorname *{Pr}(M = Y|X = x)$  (expert is correct). To achieve this goal of probability estimation, the commonly used method is to combine the obtained scoring function  $\pmb{g}^{*}$  with a transformation function  $\psi$  to make the composite function  $\psi \circ g^{*}$  a probability estimator  $\hat{\eta}^{\psi}(\pmb {g}(\pmb {x}))$ . For example, the softmax function is frequently used as  $\psi$  in ordinary multi-class classification to map the scoring function from  $\mathbb{R}^K$  to  $\Delta^K$ . So far, we have not discussed whether these probability estimates are valid.
In the task of L2D for classification, we aim to estimate both  $\pmb{\eta}(\pmb{x}) \in \Delta^K$  and  $\operatorname{Pr}(M = Y|X = \pmb{x}) \in [0,1]$  with a  $K + 1$ -dimensional estimator, where its  $y$ -th dimension is the estimate of  $\eta_y(\pmb{x})$  for  $y \in \mathcal{V}$  and  $K + 1$ -th dimension is the estimate of  $\operatorname{Pr}(M = Y|X = \pmb{x})$ . Correspondingly, the transformation  $\psi$  should be from  $\mathbb{R}^{K + 1}$  to a range  $\mathcal{P}$  that contains  $\Delta^K \times [0,1]$ . It was shown in Theorem 1 of Mozannar and Sontag [30] that a probability estimator  $\hat{\eta}^{\mathrm{sm}}$  with the softmax output of scorer function  $\pmb{g}$  and an extra fractional transformation guarantees to recover class probabilities and expert accuracy at  $\pmb{g}^* \in \mathrm{argmin}_{\pmb{g}} R_{L_{CE}}(\pmb{g})$ , which is formulated as:
$$
\hat {\eta} _ {y} ^ {\mathrm {s m}} (\boldsymbol {g} (\boldsymbol {x})) = \psi_ {y} ^ {\mathrm {s m}} (\boldsymbol {g} (\boldsymbol {x})) / \left(1 - \psi_ {K + 1} ^ {\mathrm {s m}} (\boldsymbol {g} (\boldsymbol {x}))\right), \quad \forall y \in \mathcal {Y} \cup \{K + 1 \}, \tag {5}
$$
It is noticeable that the range of this estimator is  $\hat{\eta}^{\mathrm{sm}}\in \Delta^K\times [0, + \infty ]$  : the estimate of expert accuracy  $\hat{\eta}_{K + 1}^{sm}$  is unbounded above and will approach  $+\infty$  if  $\psi_{K + 1}^{\mathrm{sm}}(\pmb {u})\to 1$  . Verma and Nalisnick [44] pointed out that the unboundedness can hurt the performance of this probability estimator: if  $\psi_{K + 1}^{\mathrm{sm}}(\pmb {g}(\pmb {x})) > 1 / 2$  , the estimated expert accuracy will be greater than 1 and thus meaningless. Experimental results also show the frequent occurrence of such meaningless results due to the overconfidence of deep models [19]. We further illustrate such miscalibration in Figure 2.
To mitigate this problem, a new OvA-based surrogate loss that can induce a bounded probability estimator while remaining consistent is proposed in [44], which has the following formulation:
$$
L _ {\mathrm {O v A}} (\boldsymbol {g} (\boldsymbol {x}), y, m) = \xi \left(g _ {y} (\boldsymbol {x})\right) + \sum_ {y ^ {\prime} \neq y} ^ {K + 1} \xi \left(- g _ {y ^ {\prime}} (\boldsymbol {x})\right) + [   [ m = y ]   ] \left(\xi \left(g _ {K + 1} (\boldsymbol {x})\right) - \xi \left(- g _ {K + 1} (\boldsymbol {x})\right)\right), \tag {6}
$$
where  $\xi$  is a binary proper composite [46, 40] loss. Its induced probability estimator  $\hat{\eta}^{\mathrm{OvA}}$  is:
$$
\hat {\eta} _ {y} ^ {\mathrm {O v A}} (\boldsymbol {g} (\boldsymbol {x})) = \psi_ {\xi} \left(g _ {y} (\boldsymbol {x})\right), \quad \forall y \in \mathcal {Y} \cup \{K + 1 \}, \tag {7}
$$
where  $\psi_{\xi}$  is a mapping to [0, 1] determined by the binary loss, which makes  $\hat{\eta}^{\mathrm{OvA}}$  bounded. It is also experimentally shown that the  $\hat{\eta}^{\mathrm{OvA}}$  outperformed  $\hat{\eta}^{\mathrm{sm}}$  [30] in the task of probability estimation.
Figure 2: Illustration of the miscalibration of estimator (5) on a binary classification with deferral problem. The unbounded estimator (5) first estimates  $\frac{[\eta(\pmb{x});\mathrm{Pr}(M = Y|X = \pmb{x})]}{1 + \mathrm{Pr}(M = Y|X = \pmb{x})}$  that takes value in the 2-D probability simplex denoted by the left triangle and then obtain the final estimate with the fractional transformation (5). However, when over-confidence occurs in the 2-D simplex, i.e., the softmax output lies in the invalid region that  $\psi_3^{\mathrm{sm}} > 1 / 2$  (5) will magnify the calibration error of the estimates and clipping it to a valid estimate cannot solve this problem.
Given the success of the OvA-based loss, can we assert that softmax parameterization is inferior to the OvA strategy in probability estimation for L2D? We argue in this paper that such a conclusion should not be drawn so quickly. We should not discourage the use of softmax parameterization based solely on the specific implementation (3), since there may be other implementations of softmax that can resolve the issues with  $\hat{\eta}^{\mathrm{sm}}$ . Furthermore, the great success of softmax parameterization in deep learning models suggests that this potential implementation could also achieve outstanding performance. In this paper, we focus on finding such an implementation and show its superiority both theoretically and experimentally.
3 Problem with Symmetric Losses for L2D with Probability Estimation
Before beginning the search for a suitable implementation of softmax parameterization for L2D, it is important to determine the cause of the unboundedness of  $\hat{\eta}^{\mathrm{sm}}$ . Is it due to the use of softmax parameterization or some other factor? If the reason is the former one, any attempts to create a bounded probability estimator with a softmax function for L2D will be in vain. Therefore, it is crucial to identify the root cause of the unboundedness before proceeding with further efforts.
Surprisingly, we find that such unboundedness is not a particular problem of softmax parameterization and is a common one shared by many loss functions. Recall that the loss (3) with unbounded estimator  $\hat{\eta}^{\mathrm{sm}}$  is a special case of the consistent surrogate (4) by setting the multi-class loss  $\phi$  to softmax cross-entropy loss. We show that even if we use other losses beyond the softmax function and choose other losses such as the standard OvA losses ((14) in Zhang [52]), the induced probability estimators are still inevitably unbounded (or bounded but induced from an unbounded one) as long as we are using  $\phi$  with symmetric structure:
Theorem 1. (Impossibility of non-trivial bounded probability estimator with symmetric losses) If a surrogate loss  $L_{\phi}$  defined as (4) has probability estimators and is induced from a symmetric consistent multi-class loss  $\phi$  such that that  $P\phi(\boldsymbol{u}) = \phi(P\boldsymbol{u})$ , where  $P$  is any permutation matrix, it must have an unbounded probability estimator  $\hat{\eta}$ : let  $\boldsymbol{g}^* \in \mathrm{argmin} R_{L_{\phi}}(\boldsymbol{g})$ , we have that  $\forall \hat{\eta}(\cdot)$  where  $\hat{\eta}(\boldsymbol{g}^*(\boldsymbol{x})) = [\boldsymbol{\eta}(\boldsymbol{x}); \operatorname*{Pr}(M = Y | X = \boldsymbol{x})]$ ,  $\hat{\eta}$  is not bounded above. Furthermore, any bounded probability estimator  $\hat{\eta}'$  for  $L_{\phi}$  must be a piecewise modification of the unbounded estimator  $\hat{\eta}$ : denote by  $\mathcal{U} = \cup_{\beta} \mathrm{argmin}_{\boldsymbol{u}} \sum_{y=1}^{K+1} \beta_y L_{\phi}(\boldsymbol{u}, y)$  for all  $\beta \in \Delta^K \times [0,1]$ , we have that  $\hat{\eta}'$  is equal to  $\hat{\eta}$  on  $\mathcal{U}$ .
The proof is provided in Appendix A. Intuitively, the existence of such an unbounded estimator is caused by the fact that symmetric loss induced  $L_{\phi}$  neglects the asymmetric structure of the probabilities to be estimated by modeling  $\frac{[\eta(\boldsymbol{x}) : \operatorname*{Pr}(M = Y | X = \boldsymbol{x})]}{1 + \operatorname*{Pr}(M = Y | X = \boldsymbol{x})}$  directly. Furthermore, all potential bounded probability estimators are piecewise modifications of the unbounded estimator sharing the same values on  $\mathcal{U}$  with  $\hat{\eta}$ , which means that they are generated by substituting the invalid values of the unbounded estimator with a valid one, e.g., we can obtain a bounded estimator based on the unbounded one (5) by clipping it to 1 if its estimated expert accuracy is larger than 1.
However, such modifications to make an unbounded estimator bounded are not very useful. Due to the training process's complexity and the distribution's arbitrariness, it is hard to design an appropriate modification of an unbounded estimator. For example, though we can bound the expert accuracy of (5) by 1 when it generates an invalid value that is larger than 1, it is still an overly confident estimation. Meanwhile, we do not know how to upper bound it by a value lower than 1 without prior knowledge of  $\operatorname*{Pr}(M = Y|X = x)$ . Overall, training with  $L_{\phi}$  in Theorem 1 can easily lead to a model that generates invalid probability estimation with an unbounded estimator  $\hat{\eta}$ , while most modified bounded estimators cannot efficiently solve this problem. We will experimentally verify this point in Section 5.