SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jarosław Blasiok
jb4451@columbia.edu
Parikshit Gopalan
parik.g@gmail.com
Preetum Nakkiran
preetum.nakkiran@gmail.com
Optimizing proper loss functions is popularly believed to yield predictors with good calibration properties; the intuition being that for such losses, the global optimum is to predict the ground-truth probabilities, which is indeed calibrated. However, typical machine learning models are trained to approximately minimize loss over restricted families of predictors, that are unlikely to contain the ground truth. Under what circumstances does optimizing proper loss over a restricted family yield calibrated models? What precise calibration guarantees does it give? In this work, we provide a rigorous answer to these questions. We replace the global optimality with a local optimality condition stipulating that the (proper) loss of the predictor cannot be reduced much by post-processing its predictions with a certain family of Lipschitz functions. We show that any predictor with this local optimality satisfies smooth calibration as defined in Kakade and Foster [2008], Blasiok et al. [2023b]. Local optimality is plausibly satisfied by well-trained DNNs, which suggests an explanation for why they are calibrated from proper loss minimization alone. Finally, we show that the connection between local optimality and calibration error goes both ways: nearly calibrated predictors are also nearly locally optimal.
In supervised prediction with binary labels, two basic criteria by which we judge the quality of a predictor are accuracy and calibration. Given samples from a distribution  $\mathcal{D}$  on  $\mathcal{X} \times \{0,1\}$  that corresponds to points  $x$  from  $\mathcal{X}$  with binary labels  $y \in \{0,1\}$ , we wish to learn a predictor  $f$  that assigns each  $x$  a probability  $f(x) \in [0,1]$  that the label is 1. Informally, accuracy measures how close the predictor  $f$  is to the ground-truth  $f^{*}(x) = \mathbb{E}[y|x]$ . Calibration [Dawid, 1982, Foster and Vohra, 1998] is an interpretability notion originating from the literature on forecasting, which stipulates that the predictions of our model be meaningful as probabilities. For instance, on all points where  $f(x) = 0.4$ , calibration requires the true label to be 1 about  $40\%$  of the time. Calibration and accuracy are complementary notions: a reasonably accurate predictor (which is still far from optimal) need not be calibrated, and calibrated predictors (e.g. predicting the average) can have poor accuracy. The notions are complementary but not necessarily conflicting; for example, the ground truth itself is both optimally accurate and perfectly calibrated.
Proper losses are central to the quest for accurate approximations of the ground truth. Informally a proper loss ensures that under binary labels  $y$  drawn from the Bernoulli distribution with parameter  $v^{*}$
Here we focus on the accuracy of the predictor  $f$  as opposed to the accuracy of a classifier induced by  $f$ .
the expected loss  $\mathbb{E}[\ell(y, v)]$  is minimized at  $v = v^{*}$  itself. Familiar examples include the squared loss  $\ell_{\mathrm{sq}}(y, v) = (y - v)^2$  and the cross-entropy loss  $\ell_{\mathrm{xent}}(y, v) = y \log(1 / v) + (1 - y) \log(1 / (1 - v))$ . A key property of proper losses is that over the space of all predictors, the expected loss  $\mathbb{E}_{\mathcal{D}}[\ell(y, f(x))]$  is minimized by the ground truth  $f^{*}$  which also gives calibration. However minimizing over all predictors is infeasible. Hence typical machine learning deviates from this ideal in many ways: we restrict ourselves to models of bounded capacity such as decision trees or neural nets of a given depth and architecture, and we optimize loss using algorithms like SGD, which are only expected to find approximate local minima in the parameter space. What calibration guarantees transfer to this more restricted but realistic setting?
The State of Calibration & Proper Losses. There is a folklore belief in some quarters that optimizing a proper loss, even in the restricted setting, will produce a calibrated predictor. Indeed, the user guide for scikit-learn [Pedregosa et al., 2011] claims that LogisticRegression returns well calibrated predictions by default as it directly optimizes Log loss<sup>2</sup>. Before going further, we should point out that this statement in its full generality is just not true: for general function families  $\mathcal{C}$ , minimizing proper loss (even globally) over  $\mathcal{C}$  might not result in a calibrated predictor. Even if the class  $\mathcal{C}$  contains perfectly calibrated predictors, the predictor found by loss minimization need not be (even close to) calibrated. A simple example showing that even logistic regression can fail to give calibrated predictors is provided in Appendix B.
Other papers suggest a close relation between minimizing a proper loss and calibration, but stop short of formalizing or quantifying the relationship. For example, in deep learning, Lakshminarayanan et al. [2017] states calibration can be incentivised by proper scoring rules. There is also a large body of work on post-hoc recalibration methods. The recipe here is to first train a predictor via loss minimization over the training set, then compose it with a simple post-processing function chosen to optimize cross-entropy on a holdout set [Platt, 1999, Zadrozny and Elkan, 2002]. See for instance Platt scaling [Platt, 1999], where the post-processing functions are sigmoids, and which continues to be used in deep learning (e.g. for temperature scaling as in Guo et al. [2017]). Google's data science practitioner's guide recommends that, for recalibration methods, the calibration [objective] function should minimize a strictly proper scoring rule [Richardson and Pospisil, 2021]. In short, these works suggest recalibration by minimizing a proper loss, sometimes in conjunction with a simple family of post-processing functions. However, there are no rigorous bounds on the calibration error using these methods, nor is there justification for a particular choice of post-processing functions.
Yet despite the lack of rigorous bounds, there are strong hints from practice that in certain settings, optimizing a proper-loss does indeed yield calibrated predictors. Perhaps the strongest evidence comes from the newest generation of deep neural networks (DNNs). These networks are trained to minimize a proper loss (usually cross-entropy) using SGD or variants, and are typically quite far from the ground truth, yet they turn out to be surprisingly well-calibrated. This empirical phenomenon occurs in both modern image classifiers [Minderer et al., 2021, Hendrycks* et al., 2020], and language models [Desai and Durrett, 2020, OpenAI, 2023]. The situation suggests that there is a key theoretical piece missing in our understanding of calibration, to capture why models can be so well-calibrated "out-of-the-box." This theory must be nuanced: not all ways of optimizing a proper loss with DNNs yields calibration. For example, the previous generation of image classifiers were poorly calibrated [Guo et al., 2017], though the much smaller networks before them were well-calibrated [Niculescu-Mizil and Caruana, 2005]. Whether DNNs are calibrated, then depends on the architecture, distribution, and training algorithm—and our theory must be compatible with this.
In summary, prior work suggests a close but complicated relationship between minimizing proper loss and calibration. On one hand, for simple models like logistic regression, proper loss minimization does not guarantee calibration. On the other hand, for DNNs, certain ways (but not all ways) of optimizing a proper loss appears to yield well-calibrated predictors. The goal of our work is to analyze this phenomena from a theoretical perspective.<sup>3</sup> Our motivating question is: What minimal conditions on model family and training procedure guarantee that optimizing for a proper loss provably yields small calibration error?
$^{2}$ https://scikit-learn.org/stable/modules/calibration.html. Log-loss is  $\ell_{\text{xent}}$  in our notation.
3For simplicity, we avoid generalization concerns and consider the setting where we minimize loss directly on the population distribution. This is a reasonable simplification in many learning settings with sufficient samples, such as the 1-epoch training of many Large-Language-Models.
Figure 1: Schematic depiction of the loss landscape of a proper loss on the population distribution. The red dot represents a well-calibrated predictor  $f$ . The blue curve represents all predictors  $\{\kappa \circ f : \kappa \in K_{\ell}\}$  obtained by admissible post-processing of  $f$ . Since the proper loss cannot be decreased by post-processing (the red dot is a minimal point on the blue curve), the predictor  $f$  is well-calibrated, even though it is not a global minimum.
The main contribution of this work is to identify a local optimality condition for a predictor that is both necessary and sufficient to guarantee calibration. The local optimality condition requires that the loss  $\ell$  cannot be reduced significantly through post-processing using a family of functions  $K_{\ell}$  that have certain Lipschitz-ness guarantees, where post-processing means applying a function  $\kappa :[0,1]\to [0,1]$  to the output of the predictor. This condition is illustrated in Figure 1; it is distinct from the standard local optimality condition of vanishing gradients. We prove that a predictor satisfying this local optimality condition is smoothly calibrated in the sense of Kakade and Foster [2008], Gopalan et al. [2022b], Blasiok et al. [2023b]. Smooth calibration is a consistent calibration measure that has several advantages over the commonly used Expected Calibration Error (ECE); see the discussion in Kakade and Foster [2008], Blasiok et al. [2023b]. Quantitatively for the case of the squared loss, we present a tight connection between smooth calibration error and the reduction in loss that is possible from post-processing with  $K_{\ell}$ . For other proper loss functions  $\ell$ , we give a tight connection between post-processing with  $K_{\ell}$  and a calibration notion we call dual smooth calibration. The fact that the connection goes both ways implies that a predictor that does not satisfy our local optimality condition is far from calibrated, so we have indeed identified the minimal conditions needed to ensure calibration.
Implications. Heuristically, we believe these theoretical results shed light on why modern DNNs are often calibrated in practice. There are at least three possible mechanisms, at varying levels of formality.
First, informally: poor calibration implies that the test loss can be reduced noticeably by post-processing with a simple function. But simple post-processings can be represented by adding a few layers to the DNN—so we could plausibly improve the test loss by adding a few layers and re-training (or continuing training) with SGD. Thus, if our DNN has been trained to the point where such easy gains are not left on the table (as we expect of state-of-the-art models), then it is well-calibrated.
The second possible mechanism follows from one way of formalizing the above heuristic, yielding natural learning algorithms that provably achieve calibration. Specifically, consider Structural Risk Minimization (SRM): globally minimizing a proper loss plus a complexity regularizer, within some restricted function family. In Section 5, we prove SRM is guaranteed to produce a calibrated predictor, provided: (1) the function family is closed under post-processing and (2) the complexity regularizer grows only mildly under post-processing. If the informal "implicit regularization hypothesis" in deep learning is true, and SGD on DNNs is equivalent to SRM with an appropriate complexity-regularizer [Arora et al., 2019, Neyshabur et al., 2014, Neyshabur, 2017, Zhang et al., 2021], then our results imply DNNs are well-calibrated as long as the regularizer satisfies our fairly mild assumptions.
Finally, a third mechanism for calibration involves a heuristic assumption about the optimizer (SGD). Informally, for a sufficiently deep network, updating the network parameters from computing the function  $f$  to computing the post-processed  $(\kappa \circ f)$  may be a "simple" update for SGD. Thus, if it were possible to improve the loss via such a simple post-processing, then SGD would have already exploited this by the end of training.
These algorithms and intuitions also suggest a practical guidance: for calibration, one should optimize loss over function families that are capable of computing rich classes of post-processing functions. The universality properties of most DNN architectures imply they can compute a rich family of post-processing functions with small added depth [Cybenko, 1989, Lu et al., 2017, Yun et al., 2019, Zhou, 2020] but the family of functions used in logistic regression (with logits restricted to being affine in the features) cannot.
Our results are consistent with the calibration differences between current and previous generation models, when viewed through the lens of generalization. In both settings, models are expected to be locally-optimal with respect to train loss [Carrell et al., 2022], since this is the explicit optimization objective. Calibration, however, requires local-optimality with respect to test loss. Previous generation image classification models were trained to interpolate on small datasets, so optimizing the train loss was very different from optimizing the test loss [e.g. Guo et al., 2017, Mukhoti et al., 2020]. Current generation models, in contrast, are trained on massive datasets, where optimizing the train loss is effectively equivalent to optimizing the test loss—and so models are close to locally-optimal with respect to both, implying calibration.
Organization. We first give a technical overview of our results in Section 2. In Section 3 we prove our main result in the case of square-loss. Then in Section 4 we extend this result to a wide family of proper losses. Finally, in Section 5 we present natural algorithms which provably achieve calibration, as a consequence by our results. Additional related works are presented in Appendix A.
In this section, we present a formal but still high-level overview of our results. We first set some notation. All our probabilities and expectations are taken with respect to a distribution  $\mathcal{D}$  over  $\mathcal{X} \times \{0,1\}$ . A predictor is a function  $f: \mathcal{X} \to [0,1]$  that assigns probabilities to points, the ground-truth predictor is  $f^{*}(x) = \mathbb{E}[y|x]$ . A predictor is perfectly calibrated if the set  $A = \{v \in [0,1] : \mathbb{E}[y|f(x) = v] \neq v\}$  has measure 0 (i.e.  $\operatorname{Pr}_{\mathcal{D}}(f(x) \in A) = 0$ ). A post-processing function is a function  $\kappa: [0,1] \to [0,1]$ . A loss function is a function  $\ell: \{0,1\} \times [0,1] \to \mathbb{R}$ , where  $\ell(y,v)$  represents the loss suffered when we predict  $v \in [0,1]$  and the label is  $y \in \{0,1\}$ . Such a loss is proper if when  $y \sim \mathrm{Ber}(v^{*})$  is sampled from the Bernoulli distribution with parameter  $v^{*}$ , the expected loss  $\mathbb{E}[\ell(y,v)]$  is minimized at  $v = v^{*}$ , and we say a loss is strictly proper if  $v^{*}$  is the unique minimizer.
As a warm-up, we present a characterization of perfect calibration in terms of perfect local optimally over the space of all possible (non-Lipschitz) post-processing functions.
Claim 2.1. For every strictly proper loss  $\ell$ , a predictor  $f$  is perfectly calibrated iff for every post-processing function  $\kappa$ ,
$$
\underset {\mathcal {D}} {\mathbb {E}} [ \ell (y, f (x)) ] \leq \underset {\mathcal {D}} {\mathbb {E}} [ \ell (y, \kappa (f (x))) ]. \tag {1}
$$
This is true, because if  $f$  is perfectly calibrated, and  $\kappa$  arbitrary, then after conditioning on arbitrary prediction  $f(x)$  we have:
$$
\mathbb {E} [ \ell (y, f (x)) | f (x) = v ] = \underset {y \sim \operatorname {B e r} (v)} {\mathbb {E}} [ \ell (y, v) ] \leq \underset {y \sim \operatorname {B e r} (v)} {\mathbb {E}} [ \ell (y, \kappa (v)) ],
$$
and (1) follows by averaging over  $v$ . On the other hand if  $f$  is not perfectly calibrated, we can improve the expected loss by taking  $\kappa(v) \coloneqq \mathbb{E}[y|f(x) = v]$  — by the definition of the strictly proper loss, for each  $v$  we will have  $\mathbb{E}[\ell(y, \kappa(f(x))|f(x) = v] \leq \mathbb{E}[\ell(y, f(x))|f(x) = v]$ , and on a set of positive probability the inequality will be strict.
While we are not aware of this precise claim appearing previously, statements that are similar in spirit are known; for instance, by the seminal work of Foster and Vohra [1998] which characterizes calibration in terms of swap regret for the squared loss. In our language they show the equivalent Claim 2.1 for  $\ell$  being a squared loss, and  $\kappa$  restricted to be of the form  $\kappa(v_0) := w_0$ ,  $\kappa(v) := v$  for all  $v \neq v_0$ .
Claim 2.1 connects calibration to a local optimality condition over the space of all possible post-processing functions. If this is satisfied, it guarantees calibration even though the loss might be far from the global minimum. This sheds light on classes of predictors such as decision trees or branching programs, which are closed under composition with arbitrary post-processing functions,
since this amounts to relabelling the leaf nodes. It tells us that such models ought to be calibrated from proper loss minimization, as long as the optimization achieves a fairly weak notion of local optimality.
The next step is to ask whether DNNs could satisfy a similar property. Closure under arbitrary post-processing seems too strong a condition for DNNs to satisfy — we should not expect DNNs to be able to express arbitrarily discontinuous uni-variate post-processing functions. But we do not expect or require DNNs to be perfectly calibrated, only to be close to perfectly calibrated. Measuring the distance from calibration involves subtle challenges as highlighted in the recent work of Blasiok et al. [2023b]. This leads us to the following robust formulation which significantly generalizes Claim 2.1:
- Using the well known relation between proper losses and Bregman divergences [Savage, 1971, Gneiting and Raftery, 2007], we consider proper loss functions where the underlying convex functions satisfy certain differentiability and smoothness properties. This is a technical condition, but one that is satisfied by all proper loss functions commonly used in practice.
- We only allow post-processing functions that satisfy certain Lipschitzness properties. Such functions are simple enough to be implementable by DNNs of constant depth. Indeed we heuristically believe that state-of-the-art DNNs will automatically be near-optimal w.r.t. such post-processings, as mentioned in Section 1.1.
- We measure calibration distance using the notion of smooth calibration from Kakade and Foster [2008] and strengthenings of it in Gopalan et al. [2022b], Błasiok et al. [2023b]. Smooth calibration plays a central role in the consistent calibration measures framework of Błasiok et al. [2023b], where it is shown using duality that the smCE is linearly related with the Wasserstein distance to the nearest perfectly calibrated predictor [Błasiok et al., 2023b]. In practice, there exist sample-efficient and linear-time estimators which approximate the smooth calibration error within a quadratic factor [Błasiok et al., 2023b].
With these notions in place, we show that the post-processing gap of a predictor, which measures how much the loss can be reduced via certain Lipschitz post-processing functions provides both an upper and a lower bound on the smooth calibration error. This result gives us a robust and quantitative version of Claim 2.1. The main technical challenge in formulating this equivalence is that for general  $\ell$ , the class of post-processing functions that we allow and the class of smooth functions that we use to measure calibration error are both now dependent on the loss function  $\ell$ , they are Lipschitz over an appropriately defined dual space to the predictions defined using convex duality. In order to keep the statements simple, we state our results for the special case of squared loss and cross entropy loss below, defering the full statement to the technical sections.
Squared loss. For the squared loss  $\ell_{\mathrm{sq}}(y,v) = (y - v)^2$ , we define the post-processing gap  $\mathsf{pGap}_{\mathcal{D}}(f)$  to be the difference between the expected squared loss  $\mathbb{E}[\ell_{\mathrm{sq}}(y,f(x))]$  of a predictor  $f$  and the minimum expected squared loss after we post-process  $f(x)\mapsto f(x) + \eta (f(x))$  for some 1-Lipschitz  $\eta$ . The Brier score [Brier et al., 1950, Foster and Vohra, 1998] uses the squared loss of a predictor as a calibration measure;  $\mathsf{pGap}$  can be viewed as a differential version of Brier score.
Definition 2.2 (Post-processing gap). Let  $K$  denote the family of all post-processing functions  $\kappa : [0,1] \to [0,1]$  such that the update function  $\eta(v) = \kappa(v) - v$  is 1-Lipschitz. For a predictor  $f: \mathcal{X} \to [0,1]$  and a distribution  $\mathcal{D}$  over  $\mathcal{X} \times \{0,1\}$ , we define the post-processing gap of  $f$  w.r.t.  $\mathcal{D}$  to be
$$
\mathsf {p G a p} _ {\mathcal {D}} (f) := \mathbb {E} _ {(x, y) \sim \mathcal {D}} [ \ell_ {\mathbf {s q}} (y, f (x)) ] - \inf  _ {\kappa \in K} \mathbb {E} _ {(x, y) \sim \mathcal {D}} [ \ell_ {\mathbf {s q}} (y, \kappa (f (x))) ].
$$
We define the smooth calibration error  $\mathsf{smCE}_{\mathcal{D}}(f)$  as in Kakade and Foster [2008], Gopalan et al. [2022b], Błasiok et al. [2023b] to be the maximum correlation between  $y - f(x)$  and  $\eta (f(x))$  over all bounded 1-Lipschitz functions  $\eta$ . Smooth calibration is a consistent measure of calibration [Błasiok et al., 2023b]. It does not suffer the discontinuity problems of ECE [Kakade and Foster, 2008], and is known to be linearly related with the Wasserstein distance to the nearest perfectly calibrated predictor (Błasiok et al. [2023b]). In particular, it is 0 if and only if we have perfect calibration. We refer the reader to these papers for a detailed discussion of its merits.
Definition 2.3 (Smooth calibration error). Let  $H$  be the family of all 1-Lipschitz functions  $\eta : [0,1] \to [-1,1]$ . The smooth calibration error of a predictor  $f: \mathcal{X} \to [0,1]$  with respect to distribution  $\mathcal{D}$
over  $\mathcal{X}\times \{0,1\}$  is defined as
$$
\mathsf {s m C E} _ {\mathcal {D}} (f) := \sup _ {\eta \in H} \mathbb {E} _ {(x, y) \sim \mathcal {D}} [ (y - f (x)) \eta (f (x)) ].
$$
Our main result is a quadratic relationship between these two quantities:
Theorem 2.4. For any predictor  $f: \mathcal{X} \to [0,1]$  and any distribution  $\mathcal{D}$  over  $\mathcal{X} \times \{0,1\}$ ,
$$
\operatorname {s m C E} _ {\mathcal {D}} (f) ^ {2} \leq \operatorname {p G a p} _ {\mathcal {D}} (f) \leq 2 \operatorname {s m C E} _ {\mathcal {D}} (f).
$$
In Appendix D we show that the constants in the inequality above are optimal.
Cross-entropy loss. For the cross entropy loss  $\ell_{\mathrm{xent}}(y,v) = -y\ln v - (1 - y)\ln (1 - v)$ , we observe its close connection to the logistic loss  $\ell^{(\psi)}(y,t) = \ln (1 + e^t) - yt$  given by the equation
$$
\ell_ {\text {x e n t}} (y, v) = \ell^ {(\psi)} (y, t) \tag {2}
$$