Towards Uncertainties in Deep Learning that Are Accurate and Calibrated

1 INTRODUCTION . Probabilistic forecasts can be characterized by two properties—calibration and sharpness ( Gneiting et al. , 2007 ) . Intuitively , calibration means that a 90 % confidence interval contains the true outcome 90 % of the time . Sharpness means that these confidence intervals are narrow . These qualities are grounded in the statistics literature on proper scoring rules , and are widely used to evaluate forecasts ( Gneiting and Raftery , 2005 ; 2007 ) in domains such as medicine ( Saria , 2018 ) , robotic control ( Malik et al. , 2019 ) , and human-in-the-loop learning ( Werling et al. , 2015 ) . This paper introduces algorithms that ensure the calibration of any predictive machine learning model while maintaining sharpness . They apply to both classification and regression tasks and guarantee the strong property of distribution calibration ( which generalizes standard quantile and classification-based calibration ; Song et al . ( 2019 ) ) in any model , including deep learning models , which are often miscalibrated ( Guo et al. , 2017 ) . Unlike existing methods for distribution calibration ( Song et al. , 2019 ) , ours can be used with any model ( not just ones that output Gaussians ) , are very simple to implement in differentiable programming frameworks , and have theoretical guarantees . Importantly , our algorithms achieve for the first time a long-standing statistical principle that forecasts should “ maximize sharpness subject to being calibrated ” proposed by Gneiting et al . ( 2007 ) . We prove that under some assumptions this principle is achievable in modern machine learning models in a black-box manner and without sacrificing overall performance . This lends strong support for this principle as a way of reasoning about uncertainty in machine learning . In a sense , calibration is a rare free lunch in machine learning , and we argue that it should be enforced in predictive models and taken advantage of in downstream applications . Empirically , we find that our method consistently outputs well-calibrated predictions across a wide range of experiments , while improving performance on downstream tasks with minimal implementation overhead . Contributions . In summary , we make three contributions . We propose a new recalibration technique that ( a ) is among the only to guarantee distribution calibration besides Song et al . ( 2019 ) . Unlike Song et al . ( 2019 ) we can ( b ) recalibrate any parametric distribution ( not just Gaussians ) and ( c ) our method is simpler . While theirs is based on variational inference in Gaussian processes , ours uses a neural network that can be implemented in a few lines of code , which encourages adoption . Our method ( d ) applies to both classification and regression and ( e ) outperforms methods by Song et al . ( 2019 ) and Kuleshov et al . ( 2018 ) as well as Platt and temperature scaling . We also formally prove that our technique produces asymptotically distributionally calibrated forecasts while minimizing regret . Most methods ( e.g. , Platt , Kuleshov scaling , Song et al. , etc . ) do not have a correctness proof , except for conformal prediction , which is significantly more complex . Finally , our analysis formalizes the well-known paradigm of Gneiting et al . ( 2007 ) and provides the first method that provably achieves it . This lends strong support for this principle and influences how one should reason about uncertainty in machine learning . An important takeaway is that calibration can be achieved in most applications of machine learning with very little cost . As such , calibration can be a rare free lunch that we believe should be leveraged throughout machine learning . 2 BACKGROUND . 2.1 PREDICTIVE UNCERTAINTY IN MACHINE LEARNING . Supervised machine learning models commonly predict a probability distribution over the output variables — e.g . class membership probabilities or the parameters of an exponential family distribution . These predictive uncertainties are useful for interpretability , safety , and downstream decisionmaking . Aleatoric uncertainty captures the inherent noise in the data , while epistemic uncertainty arises from not having a large enough dataset to estimate model parameters ( Kendall and Gal , 2017 ) . Notation . Formally , we say that a machine learning forecaster H : X → ∆ ( Y ) outputs a probability distribution F ( y ) : Y → [ 0 , 1 ] in the space ∆ ( Y ) of distributions over y . We use f to denote the probability density or probability mass function associated with F . The model H is trained on a labeled dataset xt , yt ∈ X ×Y for t = 1 , 2 , ... , T of i.i.d . realizations of random variablesX , Y ∼ P , where P is the data distribution . 2.2 WHAT DEFINES GOOD PREDICTIVE UNCERTAINTIES ? . The standard tool in statistics for evaluating the quality of predictive uncertainties is a proper scoring rule ( Gneiting and Raftery , 2007 ) . Formally , a scoring rule S : ∆ ( Y ) × Y → R assigns a “ score ” to a probabilistic forecast F ∈ ∆ ( Y ) and a realized outcome y ∈ Y . Given a true distribution G ∈ ∆ ( Y ) for y , we use the notation S ( F , G ) for the expected score S ( F , G ) = Ey∼GS ( F , y ) . We say that a score S is proper if it is minimized by G when G is the true distribution for y : S ( F , G ) ≥ S ( G , G ) for all F . When S is proper , we also refer to it as a proper loss . An example of a proper loss is the log-likelihood S ( F , y ) = log f ( y ) , where f is the probability density or probability mass function of F . Another common loss is the check score ρτ ( y , f ) = τ ( y − f ) if y ≥ f and ( 1 − τ ) ( f − y ) otherwise ; it can be used to estimate the τ -th quantile of a distribution . See Table 1 for additional examples . What are the qualities of a good probabilistic prediction , as measured by a proper scoring rule ? It can be shown that every proper score is a sum of the following terms ( Gneiting et al. , 2007 ) : proper loss = calibration−sharpness + irreducible term︸ ︷︷ ︸ refinement term . Thus , there are precisely two qualities that define an ideal forecast : calibration and sharpness . We examine each of them next . 2.3 CALIBRATION AND SHARPNESS — TWO QUALITIES OF AN IDEAL PREDICTION . Formally , calibration can be defined by the equation P ( Y = y | FX = F ) = f ( y ) for all y ∈ Y , F ∈ ∆ ( Y ) , ( 1 ) where X , Y ∼ P are random variables corresponding to the input features and targets , and FX = H ( X ) is the forecast at X , itself a random variable that takes values in ∆ ( Y ) . We use f to denote the probability density or probability mass function associated with F . When Y = { 0 , 1 } and FX is a Bernoulli distribution with parameter p , we can write ( 1 ) as P ( Y = 1 | FX = p ) = p. This has a simple intuition : the true probability of Y = 1 is p conditioned on predicting it as p. Equation 1 extends beyond binary classification to arbitrary distributions . For example , if F is a Gaussian with variance σ2 , this definition asks that the data distribution conditioned on predicting F also has variance σ2 . This recently proposed definition is called distribution calibration ( Song et al. , 2019 ) . A closely related , but weaker concept is quantile calibration ( Kuleshov et al. , 2018 ) , which asks that a 90 % confidence interval contains the true value 90 % of the time . Formally , it can be written as : P ( Y ≤ CDF−1FX ( p ) ) = p for all p ∈ [ 0 , 1 ] , Quantile calibration is implied by distributional calibration ( Song et al. , 2019 ) . Calibration by itself is not sufficient to produce a useful forecast . For example , it is easy to see that a binary classifier that always outputs P ( Y = 1 ) as the probability that Y = 1 is calibrated ; however it does not even use the features X and thus can not be accurate . In order to be useful , forecasts must also be sharp . Intuitively , this means that predicted confidence intervals should be as tight as possible around a single value . This is captured by proper scoring rules as part of a refinement term ( see Table 1 ) , which equals an irreducible term minus a sharpness term ( Murphy , 1973 ; Brocker , 2009 ) . The latter is maximized when we minimize the scoring rule . Are Modern Machine Learning Models Calibrated And Sharp ? Most machine learning models are not calibrated out-of-the-box ( Niculescu-Mizil and Caruana , 2005 ; Guo et al. , 2017 ) . Two reason for this are the limited expressivity of the model H—we can not perfectly fit the entirety of the level curves of the data distribution—and computational approximations—computing extract predictive uncertainties may be intractable , and approximations are not entirely accurate . A final reason stems from how models are trained—since we can not fit a perfect H , standard objective functions induce a tradeoff between sharp and calibrated forecasts . Next , we will show that by training models differently , we can achieve calibration without sacrificing performance . 3 ENSURING DISTRIBUTION CALIBRATION IN ANY MODEL . This section introduces algorithms that ensure the distirbutional calibration of any predictive machine learning model while maintaining sharpness . Unlike existing methods for distribution calibration , ours can be used with any model ( not just ones that output Gaussians ) , are very simple to implement in differentiable programming frameworks , and have theoretical guarantees . We first assume there exists a parameterization Φ1 of the probabilities returned by forecaster H : for each p ∈ ∆ ( Y ) returned by H , there exist parameters φ ∈ Φ1 that describe p. The φ can be the natural parameters of an exponential family distribution , such as ( µ , σ2 ) describing a Gaussian . We consider a class of algorithms based on a classic approach called recalibration . First , we train a base forecaster H to minimize a proper loss L. Then , we train an auxiliary model R : Φ1 → Φ2 ( called the recalibrator ) over the outputs of H that outputs the parameters φ2 ∈ Φ2 of another distribution such that L is minimized . Here Φ2 is a second parameterization of ∆ ( Y ) ( possibly the same ) . As a result , the forecasts ( R ◦H ) ( X ) will be calibrated . We provide details in Algorithm 1 . Algorithm 1 Calibrated Learning of Probabilistic Models . Input : Model H : X → Φ1 , recalibrator R : Φ1 → Φ2 , training set D , recalibration set C Output : Recalibrated model R ◦H : X → Φ2 . 1 . Fit the base model on D : minH ∑ ( x , y ) ∈D L ( H ( x ) , y ) 2 . Fit the recalibration model R on the output of H on C : minR ∑ ( x , y ) ∈C L ( ( R ◦H ) ( x ) , y ) Implementing this approach requires choosing parameterizations of probabilities Φ1 , Φ2 , a recalibration model R , and an objective L. We discuss these choices below ; then we clarify how 1 differs from existing recalibration algorithms . Parameterizing Probability Distributions . When fitting a classification model , each distribution ∆ ( Y ) is a categorical and can be parameterized via its K ≥ 2 class membership probabilities . In regression , most widely used models such as neural networks already output parameterized probabilities , in which the φ are usually the natural parameters of an exponential family model . In the most general case , if we only have black-box access to a density function or a cumulative distribution function , we may form a d-dimensional representation by evaluating the distribution at a grid of d points . For example , if we have black-box access to a quantile function F−1 , we may featurize F via its sequence of quantiles φ ( F ) = ( F−1 ( αi ) ) di=1 for some sequence of d levels αi , possibly chosen uniformly in [ 0 , 1 ] . In addition to the above techniques , the representation of output probabilities in Φ2 coming from R can leverage flexible invertible models of the CDF , following methods developed in the normalizing flow literature , including monotonic neural networks , sum-of-squares polynomials ( Wehenkel and Louppe , 2019 ; Jaini et al. , 2019 ) spline functions ( Muller et al. , 2019 ; Durkan et al. , 2019 ) , piecewise separable models ( Wehenkel and Louppe , 2019 ) , and others . Choosing a Recalibrator . Ideal recalibrators are highly effective at optimizing the proper loss L ( see Section 4 ) . In a simple setting like binary classification , our task reduces to one-dimensional density estimation ; in such cases we can provably achieve calibration asymptotically by using kernel density estimation for the recalibrator R , while controlling the kernel width as a function of the dataset size to trade off overfitting and underfitting ( Wasserman , 2006 ) . In regression settings , we may rely on other non-parametric techniques such as Gaussian processes . An alternative approach is to rely on expressive neural networks ; although their optimization is a non-convex , they are very effective at fitting proper losses L , feature mature regularization techniques , and can be implemented easily within deep learning frameworks , possibly within the same computation graph as a neural forecaster H , which can simplify deployment . In the classification setting , a natural architecture for R is a sequence of dense layers mapping the simplex ∆K into ∆K . In regression settings , R needs to output a density function : a natural architecture for this is a mixture density network ( MDN ; Bishop ( 1994 ) ) . Choosing a Proper Loss A natural choice of proper loss is the log-loss . It applies in both calibration and regression ; optimizing it is a standard supervised learning problem . In regression settings , we found that using the quantile loss L = Eτ∈U [ 0,1 ] Ey∼Gρτ ( y − F−1 ( τ ) ) ( see Table 1 ) was numerically stable and produced the best performance . This objective fits a model Rθ ( τ ; φ ) to estimate the τ -th conditional quantile F−1 ( τ ) at φ . When Rθ ( τ ; φ ) is a neural network that takes in τ and x , we minimize the quantile loss Eτ∈U [ 0,1 ] Ey∼Gρτ ( y − F−1 ( τ ) ) using gradient descent , approximating both expectations using Monte Carlo ( details are in the experiments section ) . Comparing Against Song et al . Interestingly , the method of Song et al . ( 2019 ) is a special case of ours when Φ1 consists of Gaussian natural parameters , Φ2 consists of parameters for the Beta link function , R is a Gaussian process , and L is the log-likelihood . However , the resulting problem can only be solved using variational inference , which is slow and complex to implement . Our framework instead admits simple solutions based on gradient descent . Comparing Against Other Approaches . Algorithm 1 performs recalibration like many previous methods ( e.g. , Platt , Kuleshov et al. , Song et al. , etc . ) ; it may thus appear to be the same as these methods . However , that is not the case . First , existing recalibration approaches operate over the space of probabilities ( class probabilities or CDF values ) ; ours operates over functional parameters . This is what enables it to achieve distribution rather than quantile calibration ( Kuleshov et al. , 2018 ) . Our approach also involves novel recalibration objectives ( e.g. , the quantile divergence in regression ) which differ from the calibration error of Kuleshov et al . ( 2018 ) . We also use different types of models ( small neural networks instead of isotonic regression ) and different optimization procedures used ( stochastic gradient descent instead of variational inference ) . Thus , our recalibration strategy is distinct from previous work .

Summary: The paper proposes a new method for calibrating the uncertainty estimates of supervised machine learning models, i.e., ensuring that they match the probabilities of the data distribution. After regular model training, a calibrator model is trained to transform the model output, using the same loss but a separate dataset. Theoretical analysis and empirical experiments are conducted to support the claim that this is a way to calibrate "any model" at "almost no cost".