Measuring Calibration in Deep Learning

1 INTRODUCTION . The reliability of a machine learning model ’ s confidence in its predictions is critical for high risk applications , such as deciding whether to trust a medical diagnosis prediction ( Crowson et al. , 2016 ; Jiang et al. , 2011 ; Raghu et al. , 2018 ) . One mathematical formulation of the reliability of confidence is calibration ( Murphy & Epstein , 1967 ; Dawid , 1982 ) . Intuitively , for class predictions , calibration means that if a model assigns a class with 90 % probability , that class should appear 90 % of the time . Calibration is not directly measured by proper scoring rules like negative log likelihood or the quadratic loss . The reason that we need to assess the quality of calibrated probabilities is that presently , we optimize against a proper scoring rule and our models often yield uncalibrated probabilities . Recent work proposed Expected Calibration Error ( ECE ; Naeini et al. , 2015 ) , a measure of calibration error which has lead to a surge of works developing methods for calibrated deep neural networks ( e.g. , Guo et al. , 2017 ; Kuleshov et al. , 2018 ) . In this paper , we show that ECE has numerous pathologies , and that recent calibration methods , which have been shown to successfully recalibrate models according to ECE , can not be properly evaluated via ECE . Calibration ( and uncertainty quantification generally ) is critical in autonomous vehicles , the exploration phase of may algorithms , medical applications , and many more safety-critical applications of machine learning . A suite of recent papers ( e.g. , Lee et al. , 2017 ; Vaicenavicius et al. , 2019 ; Thulasidasan et al. , 2019 ; Kumar et al. , 2018 ; Guo et al. , 2017 ; Seo et al. , 2019 ) use ECE to validate their models ’ calibration . We identify concerns with that methodology . As this metric has become the default choice for measuring calibration in industry and for researchers , we expect our criticism to change the decisions of anyone attempting to train well-calibrated classification models . We recommend that rather than using Expected Calibration Error , practitioners use Static Calibratinon Error or Adaptive Calibration . 2 BACKGROUND & RELATED WORK . 2.1 MEASUREMENT OF CALIBRATION . Assume the dataset of features and outcomes { ( x , y ) } are i.i.d . realizations of the random variables X , Y ∼ P. We focus on class predictions . Suppose a model predicts a class y with probability p̂ . The model is calibrated if p̂ is always the true probability . Formally , P ( Y = y | p̂ = p ) = p for all probability values p ∈ [ 0 , 1 ] and class labels y ∈ { 0 , . . . , K − 1 } . The left-hand-side denotes the true data distribution ’ s probability of a label given that the model predicts p̂ = p ; the right-handside denotes that value . Any difference between the left and right sides for a given p is known as calibration error . Expected Calibration Error ( ECE ) . To approximately measure the calibration error in expectation , ECE discretizes the probability interval into a fixed number of bins , and assigns each predicted probability to the bin that encompasses it . The calibration error is the difference between the fraction of predictions in the bin that are correct ( accuracy ) and the mean of the probabilities in the bin ( confidence ) . Intuitively , the accuracy estimates P ( Y = y | p̂ = p ) , and the average confidence is a setting of p. ECE computes a weighted average of this error across bins : ECE = B∑ b=1 nb N |acc ( b ) − conf ( b ) | , where nb is the number of predictions in bin b , N is the total number of data points , and acc ( b ) and conf ( b ) are the accuracy and confidence of bin b , respectively . ECE as framed in Naeini et al . ( 2015 ) leaves ambiguity in both its binning implementation and how to compute calibration for multiple classes . In Guo et al . ( 2017 ) , they bin the probability interval [ 0 , 1 ] into equally spaced subintervals , and they take the maximum probability output for each datapoint ( i.e. , the predicted class ’ s probability ) . We use this for our ECE implementation . 2.2 PROPER SCORING RULES & OTHER MEASURES OF CALIBRATION .. Many classic methods exist to measure the accuracy of predicted probabilities . For example , the Brier score ( quadratic loss ) measures the mean squared difference between the predicted probability and the actual outcome ( Gneiting & Raftery , 2007 ) . This score can be shown to decompose into a sum of metrics , including calibration error . In practice , training against this proper scoring rule , like training against log likelihood , does not guarantee a well calibrated model . The Hosmer-Lemeshow test is a popular hypothesis test for assessing whether a model ’ s predictions significantly deviate from perfect calibration ( Hosmer & Lemesbow , 1980 ) . The reliability diagram provides a visualization of how well-calibrated a model is ( DeGroot & Fienberg , 1983 ) . Adaptivity was explored by ( Nguyen & O ’ Connor , 2015 ) and applied in ( Hendrycks et al. , 2018 ) . Kuleshov et al . ( 2018 ) extends ECE to the regression setting . Unlike the Brier score and Hosmer-Lemeshow , we ’ d like the metric to be scalar-valued in order to easily benchmark methods , and to only measure calibration . 2.3 POST-PROCESSING METHODS . 2.3.1 STANDARD APPROACHES FOR MULTICLASS CLASSIFICATION . One common approach to calibration is to apply post-processing methods to the output of any classifier without retraining . The two most popular post-processing methods are the parametric approach of Platt scaling ( Platt et al. , 1999 ) and the non-parametric approach of isotonic regression ( Zadrozny & Elkan , 2002 ) . Platt scaling fits a logistic regression model to the logits of a classifier , on the validation set , which can be used to compute calibrated predictions at test time . The original formulation of Platt scaling for neural networks ( Niculescu-Mizil & Caruana , 2005 ) involves learning scalar parameters a , b ∈ R on a held-out validation set and then computing calibrated probabilities p̂i given the uncalibrated logits vector zi on the test set as p̂i = σ ( azi + b ) . These parameters are typically estimated by minimizing the negative log likelihood . Platt scaling can be extended to the multiclass setting by considering higher-dimensional parameters . In matrix scaling a is replaced with W ∈ RK∗K while we consider b ∈ RK . As for vector scaling ; W ∈ RK . Calibrated probabilities for either of these extensions can then be computed as p̂i = max k σ ( Wzi + b ) . An even simpler extension is temperature scaling ( Guo et al. , 2017 ) which reduces the set of regression parameters to the inverse of a single scalar T > 0 such that p̂i = max k σ ( zi/T ) . On the other hand , isotonic regression is a common non-parametric processing method that finds the stepwise-constant non-decreasing ( isotonic ) function f that best fits the data according to a mean-squared loss function ∑ i ( f ( pi ) − yi ) 2 where pi are the uncalibrated probabilities and yi the labels . The standard approach for extending isotonic regression to the multiclass setting is to break the problem into many binary classification problems ( e.g . one-versus-all problems ) , to calibrate each problem separately , and then to combine the calibrated probabilities ( Zadrozny & Elkan , 2002 ) . 3 ISSUES WITH CALIBRATION METRICS . 3.1 NOT COMPUTING CALIBRATION ACROSS ALL PREDICTIONS . Expected Calibration Error was crafted to mirror reliability diagrams , which are structured around binary classification such as rain vs not rain ( DeGroot & Fienberg , 1983 ) ( Figure 1 ) . A consequence is that the error metric is reductive in a multi-class setting . In particular , ECE is computed using only the predicted class ’ s probability , which implies the metric does not assess how accurate a model is with respect to the K − 1 other probabilities . We examine increasing the importance of omitted predictions via label noise . As label noise increases , trained models become less confident in their predictions . Secondary and tertiary predictions are correct with higher frequency , and so for many samples the prediction with the correct label isn ’ t included in ECE ’ s measure of calibration error . We find that ECE becomes a worse approximation of the calibration error as relevant predictions not included in the ECE metric become common ( Figure 1 ) . In Figure 2 , we observe that calibration error , when measured in a per-class manner , is non-uniform across classes . Guaranteeing calibration for all class probabilities is an important property for many applications . For example , a physician may care deeply about a secondary or tertiary prediction of a disease . Decision costs along both ethical and monetary axes are non-uniform , and thus clinical decision policies necessitate a comprehensive view of the likelihoods of potential outcomes ( e.g. , Tsoukalas et al. , 2015 ) . Alternatively , if a true data distribution exhibits high data noise ( also known as high aleatoric uncertainty , or weak labels ) , then we ’ d like models to be calibrated across all predictions as no true class label is likely . 3.2 FIXED CALIBRATION RANGES . One major weakness of evenly spaced binning metrics is caused by the dispersion of data across ranges . In computing ECE , there is often a large leftward skew in the output probabilities , with the left end of the region being sparsely populated and the rightward end being densely populated . Sharpness , which is the desire for models to always predict with high confidence , i.e. , predicted probabilities concentrate to 0 or 1 , is a fundamental property ( Gneiting et al. , 2007 ) . ( That is , network predictions are typically very confident . ) This causes only a few bins to contribute the most to ECE—typically one or two as bin sizes are 10-20 in practice ( Guo et al. , 2017 ) . 3.3 BIAS-VARIANCE TRADEOFF . ‘ Ground truth ’ is the calibration error we would measure if we could see an infinite number of predictions at a particular confidence level . That would give us a long-run error for every prediction , which we could aggregate into an overall calibration error . Because we don ’ t have that infinite set of reference predictions for every real-valued prediction , we try to approximate that set by using the values that are close together as reference predictions . As we increase the number of datapoints in a bin , we get a more accurate estimate of the calibration in a given range through the added ( seen as a reduction in variance ) . But as the datapoints in the bin start to differ from one another due to the extension of the range , they bias the estimate of the calibration within that bin ( we refer to this as bias ) . Selecting the number of bins has a bias-variance tradeoff as it determines how many data points fall into each bin and therefore the quality of the estimate of calibration from that bin ’ s range . In particular , a larger number of bins causes more granular measures of calibration error ( low bias ) but also a high variance of each bin ’ s measurement as bins become sparsely populated . This tradeoff compounds particularly with the problem of fixed calibration ranges , as due to sharpness certain bins have many more data points than others .

Summary: 	This paper proposes two novel scalar-valued evaluation measures, namely Static Calibration Error (SCE) and Adaptive Calibration Error (ACE)  for evaluating the quality of calibration (the accuracy of estimating p(y|x) in classification) in deep learning. The authors suggested that the existing Expected Calibration Error (ECE), which is the current most popular scalar-valued metric for calibration, has several drawbacks, especially in the multi-class scenario. Intuitively, ECE only focuses on the predicted class probability (argmax) and ignores class probabilities of other K-1 classes, which implies that ECE may fail to capture the class probability estimates for all classes. They also illustrated the drawback of ECE under label noise.