Improved Uncertainty Post-Calibration via Rank Preserving Transforms

1 INTRODUCTION . Modern machine learning models such as deep neural networks have achieved high performance on many challenging tasks , and have been put into production that impacts billions of people ( LeCun et al. , 2015 ) . It is increasingly critical that the outputs of these models are comprehensible and safe to use in downstream applications . However , high-accuracy classification models often exhibit the failure mode of miscalibration : the output probabilities of these models do not reflect the true accuracies , and tend to be over-confident ( Guo et al. , 2017 ; Lakshminarayanan et al. , 2017 ) . As the output probabilities are typically comprehended as ( an estimate of ) true accuracies and used in downstream applications , miscalibration can negatively impact the decision making , and is especially dangerous in risk-sensitive domains such as medical AI ( Begoli et al. , 2019 ; Jiang et al. , 2012 ) or self-driving cars ( Michelmore et al. , 2018 ) . It is an important question how to properly calibrate these models so as to make the output probabilities more trustworthy and safer to use . Existing methods for uncertainty calibration can roughly be divided into two types . Diversity-based methods such as ensembles ( Lakshminarayanan et al. , 2017 ; Wen et al. , 2020 ) and Bayesian networks ( Gal & Ghahramani , 2016 ; Maddox et al. , 2019 ; Dusenberry et al. , 2020 ) work by aggregating predicted probability over multiple models or multiple times on a randomized model . These methods are able to improve both the accuracy and the uncertainty calibration over a single deterministic model ( Ovadia et al. , 2019 ) . However , deploying these models requires either storing all the ensemble members and/or running multiple random variants of the same model , which makes them memory-expensive and runtime-inefficient . On the other hand , post-calibration methods work by learning a calibrator on top of the output probabilities ( or logits ) of an existing well-trained model ( Platt et al. , 1999 ; Zadrozny & Elkan , 2001 ; 2002 ; Guo et al. , 2017 ; Ding et al. , 2020 ) . For a K-class classification model that outputs logits z = z ( x ) ∈ RK , post-calibration methods learn a calibrator f : RK → RK using additional holdout data , so that f ( z ) is better calibrated than the original z . The architectures of such calibrators are typically simple : A prevalent example is the temperature scaling method which learns fT ( z ) = z/T with a single trainable parameter T > 0 by minimizing the negative log-likelihood ( NLL ) loss on holdout data . Such simple calibrators add no overhead to the existing model , and is empirically shown to improve the calibration significantly on a variety of tasks and models ( Guo et al. , 2017 ) . Despite its empirical success , the design of post-calibration methods is not yet fully satisfactory : In practice , simple calibrators such as temperature scaling often underfit the calibration loss on its training data , whereas more complex calibrators can often overfit—see Figure 1 for a quantitative illustration of this effect . While the underfitting of simple calibrators are perhaps due to their limited expressive power , the overfitting of complex calibrators is also believed to be natural since the holdout dataset used for training the calibrators are typically small ( e.g . a few thousands of examples ) . One concrete example is the matrix scaling method which learns a matrix calibrator fW , b ( z ) = Wz + b involving O ( K2 ) trainable parameters . When K is large , matrix scaling often tend to overfit and hurt calibration , despite being a strict generalization of temperature scaling ( Guo et al. , 2017 ) . It is further observed that the overfitting can not be easily fixed by applying common regularizations such as L2 on the calibrator ( Kull et al. , 2019 ) . These empirical evidence seems to suggest that complex calibrators with a large amount of parameters are perhaps not recommended in designing post calibration methods . In this paper , we show that in contrast to the prior belief , large calibrators do not necessarily overfit ; it is rather a lack of accuracy constraint on the calibrator that may have caused the overfitting . Observe that matrix scaling , unlike temperature scaling , is not guaranteed to maintain the accuracy of the model : it applies a general affine transform z 7→ Wz + b on the logits and can modify their rank ( and thus the predicted top label ) , whereas temperature scaling is guaranteed to preserve the rank . When trained with the NLL loss , a calibrator that does not maintain the accuracy may attempt to improve the accuracy at the cost of hurting ( or not improving ) the calibration . Motivated by this observation , this paper proposes Neural Rank-Preserving Transforms ( NRPT ) , a method for learning calibrators that maintain the accuracy of the model , yet are complex enough for yielding better calibration performance than simple calibrators such as temperature scaling . Our key idea is that a sufficient condition for the calibrator to maintain the accuracy is for it to preserve the rank of the logits : any mapping that preserves the rank of the logits will not change the predicted top label . We instantiate this idea by designing a family of calibrators that perform entrywise monotone transforms on each individual logit ( or log-probability ) : for a K-class classification problem , NRPT scales each logit as zi → f ( zi , x ) , where zi ∈ R is the i-th logit ( 1 ≤ i ≤ K ) , x ∈ Rd is the original input features , and f : R× Rd → R is monotonically increasing in its first argument but otherwise arbitrary . As f is monotone , we have f ( z1 , x ) ≤ f ( z2 , x ) if z1 ≤ z2 , and thus f preserves the rank of the logits . This method strictly generalizes temperature scaling ( which uses f ( zi , x ) = zi/T ) and local temperature scaling ( which uses f ( zi , x ) = zi/T ( x ) ) ( Ding et al. , 2020 ) . The fact that f can depend on x further helps improve the expressivity of f and allows great flexibility in the architecture design . We compare our instantiation of NRPT against temperature scaling and matrix scaling in Figure 1 , in which we see that NRPT is indeed able to fit the training loss better than temperature scaling and does not suffer from overfitting . Our contributions We propose Neural Rank-Preserving Transforms ( NRPT ) , an improved method for performing uncertainty post-calibration on a trained classifier while maintaining its accuracy ( Section 3 ) . NRPT learns calibrators that scale the logits using general monotone transforms , are individualized to the original input features , and allow learning with any calibration loss function ( not restricted to those that correlates with the accuracy ) . We show experimentally that NRPT improves the expected calibration error ( ECE ) significantly over existing post-calibration methods on large-scale image and text classification tasks such as CIFAR-100 , ImageNet , and MNLI ( Section 4.1 ) . NRPT can further match diversity-based methods such as deep ensembles , while using a much less number of additional parameters . We further demonstrate the strong calibration ability of NRPT beyond the ECE , by showing that it improves on the accuracy among top-confidence predictions , as well as the tradeoff between ECE and sharpness of prediction ( Section 4.2 ) . Due to the space constraint , we defer the discussions of additional related work to Appendix A and additional experimental details and results to the later Appendices . 2 BACKGROUND ON UNCERTAINTY CALIBRATION . We consider K-class classification problems where X ∈ Rd is the input ( features ) , Y ∈ [ K ] : = { 1 , . . . , K } is the true label , and ( X , Y ) follows some underlying joint distribution . Let p̂ : Rd → ∆K be a prediction model ( for example , a neural network learned from data ) that maps inputs to probabilities , where ∆K : = { ( p1 , . . . , pK ) : pi ≥ 0 , ∑ i pi = 1 } is the set of all probability distributions on [ K ] . We say p̂ is perfectly calibrated if P ( Y = k | p̂ ( X ) = p ) = pk for all p ∈ ∆K , k ∈ [ K ] . In other words , a model is perfectly calibrated if when the model predicts p̂ ( X ) = p , the conditional distribution of Y is exactly p. It is difficult to evaluate perfect calibration from finite data , as for almost all p we do not receive samples that satisfy the exact conditioning p̂ ( x ) = p. This motivates considering alternative scalar metrics for calibration that can be estimated from data . ECE The Expected Calibration Error ( ECE ) is a commonly used metric that measures calibration through grouping examples according to the confidence ( i.e . the top predicted probability ) ( Naeini et al. , 2015 ; Guo et al. , 2017 ) . Let { ( xi , yi ) } ni=1 be the evaluation dataset on which we wish to evaluate the calibration of a model p̂ . Define the intervals Im = ( m−1M , m M ] , where M > 0 is a ( fixed ) number of bins , and partitions the examples into M bins according to the confidence : Bm = { i : maxk p̂ ( xi ) k ∈ Im } . Define the accuracy and confidence within Bm as acc ( Bm ) : = 1 Bm ∑ i∈Bm 1 { arg max k p̂ ( xi ) k = yi } and conf ( Bm ) : = 1 Bm ∑ i∈Bm max k p̂ ( xi ) k The ECE is then defined as the ( weighted ) average difference between accuracy and confidence : ECE ( p̂ ) : = M∑ m=1 |Bm| n |acc ( Bm ) − conf ( Bm ) | . ( 1 ) The ECE is a sensible calibration metric since it is a binned approximation of the top-label calibration error ( TCE ) that measures the difference between accuracy and confidence under exact conditioning : TCE ( p̂ ) : = E [ ∣∣∣∣P ( arg max k p̂ ( X ) k = Y | max k p̂ ( X ) k ) −max k p̂ ( X ) k ∣∣∣∣ ] . Debiased ECE Recent work shows that the ECE has an inherent positive bias and proposes the Debiased ECE that approximately removes this bias using Gaussian bootstrapping ( Kumar et al. , 2019 ) : DECE ( p̂ ) : = ECE ( p̂ ) − ( ER1 : M [ M∑ m=1 |Bm| n |conf ( Bm ) −Rm| ] − ECE ( p̂ ) ) , where Rm ∼ N ( acc ( Bm ) , acc ( Bm ) ( 1−acc ( Bm ) ) |Bm| ) . Kumar et al . ( 2019 ) showed that the debiased ECE is a typically more accurate estimator of the TCE than ECE , especially when the TCE is relatively small . In our experiments , we use both the ECE and the debiased ECE for evaluating calibration . NLL The Negative Log-Likelihood ( NLL ) , typically used as loss function for training classifiers , is also a measure of calibration : NLL ( p̂ ) : = 1 n n∑ i=1 − log p̂ ( xi ) yi . ( 2 ) NLL is a proper scoring rule ( Lakshminarayanan et al. , 2017 ) in the sense that the population minimizer over all possible p̂ is achieved at the ground truth conditional distribution p ? ( Hastie et al. , 2009 ) . In general , the NLL measures the distance between p̂ and p ? , and is thus a joint metric of accuracy and calibration . Predictive entropy ( sharpness ) While we are mostly concered about the accuracy and calibration of a model , these two metrics alone do not fully guarantee a proper uncertainty quantification . For example , any high-accuracy model can be calibrated in a “ trivial ” way such that the ECE becomes exactly 0 , by mapping the confidence on all examples to be equal to the ( overall ) accuracy of the model , and rescaling the non-top probabilities accordingly . In order to prevent such trivial calibration , we additionally measure the sharpness of the predictions using the predictive entropy ( Lakshminarayanan et al. , 2017 ) : PEnt ( p̂ ) = 1 n n∑ i=1 K∑ k=1 −p̂ ( xi ) k log p̂ ( xi ) k. Lower predictive entropies indicate sharper predictions ( i.e . predictions closer to delta distributions than the uniform distribution ) . In general , the predictive entropy is not necessarily related to the calibration ; however , for models that have the same accuracies , we observe that the predictive entropy is typically negatively correlated with calibration—sharper predictions are usually less calibrated .

Summary: This paper introduces a new post-processing calibration method that can keep classification ranking, like the simple (local) temperature scaling methods, but has larger capacity, comparable to matrix scaling.  Essentially it tries to achieve the best of both approaches by avoiding the underfitting and overfitting issues of temperature and matrix scaling, respectively.  The method proposed essentially learns a temperature and a bias parameter that depends on the image, and two matrices that are global to all images.  During post-processing calibration training, the constraining of the matrices and the temperature to be strictly positive guarantees that the calibration is monotonically increasing, guaranteeing the preservation of classification ranking.  The paper further shows that this calibration model can be trained in a post-processing approach with NLL and ECE losses (the training with the ECE loss is claimed as another novelty).  Experiments are performed on CIFAR100, ImageNet, and MNLI, and results show that the proposed post-processing calibration stage trained with NLL improves NLL and Entropy for CIFAR100 and ImageNet, and the one trained with ECE improves the ECE related measures.  For MNLI, results seem to show that the calibration with NLL loss provides the best general result.  Additional results that show the value of the method for other calibration measures also display the value of the proposed approach.