Improved Uncertainty Post-Calibration via Rank Preserving Transforms

1 INTRODUCTION . Modern machine learning models such as deep neural networks have achieved high performance on many challenging tasks , and have been put into production that impacts billions of people ( LeCun et al. , 2015 ) . It is increasingly critical that the outputs of these models are comprehensible and safe to use in downstream applications . However , high-accuracy classification models often exhibit the failure mode of miscalibration : the output probabilities of these models do not reflect the true accuracies , and tend to be over-confident ( Guo et al. , 2017 ; Lakshminarayanan et al. , 2017 ) . As the output probabilities are typically comprehended as ( an estimate of ) true accuracies and used in downstream applications , miscalibration can negatively impact the decision making , and is especially dangerous in risk-sensitive domains such as medical AI ( Begoli et al. , 2019 ; Jiang et al. , 2012 ) or self-driving cars ( Michelmore et al. , 2018 ) . It is an important question how to properly calibrate these models so as to make the output probabilities more trustworthy and safer to use . Existing methods for uncertainty calibration can roughly be divided into two types . Diversity-based methods such as ensembles ( Lakshminarayanan et al. , 2017 ; Wen et al. , 2020 ) and Bayesian networks ( Gal & Ghahramani , 2016 ; Maddox et al. , 2019 ; Dusenberry et al. , 2020 ) work by aggregating predicted probability over multiple models or multiple times on a randomized model . These methods are able to improve both the accuracy and the uncertainty calibration over a single deterministic model ( Ovadia et al. , 2019 ) . However , deploying these models requires either storing all the ensemble members and/or running multiple random variants of the same model , which makes them memory-expensive and runtime-inefficient . On the other hand , post-calibration methods work by learning a calibrator on top of the output probabilities ( or logits ) of an existing well-trained model ( Platt et al. , 1999 ; Zadrozny & Elkan , 2001 ; 2002 ; Guo et al. , 2017 ; Ding et al. , 2020 ) . For a K-class classification model that outputs logits z = z ( x ) ∈ RK , post-calibration methods learn a calibrator f : RK → RK using additional holdout data , so that f ( z ) is better calibrated than the original z . The architectures of such calibrators are typically simple : A prevalent example is the temperature scaling method which learns fT ( z ) = z/T with a single trainable parameter T > 0 by minimizing the negative log-likelihood ( NLL ) loss on holdout data . Such simple calibrators add no overhead to the existing model , and is empirically shown to improve the calibration significantly on a variety of tasks and models ( Guo et al. , 2017 ) . Despite its empirical success , the design of post-calibration methods is not yet fully satisfactory : In practice , simple calibrators such as temperature scaling often underfit the calibration loss on its training data , whereas more complex calibrators can often overfit—see Figure 1 for a quantitative illustration of this effect . While the underfitting of simple calibrators are perhaps due to their limited expressive power , the overfitting of complex calibrators is also believed to be natural since the holdout dataset used for training the calibrators are typically small ( e.g . a few thousands of examples ) . One concrete example is the matrix scaling method which learns a matrix calibrator fW , b ( z ) = Wz + b involving O ( K2 ) trainable parameters . When K is large , matrix scaling often tend to overfit and hurt calibration , despite being a strict generalization of temperature scaling ( Guo et al. , 2017 ) . It is further observed that the overfitting can not be easily fixed by applying common regularizations such as L2 on the calibrator ( Kull et al. , 2019 ) . These empirical evidence seems to suggest that complex calibrators with a large amount of parameters are perhaps not recommended in designing post calibration methods . In this paper , we show that in contrast to the prior belief , large calibrators do not necessarily overfit ; it is rather a lack of accuracy constraint on the calibrator that may have caused the overfitting . Observe that matrix scaling , unlike temperature scaling , is not guaranteed to maintain the accuracy of the model : it applies a general affine transform z 7→ Wz + b on the logits and can modify their rank ( and thus the predicted top label ) , whereas temperature scaling is guaranteed to preserve the rank . When trained with the NLL loss , a calibrator that does not maintain the accuracy may attempt to improve the accuracy at the cost of hurting ( or not improving ) the calibration . Motivated by this observation , this paper proposes Neural Rank-Preserving Transforms ( NRPT ) , a method for learning calibrators that maintain the accuracy of the model , yet are complex enough for yielding better calibration performance than simple calibrators such as temperature scaling . Our key idea is that a sufficient condition for the calibrator to maintain the accuracy is for it to preserve the rank of the logits : any mapping that preserves the rank of the logits will not change the predicted top label . We instantiate this idea by designing a family of calibrators that perform entrywise monotone transforms on each individual logit ( or log-probability ) : for a K-class classification problem , NRPT scales each logit as zi → f ( zi , x ) , where zi ∈ R is the i-th logit ( 1 ≤ i ≤ K ) , x ∈ Rd is the original input features , and f : R× Rd → R is monotonically increasing in its first argument but otherwise arbitrary . As f is monotone , we have f ( z1 , x ) ≤ f ( z2 , x ) if z1 ≤ z2 , and thus f preserves the rank of the logits . This method strictly generalizes temperature scaling ( which uses f ( zi , x ) = zi/T ) and local temperature scaling ( which uses f ( zi , x ) = zi/T ( x ) ) ( Ding et al. , 2020 ) . The fact that f can depend on x further helps improve the expressivity of f and allows great flexibility in the architecture design . We compare our instantiation of NRPT against temperature scaling and matrix scaling in Figure 1 , in which we see that NRPT is indeed able to fit the training loss better than temperature scaling and does not suffer from overfitting . Our contributions We propose Neural Rank-Preserving Transforms ( NRPT ) , an improved method for performing uncertainty post-calibration on a trained classifier while maintaining its accuracy ( Section 3 ) . NRPT learns calibrators that scale the logits using general monotone transforms , are individualized to the original input features , and allow learning with any calibration loss function ( not restricted to those that correlates with the accuracy ) . We show experimentally that NRPT improves the expected calibration error ( ECE ) significantly over existing post-calibration methods on large-scale image and text classification tasks such as CIFAR-100 , ImageNet , and MNLI ( Section 4.1 ) . NRPT can further match diversity-based methods such as deep ensembles , while using a much less number of additional parameters . We further demonstrate the strong calibration ability of NRPT beyond the ECE , by showing that it improves on the accuracy among top-confidence predictions , as well as the tradeoff between ECE and sharpness of prediction ( Section 4.2 ) . Due to the space constraint , we defer the discussions of additional related work to Appendix A and additional experimental details and results to the later Appendices . 2 BACKGROUND ON UNCERTAINTY CALIBRATION . We consider K-class classification problems where X ∈ Rd is the input ( features ) , Y ∈ [ K ] : = { 1 , . . . , K } is the true label , and ( X , Y ) follows some underlying joint distribution . Let p̂ : Rd → ∆K be a prediction model ( for example , a neural network learned from data ) that maps inputs to probabilities , where ∆K : = { ( p1 , . . . , pK ) : pi ≥ 0 , ∑ i pi = 1 } is the set of all probability distributions on [ K ] . We say p̂ is perfectly calibrated if P ( Y = k | p̂ ( X ) = p ) = pk for all p ∈ ∆K , k ∈ [ K ] . In other words , a model is perfectly calibrated if when the model predicts p̂ ( X ) = p , the conditional distribution of Y is exactly p. It is difficult to evaluate perfect calibration from finite data , as for almost all p we do not receive samples that satisfy the exact conditioning p̂ ( x ) = p. This motivates considering alternative scalar metrics for calibration that can be estimated from data . ECE The Expected Calibration Error ( ECE ) is a commonly used metric that measures calibration through grouping examples according to the confidence ( i.e . the top predicted probability ) ( Naeini et al. , 2015 ; Guo et al. , 2017 ) . Let { ( xi , yi ) } ni=1 be the evaluation dataset on which we wish to evaluate the calibration of a model p̂ . Define the intervals Im = ( m−1M , m M ] , where M > 0 is a ( fixed ) number of bins , and partitions the examples into M bins according to the confidence : Bm = { i : maxk p̂ ( xi ) k ∈ Im } . Define the accuracy and confidence within Bm as acc ( Bm ) : = 1 Bm ∑ i∈Bm 1 { arg max k p̂ ( xi ) k = yi } and conf ( Bm ) : = 1 Bm ∑ i∈Bm max k p̂ ( xi ) k The ECE is then defined as the ( weighted ) average difference between accuracy and confidence : ECE ( p̂ ) : = M∑ m=1 |Bm| n |acc ( Bm ) − conf ( Bm ) | . ( 1 ) The ECE is a sensible calibration metric since it is a binned approximation of the top-label calibration error ( TCE ) that measures the difference between accuracy and confidence under exact conditioning : TCE ( p̂ ) : = E [ ∣∣∣∣P ( arg max k p̂ ( X ) k = Y | max k p̂ ( X ) k ) −max k p̂ ( X ) k ∣∣∣∣ ] . Debiased ECE Recent work shows that the ECE has an inherent positive bias and proposes the Debiased ECE that approximately removes this bias using Gaussian bootstrapping ( Kumar et al. , 2019 ) : DECE ( p̂ ) : = ECE ( p̂ ) − ( ER1 : M [ M∑ m=1 |Bm| n |conf ( Bm ) −Rm| ] − ECE ( p̂ ) ) , where Rm ∼ N ( acc ( Bm ) , acc ( Bm ) ( 1−acc ( Bm ) ) |Bm| ) . Kumar et al . ( 2019 ) showed that the debiased ECE is a typically more accurate estimator of the TCE than ECE , especially when the TCE is relatively small . In our experiments , we use both the ECE and the debiased ECE for evaluating calibration . NLL The Negative Log-Likelihood ( NLL ) , typically used as loss function for training classifiers , is also a measure of calibration : NLL ( p̂ ) : = 1 n n∑ i=1 − log p̂ ( xi ) yi . ( 2 ) NLL is a proper scoring rule ( Lakshminarayanan et al. , 2017 ) in the sense that the population minimizer over all possible p̂ is achieved at the ground truth conditional distribution p ? ( Hastie et al. , 2009 ) . In general , the NLL measures the distance between p̂ and p ? , and is thus a joint metric of accuracy and calibration . Predictive entropy ( sharpness ) While we are mostly concered about the accuracy and calibration of a model , these two metrics alone do not fully guarantee a proper uncertainty quantification . For example , any high-accuracy model can be calibrated in a “ trivial ” way such that the ECE becomes exactly 0 , by mapping the confidence on all examples to be equal to the ( overall ) accuracy of the model , and rescaling the non-top probabilities accordingly . In order to prevent such trivial calibration , we additionally measure the sharpness of the predictions using the predictive entropy ( Lakshminarayanan et al. , 2017 ) : PEnt ( p̂ ) = 1 n n∑ i=1 K∑ k=1 −p̂ ( xi ) k log p̂ ( xi ) k. Lower predictive entropies indicate sharper predictions ( i.e . predictions closer to delta distributions than the uniform distribution ) . In general , the predictive entropy is not necessarily related to the calibration ; however , for models that have the same accuracies , we observe that the predictive entropy is typically negatively correlated with calibration—sharper predictions are usually less calibrated .

Summary: This paper proposes a post-calibration technique that is meant to be more powerful than temperature scaling without introducing overfitting. The authors argue that previous attempts to generalize temperature scaling have a tendency to overfit not because of the additional parameters, but rather because they are not rank preserving (and therefore can change model predictions / accuracy). The key idea is to use a higher-capacity (2-layer neural network) calibrator that is constrained to be rank-preserving so that model accuracy remains unchanged. The proposed method is compared against baselines on classification tasks from vision (CIFAR-100, ImageNet) and language (MNLI).