Revisiting the Calibration of Modern Neural Networks

1 Introduction . Neural networks , especially vision models , are increasingly used in safety-critical applications such as autonomous driving ( Bojarski et al. , 2016 ) , medical diagnosis ( Esteva et al. , 2017 ; Jiang et al. , 2012 ) , and meteorological forecasting ( Sønderby et al. , 2020 ) . For such applications , it is essential that model predictions are not just accurate , but also well calibrated . Model calibration refers to the accuracy with which the scores provided by the model reflect its predictive uncertainty . For example , in a medical application , we would like to defer images for which the model makes low-confidence predictions to a physician for review ( Kompa et al. , 2021 ) . Skipping human review due to confident , but incorrect , predictions , could have disastrous consequences . While intense research and engineering effort has focused on improving the predictive accuracy of models , less attention has been given to model calibration . In fact , over the last few years , there have been many reports that calibration of modern neural networks can be surprisingly poor , despite the advances in accuracy ( e.g . Guo et al . 2017 ; Lakshminarayanan et al . 2017 ; Malinin & Gales 2018 ; Thulasidasan et al . 2019 ; Hendrycks et al . 2020b ; Ovadia et al . 2019 ; Wenzel et al . 2020 ; Havasi et al . 2021 ; Rahaman & Thiery 2020 ; Leathart & Polaczuk 2020 ) . Some works suggest a trend for larger , more accurate models to be worse calibrated ( Guo et al. , 2017 ) . These concerns are more relevant than ever , since the architecture size , amount of training data , and computing power used by state-of-the-art models continue to increase . At the same time , rapid advances in model architecture ( Tolstikhin et al. , 2021 ; Dosovitskiy et al. , 2021 ) and training approaches ( Chen et al. , 2020 ; Mahajan et al. , 2018 ; Radford et al. , 2021 ) raise the question whether past results on calibration , largely obtained on standard convolutional architectures , extend to current state-of-the-art models . Since model advances are quickly translated to real-world , safety-critical applications ( e.g . Mustafa et al . 2021 ) , there is an urgent need to re-assess the calibration properties of current state-of-the-art models . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Contributions . To address this need , we provide a systematic comparison of recent image classification models , relating their accuracy , calibration , and design features . We find that : 1 . The best current models , including the non-convolutional MLP-Mixer ( Tolstikhin et al. , 2021 ) and Vision Transformers ( Dosovitskiy et al. , 2021 ) , are well calibrated compared to past models and their performance is more robust to distribution shift . 2 . In-distribution calibration slightly deteriorates with increasing model size , but this is outweighed by a simultaneous improvement in accuracy . 3 . Under distribution shift , calibration improves with model size , reversing the trend seen indistribution . 4 . Accuracy and calibration are correlated under distribution shift , such that optimizing for accuracy may also benefit calibration . 5 . Model size , pretraining duration , and pretraining dataset size can not fully explain differences in calibration properties between model families . Our results suggest that further improvements in model accuracy will continue to benefit calibration . They also hint at architecture as an important determinant of model calibration . We provide code and a large dataset of calibration measurements , comprising 180 distinct models from 16 families , each evaluated on 79 ImageNet-scale datasets and 28 metric variants.1 2 Related Work . Measures of model calibration . The losses that are commonly used to train classification models , such as cross-entropy and squared error , are proper scoring rules ( Gneiting et al. , 2007 ) and are therefore guaranteed to yield perfectly calibrated models at their minimum—in the infinite-data limit . However , in practice , due to model mismatch and overfitting , even losses based on proper scoring rules may result in poor model calibration . Miscalibration is commonly quantified in terms of Expected Calibration Error ( ECE ; Naeini et al . 2015 ) , which measures the absolute difference between predictive confidence and accuracy . We focus on ECE because it is a widely used and accepted calibration metric . Nevertheless , it is well understood that estimating ECE accurately is difficult because estimators can be strongly biased and many estimator variants exist ( Nixon et al. , 2019 ; Roelofs et al. , 2020 ; Vaicenavicius et al. , 2019 ; Gupta et al. , 2021 ) . Section 5 discusses these issues and our approaches to mitigate them . Alternatives to ECE include likelihood measures , Brier score ( Brier , 1950 ) , Bayesian methods ( Gelman et al. , 2013 ) , and conformal prediction ( Shafer & Vovk , 2008 ) . Further , model calibration can be represented visually with reliability diagrams ( DeGroot & Fienberg , 1983 ) . Figure 8 and Appendix F provide likelihoods , Brier scores , and reliability diagrams for our main analyses . Empirical studies of model calibration . There have been many recent empirical studies on the robustness ( accuracy under distribution shift ) of image classifiers ( Geirhos et al. , 2019 ; Taori et al. , 2020 ; Djolonga et al. , 2020 ; Hendrycks et al. , 2020a ) . Several works have also studied calibration . Most notable is Guo et al . ( 2017 ) , who found that “ modern neural networks , unlike those from a decade ago , are poorly calibrated ” , that larger networks tend to be calibrated worse , and that “ miscalibration worsen [ s ] even as classification error is reduced. ” Other works have corroborated some of these findings ( e.g. , Thulasidasan et al . 2019 ; Wen et al . 2021 ) . This line of work suggests a trend that larger models are worse calibrated , which would have major implications for research toward bigger models and datasets . We show that for more recent models , this trend is negligible in-distribution and in fact reverses under distribution shift . Ovadia et al . ( 2019 ) empirically study calibration under distribution shift and provide a large comparison of methods for improving calibration . They report that both accuracy and calibration deteriorate with distribution shift . While we observe the same trend , we find that the calibration of some recent model families decays so slowly under distribution shift that the decay in accuracy is likely more relevant in practice ( Section 4.3 ) . Ovadia et al . also find that , across methods for improving calibration , improvements on in-distribution data do not necessarily translate to out-of-distribution data . This finding may suggest that there is little correlation between in-distribution and out-of-distribution calibration in general . However , our 1Available at https : //github.com/google-research/robustness_metrics/tree/master/ robustness_metrics/projects/revisiting_calibration . results show that , across model architectures , the models with the best in-distribution calibration are also the best-calibrated on a range of out-of-distribution benchmarks . The important implication of this result is that designing models based on in-distribution performance likely also benefits their out-of-distribution performance . Improving calibration . Many strategies have been proposed to improve model calibration such as post-hoc rescaling of predictions ( Guo et al. , 2017 ) , averaging multiple predictions ( Lakshminarayanan et al. , 2017 ; Wen et al. , 2020 ) , and data augmentation ( Thulasidasan et al. , 2019 ; Wen et al. , 2021 ) . Here , we focus on the intrinsic calibration properties of state-of-the-art model families , rather than methods to further improve calibration . As a baseline on top of a model ’ s intrinsic calibration properties , we study temperature scaling ( Guo et al. , 2017 ) . It is effective in improving calibration and so simple that it can be applied in many cases at minimal additional cost , in contrast to many more sophisticated methods . Temperature scaling re-scales a model ’ s logits by a single parameter , chosen to optimize the model ’ s likelihood on a held-out portion of the training data . This temperature factor changes the model ’ s confidence , i.e. , whether the model predictions are on average too certain ( overconfident ) , optimally confident , or too uncertain ( underconfident ) . The classification accuracy of the model is not affected by temperature scaling . A large fraction of model miscalibration is typically due to average over- or underconfidence , e.g . due to suboptimal training duration ( Guo et al. , 2017 ) . By normalizing a model ’ s confidence , temperature scaling not only improves calibration , but also removes a primary confounder that can hide trends in calibration between models ( see Section 4.2 and Appendix D ) . Therefore , we study both unscaled and temperature-scaled predictions in the paper . 3 Definitions and Notation . We consider the multi-class classification problem , as analyzed by Bröcker ( 2009 ) , where we observe a variable X and predict a categorical variable Y ∈ { 1 , 2 , . . . , k } . We model our predictor f as a function that maps every input instanceX to a categorical distribution over k labels , represented using a vector f ( X ) belonging to the ( k − 1 ) -dimensional simplex ∆ = { p ∈ [ 0 , 1 ] k | ∑k y=1 py = 1 } . Intuitively , a model f is well-calibrated if its output truthfully quantifies the predictive uncertainty . For example , if we take all data points x for which the model predicts [ f ( x ) ] y = 0.3 , we expect 30 % of them to indeed take on the label y . Formally , the model f is said to be calibrated if ( Bröcker , 2009 ) ∀p ∈ ∆ : P ( Y = y | f ( X ) = p ) = py . ( 1 ) We will focus on a slightly weaker , but more practical condition , called top-label or argmax calibration ( Kumar et al. , 2019 ; Guo et al. , 2017 ) . This requires that the above holds only for the most likely label , i.e. , ∀p∗ ∈ [ 0 , 1 ] P ( Y ∈ arg max p | max f ( X ) = p∗ ) = p∗ , ( 2 ) where the max and arg max act coordinate-wise . The most common measure of the degree of miscalibration is the Expected Calibration Error ( ECE ) , which computes the expected disagreement between the two sides of eq . ( 2 ) E [ |p∗ − E [ Y ∈ arg max f ( X ) | max f ( X ) = p∗| ] . ( 3 ) Unfortunately , eq . ( 3 ) can not be estimated without quantization as it conditions on a null event . Hence , one typically first buckets the predictions intom binsB1 , . . . , Bm based on their top predicted probability , and then takes the expectation over these buckets . Namely , if we are given a set of n i.i.d . samples ( x1 , y1 ) , . . . , ( xn , yn ) distributed as P ( X , Y ) , then we assign each j ∈ { 1 , . . . , n } to a bucket Bi based on max f ( xj ) . Then , we compute in each bucket Bi the confidence ( Bi ) = 1 |Bi| ∑ j∈Bi max f ( xj ) and the accuracy ( Bi ) = 1 |Bi| ∑ j∈BiJyj ∈ arg max f ( xj ) K , where J·K is the Iverson bracket . Finally , we construct an estimator by taking the expectation over the bins ÊCE = m∑ i=1 |Bi| n |accuracy ( Bi ) − confidence ( Bi ) | . ( 4 ) In Section 5 we discuss the statistical properties of this estimator , possible pitfalls , and several mitigation strategies .

Summary: The paper systematically evaluates the relation between model calibration and accuracy. The main finding is that the most recent models, notably those not using convolutions, are among the best calibrated. In addition, the analysis shows that model size and amount of pretraining do not explain these differences. Experimental evaluation is extensive and very carefully performed.