Revisiting the Calibration of Modern Neural Networks

1 Introduction . Neural networks , especially vision models , are increasingly used in safety-critical applications such as autonomous driving ( Bojarski et al. , 2016 ) , medical diagnosis ( Esteva et al. , 2017 ; Jiang et al. , 2012 ) , and meteorological forecasting ( Sønderby et al. , 2020 ) . For such applications , it is essential that model predictions are not just accurate , but also well calibrated . Model calibration refers to the accuracy with which the scores provided by the model reflect its predictive uncertainty . For example , in a medical application , we would like to defer images for which the model makes low-confidence predictions to a physician for review ( Kompa et al. , 2021 ) . Skipping human review due to confident , but incorrect , predictions , could have disastrous consequences . While intense research and engineering effort has focused on improving the predictive accuracy of models , less attention has been given to model calibration . In fact , over the last few years , there have been many reports that calibration of modern neural networks can be surprisingly poor , despite the advances in accuracy ( e.g . Guo et al . 2017 ; Lakshminarayanan et al . 2017 ; Malinin & Gales 2018 ; Thulasidasan et al . 2019 ; Hendrycks et al . 2020b ; Ovadia et al . 2019 ; Wenzel et al . 2020 ; Havasi et al . 2021 ; Rahaman & Thiery 2020 ; Leathart & Polaczuk 2020 ) . Some works suggest a trend for larger , more accurate models to be worse calibrated ( Guo et al. , 2017 ) . These concerns are more relevant than ever , since the architecture size , amount of training data , and computing power used by state-of-the-art models continue to increase . At the same time , rapid advances in model architecture ( Tolstikhin et al. , 2021 ; Dosovitskiy et al. , 2021 ) and training approaches ( Chen et al. , 2020 ; Mahajan et al. , 2018 ; Radford et al. , 2021 ) raise the question whether past results on calibration , largely obtained on standard convolutional architectures , extend to current state-of-the-art models . Since model advances are quickly translated to real-world , safety-critical applications ( e.g . Mustafa et al . 2021 ) , there is an urgent need to re-assess the calibration properties of current state-of-the-art models . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Contributions . To address this need , we provide a systematic comparison of recent image classification models , relating their accuracy , calibration , and design features . We find that : 1 . The best current models , including the non-convolutional MLP-Mixer ( Tolstikhin et al. , 2021 ) and Vision Transformers ( Dosovitskiy et al. , 2021 ) , are well calibrated compared to past models and their performance is more robust to distribution shift . 2 . In-distribution calibration slightly deteriorates with increasing model size , but this is outweighed by a simultaneous improvement in accuracy . 3 . Under distribution shift , calibration improves with model size , reversing the trend seen indistribution . 4 . Accuracy and calibration are correlated under distribution shift , such that optimizing for accuracy may also benefit calibration . 5 . Model size , pretraining duration , and pretraining dataset size can not fully explain differences in calibration properties between model families . Our results suggest that further improvements in model accuracy will continue to benefit calibration . They also hint at architecture as an important determinant of model calibration . We provide code and a large dataset of calibration measurements , comprising 180 distinct models from 16 families , each evaluated on 79 ImageNet-scale datasets and 28 metric variants.1 2 Related Work . Measures of model calibration . The losses that are commonly used to train classification models , such as cross-entropy and squared error , are proper scoring rules ( Gneiting et al. , 2007 ) and are therefore guaranteed to yield perfectly calibrated models at their minimum—in the infinite-data limit . However , in practice , due to model mismatch and overfitting , even losses based on proper scoring rules may result in poor model calibration . Miscalibration is commonly quantified in terms of Expected Calibration Error ( ECE ; Naeini et al . 2015 ) , which measures the absolute difference between predictive confidence and accuracy . We focus on ECE because it is a widely used and accepted calibration metric . Nevertheless , it is well understood that estimating ECE accurately is difficult because estimators can be strongly biased and many estimator variants exist ( Nixon et al. , 2019 ; Roelofs et al. , 2020 ; Vaicenavicius et al. , 2019 ; Gupta et al. , 2021 ) . Section 5 discusses these issues and our approaches to mitigate them . Alternatives to ECE include likelihood measures , Brier score ( Brier , 1950 ) , Bayesian methods ( Gelman et al. , 2013 ) , and conformal prediction ( Shafer & Vovk , 2008 ) . Further , model calibration can be represented visually with reliability diagrams ( DeGroot & Fienberg , 1983 ) . Figure 8 and Appendix F provide likelihoods , Brier scores , and reliability diagrams for our main analyses . Empirical studies of model calibration . There have been many recent empirical studies on the robustness ( accuracy under distribution shift ) of image classifiers ( Geirhos et al. , 2019 ; Taori et al. , 2020 ; Djolonga et al. , 2020 ; Hendrycks et al. , 2020a ) . Several works have also studied calibration . Most notable is Guo et al . ( 2017 ) , who found that “ modern neural networks , unlike those from a decade ago , are poorly calibrated ” , that larger networks tend to be calibrated worse , and that “ miscalibration worsen [ s ] even as classification error is reduced. ” Other works have corroborated some of these findings ( e.g. , Thulasidasan et al . 2019 ; Wen et al . 2021 ) . This line of work suggests a trend that larger models are worse calibrated , which would have major implications for research toward bigger models and datasets . We show that for more recent models , this trend is negligible in-distribution and in fact reverses under distribution shift . Ovadia et al . ( 2019 ) empirically study calibration under distribution shift and provide a large comparison of methods for improving calibration . They report that both accuracy and calibration deteriorate with distribution shift . While we observe the same trend , we find that the calibration of some recent model families decays so slowly under distribution shift that the decay in accuracy is likely more relevant in practice ( Section 4.3 ) . Ovadia et al . also find that , across methods for improving calibration , improvements on in-distribution data do not necessarily translate to out-of-distribution data . This finding may suggest that there is little correlation between in-distribution and out-of-distribution calibration in general . However , our 1Available at https : //github.com/google-research/robustness_metrics/tree/master/ robustness_metrics/projects/revisiting_calibration . results show that , across model architectures , the models with the best in-distribution calibration are also the best-calibrated on a range of out-of-distribution benchmarks . The important implication of this result is that designing models based on in-distribution performance likely also benefits their out-of-distribution performance . Improving calibration . Many strategies have been proposed to improve model calibration such as post-hoc rescaling of predictions ( Guo et al. , 2017 ) , averaging multiple predictions ( Lakshminarayanan et al. , 2017 ; Wen et al. , 2020 ) , and data augmentation ( Thulasidasan et al. , 2019 ; Wen et al. , 2021 ) . Here , we focus on the intrinsic calibration properties of state-of-the-art model families , rather than methods to further improve calibration . As a baseline on top of a model ’ s intrinsic calibration properties , we study temperature scaling ( Guo et al. , 2017 ) . It is effective in improving calibration and so simple that it can be applied in many cases at minimal additional cost , in contrast to many more sophisticated methods . Temperature scaling re-scales a model ’ s logits by a single parameter , chosen to optimize the model ’ s likelihood on a held-out portion of the training data . This temperature factor changes the model ’ s confidence , i.e. , whether the model predictions are on average too certain ( overconfident ) , optimally confident , or too uncertain ( underconfident ) . The classification accuracy of the model is not affected by temperature scaling . A large fraction of model miscalibration is typically due to average over- or underconfidence , e.g . due to suboptimal training duration ( Guo et al. , 2017 ) . By normalizing a model ’ s confidence , temperature scaling not only improves calibration , but also removes a primary confounder that can hide trends in calibration between models ( see Section 4.2 and Appendix D ) . Therefore , we study both unscaled and temperature-scaled predictions in the paper . 3 Definitions and Notation . We consider the multi-class classification problem , as analyzed by Bröcker ( 2009 ) , where we observe a variable X and predict a categorical variable Y ∈ { 1 , 2 , . . . , k } . We model our predictor f as a function that maps every input instanceX to a categorical distribution over k labels , represented using a vector f ( X ) belonging to the ( k − 1 ) -dimensional simplex ∆ = { p ∈ [ 0 , 1 ] k | ∑k y=1 py = 1 } . Intuitively , a model f is well-calibrated if its output truthfully quantifies the predictive uncertainty . For example , if we take all data points x for which the model predicts [ f ( x ) ] y = 0.3 , we expect 30 % of them to indeed take on the label y . Formally , the model f is said to be calibrated if ( Bröcker , 2009 ) ∀p ∈ ∆ : P ( Y = y | f ( X ) = p ) = py . ( 1 ) We will focus on a slightly weaker , but more practical condition , called top-label or argmax calibration ( Kumar et al. , 2019 ; Guo et al. , 2017 ) . This requires that the above holds only for the most likely label , i.e. , ∀p∗ ∈ [ 0 , 1 ] P ( Y ∈ arg max p | max f ( X ) = p∗ ) = p∗ , ( 2 ) where the max and arg max act coordinate-wise . The most common measure of the degree of miscalibration is the Expected Calibration Error ( ECE ) , which computes the expected disagreement between the two sides of eq . ( 2 ) E [ |p∗ − E [ Y ∈ arg max f ( X ) | max f ( X ) = p∗| ] . ( 3 ) Unfortunately , eq . ( 3 ) can not be estimated without quantization as it conditions on a null event . Hence , one typically first buckets the predictions intom binsB1 , . . . , Bm based on their top predicted probability , and then takes the expectation over these buckets . Namely , if we are given a set of n i.i.d . samples ( x1 , y1 ) , . . . , ( xn , yn ) distributed as P ( X , Y ) , then we assign each j ∈ { 1 , . . . , n } to a bucket Bi based on max f ( xj ) . Then , we compute in each bucket Bi the confidence ( Bi ) = 1 |Bi| ∑ j∈Bi max f ( xj ) and the accuracy ( Bi ) = 1 |Bi| ∑ j∈BiJyj ∈ arg max f ( xj ) K , where J·K is the Iverson bracket . Finally , we construct an estimator by taking the expectation over the bins ÊCE = m∑ i=1 |Bi| n |accuracy ( Bi ) − confidence ( Bi ) | . ( 4 ) In Section 5 we discuss the statistical properties of this estimator , possible pitfalls , and several mitigation strategies .

Summary: The paper evaluates numerous recent models in terms of uncertainty calibration. The models evaluated are recent advances in relation to a previous seminal paper on the topic. The findings of this work indicate that previous trends leading to overconfidence with larger models may not apply to the current state-of-the-art models.