Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning

video / code / blog post 1 INTRODUCTION . Deep neural networks ( DNNs ) have become one of the most popular families of machine learning models . The predictive performance of DNNs for classification is often measured in terms of accuracy . However , DNNs have been shown to yield inaccurate and unreliable probability estimates , or predictive uncertainty ( Guo et al. , 2017 ) . This has brought considerable attention to the problem of uncertainty estimation with deep neural networks . There are many faces to uncertainty estimation . Different desirable uncertainty estimation properties of a model require different settings and metrics to capture them . Out-of-domain uncertainty of the model is measured on the data that does not follow the same distribution as the training dataset ( out-of-domain data ) . Out-of-domain data can include images corrupted with rotations or blurring , adversarial attacks ( Szegedy et al. , 2013 ) or data points from a completely different dataset . The model is expected to be resistant to data corruptions and to be more uncertain on out-of-domain data than on in-domain data . On the contrary , in-domain uncertainty of the model is measured on data taken from the training data distribution , i.e . data from the same domain . In this setting the model is expected to produce reliable probability estimates , e.g . the model shouldn ’ t be too overconfident in its wrong predictions . Pitfalls of metrics We show that many common metrics of in-domain uncertainty estimation ( e.g . log-likelihood , Brier score , calibration metrics , etc . ) are either not comparable across different models or fail to provide a reliable ranking . We address some of the stated pitfalls and point out more reasonable evaluation schemes . For instance , although temperature scaling is not a standard for ensembling techniques , it is a must for a fair evaluation . With this in mind , the ∗Equal contribution §HSE refers to National Research University Higher School of Economics †Skoltech refers to Skolkovo Institute of Science and Technology ‡HSE refers to Samsung-HSE Laboratory , National Research University Higher School of Economics calibrated log-likelihood avoids most of the stated pitfalls and generally is a reasonable metric for in-domain uncertainty estimation task . Pitfalls of ensembles Equipped with the proposed evaluation framework , we are revisiting the evaluation of ensembles of DNNs—one of the major tools for uncertainty estimation . We introduce the deep ensemble equivalent ( DEE ) score that measures the number of independently trained models that , when ensembled , achieve the same performance as the ensembling technique of interest . The DEE score allows us to compare ensembling techniques across different datasets and architectures using a unified scale . Our study shows that most of the popular ensembling techniques require averaging predictions across dozens of samples ( members of an ensemble ) , yet are essentially equivalent to an ensemble of only few independently trained models . Missing part of ensembling In our study , test-time data augmentation ( TTA ) turned out to be a surprisingly strong baseline for uncertainty estimation and a simple way to improve ensembles . Despite being a popular technique in large-scale classification , TTA seems to be overlooked in the community of uncertainty estimation and ensembling . 2 SCOPE OF THE PAPER . We use standard benchmark problems of image classification which comprise a common setting in research on learning ensembles of neural networks . There are other relevant settings where the correctness of probability estimates can be a priority , and ensembling techniques are used to improve it . These settings include , but are not limited to , regression , language modeling ( Gal , 2016 ) , image segmentation ( Gustafsson et al. , 2019 ) , active learning ( Settles , 2012 ) and reinforcement learning ( Buckman et al. , 2018 ; Chua et al. , 2018 ) . We focus on in-domain uncertainty , as opposed to out-of-domain uncertainty . Out-of-domain uncertainty includes detection of inputs that come from a completely different domain or have been corrupted by noise or adversarial attacks . This setting has been thoroughly explored by ( Ovadia et al. , 2019 ) . We only consider methods that are trained on clean data with simple data augmentation . Some other methods use out-of-domain data ( Malinin & Gales , 2018 ) or more elaborate data augmentation , e.g . mixup ( Zhang et al. , 2017 ) or adversarial training ( Lakshminarayanan et al. , 2017 ) to improve accuracy , robustness and uncertainty . We use conventional training procedures . We use the stochastic gradient descent ( SGD ) and use batch normalization ( Ioffe & Szegedy , 2015 ) , both being the de-facto standards in modern deep learning . We refrain from using more elaborate optimization techniques including works on superconvergence ( Smith & Topin , 2019 ) and stochastic weight averaging ( SWA ) ( Izmailov et al. , 2018 ) . These techniques can be used to drastically accelerate training and to improve the predictive performance . Thus , we do not comment on the training time of different ensembling methods since the use of these and other more efficient training techniques would render such a comparison obsolete . A number of related works study ways of approximating and accelerating prediction in ensembles . The distillation mechanism allows to approximate the prediction of an ensemble by a single neural network ( Hinton et al. , 2015 ; Balan et al. , 2015 ; Tran et al. , 2020 ) , whereas fast dropout ( Wang & Manning , 2013 ) and deterministic variational inference ( Wu et al. , 2018 ) allow to approximate the predictive distribution of specific stochastic computation graphs . We measure the raw power of ensembling techniques without these approximations . All of the aforementioned alternative settings are orthogonal to the scope of this paper and are promising points of interest for further research . 3 PITFALLS OF IN-DOMAIN UNCERTAINTY ESTIMATION . No single metric measures all the desirable properties of uncertainty estimates obtained by a model of interest . Because of this , the community is using many different metrics in an attempt to capture the quality of uncertainty estimation , such as the Brier score ( Brier , 1950 ) , log-likelihood ( Quinonero-Candela et al. , 2005 ) , metrics of calibration ( Guo et al. , 2017 ; Nixon et al. , 2019 ) , performance of misclassification detection ( Malinin & Gales , 2018 ) , and threshold–accuracy curves ( Lakshminarayanan et al. , 2017 ) . In the section we highlight the pitfalls of the aforementioned metrics , and demonstrate that these pitfalls can significantly affect evaluation , changing the ranking of the methods . Notation We consider a classification problem with a dataset that consists of N training and n testing pairs ( xi , y∗i ) ∼ p ( x , y ) , where xi is an object and y∗i ∈ { 1 , . . . , C } is a discrete class label . A probabilistic classifier maps an object xi into a predictive distribution p̂ ( y |xi ) . The predictive distribution p̂ ( y |xi ) of a deep neural network is typically defined by the softmax function p̂ ( y |x ) = Softmax ( z ( x ) /T ) , where z ( x ) is a vector of logits and T is a scalar parameter standing for the temperature of the predictive distribution . This scalar parameter is usually set to T = 1 or is tuned on a validation set ( Guo et al. , 2017 ) . The maximum probability maxc p̂ ( y = c |xi ) is called the confidence of a classifier p̂ on an object xi . I [ · ] denotes the indicator function throughout the text . 3.1 LOG-LIKELIHOOD AND BRIER SCORE . The average test log-likelihood LL = 1n ∑n i=1 log p̂ ( y = y ∗ i |xi ) is a popular metric for measuring the quality of in-domain uncertainty of deep learning models . It directly penalizes high probability scores assigned to incorrect labels and low probability scores assigned to the correct labels y∗i . LL is sensitive to the softmax temperature T . The temperature that has been implicitly learned during training can be far from optimal for the test data . However , a nearly optimal temperature can be found post-hoc by maximizing the log-likelihood on validation data . This approach is called temperature scaling or calibration ( Guo et al. , 2017 ) . Despite its simplicity , the temperature scaling results in a notable improvement in the LL . While ensembling techniques tend to have better temperature than single models , the default choice of T = 1 is still suboptimal . Comparing the LL with suboptimal temperatures—that is often the case in practice—can potentially produce an arbitrary ranking of different methods . Comparison of the log-likelihood should only be performed at the optimal temperature . Empirically , we demonstrate that the overall ordering of methods and also the best ensembling method according to the LL can vary depending on temperature T . While this applies to most ensembling techniques ( see Figure 10 ) , this effect is most noticeable on experiments with data augmentation on ImageNet ( Figure 1 ) . We introduce a new metric called the calibrated log-likelihood that is the log-likelihood at the optimal temperature . The calibrated log-likelihood considers a model and a post-training calibration as a unified system , targeting to measure all models in the equal conditions of optimal temperature . That allows to avoid measuring calibration error that can be eliminated by a simple temperature scaling . The metric significantly affects the results of the comparison . For example , in Figure 10 the differences between Bayesian ( VI , K-FAC , SWAG , dropout ) and conventional non-Bayesian networks become much less pronounced , and in most cases making conventional non-Bayesian networks match the performance of Bayesian ones ( VI , K-FAC , Dropout ) on ResNet110 , ResNet164 , and WideResNet . We show how to obtain an unbiased estimate of the calibrated log-likelihood without a held-out validation set in Section 3.5 . LL also demonstrates a high correlation with accuracy ( ρ > 0.86 ) , that in case of calibrated LL becomes even stronger ( ρ > 0.95 ) . That suggests that while ( calibrated ) LL measures the uncertainty of the model , it still has a significant dependence on the accuracy and vice versa . A model with higher accuracy would likely have a higher log-likelihood . See Figure 9 in Appendix C for more details . Brier score BS = 1n 1 C ∑n i=1 ∑C c=1 ( I [ y∗i = c ] − p̂ ( y = c |xi ) ) 2 has also been known for a long time as a metric for verification of predicted probabilities ( Brier , 1950 ) . Similarly to the log-likelihood , the Brier score penalizes low probabilities assigned to correct predictions and high probabilities assigned to wrong ones . It is also sensitive to the temperature of the softmax distribution and behaves similarly to the log-likelihood . While these metrics are not strictly equivalent , they show a high empirical correlation for a wide range of models on CIFAR-10 , CIFAR-100 and ImageNet datasets ( see Figure 8 in Appendix C ) . 3.2 MISCLASSIFICATION DETECTION . Detection of wrong predictions of the model , or misclassifications , is a popular downstream problem relevant to the problem of in-domain uncertainty estimation . Since misclassification detection is essentially a binary classification problem , some papers measure its quality using conventional metrics for binary classification such as AUC-ROC and AUC-PR ( Malinin & Gales , 2018 ; Cui et al. , 2019 ; Możejko et al. , 2018 ) . These papers use an uncertainty criterion like confidence or predictive entropyH [ p̂ ( y |xi ) ] as a prediction score . While these metrics can be used to assess the misclassification detection performance of a single model , they can not be used to directly compare misclassification performance across different models . Correct and incorrect predictions are specific for every model , therefore , every model induces its own binary classification problem . The induced problems can differ significantly , since different models produce different confidences and misclassify different objects . In other words , comparing such metrics implies a comparison of performance of classifiers that solve different classification problems . Such metrics are therefore incomparable . AUCs for misclassification detection can not be directly compared between different models . While comparing AUCs is incorrect in the setting of misclassification detection , it is correct to compare these metrics in many out-of-domain data detection problems . In that case , both objects and targets of the induced binary classification problems remain the same for all models . All outof-domain objects have a positive label and all in-domain objects have a negative label . Note that this condition does not necessarily hold in the problem of detection of adversarial attacks . Different models generally have different inputs after an adversarial attack , so such AUC-based metrics might still be flawed .

Summary: This paper mainly concerns the quality of in-domain uncertainty for image classification. After exploring common standards for uncertainty quantification, the authors point out pitfalls of existing metrics by investigating different ensembling techniques and introduce a novel metric called deep ensemble equivalent (DEE) that essentially measures the number of independent models in an ensemble of DNNs. Based on the DEE score, a detailed evaluation of modern DNN ensembles is performed on CIFAR-10/100 and ImageNet datasets.