How to measure deep uncertainty estimation performance and which models are naturally better at providing it

1 INTRODUCTION . Deep neural networks ( DNNs ) show great performance in a wide variety of application domains including computer vision , natural language understanding and audio processing . Successful deployment of these models , however , is critically dependent on providing an effective uncertainty estimation of their predictions in the form of some kind of selective prediction or providing a probabilistic confidence score for their predictions . But how should we evaluate the performance of uncertainty estimation ? Let us consider two classification models for the stock market that predict whether a stock ’ s value is about to increase , decrease or remain neutral ( three-class classification ) . Suppose that model A has a 95 % true accuracy , and generates a confidence score of 0.95 on every prediction ( even on misclassified instances ) ; model B has a 40 % true accuracy , but always gives a confidence score of 0.6 on correct predictions , and 0.4 on incorrect ones . Model B can be utilized easily to generate perfect investment decisions . Using selective prediction ( Geifman & El-Yaniv , 2017 ) , Model B will reject all investments on stocks whenever the confidence score is 0.4 . While model A offers many more investment opportunities , each of its predictions carries a 5 % risk of failure . Among the various metrics proposed for evaluating the performance of uncertainty estimation are : Area Under the Receiver Operating Characteristic ( AUROC or AUC ) , Area Under the RiskCoverage curve ( AURC ) ( Geifman et al. , 2018 ) , selective risk or coverage for a selective accuracy constraint ( SAC ) , Negative Log-likelihood ( NLL ) , Expected Calibration Error ( ECE ) , which is of- ten used for evaluating a model ’ s calibration ( see Section 2 ) and Brier score ( Brier , 1950 ) . All these metrics are well known and are often used for comparing the uncertainty estimation performance of models ( Moon et al. , 2020 ; Nado et al. , 2021 ; Maddox et al. , 2019 ; Lakshminarayanan et al. , 2017 ) . Somewhat surprisingly , NLL , Brier , AURC , and ECE all fail to reveal the uncertainty superiority of Model B in our investment example ( see Appendix A for the calculations ) . Both AUROC and SAC , on the other hand , reveal the advantage of Model B perfectly ( see Appendix A for details ) . It is not hard to construct counter examples where these two metrics fails and others ( e.g. , ECE ) succeed . The risk-coverage ( RC ) curve ( El-Yaniv & Wiener , 2010 ) is perhaps one of the most informative and practical representations of the overall uncertainty profile of a given model . In general , though , two RC curves are not necessarily comparable if one does not fully dominate the other ( see Figure 2 ) . The advantage of scalar metrics such as the above is that they summarize the model ’ s overall uncertainty estimation behavior by reducing it to a single scalar . When not carefully chosen , however , these reductions could result in a loss of vital information about the problem ( for example , reducing an RC curve to an AURC does not show that Model B has an optimal 0 risk if the coverage is smaller than 0.4 ) . Thus , the choice of the “ correct ” single scalar performance metric unfortunately must be task-specific . When comparing the uncertainty estimation performance of deep architectures that exhibit different accuracies , we find that AUROC and SAC can effectively “ normalize ” accuracy differences that plague the usefulness of other metrics ( see Section 2 ) . This normalization is essential to our study where we compare uncertainty performance of hundreds of models that can greatly differ in their accuracies . In applications where risk ( or coverage ) constraints are dictated ( Geifman & El-Yaniv , 2017 ) , the most straightforward and natural metric is the SAC ( or selective risk ) , which directly measures the coverage ( resp. , risk ) given at the required level of risk ( resp. , coverage ) constraint . We demonstrate this in Appendix J , evaluating which models give the most coverage for a SAC of 99 % . Sometimes , however , such constraints are unknown in advance , or even irrelevant , e.g. , the constructed model should serve a variety of risk constraint use cases , or the model may not be allowed to abstain from predicting at all . In this paper we conduct a comprehensive study of DNNs ’ ability to estimate uncertainty by evaluating 484 models pretrained on ImageNet ( Deng et al. , 2009 ) , taken from the PyTorch and timm respositories ( Paszke et al. , 2019 ; Wightman , 2019 ) . We identify the main factors contributing to or harming the confidence ranking of predictions ( “ ranking ” for short ) , calibration and selective prediction . Furthermore , we also consider the source of uncertainty as either internal ( stemming from either the aleatoric or epistemic uncertainty of the model ( Kiureghian & Ditlevsen , 2009 ) ) or external ( originating from unseen or unknown class-out-of-distribution ( C-OOD ) data ) and evaluate these models in multiple ways . After first evaluating models solely on in-distribution ( ID ) data , we then define and test two ways of evaluating C-OOD data , each of which also divides the data into different groups by how difficult it is for the model to distinguish instances as external . Our study lead to quite a few new observations and conclusions ; ( 1 ) Training regimes incorporating any kind of knowledge distillation ( KD ) ( Hinton et al. , 2015 ) leads to DNNs with improved uncertainty estimation performance evaluated by any metric , in both internal and external settings ( i.e. , leading also to better C-OOD detection ) , more than by using any other training tricks ( such as pretraining on a larger dataset , adversarial training , etc. ) . ( 2 ) Some architectures are naturally superb at all aspects of uncertainty estimation and in all settings , e.g. , vision transformers ( ViTs ) ( Dosovitskiy et al. , 2020 ; Steiner et al. , 2021 ) , while other architectures tend to perform worse , e.g. , EfficientNetV2 and GENet ( Tan & Le , 2021 ; Lin et al. , 2020 ) . These results are visualized in Figure 1 . ( 3 ) The superiority of ViTs remains even when the comparison considers the models ’ sizes—meaning that for any size , ViTs outperform the competition in uncertainty estimation performance , as visualized in Appendix B in Figures 9 and 10 . ( 4 ) The simple post-training calibration method of temperature scaling ( Guo et al. , 2017 ) , which is known to improve ECE , for the most part also improves ranking ( AUROC ) and selective prediction—meaning not only does it calibrate the probabilistic estimation for each individual instance , but it also improves the partial order of all instances induced by those improved estimations , pushing instances more likely to be correct to have higher confidence than instances less likely to be correct ( see Section 3 ) . ( 5 ) Contrary to previous work by Guo et al . ( 2017 ) , we observe that while there is a strong correlation between accuracy/number of parameters and ECE or AUROC within each specific family of models of the same architecture , the correlation flips between a strong negative and a strong positive correlation depending on the type of architecture being observed . For example , as ViT architectures increase in size and accuracy , their ECE deteriorates while their AUROC improves . The exact opposite , however , could be observed in XCiTs ( El-Nouby et al. , 2021 ) as their ECE improves with size while their AUROC deteriorates . ( see Appendix G ) . ( 6 ) The best model in terms of AUROC or SAC is not always the best in terms of calibration , as illustrated in Figure 1 , and the trade-off should be considered when choosing a model based on its application . Due to lack of space , a number of additional interesting observations are briefly mentioned in the paper without supporting empirical evidence ( which is provided in the appendix ) . 2 HOW TO EVALUATE DEEP UNCERTAINTY ESTIMATION PERFORMANCE . Let X be the input space and Y be the label space . Let P ( X , Y ) be an unknown distribution over X × Y . A model f is a prediction function f : X → Y , and its predicted label for an image x is denoted by ŷf ( x ) . The model ’ s true risk w.r.t . P is R ( f |P ) = EP ( X , Y ) [ ` ( f ( x ) , y ) ] , where ` : Y × Y → R+ is a given loss function , for example , 0/1 loss for classification . Given a labeled set Sm = { ( xi , yi ) } mi=1 ⊆ ( X × Y ) , sampled i.i.d . from P ( X , Y ) , the empirical risk of model f is r̂ ( f |Sm ) , 1m ∑m i=1 ` ( f ( xi ) , yi ) . Following Geifman et al . ( 2018 ) , for a given model f we define a confidence score function κ ( x , ŷ|f ) , where x ∈ X , and ŷ ∈ Y is the model ’ s prediction for x , as follows . The function κ should quantify confidence in the prediction of ŷ for the input x , based on signals from model f . This function should induce a partial order over instances in X , and is not required to distinguish between points with the same score . The most common and well-known κ function for a classification model f ( with softmax at its last layer ) is its softmax response values : κ ( x , ŷ|f ) , f ( x ) ŷ ( Cordella et al. , 1995 ; De Stefano et al. , 2000 ) . While this is the main κ we evaluate , we also test the popular uncertainty estimation technique of Monte-Carlo dropout ( MC-Dropout ) ( Gal & Ghahramani , 2016 ) , which is motivated by Bayesian reasoning . Although these methods use the direct output from f , κ could be a different model unrelated to f and unable to affect f ’ s predictions . Note that to enable a probabilistic interpretation , κ can only be calibrated if its values reside in [ 0 , 1 ] whereas for ranking and selective prediction any value in R can be used . A selective model f ( El-Yaniv & Wiener , 2010 ; Chow , 1957 ) uses a selection function g : X → { 0 , 1 } to serve as a binary selector for f , enabling it to abstain from giving predictions for certain inputs . g can be defined by a threshold θ on the values of a κ function such that gθ ( x|κ , f ) = 1 [ κ ( x , ŷf ( x ) |f ) > θ ] . The performance of a selective model is measured using coverage and risk , where coverage , defined as φ ( f , g ) = EP [ g ( x ) ] , is the probability mass of the non-rejected instances inX . The selective risk of the selective model ( f , g ) is defined asR ( f , g ) , EP [ ` ( f ( x ) , y ) g ( x ) ] φ ( f , g ) . These quantities can be evaluated empirically over a finite labeled set Sm , with the empirical coverage defined as φ̂ ( f , g|Sm ) = 1m ∑m i=1 g ( xi ) , and the empirical selective risk defined as r̂ ( f , g|Sm ) , 1 m ∑m i=1 ` ( f ( xi ) , yi ) g ( xi ) φ̂ ( f , g|Sm ) . Similarly , SAC is defined as the largest coverage available for a specific accuracy constraint . A way to visually inspect the behavior of a κ function for selective prediction can be done using an RC curve—a curve showing the selective risk as a function of coverage , measured on some chosen test set ; see Figure 2 for an example . The AURC and E-AURC metrics were defined by Geifman et al . ( 2018 ) for quantifying the selective quality of κ functions via a single number , with AURC being defined as the area under the RC curve . AURC , however , is very sensitive to the model ’ s accuracy , and in an attempt to mitigate this , E-AURC was suggested . The latter also suffers from sensitivity to accuracy , as we demonstrate in Appendix C. Let us consider the two models in Figure 2 for risk-sensitive deployment ; EfficientNetV2-XL ( Tan & Le , 2021 ) and ViT-B/32-SAM ( Chen et al. , 2021a ) . While the former model has better overall accuracy and AURC ( metrics that could lead us to believe the model is best for our needs ) , it can not guarantee a Top-1 ImageNet selective accuracy above 95 % for any coverage . ViTB/32-SAM , on the other hand , can provide accuracies above 95 % for all coverages below 50 % . When there are requirements for specific coverages , the most direct metric to utilize would be the matching selective risks , by which we can select the model offering the best performance for our task . If instead a specific range of coverages is specified , we could measure the area under the RC curve for those coverages : AURCC ( κ , f |Sm ) = 1|C| ∑ c∈C r̂ ( f , gc|Sm ) , with C being those required coverages . Lastly , if a certain accuracy constraint is specified , the chosen model should be the one providing the largest coverage for that constraint ( the largest coverage for a certain SAC ) . Often , these requirements are not known or can change as a result of changing circumstances or individual needs . Also , using metrics sensitive to accuracy such as AURC makes designing architectures and methods to improve κ very hard , since an improvement in these metrics could be attributed to either an increase in overall accuracy ( if such occurred ) or to a real improvement in the model ’ s “ metacognition ” . Lastly , some tasks might not allow the model to abstain from making predictions at all , but instead require interpretable and well-calibrated probabilities of correctness , which could be measured using ECE .

Summary: The paper provides an empirical comparison of uncertainty estimates obtained from 484 deep neural networks (DNN), trained for image classification tasks on the ImageNet dataset. They compared uncertainty estimation performance of different architectures and training strategies (knowledge distillation) on many quantitative metrics such as AUC-ROC on classification tasks, Area under the risk coverage curve (AURC), etc. The authors finally summarize their findings on what architectures are best at providing uncertainty estimations and how to compare different methods on OoD detection and uncertainty in in-distribution samples.