Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures

Uncertainty quantification for complex deep learning models is increasingly important as these techniques see growing use in high-stakes , real-world settings . Currently , the quality of a model ’ s uncertainty is evaluated using point-prediction metrics such as negative log-likelihood or the Brier score on heldout data . In this study , we provide the first large scale evaluation of the empirical frequentist coverage properties of well known uncertainty quantification techniques on a suite of regression and classification tasks . We find that , in general , some methods do achieve desirable coverage properties on in distribution samples , but that coverage is not maintained on out-of-distribution data . Our results demonstrate the failings of current uncertainty quantification techniques as dataset shift increases and establish coverage as an important metric in developing models for real-world applications . 1 INTRODUCTION . Predictive models based on deep learning have seen dramatic improvement in recent years ( LeCun et al. , 2015 ) , which has led to widespread adoption in many areas . For critical , high-stakes domains such as medicine or self-driving cars , it is imperative that mechanisms are in place to ensure safe and reliable operation . Crucial to the notion of safe and reliable deep learning is the effective quantification and communication of predictive uncertainty to potential end-users of a system . Many approaches have recently been proposed that fall into two broad categories : ensembles and Bayesian methods . Ensembles ( Lakshminarayanan et al. , 2017 ) aggregate information from many individual models to provide a measure of uncertainty that reflects the ensembles agreement about a given data point . Bayesian methods offer direct access to predictive uncertainty through the posterior predictive distribution , which combines prior knowledge with the observed data . Although conceptually elegant , calculating exact posteriors of even simple neural models is computationally intractable ( Yao et al. , 2019 ; Neal , 1996 ) , and many approximations have been developed ( Hernández-Lobato & Adams , 2015 ; Blundell et al. , 2015 ; Graves , 2011 ; Pawlowski et al. , 2017 ; Hernández-Lobato et al. , 2016 ; Louizos & Welling , 2016 ; 2017 ) . Though approximate Bayesian methods scale to modern sized data and models , recent work has questioned the quality of the uncertainty provided by these approximations ( Yao et al. , 2019 ; Wenzel et al. , 2020 ; Ovadia et al. , 2019 ) . Previous work assessing the quality of uncertainty estimates have focused on calibration metrics and scoring rules such as the negative-loglikelihood ( NLL ) , expected calibration error ( ECE ) , and Brier score . Here we provide a complementary perspective based on the notion of empirical coverage , a well-established concept in the statistical literature ( Wasserman , 2013 ) that evaluates the quality of a predictive set or interval instead of a point prediction . Informally , coverage asks the question : If a model produces a predictive uncertainty interval , how often does that interval actually contain the observed value ? Ideally , predictions on examples for which a model is uncertain would produce larger intervals and thus be more likely to cover the observed value . More formally , given features xn ∈ Rd and a response yn ∈ R , coverage is defined in terms of a set Ĉn ( x ) and a level α ∈ [ 0 , 1 ] . The set Ĉn ( x ) is said to have coverage at the 1− α level if for all distributions P ∈ Rd × R where ( x , y ) ∼ P , the following inequality holds : P { yn ∈ Ĉn ( xn ) } ≥ 1− α ( 1 ) The set Ĉn ( x ) can be constructed using a variety of procedures . For example , in the case of simple linear regression a prediction interval for a new point xn+1 can be constructed1 using a simple , closed-form solution . Figure 1 provides a graphical depiction of coverage for two hypothetical regression models . A complementary metric to coverage is width , which is the size of the prediction interval or set . Width can provide a relative ranking of different methods , i.e . given two methods with the same level of coverage we should prefer the method that provides intervals with smaller widths . Contributions : In this study we investigate the empirical coverage properties of prediction intervals constructed from a catalog of popular uncertainty quantification techniques such as ensembling , Monte-Carlo dropout , Gaussian processes , and stochastic variational inference . We assess the coverage properties of these methods on nine regression tasks and two classification tasks with and without dataset shift . These tasks help us make the following contributions : • We introduce coverage and width as a natural and interpretable metrics for evaluating predictive uncertainty . • A comprehensive set of coverage evaluations on a suite of popular uncertainty quantification techniques . • An examination of how dataset shift affects these coverage properties . 2 BACKGROUND AND RELATED WORK . Obtaining Predictive Uncertainty Estimates Several lines of work focus on improving approximations of the posterior of a Bayesian neural network ( Graves , 2011 ; Hernández-Lobato & Adams , 2015 ; Blundell et al. , 2015 ; Hernández-Lobato et al. , 2016 ; Louizos & Welling , 2016 ; Pawlowski et al. , 2017 ; Louizos & Welling , 2017 ) . Yao et al . 1A well-known result from the statistics literature ( c.f . chapter 13 of Wasserman ( 2013 ) ) is that the interval is given by ŷn+1± tn−2sy √ 1/n+ ( xn+1 − x̄ ) 2/ ( ( n− 1 ) s2x ) , where ŷn+1 is the predicted value , tn−2 is the 1 − α/2 critical value from a t-distribution with n − 2 degrees of freedom , x̄ is the mean of x in the training data , and sy , sx are the standard deviations for y and x respectively . such that ( 1 ) holds asymptotically . However , for more complicated models such as deep learning , closed form solutions with coverage guarantees are unavailable , and constructing these intervals via the bootstrap ( Efron , 1982 ) ) can be computationally infeasible or fail to provide the correct coverage ( Chatterjee & Lahiri , 2011 ) . ( 2019 ) provide a comparison of many of these methods and highlight issues with common metrics of comparison , such as test-set log likelihood and RMSE . Good scores on these metrics often indicates that the model posterior happens to match the test data rather than the true posterior ( Yao et al. , 2019 ) . Maddox et al . ( 2019 ) developed a technique to sample the approximate posterior from the first moment of SGD iterates . Wenzel et al . ( 2020 ) demonstrated that despite advances in these approximations , there are still outstanding challenges with Bayesian modeling for deep networks . Alternative methods that do not rely on estimating a posterior over the weights of a model can also be used to provide uncertainty estimates . Gal & Ghahramani ( 2016 ) , for instance , demonstrated that Monte Carlo dropout is related to a variational approximation to the Bayesian posterior implied by the dropout procedure . Lakshminarayanan et al . ( 2017 ) used ensembling of several neural networks to obtain uncertainty estimates . Guo et al . ( 2017 ) established that temperature scaling provides well calibrated predictions on an i.i.d test set . More recently , van Amersfoort et al . ( 2020 ) showed that the distance from the centroids in a RBF neural network yields high quality uncertainty estimates . Liu et al . ( 2020 ) also leveraged the notion of distance ( in this case , the distance from test to train examples ) to obtain uncertainty estimates with their Spectral-normalized Neural Gaussian Processes . Assessments of Uncertainty Properties under Dataset Shift Ovadia et al . ( 2019 ) analyzed the effect of dataset shift on the accuracy and calibration of Bayesian deep learning methods . Their large scale empirical study assessed these methods on standard datasets such as MNIST , CIFAR-10 , ImageNet , and other non-image based datasets . Additionally , they used translations , rotations , and corruptions ( Hendrycks & Gimpel , 2017 ) of these datasets to quantify performance under dataset shift . They found stochastic variational inference ( SVI ) to be promising on simpler datasets such as MNIST and CIFAR-10 , but more difficult to train on larger datasets . Deep ensembles had the most robust response to dataset shift . Theoretical Coverage Guarantees The Bernstein-von Mises theorem connects Bayesian credible sets and frequentist confidence intervals . Under certain conditions , Bayesian credible sets of level α are asymptotically frequentist confidence sets of level α and thus have the same coverage properties . However , when there is model misspecification , coverage properties no longer hold ( Kleijn & van der Vaart , 2012 ) . Barber et al . ( 2019 ) explored under what conditions conditional coverage guarantees can hold for arbitrary models ( i.e . guarantees for P { yn ∈ Ĉn ( x|x = xn ) } , which are per sample guarantees ) . They show that even when these coverage properties are not desired to hold for any possible distribution , there are provably no methods that can give such guarantees . By extension , no Bayesian deep learning methods can provide conditional coverage guarantees . 3 METHODS . In both the regression and classification settings , we analyzed the coverage properties of prediction intervals and sets of five different approximate Bayesian and non-Bayesian approaches for uncertainty quantification . These include Dropout ( Gal & Ghahramani , 2016 ; Srivastava et al. , 2015 ) , ensembles ( Lakshminarayanan et al. , 2017 ) , Stochastic Variational Inference ( Blundell et al. , 2015 ; Graves , 2011 ; Louizos & Welling , 2016 ; 2017 ; Wen et al. , 2018 ) , and last layer approximations of SVI and Dropout ( Riquelme et al. , 2019 ) . Additionally , we considered prediction intervals from linear regression and the 95 % credible interval of a Gaussian process with the squared exponential kernel as baselines in regression tasks . For classification , we also considered temperature scaling ( Guo et al. , 2017 ) and the softmax output of vanilla deep networks ( Hendrycks & Gimpel , 2017 ) . 3.1 REGRESSION METHODS AND METRICS . We evaluated the coverage properties of these methods on nine large real world regression datasets used as a benchmark in Hernández-Lobato & Adams ( 2015 ) and later Gal and Ghahramani ( Gal & Ghahramani , 2016 ) . We used the training , validation , and testing splits publicly available from Gal and Ghahramani and performed nested cross validation to find hyperparameters and evaluated coverage properties , defined as the fraction of prediction intervals which contained the true value in the test set . On the training sets , we did 100 trials of a random search over hyperparameter space of a multi-layer-perceptron architecture with an Adam optimizer ( Kingma & Ba , 2015 ) and selected hyperparameters based on RMSE on the validation set . Each approach required slightly different ways to obtain a 95 % prediction interval . For an ensemble of neural networks , we trained N = 40 vanilla networks and used the 2.5 % and 97.5 % quantiles as the boundaries of the prediction interval . For dropout and last layer dropout , we made 200 predictions per sample and similarly discarded the top and bottom 2.5 % quantiles . For SVI , last layer SVI ( LL SVI ) , and Gaussian processes we had approximate variances available for the posterior which we used to calculate the prediction interval . We calculated 95 % prediction intervals from linear regression using the closed-form solution . Then we calculated two metrics : • Coverage : A sample is considered covered if the true label is contained in this 95 % predic- tion interval . We average over all samples in a test set to estimate the coverage of a method on this dataset . • Width : The width is the average over the test set of the ranges of the 95 % prediction intervals . Coverage measures how often the true label is in the prediction region while width measures how specific that prediction region is . Ideally , we would have high levels of coverage with low levels of width on in-distribution data . As data becomes increasingly out of distribution , we would like coverage to remain high while width increases to indicate model uncertainty .

Summary: Paper provides an  evaluation of the reliability of confidence levels of well known uncertainty quantification techniques in deep learning on classification and regression tasks. The question that the authors are trying to answer empirically is: when a model claims accuracy at a confidence level within a certain interval , how often does the actual accuracy   fall within  that interval?  This is conceptually similar to the recent slew of papers seeking to empirically evaluate the softmax calibration of deep models where the question there is how often do predicted probabilities of the winning class reflect the true probability of the correct answer, but in this paper the focus is on confidence level and confidence intervals.