Towards More Realistic Neural Network Uncertainties

1 Introduction . Having attracted great attention in both academia and digital economy , deep neural networks ( DNNs ) ( Goodfellow et al. , 2016 ) are about to become vital components of safety-critical systems . Applications like early diagnoses of severe diseases and safe automated transport seem to be within reach ( Liu et al. , 2014 ; Bojarski et al. , 2016 ) . To actually deliver on these promises , the considerable potential of such safety-critical systems to harm humans and to cause severe damages has to be minimized . This fact comes with new challenges for the development of DNNs : next to the performance itself , further requirements such as low latency and high robustness gain in importance . Furthermore , safety-critical systems do not tolerate failures and hence have to be monitored and assessed at runtime to ensure safe functioning . One such mean of understanding the state of a software system is measuring the statistical uncertainty of the system module given the current input . Quantifying such uncertainties helps to make decisions especially in situations of partial availability of the relevant information . In particular in modular systems , subsequent modules should profit from the addition of knowledge about the uncertainty within the processing of the current input . Amongst others , Monte Carlo ( MC ) dropout and variational inference are promising approaches to estimate the prediction uncertainty of DNNs ( see section 2 for more details ) . These approaches try to estimate more realistically the statistical uncertainties of DNNs that go beyond computing dispersion metrics on the DNN ’ s softmax output which is known to be rather easily fooled by adversarial perturbations ( Nguyen et al. , 2015 ) . For uncertainty estimates to be used in safety-critical systems , we require them to be realistic ( Horwood et al. , 2014 ) , i.e . we require these estimates to resemble the residuals ( the fitting errors ) of the neural network outcomes ( Figure 1 ) . This poses a conceptual challenge as standard optimization schemes do not allow for direct training of realistic uncertainties . Therefore , high predictive performance and uncertainty realism might be largely unrelated to one another . It is desirable to achieve these two objectives at the same time . For regression , we put forward an evaluation scheme to test for uncertainty realism . For classification , we argue that existing assessment methods already ( partly ) satisfy these realism demands . Instead we propose a novel approach to variational inference that provides more realistic uncertainty estimates compared to existing approaches . In detail , our contribution is as follows : • We propose a statistical test that allows to evaluate the realism for uncertainty mechanisms in regression tasks . These test outcomes are empirically analyzed for a 4D regression task in the vision domain ( object detection with SqueezeDet and MC dropout ) . Further analyses call for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors . • We introduce a probabilistic U-Net-like FRRN semantic segmentation network and systematically assess the realism of the two uncertainty mechanisms it naturally provides . We find probabilistic FRRN to significantly improve uncertainty realism compared to ( plain ) FRRN . 2 Related work . In literature , it is common to distinguish uncertainties by their source : data-inherent uncertainty is called aleatoric uncertainty ( e.g . rolling a dice ) and uncertainty due to an limited amount of training data is called epistemic uncertainty ( Der Kiureghian & Ditlevsen , 2009 ; Senge et al. , 2014 ; Kendall & Gal , 2017 ; Depeweg et al. , 2017 ) . This paper focuses on epistemic uncertainty . An approach that takes epistemic uncertainty into account are Bayesian neural networks ( BNNs ) ( MacKay , 1992 ) which represent the weights by random variables . Learning a BNN requires to calculate conditional distributions of the weights given the data . Already in early applications , approximate solutions are used ( MacKay , 1992 ; Neal , 1993 ) . Many recent applications of BNNs to deep architectures are based on variational approximations to Bayesian inference ( Attias , 2000 ) , e.g . MC dropout ( Gal & Ghahramani , 2016 ) , early stopping ( Duvenaud et al. , 2016 ) and weight decay ( Blundell et al. , 2015 ) . For our experiments , we apply MC dropout to a regression problem in the detection setting . MC dropout was already used for different regression tasks ( Kendall & Gal , 2017 ) and also in the detection setting ( Miller et al. , 2018 ) . Bayesian frameworks are also used in other settings than BNNs : the Probabilistic U-Net ( Kohl et al. , 2018 ) , a model for segmentation , uses a neural Network as a feature extractor , the weights of which are not treated as random variables . To obtain a distribution over possible segmentations , it is assumed that there are hidden random variables that generate a distribution over the output space . The probabilistic U-Net was introduced to model an aleatoric uncertainty , namely the label ambiguity caused by different ground truth segmentations . The approximations mentioned above both represent the output by a sample from the approximate posterior distribution . The approximations make it necessary to assess the quality of the uncertainty measures . One way to assess the quality of an uncertainty estimate is critically studying the algorithm itself . For example Osband ( 2016 ) points out that MC dropout ( Gal & Ghahramani , 2016 ) — unlike a corresponding BNN—does not converge to concentrated distributions in the infinite data limit ( see also Gal et al . ( 2017 ) ) and Bishop ( 2006 ) shows that variational approaches tend to underestimate the variance . An alternative way is to treat the uncertainty mechanism as a black box and assess how well the estimated uncertainties fit to the reality . Lacking a ground truth label for the uncertainty , this comparison has to be indirect ( Lakshminarayanan et al. , 2017 ) . The assessment approaches differ between classification and regression tasks . Classification For the classification task , it is typical to determine an uncertainty score ( either directly on the network output ( Gal & Ghahramani , 2016 ; Hendrycks & Gimpel , 2017 ) or by an additional model ( Oliveira et al. , 2016 ) ) and to raise a flag if this score surpasses a threshold . Given an approximation to a posterior distribution ( or a sample from it ) it is possible to derive scores based on the expected softmax and higher moments . Typical such scores on the expected softmax are the entropy ( Mukhoti & Gal , 2018 ) or the maximum ( Hendrycks & Gimpel , 2017 ) . Scores based on the second moment are the variance ( of the winning class in sampling or of the expected softmax ) and mutual information ( MI ) ( Mukhoti & Gal , 2018 ) . To make evaluation of the above scores independent of a threshold value , areas under the receiver operating characteristic ( AUROC ) , the precision-recall curve ( AUPRC ) or the negative predictive value versus the recall curve are considered ( Hendrycks & Gimpel , 2017 ; Mukhoti & Gal , 2018 ) . Alternatively , one can assess the calibration quality of the resulting softmax ( Lakshminarayanan et al. , 2017 ; Guo et al. , 2017 ; Gal et al. , 2017 ) . Regression For regression tasks , it is suitable to represent the uncertainty of an approximation to the posterior distribution ( or of an sample from it ) by its ( empirical ) covariance matrix . A very strict way to assess this matrix is the application of a suitable statistical test , as it is in astronautics ( covariance realism , see Horwood et al . ( 2014 ) for details ) . A less strict metric is a multidimensional extension of the standardized mean-squared error ( SMSE ) ( Rasmussen , 2003 ) that was used by Fruehwirt et al . ( 2018 ) to assess a one-dimensional regression model based on MC dropout ( this work also covers other methods than MC dropout ) . More frequently , models are assessed by the average log-likelihood ( Blei et al. , 2006 ; Walker et al. , 2016 ; Gal & Ghahramani , 2016 ) . The methods described above assess the complete covariance matrix . For further assessments this matrix is reduced to a score measuring a certain aspect of the corresponding ellipsoid . Plausible scalar scores include the determinant , the maximal eigenvalue or the maximal diagonal entry of the matrix . A completely different method of uncertainty estimate assessment is investigating its behavior . For example Kendall & Gal ( 2017 ) check if the epistemic uncertainty decreases when increasing the training set or Wirges et al . ( 2019 ) study how the level of uncertainty depends on the distance of the object for some 3D environment regression task . 3 Towards more realistic neural network uncertainties . An uncertainty mechanism should be realistic . To assess its realism we need suitable metrics . The metrics should help us to decide if—or to which degree—the uncertainty reflects the actual statistical weaknesses of the model . Earlier attempts to uncertainty realism can be found in Mukhoti & Gal ( 2018 ) . For regression tasks , we derive a mathematical criterion based on Mahalanobis distances . For classification , existing assessment methods already allow to judge how well uncertainties point out the weaknesses of the model . Here , we propose a novel approach to variational inference that provides more realistic uncertainty estimates compared to existing approaches . Regression . For regression tasks , uncertainty mechanisms like MC dropout produce an output sample instead of a single output . The sample covariance matrix is a measure for the network uncertainty . In other words we fit a Gaussian to the sample of predictions and call this Gaussian the posterior predictive distribution . There are multiple ways to assess how good this posterior predictive distribution fits to test data . The most common way is to calculate the negative log-likelihood of the test data . However , test log-likelihood is a combined evaluation metric for means and variances and thus it is not ideally suited for analyzing the quality of the estimated variances . Instead , it is preferable to assess the variances separately . We propose to do this by means of a statistical test : For conceptual clarity , we consider a simple 1D regression task for the moment . A neural network with MC dropout is trained to predict yi , gt from xi . At test time , the network generates—for each input xi—a sample { yi , k } that is summarized as ( µi , σi ) . The network residual is ξi = yi , gt − µi . Here , uncertainty realism boils down to requiring : ξi ∼ N ( 0 , σi ) . As only one ξi is available for a given xi , rewriting this criterion as ξ2i σ2i = ( yi , gt − µi ) 2 σ2i ∼ χ2 ( d = 1 ) ( 1 ) allows to actually test it—independent of a specific dataset . Note that not only unrealistic uncertainties but also regression bias could lead to violations of this condition . Generalizing this statistical criterion from 1D to a higher dimensional regression task , the covariance realism can be assessed using the squared Mahalanobis distance ( Horwood et al. , 2014 ) : M2µi , Σi ( y ) = ( y− µi ) T Σ−1i ( y− µi ) , where µi and Σi are derived from the uncertainty mechanism for data point xi . If y is sampled from a Gaussian distribution with mean µ and covariance matrix Σ , M2µ , Σ ( y ) follows a chi-squared distribution χ2 ( d ) , where d is the dimension of y . Note that this statement holds for every choice of µ and Σ and hence also in our setting , where each observation yi , gt is assumed to be a realization of a Gaussian with different parameters { µi , Σi } . Hence , if the estimates µi and Σi are realistic , a strict uncertainty realism criterion is given by requiring the setMgt = { M2µi , Σi ( yi , gt ) } to follow a χ 2 ( d ) -distribution .

Summary: This paper considers the important open question of how to ensure that uncertainty estimates of neural network predictions actually reflect real world error distributions. The paper introduces an uncertainty quality metric along with a statistical test based off this metric that enables a binary decision of whether to accept a model’s uncertainties as realistic. The paper also introduces a new model for supervised image-to-image tasks that combines two existing uncertainty mechanisms, and achieves reasonable uncertainty estimates, in particular, demonstrating robustness to out-of-sample data.