Towards More Realistic Neural Network Uncertainties
Authors: Joachim Sicking
Alexander Kister
Matthias Fahrland
Stefan Eickeler
Fabian Hueger
Stefan Rueping
Peter Schlicht
Tim Wirtz
Venue:  (2020)
Keywords: uncertainty
variational inference
MC dropout
variational autoencoder
evaluation

Abstract: Statistical models are inherently uncertain. Quantifying or at least upper-bounding their uncertainties is vital for safety-critical systems. While standard neural networks do not report this information, several approaches exist to integrate uncertainty estimates into them. Assessing the quality of these uncertainty estimates is not straightforward, as no direct ground truth labels are available. Instead, implicit statistical assessments are required. For regression, we propose to evaluate uncertainty realism---a strict quality criterion---with a Mahalanobis distance-based statistical test. An empirical evaluation reveals the need for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors. Alongside, we transfer the variational U-Net classification architecture to standard supervised image-to-image tasks. It provides two uncertainty mechanisms and significantly improves uncertainty realism compared to a plain encoder-decoder model.

Under review as a conference paper at ICLR 2020
TOWARDS MORE REALISTIC NEURAL NETWORK UNCERTAINTIES
Anonymous authors Paper under double-blind review
A ABSTRACT
Statistical models are inherently uncertain. Quantifying or at least upper- bounding their uncortainties is vital for safety-critical systems. While standard neural networks do not report this information, several approaches exist to integrale uncertainly estimates into them. Assessig the quality of these uncertainly estimates not straightforward, labels are available. Instead, implicit statistical assessments are required. For regression, we propose evaluate uncertainty realism a strict quality criterion with a Mahalanobis distance-based statistical test. An empirical evaluation reveals the need for uncertainty mcasures that are appropriate to upper-bound hcavy-tailed empirical errors. Alongside, we transfer the variational U-Net classification architecture to standard supervised image- to-image tasks. It provides two uncertainly mechanisms and significantly improves uncertainty realism compared to a plain encoder-decoder model.
direct ground truth
INTRODUCTION
Having attracted great attention in both academia and digital economy, deep neural networks (DNNs) (Goodfellow et al.) 2016) are about to become vital components of safety-critical systems. Applications like carly diagnoses of severe discases and safe automated transport seem to be within reach (Liu et al., 2014 Bojarski et al.) 2016). To actually deliver on these promises, the considerable potential of such salety-critica systems to harm humans and to cause severe damages has to be minimized. This fact comes with new challenges for the development of DÑNs: next to the performance itself, further requirements such as low latency and high robustness gain in importance. Furthermore, safety-critical systems do not tolerate failures and hence have to be monitored and assessed at runtime to ensure safe functioning.
One such mean of understanding the state of software system is measuring the statistical uncertainty of the system module given the current input. Quantifying such uncertainties helps to make decisions especially in situnti of partial availability of the relevant informa- tion. In particular in modular systems, subsequent modules should profit from the addition of knowledge about the uncertainty within the processing of the current input.
Amongst others, Monte Carlo (MC) dropout and variational inference are promising ap- proaches to estimate the prediction uncertainty of DNNs (see scction 2 for more details). These approaches try to estimate more realistically the statistical uncertainties of DNNs that go beyond computing dispersion metrics on the DNN's softmax output which is known to be rather easily fooled by adversarial perturbations (Nguyen et al. 2015
For uncertainty estimates to be used in safety-critical systems, we require them to be realistic (Horwood et al. 2014), ie. require these estimates to resemble the residuals (the fitting errors) the neural network outcomes (Figure 1). This poses a conceptual challenge standard optimization schemes do not allow for direct training of realistic uncertaintics. Therefore, high prodictive performance and uncertainty realism might be largely unrelated to one another. It is desirable to achieve these two objectives at the same time.
For regression, we put forward an evaluation scheme to test for uncertainty realism. For classification, we argue that existing assessment. methods already (partly) satisfy these

---

Under review as a conference paper at ICLR 2020
Figure 1: Symbolic image illustrating uncertainty realism. The estimated uncertainties (contour lines) are too broad (left), realistic (middle), too narrow (right) compared to the deviations of model outputs (black crosses) from ground truth (red circle).
realism demands. Instead we propose a novel approach to variational inference that provides more realistic uncertainty estimates compared to existing approaches.
In detail, our contribution is as follows:
We propose statistical test that allows to evaluate the realism for uncertainty mechanisms in regression tasks. These test outcomes are empirically analyzed for a 4D regression lask in the vision domain (object detection with SqueezeDel and MC dropout). Further analyses call for uncertainty measures that are appropriate to upper-bound heavy-tailed empirical errors.
We introduce probabilistic U-Net-like FRRN semantic segmentation network and systematically assess the realism of the two uncertainty mechanisms it naturally provides. We find probabilistic FRRN significantly improve uncertainty realism compared to (plain) FRRN.
2 RELATED WORK
In literature, it is common to distinguish uncertainties by their source: data-inherent uncertainty is called aleatoric uncertainty (e.g. rolling dice) and uncertainty due an limited amount of training data is called epislemic uncerlainly (Der Kiureghian & Ditlevsen, 2009 Senge et al. 2014 Kendall & Gal 2017 Depeweg el al. 2017). This paper locuses on
epistemic uncertainty.
An approach that takes epistemic uncertainty into account are Bayesian neural networks (BNNs) (MacKay) 1992) which represent the weights by random variables. Learning BNN requires to calculate conditional distributions of the weights given the data. Already in carly applications, approximate solutions used (MacKay) 1992 Neal 1993). Many
recent applications of BNNs to deep architectures are based on variational approximation to Bayesian inference (Attias 2000), e.g. MC dropout (Gal & Ghahramani 2016). early stopping (Duvenaud et al.) 2016 and weight decay (Blundell et al. 2015). For our experiments, we apply MC dropout
a regression problem in the detection setting. MC dropout was already used for different regression tasks (Kendall & Gal 2017) and also in the detection
sctting Miller et al. 2018).
the Probabilistic U-Net [Kohl al. 22018), model for segmentation, uses a neural Network as feature extractor, the weights of which are not treated as random variables. To obtain distribution over possible segmentations,
Bayesian frameworks are also used in other settings than BNNs:
assumed that there are hidden random variables that generate the output space. The probabilistic U-Net was introduced to model an aleatoric uncertainty, namely the label ambiguity caused by different ground truth segmentations. The approximations mentioned above both represent the output by a sample from the approximate posterior distribution. The approximations make it necessary to assess the quality of the uncertainty mcasures.
distribution
One way to assess the quality of an uncertainty estimate is critically studying the algorithm itself. For example Osband (0166) points out that MC dropout (Gal & Ghahramani 2016
2

---

Under review as a conference paper at ICLR 2020
unlike corresponding BNN does not converge to concentrated distributions in the infinite data limit (see also Gal et al. (2017)) and Bishop (2066) shows that variational approaches tend to underestimate the variance. An alternative way is to treat the uncertainty mechanism as a black box and assess how well the estimated uncertainties fit. to the reality. Lacking a ground truth label for the uncertainty, this comparison has to be indirect (Lakshminarayanan et al. 2017). The assessment approaches differ between classification and regression tasks.
Classification For the classification task, it is typical to determine an uncertainty score (either directly on the network output (Gal & Ghahramani 2016 Hendrycks & Gimpel 2017) or by an additional model (Oliveira et al.) 2016]] and to raise a flag this score surpasses a threshold. Given an approximation to a posterior distribution (or sample from it) is possible derive scores based on the expected softmax and higher moments. Typical such scores on the expected softmax are the entropy (Mukhoti & Gal 2018) or the maximum (Hendrycks & Gimpel 2017). Scores based on the second moment, are the variance (of the winning class in sampling or of the expected softmax) and mutual information (MI) Mukhoti & Gal 2018).
To make evaluation of the above scores independent of threshold value, areas under the receiver operating characteristic (AUROC), the precision-recall curve (AUPRC) or the negative predictive value versus the recall curve are considered (Hendrycks & Gimpel 2017 Mukhoti & Gal 2018). Alternatively, one can assess the calibration quality of the resulting softmax [Lakshminarayanan et al. 2017 Guo et al. 2017 Gal et al. 2017
Regression For regression tasks, it suitable to represent the uncortainty of an approxi- mation to the posterior distribution (or of an sample from it) by its (empirical) covariance matrix. A very strict. way to assess this matrix is the application of a suitable statistical test, as it is in astronautics (covariance realism, see Horwood et al. (2014) for details). less strict metric is a multidimensional extension of the standardized mean-squared error (SMSE) (Ras) mussen 2003) that was used by Fruehwirt al. (2018) to assess a one-dimensional regression model based on MC dropout (this work also covers other methods than MC dropout). More frequently, models are assossed by the average log-likelihood (Blei et al. 2006 Walker et al.
2016 Gal & Ghahramani 2016).
The methods described above assess the complete covariance matrix. For further assessments this matrix is reduced to a score measuring certain aspect of the corresponding ellipsoid. Plausible scalar scores include the determinant, the maximal eigenvalue or the maximal diagonal entry of the matrix. A completely different method of uncertainty estimate assessment is investigating its behavior. For example Kendall & Gal (2017) check if the
epistemic uncortainty decreases when increasing the training set or Wrrgges al. (2019) study how the level of uncertainty depends on the distance of the object for some 3D environment regression task.
3 TOWARDS MORE REALISTIC NEURAL NETWORK UNCERTAINTIES
An uncertainty mechanism should be realistic. To assess its realism we need suitable metrics. The metrics should help us to decide if or to which degree- the uncertainty reflects the actual statistical weaknesses of the model. Earlier attempts to uncertainty realism can be found in Mukhoti & Gal (2018). For regression tasks, we derive a mathematical critcrion based on Mahalanobis distances. For classification, existing assessment methods alrcady allow to judge how well uncertainties point out the weaknesses of the model. Here, we propose novel approach to variational inference that provides more realistic uncertainly estimates compared to existing approaches.
Regression
For regression tasks, uncertainty mechanisms like MC dropout produce an output sample instead of a single output. The sample covariance matrix is measure for the network uncertainty. In other words we a Gaussian to the sample of predictions and call this Gaussian the posterior predictive distribution. There are multiple ways to assess how good this posterior predictive distribution fits to test data. The most common way is to calculate the negative log-likelihood of the test data. However, test log-likelihood is combined
3

---

Under review as a conference paper at ICLR 2020
Encoder
latent space
Sample 2,22m
FRRN Decoder
1x1 dropout
Encoder
Figure Architecture of the proposed variational FRRN (at inference). The two uncertainty mechanisms latent space sampling and MC dropord are highlighted in red.
evaluation metric for means and variances and thus it is not ideally suited for analyzing the quality of the estimated variances. Instead, preferable to assess the variances separately. We propose to do this by means of statistical test:
For conceptual clarity, we consider a simple 1D regression task for the moment. A neural network with MC dropout is trained to predict Yigtt from zς. At test time, the network generates for each input F; a sample (v.,} that is summarized as (1,, The network residual is & Vi.tt - Hq. Here, uncertainty realism boils down to requiring: & o,) As only one & is available for a given 2,, rewriting this criterion as
e (Uiar n:)² (2==1)
(1)
of
allows actually test independent of specific dataset. Note that not only unrealistic uncertainties but also regression bias could lead to violations of this condition.
Generalizing this statistical criterion from 1D to higher dimensional regression task, the covariance realism can be assessed using the squared Mahalanobis distance (Horwood et al. 2014):
())))) ML. (y (y-ru 14) E p4)
where μ4; and E, derived from the uncertainty mechanism for data point x, If y is sampled from a Gaussian distribution with mean μ and covariance matrix E, M2 M2,x(U) (v)) follows a chi-squared distribution (d), where d is the dimension of y. Note that this statement holds for every choice of μ and Σ and hence also our setting, where each observation Us,gt is assumed to be a realization of Gaussian with different, parameters {p:: Σ.}. Hence, if the estimates μ1, and Σ; are rcalistic, strict uncertainty realism critcrion given by requiring the set Mgt = {M², E, (Frgg)) to follow ("(d)-distribution.
Classification
For classification tasks, not only the study of softmax variances but also the study of the mean softmax is interesting because it provides (inter-class) uncertainty information. This in contrast to regression tasks. To treat these dilferent uncertainly sources in a uniform way, we derive scalar scores from them, e.g. entropy or variance.
The practical use of such uncertainty score is to identify inputs for which the model is wrong. The better we are in making this decision, the more realistic is the uncertainty score. This realism is quantified by AUC values.
standard uncertainty mechanism for classification tasks is MC dropout (Gal & Ghahramani 2016) an approximation of BNN. However, it makes use of an uninformed prior over the weights. We propose an alternative mechanism by studying variational inference- based uncertainty inspired by the probabilistic U-Net. (Kohl et al. 2018). In contrast, our
4

---

Under review as a conference paper at ICLR 2020
approach of using the variational architecture (Figure) 2 is to focus less creating different interpretations of an input image, but to treat the latent space as a more sophisticated and focused mechanism to encode data-inherent uncertainty. Furthermore, instead of using U-Net as the base structure for the segmentation task and the encoders, we use an FRRN- based (Pohlen et al. 2017) architecture (see appendix A for details). Additionally, to combine the latent space sampling with MC dropout by inserting dropout layer right before the last convolution of the network. To the best of our knowledge, this architecture yields the first DNN that allows for evaluation and fusion of the two mentioned unccrtainty mechanisms.
allow
4 EMPIRICAL EVALUATION
Following the outlined assessment. scheme for neural network uncertainties, we evaluate their realism for the regression lask of object detection. Moreover, we analyze the realism of the two uncertainty mechanisms of the novel variational FRRN for semantic segmentation.
4.1 REGRESSION OBJECT DETECTION
We consider SqueezeDet (Wu et al. 2017), a lightweight single-stage detector network. that uses a pre-trained SqueezeNet as its backbonc. It is trained on KITTI (Geiger et al. 2012) and returns four-tuple (d 4) of center coordinates, width and height of a 2D bounding box for detected objects. The uncertainty mechanism we use is MC dropout (p = 0.5) before the last convolutional layer. The MC sampling of the dropout layer is done before the thresholding and non-maximum suppression stage. The output sample } for a given input o, with ground truth Vi.tt is characterized by its mean My and its covariance Eg. Following section 3 calculate the squared Mahalanobis distance M Ma, (2 Mgt- {M M. (t) denotes the set of squared Mahalanobis distances for the entire test set (æ; gt). To apply the strict uncertainty realism criterion, we test if this set is drawn from x" (d = 4). This yields a p-value close to zero and therefore we have to reject the hypothesis and find the uncertainty mechanism to be unrealistic: MC dropout does not provide realistic uncertainty estimates in this empirical setting. In contrast, the set of intra-sample squared Mahalanobis distances Msample- (M,,,,, (y,,)) is-in accordance with theory- distribbi A comparison of these distributions is shown in Figure 3 (left panel, for per-component visualizations see Figure in the appendix). The width of the MC sample distribution is one order of magnitude smaller than the actual estimation errors highlighting that variational approaches underestinate variances (Bishop) 2006). Handling this deviation in an ad-hoc fashion by scaling the variance of Msample to match the variance of Mgt is shown in Figure 3 (middle panel). The higher moments of MMg are still deviating and cause a decay that is slower than exponential, iee. a fat tail.
While Mahalanobis distances allow for a combined assessment of the realism of the size and orientations of the covariance ellipsoid, an isolated assessment of the covariance orientation can be done by analyzing the angle a = L(D My yge) enclosed by the largest eigenvector of the covariance maax and the estimation error My Vyt (Figure right panel). The high resemblance of the angle distribution with the distribution of the differential solid angle of the 3-sphere emphasizes that no covariance orientation realism is given.
Following section 3 we check if MC dropout is-at at least- good indicator for realism, i.e. the mean estimation error increases monotonically with the uncertainty score Ey. Here, we approxinate the covariance with its determinate det(Ey) and its largest component ymax as rough scalar measures of its size. Figure 4 (left and right panel) suggests that such a monotonicity is given. Due to the discussed fat tails, however, upper-bounding estimation errors by uncertainty estimates cannot be done using multiples of the standard deviation. The empirical 99% quantiles (Figure 4 green) are underestimated by the- in case of canaliity-correponddin.
(red), partly by far. Therefore, residual risks should be quantified using appropriate tail measures such quantiles or tail mean values.
5

---

Under review as a conference paper at ICLR 2020
Mg
Map
10
Moarge
Moamgle
rel. fraquancy
frequency
0.4
10:3
10"2
10
rel.
0.2
10-4
10.1
0.0
40
80
Matatencbis distance
Mahallanbbis distance
Figure 3: Assessment of uncertainty realism for the regression task. Left: empirical dis- tributions Mammll and Mgt, middle: rescaled Msample to match the variance of Mgt. right: distribution of angles a enclosed by the error direction and the covariance ellipsoid orientation.
30
50%
99% quantiv
40
30
30
20
10
10-2
10
100
10a
10
107
10-1
10P
10a
102
Figure 4: Uncertainty scores and estimation errors (left: covariance determinant, right: largest covariance element).
4.2 CLASSIFICATION SEMANTIC SEGMENTATION
The proposed variational FRRN (Figure appendix A.2 trained for x 10 steps on the Cityscapes dataset (Cordts et al. 2016 with a batch size of 4 and drop rate of 0.5. The experiments are conducted on the images of the city "Münster'. In order to compare the three uncertainty mechanisms that the variational FRRN provides, each test set image processed three times: with only MC dropout switched-on (MC), with only latent space sampling switched-on (CVAE) and with both mechanisms switched-on (CVAE + MC). Regardless of the chosen uncertainty mochanism, the number of samples is fixed to 50. As uncertainty scores consider the highest probability and the entropy of the mean softmax, the variance of the winning class and the MI (see section 2 To evaluate the realism of the mechanisms not only within the training data distribution but also out-of-distribution images are vertically flipped and again processed three times as described above. The uncertainty realism is measured by calculating the respective areas under the ROC (AUROC) and the precision-recall curve (AUPRC) where we take the correctly classified pixels as positive. As a bascline we use the plain softmax output without any uncortainty mochanism being switched-on.
Table showss that for in-data samples the plain softmax probability reasonably accurate uncertainty estimation and camnot. be outperformed by the advanced mechanisms. This may be related to the high accuracy of the network because good predictions require comparably less challenging uncertainty estimations. An example for in-data prediction and the corresponding estimated uncertainty can seen Figure 7 in the appendix.
For out-of-data images, however, the sampling mechanisms allow a significant improvement. Sampling with the variational FRRN yields the most realistic uncertainties with an AUROC value of 0.72 when using the entropy score and an AUPRC of 0.755 when using the mean softmax probability. The influence of the different scores on the assessment generally small. Fusing latent space sampling with MC dropout does not lead to further improvement.

---

Under review as a conference paper at ICLR 2020
Ground Truth
Segmentation using CVAE
Degmentation using MC
10
1.0
10
66
00
62-
02
Uncertainty using CVAF
Uncerfainty using MC
Figure 5: Segmentations and uncortainty maps producod by variational FRRN. Segmentations: ground truth (top left), FRRN with CVAE sampling (top middle). FRRN with MC dropout (top right). Uncertainly maps: FFRN + CVAE sampling (bottom left), FRRN + MC dropout (bottom right). Uncertainties of correctly classified pixels are shown in green, of misclassified pixels in red.
Table 1: Realism assessment for pixel-wise classification (in-sample and out-of-sample). Compared mechanisms are the softmax baseline (none), MC sampling (MC), latent space sampling (CVAE) and combination of the latter (CVAE MC).
mechanism
in-sample
out-of-sample
AUROC
AUPRC
AUROC
AUPRC
none
max. comp. softmax
0.945
0.997
0.642
0.643
entropy
0.914
0.997
0.639
0.640
mean-buaed scores [(by)
MC
max. comp. softmax
0.945
0.997
0.651
0.653
MC
entropy
0.943
0.997
0.657
0.655
MC
MI
0.943
0.997
0.654
0.655
CVAB
max. comp. softmax
0.718
0.755
CVAB
eutropy
0.944
0.977
0.720
0.751
CVAB
0,926
0.995
0.717
0.747
CVAE + MC
comp. softmax
0.916
0.997
0.713
0.742
CVAB + MC
entropy MI
0.944
0.997
0.714
0.742
CVAR MC
0.919
0.996
0.706
0.735
covariance banrrience-used scores [Ey)
variance max. comp. softmax
0.655
CVAB
variance of max. comp. softmax
0.936
0.997
0.718
0.753
CVAE + MC
variance of max, comp. softmax
0.914
0.995
0.702
0.733
Figure 5 depicts the predictions and uncertainty estimations of the network given an out- of-data input image. As expected, the network fails correctly classify the majority of the pixels. However, both applied mechanisms allow to detect reasonable amount of the occurring misclassifications and assign a low uncertainty regions with many true positives.
Figure shows the ROC and precision-recall curves for the different mechanisms and the entropy score. It can be seen that the general shape is similar for all mechanisms. Further results for using a ROC curve-based thresholding to reject predictions with high chance of being wrong qualitatively (Figure (5) and empirically (Figure Table Ih we see that MC dropout and the latent space sampling seem to express the same kind of uncertainty. Both mechanisms generate better probability ostimations for out-of-data samples compared to plain softmax.
be found in the appendix (Figure 8 Comparing the mechanisms
At first glance, the similarity between the two mechanisms is surprising because they function in different ways. However, both are based on the principles of variational inference, thus approximate the posterior by sampling from the model parameters. Fusing both
7

---

Under review as a conference paper at ICLR 2020
1.0
1.0
1.0
Thee Positive Rate
0.8
Thue Positves Rote
0.8
0.8
0.8
0.6-
Precision
0.6
Procision
0.6
0.4
0.4
0.4
0.4
0.2
08.00
0.25 0.50 0.75
1.00
8.00
0.25
0.75 Positive
0.50
1.00
8..0
0.25
0.50 Recall
0.75
1.00
80.0
0.25
0.50
0.75
1.00
False Positive
Softmox Samplc!
dropout
MC
Figure 6: In-sample and out-of-sample comparison different uncertainty mechanisms. a) and b) show in-sample and out-of-sample ROC curves, c) and d) the equivalent precision-recall curves. Class entropy is used as uncertainty score.
mechanisms leads to different sample of parameters, but from the same distribution, which may be why we see no further improvements in the uncertainty estimate.
5 DISCUSSION AND FUTURE WORK
We argue to evaluate uncertainty realism in regression tasks with a Mahalanobis distance- based statistical test. We improve uncertainty realism in classification task by using the proposed variational FRRN instead of a (plain) FRRN.
Regression For the considered regression task in the vision domain, MC dropout does not provide realistic uncertainty estimates as it does not the orientation, width or non- normality of the estimation error distribution. However, it is important to highlight that our experimental setting adds further approximations to the approximations underlying MC dropout (small network, dropout applied only to one layer). This setup is commonly used though and we emphasize to carefully validate MC dropout results for each safety-critical application. While not being realistic, we find it to be a good indicator for realism. The observed nor-normality of the error distribution indicates that naive uncertainty upper- bounds on regression errors might fall short. Instead, we recommend the usage of appropriate uncertainty measures such as quantiles or tail mean values.
Classification When comparing the MC dropout with the latent space sampling technique, it scems that both approximate the same form of uncertainty. While small scale differences appear in examplary images, the overall estimation focuses on the same aspects and empirical evaluations show similar tendencies. Both mechanisms allow to reason about epistemic uncertainties. We state that the similarities originate from the mutual base idea of estimating the models uncertainty by approximating the posterior distribution. Compared to MC dropout, the latent space sampling seems to be the more informed uncertainty mechanism.
Future work Uncertainty realism might be improved by using multiple dropouts in the MC setting, adjusting loss functions and other training hyper paramcters or using thresholds per class for the score. Studying other out-of-data dircctions (c.g. lighting conditions, fog and unknown classes) and tasks from other domains could strengthen our results. A shortcoming of probabilistic U-Net. is that the latent. space sample only used an additional channel which the network can choose to ignore. PHiSeg (Baumgartner et al. 2019) is an extension of the probabilistic U-Net that addresses this. Transferring the PHiSeg approach to our setting and studying the resulting uncertainties seemss promising, Important further work to prove residual risk bounds and formalize requirements for uncertainty estimates of neural networks. Relating uncertainty estimates to adversarial robustness needs further research.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Hagai Attias. A variational bayesian framework for graphical models. In Advances in Neural Information Processing Systems, pp. 209-210100 109--15,- 2000.
Christian F Baumgartner, Kerem C Tezcan, Krishna Chaitanya, Andreas M Hötker, Urs J Muehlematler, Khoschy Schawkal, Anton Becker, Olivio Donati, and Ender Konukoglu. Phiseg: Capturing uncertainly in medical image segmentation. arXic preprial orXiv:1906 04045, 2019.
Christopher M Bishop. Pallern Recognition and Machine Learning. Springer, 2006.
David M Blei, Michael Jordan, et al. Variational inference for dirichlet process mixtures. Bayesion Analysis, 1 111):221-143, 2006.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint Xrx: 1amx1555.05444 2015.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars. Xiv preprint arv: 1101110047.990 2016.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding, In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Stefan Depeweg, José Miguel Srrnándde-Lobbatte Finale Doshi- Velez, and Steffen Udluft. Decomposition of uncertainty in bayesian deep learning for efficient and risk-scnsitive lcarning. arXiv preprint orXio:1710.. 07288, 2017.
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does matter? Structural Safety, 31 12):1005-1,, 2009.
David Duvenaud, Dougal Maclaurin, and Ryan Adams. Early stopping as nonparametric variational inference. In Artificial Intelligence and Stotislics, pp. 1070-1077, 2016.
Wolfgang Fruchwirt, Adam D Cobb, Martin Mairhofer, Leonard Weydemann, Heinrich Garn, Reinhold Schmidt, Thomas Benke, Peter Dal-Biance Gerhard Ransmayr, Markus Waser, et al. Bayesian deep neural nelworks for low-cost neuroplyssilloical markers of alazhemmer's disease severity. Xiv preprinl ar arXiv:1812.00 0494.99924494 2018.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. International Conference on Machine Leurning, pp- 1050-1059, 2016.
Yarin Gal, Jiri Hron, and Alex Kendall. Concrete Dropout. In Advances in Neural Information Processing Systems 30 (NIPS), 2017.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference Computer Vision and Pattern Recognition (CVPR), 2012.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. calibration modern neural networks. CoRR, hhs/1706.044999 2017. URL http::/arxivv. wwwparwwww...oraaeoo hatp///a.////..../ 14999 04599
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of- distribution examples neural networks. CoRR, abs/1610.00116, 2017.
Joshua Horwood, Jeffrey Aristoff, Navraj Singh, Aubrey B Poore, and Matthew D Hejduk. Beyond covariance realism: new metric for uncertainty realism. In Signal and Data Processing of Small Targets 2014, volume 9092, pp. 90920F. International Society for Optics and Photonics, 2014.

---

Under review as a conference paper at ICLR 2020
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems, pp. 5574-5584, 2017.
Simon Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R Ledsam, Klaus Maier-Hein, SM Ali Eslami, Danilo Jimenez Rezende, and Olaf Rouneberger. A probabilistic u-net for segmentation of ambiguous images. In Advonces in Neural Information Processing Systems, pp. 1965-6975, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainly estimation using deep ensembles. In Advances in Neurul Information 2017.
Processing Systems, pp.
Siqi Liu, Sidong Liu, Weidong Cai, Sonia Pujol, Ron Kikinis, and Dagan Feng. Early diagnosis of alzheimer's disease with deep learning. In 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI), pp. 1015 1018. IEEE, 2014.
David JC MacKay. A practical bayesian framework for backpropagation networks. Newral Computation, 4(3):448-472, 1992.
Dimity Miller, Lachlan Nicholson, Feras Dayoub, and Niko Sünderhauf. Dropout sampling for robust object detection in open-set conditions. In 2018 IEEE International Conference Robotics and Automation (ICRA), pp. 1-7. IEEE, 2018.
Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmentation. orXiv preprint srXiv:1811.12700 2018.
Radford M Neal. Bayesian learning via stochastic dynamics. In Advonces in Neural Information Processing Systems, pp. 475-482, 1993.
Auh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for uncrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427-436, 2015.
Ramon Oliveira, Pedro Tabacof, and Eduardo Valle. Known unknowns: Uncertainty quality in bayesian neural networks. arXiv preprint arXiv:1612.0123 2016.
Ian Osband. Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout. In NIPS Workshop on Bayesion Deep Learning, 2016.
Tobias Pohlen, Alexander Hermans, Markus Mathias, and Bastian Leibe. Full-resolution residual networks for semantic segmentation in street scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognation, pp. 4151-4160, 2017.
Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer School on Machine Learning, pp. 63-71. Springer, 2003.
Robin Senge, Stefan Bösner, Krzysztof Dembezyúski, Jörg Haasenritter, Oliver Hirsch,
Norbert Donner-Banzhoff, and Eyke Hüllermeier. Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty. Informalion Sciences, 255:16-25 29, 2014.
Jacob Walker, Carl Doersch, Abhinav Gupta, and Martial Hebert. An uncertain future: Forecasting from static images using variational autoencoders. In European Conference on Computer Vision, pp. 835 851. Springer, 2016.
Sascha Wirges, Marcel Reith-Braun, Martin Lauer, and Christoph Stiller. Capturing object detection uncertainty in multi-layer grid maps. orXiv preprint 1rXi::1901.1113 11284, 2019.
Bichen Wu, Forrest landola, Peter H Jin, and Kurt Keutzer. Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion Workshops, pp. 129-137 2017.
10

---

Under review as a conference paper at ICLR 2020
A
ADDITIONAL IMPLEMENTATION DETAILS
A.1 BASE FRRN
In this section we describe the FRRN base architecture that in section 4.2 Initially, the input to the FRRN gets fed into convolutional layer with 5 x 5-kernel, followed by three residual units consisting of two convolutions with skip connection. The data flow then splits into a pooling stream and a residual stream. While the data the pooling stream gets down- and up-sampled stream remains on full resolution, retaining the spatial information. The two streams get combined each scale in so called full-resolution residual units (FRRUs) by pooling the residual stream to the other streams dimensions and merging it through concatenation and two convolutions. The last upsampling and stream concatenation is followed by another three residual units and 1 x convolution to classify the pixels. Finally softmax layer is used to calculate the class probabilities.
for the experiments
extract features, the residual
We start with base number of 24 channels and double, respectively halve the number with each down- and up-sampling step. For dimension reduction or enlargement we use max pool and bilinear upscale operations. The residual stream is kept consistently 16 channels, thus after a final concatenation we end with 40 feature maps that are used for the classification. If not stated otherwise all convolutions were run with 3 x 3-kernels and are followed by ReLU activation.
A.2 VARIATIONAL FRRN
Compared the architecture of the probabilistic U-Net, we made various changes: As the base structure use full-resolution residual network (FRRN, see appendix A. and its feature extraction part and thus the downsampling operations alongside the full-reoolutinn residual units for the variational encoders (Figure 2). To use the features of both the residual and the pooling stream for the probability estimation in the encoders, the final activations of the residual stream are pooled and concatenated with the pooling stream. The resulting final feature maps are global average pooled and a 1 x 1 convolution used to predict a mcan and variance. As proposed by Kohl et al. (2018) use dimension of six for the latent space and broadcast the drawn sample to the last activation map of the FRRN, followed by three convolutions.
11

---

Under review as a conference paper at ICLR 2020
B ADDITIONAL RESULTS FOR THE VARIATIONAL FRRN
B.1 QUALITATIVE RESULTS
Figure shows the predictions and corresponding uncertainty estimates of the variational FRRN. Especially class boundaries and distant objects are assigned high uncertainty. Note that the selected input image is challenging: The fence on the left side of the imago is sce-through. Whilc it is labelled as "fence", the network sees the objects behind it. The resulting muislassificattinn ions are assigned high uncertainty.
Ground Truth
Segmentation using CVAE
Segmentation using MC
3.9
Uncertainty using CVAE
Uncertainty using MC
Figure 7: Uncertainty of correct and false classified pixels. Upper row: The flipped input image, overlayed with the Ground truth segmentation as well as the predictions based on the variational FRRN and MC dropout (50 samples each). The lower row shows the normalized, pixelwise uncertainty of correctly classified pixels (green) and misclassified pixels (red). Gray regions are not included training and evaluation.
B.2 QUANTITATIVE RESULTS
The ROC curves allow to find threshold that maximises the trade-off between the true positive and false negative rate. Finding this threshold and applying to the uncertainty estimations allows to reject pixels that have high chance of being misclassified. Figure shows the amount of false negatives (FN) and true positives (TP) that were detected as being "misclassified".
100
100
80
80
Rejections [96]
Rejections [%6]
60
60
40
40
20
Baseline
MC
varFRRN
Baseline
MC
varERRN
VarFRRN MC
VarFRRN MC
Rejected
Rejected
Figure Rejection rates of FN and TP predictions with high uncertainty. The threshold is based on the ROC curve and the entropy score. Left: Rejections in-data samples, right: on out-of-data samples.
12

---

Under review as a conference paper at ICLR 2020
C EXTENDED EVALUATION OF THE UNCERTAINTY REALISM OF SQUEEZEDET USING MC DROPOUT
10"2
Bet. ppl a.
10
(1g +4 /o)
10:24
frequency
Trequenccy
10
10
10
10
10
10
-75
-50
-25
0
25
50
75
-75
-50
-25
25
50
75
rel. ceviation
rel.. deviaaion
10:
Daau 140
Du. p2) /o.
10-2
rel. frequency
rel. frequency
10-3
20"3.
-75
deviation
Figure 9: Assessment of uncertainty realism for the regression task. Distributions of scaled MC-dropout samples and estimation errors in x-direction (top left), y-direction (top right), for width (bottom left) and height (bottom right).
13


Reviews:
[{"review_id": "HylmsHQCYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers the important open question of how to ensure that uncertainty estimates of neural network predictions actually reflect real world error distributions. The paper introduces an uncertainty quality metric along with a statistical test based off this metric that enables a binary decision of whether to accept a model’s uncertainties as realistic. The paper also introduces a new model for supervised image-to-image tasks that combines two existing uncertainty mechanisms, and achieves reasonable uncertainty estimates, in particular, demonstrating robustness to out-of-sample data.\n\nI am tending to reject, because although each of the two distinct contributions are good starts on interesting approaches, neither provides a convincing solutions for the main question, and the two contributions are quite distinct, so that the paper lacks a consistent thread.\n\nFirst, the Mahalanobis-based uncertainty evaluation makes sense, but it is not clear what it adds beyond the standard average negative log-likelihood (NLL) metric. Mahalanobis distance increases monotonically with negative log-likelihood, so is there any reason to expect it is a better way to compare models based on their uncertainty? Is there some experiment that could show that the new metric is better than NLL at evaluating uncertainty realism?\n\nThe statistical test could potentially be a nice real world tool for deciding whether to trust neural network uncertainties. However, the paper only applies the test in one case, where the MC dropout model is shown to have unrealistic uncertainties. To show that this test is useful, there should also be experiments where uncertainty estimates are shown to be realistic. Is there some alternative to MC dropout for regression or some improvements to the algorithm that could yield realistic uncertainties under this statistical test? E.g., would the U-Net-based model in the paper pass the test if it were adapted to regression problems?\n\nSecond, the new U-Net-based classification model looks like a reasonable approach, but it is disjoint from the new statistical test, and it is not clear that the method yields improvements to uncertainty realism, since there are no comparisons to external results. Since only a new architecture is used, it is not clear that the deficiencies in uncertainty realism are not architecture-specific. Is it that there are no existing architectures that can be applied to this problem? Also, is there some more realistic setting where the CVAE approach would improve in-sample scores, i.e., where out-of-sample data is not generated synthetically? Can the statistical test be adapted to the classification setting? Similar, to the case of regression, the uncertainty realism metrics used for classification are tightly coupled with the prediction accuracy; is there a way to decouple these, would one want to? The conclusion that CVAE is better than MC for this problem is solid, but is there a more general conclusion to be drawn? E.g., could a CVAE model yield realistic estimates in the regression setting?\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Bkl1amlrFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper a new method for evaluating prediction uncertainties for regression and classification tasks.\n\nI argue this work should be rejected. For regression the method is based on Gaussian assumption, which has no reason to be correct (especially for BayesianNN), and for classification the method is not clear at all.\n\nDetailed remarks:\n- For a BNN, there is no reason why p(y|x) is Gaussian, so even if the residual is rejected as chi-square this doesn't mean anything.\n- I am unclear as to how the classifier uncertainty is measured. \n- The paper claims that \"it is preferable to assess the variances separately.\" which I agree with, but a mistake in the mean would effect your chi-square test, so it is not separated.\n- I am unclear as to what statistical test you perform to accept/reject the chi-square hypothesis.\n ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}, {"review_id": "B1l5_QS3FB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe paper makes the following two contributions: 1) a new metric to measure the realism of uncertainty estimates for regression problems which uses a Mahalanobis distance-based statistical test. 2) a new probabilistic architecture for semantic segmentation.\n\nOverall I do not think that the paper is qualifies for acceptance, because a) both contributions are only loosely connected and b) some parts are confusingly written or poorly motivated, making the paper hard to follow.\n\nAfter reading the paper it still remains unclear to me why the proposed statistical test is superior to other popular metrics, such as the log-likelihood or the metrics proposed by Mukhoti and Gal. I think the paper would benefit from a more detailed discussion that highlights the differences between the proposed metric and other commonly used metrics.\nFurthermore, in the experiments, the paper only shows that MC dropout doesn't achieve realistic uncertainty estimates. I think concluding that variational approaches underestimate the variance is a bit of stretch (see section 4.1) , i e. it would be more convincing if other approaches (e.g Blundell et al. Hernandes-Lobato and Adams) are also considered.\nThe paper also just states that uncertainty estimates obtained by MC dropout are unrealistic but doesn't elaborates how to improve them.\n\nThe second contribution, a probabilistic architecture for semantic segmentation, is not introduced in the main paper. Instead details are only provided in the appendix. In my opinion the paper would be easier to follow if it would contain a section that motivates and described the proposed architecture before jumping directly to the experiments.\n\nMinor comments:\n\n- the acronym FRRN is not defined\n\n- Section 4.2 last paragraph: I don't understand why samples obtain by MC dropout and from the latent space are considered to be from the same distribution? While both methods approximate the weight posterior they use different approximations for that.\n\n- In the introduction the paper states that the proposed segmentation network is a U-net like FRRN architecture , however in section 3 it says instead of a U-net based architecture a FRRN based architecture is used. This is somewhat confusing.\n\n- Section 4.1: how are the variances scaled for Figure 3 middle?\n\n- Section 4.1 It would be also interesting if other metrics, such as log-likelihood or RMSE, to see how well the model is able to fit the data.\n\n- I am also surprised that CVAE + MC underperforms to just using CVAE?\n\n- Section 4.1: how are the variances scaled for Figure 3 middle?\n\nReferences:\n\nEvaluating Bayesian Deep Learning Methods for Semantic Segmentation\nJishnu Mukhoti, Yarin Gal\narXiv:1811.12709 [cs.CV]\n\nWeight Uncertainty in Neural Networks\nCharles Blundell, Julien Cornebise, Koray Kavukcuoglu, Daan Wierstra\nICML 2015\n\n\nHernández-Lobato J. M. and Adams R.\nProbabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks,\nICML 2015", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes two contributions to improve uncertainty in deep learning.  The first is a Mahalanobis distance based statistical test and the second a model architecture.  Unfortunately, the reviewers found the message of the paper somewhat confusing and particularly didn't understand the connection between these two contributions.   A major question from the reviewers is why the proposed statistical test is better than using a proper scoring rule such as negative log likelihood.  Some empirical justification of this should be presented.
Decision: Reject