On the Practicality of Deterministic Epistemic Uncertainty
Authors: Janis Postels
Mattia Segu
TAO SUN
Luca Sieber
Luc Van Gool
Fisher Yu
Federico Tombari
Venue: ICLR 2022 Submitted (2022)
Keywords: uncertainty
epistemic uncertainty
uncertainty calibration

Abstract: A set of novel approaches for estimating epistemic uncertainty in deep neural networks with a single forward pass has recently emerged as a valid alternative to Bayesian Neural Networks. On the premise of informative representations, these deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution (OOD) data while adding negligible computational costs at inference time. However, it remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications - both prerequisites for their practical deployment. To this end, we first provide a taxonomy of DUMs and evaluate their calibration under continuous distributional shifts. Then, we extend them to semantic segmentation. We find that, while DUMs scale to realistic vision tasks and perform well on OOD detection, the practicality of current methods is undermined by poor calibration under distributional shifts.

Under review as a conference paper at ICLR 2022
ON THE PRACTICALITY OF DETERMINISTIC EPISTEMIC UNCERTAINTY
Anonymous authors Paper under double-blind review
ABSTRACT
A set of novel approaches for estimating epistemic uncertainty in deep neural networks with single forward pass has recently emerged as a valid alternative Bayesian Neural Networks. On the premise of informative representations, these deterministic uncertainty methods (DUMs) achieve strong performance on detecting oft-tf--iittibutiinn (OOD) data while adding negligible computational costs at inference time. However, it remains unclear whether DUMs are well calibrated and can seamlessly scale real-world application both prerequisites for their practical deployment. To this end, we first provide a taxonomy of DUMs, and evaluate their calibration under continuous distributional shifts. Then, we extend them semantic segmentation. We find that, while DUMs scale to realistic vision tasks and perform well on OOD detection, the practicality current methods is undermined by poor calibration under distributional shifts.
INTRODUCTION
Despite the dramatic enhancement of predictive performance of deep learning (DL), its adoption remains limited due to unpredictable failure on out-of-distribution (OOD) samples ( and adver- sarial attacks (3). Uncertainty estimation techniques aim at bridging this gap by providing accurate confidence levels on model's output, allowing for safe deployment of neural networks (NNs) in safety-critical tasks, e.g. autonomous driving or medical applications.
While Bayesian Neural Networks (BNNs) represent the predominant holistic solution for quantifying uncertainty 44 5) exactly modelling their full posterior is often intractable, and scalable versions usually require expensive variational approximations (6 g 10). Moreover, has recently been shown that true Bayes posterior can also lead to poor uncertainty (II). Thus, efficient approaches to uncertainty estimation largely remain an open problem, limiting the adoption within real-time applications under strict memory, time and safety requirements.
Recently, a promising line of work emerged estimating epistemic uncertainty deterministic NNs with single forward pass. By regularizing the hidden representations of model, these methods represent an efficient and scalable solution to epistemic uncertainty estimation and to the related OOD detection problem. Compared traditional uncertainty estimation techniques, Deterministic Uncertainty Methods (DUMs) quantify epistemic uncertainty by measuring distances (12) 13) or estimating the distribution of latent representations d4 15 [] detection is prerequisite for a safe deployment of DL in previously unseen scenarios, the calibration i.e. how well uncertainty correlates with model performance of such methods under continuous distributional shifts is equally important. Measuring the calibration of an epistemic uncertainty estimate on shifted data investigates whether entails information about the predictive performance of the model. This an essential requirement for uncertainty and, unlike OOD detection, and evaluation that not model-agnostic i.e. one cannot perform well without taking the predictive model into account. Nonethcless, previous work falls short of investigating calibration, and solely focuses OOD detection [3) [8)
IIZ I8 19 220 21.. While OOD
19). Further, DUMs have thus far only been evaluated on toy datasets for binary classification, small scalle image classification tasks (13) 19) and toy prediction problems in natural language processing (13). Despite claiming to solve practical issues of traditional uncertainty estimation approaches, the practicality of DUMs remains to be assessed on more challenging tasks.
This work investigates the most crucial questions on the safety of DUMs in practical applications and addresses their shortcomings. In particular: (i) we provide the first analysis of the calibration

---

Under review as a conference paper at ICLR 2022
of DUMs under continuous distributional shifts; (ii) evaluate the sensitivity of DUMs to their regularization strength; (iii) we scale DUMs to dense prediction tasks, e.g. semantic segmentation, and evaluate them under synthetic and realistic continuous distributional shift; (iv) we find that the practicality of many DUMs is undermined by their poor calibration under both synthetic and realistic distributional shifts.
RELATED WORK
Sources of Uncertainty. Uncertainty in a model's predictions can arise from two different sources 22 23). While aleatoric uncertainty encompasses the noise inherent in the data and is consequently irreducible (22), epistemic uncertainty quantifies the uncertainty associated with choosing the model parameters based on limited information, and vanishes - in principle- in the limit of infinite data. This work distinguishes two properties of epistemic uncertainty its performance on detecting OOD samples and its calibration (i.e. its correlation with model performance under distributional shifts). While the latter has been explored in the case of probabilistic approaches to uncertainty estimation (24) we are the first to investigate the behaviour of DUMs in this scenario. Notably, 25) evaluates prominent scalable epistemic uncertainty estimates on semantic segmentation. However, they investigate calibration only on in-distribution data. Further, although (15) evaluates the calibration of their approach, they do so exclusively on in-distribution data.
BNNs (26) 5) represent a principled way of measuring uncertainty. However, their intractable poste- rior distribution requires approximate inference methods, such Markov Chain Monte-Carle (26) Variational Inference (VT) (4). While these methods traditionally struggle with large datasets and architectures, variety of scalable approaches often based VI have recently emerged.
Deep Ensembles, which typically consist of identical models trained from different initializations, have been introduced to the deep learning community by Lakshminarayanan al. (27) and extended by 28 While deep ensembles are widely regarded as a strong baseline for estimating epistemic uncertainty, they come with high computational as well as memory costs.
Efficient Approaches. Recently, approaches based on stochastic regularization have been de- veloped 6 7 & 30;
31.. By keeping stochasticity inference time, they estimate uncertainty using multiple forward passes. Another line of work estimates the posterior using the Laplace- approximation 32 34) Moreover, efficient ensemble methods were proposed producing predic- 28 36 33). Despite promising results on large-scale tasks while being parameter-efficien, these methods still require sampling through the model, which can render them impractical given limited compute. To estimate uncertainty in real-time and resource-demanding tasks, recent work has focused on providing uncertainty estimates with a single forward pass. One line of work proposes principled approach for variance propagation in NNs (9) 38 10). Notably, another line of work proposes efficient approaches to estimate alcatoric uncertainty (39 40 4T).
tions using a single model (35) 29;
Recently, DUMs showed promising results OOD detection. By leveraging distances and densities in the feature space of a NN, these methods provide confidence estimates while adding negligible computational cost. Since they are united in their deterministie treatment of the weights, term them Deterministic Uncertainty Methods (DUMs). The next section provides a taxonomy of DUMs.
TAXONOMY FOR DETERMINISTIC UNCERTAINTY QUANTIFICATION
This section categorizes existing DUMs. To quantify epistemic uncertainty deterministically, the distribution of the hidden representations of NN needs to be sensitive to the input distribution. However, discriminative models suffer from the fundamental problem of feature collapse (13) 19). Thus, we firstly categorize DUMs according to the regularization method used to counteract feature collapse (See. 3.1) Moreover, we cluster DUMs according to the method used for uncertainty estimation SSec. 3.2) Tab. shows an overview of the resulting taxonomy.
Feature Collapse Discriminative models can learn to discard large part of their input information, as exploiting spurious correlations may lead to better performance on the training data distribution (46) 47). Such invariant representations learned may be blind to distributional shifts, resulting in collapse of OOD embeddings to in-distribution features This problem is known as feature collapse (13), and makes OOD detection based high-level representations impossible.
2

---

Under review as a conference paper at ICLR 2022
Uncertainty Estimation Method
Discriminative
Generative
DUMs
Distance from class centroid
Gaussian
Gaussian Mixture Models
Normalizing
Processes
Flows
Regu- larization
Distance awareness
DCS (12)
DUQ (33
SNGP (5))
DUE (8))
DDU (19)
Invertible
networks
Informative representations
DCU 16 42).
(43 44 45
MIR 20
PostNet
Table I: Taxonomy of DUMs, Methods are grouped according to their regularization of the hidden representations (rows), and their uncertainty estimation method (columns).
3.1 REGULARIZATION OF HIDDEN REPRESENTATIONS
We group DUMs according to how feature collapse is tackled. We identify two main paradigms distance awareness and informative representations which we discuss in Sec. 3.1.I and [eec. 3.1.2.
3.1.1 DISTANCE AWARENESS
Essentially, distance- aware hidden representations avoid feature collapse by relating distances be- tween latent representations to distances in the input space. Therefore, one constrains the bi- bi-Lipschitz constant, as enforces a lower and an upper bound expansion and contraction performed by model. A lower bound enforces that different inputs are mapped to distinct representations and, thus, provides a solution to feature collapse. The upper bound enforces smoothness, i.e. small changes in the input do not result in large changes in the latent space. While there exist other approaches, e.g. (48), recent proposals have primarily adopted two methods to impose the bi-Lipschitz constraint.
The two-sided Gradient Penalty relates changes in the input to changes in feature space by di- rectly constraining the gradient of the input (13). Spectral Normalization (SN) (49) is less computationally-ddmmmne
alternative. SN is applicable to residual layers and normalizes the weights W of each layer using their spectral norm sn(W) to constrain the bi-Lipschitz constant. Various DUMs SNGP (15), DUE (18) and DDU (19) rely SN to enforce distance-awarenes of hidden representations. More details on gradient penalty and SN can be found in the supplement.
Note that the Lipschitz constraint defined with respect fix distance measure, which can be difficult to choose for many high-dimensional data distributions. A popular choice, e.g. (13 15, 18),
the Lg distance. Moreover, principled approaches to provide exact singular values in convolutional layers (50) result in prohibitive computational complexity; the spectral normalization approximations typically adopted by the methods previously described have been found to be sub-optimal (51), and its interaction with losses, architecture and optimization is yet to be fully understood (52)
3.1.2 INFORMATIVE REPRESENTATIONS
While distance-awareness achieves remarkable performance on OOD detection, it does not explicitly preserve sample-specific information. Thus, depending on the underlying distance metric it may discard useful information about the input act overly sensitive. An alternative line of work avoids feature collapse by learning informative representations (14 [6 inative models to preserve information in its hidden representations beyond what is required to solve a task independent of the choice of an underlying distance metric. Notably, while representations that aware of distances in the input space are also informative, both categories remain fundamentally different their approach to feature collapse. While distance-awareness based on the choice of specific distance metric tying together input and latent space, informative representations incentivize a NN to store more information about the input using an auxiliary task (00 [6) or forbid information We identify three distinct families of approaches.
43 45). thus forcing discrim-
loss by construction 433 44
45)
Contrastive learning (53) has emerged as an approach for learning representations that are both informative and discriminative. This is utilized by Wu al. (16) and Winkens ef al. (42), who apply
3

---

Under review as a conference paper at ICLR 2022
SimCLR (54) to regularize hidden representations for discriminative task by using contrastive loss for pretraining and fine-tuning to force representations to discriminate between individual instances.
Reconstruction regularization (20) instead forces the intermediate activations to fully represent the imput. This is achieved by adding a decoder branch fed with the activations of a given layer to reconstruct the input. We term it MIR (Maximally Informative Representations).
Entropy regularization. PostNet (17) learns the classtcondi distribution of hidden representa- tions end-to-end using Normalizing Flow (NF) parameterizing Dirichlet distribution. This allows them to enforce informative representations by implicitly encouraging large entropy of the NF during training. We refer the supplement for details.
Invertible Neural Networkss (INNs) (55) 43 45 built via a cascade of homeomorphic layers, cannot discard information except at the final classification stage. Consequently, the mutual informa- tion between input and hidden representation is maximized by construction. Interestingly, Behrmann et al. (56) showed that a ResNet is invertible if its Lipschitz constant lower than 1, meaning that invertible ResNets both possess highly ionoly-ifformatve representations and satisfy However, note that this is not a necessary condition for invertibility, and thus information preservation.
awareness.
3.2 UNCERTAINTY ESTIMATION
We distinguish two directions regarding uncertainty estimation in DUMs generative and discrimina- tive approaches. While generative approaches use the likelihood produced by an explicit generative model of the distribution of hidden representations directly use the predictions based on regularized representations to quantify uncertainty.
proxy for uncertainty, discriminative methods
Generative approaches estimate the distribution of hidden representations post-training and use the likelihood as uncertainty metric. Wu et al. (16) propose method to estimate the distribution in the feature space, where the variance of the distribution is used as a confidence measure. MIR (20). DDU (19) and DCU (42) fit a condiicon GMM to their regularized hidden representations and use the log-likelihood as an epistemic uncertainty proxy. DEUP (57) uses the log- ligleiiei of a normalizing flow in combination with an aleatoric uncertainty estimate to predict the generalization error. special instance of the generative approaches are INNs as they directly estimate the training data distribution. The likelihood of the input data is used as proxy for uncertainty. While this idea is appealing. it can lead to training difficulties, imposes strong constraints on the underlying model and still remains susceptible to OOD data (58). PostNet (17) is a hybrid approach which estimates the distribution of hidden representations of each class using a separate NF end-to-end. Its loglliillihoods parameterize Dirichlet distribution. We categorize PostNet generative approach since their epistemic uncertainty is the log-likelihood of the NF associated with the predicted class.
Discriminative approaches use the predictions to directly assess confidence. Mandelbaum al. (12) propose to use a Distance-based Confidence Score (DCS) learning a centroid for each class end-to-end. Similarly, DUQ (13) builds on Radial Basis Function (RBF) networks (59) and proposes novel centroid updating scheme. Both estimate uncertainty as the distance between the model output and the closest centroid. DUMs adopting SN [55] 18) (preserving L2 distances) typically replace the softmax layer with Gaussian processes (GPs) with RBF kernel, extending distance awareness to the output layer. In particular, SNGP (5)) relies on a Laplace approximation of the GP based on the random Fourier feature (RFF) expansion of the GP posterior (60). DUE (18) uses the inducing point approximation (61 62), incorporating a large number of inducing points without overfitting (63). The uncertainty is derived the Dempster- Dempster-Shafer metric (15), resp. the softmax entropy (18)
4 EVALUATION OF DETERMINISTIC EPISTEMIC UNCERTAINTY
We measure the calibration of DUMs under continuous distributional shifts in two parts. Firstly, evaluate DUMs on image classification using synthetic corruptions (Sec. 4.1.I). Then, we extend DUMs to a large-scale dense prediction task semantic segmentation (Sec. 4.2) where evaluate their calibration on synthetic corruptions (Sec. 4.2.1) based on Cityscapes and on more realistic distributional shifts (Sec. 4.2.2) based on data collected in the simulation environment CARLA (64).
Baselines. We compare DUMs with two baselines for epistemic uncertainty Monte-Carlo (MC) dropout (7) and deep ensembles (27). Moreover, we report the softmax entropy of a vanilla NN
4

---

Under review as a conference paper at ICLR 2022
simple baseline. We refer to the supplement for details on uncertainty estimation in our baselines. Note, that the softmax entropy is expected to yield suboptimal calibration under distributional shifts since it quantifies alcatoric uncertainty while adding no computational overhead.
Methods. evaluate DUQ (3), SNGP (15) and DDU (19 as representatives of distance- awwannesssseessss since these cover both techniques SN and gradient penalty and apply different techniques for uncertainty estimation. We exclude DUE (18) since provides limited additional insights given SNGP. Moreover, we exclude DCS (12) since it only leads a marginal improvement in their own experiments and their contrastive loss only operates on class centroids and, thus, is not expected to lead distance awareness within clusters. Furthermore, we evaluate MIR (2)), DCU 422) and PostNet (17) as representatives of informative representations. However, we do not scale DCU and PostNet to semantic segmentation. DCU with its constrastive pretraining based on SimCLR (54)
computationally too demanding due to large batch sizes. PostNet does not scale to semantic segmen- tation due instabilities arising from the end-to-end training of the NF for learning the distribution of hidden representations. They require small hidden dimension (<10) which already leads to poor testset performance on CIFARIOO 100. Further, we do not evaluate methods based on invertible neural networks (44 (55) since they 1) enforce strict constraints the underlying architecture (e.g. fixed dimensionality of hidden representations) and often lead to training instabilities.
Calibration Metrics. Typical calibration metrics are Expected Calibration Error (ECE) (65) and Brier score (66). However, since most DUMs, except SNGP, do not provide uncertainty in form of probabilistic forecast, cannot rely measuring the calibration of probabilities. Thus, will exploit another desired property of uncertainty to quantify calibration, namely the ability to distinguish correct from incorrect predictions. In fact, this property is a relaxation of calibrated probabilities, as it is independent of the absolute value of uncertainty estimates and only relies on the ordering of predictions relative to each other. We assess the calibration of uncertainty estimates under distributional shifts using two metrics. Firstly, we report the Area Under the Receiver Operating Characteristic (AUROC) obtained when separating correct and incorrect predictions based on uncertainty. Moreover, we introduce a new metric, Relative Area Under the Lift Curve (rAULCC), based on the Area Under the Lift Curve (AULC) (67). The AULC is obtained by ordering the predictions according increasing uncertainty and plotting the performance of all samples with an uncertainty value smaller than a certain quantile of the uncertainty against the quantile itself.
Formally, given set of uncertainty quantiles qi E [0, 1], i E [1..... S), with some quantile step width 0 < s < 1 and the function F(g.) which returns the accuracy of all samples with uncertainty r(g.)
gi. the AULC is defined as AU -1 Eie(,....s) $ F Here, FR(-) refers to baseline uncertainty estimate that corresponds to random guessing, and subtract to shift the performance of the random baseline to zero. Note, if an uncertainty estimate is anti-correlated with a models' performance, this score can also be negative. To alleviate bias towards better performing models, we further compute the rAULC by dividing the AULC by the AULC of hypothetical (optimal) uncertainty estimation that perfectly orders samples according model performance. In classification we measure AUROC and rAULC on the image-level, in semantic segmentation on the pixel-level. In all experiments we set the quantile step width to s N where the number of predictions.
We compute AUROC and rAULC on continuous distributional shifts 1) across all severities of distributional shifts (including the clean testset) and 2) for each severity separately. We use the former method to establish quantitative comparison among the methods and the latter to depict the calibration evolution qualitatively as a function of the distributional shift's s severity.
4.1 IMAGE CLASSIFICATION
Datasets. We train DUMs on CIFAR-10
10 and CIFAR-100 (68) and evaluate on the corrupted versions of their test set CIFAR10/100-C (69) (Sec. T.T.T). These include 15 synthetic corruptions, each with 5 levels of severity. Moreover, we explore how sensitive the calibration of DUMs is to the choice of regularization strength on MNIST (70) and FashionMNIST (T)).
Models and optimization. Each method shares the same backbone architecture and uses method- specific prediction head. When training on CIFAR- 10/100, the backbone architecture is a ResNet- 50 (72) For the experiments regarding hyperparameter sensitivity MNIST and Fashion-MNIST, we employ a multilayer perceptron (MLP) as feature extractor with 3 hidden layers of 100 dimensions each and ReI.U activation functions. Each DUM has hyperparameter for the regularization of its
5

---

Under review as a conference paper at ICLR 2022
hidden representations. We choose the hyperparameter such that it minimizes the validation loss. All results are averaged over independent runs. The standard deviation and optimization details can be found in the supplement where not present in the main paper.
4.1.1 CONTINUOUS DISTRIBUTIONAL SHIFTS
Method
CIFARIO-C AUROC
CIFAR100-C AUROC
ACC
rAULC
ACC
rAULC
Softmax
0.882
0.782
0.708
0,610
0.762
0.596
MC Dropout
0.885
0.866
0.829
0.615
0.818
0.726
Ensemble (7
0.910
0.85 0
0.833
0.628
0.824
0.713
SNGP (15
0.903
0.833
0.766
0.611
0.788
0.623
DDU 19
0.884
0.673
0.441
0.609
0.635
0.339
MIR 20
0.889
0.79
0.697
0.617
0.726
0.514
DUQ
13
0.860
0.773
0.614
DCU
(22)
0.945
0.794
0.706
0.642
0.750
0.558
PostNet (17)
0.882
0.784
0.676
0.520
0.743
0.603
Table We compare Softmax, MC Dropout (7), Deep Ensembles, SNGP, DDU, MIR, DUQ, DCU and PostNet on CIFAR10/100-C. We evaluate the accuracy (ACC) the uncorrupted testset, AUROC and rAULC. Ensembles and MC dropout demonstrate better uncertainty calibration than most DUMs. Only SNGP consistently outperforms the softmax entropy. DCU's superior performance is expected since uses expensive contrastive pretraining. DUQ did not converge on CIFAR100-C due to training instabilities arising from dynamically updated cluster centroids.
Tab. 2 reports testset accuracy and calibration for the baselines and DUMs. AUROC and rAULC are computed for each corruption across all severity levels, then averaged over all corruptions. Further, Fig. I depicts our metrics depending on the severity of corruptions. We generally observe that ensembles and MC dropout demonstrate best performance in terms of calibration. Further, SNGP is the only DUM that consistently outperforms the softmax entropy. Overall we observe that DUMs using the distribution of hidden representations for estimating epistemic uncertainty yield worse calibration. Among these we find that regularizing hidden representations by enforcing distance awareness (DDU) yields the worst calibration. The superior performance of DCU in terms of testset accuracy originates from their extensive contrastive pretraining which includes extensive data augmentation and prolonged training schedule. note that DUQ (13) did not converge on CIFAR 100 due to training instabilities. These arise from maintaining the class centroids, which become very noisy for 100 classes with only 600 samples per class.
Moreover, report OOD detection performance of models used in Tab. 2 and Fig. in the supple- ment. Interestingly, despite competitive performance of OOD detection DUMs fall short terms of uncertainty calibration compared to MC dropout and deep ensembles.
4.1.2 SENSITIVITY HYPERPARAMETERR
We are interested in the qualitative impact of the regularization strength on the uncertainty calibration. Therefore, we train DUQ (13,, SNGP (15), MIR (20) and DDU (19) using various regularization strengths on MNIST and evaluate on continuously shifted data by rotating from 0 to 180 degrees in steps of 20 degrees. FFg.. 2 depicts the test accuracy against the rAULC for various regularization strengths. Only for MIR we observe a clear, positive correlation between regularization strength and calibration. The supplement depicts similar results on FashionMNIST (
4.2 SEMANTIC SEGMENTATION
This section evaluates whether DUMs seamlessly scale to realistic vision tasks and compares their behaviour under synthetic and realistic continuous distributional shifts with the softmax entropy, MC dropout and ensembles. Therefore, we apply MIR (20), SNGP (15) and DDU (19) to semantic segmentation. Note that DUQ (13) did not converge this task.

---

Under review as a conference paper at ICLR 2022
sagp ddu mir dug softmax
onsemble dropout winkens PostNet
ACC- gm
AUROC gn
rAULC gm
ACC mb
AUROC mb
rAULC mb
0.8
0.0 0.5
80
05
0.4
0.5
000
0.2 0.6
03
0.000
022
0.5
02
077
0
0
4
Figure Softmax entropy, ensembles 27.. MC dropout (7.. DUQ (3), SNGP (5). MIR (0).
DDU (19) and DCU (22 on CIFARIO-2 (upper row) and CIFAR 100- (lower row) (69 We show the accuracy, AUROC and rAULC on the corruptions gaussian noise (gn) and motion blur (mb) against the corruption severity. While all methods, except DCU, demonstrate a similar accuracy, DUMs in particular methods based on generative modeling of hidden representations yield worse calibration. DUQ did not converge on CIFAR 100. Other corruptions are included in the supplement.
0.998
DUQ
SNGP
DDU
MIR
Accuracy
D.316
x
xxx
0.2
0.91
91
0.00
X
xXxX
XxX
0.07
x
X
40
60
80
0.97
0.58
0.6
0.58
0.6
0.62
0.35
0.4
0.45
0.5 0.6
0.7
rAULC
TAULC
rAULC
rAULC
Figure 2: We analyse the sensitivity of DUQ 13), SNGP (15), MIR (00) and DDU (19) to their regularization strength. Therefore, we train models on MNIST and evaluate on continuously shifted data by rotating from 0 to 180 degrees in steps of 20 degrees. We plot the accuracy on the unperturbed testset against the rAULC computed using data from all levels of perturbation. Only for MIR we observe a clear, positive correlation between regularization strength and calibration. For DDU and SNGP, smaller regularization parameter denotes stronger regularization.
We consider semantic segmentation multidimensional classification problem, where each pixel of the output mask represents an independent classification problem. Given an image x with n pixels y {21, yn J. the predictive distribution factorizes according to p(y x) ply1 x)P(1/2
x) . p(yn x). We evaluate the calibration of the pixel-level uncertainty in our experiments.
Datasets. We evaluate synthetic distributional shifts using corrupted version of Cityscapes (73) (Cityscapes-C (74) which contains the same corruptions as LIFAR10/100-C To further benchmark DUMs in realistically and continuously changing environment, collect synthetic dataset for semantic segmentation. We use the CARLA Simulator (64) for rendering the images and segmentation masks. The classes definition is aligned with the CityScape dataset (73). Training data is collected from four towns in CARLA. We produce 32 sequences from each town. Vehicles and pedestrians are randomly generated for each sequence. Every sequence has 500 frames with a sampling rate of 10 FPS. We uniformly sample validation set. We introduce continuous distributional shifts by varying the
and weather conditions (visual examples and details on data collection are in the supplment). The time-of-the-dae is parameterized by the sun's 's altitude angle, where 90º means mid-day (training data) and 0 means dust/dawn. We produce samples with altitude angles from 90" to 15" by steps of 5", and 15 to -50, where the environment changes sharply, in 1 steps. In order to continuously change the weather conditions, we increase the magnitude of the rain in four steps (see supplement for visual examples). We refer this dataset as CARLA-C.
Backbone. We adopt Dilated ResNet (DRN) 775 76) as semantic segmentation backbone since is based residual connections allowing the SN. Using dilated convolutions it improves spatial accuracy, achieving satisfactory results on CityScapes (73). We adopt the variant DRN-A-50. All results averaged across 5 independent repetitions.
SNGP. DRN uses 1 x convolutions at the last layer to map the latest feature map to the predicted segmentation mask. This works under the assumption that all pixels in the output mask are i.i.d.
7

---

Under review as a conference paper at ICLR 2022
random variables. Following this intuition, we extend SNGP semantic segmentation by fitting GP: R + RC at pixel level that maps from the deep feature dimension 2 to the number of classes c. By keeping the GP kernel parameters shared across all pixels, we simulate 1 1 convolutional GPs, ie. a: (Hz x Wi x 2) + (H₂ x Wi x C), where a convolves the GP, H, and W are, respectively, feature map height and width at layer 1, Z is the number of latent features and C' is the number of output classes. For details about the GP we refer to (15) or the supplement.
MIR and DDU require fitting the distribution of hidden representations. We fit a Gaussian mixture model (GMM) with 20 components (i.e. number of classes) to each spatial location of the hidden representations using features extracted from the training data independently. This assumes that the distribution is translation invariant and factorizes along the spatial dimensions of the latent space. Pixel-level uncertainties are then computed using bi-cubic interpolation following a similar procedure as (77 in this framework. We refer to the supplement for more details
4.2.1 CITYSCAPES CORRUPTED
We evaluate the softmax entropy, ensembles (7), MC dropout 7 SNGP 15) MIR 20). DDU (I9)
on Cityscapes-C (74). Tab. and Fig. depict mean Intersection over Union (mloU) and calibration
performance in terms of AUROC and rAULC. Ensembles and MC dropout yield the best calibration, while among DUMs only SNGP consistently ourperforms the softmax entropy.
Method
Cityscapes-C AUROC
CARLA-C AUROC
mloU
TAULC
mIoU
rAULC
Softmax
0.503
0,815
0.737
0.422
0.854
0.818
MC Dropout
0.506
0.846
0.785
0,410
0.843
0.730
Ensemble 27
0.525
0.835
0.751
0.428
0.863
0.812
SNGP (5))
0.519
0.833
0.759
0.424
0.853
0.813
DDU 19
0.505
0.731
0.542
0.408
0.467
-0.038
MIR 20
0.504
0.729
0.564
0.412
0.744
0.619
Table 3: We compare semantic segmentation using Softmax, MC Dropout 7. Deep Ensembles, SNGP, DDU and MIR on Cityscapes-C. and CARLA-C. We evaluate the mloU the uncorrupted testset and AUROC/AULC across all levels of corruption. Again, ensembles and MC dropout yield better calibrated uncertainty than most DUMs. Most notably, only SNGP consistently outperforms the softmax entropy. DUMs using an explicit generative model of hidden representations to estimate uncertainty perform partieular bad on realistic distributional shifts (CARLA-C).
Solumax
Ensemble
SNGP
MC Dropout
MIR
DDU
mloU
AUROC
TAULC
0.1
0.8
0.5
0.2
0.6
0.4
0
0.5
0.9
0.8
0.4
0.8
0.6
0.3
0.7
0.2
0.4
0
2
4
2
2
Severity
Severity
Severity
Figure 3: Softmax entropy, ensembles (27), MC dropout 77. SNGP (5), MIR 20), DDU 19 on Cityscapes-C (74). We show the mloU, AUROC and rAULC on the corruptions gaussian noise (upper) and motion blur (lower) depending on the corruption severity. While all methods demonstrate a similar mloU, DUMs in particular methods based on generative modeling of hidden representations yield worse calibration across corruption severities. Other corruptions are the supplement.
8

---

Under review as a conference paper at ICLR 2022
Softmax
Ensemble
SNGP
MIR
MC Dropout
DDU
Accuracy - time_ day
AUROC- time_ol_de day
FAULC time day
0.4
0.8
0.5
0
0.3
0.1
0..5
Sun angle
Sun angle [']
Sun angle [']
Accuracy strensh
AUROC rain strength
rAULC rain strength
0.4
0.8
0.5
0.3
0.6
0
0.4
0..
0.2
3
Rain strength
Rain strenntth
Rain strongth
Figure 4: Softmax entropy, ensembles (27). MC dropout (7), SNGP (15), MIR 20), DDU 19 on CARLA-C (74). We show the mloU, AUROC and rAULC for each method on the corruptions "time of day" (upper) and "rain" (lower) depending on the corruption severity. While all methods demonstrate similar mIoU pattern, DUMs in particular methods based on generative modeling of hidden representations yield worse calibration across corruption severities.
4.2.2 REALISTIC CONTINUOUS DISTRIBUTIONAL SHIFTS
Similarly, evaluate the softmax entropy, ensembles (27), MC dropout 77, SNGP (15), MIR (20). DDU (19) on CARLA-C. (Tab. 3) and Fig. 4 depict mloU and calibration performance in terms of AUROC and rAULC. Ensembles yield the best calibration. Among DUMs only SNGP consistently ourperforms the softmax entropy which is in line with the results on image classification (Sec. 1.1.1
CONCLUSION & DISCUSSION
This work investigated the calibration under continuous distributional shifts of DUMs which recently showed good OOD detection performance and are interesting for practical applications in need of effi- cient uncertainty quantification. Overall, we observe that such uncertainty estimates are considerably worse calibrated than scalable Bayesian methods. We observe this on image classification (Sec. 1.1.1 as well as semantic segmentation (Sec. 4.2) and synthetic as well more realistic distributional shifts. SNGP (15) denotes the only DUM that consistently yields better calibrated uncertainties under continuous distributional shifts than the softmax entropy. Simultancously. SNGP is the only DUM which derives there uncertainty from its predictive distribution.
In particular, our experiments reveal that methods relying the distribution of hidden representations to quantify uncertainty (42) 20 19) are poorly calibrated. Arguably, it understandable that these methods are worse calibrated than SNGP since they do not take into account the predictive distribution. The underlying assumption is that locations in feature space entail information about the correctness of predictions. While this is arguably true, features also contain additional information that render them suboptimal for judging the correctness of predictions due to ambiguities. We refer to the supplement for further theoretical consideration of the limitations of DUMs. Overall, this underlines the necessity to refrain from DUMs that purely rely of distance or log-likelihood in the feature space when well calibrated uncertainties are required.
Moreover, another desirable property for such methods would be that the strength of the feature space regularization correlates with the quality of the uncertainty. Due to the original purpose of most DUMs, this would be at least expected for OOD detection. However, this is not verified for Lipschitz regularization by our investigation Sec. 4.1.I). We hypothesize that this originates from the choice of metric for regularization i.e. Le distance which is not meaningful in the image space. We hope that our findings will foster future research on making these promising family of methods better calibrated and more broadly applicable.

---

Under review as a conference paper at ICLR 2022
REFERENCES
[ Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classifi- cation tasks to a new unlabeled sample. Advances neural information processing systems, 24:2178-2186 2011.
[2] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International Conference Machine Learning, pages 10-18. PMLR, 2013.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfel- low, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv: 2013.
[4] Geoffrey E Hinton and Drew Van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference on Computational learning theory, pages 5-13, 1993.
[5] Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
[6] Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparam- eterization trick. In Advances in neural information processing systems, pages 2575-2255 -5883, 2015.
[7] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty deep learning. In international conference on machine learning, pages 1050-1059, 2016.
[8] Guodong Zhang. Shengyang Sun, David Duvenaud, and Roger Grosse. Noisy natural gradient variational inference. In International Conference Machine Learning, pages 5852-5861.
PMLR, 2018
[9] Janis Postels, Francesco Ferroni, Huseyin Coskun, Nassir Navab, and Federico Tombari. Sampling-fiee free epistemic uncertainty estimation using approximated variance propagation. In Proceedings of the IEEE International Conference Computer Vision, pages 2931-2940, 2019.
[10] Antonio Loquercio, Mattia Segu, and Davide Scaramuzza. A general framework for uncertainty estimation in deep learning. IEEE Robotics and Automation Letters, 5(2):3153-3160, 2020.
[11] Florian Wenzel, Kevin Roth, Bastiaan Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good is the bayes posterior in deep neural networks really? International Conference on Machine Learning, pages 10248-1000 10259. PMLR, 2020.
[12] Amit Mandelbaum and Daphna Weinshall. Distance-based confidence score for neural network classifiers. arXiv preprint arXiv:1709.09844 2017.
[13] Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using single deep deterministic neural network. In International Conference Machine Learning, pages 9690-9700. PMLR, 2020.
[14] Alexander A Alemi, Ian Fischer, and Joshua V Dillon. Uncertainty in the variational information bottleneck. arXiv preprint arXiv:1807.00900 2018.
[15] Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax- Weiss, and Balaji Laksh- minarayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Conference on Neural Information Processing Systems, 2020.
[16] Mike Wu and Noah Goodman. A simple framework for uncertainty in contrastive learning.
arXiv preprint arXiv:20100000338 2020.
10

---

Under review as a conference paper at ICLR 2022
[17] Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncer- tainty estimation without ood samples via density-bassee -aseed pseudo-counts. Advances in Neural Information Processing Systems, 33:1356-1367, 2020.
[18] Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. Improving deterministic uncertainty estimation in deep learning for classification and regression. arXiv preprint arXiv:2 1409, 2021.
[19] Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. De- terministic neural networks with appropriate inductive biases capture epistemic and alcatoric uncertainty. arXiv preprint arXiv:2 1152,, 2021.
[20] Janis Postels, Hermann Blum, Yannick Strümpler, Cesar Cadena, Roland Siegwart, Luc Van Gool, and Federico Tombari. The hidden uncertainty in neural networks activations. arXiv preprint arXiv:20/2.03008, 2020.
[21] Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, and Stephan Günne- mann. Natural posterior network: Deep bayesian predictive uncertainty for exponential family distributions. arXiv preprint arXiv:2 2021.
[22] Armen Der Kiureghian and Ove Ditlevsen. Alcatory or epistemic? does matter? Structural safery, 31(2):105-113 2009.
[23] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 5580-5590, 2017.
[24] Jasper Snock, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan Sebastian Nowozin, Sculley, Joshua Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, pages 13969-133 13980, 2019.
[25] Fredrik K Gustafsson, Martin Danelljan, and Thomas B Schon. Evaluating scalable bayesian deep learning methods for robust computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 318 319, 2020.
[26] Radford M Neal et al. Meme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
[27] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems, pages 4402-6413, 2017.
[28] Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXiv:2002.06711, 2020.
[29] Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran. Efficient and scalable bayesian neural nets with rank-1 factors. In International conference machine learning, pages 2782-2777 2792. PMLR, 2020.
[30] Mattias Teye, Hossein Azizpour, and Kevin Smith. Bayesian uncertainty estimation for batch normalized deep networks. In International Conference on Machine Learning, pages 4907-4916, 2018.
[31] Ian Osband, Zheng Wen, Mohammad Asghari, Morteza Ibrahimi, Xiyuan Lu, and Benjamin Van Roy. Epistemic neural networks. arXiv preprint arXiv:2107.08924, 2021.
[32] Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable laplace approximation for neural networks. In 6th International Conference on Learning Representations, ICLR 2018- Conference Track Proceedings, volume 6. International Conference on Representation Learning, 2018.
11

---

Under review as a conference paper at ICLR 2022
[33] Jongseok Lee, Matthias Humt, Jianxiang Feng, and Rudolph Triebel. Estimating model uncertainty of neural networks in sparse information form. International Conference on Machine Learning, pages 5702-5713, 13. PMLR. 2020.
[34] Apoorva Sharma, Navid Azizan, and Marco Pavone. Sketching eurvature for efficient out-of- distribution detection for deep neural networks. arXiv preprini urXiv:2102.12550 2021.
[35] Christian Rupprecht, Iro Laina, Robert DiPietro, Maximilian Baust, Federico Tombari, Nassir Navab, and Gregory D Hager. Learning in an uncertain world: Representing ambiguity through multiple hypotheses. In Proceedings of the IEEE International Conference on Computer Vision, pages 3591-3600, 2017.
[36] Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew Mingbo Dai, and Dustin Tran. Training independent subnetworks for robust prediction. In International Conference on Learning Representations, 2020.
[37] Alexandre Rame, Remy Sun, and Matthieu Cord. Mixmo: Mixing multiple inputs for multiple outputs via deep subnetworks. Proceedings of the IEEE International Conference Computer Vision, 2021.
[38] Manuel HauBmann, Fred A Hamprecht, and Melih Kandemir. Sampling- freee variational inference of bayesian neural networks by variance backpropagation. In Uncertainty in Artificial Intelligence, pages 563-573. PMLR, 2020.
[39] Christopher M Bishop. Mixture density networks. 1994.
[40] Philipp Oberdiek, Matthias Rottmann, and Hanno Gottschalk. Classification uncertainty of deep neural networks based on gradient information. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pages 113-125. Springer, 2018.
[41] Scong Joon Oh, Kevin Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, and Andrew Gallagher. Modeling uncertainty with hedged instance embedding. Proceedings of the International Conference on Learning Representations, 2019.
[42] Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R Led- sam, Patricia Mac Malliliiass, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, et al. Con- trastive training for improved out-of-distribution detection. arXiv preprint urXiv:2007.05566 2020.
[43] Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini, Ralf Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich Köthe. Analyzing inverse problems with invertible neural networks. arXiv preprint arXiv: 888.8000000 2018.
[44] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshninarayanan. Hybrid models with deep and invertible features. In International Conference on Machine Learning, pages 4723-4732. PMLR, 2019.
[45] Lynton Ardizzone, Radek Mackowiak, Carsten Rother, and Ullrich Köthe. Training normalizing flows with the information bottleneck for competitive generative classification. Advances in Neural Information Processing Systems, 33, 2020.
[46] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: founda- tions and learning algorithms. The MIT Press, 2017.
[47] Mattia Segå, Alessio Tonioni, and Federico Tombari. Batch normalization embeddings for deep domain generalization. arXiv preprint arXiv:2011.12677 2020.
[48] Anton Obukhov, Maxim Rakhuba, Alexander Liniger, Zhiwu Huang, Stamatios Georgoulis, Dengxin Dai, and Lue Van Gool. Spectral tensor train parameterization deep learning layers. In International Conference Artificial Intelligence and Statistics, pages 3547-3555. PMLR, 2021.
[49] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv: arXiv:1802 08880000997 2018.
12

---

Under review as a conference paper at ICLR 2022
[50] Hanie Sedghi, Vineet Gupta, and Philip M Long. The singular values of convolutional layers. arXiv preprint arXiv: 10408, 2018.
[51] Sahil Singla and Soheil Feizi. Bounding singular values of convolution layers. arXiv preprint arXiv:1911. 10258, 2019.
[52] Mihaela Rosca, Theophane Weber, Arthur Gretton, and Shakir Mohamed. A case for new neural network smoothness constraints. arXiv preprint arXiv:2012.07969, 2020.
[53] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.033: 000748 2018.
[54] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMIR, 2020.
[55] Jöm-Henrik Jacobsen, Arnold Smeulders, and Edouard Oyallon. i-revnet: Deep invertible networks. arXiv preprint 1rXiv:1802.07088. 2018.
[56] J Behrmann, W Grathwohl, RTQ Chen, D Duvenaud, and JH Jacobsen. Invertible residual networks. arxiv e-prints. arXiv preprint arXiv:1811.00995 2018
[57] Moksh Jain, Salem Lahlou, Hadi Nekoci, Victor Butoi, Paul Bertin, Jarrid Rector-Brooke Maksym Korablyov, and Yoshua Bengio. Deup: Direct epistemic uncertainty prediction. arXiv preprint arXiv:2102.08500) 2021.
[58] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In International Conference on Learning Representations, 2018.
[59] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (1:21)222772333 2324, 1998.
[60] Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer school on machine learning, pages 63-71. Springer, 2003.
[61] Michalis Titsias. Variational learning of inducing variables in sparse gaussian processes. In Artificial intelligence and statistics, pages 567-574. PMLR, 2009.
[62] James Hensman, Alexander Matthews, and Zoubin Ghahramani. Scalable variational gaussian process classification. In Artificial Intelligence and Statistics, pages 351-360. PMLR, 2015.
[63] David Burt, Carl Edward Rasmussen, and Mark Van Der Wilk. Rates of convergence for sparse variational gaussian process regression. In International Conference on Machine Learning, pages 862-871. PMLR, 2019.
[64] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1-16, 2017.
[65] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht, Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015.
[66] GLENN BRIER. Verification of forecasts expressed terms probability. Monthly Weather Review, 78(1):1-3, 1950.
[67] Miha Vuk and Tomaz Curk. Roc curve, lift chart and calibration plot. Metodoloski zvezki, 3()):99, 2006.
[68] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar- 10 dataset. online: http:///ww. cs. toronto. edu/kriz/cifar himl, 55:5, 2014.
13

---

Under review as a conference paper at ICLR 2022
[69] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustmess to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
[70] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun, com/exdb/mnist// 1998.
[71] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: (00iv:770.007200 708.07747, 2017.
[72] Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016
[73] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[74] Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness in object detection:
Autonomous driving when winter is coming. arXiv preprint urXiv:1907.0748% 2019.
[75] Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. In International Conference Learning Representations (ICLR), 2016.
[76] Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. Dilated residual networks. In Computer Vision and Pattern Recognition (CVPR), 2017.
[77] Hermann Blum, Paul-Edouard Sarlin, Juan Nieto, Roland Siegwart, and Cesar Cadena. The fishyscapes benchmark: Measuring blind spots in semantic segmentation. arXiv preprint 1944v:190..03215, 2019.
14


Reviews:
[{"review_id": "piYOUeKnmBp", "reviewer": "Reviewer_JspH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The reasons invoked to include some of the cited DUM approaches are not very convincing, since the point of a benchmark is to evaluate all the relevant approaches according to the same yardstick. Since there is no methodological novelty in this paper, I would suggest to open up the comparison to a slightly larger set of methods than what was done here.\n\nNonetheless, the provided comparisons can be useful for future research aiming at proposing novel approaches and possibly avoiding all the possible comparisons, focusing instead on the methods which seem to perform well in this paper. These comparisons also point to DUM approaches being generally insufficiently well calibrated (especially those relying on the distribution of hidden representations).\n\nThere is another deterministic estimator of epistemic uncertainty that could have been considered, of a very different nature (but only applicable with one can have an estimator of aleatoric uncertainty, e.g. for pairs of examples with the same x but different y): DEUP (Lahlou et al, 2021). They train a 2nd network to predict the per-example out-of-sample error of the main predictor.", "summary_of_the_paper": "\nThis paper reviews and compares a class of epistemic uncertainty estimation methods that avoid sampling (hence called deterministic) as well as multiple models (like ensembles) because of their memory footprint.", "summary_of_the_review": "\nThis paper compares experimentally a number of epistemic uncertainty estimators within a fairly limited but interesting class of methods. This could be useful to practitioners of the field. A broader set of comparisons could improve the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "dk8VofQFtRU", "reviewer": "Reviewer_ZZs6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n\n(1)\tThe paper is well-organized. It first introduces a summary of DUMs and then provides experiments on both image classification tasks and real-world applications, i.e., semantic segmentation.\n\n(2)\tThe experiments are systematic, including all the necessary evaluation metrics and different levels of distribution shifts.\n\nWeaknesses:\n\nFor the method:\n\n(1)\tSome important deterministic uncertainty methods such as the Dirichlet-based methods [1,2,3] are missing in the paper.  Moreover, the authors cite the Posterior network [2] work as one of the DUMs but did not evaluate it.  In addition, [4], that performed the evaluation of the Dirichlet-based methods , also considered the uncertainty robustness under distribution shifts. The authors should provide a discussion and comparison of their work with [4].\n\n(2) The authors should discuss in the body of the papers the about underlying reasons for the issues associated with the DUM methods and even propose possible solutions to address or avoid these problems.\n\n(3)\tThe authors should discuss each deterministic uncertainty method in terms of the strengths/weaknesses of each method and the possible assumptions. Some necessary equations should also be provided. Then, we can have a clear mind about which method to choose for experiments\n\n(4)\tThe authors mainly evaluate the uncertainty robustness under distributional shifts. However, the author should also provide some discussion about the OOD detection results. Could DUMs perform as well as Ensemble and MC-dropout when there are no distributional shifts?\n\nFor the experiments\n\n(1)\tThe experiment results lack the necessary analysis. For example, why do most DUMs have much worse calibration performance than softmax? \n\n(2) 5 DUMs are considered for image classification while only 3 DUMs are considered for semantic segmentation. The performance of DUMs may not be representative of semantic segmentation.\n\n(3)\tThe experiments did not consider the regression tasks.\n\n(4)\tFigures 1,2,3,4 are very confusing. It is hard to distinguish different colors from figures. The authors may use different curves when plotting the results to distinguish DUMs with Ensemble and MC-dropout.\n\n[1] Malinin, Andrey, and Mark Gales. \"Predictive uncertainty estimation via prior networks.\" arXiv preprint arXiv:1802.10501 (2018).\n\n[2] Charpentier, Bertrand, Daniel Zügner, and Stephan Günnemann. \"Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts.\" arXiv preprint arXiv:2006.09239 (2020).\n\n[3] Malinin, Andrey, Bruno Mlodozeniec, and Mark Gales. \"Ensemble distribution distillation.\" arXiv preprint arXiv:1905.00076 (2019).\n\n[4] Kopetzki, Anna-Kathrin, et al. \"Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?.\" International Conference on Machine Learning. PMLR, 2021.\n", "summary_of_the_paper": "This paper mainly summarizes and evaluates the existing methods for estimating epistemic uncertainty through a single pass of the neural networks. The authors categorize deterministic uncertainty methods (DUMs) based on how latent representation is learned under regularization and how uncertainty is quantified. For evaluating DUMs, they provide some analyses on the uncertainty calibration performance under different-level distribution shifts for image classification and semantic segmentation. Specifically, they demonstrate that DUMs cannot generate well-calibrated uncertainty under distribution shifts compared to the MC-dropout and Ensemble methods.", "summary_of_the_review": "Overall, this paper provides the evaluation for some determinist uncertainty estimation methods focusing on the robustness of uncertainty under distributional shifts. However, the paper should include more related methods in comparison with a discussion about the pros/cons and possible assumptions for each method. The experiment results should also be analyzed in detail. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "b8qWC9KUyUv", "reviewer": "Reviewer_WCgZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Title of the paper seems too broad and not representative of the content. “Practicality” is a very general term. The analysis and findings seem rather specific. It mainly seems to be that ”DUMs are not well calibrated under continuous distributional shifts”.\n\nA comprehensive review of DUMs could be of great benefit, if done correctly. The area is a bit non-structured and the evaluations are not standardized. However, the current version of this paper falls short in helping in both aspects.   \n\nLiterature review: \nThe explanations are very surface level and non-comprehensive. As an example, the literature review on the “discriminative methods” summarizes every approach in one very vague sentence, requiring the reader to refer to the papers to actually be able to understand. Given that the work is mostly a review of other methods, it should at least serve as a good survey, for a non expert reader. The current version of the manuscript falls short as is.\n\nBesides that, the literature review of uncertainty estimation is incomplete (see [1, 2, 3, 4, 5] as examples). MIMO approaches [2, 3] are similar to deep ensembles but without the computational and memory requirements. One may argue that these approaches fall much closer to DUMs as opposed to Ensemble methods in terms of efficiency (memory & computation). Especially given that they are mostly image based, they are directly applicable here. They’re not even mentioned in the literature review.\n\nThe terms out-of-distribution detection, epistemic uncertainty estimation, aleatoric uncertainty estimation, and calibration. are not very well defined in the community when it comes to their quantification. Especially OOD detection (maybe the most well defined one) and epistemic uncertainty estimation, are often used interchangeably in many papers (debatable if correctly or incorrectly).  \nWhen it comes to adding synthetic corruption to the image (input), how is that not aleatoric uncertainty? One may argue that OOD detection is a better proxy for epistemic (although not perfect), and calibration on corruptions (noise in input) is better for aleatoric uncertainty. One example for each: [1] for using input variation / corruption for aleatoric uncertainty evaluation. [4] for using OOD detection as a proxy for epistemic uncertainty estimation. If that is the case, why should models designed for epistemic uncertainty estimation, be the best candidates for aleatoric uncertainty estimation anyway. \n\nIf the authors disagree with these conventions they should at least provide a comprehensive justification on why they think things should be different, and they should propose clear evaluation protocols for each type of uncertainty estimation. \n\n\n\n[1] Modeling Uncertainty With Hedged Instance Embedding\n\n[2] MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks\n\n[3] Training Independent Subnetworks For Robust Prediction\n\n[4] Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks\n\n[5] Epistemic Neural Networks\n", "summary_of_the_paper": "This paper analyzes deterministic uncertainty estimation methods, in terms of their calibration under distributional shift. They provide a literature review, propose a quantification metric, and compare performance of several uncertainty estimation methods.", "summary_of_the_review": "Overall I think the experiments of the paper are insightful and have merit, but the presentation, positioning, and claims could be significantly improved. Maybe if the paper’s focus, description, and claims were concentrated around “evaluating calibration of deterministic epistemic uncertainty models under distribution shift”, it would be more representative of the experiments, and it could become a more conclusive work. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MU26PgWlJIJ", "reviewer": "Reviewer_z2yi", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n- The motivation of the paper is strong, these new uncertainty quantification methods (DUMs) can raise some doubts as they do not do the usual bayesian model averaging (like ensembles or dropout/dropconnect), and as far as I know, there is no previous comparison of all these methods, particularly in more realistic settings. There is the important question if these methods work well in more complex settings.\n- The authors propose a taxonomy on DUM method which I find useful for the future and very positive. The survey and description in the first pages of the paper is also very useful for the community.\n- The evaluation seems to be correct, the dataset selection for image classification and segmentation experiments is excellent, and the metric selection is good, except for the rAULC metric which is not that common, but seems appropriate as not all evaluated methods produce probabilities. The authors argue why standard calibration errors cannot be used, but I think it could be an improvement to normalize confidence scores produced by DUMs to be able to use calibration errors, I do not see this as a big issue more than using a standard metric. For example gradient uncertainty can be normalized to produce scores in [0, 1] and ECE can be computed for these predictions.\n- The supplementary material contains many additional results of interest, like variations of DUM hyper-parameter effects, justification about issues with intermediate activations, full results for corrupted versions of all datasets, and OOD detection without dataset shift (standard OOD detection benchmarks), showing that some DUMs work better in some datasets, and ensembles on SVHN.\n- The additional segmentation results in the supplementary are very interesting, in particular Fig 7 and Fig 8, as it shows how MIR produces radically different uncertainty as seen in the heatmaps of uncertainty (last column), in comparison with other methods with mostly produce high uncertainty in the borders between regions, acting as a kind of edge detector, and this could lead to additional insights on why some methods fail.\n- There are clear conclusions from this paper, that DUM methods do not perform OOD at acceptable levels for both classification and segmentation (except for DCU), and their calibration also suffers considerably. All of this in distribution shift settings.\n- Figure 1 shows that DUMs suffer from the same issue with dataset distribution shift, where OOD/calibration performance suffers when corrupting input images, same as Ovadia et al. 2019.\n- The paper is overall well written, and I had no trouble reading it. \n- I find that the analysis in this paper is original and novel, there is small epsilon contribution to the state of the art, which should enable future research on DUMs.\n\nWeaknesses\n\n- A strong theoretical foundation on why these methods fail would strengthen the paper. The authors perform an analysis of regularization strength on MNIST (Easy but not many conclusions can be drawn from this dataset) and about distributions of intermediate features, but I think a full theoretical paper on this topic would be required.\n\nMinor Issues\n\n- It is possible that the paper would benefit from a \"page 2 figure\" to introduce the reader. There is no space, but I think putting Table 1 on top of the second page would serve this purpose.\n- In Figure 1, I think axis tick labels can be reduced a bit in size or even factored across different plots (putting the tick labels in the first plots on the right and bottom rows) to save space and make the plots a bit bigger, for readability.\n- One DUM that I think was left out is gradient-based uncertainty, for this please see \" Classification uncertainty of deep neural networks based on gradient information\" Oberdiek et al. 2018. I am not sure if adding it would change the conclusions, but maybe it can be included in the taxonomy.\n- I totally agree with the authors that DUQ could diverge on CIFAR100-C, I have experienced this using DUQ on small datasets. I suggest that the authors remove the DUQ centroid learning algorithm (the running average over input features) and train the centroids using gradient descent. This will probably solve the issue.\n- For readability, in Figure 6 of the supplementary, it is better to include the metric in the label of the y axis. Same for Figures 11 to 20, the reader will be pleased to find metrics and information in the x/y axis labels.", "summary_of_the_paper": "This paper is an analysis and benchmark comparison of deterministic uncertainty quantification models under dataset shift. The authors evaluate DUM methods, and show that while they work well in academic datasets (mostly CIFAR10/100), they seem to fail in out of distribution detection on a more realistic setting (Semantic segmentation on CityScapes and CARLA simulated images) and in distribution shift settings (corrupting input images), particularly in terms of lower quality uncertainty calibration and out of distribution detection performance, which has practical implications for the use of these algorithms.\n", "summary_of_the_review": "I believe that this is a good paper, with important conclusions and insights for the uncertainty quantification community, it is correctly evaluated, and should be accepted. There are no major issues to be dealt with.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper performs an empirical evaluation of deterministic methods for the quantification of epistemic uncertainty.  There is no new algorithm.  The main contribution is the empirical evaluation.  This empirical evaluation will be useful for the community. It is an independent evaluation that casts some doubts on the calibration of several existing deterministic techniques, which will spur additional research. However, the paper is not well written. As pointed out by the reviewers, the paper does not provide much background. It refers to many  concepts without defining them. The concepts are not new (references are provided for each concept), but since the paper does not describe any new technique it should do a good job at explaining those concepts. The authors added some explanations in the supplementary material, but some of those explanations should really be in the main paper.  The most important issue with the paper is that it does not explain why the deterministic techniques do not seem to be well calibrated. The authors added a "theoretical justification" in section 6.1, but it amounts to saying that deterministic methods make a point estimate, which is too general to explain much. An important factor for proper generalization and calibration is the inductive bias of the model. At the end of the day, if we generate data from a model, then that model will be better calibrated than the other models. So a discussion of the inductive bias of each model and how this inductive bias relates to the properties of each dataset would have been much more insightful.
Decision: Reject