Conservative Uncertainty Estimation By Fitting  Prior Networks
Authors: Kamil Ciosek
Vincent Fortuin
Ryota Tomioka
Katja Hofmann
Richard Turner
Venue:  (2020)
Keywords: uncertainty quantification
deep learning
Gaussian process
epistemic uncertainty
random network
prior
Bayesian inference

Abstract: Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks. In this paper, we theoretically justify a scheme for estimating uncertainties, based on sampling from a prior distribution. Crucially, the uncertainty estimates are shown to be conservative in the sense that they never underestimate a posterior uncertainty obtained by a hypothetical Bayesian algorithm. We also show concentration, implying that the uncertainty estimates converge to zero as we get more data. Uncertainty estimates obtained from random priors can be adapted to any deep network architecture and trained using standard supervised learning pipelines. We provide experimental evaluation of random priors on calibration and out-of-distribution detection on typical computer vision tasks, demonstrating that they outperform deep ensembles in practice.

Published as a conference paper at ICLR 2020
CONSERVATIVE UNCERTAINTY ESTIMATION BY FITTING PRIOR NETWORKS
Richardd Kamil Ciosek', Vincent Fortuin': 12, Ryota Tomioka', Katja Hofmann', Richard Turner
ABSTRACT
Obtaining high-quality uncertainty estimates essential for many applications of deep neural networks. In this paper, we theoretically justify a scheme for esti- mating uncertainties, based on sampling from prior distribution. Crucially, the uncertainty estimates are shown to be conservative in the sense that they never un- derestimate a posterior uncertainty obtained by a hypothetical Bayesian algorithm. We also show concentration, implying that the uncertainty estimates converge to zero as we get more data. Uncertainty estimates obtained from random priors can be adapted to any deep network architecture and trained using standard su- pervised learning pipelines. We provide experimental evaluation of random priors calibration and cuttof-distribution detection typical computer vision tasks, demonstrating that they outperform deep ensembles in practice.
INTRODUCTION
Deep learning has achieved huge success in many applications. In particular, increasingly often, is used component in decision-making systems. In order to have confidence in decisions made by such systems, is necessary to obtain good uncertainty estimates, which quantify how certain the network is about given output. In particular, if the cost of failure is large, for example where the automated system has the capability to accidentally hurt humans, the availability and quality of uncertainty estimates
determine whether the system is safe deploy all (Carvalho 2016
Leibig et al. 2017: Michelmore et al. (018)). Moreover, when decisions are made sequentially, good uncertainty estimates are crucial for achieving good performance quickly (Bellemare et al. 2016
Houthooft et al. 2016 Ostrovski et al., 2017 Burda et al. 2018).
Because any non- non-Bayesian inference process is potentially sub-optimal (De Finetti 1937), these uncertainty estimates should ideally be relatable to Bayesian inference with a useful prior. Deep en- sembles (Lakshminarayanan et al., 2017). one of the most popular methods available for uncertainty estimation in deep networks today, struggle with this requirement. While deep ensembles can be related (Rubin 1981) to Bayesian inference settings where the individual models are trained on subsets of the data, this is not how they are used in practice. In order to improve data efficiency, all ensembles are typically trained using the same data (Lakshminarayanan et al.] 2017), resulting in method which does not have theoretical justification. Moreover, deep ensembles overconfident uncertainty estimates in practice. On the other hand, Monte-Carlo dropout can be viewed (Gal & Ghahramani 2016) as certain form of Bayesian inference. However, doing variational inference to a quasi-KL.
give
quires requires either a limit to be taken or a generalization divergence (Hron et al. 2018). In practice, MC dropout can give arbitrarily overconfident estimates FFong et al.] 2019). More broadly, a category of approaches, known as Bayesian Neural Networks 1996). maintains a distribution over the weights of the neural network. These methods have sound Bayesian justification. but training them is both difficult and carries an accuracy penalty, particularly for networks with convolutional architectures (Osaa et al. 2019). Moreover, tuning BNNs is hard and achieving a good approximation to the posterior is difficult (Brosse et al.) 2018).
Blundell 2015; Welling & Teh.
We use another way of obtaining uncertainties for deep networks, based on fitting ran- dom priors (Osband et al.) 2018 2019). Random priors are easy to train and were found to work very well in practice (Burda et al., 2018). To obtain the uncertainty estimates, Affiliations: 1. Microsoft Research Cambridge; 2. ETH Zurich; 3. University of Cambridge. The second author was an intern Microsoft when contributing this work.

---

Published as a conference paper at ICLR 2020
first train predictor network to fit prior. Two amples of prior-prrec predictor pairs are shown in the top two plots of Figure Faced with novel input point, obtain an uncertainty (Figure bottom plot) by mea- suring the error of the predictor network against this pattern. Intuitively, these errors will be small close to the training points, but large far from them. The pat- terns themselves are drawn from randomly initialized (and therefore untrained) neural networks. While this way of estimating uncertainties was known before (Os- band et al., 2019), did not have theoretical justifi- cation beyond Bayesian linear regression, which is too limiting for modern applications.
0.1
Prior Predictor
Training
-0.2
-4
-2
2
4
Prior
0.3
Predictor
Training data
Contributions We provide sound theoretical frame- work for obtaining uncertainty estimates by fitting ran- dom priors, method previously lacking principled justification. Specifically, we justify estimates in the uncertainty of the output of neural networks with any architecture. In particular, we show in Lemma | and Proposition
0.2
0.1
-
[ that these uncertainty estimates are con- servative, meaning they are never more certain than Bayesian algorithm would be. Moreover, in Proposi- tion 2 we show concentration, i.e. that the uncertainties become zero with infinite data. Empirically, we evalu- ate the calibration and out-of-distribution performance of our uncertainty estimates on typical computer vision tasks, showing practical benefit over deep ensembles and MC dropout.
Ground truth Prediction Training data
0
-4
-2
Figure 1: On top. two predictors (green) were trained to fit two randomly- generated priors (red). On the bottom, we obtain uncertainties from the difference between predictors and priors. Dots cor- respond to training points Ię-
PRELIMINARIES
We are going to reason about uncertainty within the for- mal framework of stochastic processes. We now intro- duce the required notations.
A stochastic process is collection of random variables (f(z)). We consider processes where x R.K and the
f(x) takes values in R A stochastic process has exchangeable outputs if the dis- tribution does not change when permuting the M entries in the output vector. Allowing a slight abuse of notation, denote the liite-dimensional distribution of the process {z() } for the set X {x; )==1....N as f( .N = f(X). In practice, the finie-dimeensonal distribution reflects the idea of restricting the process to points I1, N and marginalizing over all the other points. Infer- ence can be performed on stochastic processes similarly to probability distributions. In particular, we start with some prior process {x()), observe set of N training points X {2: ie...... N....N and labels | and then consider the posterior process (x)) whose soiite-dimnssioall distributions are given by fxy ...
f ........ . T1..... NN V1.... yNN) for any set of testing points rī ...... We use subscripts to denote conditioning on the dataset throughout the paper. We denote the variance of (x,(x.) with ox (x=). stochastic process called Gaussian all its finite-dimensional distributions are Gaussian. Given test point v., we denote the posterior GP mean with yx, (I,) and posterior GP variance with o (x.). We provide more background on GPs in Appendix
ESTIMATING UNCERTAINTY FROM RANDOM PRIORS
Intuition Uncertainties obtained from random priors have appealing intuitive justification. Consider the networks in the top part of Figure We start with a randomly initialized prior network, datapoint, train the predictor network (green) match this
shown in red. Whenever
2

---

Published as a conference paper at ICLR 2020
prior. Uncertainties then be obtained by considering the squared error between the prior and the predictor at a given point. An example uncertainty estimate is shown as the shaded blue area in the bottom of Figure I While it may at first seem that the squared emrrr is poor measure of uncertainty because it can become very small by random chance, we formally show Section 4.1 that this is very improbable. In Section[4.2] more data. Similarly to GP inference, uncertainty estimation in our framework does not depend on the regression label. The prediction mean (blue curve in the bottom part of Figure I is obtained by fitting a completely separate neural network. In section 6 we discuss how this framework avoids the overconfidence characteristic of deep ensembles (Lakshminarayanail al. 2017).
show that this error goes down to zero
observe
The process of obtaining network uncertainties involves randomly initialized prior networks, which are never trained. While this may at first appear very different from they way deep learning is normally done, these random networks are a crucial component of our method. We show in Section 4.1 that the random process that corresponds to initializing these networks can be interpreted prior a Bayesian inference procedure. A prior conveys the information about how the individual data points are related. The fact that we are using random networks has both practical and theo- retical benefits. Practically, since the prior does not depend on the data, there is no way that it can overfit. The use of random priors also has strong empirical support randomlyy initialized networks have been recently used as priors to obtain state-of-the-art performance on computer vision tasks (Ulyanov al. 2018 Cheng al.. 2019). Theoretically. using random priors satisfies the likelihood principle (Robert 2007). Moreover, random priors can be viewed as a safe choice since they make the minimum reasonable assumption that the network architecture is appropriate for the task. In fact, whenever deep learning is used, with or without uncertainty estimates, practitioners are already implicitly making that assumption.
Prior
Algorithm The process of training the predictor networks is shown in Algorithm D The function TRAIN-UNCERS UNNERUNNERTASNIIS first generates random priors, L.e neural networks with random weights. In our notation, it corresponds to sampling functions from the prior process {fxx)) These priors, eval- uated at points from the dataset X {=:)ee...... are then used labels for supervised learning, per- formed by the function FIT. After training, when we want to obtain an uncertainty estimate & given test point I., we use the formula
Algorithm Training the predictors.
function TRAIN- (UAAI-UNCERTAINTIESXX))
for 11... ..... do
I [f(r) hx < FIT(X, end for return fü hx, end function
D random prior
function FIT(X, ((X))
L(h) Exex (f) - h(*)ll2
(2(x)) max m0,00, (z.)) ) + (0,
02). (1)
hx +- OPTIMIZE(L) > SGD or similar
return hxy: return trained predictor end function
Here, the quantity 0² is the sample mean of the squared error. We will show in Section 4 that it is an unbiased estimator based estimate of the standard deviation of squared error across bootstraps, needed to quantify our uncertainty about what the uncertainty is. The
variable that models the uncertainty. On the other hand, Gr is the sample-
3 controls the degree to which this uncertainty is taken into account. Formally, the quantities are defined
(x. 2 EB MB [lf(x))) hxxs(()))1
(2)
0 (x.) A DB 1 (63(1x.) (11((=))))))))01)1)))))))) -xxx
In the above equations, B is the number of prior functions and each prior and predictor network has M outputs. Because the predictors are trained independently, uncertainty estimates obtained from each of the B predictor-prior pairs independent. We defer the discussion of details of network architecture to Section 5 Our experiments (Section 7 show that it is often sufficient to use B in practice.
3

---

Published as a conference paper at ICLR 2020
4 THEORETICAL RESULTS
In Section we introduced a process for obtaining uncertainties in deep learning. We now seek to provide formal justification. We define the expected uncertainties as
(1r)) - Ey [[][(.)]] E, 1 [1(x)))) (xx(xx)))
(4)
In other words, 03 is the expected version of the sample-based uncertainties (z.) introduced in equation 2 Since Bayesian inference known to be optimal (De Finetti 1937; Jaynes 2003
Robert 2007), p. the most appealing way of justifying uncertainty estimates 0i and o is to relate them to Bayesian posterior x (sxx). We do this in stages. First, in Section 4.1 we prove that the obtained uncertainties are larger than ones arrived at by Bayesian inference. This means that our uncertainties are conservative, ensuring that our algorithm is never more certain than it should be. Next, in Section 4.2 show that uncertainties concentrate, i... they become small get more and more data. These two properties are sufficient to justify the use of our uncertainties in many applications.
4.1 UNCERTAINTIES FROM RANDOM PRIORS ARE CONSERVATIVE
From the point of view of safety, it is preferable to overestimate the ground truth uncertainty than to underestimate it. We now show that this property holds for uncertainties obtained from random priors. We first justify conservatism for the expected uncertainty a defined in equation 4 and then for the sampled uncertainty de defined in equation 2
Amortized Conservatism We first consider a weak form of this conservatism, which we call amortized. It guarantees that o2 is never smaller than the average posterior uncertainty across labels sampled from the prior. Formally, amortized conservatism holds if for any test point 2, we have
0 (x..) 2 Ex(x) [x+))
Here , corresponds to the second moment of the posterior process {1x)( We will introduce stronger version of conservatism, which does not have an expectation the right-hand side, later in this section (eq. S). For now, we concentrate on amortized conservatism. In Lemma (proof in appendix), we show that it holds under very general conditions.
Lemma 1. For any function h [K++))
RM,, for any test point x. E RKK and for any stochastic process {f(x)) with all second moments finite and exchangeable outputs
(x.) Ex(x) (o.) + M (xx)-1xx(xx.)1P
(6)
Relation to GP Lemma holds for any prior process {f(x)) However, the prior process used by Algorithm iis not completely arbitrary. The fact that prior samples are obtained by initializing neural networks with independently sampled weights gives additional structure. In fact, it can be shown that randomly initialized neural networks become close to GPs as the width of the layers increases. While the original result due to Neal (1966) held for a simple network with one hidden layer, it has been extended to a wide class of popular architectures, including to CNNs and RNNs of arbitrary depth (Matthews et al. 2018 Lee et al.) 2018, Novak et al. 2019, Williams 1997
Le Roux & Bengio 2007] Hazan & Jaakkola 2015 Daniely et al.) [016]] Garriga- Alonso et al. 2019). Recently, it has been shown to hold for a broad class of functions trainable by gradient descent (Yang, 2019). While the precise statement of these results involves technicalities which fall beyond the scope of this paper, we recall the key insight. For a family of neural networks (rW (x)). where the weights are sampled independently and W is the width of the hidden layers, there exists a limiting kernel function lixo such that
lim H ( CP(O, koc) W00
In other words, as the size of the hidden layers increases, the stochastic process obtained by initializ- ing networks randomly converges in distribution to a GP. In the context of our uncertainty estimates, this makes it reasonable for W large enough to consider the prior to be GP. We stress that the GP assumption has to hold only for the prior network, which is never trained. We do not make any assumptions about connections between the predictor training process and GPs.
4

---

Published as a conference paper at ICLR 2020
Strict Conservatism Denoting the posterior GP variance with ox (x.), we define uncertainty timates to be strictly conservative when
(x.) 2 xx (x.).
(8)
This statement is stronger than the amortized conservatism in equation 5 Intuitively, equation can be interpreted as saying that our uncertainty estimates are never too small. This confirms the intuition expressed by [Brdda et al. (2018) that random priors do not overfit. Below, in Proposition we outline how to guarantee strict conservatism formally. is proved in Appendix F.I
Proposition 1 (Strict Conservatism Expectation). Assume that f is a GP. Then for any function : R BRKK RM have
(ar)) = -0 (x,) B5(x) |t (=l) hx hx
Moreover, equality holds if and only if hx,( pxz(x.).
Conservatism with Finite Bootstraps Lemma above shows conservatism for expected uncer- tainties, i.e. 8² introduced equation However, in practice we have to estimate this expectation
using a finite number of bootstraps, and use the sampled uncertainties oi defined in equation 2 We now state conservatism guarantee that holds even in the case of just one bootstrap (B 1). The proof is deferred to Appendix F.T
Corollary (Strict Conservatism for Finite Bootstraps). Assume that f is a GP. Assume that the random variable 62, (Ix.) has finite variance upper bounded by von- Then with probability 1 for any function h Nx + R.A,, we have
(x,)) + 1 JUB 2 (2)(2)) ) 2 (x.).
(10)
However, applying Corollary requuires the knowledge of OUB. We now provide an upper bound. Lemma 2. Assume that the GP {{x) is zero mean with exchangeable outpuls and the function hxy takes values in [-U, UJM Assume that permuting the outputs of produces the same permu- tation in the outputs of hx With probability 1 - 5, we have
Var. [o (xx)) S VUB:
(11)
where UUR is expressible in terms of observable quantities.
The proof and the explicit formula for TUB is deferred to Appendix F.I vatism is desired, but absolutely essential, and replace rUB with the sample-hased estimate i (x.). defined in equation 2 In this case, the con- servatism guarantee is only approximate. This how we obtained equation used by the algorithm in practice.
cases where conser- avoid the torturous calculation of Lemma
4.2 UNCERTAINTIES FROM RANDOM PRIORS CONCENTRATE
While the conservatism property in Proposition is appealing, it is not sufficient its for the uncertainty estimates to be useful. We also need concentration, i.e. a guarantee that the uncertainties g² become small with more data. We can gurantee this formally by assuming that the class of neural networks being fitted is Lipschitz-continuous and bounded. Intuitively, by assumption of Lipschitz continuity, the predictors hx/ cannot behave very differently on points from the training and test sets, since both come from the same data distribution. We can then show concentration by using standard Rademacher tools to obtain a bound on the expected uncertainty in terms of the squared error on the training set. This process is formalized in Proposition 2
Proposition 2. If the training converges, i.e. the training loss MN EN (l. (xx( oA for arbitrarily large training sets, then assuming the predictors hx bounded and Lipschitz continuous with constant L, then under technical conditions the uncertainties concentrate, i.e. G2 g²(x.) N and B with probability 1.
1
5

---

Published as a conference paper at ICLR 2020
B bootstraps
B
bootstraps
f
M outputs
hxs
M outputs
prior networks are randomly sampled
predictors are trained on x; R
Figure 2: Architecture of the random prior networks and predictor networks hxy. The predictor networks f typically share the same architectural core, but have additional layers relative to the prior networks. Both the green and red parts of the predictor networks trained.
The proof and the technical conditions are given in Appendix F training error is zero for arbitrarily large training sets, which might at first seem unrealistic. We argue that this assumption is in fact reasonable. The architecture of our predictor networks (Figure 2 right diagram) is a superset of the prior architecture (Figure 2 left diagram), guarantecing the existence weight settings for the predictor that make the training loss zero. Recent results on deep learning optimization (Du et al. 2019 Allen-Zhu et al. 2019) have shown that stochastic gradient descent can in general be expected to find representable functions.
Proposition assumes that the
5 PRACTICAL CONCLUSIONS FROM THE THEORY
We now re-visit the algorithm we defined in Section 3 with the aim of using the theory above to obtain practical improvements in the quality the uncertainty estimates.
Architecture and Choosing the Number of Bootstraps Our conservatism guarantee in Proposi- tion [ holld for any architecture for the predictor f- In theory, the predictor could be completely arbitrary and does not even have to be a deep network. In particular, there no formal requirement for the predictor architecture to be the same the prior. On the other hand, to show concentration Proposition
2 we had to ensure that the prior networks are representable by the predictor. In the architecture shown in Figure 2 where the predictor mirrors the prior, but has additional layers, giving it more representational power. Moreover, the architecture requires choos- ing the number of bootstraps B. Our experiments in Section 7 show that even using B 1, i.e. one bootstrap, produces uncertainty estimates of high quality in practice.
practice,
Modeling Epistemic and Aleatoric Uncertainty Proposition T and Proposition 2 hold for any Gaussian Process prior. By choosing the process appropriately, can model both epistemic and aleatoric uncertainty. Denote by (n(x)) a stochastic process obtained by randomly initial- izing neural networks and denote by { } the noise term, modeling the alcatoric (observa- tion) noise, where samples are obtained from E(z) ~ NN 1) at cach independently (see [Ap- pendix for more background on aleatoric noise). We can now choose the prior process as sum [f(=)] {n(x) + ( of epistemic component (n(x:) and the noise term. The amount of aleatoric uncertainty can be adjusted by choosing o²--
Prior Choice, Weight Copying and Conservatism One question that can asked about our architecture (Figure is whether it is possible for the predictor to exactly copy the prior weights, giving zero uncertainty everywhere. one-dimensional regression problem, o 0 and the both the priors and predictors are linear functions. In this case, after training on two points, the predictors will agree with the priors every- where and uncertainty estimates will be zero. However, this is still consistent with conservatism guarantee The reason for this is once we assume such a linear prior, we are comparing to GP with linear kernel. But GP with that kerne will also have uncertainty after seeing two samples.
useful edge case to consider here is when we are solving

---

Published as a conference paper at ICLR 2020
In practice this means that we have to choose the architecture of the prior networks be expressive enough, which is no different from choosing a reasonable prior for Bayesian inference. Empirically, the tested network architecture did not show weight copying.
PRIOR WORK
Randomized Prior Functions (RPFs) Our work was inspired by, and builds on, Randomised Prior Functions (Osband et al. 2019 2018), but it is different in two important respects. First, the existing theoretical justification for RPFs only holds for Bayesian linear regression (Osband et al.) 2018 equation 3) with non-zero noise added to the priors. In contrast, our results are much more general and hold for any deep network with or without added aleatoric noise. Second, we are target- ing a different setting. While RPFs were designed as way of sampling functions from the posterior, we provide estimates of posterior uncertainty at a given test point. Our algorithm is based on the work by Burda et al. (2018), who applied RPFs to exploration in MDPs, obtaining state-of-the art results, but without justifying their uncertainty estimates formally. Our paper provides this miss- ing justification, while also introducing way of quantifying the error in estimating the uncertainty itself. Moreover, since Burda et al. (2018) focused on the application of RPFs to Reinforcement Learning, they only performed of -utto--iisttiuttion evaluation on the relatively easy MNIST dataset (LeCun) 1998). In contrast, in Section / we evaluate the uncertainties on more complex vision tasks. The term prior networks has also been used (Malinin Gales| 2018) to denote deep networks that output the parameters of prior distribution, an approach fundamentally different from our work.
Deep Ensembles The main alternative approach for obtaining uncertainties in deep learning are deep ensembles (Lakshminarayanan et al., 2017). Building on the bootstrap (Efron & Tibshirani] 1994). deep ensembles maintain several models and quantify epistemic uncertainty by measuring how their outputs vary. Crucially, deep ensembles use representations trained on regression labels, and tend to learn similar representations for different inputs with similar labels, which can lead to over-fitting the uncertainty estimates. A useful edge case to consider is if the each of the models in the ensemble is convex in the weights. In this case, models in a deep ensemble will all converge to the same weights and produce zero uncertainty. While deep learning models used in practice aren't t normally convex, show empirically in section 7 that deep ensembles can give overconfident un- certainty estimates in practical vision tasks, particularly on points that have the same label as points in the training set. Since our method avoids overconfidence, it can be understood as complementary to deep ensembles, to be used in situations where obtaining conservative estimates is more important than the representational benefit of using labels. In practice, deep ensembles also require using more bootstraps to achieve the same OOD performance. Moreover, they do not have theoretical support in the case when all the members of the ensemble are trained on the same data, which how they are used in practice (Lakshminarayann
et al. 2017).
Dropout In cases where it is not economical to train more than one network, uncertainties can be obtained with dropout (Srivastava et al. 2014 Gal & Ghahramani] 2016). Monte-Carlo dropout can be viewed (Gal & Ghahramani] 2016) as a form of approximate Bayesian inference. However, to do so requires a rather unnatural approximating family from the perspective of approximate inference. Also, one has then either to take a limit or generalize variational inference to KLL (Hron al.) (018)) divergence. In addition, dropout can be interpreted in terms of MAP inference (Nalis- nick et al.) 2019). Another alternative view of MC dropout is as an ensemble method in which the ensemble members have shared parameters (which means they are trained together) and where the ensembling is applied at test time too. This latter view is arguably as natural as the Bayesian inter- pretation. For this reason we discuss MC dropout separately from BNNs. Since dropout implicitly approximates non- Gon-Gaasiian weight distribution with Gaussians, it exhibits spurious patterns in the obtained uncertainties, which can lead to arbitrarily overconfident estimates (Foong et al. 2019). In contrast, due to the conservatism property, random priors avoid such overconfidence.
Bayesian Neural Networks (BNNs) Bayesian Neural Networks (Blundell et al. 2015 Kingma &
Welling 2014 Rezende et al. 2014 Welling & Tch 2011 Brosse et al. 20188) explicitly model the
'The existing justification of RPFs variance.
Oshand et al.
2019 Section 5.3.1) involves division by the noise
7

---

Published as a conference paper at ICLR 2020
distribution over weights of neural network. While BNNs provide link between deep learning and Bayesian inference, they are very slow to train. Even recent tuned implementations of BNNs (Osawa et al. 2019) are several times slower than supervised learning. This happens despite using battery of technical optimizations, including distributed training and batch normalization. Moreover, modern convolutional BNNs still carry significant accuracy penalty when deployed with realistic settings of prior variance
7 EXPERIMENTS
Encouraged by the huge empirical success of random priors in Reinforcement Learning (Burda et al, 2018), we wanted to provide an evaluation in more typical supervised learning setting. We tested the uncertainties two ways. First, we investigated calibration, i.e. whether we can expect a higher accuracy for more confident estimates. Next, we checked whether the uncertainties can be used for out-of-distribution detection. We compared to two competing approaches for uncertainty detection: deep ensembles (Lakshminarayanan et al., 2017) and spatial concrete dropout (Gal al. 2017). The same ResNet architecture served as a basis for all methods. Details of the implementation are provided in Appendix A
Oft-st-Distiituttio Detection We evalu- ated the uncertainty estimates on out-of- distribution detection. To quantify the results, we evaluated the area under the ROC curve (AUROC) for the task of deciding whether given image comes from the same distribution or not. All methods were trained on four classes from the CIFAR-10 (Krizhevsky et al., 2209) dataset (training details provided [Ap- pendix A). We then tested the resulting net- works on images from withheld classes and the SVHN dataset (Netzer et al. 2011]. which contains completely different images. Results are shown in Table [I] Considering the statisti- cal errors (see/Appendix B), random priors per- formed slightly better than deep ensembles with adversarial training for
DE +AT
RP
DE
DR
B=1
Train V. cat/deer
0.99
0.83
0.96
0.81
Train V. vehicles
1.00
0.82
0.96
0.76
Train excluded
1.00
0.82
0.96
0.77
Train SVHN
0.95
0.88
0.96
0.86
B=10
Train V. cat/derr
1.00
0.95
0.99
0.82
Train vehicles
1.00
0.92
0.98
0.78
Train v. excluded
1.00
0.93
0.98
0.79
Train v. SVHN
0.97
0.94
0.99
0.87
Table 1: Oft-of-distribution AUROC for random
priors (RP), deep ensembles (DE), deep ensem- bles with adversarial training (DE+AT) and spa- tial concrete dropout (DR). Estimated confidence intervals are provided in Appendix B
1 and about the same for B = 10. For dropout, B refers to the number of dropout samples. Dropout per- formed worse, but quality of the uncertainties, we also show uncertainty histograms in Figure 3 The figure shows the distribution of uncertainty estimates for seen data (top row) vs. unseen data (bottom row) for bootstrap sizes = (1,5, 5, 10]. The main conclusion is that uncertainties obtained from random pri- ors are already well-separated with B 1, while deep ensembles need more bootstraps achieve the full separation between test and train examples. We provide additional experimental results, showing OOD accuracy and an evaluation on CIFAR 100 Appendix B
cheaper to train. In order to gain more finely insight into the
Calibration Good uncertainty estimates have the property that accuracy increases as we become more certain, a property known as calibration. We measured it by evaluating average accuracy on the subset of images with uncertainty smaller than a given value. We trained on four classes from the CIFAR- 10 (Krizhevsky et al. 2009) dataset. We then tested the resulting networks on the whole dataset, which included both the seen and unseen classes. Results are shown in Figure Ideally, in calibrated method, these curves should be increasing, indicating that method always becomes more accurate it becomes more confident. In coarse terms, Figure 4 confirms that all methods except a degenerate deep ensemble with only one bootstrap are roughly monotonic. However, uncertainty estimates from random priors more stable, showing monotonicity on finer scale well as on large scale. Interestingly, calibration improved only slightly when increasing the number of bootstraps B.
"See appendix E of the paper by Osawa al. 2019
8

---

Published as a conference paper at ICLR 2020
RP. scen
DC seen
DE+AT, seen
DR. seen
0.010
B
8 5
8
0.005
1:00
0.01
0.02
0.010
6.005
0.000
0.00
0.00
0.000
0.00
0.25
0.75
1.00
0.00
0.25
0.75
1.00
0.00
0.25
0.75
1.00
0.010
0.02
0.015
0.03
10
B
10
10
10
0.005
0.01
6.000
0.00
0.000
0.00
Figure 3: Distribution of uncertainty estimates for various algorithms. Top row shows seen data, bottom row shows unseen data from CIFAR-10. For random priors (RP), uncertainties are ô². For other algorithms, they are mP)) where py is the averaged output of models in ensemble Lakshminarayanan et al. 2017).
1.0
1.0
1.0
0.3
0.9
0.9
0.8
Accuracy
0.8
Accuracy
0.8
0.7
0.7
RP
0.6
0.6
12
10
5
12
10
8
10
Number of points includee (x1000)
Number of points included (x1000)
Number of points incluted (x1000)
Figure 4: Calibration curves showing the relationship between uncertainty (horizontal axis) and
accuracy (vertical axis) for B 1,5, 10 on CIFAR-10.
Subsampling Ablation the previous ex-
RP
DE
DR
+AT
periment, we kept the architectural and opti- mization choices fixed across algorithms. This ensured a level playing field, but meant that we were not able to obtain zero training error on the predictor networks used by random priors. However, we also wanted to evaluate random priors in the setting of near-zero training error. To do this, we used smaller set of training im- ages, while still keeping the network architec- ture the same. This allowed us to obtain near- complete convergence (details in Amppendixx A). Results of this ablation are shown in Figures and and 6 as well as Table analogous to our results on the full dataset presented above. In this sub-sampled regime, the random prior method easily outper- formed competing approaches, showing better calibration (Fig. (5). The histograms in Figure 6 also demonstrate good separation between seen and unseen data. In the -uitt--diittiuution benchmarks reported in Table [ the random prior method has comfortably outperformed the baselines. While this training regime is not practical for real-life tasks, it demonstrates the potential performance of random priors when trained to full convergence.
B=1
Train excluded
1.00
0.90
0.89
0.91
Train v. SVHN
1.00
0.95
0.94
0.97
B=10
Train excluded
1.00
0.94
0.90
0.92
Train v. SVHN
1.00
0.97
0.95
0.97
Table 2: Out-of-distribbuiio AUROC for the same
models as above (see Tab. [ on subsampled data. Numbers are accurate up #0.01.
Sensitivity to Initialization Scale We performed an ablation to test the robustness of our algorithm to the scaling of the weight initialization in the prior. Results are shown in Figure 7 where we plot the relationship between initialization scale (taken from the set {0.01, 0.1, 1.0. 2.0, 5.0, 10.0)) and AUROC performance on the CIFAR- 10 task. OOD performance is relatively robust with respect to the weight initialization within one order of magnitude.

---

Published as a conference paper at ICLR 2020
0.0
0.0
3.0
0.6
0.6
0.6
DE
DE+AM
DE+AT
D1
01
05
0.5
0.5
Number included
Number included
Number included
Figure 5: The relationship between uncertainty (horizontal axis) and accuracy (vertical axis) for 1, 5, 10 on subset of 75 samples from CIFAR-10, In well-calibrated models, accuracy increases as uncertainty declines.
RP.
DE+AT,
0.04
0.04
0.04
0.30
0.02
0.00
0.00
0.00
0.00
5.0
0.25
9.00
0.50
1.00
0.25
1.00
RP. urisen
DE
AT unsewn
DR. unseer
0.04
8-1
0.05
10
0.02
10
0.02
10
0.02
8
3.0
2.5
5.0
7.5
10.0
9.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
3.75
1.00
3.00
0.25
0.50
5.75
1.00
Figure 6: Distribution of uncertainty estimates for various algorithms. Top row shows data, bottom row shows unseen data from CIFAR-10, where we trained on sample of 75 images from the training For random priors (RP), uncertainties 62, For other algorithms, they max(p,), where Py is the averaged output of models in ensemble [akssminarayaaaa et al. 2017
Summary of experiments have shown that uncertainties obtained from random priors achieve competitive performance with fewer boot- straps in a regime where the network architecture is typical for standard supervised learning workloads. Random priors showed superior perfor- mance in a regime where the predictors can be trained to near-zero loss.
Figure 7: Robustness of OOD perfromance initialization scale. Conf. bars present, but small, denoting high confidence. Horizontal axis logarithmic.
8 CONCLUSIONS
We provided a theoretical justification for the use of random priors for obtaining uncertainty estimates in the context of deep learning. We have shown that the obtained uncertainties are conservative and that they con- centrate for any neural network architecture. We performed an extensive empirical comparison, showing that random priors perform similarly to deep ensembles in typical supervised training setting, while outperforming them in regime where we are able to accomplish near-zero training loss for the predictors.
REFERENCES
Zeyuan Allen-Zhu. Yuanzhi Li. Zhao Song. convergence theory for deep learning via over- parameterization. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th In- ternational Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 242-252, Long Beach, Californa, USA, 09-15 Jun 2019. PMLR. URI. http: http:://pooceeiiie ing. m1 m.r.press/vvl v0tsss/vvv//// 11en zhul 99..htm
10

---

Published as a conference paper at ICLR 2020
Mare Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation. In Advances Neural Information Processing Systems, pp. 1471-1479, 2016.
Charles Blundell, Julien Coreeiise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint 155444150,,05424, 2015.
Nicolas Brosse, Alain Durmus, and Eric Moulines. The promises and pitfalls of stochastic gradient langevin dynamics. In Advances in Neural Information Processing Systems, pp. 8268-8278. 2018.
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. arXiv preprint arXiv: 12894, 2018.
Ashwin Mark Carvalho. Predictive Control under Uncertainty for Safe Autonomous Driving: Integrating Data- Driven Forecasts with Control Design. PhD thesis, UC Berkeley, 2016.
Zezhou Cheng, Matheus Gadelha, Subhransu Maji, and Daniel Sheldon. A bayesian perspective on the deep image prior. In Proceedings the IEEE Conference Computer Vision and Pattern Recognition, pp- 5443 $,51, 2019.
Amit Daniely, Roy Frostig, and Yoram Singer. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity. In Advances In Neural Information Processing Systems, 2253-2261, 2016.
Bruno De Finetti. La prévision: ses lois logiques, sources subjectives. In Annales de I' l'institut Henri Poincaré, pp. 1-68, 1937.
Simon Du Jason Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. In International Conference on Machine Learning, pp. 1675-1685, 2019.
John Duchi. Probability bounds, 2009.
Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap. SIAM Review, 36())667746 678, 1994. doi: 10. 137/11376667
Andrew YK Foong, David R Burt, Yingzhen Li, and Richard E Turner. Pathologies of factorised gaussian and dropout posteriors in bayesian neural networks. arXiv preprint 199:::90990001999 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33nd International Conference on Machine Learning, JCML 2016, New York City, NY, USA. June 19-24, 2016, pp. 1050-100 1059, 2016. press / v48/ gwwp6.............../........../ html
mir.
Yarin Gal, Jini Hron, and Alex Kendall. Concrete dropout. In Advances in Neural Information Processing Systems Annual Conference on Neural Information Prucessing Systems 2017, 4-9 December 2017, Long Beacc, CA, USA, pp. 3581-3590, 2017.
Adri Garriga-Alonso, Carl Edward Rasmussen, and Laurence Aitchison. Deep convolutional networks as shallow gaussian processes. In International Conference on Learning Representations, 2019. URL uttpttps:
//openreview, net. forum?id=kkk fal.llssiooo fm10ckm
Tamir Hazan and Tommi Jaakkola. Steps toward deep kernel methods from infinite neural networks. arXiv preprint 150.00550..05 2015.
Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbecl. Vime: Variational information maximizing exploration. In Advances in Neural Information Processing Systems, pp. 1109 1117, 2016.
Jiri Hron, Alexander G. de G. Matthews. and Zoubin Ghahramani. Variational bayesian dropout: pit- falls and fixes. Proceedings of the 35th International Conference on Machine Learning, ICML 2018. Stockholmsmäisvnn Stockholm, Sweden, July 10-15, 2018, pp. 2024-2033, 2018. URL kttp: / /pprccediigss.o www.:mmrr..p.....mmmmmmmmm wwt.pprss//////roooo.. 18a.
Edwin T Jaynes. Probability theory: The logic of science. Cambridge university press, 2003.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv: 1412.6980, 2014.
11

---

Published as a conference paper at ICLR 2020
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceed- ings, 2014, URLp::/:///axx ozg/ aba/ abaaaaaa1111 1312. 6114
Alex Krizhevsky, Geoffrey Hinton, et Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pp. 6402-6413,
2017.
Nicolas Le Roux and Yoshua Bengio. Continuous neural networks. In Artificial Intelligence and Statistics, pp- 404-411, 2007.
Yann LeCun. The mnist database of handwritten digits. http://yamn. lecun. com/exdh/nnist// 1998.
Jaehoon Lee, Jascha Sohl-dickstein, Jeffrey Pennington, Roman Novak, Sam Schoenholz, and Yasaman Bahri.
Deep neural networks as gaussian processes. International Conference on Learning Representations, 2018. URLhhttps://ppeneviiie wwtp://rmt/oo/mme//////
http:///pppppeeeeeeeeeee//rrnrn B
1A-M-A----0 0-
Christian Leibig, Vaneeda Allken, Murat Seçkin Ayhan, Philipp Berens, and Siegfried Wahl. Leveraging un-
certainty information from deep neural networks for disease detection. Scientific reports, 7(1):17816, 2017.
Ulrike von Luxburg and Olivier Bousquet. Distance-basee classification with lipschitz functions. Journal of Muchine Learning Reseurch, 5(Ju)):669-60) 69066669999966 2004.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems, pp. 7047-7058, 2018
AGDG Matthews, M Rowland, J Hron, RE Turner, and Z Ghahramani. Gaussian process behaviour in wide deep neural networks. In Proceedings of the 6th International Conference Learning Reprexentations, 2018.
Rhiannon Michelmore, Marta Kwiatkowska, Yarin Gal. Evaluating uncertainty quantification in end-to-end autonomous driving control. arXiv preprint arXiv: 10o.:811.0600 1.06817, 2018.
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
Kevin P Murphy. Machine learning: a probabilliticc perspective. MIT press, 2012.
Eric Nalisnick, Jose Miguel Hermandez- and Padhraic Smyth. Dropout as structured shrinkage prior. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 4712-4722 Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL htt.tttppppprrcceediino ml press /97/
malisnickl html
Radford M Neal. Bayesian leamning for neural networks. Phd Thesis, 1996.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Daniel A. Abolafia. Jeffrey Penning- ton, and Jascha Sohl-dickstein. Bayesian deep convolutional networks with many channels are gaussian pro- In International Conference Learning Representations, 2019. URL htttt:://oppnreeeeeese
/et/forure iiniiidd-ii 1i-Bidl33 1g30 1
K. Osawa, S. Swaroop, A. Jain, R. Eschenhagen, R. E. Turner, Yokota, and M. E. Khan, Practical deep learn- ing with bayesian principles. The 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, Dec. 8- 8-14 2019.
Ian Osband, John Aslanides, and Albin Cassirer. Randomized prior functions for deep reinforcement learning. In Advances in Newral Information Processing Systems, pp. 8617-8629, 2018.
lan Osband, Benjamin Van Roy, Daniel J. Russo, and Zheng Wen. Deep exploration via randomized value func- tions. Journal of Machine Learning Research, 20(124): 1-622 2019. URL w // jmlr org/ papers/
v20/18 8- html
12

---

Published as a conference paper at ICLR 2020
Georg Ostrovski, Mare Bellemare, Aäron den Oord, and Rémi Munos. Count-based exploration with neural density models. Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 2721-2730 JMLR. org, 2017.
Pedregosa, G. Varoquaux, Gramfort, Michel. B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, Cournapcan, M. Brucher, Perrot, and Duches- nay. Scikit-lcam: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Eric P. Xing and Tony Jebara (eds.), Proceddiggs of the 31 Inter- national Conference Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 1278-1286, Bejing, China, 22-24 2014, PMIR. URL URL htt.httt://pooeeeeee ngs /.r.eess//
rezende htm.
Christian Robert. The Bayesian choice: from decisiin--teeerriicc foundttion computaaional implementation. Springer Science & Business Media, 2007.
Donald Rubin. The bayesian bootstrap. The annals of statistics, pp. 130-134, 1981.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929-1958, 2014.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky, Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 9446-9454, 2018.
Max Welling
Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-I1), pp. 681-688, 2011.
Christopher KI Williams. Computing with infinite networks. Advances in neural information processing systems, pp. 295-301, 1997.
Christopher Williams and Carl Edward Rasmussen. Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
Greg Yang. Wide feedforward recurrent neural networks any architecture are gaussian processes. Neural Information Procesing Systems (NeurIPS), 2019.
13

---

Published as a conference paper at ICLR 2020
APPENDICES
APPENDIX A REPRODUCIBILITY AND DETAILS OF EXPERIMENTAL SETUP
APPENDIX A.1 SYNTHETIC DATA
For the ID regression experiment synthetic data (Fig used feed-forward neural networks with 2 layers of 128 units each and 1-dimensional output layer. We used an ensemble size of 5. The network was trained on 20 points sampled from the negative domain of sigmoid function and tested on 20 points sampled from the positive domain.
APPENDIX A.2 EXPERIMENTAL SETUP
Model architecture For the CIFAR-10 experiments, we adapted the setup from the cifarl0-fast model For the network predicting the mean, we used the exact same archi- tecture as in this model. For the prior networks in our uncertainty estimators, the architecture for the prior network was the same as the mean network, but using a final linear layer instead of the softmax layer. We used squared eтror on that last layer to get the uncertainties. For the predictor networks
uncertainty estimators, we added two additional layers at end to make sure prior functions are learnable (see Fig. 2
We followed Burda et al. (2018) in choosing the output size to be optimizer (Kingma & Ba 2014) with learning rate of 0.0001. We optimized the initialization scale of our networks as a hyperparameter on the grid {0.01, 0.1, 1.0, 2.0, 10.0} and chose 2.0. We chose scaling factor of B 1.0 for the uncertainty bonus of the random priors and fixed it for all experiments.
512 and using the Adam
Data For the CIFAR-10 experiment, we trained on the classes (bird, dog. frog. horse} and ex- cluded {cat, deer, airplane, automobile, ship, truck). For the small CIFAR-10 ablation experiment, we trained on 75 images sampled from the classes (ship, truck) } and excluded the remaining classes.
Training Error The training error was 0.57 # 0.20 on the CIFAR experiment and 0.03 + 0.02 on the sub-sampled ablation (the symbol denotes 90% confidence intervals).
oottof-distribution classification For computing the areas under the receiver-operator charac- teristic curves (AUROC) in the OOD classification tables, we used the roc_auc_scoR function from the Python package sklearn (Pedregosa al. [201], using the predicted uncertainties as predicted label scores and binary labels for whether or not the samples were from the training set.
APPENDIX B
ADDITIONAL RESULTS
APPENDIX B.
CONFIDENCE INTERVALS FOR AUROCS
We provide confidence intervals for AUROC measurements in Table
DE
RP
DE
DR
+AT
B=l
Train V. cat/deer
0.99 0.002
0.83 + 0.065
0.96 0.008
0.81 0.001
Train v. vehicles
1.00 + 0.000
0.82 + 0.070
0.96 + 0.007
0.76 + 0.001
Train v. excluded
1.00 - 0.001
0.82 + 0.069
0.96 0.007
0.77 1 0.002
Train SVHN
0.95 - 0.013
0.88 + 0.101
0.96 + 0.009
0.86 + 0.002
Table 3: Out-of distribuutin AUROC for random priors (RP), deep ensembles (DE), deep ensembles
with adversarial training (DE+AT) and spatial concrete dropout (DR). The errors are computed from ten samples each in the B = 1 case. The + symbol denotes one standard error.
14

---

Published as a conference paper at ICLR 2020
APPENDIX B.2 OOD CLASSIFICATION ACCURACIES
In addition AUROC results, also provide accuracy figures on the same OOD tasks. The thresholding for classification was obtained by cross-validation.
They in Table and
DE +AT
RP
DE
DR
B=1
Train cat/deer
0.001
0.83 0.008
0.97 0.006
0.82 0.000
Train v. vehicles
0.99 1 0.001
0.81 1: 0.008
0.96 0.004
0.86 0.000
Train excluded
0.98 1 0.001
0.87 1: 0.022
0.97 1 0.007
0.70 0.002
Train SVHN
0.91 + 0.006
0.91 + 0.025
0.96 + 0.008
0.78 0.001
B=10
Train cat/deer
0.98
0.88
0,96
0.82
Train vehicles
0.99
0.87
0.95
0.86
Train excluded
0.99
0.89
0.96
0.71
Train SVHN
0.92
0.96
0.78
Table 4: Out-of-distribution classification accuracy for random priors (RP), deep ensembles (DE), deep ensembles with adversarial training (DE+AT) and spatial concrete dropout (DR). These values augment the AUROC values reported in Table The + symbol denotes one standard error.
DE +AT
RP
DE
DR
B=1
Train excluded
1.00
0.90
0.88
0.91
Train SVHN
1.00
0.95
0.90
0.97
B=10
Train v. excluded
1.00
0.95
0.89
0.91
Train SVIIN
1.00
0.97
0.95
0.96
Table 5: cattof-ddisributton accuracy for the same models as above (see Tab. 4 on subsampled data. These values augment the AUROC values reported in Table
APPENDIX B.3 SUPERVISED IN-DISTRIBUTION PLASSIFICATION ACCURACIES
DE
RP*
DE
DR
+AT
CIFAR-10 Subsampled CIFAR-10
0.86
0.88
0.86
0.86
0.82
0.81
0.82
0.75
CIFAR-100
0.90
0.91
0.90
0.89
Table 6: In-distribution supervised classification accuracies the respective test sets of the different data sets random priors (RP), deep ensembles (DE), deep ensembles with adversarial training (DE+AT)) and spatial concrete dropout (DR).
*Since random priors do not have an intrinsic supervised prediction model, we used the predictions from the DE+AT' model in all experiments instead, setting B 1.
APPENDIX B.4 CIFAR100 EXPERIMENT
As additional empirical support for our method, we ran experiments on another data set, namely CIFAR- 100 (Krizhevsky et al. 2009). Again, we include 5 classes in the training set and exclude the remaining classes. The results are reported the following (Figs. g Tabs. 7 g). They
qualitatively and quantitatively support the same conclusions as previous experiments.
APPENDIX C BACKGROUND ON BAYES RISK
For completeness, we recall the definition Bayes Risk. We are often interested in minimizing the Mean Squared Error F, [[[(x.) w)2]. where 24 is given test point and w is variable
15

---

Published as a conference paper at ICLR 2020
RP. seen
DE. seen
DC+AT, seen
DR. 525n
0.04
0.04
B-1
0.010
5
5
0.005
10
0.62
0.02
0.01
0.000
0.00
0.00
0.00
0.00
0.25
0.75
1.00
0.00
0.75
1.00
0.50
0.75
1.00
3.04
0.04
0.02
0.010
0.005
10
0.02
10
0.02
10
0.01
10
6.000
0.00
0.00
0.00
1,00
1.00
0.75
Figure 8: Distribution of uncertainty estimates for various algorithms. Top row shows seen data, bottom row shows unseen data from CIFAR-100. For random priors (RP), uncertainties are 8². For other algorithms, they are - mP)) where py is the averaged output of models in ensemble Lakshminarayaaan et al. 2017).
10
10
1.0
1.0
0.9
0.9
0.9
Accuracy
Accuraoy
0.7
0.6
6.6
0.6
DE+AT
DE-AT
DE+AT
DR
DR
DR
0.5
0.5
0.5
0.75
1.25
0.75
1.25
0.75
0.50
points included [x1000]
Number points included [x1000]
points included [x1000]
Figure 9: The relationship between uncertainty (horizontal axis) and accuracy (vertical axis) for B 1, 5, 10 on samples from CIFAR-100, In well-calibrated models, accuracy increases as uncertainty declines.
allowed to adjust. A known result of Bayesian decision theory Robertt 2007 Murphy 2012) is that the minimizer of the MSE is given by the expected value of f, i.e.
arg min E, [[[(x.) w) = E, [f(x))] uc
(12)
Equation 12 holds for any stochastic process f. including when f is posterior process obtained by conditioning on some dataset. consequence of equation [2]s that is impossible to obtain MSE lower than the one obtained by computing the posterior mean of f.
APPENDIX D GAUSSIAN PROCESSES
stochastic process is Gaussian (Williams & Rasmussen 2006), if all its finite-dimensional distri- butions are Gaussian. The main advantage of GPs that the posterior process can be expressed in a tractable way. GPs are often used for regression, where we are learning an unknown function R + from noisy observations. Since a Gaussian distribution completely identified by its first two moments, a GP can be defined by a mean function and a covariance function. Formally, the R. a positive-definite kernel function k R x R - R. GPs can be used to model two kinds of uncertainty: epistemic un- certainty, which reflects lack of knowledge about unobserved values of & and aleatoric uncertainty, which reflects measurement noise. When performing regression, start with zero-mean prior GP( k) and then observe N training points {æt ! and labels y = {yr J= where
notation GP (p. k) refers a GP with with mean function μ : R
We depart from standard notation, which uses f. hecause we will be using f to denote a sample from the prior process.
16

---

Published as a conference paper at ICLR 2020
DE
RP
DE
DR
+AT
Train v. excluded
1,00 + 0.000
0.93 + 0.003
0.98 + 0.001
0.88 = 0.002
Train SVHN
1,00 + 0.000
0.96 + 0.004
0.99 + 0.001
0.82
0.002
B=10
Train V. excluded
1.00
0.96
0.99
0.90
Train SVHN
1.00
0.99
1.00
0.82
Table 7: Out-of-distrbattion classification AUROCs on CIFAR-100 for random priors (RP), deep
ensembles (DE), deep ensembles with adversarial training (DE+AT) and spatial concrete dropout (DR). The + symbol denotes one standard error.
DE +AT
RP
DE
DR
B=1
Train excluded
1.00 + 0.001
0.91 + 0.002
0.97
0.001
0.82
0.003
Train SVHN
0.97 + 0.003
0.95
0.003
0.99
0.001
0.74 0.003
B=10
Train excluded
1.00
0.94
0.98
0.83
Train SVHN
0.98
0.98
0.99
0.74
Table 8: Out-of-distribution classification accuracy CIFAR-100 for random priors (RP), deep ensembles (DE), deep ensembles with adversarial training (DE+AT) and spatial concrete dropout (DR). The + symbol denotes standard error. These values augment the AUROC values reported Table 7
= ((x;) 4. Here, the i.i.d. random variables ~ AO o model the aleatoric noise. We obtain the posterior process on (P(xx kx). For GPs, the mean and covariance of the posterior GP on y evaluated at can be expressed as
(x,( = k (K + +0 and (x.) (x.. + 0 k.r - K_ (K + - +
(13)
(14)
particular, the posterior covariance does not depend on y. In the formula above, we use the kernel matrix K E R- x IRN defined Kij KT)) where x; and I, are in the training set. We also use the notation k, E N for the vector of train-test correlations {k, 1 k(x,, where is in the training set and k(x*,x") is similarly defined. The shorthand o (z.) introduced in equation 14 denotes the posterior variance at single point.
APPENDIX E LIST OF SYMBOLS DENOTING VARIANCE
Below, we give list of symbols used for variance of various random variables.
Xf posterior variance of stochastic process
posterior variance of Gaussian process prior variance of stochastic process sample-based estimate of prior GP variance
g²
combined uncertainty estimate (see equation
82
cample-based mean part of uncertainty estimate (see equation 2
E, [d2]
0
sample- based variance part of uncertainty estimate (see equation
UUB
upper bound on variance 6²
o²
aleatoric variance (observation noise)
APPENDIX F PROOFS
We now give formal proofs for the results in the paper.
17

---

Published as a conference paper at ICLR 2020
APPENDIX F.1 PROOFS RELATING TO CONSERVATISM
Lemma For any function (N((())) RR,, for any test point I, E RKK and for any stochastic process ((( with all second moments finite and exchangeable outputs
(x,) EB(x) (x.)( + (xxx((())
(6)
Proof. We prove the statement by re-writing the expression on the left.
(x,)) = t E(x)),(2.) [11 -
(15)
- M E((x) E(G.))(())
[I] - hx (x,((=)))
(16)
= 1
M Fx(x)
E((o.1((0)
12 -
(17)
के E((x
B((..IS(X)
MM Em (xr)) 21 (xx 3 (z.) + (x., (x,))
(18)
EM pm (x:. + (xx))) 2p mm (z. / (z.) (h (z.)))
MM Ex(x
(19)
E pm (x.) (uxxm (z.) - h (2.)))
Mi Ex(x)
(20)
Ex(x)) (xx) + -xxx)(t))hxxxx_..1
(21)
Here, the equality in (T)) holds by definition of conditional probability. The equality in 19 holds by definition of posterior mean and the equality 21 follows by assumption that the process has exchangeable outputs. While this argument follows a similar pattern standard result about Bayesian Risk (see Appendix Appendix C). it is not identical because the function hxy depends on f.
Proposition 1 (Strict Conservatism Expectation). Assume that f is a GP. Then for any function h: N R we have
(1(x))) = (xx) (I.) Es(x) [t (lux((a.) hxf(x=))11
20
Moreover, equality holds if and only if (xx))x))) pxf(x.).
Proof. We instantiate Lemma [ by setting í to be a GP. By equation 14, the posterior covariance of GP does not depend the target values, i.e. (xx) = ox (z.). The first part of the result can be shown by pulling (z.) out of the expectation. Moreover, since || is a norm and hence positive semi-definite, equality holds if and only if hxz(xx)) = (x)(x.).
Lemma Assume that the random variable (z.) has finite variance upper bounded by UpB- With probability 1 5, we have (x)) + 58 2 (2.).
Proof. The proof is standard, but we state it in our notation for completeness. Applying Chebyshev's inequality to the random variable o& (x..), we have that Prob (z.) - -6 oz.x1) 2 "UB implying the statement.
S 6.
Corollary (Strict Conservatism for Finite Bootstraps). Assume that f is a GP. Assume that the random variable 2 (Ix) has finite variance upper bounded by Ven- Then with probability 1 for any function h IR N R,M, we have
(z.))) + UUB 2 (z.x)) 2 (x.).
(10)
Proof. Combine Lemma 3 and Proposition
18

---

Published as a conference paper at ICLR 2020
Lemma 2. Assume that the GP {f(z)} is mean with exchangeable outputs and the function hxy takes values in [-U, UTM, Assume that permuting the outputs of produces the same permu- tation in the outputs hx With probability 6, we have
fa.. [ (c.)] S vUB.
(11)
where vuB expressible in terms of observable quantities.
Proof. We seek to decompose the variance of o (1xx) into the part that comes from the prior and the part that comes from the fitted function fm fm.
ya.......JJa
[ (x.)]
(22)
/ar......ss /8
E MB (zx)) - (x.)))
(23)
Vary LT (x..) (1+)))
(24)
B ME Vary
(f"' h (z.))²
(25)
B MJ Em Eis Covy [1f (ffT( (x.)
(f((z) (x,)
(26)
S MT M Vary (xx) - 1x (x=))
(27)
B Var, [(f"(x)) - hx pm (2x))²]
(28)
I BB Ey (x." h (x.))
(29)
B (E, [(("(..) - F., (x fm (xx)) + F., ((T"()))
4 Ey (f.) (xx) (1= + E, (m (xx))"))
(30)
Here, line 27 holds by exchangeability of outputs and the Cauchy- Sewaaaiiiequualityy
Since f (x.,) is has support in [-U,U], we have
Ey (x.)) s U2 E, ((xx)=()))))) U Ey ((xx)=(xx)))) S UO.
(31)
Moreover, since (xx) Gaussian and zero mean, we can write out the moments explicitly.
FG ((*"(( 3(E,, (x))))
Ey ((P"((z)) = 15(E, (x.))=)))
(32)
Since f(x.) is Gaussian, probabilistic confidence interval. In particular, we know that E, [(FT(xx)))²) S (z.)) Ba-1 with
sample-based estimate of the prior variance and obtain an
x
probability - 5, where x denotes the inverse CDF of the Chi-Squared distribution with Bg Bp-1
degrees of freedom. We denote this upper bound with wUB = = ()
We proceed by bounding the individual terms in equation 30 separattlly E, ((TT(x.)) 3(E, ((*"(x))))1)
- Ey (I." (/xx) S v Ey (I. Ey ((.))((xx)))
Ey [1TT (1 (1. {) < E, [1
(T (x.) [(1xpm[[[pm) (xx))
Fly [Smh (.. (hx (x.)) v F. (x))())))) By pm (1.)))
Combining the above, equation we obtain
30 and the bounds on individual moments in equations 31 and
32
Var (x.)) < B ( + 115w U + 6v 6/wuin + 4 + U3)
(33)
Vhn
Here, WUB (x.) Ba_1 (xx) sasple-based estimate of the prior variance obtained with Bo samples, where x2 denotes the inverse CDF of the Chi-Squared distribution with Bo 1 degrees of freedom.
B ())
19

---

Published as a conference paper at ICLR 2020
APPENDIX F.2 PROOFS RELATING TO CONCENTRATION
We now proceed to the proofs showing concentration. We begin by formally defining class of predictor networks.
Definition (Class Hu of Lipschitz networks). Consider functions h RK + RM, Let j.j -,...,, index the outpuls of the function. We define Hy so that each h Hu has the following properties for each j.j'. (P1) h is Lipschitz continuous with constant L. i.e. [1))(x) -hy h2(2²))lz S L|| z'1l2 for all x, x' with || S 1 and || S L, (P2) outputs are exchangeable, iee. (hg h E Hv) = (h : h Hu), (P3) the class is symmetric around zero, i.e. hy {h3 ma (x) U.
E Hob implies hg E {hg : h E Hv). (P4) h is bounded, i.e.
While the conditions in Definition D look complicated, they are in fact easy to check for predictor networks that follow the architecture Figure particular, Lipschitz continuity (PT) (P hold in practice because its absence would indicate extreme sensitivity to input perturbations. Output exchangeability (P2) holds since reordering the outputs does not change our architecture. Symmetry around zero (P3) holds by flipping the sign in the last network layer. Boundedness (P4) is easy to ensure by clipping outputs. In the following Lemma, we obtain bound on the expected uncertainty.
Lemma 4. Consider a target function f RKK + RMM, where j = 1,...,A. M, with the domain restricted to || I 1. Introduce constant U such thut max] S1 ((ما S U. Denote the data distribution with support on {x : || S 1] as D. Moreover, assume K 2 3. For lxg E Hy, with probability 8 we have
Em.-Dl 1 (xx)) hx MN
EE (xx()))) LU 0
(34)
Proof. The proof uses standard Rademacher tools. To avoid confusion across several conventions, we explicitly define the Rademacher complexity of function class G as:
Rx(C)) A Fm; suPgeg N Ei man(ms)) = Er,, (sup,eg N E (
(35)
Here, the random variables u, are sampled i.i.d. using discrete distribution with Prob(u; (u;
-1) = Prob(u; , 1) = = and the the second equality follows by using property (P3) We
start by applying the generie Rademacher bound (Mohri et al., 2018) to the function class M
{z1...., t1 .N tN + d to l|tz - h(x:)11 21? h Hy j. which contains the possible errors of the predictor.
1 1 (f(x)) (x,)11) Be
S
1 EN llf(x.) + +0 MN MN
(
/loR(//) N
(36)
B2
We now introduce the function class M' (5i, JIN, which models the per-output squared error. Because of property (P2) output index j. By pulling out the sum outside the supremum equation 35 we get
Ez (1, - (x:)) E Hu
M does not depend on the
RM s (MM')
(37)
by Talagrand's 's Lemma (Mohri et al.
2018 Duchi 2009).
we also have
(MM') (711.
(38)
Here, H4 = 12,12 :
h hhe Ho). By property (P1) functions in Hi are Lipschitz continuous
with constant L/U. Instantiating a known bound for Lipschitz-continuuus functions (Luxburg &
Bousquet 2004, Theorem 18 and Example 4), and using the assumption K > 3. we get NyH,) <
f
0
VV The Lemma follows by combining this with equation 37 and equation 38 plugging
into equation 36 and re-scaling by U2
20

---

Published as a conference paper at ICLR 2020
Lemma 4 allowed to relate the enror the training set to the expected error the test set. It also shows that the two will be closer for small values of the Lipschitz constant L. We now use this Lemma to show our main concentration result (Proposition 2
Proposition 2. If the training converges, i.e. the training loss MN E (xxx))) d' for arbitrarity large training sets, then assuming the predictors o bounded and Lipschitz continuous with constant L, then under technical conditions the uncertainties concentrate, i.e. (2.))) a N
and B
with probability
Proof. We are assuming the technical conditions of Lemma training loss to d in the RHS of equation 34 and letting N probability 1:
Instantiating Lemma 4 setting the oo, we obtain the following with
lim Ex, (2., = = N-
(39)
This implies:
lim (e,=0(max((0@pp - o'))l = 0.
(40)
N N
From the continuity of and hxy have that 6ī is continuous in x.. Together with the property that the expression under the expectation is non-negative, this gives that for every I..
lim max(0, 6² (x.) σ²) = O. N
(41)
Since the right-hand side does not depend on B, we also have
(x.) lim lim max(0, (x. - σ = O.
(42)
8 N +00
From the definition of foo we have that
lim 0g = 0. B-000 N N-+00
lim
(43)
We show the Lemma by combining equation and equation 43 with equation
21


Reviews:
[{"review_id": "HkgK3JURYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper shows that the MSE of a deep network trained to match fixed random network is a conservative estimate of uncertainty in expectation over many such network pairs. An experiment compares this previously proposed method to other approaches for uncertainty estimation on CIFAR 10. \n\nStrengths:\n- Obtaining uncertainty estimates for predictions of deep neural networks is an important and open research question.\n- Proposition 1 is an interesting result, although the paper does not seem to discuss its significance and implications enough.\n\nWeaknesses:\n- Proposition 1 is described as \"our uncertainty estimates are never too small\". However, as the name of the proposition suggests, it seems to only hold in expectation over models trained, which is a quite different statement.\n- Proposition 2 seems to simply state that a small network can be distilled into a large network. Maybe I missed part of the reasoning here? Otherwise, it should probably not be highlighted as a major contribution.\n- The experimental evaluation is very limited, training only on CIFAR 10. While the experiments add little value to the community, this may be acceptable for a mostly theoretical paper.\n\nClarity:\n- The paper was difficult to read and unclear in explanations. It could help to define notation upfront instead of introducing shorthands on the way.\n- In Figure 3, the Y axis limits should be fixed across seen and unseen histograms for the same method. The current presentation is a bit misleading here, as the presented method seems to have moved the most mass under this chart scaling.\n\nComments:\n- As found in prior work cited in the submission, the method tends to perform well with just one network pair. This raises the question whether the contribution of the paper that holds in expectation over many pairs and the empirical success of the approach are connected.\n- The marginal posterior variance \\sigma^2(x_\\star) appears in various forms with hat, tilde, and different subscripts. It maybe worth assigning different letters to these to avoid confusion.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJgnXst2tr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work introduces a simple technique to obtain uncertainty estimates for deep neural networks. This is achieved by having a set of random networks (i.e. neural networks where their parameters are randomly initialized) and then computing an uncertainty value based on the difference in the predictions between those random networks and networks that are trained to mimic them on a finite collection of points. The authors further show that this method results into uncertainties that are conservative, meaning that they are higher than the uncertainty of a hypothetical posterior, and concentrate, i.e. they converge towards zero when we get more and more data. The authors further draw connections to ensemble methods and discuss how such a method can be effectively realized in practice. They then evaluate their approach on an out-of-distribution detection task, they measure the calibration of their uncertainty estimates and finally perform a small ablation study for their concentration result. \n\nThis work is interesting as it seems to provide a simple way to obtain reasonable uncertainty estimates. For this reason it can potentially serve as a strong baseline for this field. The theoretical considerations also help in providing some guarantees about such an approach. Having said that, in my opinion the writing could use some more work in order to make things more clear as some critical experimental details and baselines are missing and thus do not make the method as convincing. Furthermore, I also believe that some clarifications on the theoretical aspects of this work, will help in boosting its quality. More specifically:\n\n- How exactly do you apply your method on the classification scenario? Do you select an arbitrary hidden layer of the classification model for the prior and predictor network architectures or the output logits / softmax probabilities?  In appendix A you mention the architecture but not precisely how it is employed. I believe this can be an important piece of information in order to decipher the importance of e.g. the output dimensionality on the uncertainty quality, as higher dimensional outputs might be harder to approximate thus could induce a larger squared error and hence uncertainty.\n- What is the average training error of the predictor networks for the out-of-distribution task and subsampling ablation task, i.e. how far away from concentration were the priors? \n- An effect that I found weird is the following: what happens for the out-of-distribution examples when the predictor networks can perfectly predict the prior network outputs? Wouldn’t that then imply that the uncertainty would be zero for any input (even an out-of-distribution one), as the prior network and predictor network always agree? One could imagine that for e.g. simple priors and with sufficiently dense sampling of the domain of the function this can happen in practice.\n- For the conservatism you show that your uncertainty estimate is higher, on average, than the posterior variance when you sample points from the model itself. In a sense this guarantee translates to the actual data when the prior is “correct”. How do those conservatism guarantees translate to the case when there is model misspecification, i.e. when the prior is not correct? Perhaps a small toy example would be informative.\n- For the predictor networks as described in figure 2; do you train both the green and red parts of the network or only the red parts and keep the green part fixed to the values you used for the prior f? (This helps in understanding how easy / difficult is the task of the predictor network).\n- What is the accuracy on the actual in-distribution prediction task for the RP and baselines? What did “B” correspond to for the dropout networks? Was it the number of dropout samples you averaged over to get the final predictive?\n- How sensitive are the results on the actual initialization strategy of the prior network? It would be good to see e.g. some form of performance / init variance curve in order to decipher the sensitivity. \n\nOther comments\n- It is worth pointing out that [1] showed that Monte-Carlo dropout performs approximate MAP inference, which seems more plausible than the approximate Bayesian inference perspective of [2].\n- In the introduction you argue that Bayesian neural networks rely on procedures different from standard supervised learning and thus most ML pipelines are not optimized for them in practice. Could you elaborate a bit about this statement? Variationally trained BNNs with e.g. the reparametrization trick [3, 4] are straightforward since you can just use backpropagation to update their (variational) parameters.\n- What is the x-axis for Figure 3 for the baselines? (I take it that for RP it is the \\hat{sigma}^2(x)).\n- I believe that a comparison against a simple variationally trained BNN would make the results more convincing.\n\nMisc\n- Second page, “Figure 1, top two plota” -> “Figure 1, top two plots”\n- Third page, “[…] introduced in equation 2 denotes the posterior covariance [….]” -> “[…] introduced in equation 2 denotes the posterior variance […]“\n- Fifth page, “[…] this makes it is reasonable for W large enough […]” -> “[…] this makes it reasonable for W large enough […]”\n- Sixth page, “Corollary 1 and proposition 2”; where is corollary 1? Do you mean Proposition 1?\n- Seventh page, “[…] inspired by, an builds on, […]” -> “inspired by, and builds on, […]”\n- Ninth page “montonicity” -> “monotonicity”\n\nOverall, I tend to accept this work, although, depending on the author rebuttal and other discussions, I am willing to change my rating accordingly.\n\n[1] Eric Nalisnick, José Miguel Hernández-Lobato, Padhraic Smyth, Dropout as a Structured Shrinkage Prior, 2019\n[2] Yarin Gal, Zoubin Ghahramani, Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning, 2016\n[3] Diederik P. Kingma, Max Welling, Auto-Encoding Variational Bayes, 2014\n[4] Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra, Stochastic Backpropagation and Approximate Inference in Deep Generative Models, 2014", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "B1xA7Po2tS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overview:\nThis paper introduces a new method for uncertainty estimation which utilizes randomly initialized networks. Essentially, instead of training a single predictor that outputs means and uncertainty estimates together, authors propose to have two separate models: one that outputs means, and one that outputs uncertainties. The later one consists of two networks: a randomly initialized “prior” which is fixed and is not trained, and a “predictor”, which is then trained to predict the output of the randomly initialized “prior” applied to the training samples.  \nAuthors show that under some reasonable assumptions the resulting estimates are conservative and concentrated (i.e. bounded and converge to zero with more data).\n\nWriting quality: \nOverall, the paper is relatively well-written, although it might be at times hard to follow, especially for someone who is not familiar with the original work that used randomized prior functions (Burda’18, Osband ‘18, ‘19). \n\nEvaluation:\nThe method is experimentally evaluated on a task of out-of-distribution detection on CIFAR+SVHN, and seems to perform on-par or better than the baselines (including “standard” deep ensembles and dropout networks). In addition, there are experiments that demonstrate that the model is performing relatively well in terms of calibration (whether the model predictive behaviour makes sense as the model confidence changes).\n\nDecision:\nI find the core idea behind the paper quite interesting, however, as indicated by authors themselves, it has already been studied in a slightly different context (RL, works by Burda et. al, Osband et. al). That said, authors do provide additional insides for the supervised settings, and also analyse theoretically the behaviour of uncertainty estimates. \nOverall, I cannot say I am fully convinced that the paper should be accepted as is (also see questions below), but generally I am positive about this work, and hence the final score: “weak accept”.\n\nAdditional comments / questions:\n(somewhat minor) p1: “While deep ensembles …, where the individual ensembles are trained on different data“ - here and related text, it should probably be “individual models” / “individual networks”. Generally, I am not convinced that these are strong arguments against deep ensembles.\n\n(minor) p2-p3: “2. Preliminaries” - I am not sure if this section adds much to the understanding, it would seem more natural to spend more time explaining the intuitions behind the net\n\n(kind of major) p3. “prior” - The explanation of why using a randomly initialized network makes sense is not very strict. I kind of get the general idea, but it is not clear to me why not use something less expensive, e.g. just random projections, and why do we actually need a full network. Intuitively it seems quite strange to waste a lot of capacity to fit to essentially fit a set of random weights: is it something that allows the network to avoid easily learning the “random prior”? And, more generally, can this also be considered as a “trick” to de-correlate individual predictors? I believe these points should be discussed in more detail.\n\n<update>\nI would like to thank authors for verbose response and the revised version: it is a bit more clear. \nI stand by my original rating.\n</update>\n\n\n\n\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper provides theoretical justification for a previously proposed method for uncertainty estimation based on sampling from a prior distribution (Osband et al., Burda et al.).

The reviewers initially raised concerns about significance, clarity and experimental evaluation, but the author rebuttal addressed most of these concerns.

In the end, all the reviewers agreed that the paper deserves to be accepted.
Decision: Accept (Poster)