C ONSERVATIVE U NCERTAINTY E STIMATION B Y F ITTING P RIOR N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Obtaining high-quality uncertainty estimates is essential for many applications of deep neural networks.
In this paper, we theoretically justify a scheme for estimating uncertainties, based on sampling from a prior distribution.
Crucially, the uncertainty estimates are shown to be conservative in the sense that they never underestimate a posterior uncertainty obtained by a hypothetical Bayesian algorithm.
We also show concentration, implying that the uncertainty estimates converge to zero as we get more data.
Uncertainty estimates obtained from random priors can be adapted to any deep network architecture and trained using standard supervised learning pipelines.
We provide experimental evaluation of random priors on calibration and out-of-distribution detection on typical computer vision tasks, demonstrating that they outperform deep ensembles in practice.
I NTRODUCTION
Deep learning has achieved huge success in many applications.
In particular, increasingly often, it is used as a component in decision-making systems.
In order to have conﬁdence in decisions made by such systems, it is necessary to obtain good uncertainty estimates, which quantify how certain the network is about a given output.
In particular, if the cost of failure is large, for example where the automated system has the capability to accidentally hurt humans, the availability and quality of uncertainty estimates can determine whether the system is safe to deploy at all (Carvalho, 2016;
Leibig et al., 2017; Michelmore et al., 2018).
Moreover, when decisions are made sequentially, good uncertainty estimates are crucial for achieving good performance quickly (Bellemare et al., 2016;
Houthooft et al., 2016; Ostrovski et al., 2017; Burda et al., 2018).
Because any non-Bayesian inference process is potentially sub-optimal (De Finetti, 1937), these uncertainty estimates should ideally be relatable to Bayesian inference with a useful prior.
Deep ensembles (Lakshminarayanan et al., 2017), one of the most popular methods available for uncertainty estimation in deep networks today, struggle with this requirement.
While deep ensembles can be related (Rubin, 1981) to Bayesian inference in settings where the individual models are trained on subsets of the data, this is not how they are used in practice.
In order to improve data efﬁciency, all ensembles are typically trained using the same data (Lakshminarayanan et al., 2017), resulting in a method which does not have a theoretical justiﬁcation.
Moreover, deep ensembles can give overconﬁdent uncertainty estimates in practice.
On the other hand, Monte-Carlo dropout can be viewed (Gal & Ghahramani, 2016) as a certain form of Bayesian inference. However, doing so requires requires either a limit to be taken or a generalization of variational inference to a quasi-KL divergence (Hron et al., 2018).
In practice, MC dropout can give arbitrarily overconﬁdent estimates (Foong et al., 2019).
More broadly, a category of approaches, known as Bayesian Neural Networks (Blundell et al., 2015; Welling & Teh, 2011; Neal, 1996), maintains a distribution over the weights of the neural network.
These methods have a sound Bayesian justiﬁcation, but training them is both difﬁcult and carries an accuracy penalty, particularly for networks with convolutional architectures (Osawa et al., 2019).
Moreover, tuning BNNs is hard and achieving a good approximation to the posterior is difﬁcult (Brosse et al., 2018). We use another way of obtaining uncertainties for deep networks, based on ﬁtting random priors (Osband et al., 2018; 2019). Random priors are easy to train and were found to work very well in practice (Burda et al., 2018). To obtain the uncertainty estimates, we ﬁrst train a predictor network to ﬁt a prior.
Two examples of prior-predictor pairs are shown in the top two plots of Figure 1.
Prior Predictor
Training data     0.
Prior Predictor
Training data
Ground truth Prediction
Training data them randomly initialized (and therefore untrained) neural networks
While way of estimating uncertainties was known before (Osband et al., 2019) it did not have a theoretical justiﬁcation beyond Bayesian linear regression, which is too limiting for modern applications.
Contributions We provide a sound theoretical framework for obtaining uncertainty estimates by ﬁtting random priors, a method previously lacking a principled justiﬁcation. Speciﬁcally, we justify estimates in the uncertainty of the output of neural networks with any architecture .
In particular, we show in Lemma 1 and
Proposition 1 that these uncertainty estimates are conservative , meaning they are never more certain than a
Bayesian algorithm would be.
Moreover, in Proposition 2 we show concentration, i.e. that the uncertainties become zero with inﬁnite data.
Empirically, we evaluate the calibration and out-of-distribution performance of our uncertainty estimates on typical computer vision tasks, showing a practical beneﬁt over deep ensembles and MC dropout.
P RELIMINARIES
We are going to reason about uncertainty within the formal framework of stochastic processes.
We now introduce the required notations.
A stochastic process is a collection of random variables { f ( x ) } .
We consider processes where x ∈ R K and the random-variable f ( x ) takes values in R M .
A stochastic process has exchangeable outputs if the distribution does not change when permuting the M entries in the output vector.
Allowing a slight abuse of notation, we denote the ﬁnite-dimensional distribution of the process { f ( x ) } for the set X
= { x i } i =1 ,...,N as f ( x 1 , . . .
, x N ) =
f ( X ) .
In practice, the ﬁnite-dimensional distribution reﬂects the idea of restricting the process to points x 1 , . . .
, x N and marginalizing over all the other points.
Inference can be performed on stochastic processes similarly to probability distributions.
In particular, we can start with some prior process { f ( x ) } , observe a set of N training points X
= { x i } i =1 ,...,N and labels y =
{ y i } i =1 ,...,N and then consider the posterior process { f Xy ( x ) } , whose ﬁnite-dimensional distributions are given by f Xy ( x (cid:63) 1 . . .
x (cid:63)N (cid:48) )
=
f ( x (cid:63) 1 . . .
x (cid:63)N (cid:48) | x 1 , . .
, x N , y 1 , . . .
, y N ) for any set of testing points x (cid:63) 1 . . .
x (cid:63)N (cid:48) .
We use subscripts to denote conditioning on the dataset throughout the paper.
We denote the variance of f Xy ( x (cid:63) ) with σ 2 Xf .
A stochastic process is called Gaussian if if all its ﬁnite-dimensional distributions are Gaussian.
Given a test point x (cid:63) , we denote the posterior GP mean with µ Xy ( x (cid:63) ) and posterior GP variance with σ 2 X ( x (cid:63) ) .
We provide more background on GPs in Appendix D.
E STIMATING U NCERTAINTY FROM R ANDOM P RIORS
Intuition Uncertainties obtained from random priors have an appealing intuitive justiﬁcation.
Consider the networks in the top part of Figure 1.
We start with a randomly initialized prior network, shown in red.
Whenever we see a datapoint, we train the predictor network (green) to match this prior.
Uncertainties can then be obtained by considering the squared error between the prior and the predictor at a given point.
An example uncertainty estimate is shown as the shaded blue area in the bottom of Figure 1.
While it may at ﬁrst seem that the squared error is a poor measure of uncertainty because it can become very small by random chance, we formally show in Section 4. that this is very improbable.
In Section 4.2, we show that this error goes down to zero as we observe more data.
Similarly to GP inference, uncertainty estimation in our framework does not depend on the regression label.
The prediction mean (blue curve in the bottom part of Figure 1) is obtained by ﬁtting a completely separate neural network.
In section 6, we discuss how this framework avoids the overconﬁdence characteristic of deep ensembles (Lakshminarayanan et al., 2017).
Prior The process of obtaining network uncertainties involves randomly initialized prior networks, which are never trained .
While this may at ﬁrst appear very different from they way deep learning is normally done, these random networks are a crucial component of our method.
We show in Section 4.1 that the random process that corresponds to initializing these networks can be interpreted as a prior of a Bayesian inference procedure.
A prior conveys the information about how the individual data points are related.
The fact that we are using random networks has both practical and theoretical beneﬁts. Practically, since the prior does not depend on the data, there is no way that it can overﬁt. The use of random priors also has strong empirical support – randomly initialized networks have been recently used as priors to obtain state-of-the-art performance on computer vision tasks (Ulyanov et al., 2018; Cheng et al., 2019). Theoretically, using random priors satisﬁes the likelihood principle (Robert, 2007).
Moreover, random priors can be viewed as a safe choice since they make the minimum reasonable assumption that the network architecture is appropriate for the task.
In fact, whenever deep learning is used, with or without uncertainty estimates, practitioners are already implicitly making that assumption. Algorithm function T RAIN -U NCERTAINTIES ( X ) for i = 1 . . . B do f i ∼ { f ( x ) } (cid:46) random prior h
Xf i ← F IT ( X, f i ( X ) ) end return
i h Xf i end function function F IT ( X, f i ( X ) ) L ( h ) (cid:44) (cid:80) x ∈ X (cid:107) f i ( x ) − h ( x ) (cid:107)h Xf
i ← OPTIMIZE ( L ) (cid:46) SGD or similar return
Xf i return trained predictor end function
Algorithm The process of training the predictor networks is shown in Algorithm 1.
The function
T RAIN -U NCERTAINTIES ﬁrst generates random priors, i.e. neural networks with random weights.
In our notation, it corresponds to sampling functions from the prior process { f ( x ) } .
These priors, evaluated at points from the dataset X
= { x i } i =1 ,...,N are then used as labels for supervised learning, performed by the function F IT .
After training, when we want to obtain an uncertainty estimate on φ at a given test point x (cid:63) , we use the formula
[Equation]
Here, the quantity ˆ σ 2 µ is the sample mean of the squared error.
We will show in Section 4 that it is an unbiased estimator of a variable that models the uncertainty.
On the other hand, ˆ v σ is the samplebased estimate of the standard deviation of squared error across bootstraps, needed to quantify our uncertainty about what the uncertainty is .
The hyper-parameter β controls the degree to which this uncertainty is taken into account.
Formally, the quantities are deﬁned as
[Equation]
In the above equations, B is the number of prior functions and each prior and predictor network has
M outputs.
Because the predictors are trained independently, uncertainty estimates obtained from each of the B predictor-prior pairs are independent.
We defer the discussion of details of network architecture to Section 5.
Our experiments (Section 7) show that it is often sufﬁcient to use B =in practice.
T HEORETICAL R ESULTS
In Section 3, we introduced a process for obtaining uncertainties in deep learning.
We now seek to provide a formal justiﬁcation.
We begin by setting the notation.
We deﬁne the expected uncertainties as
[Equation]
[Equation] In other words, ˜ σ 2 µ is the expected version of the sample-based uncertainties ˆ σ 2 µ ( x (cid:63) ) introduced in equation 2.
Since Bayesian inference is known to be optimal (De Finetti, 1937; Jaynes, 2003;
Robert, 2007), the most appealing way of justifying uncertainty estimates ˜ σ 2 µ and ˆ σ 2 µ is to relate them to a Bayesian posterior σ 2 Xf ( x (cid:63) ) .
We do this in two stages.
First, in Section 4.1, we prove that the obtained uncertainties are larger than ones arrived at by Bayesian inference.
This means that our uncertainties are conservative , ensuring that our algorithm is never more certain than it should be.
Next, in Section 4.2, we show that uncertainties concentrate, i.e., they become small as we get more and more data.
These two properties are sufﬁcient to justify the use of our uncertainties in many applications.
4. U NCERTAINTIES FROM R ANDOM P RIORS ARE C ONSERVATIVE
From the point of view of safety, it is preferable to overestimate the ground truth uncertainty than to underestimate it.
We now show that this property holds for uncertainties obtained from random priors.
First, we justify conservatism for the expected uncertainty ˜ σ 2 µ deﬁned in equation [-
-]4 and then for the sampled uncertainty ˆ σ 2 µ deﬁned in equation 2.
Amortized Conservatism We ﬁrst consider a weak form of this conservatism, which we call amortized.
It guarantees that ˜ σ 2 µ is never smaller than the average posterior uncertainty across labels sampled from the prior.
Formally, amortized conservatism holds if for any test point x (cid:63) we have ˜ σ
[Equation]
Here σ 2 Xf ( x (cid:63) ) corresponds to the second moment of the posterior process { f Xf ( x ) } .
We will introduce a stronger version of conservatism, which does not have an expectation on the right-hand side, later in this section (eq. 8).
For now, we concentrate on amortized conservatism. In Lemma(proof in appendix), we show that it holds under very general conditions.
Lemma 1.
For any function h : R N ×
( K +1)
→ R M , for any test point x (cid:63) ∈ R K and for any stochastic process { f ( x ) } [Equation] ∈ R K with all second moments ﬁnite and exchangeable outputs [Equation]
[Equation]
Relation to a GP Lemma 1 holds for any prior process { f ( x ) } .
However, the prior process used by Algorithm 1 is not completely arbitrary.
The fact that prior samples are obtained by initializing neural networks with independently sampled weights gives us additional structure.
In fact, it can be shown that randomly initialized neural networks become close to GPs as the width of the layers increases.
While the original result due to Neal (1996) held for a simple network with one hidden layer, it has been extended to a wide class of popular architectures, including to CNNs and RNNs of arbitrary depth (Matthews et al., 2018;
Lee et al., 2018; Novak et al., 2019; Williams, 1997; Le Roux & Bengio, 2007; Hazan & Jaakkola, 2015; Daniely et al., 2016; Garriga-Alonso et al., 2019).
Recently, it has been shown to hold for a broad class of functions trainable by gradient descent (Yang, 2019).
While the precise statement of these results involves technicalities which fall beyond the scope of this paper, we recall the key insight.
For a family of neural networks { f W ( x ) } , where the weights are sampled independently and W is the width of the hidden layers, there exist a limiting kernel function k ∞ such that
[Equation]
In other words, as the size of the hidden layers increases, the stochastic process obtained by initializing networks randomly converges in distribution to a GP.
In the context of our uncertainty estimates, this makes it reasonable for W large enough to consider the prior to be a GP.
We stress that the
GP assumption has to hold only for the prior network , which is never trained.
We do not make any assumptions about connections between the predictor training process and GPs.
Strict Conservatism Denoting the posterior GP variance with σ 2 X ( x (cid:63) ) , we deﬁne uncertainty estimates to be strictly conservative when
[Equation]
This statement is stronger than the amortized conservatism in equation 5.
Intuitively, equationcan be interpreted as saying that our uncertainty estimates are never too small.
This conﬁrms the intuition expressed by Burda et al.
(2018) that random priors do not overﬁt. Below, we outline how to guarantee strict conservatism formally. This allows us to state Proposition 1, which guarantees strict conservatism. It is proved in the appendix.
Proposition 1 (Strict Conservatism in Expectation) .
Assume that f is a GP.
Then for any function h : R N × K → R M , we have
[Equation]
Moreover, equality holds if and only if h Xf ( x (cid:63) )
= µ Xf ( x (cid:63)
[Equation]
Conservatism with Finite Bootstraps Lemma 1 above shows conservatism for expected uncertainties, i.e. ˜ σ 2 µ introduced in equation 5.
However, in practice we have to estimate this expectation using a ﬁnite number of bootstraps, and use the sampled uncertainties ˆ σ 2 µ deﬁned in equation 2.
We now state a conservatism guarantee that holds even in the case of just one bootstrap ( B = 1 ). The proof is deferred to the Appendix.
Corollary 1.
Assume that f is a GP. Assume that the random variable ˆ σ 2 µ ( x [Equation] ) has ﬁnite variance upper bounded by v UB .
Then with probability 1 − δ , for any function h : R N × K → R M , we have
[Equation]
However, applying the Corollary requires the knowledge of v UB .
We now provide an upper bound.
Lemma 2.
Assume that the GP { f ( x ) } is zero mean with exchangeable outputs and the function
h
Xf takes values in [ − U, U ] M .
Assume that permuting the outputs of f produces the same permutation in the outputs of h Xf .
With probability 1 − δ , we have
[Equation]
[Equation] Here, ˆ σ 20 ( x (cid:63) ) is a sample-based estimate of the prior variance obtained with B 0 samples and C = (cid:16)
B 0 − 1 χ 2 I ( δ ) (cid:17) 2 , where χ 2 I denotes the inverse CDF of the Chi-Squared distribution with B 0 − 1 degrees of freedom.
In cases where conservatism is desired, but not absolutely essential, we can avoid the torturous calculation of Lemma 2 and replace v UB with the sample-based estimate ˆ v σ ( x (cid:63) ) , deﬁned in equation 2.
In this case, the conservatism guarantee is only approximate.
We have done this to obtain equation 1, used by the algorithm in practice.
4. U NCERTAINTIES FROM R ANDOM P RIORS C ONCENTRATE
While the conservatism property in Proposition 1 is appealing, it is not sufﬁcient on its own for the uncertainty estimates to be useful. We also need concentration, i.e. a guarantee that the uncertainties ˆ σ 2 become small with more data.
We can gurantee this formally by assuming that the class of neural networks being ﬁtted is Lipschitz-continuous and bounded.
Intuitively, By assumption of Lipschitz continuity, the predictors h
Xf cannot behave very differently on points from the training and test sets, since both come from the same data distribution.
We can then show concentration by using standard Rademacher tools to obtain a bound on the expected uncertainty in terms of the squared error on the training set.
This process is formalized in Proposition 2.
Proposition 2.
If the training converges, i.e. the training loss 1 MN (cid:80) Ni =1 (cid:107) f
( x i ) − h
Xf ( x i ) (cid:107) 2 =
σ 2 A for arbitrarily large training sets, then assuming the predictors h
Xf are bounded and Lipschitz continuous with constant L , then under technical conditions the uncertainties concentrate, i.e.
ˆ σ 2 ( x (cid:63) ) → 0 as N → ∞ and B → ∞ with probability 1. [Figure]
The proof and the technical conditions are given in Appendix F. Proposition 2 assumes that the training error is zero for arbitrarily large training sets, which might at ﬁrst seem unrealistic.
We argue that this assumption is in fact reasonable.
The architecture of our predictor networks (Figure 2, right diagram) is a superset of the prior architecture (Figure 2, left diagram), guaranteeing the existence of weight settings for the predictor that make the training loss zero.
Recent results on deep learning optimization (Du et al., 2019; Allen-Zhu et al., 2019) have shown that stochastic gradient descent can in general be expected to ﬁnd representable functions.
P RACTICAL C ONCLUSIONS FROM THE T HEORY
We now re-visit the algorithm we deﬁned in Section 3, with the aim of using the theory above to obtain practical improvements in the quality of the uncertainty estimates.
Architecture and Choosing the Number of Bootstraps
Our conservatism guarantee in Proposition 1 holds for any architecture for the predictor
h Xf .
In theory, the predictor could be completely arbitrary and does not even have to be a deep network.
In particular, there is no formal requirement for the predictor architecture to be the same as the prior.
On the other hand, to show concentration in Proposition 2, we had to ensure that the prior networks are representable by the predictor.
In practice, we use the architecture shown in Figure 2, where the predictor mirrors the prior, but has additional layers, giving it more representational power.
Moreover, the architecture requires choosing the number of bootstraps B .
Our experiments in Section 7 show that even using B = 1 , i.e. one bootstrap, produces uncertainty estimates of high quality in practice.
Modeling Epistemic and Aleatoric Uncertainty Proposition 1 and Proposition 2 hold for any
Gaussian Process.
When choosing the process appropriately, the random prior framework can be used to model both epistemic and aleatoric uncertainty.
The amount of aleatoric uncertainty can be adjusted by choosing the right prior.
Denote by { n ( x ) } a stochastic process obtained by randomly initializing neural networks and denote by { (cid:15) ( x ) σ 2 A } the noise term, modeling the aleatoric (observation) noise, where samples are obtained from (cid:15) ( x ) ∼ N (0 , 1) at each x independently (see Appendix D for more background on aleatoric noise).
We can now choose the prior process as a sum { f ( x ) } = { n ( x ) + (cid:15) ( x ) σ 2 A } of epistemic component { n ( x ) } and the noise term.
Weight Cloning, Prior Choice and Conservatism One question that can be asked about our architecture (Figure 2) is whether it is possible for the predictor to exactly copy the prior weights, giving zero uncertainty everywhere. A useful edge case to consider here is when we are solving a one-dimensional regression problem, σ 2 A {+= 
 and the both the priors and predictors are linear functions.+}
In this case, after training on two points, the predictors will agree with the priors everywhere and uncertainty estimates will be zero. However, this is still consistent with our conservatism guarantee The reason for this is once we assume such a linear prior, we are comparing to a GP with a linear kernel . But a GP with that kernel will also have zero uncertainty after seeing two samples. In practice, this means that we have to choose the architecture of the prior networks be expressive enough, which is no different from choosing a reasonable prior for Bayesian inference.
Empirically, the tested network architecture did not show such cloning behavior.
P RIOR W ORK
Randomized Prior Functions (RPFs)
Our work was inspired by, and builds on, Randomised
Prior Functions (Osband et al., 2019; 2018), but it is different in two important respects.
First, the existing theoretical justiﬁcation for RPFs only holds for Bayesian linear regression (Osband et al., 2018, equation 3) with non-zero noise 1 added to the priors.
In contrast, our results are much more general and hold for any deep network with or without added aleatoric noise.
Second, we are targeting a different setting.
While RPFs were designed as a way of sampling functions from the posterior, we provide estimates of posterior uncertainty at a given test point.
Our algorithm is based on the work by Burda et al.
(2018), who applied RPFs to exploration in MDPs, obtaining state-of-the art results, but without justifying their uncertainty estimates formally.
Our paper provides this missing justiﬁcation, while also introducing a way of quantifying the error in estimating the uncertainty itself.
Moreover, since Burda et al.
(2018) focused on the application of RPFs to Reinforcement
Learning, they only performed out-of-distribution evaluation on the relatively easy MNIST dataset (LeCun, 1998).
In contrast, in Section 7 we evaluate the uncertainties on more complex vision tasks.
The term prior networks has also been used (Malinin & Gales, 2018) to denote deep networks that output the parameters of a prior distribution, an approach fundamentally different from our work.
Deep Ensembles
The main alternative approach for obtaining uncertainties in deep learning are deep ensembles (Lakshminarayanan et al., 2017).
Building on the bootstrap (Efron & Tibshirani, 1994), deep ensembles maintain several models and quantify epistemic uncertainty by measuring how their outputs vary.
Crucially, deep ensembles use representations trained on regression labels, and tend to learn similar representations for different inputs with similar labels, which can lead to over-ﬁtting the uncertainty estimates. A useful edge case to consider is if the each of the models in the ensemble is convex in the weights.
In this case, models in a deep ensemble will all converge to the same weights and produce zero uncertainty.
While deep learning models used in practice aren’t normally convex, we show empirically in section 7 that deep ensembles can give overconﬁdent uncertainty estimates in practical vision tasks, particularly on points that have the same label as points in the training set. Since our method avoids overconﬁdence, it can be understood as complementary to deep ensembles, to be used in situations where obtaining conservative estimates is more important than the representational beneﬁt of using labels.
In practice, deep ensembles also require using more bootstraps to achieve the same OOD performance. Moreover, they do not have theoretical support in the case when all the members of the ensemble are trained on the same data, which is how they are used in practice (Lakshminarayanan et al., 2017).
Dropout
In cases where it is not economical to train more than one network, uncertainties can be obtained with dropout (Srivastava et al., 2014; Gal & Ghahramani, 2016).
Monte-Carlo dropout can be viewed (Gal & Ghahramani, 2016) as a form of approximate Bayesian inference.
However, to do so requires a rather unnatural approximating family from the perspective of approximate inference. Also, one has then either to take a limit or generalize variational inference to a quasi-KL (Hron et al., 2018) divergence.
In addition, dropout can be interpreted in terms of MAP inference (Nalisnick et al., 2019). Another alternative view of MC dropout is as an ensemble method in which the ensemble members have shared parameters (which means they are trained together) and where the ensembling is applied at test time too. This latter view is arguably as natural as the Bayesian interpretation. For this reason we discuss MC dropout separately from BNNs.
Since dropout implicitly approximates non-Gaussian weight distribution with Gaussians, it exhibits spurious patterns in the obtained uncertainties, which can lead to arbitrarily overconﬁdent estimates (Foong et al., 2019). In Bayesian Neural Networks (BNNs) Bayesian Neural Networks (Blundell et al., 2015; Kingma & Welling, 2014; Rezende et al., 2014; Welling & Teh, 2011; Brosse et al., 2018) explicitly model the
[Figure]
distribution over weights of a neural network. While BNNs provide a link between deep learning and Bayesian inference, they are very slow to train. Even recent tuned implementations of BNNs (Osawa et al., 2019) are several times slower than supervised learning.
This happens despite using a battery of technical optimizations, including distributed training and batch normalization. Moreover, modern convolutional BNNs still carry a signiﬁcant accuracy penalty when deployed with realistic settings of prior variance.
E XPERIMENTS
Encouraged by the huge empirical success of random priors in Reinforcement Learning (Burda et al., 2018)
[Table]
we wanted to provide an evaluation in a more typical supervised learning setting.
We tested the uncertainties in two ways.
First
[Table]
we investigated calibration , i [Table] e. whether we can expect a higher accuracy for more conﬁdent estimates.
Next, we checked whether the uncertainties can be used for out-of-distribution detection.
We compared to two competing approaches for uncertainty detection: deep ensembles (Lakshminarayanan et al., 2017) and spatial concrete dropout (Gal et al., 2017) [Table]
The same ResNet architecture served as a basis for all methods.
Details of the implementation are provided in Appendix A.
[Table]
Out We evaluated the uncertainty estimates on out-ofdistribution detection.
To quantify the results, we evaluated the area under the ROC curve (AUROC) for the task of deciding whether a given image comes from the same distribution or not.
All methods were trained on four classes from the CIFAR-10 (Krizhevsky et al., 2009) dataset.
We then tested the resulting networks on images from withheld classes and on the
SVHN dataset (Netzer et al., 2011), which contains completely different images.
Results are shown in Table 1.
Considering the statistical errors (see Appendix B), random priors performed slightly better than deep ensembles with adversarial training for B = 1 and about the same for B = 10 . For dropout, B refers to the number of dropout samples.
Dropout performed worse, but was cheaper to train.
In order to gain a more ﬁnely-grained insight into [Figure] [Figure] the quality of the uncertainties, we also show uncertainty histograms in Figure 3.
The ﬁgure shows the distribution of uncertainty estimates for seen data (top row) vs. unseen data (bottom row) for bootstrap sizes B = { 1 , 5 , 10 } .
The main conclusion is that uncertainties obtained from random priors are already well-separated with B = 1 , while deep ensembles need more bootstraps to achieve the full separation between test and train examples.
Calibration Good uncertainty estimates have the property that accuracy increases as we become more certain, a property known as calibration .
We measured it by evaluating average accuracy on the subset of images with uncertainty smaller than a given value.
We trained on four classes from the CIFAR-10 (Krizhevsky et al., 2009) dataset.
We then tested the resulting networks on the whole dataset, which included both the seen and unseen classes.
Results are shown in Figure 4.
Ideally, in a calibrated method, these curves should be increasing, indicating that a method always becomes more accurate as it becomes more conﬁdent. [Table]
[Table] In coarse terms [Table] Figure 4 conﬁrms that all methods except a degenerate deep ensemble with only one bootstrap are roughly monotonic.
However, uncertainty estimates from random priors are more stable, showing monotonicity on a ﬁner scale as well as on a large scale.
Interestingly, calibration improved only slightly when increasing the number of bootstraps B .
Subsampling Ablation
In the previous experiment, we kept the architectural and optimization choices ﬁxed across algorithms.
This [Figure] ensured a level playing ﬁeld, but that we were not able to obtain zero training error on the dictor networks used by random priors.
However, we also wanted to evaluate random priors in the setting of zero training error, which is assumed by our concentration result in Section 4.2.
To do this, we used a smaller set of training images, while still keeping the network architecture the same.
This allowed us to obtain complete convergence.
Results of this ablation are shown in Figures 5 and 6, as well as Table 2, analogous to our results on the full dataset presented above.
In this sub-sampled regime, the random prior method easily outperformed competing approaches, showing better calibration (Fig. 5).
The histograms in Figure 6 also demonstrate good separation between seen and unseen data.
In the out-of-distribution benchmarks reported in Table 2, the random prior method has comfortably outperformed the baselines.
While this training regime is not practical for real-life tasks, it demonstrates the potential performance of random priors when trained to full convergence.
Summary of experiments We have shown that uncertainties obtained from random priors achieve competitive performance with fewer bootstraps in a regime where the network architecture is typical for standard supervised learning workloads.
Random priors showed superior performance in a regime where the predictors can be trained to zero loss.
C ONCLUSIONS
We provided a theoretical justiﬁcation for the use of random priors for obtaining uncertainty estimates in the context of deep learning.
We have shown that the obtained uncertainties are conservative and that they concentrate for any neural network architecture.
We performed an extensive empirical comparison, showing that random priors perform similarly to deep ensembles in a typical supervised training setting, while outperforming them in a regime where we are able to accomplish zero training loss for the predictors.
A PPENDICES A PPENDIX A R EPRODUCIBILITY AND D ETAILS OF E XPERIMENTAL S ETUP A PPENDIX A. S YNTHETIC DATA
For the 1D regression experiment on synthetic data (Fig 1), we used feed-forward neural networks with 2 layers of 128 units each and a 1-dimensional output layer.
We used an ensemble size of 5.
The network was trained on 20 points sampled from the negative domain of a sigmoid function and tested on 20 points sampled from the positive domain.
A PPENDIX A. E XPERIMENTAL S ETUP
Model architecture For the CIFAR-10 experiments, we adapted the setup from the cifar10-fast model 3 .
For the network predicting the mean, we used the exact same architecture as in this model. For the prior networks in our uncertainty estimators, the architecture for the prior network was the same as the mean network, but using a ﬁnal linear layer instead of the softmax layer. We used squared error on that last layer to get the uncertainties.
For the predictor networks in the uncertainty estimators, we added two additional layers at the end to make sure the prior functions are learnable (see Fig. 2).
We followed Burda et al.
(2018) in choosing the output size to be M = 512 and using the Adam optimizer (Kingma & Ba, 2014) with a learning rate of 0.0001.
We optimized the initialization scale of our networks as a hyperparameter on the grid { 0 . 01 , 0 . 1 , 1 . 0 , 2 . 0 , 10 . 0 } and chose 2.0, which achieved the best performance.
We chose a scaling factor of β = 1 . 0 for the uncertainty bonus of the random priors and ﬁxed it for all experiments.
Data For the CIFAR-10 experiment, we trained on the classes { bird, dog, frog, horse } and excluded { cat, deer, airplane, automobile, ship, truck } .
For the small CIFAR-10 ablation experiment, we trained on 75 images sampled from the classes { ship, truck } and excluded the remaining classes.
Training Error For Random Priors, the histogram of the training errors is the same as the histogram of the uncertainties. It is shown in the ﬁrst column of Figures 3 and 6. The training error was 0.57 on the CIFAR experiment and 0.03 on the sub-sampled ablation. Out-of-distribution classiﬁcation For computing the areas under the receiver-operator characteristic curves (AUROC) in the OOD classiﬁcation tables, we used the roc auc score function from the Python package sklearn (Pedregosa et al., 2011), using the predicted uncertainties as predicted label scores and binary labels for whether or not the samples were from the training set.
A PPENDIX B A DDITIONAL RESULTS
[Table]
v
[Table]
v
[Table] excluded [Table] A PPENDIX C B ACKGROUND ON B AYES R ISK
For completeness, we recall the deﬁnition of Bayes Risk.
We are often interested in minimizing the
Mean Squared Error E f (cid:2) ( f ( x (cid:63) ) − w ) 2 (cid:3) , where x (cid:63) is a given test point and w is a variable we are allowed to adjust.
A known result of Bayesian decision theory (Robert, 2007; Murphy, 2012) is that the minimizer of the MSE is given by the expected value of f , i.e.
[Equation]
Equation 12 holds for any stochastic process f , including when f is a posterior process obtained by conditioning on some dataset.
A consequence of equation 12 is that it is impossible to obtain a MSE lower than the one obtained by computing the posterior mean of f .
A PPENDIX D G AUSSIAN P ROCESSES
A stochastic process is Gaussian (Williams & Rasmussen, 2006), if all its ﬁnite-dimensional distributions are Gaussian.
The main advantage of GPs is that the posterior process can be expressed in a tractable way.
GPs are often used for regression, where we are learning an unknown functionφ : R K → R from noisy observations.
Since a Gaussian distribution is completely identiﬁed by its ﬁrst two moments, a GP can be deﬁned by a mean function and a covariance function.
Formally, the notation GP ( µ, k ) refers to a GP with with mean function µ : R K
→ R , a positive-deﬁnite kernel function k :
R K × R K → R .
GPs can be used to model two kinds of uncertainty: epistemic uncertainty, which reﬂects lack of knowledge about unobserved values of φ and aleatoric uncertainty, which reﬂects measurement noise.
When performing regression, we start with a zero-mean prior
GP (0 , k ) and then observe N training points
X
= { x i } i =1 ,...,N and labels y =
{ y i } i =1 ,...,N where y i =
φ ( x i ) + (cid:15) i .
Here, the i.i.d.
random variables (cid:15) i ∼ N (0 , σ 2 A ) model the aleatoric noise.
We obtain the posterior process on GP ( µ Xy , k X ) .
For GPs, the mean and covariance of the posterior
GP on y evaluated at x (cid:63) can be expressed as
[Equation]
and
[Equation]
In particular, the posterior covariance does not depend on y .
In the formula above, we use the kernel matrix K ∈ R N × R N deﬁned as K ij = k
( x i , x j ) , where x i and x j are in the training set.
We also use the notation k (cid:63) ∈ R N for the vector of train-test correlations { k (cid:63) } i
= k
( x i , x (cid:63) ) , where x i is in the training set and k ( x (cid:63) , x (cid:63) ) is similarly deﬁned.
The shorthand σ 2 X ( x (cid:63) ) introduced in equationdenotes the posterior variance at a single point.
A PPENDIX E L IST OF SYMBOLS DENOTING VARIANCE
Below, we give a list of symbols used for variance of various random variables.
σ 2 Xf posterior variance of stochastic process σ 2prior variance of stochastic process ˜ σfourth moment of prior stochastic process ˆ σsample-based estimate of prior GP variance ˆ σcombined uncertainty estimate – see equationˆ σ
[Equation]
sample-based mean part of uncertainty estimate – see equation [Equation] ˆ [Equation]
sample-based variance part of uncertainty estimate – see equationv
[Equation]
upper bound on variance of ˆ σ 2 µ σ 2 A aleatoric variance (observation noise)
A PPENDIX F P ROOFS
We now give formal proofs for the results in the paper
A PPENDIX F. P ROOFS RELATING TO C ONSERVATISM
Proposition 1 (Strict Conservatism in Expectation) .
Assume that f is a GP.
Then for any function h : R N × K → R M , we have
[Equation]
Moreover, equality holds if and only if h Xf ( x [Equation]
= µ Xf ( x (cid:63) ) .
Proof.
We instantiate Lemma 1 by setting f to be a GP.
By equation 14, the posterior covariance of a GP does not depend on the target values, i.e. σ 2 Xf ( x (cid:63) )
=
σ 2 X ( x (cid:63) ) .
The ﬁrst part of the result can be shown by pulling σ 2 X ( x (cid:63) ) out of the expectation.
Moreover, since (cid:107) · (cid:107) is a norm and hence positive semi-deﬁnite, equality holds if and only if h Xf ( x (cid:63) )
= µ Xf ( x (cid:63) ) .
Lemma 1.
For any function h : R N ×
( K +1)
→ R M , for any test point x (cid:63) ∈ R K and for any stochastic process { f ( x ) } x ∈ R K with all second moments ﬁnite and exchangeable outputs
[Equation] [Equation]
Proof.
[Equation]
Here, the equality in (16) holds by deﬁnition of conditional probability.
The equality in (19) holds by deﬁnition of posterior mean and the equality 21 follows by assumption that the process has exchangeable outputs.
While this argument follows a similar pattern to a standard result about
Bayesian Risk (see Appendix Appendix C), it is not identical because the function h
Xf depends on f .
Lemma 3.
Assume that the random variable ˆ σ 2 µ ( x (cid:63) ) has ﬁnite variance upper bounded by v UB .
With probability 1 − δ , we have ˆ σ 2 µ ( x (cid:63) ) + 1 √ δ v UB ≥ ˜ σ 2 µ ( x (cid:63) ) .
Proof.
The proof is standard, but we state it in our notation for completeness.
Applying Chebyshev’s inequality to the random variable ˆ σ 2 µ ( x (cid:63) ) , we have that Prob (cid:16) | ˜ σ 2 µ ( x (cid:63) )
− ˆ σ 2 µ ( x (cid:63) ) | ≥ 1 √ δ v UB (cid:17) ≤ δ , implying the statement. Corollary 1.
Assume that f is a GP.
Assume that the random variable ˆ σ 2 µ ( x (cid:63) ) has ﬁnite variance upper bounded by v UB .
Then with probability 1 − δ , for any function h : R N × K → R M , we have
[Equation] Proof. Combine Lemma 3 and Proposition 1. Lemma 2.
Assume that the GP { f ( x ) } is zero mean with exchangeable outputs and the function
h
Xf takes values in [ − U, U ] M .
Assume that permuting the outputs of f produces the same permutation in the outputs of h Xf .
With probability 1 − δ , we have
[Equation]
Here, ˆ σ 20 ( x (cid:63) ) is a sample-based estimate of the prior variance obtained with B 0 samples and C = (cid:16)
B 0 − 1 χ 2 I ( δ ) (cid:17) 2 , where χ 2 I denotes the inverse CDF of the Chi-Squared distribution with B 0 − 1 degrees of freedom.
Proof.
We seek to decompose the variance of ˆ σ 2 µ ( x (cid:63) ) into the part that comes from the prior and the part that comes from the ﬁtted function h
Xf m .
[Equation]
[Equation] [Equation] We now bound the two variances in equation 31 separately.
Since f ( x (cid:63) ) is Gaussian [Equation] we can use a sample-based estimate of the prior variance and obtain an probabilistic conﬁdence interval.
In particular, we know that σ 20 ( x (cid:63) ) ≤ ˆ σ 20 ( x (cid:63) )
B 0 − 1 χ
I
δ ) with probability 1 − δ , where χ 2 I denotes the inverse CDF of the Chi-Squared distribution with B 0 − 1 degrees of freedom.
Using the assumption that f is zero-mean, we now have
[Equation]
Moreover, since h Xf m ( x (cid:63) ) is bounded by U , we have
[Equation]
The bound is obtained by combining equation 31 with equations 33 and 32.
A PPENDIX F. P ROOFS RELATING TO CONCENTRATION
We now proceed to the proofs showing concentration.
We begin by formally deﬁning a class of predictor networks.
Deﬁnition 1 (Class H U of Lipschitz networks) .
Consider functions h : R K
→ R M .
Let j, j (cid:48)
= 1 , . . .
, M , index the outputs of the function.
We deﬁne H U so that each h ∈ H
U has the following properties for each j, j (cid:48) .
(P1)
h
j is Lipschitz continuous with constant L, i.e.
(cid:107) h
j ( x ) − h j
( x (cid:48) )
(cid:107) 2 ≤ L (cid:107) x − x ‘ (cid:107) 2 for all x, x (cid:48) with (cid:107) x (cid:107) ∞ ≤ 1 and (cid:107) x (cid:48) (cid:107) ∞ ≤ 1 , (P2) outputs are exchangeable, i.e. { h j : h ∈ H U }
= { h j (cid:48) :
h ∈ H U } , (P3) the class is symmetric around zero, i.e. h j ∈ { h j : h ∈ H U } implies − h j ∈ { h j : h ∈ H U } .
(P4)
h
j is bounded, i [Table] e
[Table] While the conditions in Deﬁnition 1 look complicated, they are in fact easy to check for predictor networks that follow the architecture in Figure 2.
In particular, Lipschitz continuity (P1) has to hold in practice because its absence would indicate extreme sensitivity to input perturbations.
Output exchangeability (P2) holds since reordering the outputs does not change our architecture.
Symmetry around zero (P3) holds by ﬂipping the sign in the last network layer.
Boundedness (P4) is easy to ensure by clipping outputs.
In the following Lemma, we obtain a bound on the expected uncertainty.
Lemma 4.
Consider a target function f : R K → R M , where j = 1 , . .
, M , with the domain restricted to (cid:107) x (cid:107) ∞ ≤ 1 .
Introduce a constant U such that max (cid:107) x (cid:107) ∞ ≤ 1 | f j ( x ) | ≤ U .
Denote the data distribution with support on { x : (cid:107) x (cid:107) ∞ ≤ 1 } as D .
Moreover, assume K ≥ 3 .
For h Xf ∈ H U , with probability 1 − δ we have
[Equation]
Proof.
The proof uses standard Rademacher tools.
To avoid confusion across several conventions, we explicitly deﬁne the Rademacher complexity of a function class G as: [Equation]
[Equation]
Here, the random variables u i are sampled i.i.d.
using a discrete distribution with Prob ( u i = − 1)
= Prob ( u i = 1) =
12 and the the second equality follows by using property (P3) .
We start by applying the generic Rademacher bound (Mohri et al., 2018) to the function class M = { x 1 , . . .
, x N , t 1 . . .
, t N → 1 U 2 1 M (cid:107) t i − h ( x i ) (cid:107) 2 , h ∈ H U } , which contains the possible errors of the predictor.
[Equation]
We now introduce the function class M (cid:48)
= {
[Equation]
N ,
[Equation]
∈
[Equation]
} , which models the per-output squared error.
Because of property (P2) , M (cid:48) does not depend on the output index j .
By pulling out the sum outside the supremum in equation 35, we get
[Equation]
38) by Talagrand’s Lemma (Mohri et al., 2018; Duchi, 2009), we also have [Equation]
[Equation]
Here, H 1 = { 1 U h j : h ∈ H U } .
By property (P1) , functions in H 1 are Lipschitz continuous with constant L/U .
Instantiating a known bound for Lipschitz-continuous functions (Luxburg &
Bousquet, 2004, Theorem 18 and Example 4), and using the assumption K ≥ 3 , we get ˆ R N ( H 1 ) ≤
LU O (cid:16) 1 K √ N (cid:17) .
The Lemma follows by combining this with equation 38 and equation 39, plugging into equation 36 and re-scaling by U 2 .
Lemma 4 allowed us to relate the error on the training set to the expected error on the test set.
It also shows that the two will be closer for small values of the Lipschitz constant L .
We now use this
Lemma to show our main concentration result (Proposition 2).
Proposition 2.
If the training converges, i.e. the training loss 1 MN (cid:80) Ni =1 (cid:107) f
( x i ) − h
Xf ( x i ) (cid:107) 2 =
σ 2 A for arbitrarily large training sets, then assuming the predictors h
Xf are bounded and Lipschitz continuous with constant L , then under technical conditions the uncertainties concentrate, i.e.
ˆ σ 2 ( x (cid:63) ) → 0 as N → ∞ and B → ∞ with probability 1.
Proof.
We are assuming the technical conditions of Lemma 4.
Instantiating Lemma 4, setting the training loss to zero in the RHS of equation 34 and letting N → ∞ , we obtain the following with probability 1:
[Equation]
From the continuity of f and h
Xf we have that ˆ σ 2 µ is continuous in x (cid:63) .
Together with the property that ˆ σ 2 µ is non-negative, this gives that for every x (cid:63) .
[Equation]
Since σ 2 A does not depend on B , we also have:
[Equation]
From the deﬁnition of ˆ v σ , we have that
[Equation]
Combining equation 42 and equation 43 with equation 1, we get:
[Equation]
We obtain the Lemma by comparing with equation 1.