Fixing Asymptotic Uncertainty of Bayesian Neural Networks with Infinite ReLU Features
Authors: Agustinus Kristiadi
Matthias Hein
Philipp Hennig
Venue:  (2021)
Keywords: Bayesian deep learning
Gaussian processes
uncertainty quantification

Abstract: Approximate Bayesian methods can mitigate overconfidence in ReLU networks. However, far away from the training data, even Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be overconfident. We suggest to fix this by considering an infinite number of ReLU features over the input domain that are never part of the training process and thus remain at prior values. Perhaps surprisingly, we show that this model leads to a tractable Gaussian process (GP) term that can be added to a pre-trained BNN's posterior at test time with negligible cost overhead. The BNN then yields structured uncertainty in the proximity of training data, while the GP prior calibrates uncertainty far away from them. As a key contribution, we prove that the added uncertainty yields cubic predictive variance growth, and thus the ideal uniform (maximum entropy) confidence in multi-class classification far from the training data.

Under review as a conference paper at ICLR 2021
FIXING ASYMPTOTIC UNCERTAINTY OF BAYESIAN NEURAL NETWORKS WITH INFINITE RELU FEATURES
Anonymous authors
Paper under double-blind review
ABSTRACT
Approximate Bayesian methods can mitigate overconfidence in ReLU networks. However, far away from the training data, even Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be overconfident. We suggest to fix this by considering an infinite number of ReI.U features over the input domain that are never part of the training process and thus remain at prior values. Perhaps surprisingly, we show that this model leads to a tractable Gaussian process (GP) term that be added cost overhead. The BNN then yields structured uncertainty in the proximity training data, while the GP prior calibrates uncertainty far away from them. As a key contribution, we prove that the added uncertainty yields cubic predictive variance growth, and thus the ideal uniform (maximum entropy) confidence in multi-class classification far from the training data.
pre-trained BNN's posterior at test time with negligible
INTRODUCTION
Calibrated uncertainty is crucial for safety-critical decision making by neural networks (NNs) (Amodei et al., 2016). Standard training methods of NNs yield point estimates that, even if they are highly accurate, can still be severely overconfident (Guo et al., 2017). Approximate Bayesian methods, which turn NNs into Bayesian neural networks (BNNs), can be used to address this issue. Kristiadi et al. (2020) recently showed that for binary ReLU classification networks. far away from the training data (more precisely: when scaling any input æ with a scalar a > and taking the limit oo), the uncertainty of BNNs can be bounded away from zero. This is an encouraging result when put contrast to the standard point-estimated networks, for which Hein al. (2019) showed earlier that the same asymptotic limit always yields arbitrarily high (over-) Jeoridccon Neverthe- less, BNNs can still be asymptotically overconfident (albeit less so than the standard NNs) since the aforementioned uncertainty bound can be loose. This issue is our principal interest in this paper. An intuitive interpretation is that ReLU NNs "miss out on some uncertainty" even in their Bayesian for- mulation, because they fit finite number of ReLU features to the training data, by "moving around" these features within the coverage of the data. This process has no means to encode a desideratum that the model should be increasingly uncertain away from the data.
In this work, we "add in" additional uncertainty by considering an infinite number of additional ReLU features spaced at regular intervals away from the data in the input and hidden spaces of the network. Since these features have negligible values in the data region, they do not contribute to the training process. Hence, we can consider a prior for their weights, chosen to be an independent Gaussian, and arrive at specific Gaussian process (GP) which covariance function is generaliza- tion of the classic cubic-spline kernel (Wahba, 1990). This GP prior can be added to any pre-trained trained ReLU BNN as a simple augmentation to its output. Considering the additive combination of parametric BNN and GP prior together, we arrive at another view of the method: It approximates the "full GP posterior" that models the residual of point-estimated NN (Blight & Ott, 1975; Qiu et al., 2020). In our factorization, the BNN models uncertainty around the training data, while the GP prior models uncertainty far away from them. By factorizing these parts from each other, our formulation requires no (costly) GP posterior inference, and thus offers lightweight, modular uncertainty calibration. See Fig. for illustration.
Theoretical analysis is a core contribution of this work. We show that the proposed method (i) preserves the predictive performance of the base ReL.U BNN. Furthermore, it (ii) ensures that the

---

Under review as a conference paper at ICLR 2021
1.00
0.75
0.50
(a) BNN
(b) Ours
(c) BNN-ZO
(d) Ours-Z0
Figure 1: Toy classification with a BNN and our method. Shade represents confidence, the suffix "ZO" stands for "oo"med-out". Far away from the training data, vanilla BNNs can still be overcon- fident (a, e). Our method fixes this issue while keeping predictions unchanged (b, d).
surrounding output variance asymptotically grows cubically in the distance to the training data, and thus (iii) yields uniform asymptotic confidence in the multi-class classification setting. These results extend those of Kristiadi et al. (2020) in so far as their analysis is limited to the binary classification be loose. Furthermore, of Meinke & Hein (2020) which attains maximum uncertainty far from the data for non-Bayesian point-estimate NNs. Finally, our empirical evaluation confirms our analysis and shows that the proposed method also improves uncertainty estimates in the non con-ssymtoiiic regime.
case and their bound
approach is complementary to the method
BACKGROUND
2.1 BAYESIAN NEURAL NETWORKS
Let f :RR RR. R defined by (x, 0) f(x; =: (z) be neural network. Here, @ the collection of all parameters of f. Given an i.i.d. dataset D := (am Vm Bm)mm1. mmm the standard training procedure amounts to finding point estimate the parameters which can be identified the Bayesian framework with maximum posteriori (MAP) estimation!
M
e* = arg max log p D) = arg max
Σ m=1
logp(ym fe( + log p(0).
While this point estimate may yield highly accurate predictions, it does not encode uncertainty over 0. causing an overconfidence problem (Hein al., 2019). Bayesian methods can mitigate this issue, specifically, by treating the parameter of f as random variable and applying Bayes rule. The resulting network called a Bayesian neural network (BNN). The common way to approximate the posterior p(O D) of BNN is by a Gaussian g(O D) = NN μ, Σ). which can be constructed for example by a Laplace approximation (MacKay, 1992b) or variational Bayes (Hinton & Van Camp, 1993). Given such an approximate posterior q( D) and test point æ. RN one then needs to marginalize the parameters to make predictions, i.c. we compute the integral v. = h( f)) ( (D) d0, where h is inverse link function, such the identity function for regression or the logistic-sigmoid and softmax functions for binary and multi-class classifications, respectively. Since the network f is non-linear function of this integral does not have an analytic solution. However, one can obtain a useful approximation via the following network linearization: Let x. N be test point and q D) A μ.. Σ) be a Gaussian approximate posterior. Linearizing f around a yields the following marginal distribution over the function output f f(x)):
p(f(x.) E. x.. D) =e ((f(x.) (I
(1)
where J, is the Jacobian of f(x.; 0) w.r.t. θ at (In the case of real-valued network f. we use the gradient g- = (xx; 0) instead of J..) This distribution can then be used as the predictive distribution pvv. x., D) in the regression case. For classifications, we need another approximation since not the identity function. One such approximation the generalized probit approximation
In statistical learning view, log p(ym fo(m))) is identified with empirical risk, log p(0) with regularizer. The two views are equivalent in this regard. "See Rishop (2006, Sec. 5.7.1) for more details.
2

---

Under review as a conference paper at ICLR 2021
(Gibbs, 1997; Spiegelhalter & Lauritzen, 1990; MacKay, 1992a):
x D) = exp(mxc Kuc)
p(u- cl xx,
for all c= 1, I,...,C,
(2)
Ei=1exple exp(m.: koss)
where for each = I,.... C, the real numbers Mei is the i-th component of the vector m.. and Rus := (1 / U=is) 1122 where Vri the i-th diagonal term of the matrix V.. These approxi- mations are analytically useful, but can be expensive due to the computation of the Jacobian matrix J.. Thus, Monte Carlo (MC) integration is commonly used as an alternative, i.e. we approxi- mate ys ~ 8 - h h(f(x.; ((2.:0.) 8,)) with 0, ~ q(O | D). Finally, given a classification predictive distribution ply, F., D), we define the predictive confidence of æ, as the maximum probability co. = max.e(1..... .......) ply- = e x., D) over class labels.
2.2
RELU AND GAUSSIAN PROCESSES
The ReLU activation function RcLU(2) := max(0.2) =) (Nair & Hinton, 2010) has become the de- facto choice of non-linearit in deep learning. Given arbitrary real numbers c, it can be generalized as ReLU(z;;) c) == max (0, = - c), with the "kink" at location c. An alternative formulation, useful below, is in terms of the Heaviside function H as ReI.U(z;e)
H(= c) c). We may define a collection of d such ReLU functions evaluated at some point in R as the function & : R - R k with 2 + (ReLU(2; c1), .... (e:U((2; cx) " We call this function the ReLU feature map; it can be interpreted as "placing" ReLU functions at different locations in TR.
Consider linear model R x R E defined by g(x; w) == w (xx). Suppose & regularly
places the K generalized ReLU functions centered at (c over [Cmin- Cmax. C R. where Cmin <
Cmax- If we consider Gaussian prior p(w) == N (w 0, O.o2K- K Cmin I) over the weights w then, as K goes to infinity, the distribution over (xx)
Gaussian process with mean 0 and covariance (using the shorthand Ix == g(x) and F == min (2, x²) full derivation in Appendix A):
lim cov(O2, o² (ī Cmin) K-00
(5 )
1x² - miin (x: +x) + (x -min )
=: k (x, x' Cmins 0²))
for J S Gmax- Since this expression does not depend on Crax, we consider the limit Craxx C0.
The resulting covariance function the cubic spline kernel (Wahba, 1990).
METHOD
Hein et al. (2019) showed that the confidence of point-estimated ReLU networks (i.e. feed-forward nets which use piccewise-affine activation functions and are linear in the output layer) approaches with increasing distance from the training data. For binary classification, Kristiadi et al. (2020) showed that mamussan-approxiiman away from 1, but not necessarily close to the maximum uncertainty value of 1/2. Thus, just being Bayesian such does not fix overconfidence entirely. A close look at their proof suggests that the issue is a structural limitation of the deep model itself: for any input v, and a sufficiently large scalar o, both the mean and standard deviation of the output (ax.. are linear functions of æ.. Intuitively, this issue arises because the net only has finitely many ReLU features available to "explain" the data, and thus it "lacks" ReLU features for modeling uncertainty away from the data.
ReLU BNNs f instead approach constant confidence bounded
In this section, we will utilize the cubic spline kernel to construct a new kernel and method that, intuitively speaking, adds an infinite number ReLU features away from the data to pre-trained BNNs. This construction adds the "missing" ReLU features and endows BNNs with super-quadratic output variance growth, without affecting predictions. All proofs are Appendix
3.1 THE DOUBLE-SIDED CUBIC SPLINE KERNEL
The cubic spline kernel constructed above is non-zero only (Cmin, 00) R. To make it suit- able for modeling uncertainty in an unbounded domain, we set Cmin = 0 and obtain a kernel kL, (x, xi: a2) = A (x,,, x': 0, 2) which is non-zero only (0, oo). Doing an entirely analogous
3

---

Under review as a conference paper at ICLR 2021
(a) =
(b) D 20
(c) D =
(d) D 00
The construction of our kernel in ID, as the limiting covariance of the output of a Bayesian linear model with D ReLU features. Grey curves are ReLU features while thin red curves are samples. Red shades are the +1 standard deviations of those samples.
Figure 2:
construction with infinitely many ReLU functions pointing the left, i.c ReLU(-2; c), obtain the kernel k__ (x. o²) == ( (-x, -x'; o²), which is non-zero only on (-o0, 0). We combine both into the kernel
k (2, = (x,,'/o; ki (x,
which covers the whole real line (the value at the origin (0,0) is zero) see Figure 2. For multivariate input domains, define
kes (x, x²; 0²:00) =
h (vi, x4100)
(3)
for any x, x' N with of product, since we want the associated GP to add uncertainty anywhere where at least one input call this kernel the double-sided cubic spline (DSCS) kernel. Two crucial properties of this kernel are that it has negligible values around the origin and for any x. E R. and a E R, the value k (aw,, axx) is cubic in a.
We here deliberately use a summation, instead of the alternative
dimension has non-vanishing value.
3.2 RELU-GP RESIDUAL.
Let f: RN TRD + IR be an I--layer, real-valued ReLU BNN. Suppose we place infinitely many ReLU features by following the previous construction. Then, we arrive GP(f(0) 0, k.)) of some real-valued function f(0) : RW - R over the input space RVV We can use this GP to model the "missing" uncertainty which, due to the lack of its presence, makes overconfident far-away from the data, We do so a standard manner by assuming that the "true"
zero-mean GP prior
latent function f is the sum of f and f(!);
f==f+
where 7ro) GP(F()) k..).
Under this assumption, given an input æ.. is clear that f(a) does not affect the expected output of the BNN since the GP over f(0) has zero mean. However, f(o) do additively affect the uncertainty of the BNN's output f- : f(x)) since if we assume that f. -NEE f., var f.), then it follows that T. (E f.: var fa + keo v.)). Hence, the random function f(0), resulting from placing an infinite number of ReLU features in the input space, indeed models the uncertainty residual of the BNN f. We thus call our method ReLU-GP residual (RGPR).
Unlike previous methods for modeling residuals with GPs, RGPR does not require posterior in- ference since intuitively, the additional infinitely many ReLU features are never part of the training process their "kinks" pointing away from the data. So even if we were to actively include them in the training process somehow, they would have (near) zero training gradient and just stay where and they are. The following statements illustrate this intuition more formally in GP regression under the linearization (1) by assuming w.l.o.g. that the kernel values over the dataset are negligible (by shifting and scaling until the data sufficiently close to 0 E R.)..
By contrast, product kir (ar, x x²) zero if of the k2, (Ii, sr83 o²) is
4

---

Under review as a conference paper at ICLR 2021
Proposition 1. Suppose f: :R xR - R defined by (x,0) with a prior p(O) (O O, and D = mm Ym ym Im=1 is dataset. Let f()) and f be defined as in (4), and let a. E INN be arbitrary. Under the linearization of w.r.t. 0 around 0, given that all 2i...., BM are sufficiently close the origin, the GP posterior of fx := f(xx) is given by
0) f(x; 0) is ReLU regression BNN
(5) where μ and E are the mean and covariance of the posterior of the linearized network, respectively, and 9- := Ve (z.; 0o
p(f= (x,)) [fx f(z: u) 9, Eg. + (z,
The previous proposition shows that the GP prior of f(0) does not affect the BNN's approximate posterior- is written posteriori f plus a priori f(o) Therefore, given a pre-trained BNN f with its associated posterior p(0 (D) R NO μ. Σ), we can simply add to its output f(x.;; 0) (with 0 ~ p(0 D)) a random number (x.) ~ GP( | assume that f is a pre-trained BNN.
0, ker (x., v.)). We henceforth
While the previous construction is sufficient for modeling uncertainty far away from the data, it does not model the uncertainty near the data region well. Figure 3(a) shows this behavior: placing infinitely many ReLU features over just the input space yields uncertainty that is not adapted to the data and hence, far away from them, we can still have low variance. To alleviate this issue, we additionally place infinite ReLU features on the representation space of the point- estimaett (.) [( (.:4e 11), which indeed encodes information about the data since f a trained BNN, as follows.
For each 1 = 1,..., I, - and any input a.., let N₁ be the size of the 1-th hidden layer of fr and h. (xx) =: _ be the I-th hidden units. By convention, we assume that N₂ == and h. := xx- We place for each / = ......L - 1 an infinite number of ReLU features on the representation space RRN, and thus we obtain random function f()) N + R distributed by the Gaussian process GP((() O, k.r)) Now, given that Ñ == E N₁, we define the function f = +.... f(.-1), This function is therefore function over all representation (including the input) spaces of fu. distributed by the additive Gaussian process GP(f | other words, given the representations h. == (h ( M of æ., the marginal over the function output f(h.) f. thus given by
RR - R by
0, Ei- k2)). In
p(P)= -
f. 0,
0.E.... 1=0
krs (e.(nn.
nn.0 (n..n=
Figure 3(C) visualizes the effect of this definition. The low-variance region modeled by the random function f becomes more compact around the data and can be controlled by varying the kernel hyperparameter of for each layer / 0,...,, 1. Finally, we can then model the residual in (4) using ʃ instead, i.e. we assume f = f
The generalization of RGPR to BNNs with multiple outputs is straightforward. Let f R. x R.D. R.C be a vector-valued, pre-trained, L-layer ReLU BNN. We assume that the sequence of random functions (fc R R)C is independent and identically distributed by the previous Gaussian process GP(f 0, Ei- k. Thus, defining f. = f(h.) = (f.........
we have
x(f.) = N fm
Σ 1=
Rees (n..) hh: :n: 1
Furthermore, as in the real-valued case, for any x.., the GP posterior p(F= x., D) is approximately (under the linearization of f) given by the Gaussians derived from (1) and (7):
p(J.1 x., D) ~ N ( T. fy fn((:.), EJ. + Σ kes
L-1
(n. 1 :o 1
(8)
1=0
Although the derivations above may appear involved, it is worth emphasizing that in practice, the only overheads compared to the usual MC-integrated BNN prediction step are (i) a single additional forward- hewarddpass over fa. (ii) L evaluations of the kernel kes and (ii) sampling the C-dimensional Gaussian (7). Note that their costs are negligible compared to the cost of obtaining the standard MC-prediction of f. We refer the reader Algorithm for step-by-step pseudocode.
5

---

Under review as a conference paper at ICLR 2021
120
90
60
10
(a) Input
(b) Input & hiddens (of
(c) Input & hiddens (of 10)
Figure 3: Variance of f (6) function of æ.. When J is function neural network repre-
sentations of the data (b), it captures the data region better than when is only defined on the input space (a). Increasing the kernel hyperparameters (here we assume they have the same value for all layers) makes the low-variance region more compact around the data (c).
Algorithm MC-prediction using RGPR. Differences from the standard procedure are red.
Input:
Pre-trained multi-class BNN classifier R x RD - RRC with posterior p(0 'D). Test point RRN. Prior variance hyperparameters (o ( of f. Inverse link function h. Number of MC samples S.
1: (h.) M=11 fforward x.) 2: v. (x.)) )= = Ei kes (he _ n, ;o? for
& Compute representations of æ. via a forward pass fu Compute the prior variance of
3:
1..... S do
4:
0, ~ N(e mr E)
> Sample from the (approximate) posterior of
5:
f.(x.) f(xx;;2)
> Forward pass on f using the sampled parameter
6:
fa(x.) ~ -(((h.) 10,v((x)
Sample from the marginal (7)
7:
[a(x.)) = [s(x.) + (an..)
Compute f(..; 0.)
8:
end for
9: return S S-12. h(f((x)))
Make prediction by averaging
ANALYSIS
Here, we will study the theoretical properties of RGPR. Our assumptions are mild: (i) assume that RGPR is applied only to the input space and (ii) use the network linearization technique. Assumption (i) is the minimal condition for the results presented in this section to hold similar results can also easily be obtained when hidden layers are also utilized in RGPR. Meanwhile, assumption (ii) is necessary for tractability-inn Section 6 we will validate our analysis general settings.
hold-similar
The following two propositions (i) summarize the property that RGPR preserves the original BNN's prediction and (ii) show that asymptotically, the marginal variance of the output of f grows cubically.
Proposition 2 (Invariance in Predictions). Let f R. xRP RC be any network with posterior (( μ. Σ) and be obtained from via RGPR (4). Then under the linearization of f. for any xx E R RN,, we have Bp....e.. f= Epp(f-mm... (D) fx.
Proposition 3 (Asymptotic Variance Growth), Let f :.R. R. - B.C. be C-class ReLU net- work with posterior (e p. Σ) and ff be obtained from f via RGPR over the input space, Suppose that the linearization
w.r.t. O around μ is employed. For any x. E BNN with æ. + there exists > such that for any a 2 B, the variance of each output component (1(a.))....... fe(ox.) under p(f. x. D) (8) is in (
As consequence of Proposition 3, in the binary classification case, the confidence of az, decays like 1/ 1/Va /a far away from the training data. This can be seen using the (binary) probit approximation. Thus, in this case obtain the maximum entropy in the limit of - do. In the following theorem we formalize this statement in the more general multi-class classification setting.
Theorem 4 (Uniform Asymptotic Confidence). Let f RN R - RCC be C-class ReLU
network equipped with the posterior N0 μa, E) and let be obtained from via RGPR the

---

Under review as a conference paper at ICLR 2021
Table 1: Performances of RGPRs compared to their respective base methods on the detection of far-away outliers. Error bars are standard deviations of ten trials. For each dataset, best values over each vanilla and
method (e.g. LLL against LLL-RGPR) are in bold.
MNIST
CIFARIO
SVHN
CIFARI00
Methods
MMC +
AUR t
MMC 4
AUR
MMC +
AUR
MMC +
AUR +
BNO
88.2.11.1
87.0
22.0:1.22
100.010.00
22.11:0.3
100.01:.00
10.510.11
99.6110.0
99,9=10.0
9.8=0.77
17.420.00
100.0::0.0
27.5:10.1
99.6:10000
5.940.00
99 99.940.00
LLL-RGPR
16.6.10.1
100.0:10.0
15.1:10.1
100.0:10.0
15.1100.0
100.0.10.0
4.2.10.0
100.011..0
KFL
57.2.13.0
96.0.10.8
69,5.12.5
81.2.12.5
64.9112.3
90.9111.6
41.1
81.7.1.
KFL-RGPR
28.2:
99.8=0.00
27.5+0.2
99.1
27.5+00.2
99.6+00.0
13.9
97.5++022
SWAG
83.2:1:55
55.3+:20.8
50.5:11.11
97.41:255
59.6119.7
97.5
96.0+15.0
9.6
SWAG-RGPR
27.9110.3
99.8.0..1
27.6.10.2
99.9.10.0
27.510..1
99.91.0.0
17.410.9
78.5
SVDKL 59.1110.6
99.7
46.1110.4
99.3110.1
48.5
99.4.10.0
55.3-1
80.91116
SVDRI-RGPR
22.1::0.2
100.0=0000
22.1
100.0++0.0
22.0::0.1
100.0:::.00
9.8+0..1
100.0+:0.00
input space. Suppose that the linearization of f and the generalized probit approximation (2) used for approximating the predictive distribution ply, = a (D under J. Then for any input I, E EN with Ix o 0 and for every class I...., C,
1
lim ply. at., F.D) C
5 RELATED WORK
The mitigation of the asymptotic overconfidence problem has been studied recently. Although Hein et al. (2019) theoretically demonstrated this issue, their proposed method does not fix this issue for a large enough. Kristiadi et al. (2020) showed that any Gaussian-approximated BNN could mitigate this issue even for to a constant in (0, 1). not to the ideal uniform confidence. In a non-Bai non-Bayesian framework, using Gaussian mixture models, Meinke & Hein (2020) integrate density estimates of inliers and outliers data into the confidence estimates of an NN to achieve the uniform confidence far away from the data. Nevertheless, this property has not been previously achieved in the context of BNNs.
However, the asymptotic confidence estimates of BNNs converge
Modeling the residual of a predictive model with GP has been proposed by Blight & Ott (1975); Wahba (1978); O' O'Hagan (1978); Qiu et al. (2020). The key distinguishing factors between RGPR and those methods are (i) RGPR models the residual of BNNs, in contrast to that of point-estimated novel kernel which guarantees cubic uncertainty growth, and (iii) RGPR requires no posterior inference. Nevertheless, whenever those methods uses our DSCS kernel, RGPR can seen as an economical approximation of their posterior: RGPR estimates uncertainty near the data with a BNN, while the GP-DSCS prior estimates uncertainty far away from them.
networks, (ii) RGPR
combination of weight- and function-space space models has been proposed in the context of non- parametric GP posterior sampling. Wilson et al. (2020) proposed to approximate function as the sum of a weight-space prior and function-space posterior. In contrast, RGPR models function as the sum of weight-space posterior and function-space prior in the context of parametric BNNs.
6 EMPIRICAL EVALUATIONS
Our goal in this section is (i) to validate our analysis the preceding section: we aim to show that RGPR's low confidence far-away from the training data is observable in practice, and (ii) to explore the effect of the hyperparameters of RGPR to the asn-asymptotic confidence estimates. We focus regression are in Appendix D.

---

Under review as a conference paper at ICLR 2021
6.1 ASYMPTOTIC REGIME
We use standard benchmark datasets: MNIST, CIFARIO, SVHN, and CIFARI00. We use LeNet and R8 for MNIST and the rest of the datasets, respectively. Our main reference is the method based on Blight & Ott (1975) (with our kernel): We follow Qiu et al. (2020) for combining the network and GP, and for carrying out the posterior inference. We refer to this bascline as the Blight and Ott method (BNO)-o cf. Appendix C. for an exposition about this method. The base methods, which RGPR is implemented on, are the following recently- sccentty-ropoosed BNNs: (i) last-layer Laplace (LLL., Kristiadi et al., 2020), (ii) Kronecker-factored Laplace (KFL, Ritter et al., 2018). (iii) stochastic weight Serraging-Gaussian (SWAG, Maddox et al., 2019), and (iv) stochastic variational deep kernel learning (SVDKL, Wilson et al., 2016). All the kernel hyperparameters for RGPR are set to 1. In all cases, MC-integral with 10 posterior samples is used for making prediction.
To validate Theorem 4, we construct a test dataset artificially by sampling 2000 uniform noises in [0, 1]ah and scale them with a scalar a = 2000. The goal is to distinguish test points from these outliers based on the confidence estimates. Since visual inspection of these confidence estimates as in Figure is not possible in high dimension, we measure the results using the mean maximum confidence (MMC) and area under ROC (AUR) metrics (Hendrycks & Gimpel, 2017). MMC is useful for summarizing confidence estimates, while AUR tells us the usefulness of the confidences for distinguishing between inliers and outliers.
The results are presented in Table 1. We observe that the RGPR-augmenter methods are significantly better than their respective base methods. In particular, the confidences drop, as shown by the MMC values. We also observe in Table 3 (Appendix D) that the confidence estimates close to the training data do not significantly change. These two facts together yield high AUR values, close to the ideal value of 100. Moreover, most RGPR-imbued methods achieve similar or better performance BNO baseline, likely be due to uncertainty already presents in the base BNNs. However, these confidences on far-away points are not quite the uniform confidence due to the number of MC samples used-recall that far away from the data, RGPR yields high variance; since the error of MC-integral depends both the variance and number of samples, large amount of samples are needed to get accurate MC-estimates. See Figure (Appendix D) for results with 1000 samples: in this more accurate setting, the convergence the uniform confidence happens at finite (and small) a. Nevertheless, this issue not a detrimental to the detection of far-away outliers, as shown by the AUR values in Table 1.
6.2 NON-ASYMPTOTIC REGIME
The main goal of this section is to show that RGPR can also improve uncertainty estimates near the data by varying its kernel hyperparameters. For this purpose, we use a simple hyperparameter optimization using a noise out-of-distribttion (OOD) data, similar to Kristiadi et al. (2020), to tune (a?)-the details are in Section C.2. We use LLL the base BNN.
First, we use the roNASE experiment proposed by Ovadia et al. (2019), where we measure methods' calibration at different rotation angle, see Figure 4. LLL gives significantly better perfor- mance than BNO and RGPR improves the performance further. Moreover, we use standard OOD data tasks where one distinguishes in-distribution from out-distribution samples. We do this with CIFAR10 10 as the in in-distribution dataset against various OOD datasets (more results in Appendix D). As shown in Table 2, LLL. outperforms for CIFARIO BNO and RGPR further improves L.L...
CONCLUSION
We have shown that adding "missing uncertainty" to ReLU BNNs with carefully-crafted GP prior that represents infinite ReLU features fixes the asymptotic overconfidence problem of such net- works. The core of our method is generalization of the classic cubic-spline kernel, which, when used as the covariance function of the GP prior, yields a marginal variance which scales cubically in the distance between test point and the training data. Our main strength lies in the simplicity of the proposed method: RGPR is relative straightforward to implement, and can be applied inexpensively to any pre-trained BNN. Furthermore, extensive theoretical analyses show that RGPR provides sig- nificant improvements to previous results with vanilla BNNs. In particular, we were able to show uniform confidence far-away from the training data in multi-class classifications. On less formal
8

---

Under review as a conference paper at ICLR 2021
0.8
BNO
0.0
LLL
0.5
0.2
LLL-RGPR
50
100
150
50
100
150
50
100
150
(a) Accuracy
(b) Brier Score
(c) MMC +
Figure 4: Rotated-MNIST results (averages of 10 predictions). r-axes are rotation angles. In (a), all
methods achieve similar accuracies.
Table 2: OOD data detection. Datasets in bold face the in-distribution datasets.
BNO
LLL
LLL-RGPR
Datasets
MMC
MMC 4
AUR f
MMC 4
AUR
CIFARIO
96.910.1
92.7=0011
90.320.1
SVHN
69.0:10.0
93.610.1
45.31100
96.41:011
44.0
96.010..1
LSUN
76,6+0.0
90.8+0.1
56.8=0.1
92.8+0.1
51.5+0.1
93.7=00.1
CIFARIOS 100
80.0:10.0
86.3+0.1
64.1+000
88.3+00.1
60.3+0.0
88.3+0.11
UniformNoise
75.91:0.4
94.340.1
36.720.2
99.0:10.0
25.4:0.1
99.8=0.00
Noise
61.5.10.4
96.310.1
41,81002
97.6.10.1
40.5
97.410.1
note, our construction, while derived as a post-hoc addition to the network, follows a pleasingly simple intuition that bridges the worlds of deep learning and non pam-aeatrrr/keeeee models: The RGPR model amounts to considering non non-parametric model of infinitely many ReLU features, only finitely many of which are trained as deep ReL.U network.
REFERENCES
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete Problems in AI safety. arXiv preprint arXiv:1606.06565 2016.
Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding Deep Neural Networks with Rectified Linear Units. In ICLR, 2018.
Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
BJN Blight and I. Ott. A Bayesian Approach Model Inadequacy for Polynomial Regression. Biometrika, 62, 1975.
Mark N Gibbs. Bayesian Gaussian Processes for Regression and Classification. Ph. D. Thesis, Department of Physics, University of Cambridge, 1997.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. ICML., 2017.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why ReLU Networks Yield High- confidence Predictions Far Away from the Training Data and How to Mitigate the Problem. In CVPR, 2019.
Dan Hendrycks and Kevin Gimpel. A Baseline for Detecting Misclassified and Examples in Neural Networks. In ICLR, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep Anomaly Detection with Outlier Exposure. In ICLR, 2019.
James Hensman, Alexander Matthews, and Zoubin Ghahramani. Scalable Variational Gaussian Process Classification. In AISTATS, 2015.

---

Under review as a conference paper at ICLR 2021
Geoffrey E Hinton and Drew Van Camp. Keeping the Neural Networks Simple by Minimizing the Description Length of the Weights. In COLT, 1993.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks. ICML, 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In NIPS, 2017.
David JC MacKay. The Evidence Framework Applicd Classification Networks. Neural compu- tation, 1992a.
David JC MacKay, Practical Bayesian Framework For Backpropagation Networks. Neural com- pulation, 4(3):448-472 1992b.
Wesley Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A Simple Baseline for Bayesian Uncertainty in Deep Learning. In NeurIPS, 2019.
Alexander Meinke and Matthias Hein. Towards Neural Networks that Provably Know when They don't Know. In ICLR, 2020.
Vinod Nair and Geoffrey E Hinton. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML, 2010.
Anthony O' O'Hagan. Curve Fitting and Optimal Design for Prediction. Journal of the Royal Statistical Society: Series B (Methodological). 40, 1978.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado. David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty under Dataset Shift. In NeurlPS, 2019.
Xin Qiu, Elliot Meyerson, and Risto Miikkulainen. Quantifying Neural Networks via Residual Estimation with an I/O Kernel. In ICLR, 2020.
Uncertainty in
Carl Edward Rasmussen and Christopher K. I. Williams, Gaussian Processes in Machine Learning. 2005.
Hippolyt Ritter, Aleksandar Botev, and David Barber. Scalable Laplace Approximation for Neural Networks. In ICLR, 2018.
David Spicgelhalter and Steffen L Lauritzen. Sequential Updating of Conditional Probabilities Directed Graphical Structures. Networks, 1990.
Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional Variational Bayesian
Neural Networks. In ICLR, 2019.
Grace Wahba. Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression. Journal of the Royal Statistical Society: Series B (Methodological), 40, 1978.
Grace Wahba. Spline Models for Observational Data. SIAM, 1990.
Andrew G Wilson, Zhiting Hu, Russ R Salakhutdinov, and Eric P Xing. Stochastic Variational Deep Kernel Learning. In NIPS, 2016.
James Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Mare Peter Deisenroth. Efficiently Sampling Functions from Gaussian Process Posteriors. In ICML, 2020.
10

---

Under review as a conference paper at ICLR 2021
APPENDIX A DERIVATIONS
A.1 THE CUBIC SPLINE KERNEL
Recall that have linear model f [Cmin: Cmax] RRK IR with the ReLU feature map & defined by f(æ;w) == w To ((x) over the input space [Cmin, Cmax] C R, where Cmin < Cmax- Furthermore, Φ regularly places the K generalized ReLU functions centered at (4) where Cmin K-1 (Cmax Cmin) in the input space and consider Gaussian prior p(w) N (w O, o0K k-k--(a (Cmax - Cmin) over the weight w. Then, as K goes to infinity, the distribution over the function output f(x) is a Gaussian process with mean and covariance
K
cov(f(2), f(x')) =
Cmax
Gmin
((z) = Cmax
Cmin
K
K
K
Cmax
Cmni
H(x (z' - ci)(x c)((' -
K
Cmax
Cmin
H (min(x,, c.) (c: ca( + x') + Zx'),
K
where the last equality follows from (i) the fact that both x and x' must be greater than equal to C₁ and (ii) by expanding the quadratic form in the second line.
Let min(x, x'). Since (9) is Riemann sum, in the limit of K - co, it is expressed by the following integral
{
Can 1 -) c) (c² -d(x + x') + xx') de
lim cor(f()), = d² K-0o0
min (ainf&
= =z =02H( H(ū (ā- Cmin)
2 + x') + xx' de
Cain
1 (30 (uin)
1 (2² cmin -22-cmnne + x') + (2 Cmin )
02 H(E Cminn
where we have defined == min Cmax]. The term H Cmin) has been added in the second equality as the previous expression zero if I I Cmin (since in this region, all the ReLU functions evaluate to zero). Note that
H Cmin) = (min H Cmin)
is itself a positive definite kernel. We also note that Cmax can be chosen sufficiently large so that [- Canaa Cmax]" dd contains for sure the data, e.g. this is anyway true for data from bounded domains like images in [0, 1jd, and thus can set
min(z,2²).
APPENDIX B PROOFS
Proposition 1. Suppose f RN x RDD + IR defined by (x, 0) + f( is ReLU regression BNN with prior p(0) = N B) and D = fm Vm Jm=1 is dataset. Let f(()) and f be defined as in (4), and let x, E N be arbitrary. Under the linearization of f w.r.t. 0 around O, given that all æ æi,..., BM are sufficiently close to the origin, the GP posterior of f- := f(x.) is given by
v(J. x., ~N(J. [(x;; p). 9. Eg. kee (es(
(5)
where μ and Σ are the mean and covariance of the posterior of the linearized network, respectively, and g- = (ef(x.;0)
Proof. Under the linearization of f w.r.t. O around O, we have
f(x: 0) R f(x;; 0) + Vof(x;; 0)1o o = g(x)) To

---

Under review as a conference paper at ICLR 2021
Now, the definition of RGPR implies that have
f(x) g(x) To (x);
Following O' Hagan (1978), we thus obtain the following GP prior over f. which marginal is f(x) N(f(x) 0,)(z) Bg(æ) + (x.
Suppose we write the dataset as D == (X, y) where X the data matrix and y is the target vectors, and x, E RN is arbitrary test point. Let hes = (Ru, Ei). k.,, (M)) let K,, (K + I) with Kiz == kes (æs; T;)) and o² > O sufficiently large be the (regularized) kernel matrix, and let G (g( (g(x1)..... ... g (x))) be the matrix of training "features". As Rasmussen & Williams (2005, Sec. 2.7) suggests, we have then the following GP posterior mean and variance
E(f(x.) DD g(x.)
Ken K yy g(xx) (T..)).) u)
(10)
var (f(x.) D) = k+ (I.. xx)) + k K k+ + r (
"CK,''G)"
(11)
where μu :- GK GG, and r= :- g(x.) )- GK kes Since all training points 2t,..... ,, 2a are sufficiently close the origin, by definition of the DSCS kernel, we have k.,
kes.
and K x 1/0²I. These imply that
# (B= + 1/0 G
and
g (r.).
In particular, notice that ze is approximately the posterior mean of the Bayesian linear regression on I (Bishop, 2006, Sec. 3.3). Furthermore (10) and (11) become
B(f(x.) D) g(x.) μ f( var (f(x.) D) ze kes x.) - g(x.) (B +1/0 g(x))
respectively. Notice in particular that E is the posterior covariance of the Bayesian linear regression on f. Thus, the claim follows.
Proposition 2 (Invariance in Predictions). Let f RN x IR - RC be any network with posterior (( μ. Σ) and f be obtained from f via RGPR (4). Then under the linearization of f. for any a. E NR,, we have Ep(F. (e. f. Eppff (pp(flle..D) (D. fa.
Proof. Simply compare the means of the Gaussians p(f+ xx, D) in (8) and p(f= x., D) in (1).
To prove Proposition 3 and Theorem 4, we need the following definition. Let f: R. RD RC defined by (x, 0) - fæ; 0) be feed-forward neural network which use piecewise affine activation functions (such as ReLU and called ReLU network and can be written continuous piecewise-affine function (Arora al., 2018). That is, there exists a finite set of polytopes - to as linear regions f-such that Q = R- and flo, is an affine function for each i = I,..., (Hein et al., 2019). The following lemma is central in our proofs below (the proof is in Lemma 3.1 of Hein et al. (2019)).
and are linear in the output layer. Such a network is
Lemma 5 (Hein et al., 2019). Let (Qif be the set of linear regions associated the ReLU network f : NRN, Rd + RCC, For any x E RN with F 0 there exists a positive real number B and j E {1, [I....,P) P) such that wa Qj for all a 2 B.
Proposition 3 (Asymptotic Variance Growth). Let RRN R.D - H be C-class ReLU net- work with posterior NO p., E) and J be obtained from J via RGPR over the imput space. Suppose that the linearization of I w.r.l. 0 around μ is employed. For any &. E RRN with x, * 0 there exists 3 > 0 such that for any a 2 B, the variance of each output component fooat..).....aa... under p(J. w., D) (8) in (
12

---

Under review as a conference paper at ICLR 2021
Proof. Let z. RN with + be arbitrary. By Lemma and definition of ReLU network, there exists linear region R and real number 3 such that for any a 2 B, the restriction of to can be written
fn((2 0) (az) +b, for some matrix W E RCxN and vector b E Re, which are functions of the parameter 0, evaluated at μ.. In particular, for each c= I,...,C,, C, the o-th output component of f l can written by
felR w (ax) be: where ue and be are the e-th row of W and b, respectively.
Let (I, and let fe(a..) be the e-th column of the Jacobian J(ox.) as defined in (I)- Then by definition of p(J+ xx. D), the variance of feln(oar.) e-th diagonal entry of the covariance of p(f. I., D) given by
var(fe (arfflR((cs (axx)) = je(ax.)) Eje(ax,) + ke+ (ax,, axx)).
Now, from the definition of the DSCS kernel in (3), we have
1
kes (a:E. (KE.)
N
1
Txi
N
a
N
k1
(Tods Dos))
N
E O(a)).
Furthermore, we have
Eje(ax.) [[jxx))Eje(axx)) Thus, fc(ox.) T EJee(ax, is a quadratic function of a. Therefore, var var(feln(ax..)) is ((a)). L
jc(ax.)
= (a(Vowelu) x + Vobelu) E (a(Vouwela) Vobelu).
Theorem 4 (Uniform Asymptotic Confidence). Let f :::NN: X - IRC be a C-class ReLU network equipped with the posterior N ) μ., Σ) and let I be obtained from J via RGPR over the imput space. Suppose that the linearization of ʃ and the generalized probit approximation (2) is used ax.,, f. D) under f. Then for any imput
for approximating the predictive distribution ply. a. E TRN with + O and for every class = 1....,C
lim ply- = ax., F.D) c a0000
Proof. Let x. * RN be arbitrary. By Lemma and definition of ReLU network. there exists a linear region and real number > 0 such that for any a 2 B, the restriction of f to R can be written
fln(ax) = (axx)+ +
where the matrix W E RCCN and vector b E CRC are functions of the parameter 0, evaluated at μ. Furthermore, for I,...,CC. we denote the i-th row and the i-th component of W and as w, and by respectively. Under the linearization of f, the marginal distribution (8) over the output f(w holds. Hence, under the generalized probit approximation, the predictive distribution restricted to R is given by
ply- c |/0x,, D) ~ exp(m.(ax.) K. (axx))
Σ cxa(m,(axx.) Kes (((ax)))
1 Ee exp(m;(0xx.) nis ma(axt)) - mm((x..)) (o(x.))))
=
13

---

Under review as a conference paper at ICLR 2021
where for all i 1, I,...,CC
mi(axx)= = fila(ax;; = w (ax) + b, E R,
and
<+(axz) + 7 (ux(am.)) Ress (aar., axx))) E Rzo-
In particular, for all = =,...... ..... C, note that m(ax.);; E ((a) and k(oær); @(1/a²) since (a(ar.)) + he (arr. aat.) is in ((a²) by Proposition 3. Now, notice that for any e I,..., and any E {_...., C} {c), have
Zio((x.) = (m:((xx) Nos (x(x)))) (mo(aa.)) (o((xx))
(Ki((ar.) we Ka(avx,) w.) (axx.) Ki (cxx))))))))) by
Ko((aa))b be-
(11/a)) (11/a))
(1/a2)
e @11/a
Thus, is easy to see that lima-+oo Zie( (cxx)) = 0. Hence we have
lim ply. = ox., D) a0000
lim
a t00
Eidoo exp(z1e(axx.)
1
Lize exp(O)
1 C
required.
APPENDIX C FURTHER DETAILS
C.I THE BLIGHT AND OTT'S METHOD
The Blight and Ott's method (BNO) models the residual of polynomial regression.. That is, suppose
& > RD is a polynomial basis function defined by Φ(x) = (1,x, ,...,,D-11 & is an arbitrary kernel, and un E RDD is weight vector, BNO assumes
f(z) = w Φ(z) To<x)> f(x).
where f(x) ((,,
Recently, this method has been extended to neural network. Qiu et al. (2020) apply the same idea- modeling residuals with GPs-to pre- mr-irrened networks, resulting method called RIO. Suppose that fu RN + R is a neural- with a pre-trained, point-estimated parameters μ.. Their method is defined by
f(x) = fa(x) f(x)) where f(x) ~ GP(, (o x)).
The kernel Ano is sum of RBF kernels applied on the dataset D (inputs) and the network's ' predic- tions over D (outputs), hence the name (O-iinut-outpuu As in the original Blight and Ott's method, RIO also focuses in doing posterior inference the GP. Suppose that m(x) and u(x) is the a poste- riori marginal mean and variance of the GP, respectively. Then, via standard computations, one can see that even though f is a point-estimated network, f is a random function, distributed posteriori by
F(z) N (F(z) fa(=) + m(x)).(x))
Thus, BNO and RIO effectively add uncertainty to point-estimated networks.
The posterior inference of BNO and RIO can be computationally intensive, depending the num- ber of training examples M: The cost of exact posterior inference is in ((13). While it can be alleviated by approximate inference, such via inducing point methods and stochastic optimiza- tions, the posterior inference requirement can still be a hindrance for a practical adoption of BNO and RIO, especially large problems.
14

---

Under review as a conference paper at ICLR 2021
Cont: (LLL)
0.5
Coat. (RGPP)
0
2,000
4,000
20
40
60
80
100
20
40
60
80
100
20
40
60
80
100
(a) MNIST
(b) CIFARIO
(c) SVHN
(d) CIFAR 100
Figure 5: Average confidence function of a. Top: the vanilla LLL. Bottom: LLL with RGPR.
Test data are constructed by scaling the original test sets with a. Error bars are #1 standard devia- tion. Black lines the uniform confidences. MC-integral with 1000 samples is employed.
C.2 HYPERPARAMETER TUNING
have shown in the main text (both theoretically and empirically) that the asymptotic performance of RGPR does not depend on the choice of its hyperparameters (aî ) Indeed we simply set each of to its default value for experiments and showed that RGPR could already fix the asymptotic overconfidence problem effectively.
Nevertheless, Figure 3 gives us a hint that learning these hyperparameters might beneficial for uncertainty estimation. Intuitively, by increasing (aî), one might be able to make the high confi- dence (low uncertainty) region more compact. However, if the values (af) were too large, uncertainty will be high even in the data region, resulting in underconfidenct predictions.
Borrowing the contemporary method robust learning literature (Hendrycks et al., 2019; Hein et al., 2019; Meinke & Hein, 2020, etc.) one way to train (a7 is by using the following min-max objective which intuitively balances high-confidence predictions on inliers and low-confidence predictions on outliers. Let H be the entropy functional, D the training dataset, Dout outlier dataset, o² = (af), and X E R be trade-off parameter. We define:
I((o2)==
H (p(u. (i)) D;o²))
(out) H (ply. (ou)) D;0²));
(12)
m ED
ar
Deu
where the predictive distribution P(y. x. D; x2) is as defined in Section with its dependency to 2² explicitly shown. this paper, for the outlier dataset Daus we use noise dataset constructed by Gaussian blur and contrast scaling proposed by Hein al. (2019). We found that this simple dataset is already sufficient for showing good improvements over the default values (o? = 1). Nevertheless, using more sophisticated outlier datasets, e.g. those used in robust learning literature, could potentially improve the results further. Lastly, we use the trade-off value À = and A 0.75 for experiments with LeNet/ResNet-11 and DenseNet-BC-IBn 121, respectively since found that in the latter architecture generally make the network severely underconfident.
APPENDIX D ADDITIONAL EXPERIMENTS
D.1 CLASSIFICATION
We show the behavior RGPR-imbued image classifier (LLL) in terms of a in Figure 5. While Table has already shown that RGPR makes confidence estimates close uniform, here show that the convergence to low confidence occurred for some small a. Furthermore, notice that when 1, i.e. at the test data, RGPR maintains the high confidence of the base method.
15

---

Under review as a conference paper at ICLR 2021
Table Confidence over test sets (i.e. 1) in term of MMC. Values are averaged over trials. Larger is better.
Methods
MNIST
CIFARIO
SVHN
CIFARI(0
BNO
99.2+0.0
96.9=0.0
98.5+0.0
82.1=0.1
98.5400.0
92.6:1011
91.6:1000
74.6=0.1
L.I-RGPR
97.910.0
92.510.1
91.4110.
73.8=0.1
KFL
92.910.2
86.6.10.1
90.810000
73,4.10.1
KFL-RGPR
91.700.1
86,5+0.11
90.7++0.0
72.8=002
SWAG
87.6+5.0
95.0+1..3
97.7+0.22
$2.9-009
SWAG-RGPR
86.824.0
94.92022
97.6=#0.1
51.2=10
SVDKL
99.6+0.0
97,3=0.00
98.6+0000
80,4=0.2
SVDKI-RGPP
99.6+0.00
97.5++0.0
98.6:t0.0
79.9=0.1
00
130
30
-10
-30
(a) MAP
(b) BNN
(c) BNN-RGPR
Figure 6: Toy regression with BNN and additionally, our RGPR. Shades represent =1 standard deviation.
D.2 REGRESSION
To empirically validate our method and analysis (esp. Proposition 3), present toy regression results in Figure 6. RGPR improves the BNN further: Far-away from the data, the error bar becomes wider.
For more challenging problems, we employ a subset of the standard UCI regression datasets. Our goal here, similar to the classification case, is to compare the uncertainty behavior of RGPR- augmented BNN baselines near the training data (inliers) and far-away from them (outliers). The outlier dataset is constructed by sampling 1000 points from the standard Gaussian and scale them with 2000. Naturally, the metric we choose is the predictive error bar (standard deviation), i.e. same metric used in Figure 1. Following the standard practice (see e.g. Sun et al. (2019)), we use two-layer ReLU network with 50 hidden units. The Bayesian methods used are LLL, KFL, SWAG, and stochastic variational GP (SVGP, Hensman et al., 2015) using 50 inducing points. Fi- nally, standardize the data and the hyperparameter for RGPR is set 0.001 so that RGPR does not incur significant uncertainty on the inliers.
The results are presented in Table 4. We can observe that all RGPRs retain high confidence estimates over inlier data and yield much larger error bar compared to the base methods. Furthermore, as we show in Table 5, the RGPR-augmented methods retain the base methods' predictive performances in terms of test RMSE. All in all, these findings confirm the effectiveness of RGPR in far-away outlier detection.
D.3 NON-ASYMPTOTIC REGIME
Using (12), we show the results of a tuned-RGPR on standard out-of-distribution (OOD) data detec- tion benchmark problems on LeNet/ResNet architecture Tables 2 and Furthermore, we show results for deeper network (121-layer DenseNet-BC) in Table 7. We optimize (af) using Adam with learning rate 0.1 over each validation set and the noise dataset (both contain 2000 points) for 10 epochs. Note that this process is quick since no backpropagation In general tuning the kernel hyperparameters of RGPR lead to significantly lower average confi- dence (MMC) over outliers compared to the vanilla method (LL.L) which leads higher detection
the networks is required.
16

---

Under review as a conference paper at ICLR 2021
Table Regression far-away outlier detection. Values correspond to predictive error bars (averaged over ten trials), similar to what shades represent in Figures and 2. "In" and "Out" correspond to inliers and outliers, respectively.
housing
concrete
energy
wine
Methods
In 4
Out f
In 4
Out f
In
Out
4
Out
LLL
0,405
823,215
0.324
580.616
0.252
319.890
0,126
24.176
L.L-RGPP
0.407
2504.325
0.329
3394,466
0.253
2138.909
0.129
1948.813
KFL
1.171
2996.606
1,281
2518.338
0.651
1486.745 748
475. 141
KFL-RGPR
1.165
3909.140
1.264
4258.177
0.656
2681.780
0.292
2031.481
SWAG
0.181
440.085
1.192
2770.455
0.418
1066.044
0.181
77.357
SWAG-RGPR
0.186
2403,366
1.146
4693.273
0.428
2647.922
0.187
1947.677
SVGP 0.641
2.547
0.845
3.100
0.367
2.237
0.092
0.983
SVGP-RGPR
0.641
1973.506
0.845
1932,061
0.367
1931.299
0.095
1956.027
Table 5: The corresponding predictive performance to Table 4 in terms of the RMSE metric. Values are averaged over ten trials. Smaller is better.
Methods
housing
concrete
energy
wine
LLL
7.36111.050
42.039_16.260
6.228 1. 1.864
0.423
LLI-RGPR
7.361++3.000
62.039::6.200
6.228
0,423-+:0.08
KFL
8.549
42.729=6.355
6,427=1.912
0.421::0.05
KFL-RGPR
8.466
42.608 16,40866,400
6,413.L 1 1.908
0.422.10.055
SWAG
7.265
39.308114.48
2.481 -10.755
0,429-10.047
SWAG-RGPR
7.204
4..459
2.469
:00.600
0.430
SVGP
14.512:14.751
50.657 193
5.697 1: 1..43
0.381110.057
SVGP-RGPR
14,512++4.751
50.687=7.193
5.697 1.443
0.381+00057
performance (AUR). Finally, we show the calibration performance of RGPR on the DenseNet in Table We observe that the base BNN we use, LLL, does not necessarily give good calibration per- formance. Applying RGPR improves this, making LLL better calibrated than the "gold standard" baseline BNO.
We also compare LLL-RGPR to Deep Ensemble (DE) (Lakshminarayanan et al., 2017) which has been shown to perform better compared to Bayesian methods (Ovadia et al., 2019). As we can see in Table 10, LLL.RGPR competitive DE. These results further reinforce finding that RGPR also useful in non-asymptotic regime.
Inspecting the optimal hyperparameters (af). we found that high kernel variances on higher layers tend to be detrimental to the uncertainty estimate, as measured by (12), leading to low variance values those layers, cf. Table 9. Specifically, for the LeNet architecture, found that having high kernel variance on the input (the bottom-most layer) is desirable. Meanwhile, the first residual block and the second dense block are the most impactful in terms of uncertainty estimation for the ResNet and DenseNet architectures, respectively.
17

---

Under review as a conference paper at ICLR 2021
Table 6: OOD data detection results using the hyperparameter tuning objective in (12). All values are averages and standard deviations over 10 trials.
BNO
LLL
LLL-RGPR
Datasets
MMC 4
AUR f
MMC
AUR
MMC
AUR
MNIST
99.2.10.0
98.5.10.0
91.2-10.1
FMNIST
82.340.0
89.2:t..1
70.3+0.0
92.1-1011
54,9=000
91.7=00.1
FMNIST
66,3+0.00
97.4+0.0
56.1-0.0
98.3-0.0
37.0=0.0
98.9=0.0
GrayCIFAROO
48.010..0
99.7.10.0
41.2.10.0
99.7.10.0
24.0..0.0
99,9
UniforNoiss
96.7++0.0
95.2+t0.1
92.3+0.1
94.620.1
43.3-0.2
99.2=0.00
Noise
12.9:0.1
100.0:0.00
12.
100.0
12.
100.0=0.0
SVHN
98.5+0.00
91.6=0.0
88.8=000
CIFARIO
70.9+0.0
95.0+0.0
40.7+00.
97.240.0
36.5=000
97.7=0.0
LSUN
72,2:00.0
95.1
41.5102
97.31000
36.2-0.1
98.0=0.11
CI
71,8:0.00
94.1+10.0
42.11:.00
96.710000
37.7=000
9..
UniformNoise
68.9+0..7
96.6+0.2
40,0=
97.6=0.11
31.7=0.3
98.8=0.0
Noise
66.5.10.5
96.3110.2
36.310.3
97.110.1
3..8
97.
CIFARIOO
82,240.22
74.61102
69.5-0.2
SVHN
46.8:10.0
84.4:0.2
42.7.10.0
80.3.0.22
4..0=0.0
78.1-10.2
LSUN
53.5+0.0
80.3+0.2
3988-0.1
82.6-0.2
33.0-0.1
83,7=0.2
CIFARIO
56.0:10.0
78.1
44.410.0
3.0
79.0--0.2
UniformNoise
31.6-:0.4
93.3+t0.2
21.410.2
94.7=10.2
0.6
99.1
Noise
51.9110.6
81.3.00.
47.510.7
77.1
0.64
7..
Table 7: OOD data detection results using the hyperparameter tuning objective in (12) on DenseNet- BC-121 network. All values are averages and standard deviations over trials.
BNO
LLL
LLL-RGPR
Datasets
MMC 4
AUR
MMC 1
AUR 1
MMC
AUR 1
CIFARIO
96.5.10.1
98.1=10.1
91.110.1
SVHN
86.7.10.0
83.510.11
83.2:10.0
91.3:10.1
68.81000
83.6:10.2
LSUN
78.3+-0.1
89.3+0.1
80.6+0.11
91.3==0.1
48.3=-033
94.5+0.22
CIFARI00
82.410.0
86.010.1
85.2.1000
88.910.1
57.5.10.
90.3
UniformNoise
90.6:t0.3
83.9+0.3
80.2++0.5
94.2-10.1
51.4+0.4
94.9
Noise
75.74:0.6
92.8+0.11
72.1=004
96.5-10.1
57.2-10.3
92.8:00.1
SVHN
89.910.0
97,8.0000
93,910.0
CIFARIO
47.5:t0.0
90.3+
69.5+00.0
94.1
48.9
96.4+:0.0
LSUN
38.7400.1
92,910011
67.7+000
94.7-+0.0
46.12002
97.3+00.1
CIFAR100
48.1.00.0
89.910.0
69.1
93.8
48.5110.0
96.3
UniformNoise
65.9+00.
85.8+0.1
63.6+00.5
97.3+1.11
50.4=0.4
97.2+0.11
Noise
32.0:10.3
95.2110.1
30.7.1033
99.5-10.0
29.2-10.3
99.0:10.1
CIFARI00
85.1#0.1
89,3+0.11
71.020.1
SVIIN
70.8.10.0
72.8.00.2
74.0.10.0
76.1
56.4
67.0.10.1
LSUN
67.5110.1
75.910.2
69.5-10.1
79,9-10.2
37.410.2
84,5:10,2
CIFARIO
72.3+:0.0
70.6:t0.1
74.:10.0
74.9
48.3+10.0
74.8+00.1
UniformNoise
78.8:+0.3
71.020.3
71.2400.
82,4-0.2
42.4-0.2
81.6+0.2
Noise
87.310.2
57.2.044
90.0.102
60.5
76.4.0.44
45.2.10.5
Table 8: Calibration performance of RGPR on DenseNet-BC-12) Values are expected calibra- tion errors (ECEs), averaged over ten prediction runs. RGPR makes the base BNN (LLL) more calibrated- even more than the "gold standard GP" in BNO.
Datasets
BNO
LLL
LLL-RGPR
CIFARIO
17.053++0.637
20.758=+0.445
7,451++0.20
SVHN
6.28140.092
5.705-10.17
4.929-10.110
CIFARI00
22.423-10.416
23.630-10:400
6.244
18

---

Under review as a conference paper at ICLR 2021
Table Optimal hyperparameter for each layer (or residual block and dense block for ResNet and DenseNet, respectively) on LLL.
Datasets
Input
Layer
Layer
Layer
Layer
LeNet & ResNet-18
MNIST
2.596c+02
L4499+000
2.563e-02
2.219c-02
CIFARIO
9.093e-03
3.173e+02
2.178e-01
1.229c-02
7,894e-03
SVHN
9.009c-03
5.559c+02
5.507c=00
1.390c-02
9.039c-03
CIFARIOU
7,897e-03
1.00le402
4.254c-01
9.106c-02
7.455c-03
DenseNet-BC-121
CIFARIO
1.635e-02
1,025e-01
1.102e+03
3.886++00
2.476e+00
SVHN
7.224c-03
3.798c-02
1.113c+02
2.909c+00
7.807c-03
CIFARIO0
1016e-02
5,203c-02
1.233e+02
3.335c+00
6.716c-02
Table 10: Comparison between Deep Ensemble (DE) and LLL-RGPR in terms of AUR. Results for DE are obtained from (Meinke & Hein, 2020) since we use the same networks and training protocol.
Datasets
LLL-RGPR
MNIST
EMNIST
92.1
91.7100.1
FMNIST
99.2
98.9:+0.0
GrayCIFARI0
100.0
99.9
UniformNoisc
97.9
99,2++0.0
Noise
100.0
100.0110.0
CIFARIO
SVHN
90.3
96.0.10.1
LSUN
92.0
93.7=+0.1
CIFARIOU
88.2
88.3:10.1
UniformNoise
96.6
99.8:10.0
Noise
90.3
97.4+-0.1
SVHN
CIFARIO
97.9
97.7+00.0
LSUN
97.9
98.0:10.1
CIFAR100
97.6
97.3.10.0
UniformNoise
95.6
98.8:10.00
Noise
98.2
97.84-0.1
CIFARI00
SVHI
83,2
78.1100.2
LSUN
81.6
83.7++0.2
CIFARIO
76.3
79.0:10.2
UniformNoisc
36.6
99.1+-0.1
Noise
67.5
7..81...5
19


Reviews:
[{"review_id": "uoCK_JRYx05", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Update during rebuttal: I have read other reviews and authors' answers to them. My main concern about the approach to be potentially overcomplicated has been address and I was convinced. I am raising the score for the paper. \n/============================================================================================================ \n\nThe paper addresses the issue of overconfidence of neural network outputs. Pointing to the recent results, the authors state that even Bayesian neural networks (BNNs) are overconfident in their predictions far away from the training data. They propose a method how to fix that. The method is de-facto a simple post-hoc procedure but it is backed up with theory and theoretical guarantees. \n\nStong points:\n* Interesting idea of extending BNNs with GP to address uncertainty in the areas far away from training data\n* The idea is derived theoretically and theoretical guarantees are proved\n* The method is shown to work empirically in rather extensive experiments\n\nWeak points:\n* My biggest and basically only concern about the proposed method is whether it is too complicated. Why would one need to derive GP via these infinite ReLU feature maps? What stops just to declare the usage of cubic spline kernel? This indeed allows the authors to beautifully connect finite BNN with infinite GP with the statement like the last one in the paper, but is there other motivation for that? \nMoreover, if the proposed addition to a pre-trained BNN does not depend on training data, how would it compare to something very rough and straightforward, like just adding to variance of the BNN a function that would cubically grow with the distance away from training data?    \nIn any case, I believe it would be interesting to add this kind of ablation study to empirical comparison. \n\nI am voting to weak acceptance but willing to upgrade the score if the authors provide convincing evidence about necessity of all components of the proposed method.\n\nI am voting to accept because despite of the mentioned above concerns of the method to be overcomplicated, the overall idea is interesting, the paper is well-written and easy to follow, the paper addresses the important problem that has impact for the wide audience, the proposed method provides theoretical guarantees while in practice being used as a simple post-hoc procedure.\n\nQuestions to authors:\nCould you please clarify why you need deriving GP via infinite ReLU maps and why the proposed method would be better than the rough fix to the problem of overconfidence far away from the training data described above?\n\nAdditional comments/suggestions (not too important for evaluation but might be used to improve the paper):\n1.\tReLU is not defined\n2.\tReLU features are only introduced in Section 2.2 although used in Introduction\n3.\tFirst paragraph in Section 2.1. For the full picture, some of the definitions are missing, for example, x, y, N, D, and C\n4.\tMaybe to avoid using expression: “Bayesian methods, which turn standard networks into Bayesian neural networks”\n5.\tSection 3.1. What about zeros? Two kernels covers (-\\infty, 0) and (0, \\infty) where 0 is excluded in both\n6.\tDSCS abbreviation in caption to figure 2 appears before its definition as reference to figure 2 is before this definition\n7.\tFigure 3 is unclear. The authors should probably elaborate the caption.\n8.\tWhat is n in second paragraph in Section 6.1?\n9.\tFigure 4.a – does this mean that all 3 lines coincide? Or where the other 2 lines? If the former, at least a note on this should be included, or better something like using dashed lines and different line widths can make it visible on the plot\n\nMinor:\n1.\tSecond line in Section 4. “(iii)” -> “(ii)”\n2.\tLast line in Section 6.2. “In shown in Table 2” - > “As shown in Table 2”\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "b4cw_57Wn2D", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**SUMMARY**\nThe authors consider the issue of overconfidence in ReLU NN and BNNs, particularly for data that are far (in Euclidean distance) from the training data. They address this by modeling the residual (to the NN) in the latent space with a GP. The kernel for this GP is derived as the limit of infinitely many ReLU-based random features. Specifically, this kernel has the property that it scales cubically with the norm of the input, and so causes large uncertainty away from the origin. Crucially, the GP term changes little from its prior distribution after conditioning on the data, so no expensive inference is required under the approximation made.\n\n**PROS**\nThe authors’ proposal seems to improve the overconfidence of existing methods on the tasks they study. While there is a lot of technical motivation, the paper is easy to follow and results in a simple and efficient modification to existing methods. \n\n**CONS**\nThe main weak point of the paper is the experimental section. The OOD detection task of distinguishing a test set from uniform noise, while perhaps an interesting proof of principle, does not seem very relevant in practice. The results from Table 2 are mixed when compared to the baseline LLL in tasks with real OOD data (as opposed to noise).\n\nSome of the theoretical results and proofs are quite informal or vague. For example in Proposition 1: “are close enough to the origin” \n\n**RECOMMENDATION**\nIn its current form, I vote for rejecting the paper. First, while the theoretical motivation for the method is nice, many approximations are used to produce such a simple form and more analysis is needed to establish what in particular is advantageous about the DSCS kernel over other kernels. Second, the experimental results are quite restricted and should be expanded. While seeing what the method adds to some baselines is very reasonable, it would be useful to have some discussion comparing results to the current SotA. \n\n**ADDITIONAL QUESTIONS**\nThe significance of the kernel derived from the limit of ReLU features is unclear to me. The key properties seem to be that the kernel is small when one argument is near the origin and that it is \\omega(\\alpha^2). Would another kernel with these properties work just as well?\n\nWhat is the baseline performance (in terms of accuracy or likelihood) for the methods considered in Table 1?\n\n**MINOR COMMENTS**\nAs there may be images that are very far apart but are nevertheless quite similar, it’s unclear why Euclidean distance in the input space is important. More motivation for this would be helpful. \n\n- In eq (1), f_\\mu is not defined. Should this be just \\mu?\n- In section 2.2 both d and D are used.\n- Section 3.1: “which is one-zero only on (0, \\inf)” Which argument does this refer to?\n- “the value of k(x*,x*) is cubic in x*”: What does this mean? x* is a vector.\n- The notation f ~ GP(f | mu, V) is unnecessary. f ~ GP(mu, V) is sufficient.\n- “Under this assumption, give an input x* it is clear…”: It does affect the output of the BNN due to the link function.\n- Figure 3d is reference, but does not exist.", "extra_scores": {}, "extra_text": {}}, {"review_id": "1-KP5_GnsC1", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Working under the Bayesian Neural Network setting, the authors proposed a way to inflate the resulting posterior covariance, so as to get improved predictive uncertainty quantification for new data points that are far from the training set, while also maintain similar performance when these new data are close to the training set.\n\nOverall I think the writing is clear, but I had to revisit previous sections in order to tie-up and understand all the various approximation schemes used. I find the derivation of the double-sided cubic spline and the use of infinitely many ReLU's to boost the posterior covariance novel and interesting. This method has potential to be applied to other learning algorithms and get improved confidence statement, such as Variational Bayes where it is known to produce overconfident output.\n\nDespite the authors' claim of doing an extensive theoretical analysis, I find that theoretical arguments quite heuristic in some places. It would be more informative to quantify the approximation errors incurred when using the various approximation methods discussed in this paper, in particular, network linearlization through Taylor's theorem. It seems to me that you just treating the neural network $f$ as a typical differentiable function and ignore the network structure within it by just doing $\\approx$.\n\nThe level-wise RGPR covariance kernel in (6) does not seem to be correct. For level 1, you have \n$\\boldsymbol{h}^{(1)} (\\boldsymbol{x}_{*})$. \n\nFor level 2 however, $\\boldsymbol{h}^{(2)}$ is obtained by $g(W\\boldsymbol{h}^{(1)}+\\boldsymbol{c})$ where $W$ is the weight matrix at level 1, $\\boldsymbol{c}$ is the bias and $g$ is some activation function applied entry-wise, e.g., ReLU. Since the entries of $W$ is part of $\\boldsymbol{\\theta}$ the parameter for the entire network, $W$ is random because $\\boldsymbol{\\theta}$ is assigned a prior. This implies that $\\boldsymbol{h}^{(1)}$ and $\\boldsymbol{h}^{(2)}$ are dependent and likewise for higher levels. Hence the covariance kernel of $\\hat{f}=\\hat{f}^{(0)}+\\cdots+\\hat{f}^{(L-1)}$ is not the just them sum of the individual kernels but something more complicated because they are now dependent due to $\\boldsymbol{h}_{*}$. Can the authors please clarify this?\n\nI find the addition of a kernel function to do correction quite ad-hoc and not very Bayesian. Is it possible to incorporate this term into the prior?\n\nSome other comments:\n1. Section 2.2, line 6, $c_d$ should be $c_D$. Also, is this the same $D$ for the dimension of $\\boldsymbol{\\theta}$ the network weights? Then taking $D\\to\\infty$ means you have infinite weights?\n\n2. (3) does not seem to cover $0$\n\n3. In Proposition 1, $\\boldsymbol{\\mu}$ and $\\mathrm{\\Sigma}$ are the mean and covariance of the approximate posterior of $\\boldsymbol{\\theta}$, and for $\\boldsymbol{g}_{*}$, $\\boldsymbol{0}$ should be $\\boldsymbol{\\mu}$\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agree that the proposed method for reducing overconfidence in ReLU networks is novel and interesting. However, the presentation of the theoretical results is too informal and imprecise to warrant acceptance without a strong accompanying experimental section, which is unfortunately lacking. I therefore cannot recommend acceptance of the paper in its current form.
Decision: Reject