An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
Authors: Agustinus Kristiadi
Matthias Hein
Philipp Hennig
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: bayesian deep learning
uncertainty quantification
overconfidence
gaussian processes

Abstract: A Bayesian treatment can mitigate overconfidence in ReLU nets around the training data. But far away from them, ReLU Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be asymptotically overconfident. This issue arises since the output variance of a BNN with finitely many features is quadratic in the distance from the data region. Meanwhile, Bayesian linear models with ReLU features converge, in the infinite-width limit, to a particular Gaussian process (GP) with a variance that grows cubically so that no asymptotic overconfidence can occur. While this may seem of mostly theoretical interest, in this work, we show that it can be used in practice to the benefit of BNNs. We extend finite ReLU BNNs with infinite ReLU features via the GP and show that the resulting model is asymptotically maximally uncertain far away from the data while the BNNs' predictive power is unaffected near the data. Although the resulting model approximates a full GP posterior, thanks to its structure, it can be applied post-hoc to any pre-trained ReLU BNN at a low cost.

An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence
Agustinus Kristiadi University of Tübingen agustinus krists kwi.ttadi@unniiii tuubaaeetttt-tteeeennnnnn...
Matthias Hein
University of Tübingen
matthias hein@uni
-tuebingen.de www.nhuttttteeuueennnn.....
Philipp Hennig
University of Tübingen and MPI for Intelligent Systems, Tübingen philipp
Abstract
A Bayesian treatment can mitigate overconfidence in ReLU nets around the training data. But far away from them. ReLU Bayesian neural networks (BNNs) can still underestimate uncertainty and thus be asymptotically overconfident. This issue arises since the output variance of a BNN with fimitely many features is quadratic in the distance from the data region. Meanwhile, Bayesian linear models with ReLU features converge, in the infinite-width limit, to a particular Gaussian process (GP) with variance that grows cubically so that no asymptotic overconfidence can occur. While this may seem of mostly theoretical interest, in this work, we show that it can be used in practice the benefit BNNs. We extend finite ReLU BNNs with infinite ReLU features via the GP and show that the resulting model asymptotically maximally uncertain far away from the data while the BNNs' predictive power is unaffected near the data. Although the resulting model approximates a full GP posterior, thanks to its structure, it can be applied post-hoc to any pre- trained ReLU BNN at low cost.
Introduction
Approximate Bayesian methods, which turn neural networks (NNs) into Bayesian neural networks (BNNs), can be used to address the overconfidence issue of NNs [1]. Specifically, Kristiadi et al. [2] recently showed for binary ReLU classification networks that far away from the training data, i.c. when scaling any input with scalar a > 0 and taking the limit a - oc, the confidence of (Gaussian-basee BNNs is strictly less than mee-being Bayesian" can thus mitigate overconfidence. This result is encouraging vis-à-vis standard point-estimated networks, for which Hein et al. [3] showed earlier that the same asymptotic limit always yields arbitrarily high confidence. Nevertheless, BNNs still be asymptotically overconfident, albeit less so than standard NNs, since the aforementioned uncertainty bound can be loose.
We identify that this issue arises because the variance over function outputs of BNN is asymptotically quadratic, while the corresponding mean is asymptotically linear w.r.t. Intuitively, fixing this issue requires adding an unbounded number of ReLU features with increasing distance from the training data, that the output mean stays unchanged but the associated variance grows super-quadratically And indeed there a particular Gaussian process (GP), arising from the cubic spline kernel [4], Bayesian linear model with countably infinite
which has cubic variance growth and can be ReLU features. In the context of the analysis, the fact that standard ReLU BNNs only use finitely many features makes them "miss out" on some uncertainty that should be there. In this work, "add back" this missing uncertainty into finite ReLU BNNs by first extending the cubic spline kernel
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

10
55
-10
10
-3
10
10 -5
10
-5 5 10 10 -5
0
10
(a) K =
(b) K 20
K 60
(d) K
Figure I: The construction of a GP prior with the proposed "ReLU kernel", as the limiting covariance of the output of a Bayesian linear model with K ReLU features (grey), arranged at regular intervals, oriented away from the origin. Red curves are function samples with the thick one being the mean, and the red shade their std. dev. With finite (a-c), the variance grows quadratically, leading to the asymptotic overconfidence in ReLU BNNs. But, with K away from the origin. The fact that this kernel has zero mean and negligible variance near the origin enables to easily combine this GP with standard finite pre-trained ReLU BNNs.
(d), the variance grows cubically
to cover the whole input domain (Fig. 1) and then using the resulting GP model residuals of BNNs [5-7]. Conceptually, we extend finite BNNs into infinite ones. The proposed kernel has two crucial properties: (i) It has negligible values around the origin, which we can assume without loss of generality to be the region where the data reside, and (ii) like the cubic spline kernel, its variance grows cubically in a. Using the first property, we can approximately decompose the resulting posteriori function output simply as a posteriori BNNs' output plus priori the GP's output. This extension can therefore be applied any pre-trained ReLU BNN post-hoc manner. And due to the second property, we can show that the extended BNNs are guaranteed to predict with uniform confidence away from the data. This approach thus fixes ReLU BNNs' asymptotic overconfidence, without affecting the BNNs' predictive mean. Finally, the method can be extended further while still preserving all these properties, by also modeling the representations of input points with the proposed GP. By doing so, the GP can adapt to the data well, and hence also improve the extended ReLU BNNs' non-asymptotic uncertainty.
core contribution of this paper is the theoretical analysis: We show that our method (i) models the uncertainty that ReLU BNNs lack, thus (ii) ensuring that the surrounding output variance asymptotically grows cubically in the distance to the training data, and ultimately (iii) yields uniform asymptotic confidence in the multi-class classification setting. These results extend the prior analysis in so far as is limited to the binary classification case and does not guarantee the asymptotically mexiimum-nntrop prediction. Furthermore, our approach is complementary and Hein [8], which attains maximum uncertainty far from the data for non- non-Bayesian NNs. We empirically confirm the analysis and show effectiveness in the non asymptotii regime.
the method of Meinke
2
Background
Notation We denote a test point and its unknown label as n. and ye. respectively. We denote any quantity that depends with the same subscript, in particular fa == f(xx), k. == k(x,, x.).
2.1 Bayesian Neural Networks
We focus on multi-class classification problems. Let f RN X X RC defined by (x, 0) H f(;; 0) =: fx) be C-class ReLU network-a fully-connected network equipped with the ReLU nonlinearity. Here, 0 is the collection of all parameters of f. Given an i.i.d. dataset D (xm, Fm
convolutional ferw-forward
Bm) J.M the standard training procedure amounts to finding maximum a posteriori (MAP) estimate OMAP = arg max log (( DD)
One can also apply Bayes' theorem to infer the full posterior distribution of @-ehe resulting network called a Bayesian neural network (BNN). A common way to approximate the posterior p(O D) of BNN is by Gaussian g()) == N(μ, E). Given this approximate posterior and test point ar E V the prediction is given by p(y- æ., D) = softmax (ao (()) d0. The Gaussian approximate posterior and softmax likelihood are our assumptions throughout this paper.

---

One can obtain a useful two-step closed form approximation of the previous integral as follows. First, we perform a network finearization on around μ and obtain the following marginal over f(x.):
p(- T.,, D) R NN (x.)) J_ EJ.),
(1)
where J, the D x Jacobian matrix of fo(x..) w.r.. at μ.. For brevity, let m, and V. be the above mean and covariance. To obtain the predictive distribution, then apply the generalized probit approximation [9, 10]:
ply. = c x., D) = = D) df. exp(mac Ree)
(2)
Ei=1 exp(mmx Kas))
where for each i I,.... C, the real number m.; is the i-th component of the vector m. and 7/8 V-is) -1/2 where is the i-th diagonal term of the matrix V.. Both approximations above have been shown to be good both in terms of their errors and predictive performance [10-13].'
(1 :-
While analytically useful, these approximations can be expensive due to the computation of the Jacobian matrix J, Thus, Monte Carlo (MC) integration commonly used an alternative, i.c. we approximate p(y, x., D) 1/5 >.
a- p(y. fa, (x+));; O, ~ q(0), Finally, given classification predictive distribution p(y- F,, D), we define the predictive confidence of as the maximum probability (a..c ply, = e C,, D) over class labels. Far from the data, ideally the model should produce the uniform confidence p(y.
m. D) = 1/C for all c I....,C. C.
2.2 Asymptotic Overvonfidence in BNNs
1.00
Given a fixed point estimate 4MAP, the ReLU network feas yields overconfident predictions, even for points far away from the train- ing data [3]. That is, for almost any input F. E R. one can show that there exists class c E (1...., ,C} such that lima-xo soft (ox.)) = 1. Intuitively, this issue arises because the ReLU network yields a piecewise- finitely many linear regions (the domain of each affine function). Under this setup, by scaling æ, with a, at some point one arrives at an "outer linear region" and in this region, the network is always affine-eitte increasing or decreasing even as a tends to infinity, and thus its softmax output converges to "one-hot vector".
0.75
(a) Zoomed-in
function with
1.00
BNNs, even with a simple Gaussian approximate posterior, can help to mitigate this problem in binary classifications, as shown by Kristiadi et al. [2]- The crux of their proof is the observation that
(b) Zoomed-out
Figure 2: Confidence esti-
mates of BNN.
in outer linear region, the predictive distribution (via the probit approximation) is given by²
au T x=
ply. ax.. D) D) R8
V1 1 1/880(03)))
where is the logistic-sigmoid function, is the parameter vector corresponding to the linear region and the quadratic function v maps az. to the variance of the network output. Unfortunately, both the numerator and denominator above are linear in and thus altogether p(y. only converges to a constant strictly less than 1 as a + co, not necessarily the ideal uniform confidence prediction. BNNs can therefore still be overconfident, albeit less so than the point- estimated counterpart (Fig. 2).
oz,, D)
2.3 ReLU and Gaussian processes
The ReLUU activation function ReLU(z) == max(0, 2) [14] has become the de facto choice of non- linearity in deep learning. Given an arbitrary real number c, it can be generalized as ReI.U(2;c) =
'The network linearization comes with the error of O(Il0 6MsPll?) [)) by Taylor's theorem. Meanwhile the (generalized) probit approximation, low empirical error has heen observed by [10, Fig. 1]. We omit the bias parameter for simplicity.
3

---

max(0, c), with the "kink" at location An alternative formulation, useful below, is in terms of the Heaviside function H as ReLU(2;;) c) H( c) (z -ec-c) c). We may define collection of K such ReLU functions evaluated at some point in R as the function Φ IR - R& with 2+ (ReI.U(2; Re cx)) We call this function the ReLU feature map, which can be interpreted as "placing" ReLU functions at different locations in IR.
Consider a linear model g : R x I E defined by g(x; w) :- w ((x).. Suppose & regularly places K generalized ReLU functions centered at (ci) on [Cmin Cmax] C R. where Cmin < Cmax- If we consider a Gaussian prior p(w) == . w 0, o² K
Gmin I), then as K - oc, the distribution over g Gaussian process with mean 0 and covariance (full derivation in Appendix A):
Rel (zz a'; Cmin: 0²) = H² - Cmin) (5 min) - - (I x') + (5 - Cmin) )a:r)
Here, the superscript denotes the fact that this function is over a 1-dimensional input space and == min(x,,") x'). Since the expression above does not depend Cmax, we can consider the limit Cmax + o0, and thus this kernel is non-zero on (Cmin: X0). This covariance function is the cubic spline kernel [4]. The name indicates that posterior mean of the associated GP is piecewise-cubic.
But it also has variance 1:2 (ar, I; Cmin; x²) which is cubic in x and negligible for x close to Cmin-
3 Infinite-Featury Extension for ReLU BNNs
From Section 2.2 it becomes clear that the asymptotic miscalibration of ReLU BNNs is due to the finite number of ReLU features used, which results in only quadratic variance growth. An infinite-ReLU GP with the cubic spline kernel has cubic variance growth, which, combined with the probit approximation, yields uniform confidence in the limit. But of course, full GP inference is prohibitively expensive. In this section, we propose a cheap, post-hoc way to extend any pre-traines ReLU BNN with the aforementioned GP by extending the cubic spline kernel and exploiting its two important properties. We will see that the resulting model approximates the full GP posterior and combines the predictive power of the BNN with a guarantee for asymptotically uniform confidence. While in our analysis we employ network linearization for analytical tractability, the method can be applied via MC-integration well (cf.. Section 5). All proofs are in Appendix B.
3.1 The Double-Sided Cubic Spline Kernel
The cubic spline kernel is one-sided in the sense that it has zero variance on (-o0, Cmin), and therefore unsuitable for modeling over the entire domain. This is easy to fix by first setting Cuin to obtain a kernel K ((x,a''o²)) = x';
which is non-zero only on (o). Now, by
an entirely analogous construction with infinitely many ReLU functions pointing to the opposite direction (i.e. left) via ReLU(-2; c). we obtain another kernel To (Ix x': ::);=====)= := K. '-=22--0 =z';a²)
which is non-zero only (-o0, 0). Combining them together, we obtain the following kernel, which
covers the whole real line: k² (xx (x,=x'; x'; g²) := k (x, 0²) + K (x, o'; Fig. 1. Note in particular that the variance k= 0) the origin zero. This key feature of this kernel that enables us to efficiently combine the resulting GP prior with pre-trained BNN.
For multivariate input domains, we define
N Σ
1 N
k(0, 0²) =
x (2,,
(4)
for any x, x' E N with 1. We here deliberately use summation, instead of the alternative of a product, since we want the associated GP to add uncertainty whenever at least one input dimension has non-zero value. (By contrast, a product k( x') is zero one of the k (x,, this kernel the double-sided cubic spline (DSCS) kernel. Similar to the one-dimensional case, two crucial properties of this kernel are that it has negligible variance around the origin of R and for any R.y and R, the value k(ax, ax.) ) is cubic in a.
zero.) We call

---

3.2 ReLU-GP Residual
For simplicity, we start with real-valued BNNs and discuss the generalization output later. Let f IN x ID" + R be an L-layer, real-valued ReLU BNN. Since f by itself can be asymptotically overconfident, has residual in its uncertainty estimates far from the data. Our goal is to extend f with the GP prior that arises from the DSCS kernel, to model this uncertainty residual. We do so by placing infinitely many ReLU features over its input space TEV by following the DSCS kernel construction in the previous section. Then, we arrive at a zero-me zero-mean mean GP prior GP( 0, k) over real-valued random function ʃ RRN GP prior to model the residual of f by defining
multi-diiennsioal
N - R. Following previous works [15, 6, 7], we use this
f=
where GP(OA)).
(5)
and call this method ReLU-GP residual (RGPR).
We now analyze RGPR. Besides linearization, we assume that the DSCS kernel has, without loss of generality, a negligibly small value at the data, i.e. k(=m, x.) z O for all (xm)m and any i..d. test point æ.. Note that this can always be satisfied by centering and scaling. The error of this approximation is stated in the following.
Lemma 1. Let 0 8 < and let o² be constant. For any w, x' E IRN with ||a| ||-1| we have k(æ, x'; o²) E O(8)).
Using this approximation, we show the approximate GP posterior of J.
Proposition (RGPR's GP Posterior). Let f RN x RDD + R be a ReLU BNN with weight distribution N(e μ. Σ), and let D := (Im 3m M =: (X.y) y) be dataset. Assume that | 1,2 || || S 8 for all m I....,M .M and any i.i.d. test point given an ii.dd input point x, E R. .under the linearization of f w.r.t. O around the GP posterior
EN with 8 < Then
over fo is Gaussian with mean and variance
B(F. D) = f(x.; +) hh] f(x;
(6)
Var (F. D) = g(x.) [g([) k(x,, - h- c-
(7)
respectively, where h. = (Cov (C)) ((mn)) ( ((TM))) while C is the co- variance matrix (Cov(f(a:;), (ax ))) over, the approximation error in (6) is in O [|-]]--]]] - where m = ((X: while the error in (7) is in O ((55 +| +- (=1()=1
and f(X: = (J(01) (BMii))) More-
where m = C- h...
While this result is applicable to any Gaussian weight distribution, an interesting special case is where we assume that the BNN is well-trained, i.c. we have a Gaussian (approximate) posterior p(0 (D) which induces accurate prediction and high output contidence on each of the training data, In this case, the last term of (6) is negligible since the residual y - f(X; μ1) is close zero. Moreover, notice that the last term in (7) can be upper-bounded by
h h- Amax 11.11 1
Amax Σ Cov(J( (amm))
where Amax denotes the largest eigenvalue of C-1, The last summand above can further be upper-
bounded via the Cauchy-Schwarz inequality by Cov(f f2²m) < Var(f(x.) (x)))))))))))) Var[ (xm)) But our assumption implies that pre-trained ReLU BNN, we approximately have
is close to zero for all 1,..., M. Thus, f is
J. ~ ((f(;()) g. Eg. + k:),
(8)
which can be thought of as arising from the sum of two Gaussian r.v.s. J. ~ ~N[ (L); g.. Eg.) and f. (O k. -- are back to the definition of RGPR (5). Thus, unlike previous works on modeling residuals with GPs [15, 6, 7], the GP posterior of RGPR can approximately be written as posteriori f plus priori f. RGPR can hence be applied post-hoc, after the usual training process of
5

---

oo, and the BNN. Furthermore, we see that RGPR does indeed model only the uncertainty residual of the BNN since it only affects the predictive variance. In particular, it does not affect the output mean of the BNN and thus preserves its predictive accuracy- this is often desirable in practice since the main reason for using deep ReL.U nets due to their accurate predictions.
Generalization to BNNs with multiple outputs is straightforward. Let f RN x RD + EC be vector-valued, pre-trained, L-layer ReLU BNN with posterior (O μt, 2). We assume that the following real-valued random functions (F(c) R - R) are ii.d. as the GP prior GP(O, k) (5). Thus, for any a. RR defining T. == (f A we have p(f.) NO k. k.J), and so under the linearization of f. this implies that the marginal GP posterior of RGPR is approximately given by the following C'-variate Gaussian
p(f x.. ~ (f
EJ. k.I).
(9)
We can do so since intuitively (9) is simply obtained as a result of "stacking" C independent 7. each of which satisfies Proposition 2. The following lemma shows that asymptotically, the marginal variances of fo grow cubically as we scale the test point.
Lemma 3 (Asymptotic Variance Growth). Let f. : RN x XRD + RC be a pre-trained ReLU network with posterior N(O f.. E) and J be obtained from J via RGPR. Suppose that the linearization of w.r.t. around μu is employed. For any æ= E I with æ, # there exists B > such that for any 2 3 and each c = C, the variance Var( (fo (axx)) under (9) is in ((a²).
Equipped with this result, are now ready state our main result. The following theorem shows that RGPR yields the ideal asymptotic uniform confidence of 1/C' given any pre-trained ReLU classification BNN with an arbitrary number of classes.
Theorem 4 (Uniform Asymptotic Confidence). Let f RN x RDD + RC be a C-class pre- trained ReL. network equipped with the posterior N( E) and let I be obtained from I via RGPR. Suppose that the linearization of f and the generalized probit approximation (2) is used for approximating the predictive distribution p(y. = axx, F.D) under f. For any input xs E RN with v. + O for every class = I,...,C, we have lima-to p(y- = c oz D) 1/C.
As a sketch of the proof for this theorem, consider the special case of binary classification. Here, we notice that the variance e in the probit approximation (3) is now cubic function of under RGPR, due to Lemma 3. Thus, it is easy to see that the term inside of a decays like 1/va far away from the training data. Therefore, in this case, p(y 1 aæ., D) evaluates (0) = 1/2 hence we obtain the asymptotic maximum entropy prediction.
We remark that the pre-trained assumption on ʃ in Lemma 3 and Theorem 4 can be removed. Intuitively, this is because under the scaling of a on T., the last term of (7) is in O(a?). Thus, asymptotically dominated by the ( growth induced by the DSCS kernel in the second term. We however present the statements as they are since they support the post-hoe spirit RGPR.
3.3 Extending RGPR to Non-Asymptotic Regimes
120
While the previous construction is sufficient for modeling uncertainty far away from the data, does not necessarily model the uncertainty near the data region well. Figure 3(a) shows this behavior: the variance of the GP prior equipped with the DSCS kernel grows slowly around the data and hence, even though Theorem will still apply in the limit, RGPR has a minimal effect on the uncertainty of the BNN in nom-asympootic regimes.
90
60
30
(a) Input Only
way to address this is to adapt RGPR's notion of proximity be- tween input points. This can be done by using the higher-level data representations already available from the pre-trained NN-a test point close to the data in the input space can be far from them in the representation space, thereby the DSCS kernel might assign a large variance. Based on this intuition, we extend RGPR by additionally
30
(b) Input & Hiddens
Figure 3: Variance of f.

---

placing infinite ReLU features on the representation spaces of the point-estimated network fr induced by the BNN f. where μl is the mean of the Gaussian posterior of f. as follows.
For each .,..., 1 and any imput 2,, let N₁ be the size of the I-th hidden layer fu and h be the I-th hidden representation of æ.. By convention, we assume that No- N and h a... an infinite number of ReLU features on the representation
Now, we place for each I = O, ... ,, space RM, and thus we obtain random function f()) E - I distributed as (P(O, k). Then, given that Ñ == E N₁, we define f RR R by : f f f i.e. we assume that ) are independent. This function is therefore a function over all representation (including the imput) spaces of fu. distributed the additive Gaussian process GP(O, E/ k). In other words, given all representations h. := (h: )=== of a. under Jn.. the marginal over the function output fh.. is given by
M(f.) - N (..
(0.
Σ 1=0
L-1
1 (
k
(10)
We can then use this definition of J as a drop-in replacement in (5) to define RGPR. Figure 3(b) visualizes the effect: the low-variance region modeled by ʃ becomes more compact around the data.
The analysis from the previous section still applies here since it is easy to see that the variance of f. in (10) is still cubic in a. In practice, however, it is not necessarily true anymore that each h is close to the origin in RR- To fix this, one can center and scale each h via standardization using the mean Exe D (h (1) (x)) and scaled standard deviation r'v Var D ( (x)) with r > 1, before evaluating the kemnel in (10) (these quantities only need to be computed once). Note that by tuning the DSCS kernel's 's hyperparameter o² such that confidence over the training data is preserved (cf. the next section), RGPR becomes insensitive to the choice of since intuitively the tuning procedure will make sure that the DSCS kernel does not assign large variance to the training data. Therefore, in practice set
Algorithm provides pseudocode of RGPR for classification predictions via MC-integration. The only overhead compared to the usual MC-integrated BNN prediction step are (marked in red) (i) single additional forward-pass over fu, (ii) L evaluations of the DSCS kernel k, and (iii) sampling from a C-dimensional diagonal Gaussian. Their costs negligible compared to the cost of obtaining the standard MC- -CC-rrdictionn of f. which, in particular, requires multiple forward passes.
3.4 Hyperparameter Tuning
The kemell hyperparameters (ot) =: 2² control the variance growth of the DSCS kernel. Since RGPR is a GP model, one way to tune Ø² via marginal likelihood maximization. How-
Algorithm 1 MC-prediction for RGPR. Differ- ences from the standard procedure are in red.
Input:
ever, this leads to an expensive procedure even a stochastic approximation [16] is employed since the computation of the RGPR kernel (9) requires the network's ' Jacobian and the explicit kernel matrix need to be formed. Note how- ever that those quantities are not needed for the computation of the predictive distribution via MC-integration (Algorithm 1). Hence, a cheaper yet still valid option to tune x² is to use a cross-validation (CV) which depends only on predictions over validation data Dial [17, Ch. 5).
Pre-trained I.-layer, ReI.U BNN classifier of with posterior ( μr, E). Test point RR Cen-
tering and scaling function std. Hyperparameters
(ož) (of)A-1 Number MC samples S.
1: (h. M=1 forward(fa, v.)
2: U. (x.)) = Σ k(sta(h.)) (), std(h! ooff)
3: for = 1,..., do
4:
0. ~ (N(
5:
(x.) =f(x.;0,)
6:
[.. ~ N/(0, (w.))
7: f. = fz(m.) fo (x..)
8: end for
straightforward way to perform CV is by maximizing the validation log- logeliiel (LL). That is, we maximize the objective (
return 1/5 softmax (f.())))
Ex.y..D. ED.. log p(y, | xx, D; o?). However, this tends to yield overconfident results outside the training data (Fig. 4). Thus, similar to Kristiadi al. [2], we optionally add an auxiliary
7

---

0.50
Untuned
(b) Marg. Likelihond
CV with Lu
(d) CV with Loon
Figure 4: Different objectives for tuning σ². Shades predictive confidence. Untuned o²
(1,..., 1) in (a). Dont consists uniform noise images.
term to LL. that depends on some OOD dataset Dout, resulting LOOD(O?) == (II(O²) E.C... CD. Eas1 logp(y = C., D; )²). In particular, the additional term is simply the negative cross-entropy between the predictive distribution and the uniform probability vector of length C, with A = 0.5 as proposed by Hendryeks et al. [18]. Note that both objectives can be optimized via gradient descent without the need of backprop through the network. See Fig. 4 for comparison between different objectives. In Section 5, we discuss the choice of Doun-
4 Related work
Mitigation of asymptotic overconfidence has been studied recently: Hein et al. [3] noted, demon- strated, and analyzed this issue, but their proposed method does not work for large a. Kristiadi et al. [2] showed that Bayesian treatment could mitigate this issue even as a - co. However, their analysis restricted to binary classification and the asymptotic confidence standard ReLU BNNs only converges to constant in (0, 1). In Bayesiann framework, Meinke and Hein [8] used density estimation achieve the uniform confidence far away from the data. Nevertheless, this property has not been previously achieved in the context of BNNs.
Unlike line of works that connects NNs and GPs [19-21, etc.] which studies properties of NNs as GPs in an infinite-width limit, we focus on combining finite-width BNNs with GP a posteriori. Though similar in spirit, method thus differs from Wilson al. [22] which propose combination of a weight-space prior and a function-space posterior for efficient GP posterior sampling. Our method is also distinct from other methods that model the residual of predictive model with GP [5, 15, 6, 7, etc.] since RGPR models the uncertainty residual of BNNs, in contrast to the predictive residual of point-estimated networks, and RGPR does not require further posterior inference given pre-trained BNN.
Cho and Saul [19] proposed family of kernels for deep learning, called the are-cosine kemnlss. The first-order arc-cosine kernel can be interpreted as a ReL.U kemel but only has quadratic variance growth and thus is not suitable guarantee the uniform asymptotic confidence. While higher-order arc-cosine kernels have super-quadratic variance growth, they ultimately cannot be interpreted as ReLU kernels, and hence are not as natural as the cubic-spline kernel in the context of ReLU BNNs.
Empirical Evaluations
We empirically validate Theorem in the asymptotic regime and the effect of RGPR asymptotic confidence estimates in multi-class image classification. The LeNet architecture [23] is used for MNIST, while ResNet-18 [24] is used for CIFARIO, SVHN, and CIFAR 100-R in Appendix D. For each dataset, we tune o² via a validation set of size 2000 obtained by splitting the corresponding test set. Following Hein al. [3], Dour consists of smoothed noise images, which are obtained via random permutation. blurring, and contrast rescaling of the original dataset- they do not preserve the structure the original images and thus can be considered synthetic noise images. Particularly for ResNet, we use the outputs of its residual blocks to obtain input representations h..
5.1 Asymptotic Regime
In this experiment, we use the last-layer Laplace approximation (LLL.) the base BNN, which has previously been shown to be a strong baseline [2]. Results with other, more sophisticated BNNs

---

Confidence
HNN-RCIP
Unileem
0.5
80
100
20
40
100
40 60
SU
100
20
100
(a) MNIST
(b) CIFARIO
SVHIN
(d) CIFARI00
Figure 5: Confidence of vanilla BNN (LLL) and the same BNN with RGPR, as a function of a. Test data are constructed by scaling the original test set. Curves are means, shades are 1 std. devs. Note that in (b) and (d), even though close, the BNN does not achieve the uniform confidence.
NLL
Brier
Accuracy
Confidence +
6
1.5
0.8
0.8
4
0.6
0.6
2
0.5
0.2
0.2
0
0
50
100
150
50
100
150
50
100
150
0
50
100
150
1.00
Temp. Sealing
Doe Bas.
0.75
0.50
LLL REL-RGPRROO OOD
0.00
NLL
Brier
ECE
MMC
Acc
Metric
Figure 6: (Top) Rotated- (æ-axes are rotation angles). (Bottom) Corrupted- -Ineed---FRRROO values normalized to [0, 1] and are averages over all types of corruption and all severity levels.
[25-27] are Appendix D-wwe observe similar results there. Figure 5 shows confidence estimates of both the BNN and the RGPR-imbued BNN over 1000 samples obtained from each of MNIST, CIFARIO, SVHN, and CIFAR 100 test sets, as the scaling factor a increases. As expected, the vanilla BNN does not achieve the ideal uniform confidence prediction, even for large a. This issue is most pronounced on MNIST, where the confidence estimates are far away from the ideal confidence of 0.1. Overall, this observation validates the hypothesis that BNNs have residual uncertainty, leading to asymptotic overconfidence that can be severe. We confirm that RGPR fixes this issue. Moreover, its convergence appears at a finite, small a; without pronounced effect on the original confidence.
5.2 Non-Asymptotic Regime
We report results standard dataset shift and
Table OOD data detection in of FPR #95. All values are in percent and averages over five OOD sets and prediction runs.
out-of-distribution (OOD) detection tasks. For the former, and CIFAR10-C datasets [28, 29] and mea- sure the performance using the following met- rics: negative log-likoo liglliill (NLL), the Brier score, expected calibration error (ECE), accu- racy, and average confidence. Meanwhile, for OOD detection, we use five OOD sets for each in-distribution dataset. The FPR @95 metric measures the false positive rate an OOD tector at 95% true positive rate. We use L.I.. as the base BNN for RGPR and compare against the MAP-trained network, temperature scaling [TS, 30), the method of Qiu et al. [7] with the DSCS kernel (GP-DSCS, see Appendix C), and Deep
the standard rotAted-MNIST
Methods
MNIST
CIFARIO
SVHN
CIFAR100
MAP TS
28.2
38.9
17.8
72.2
28.4
34.9
17.6
71.9
DSCS
27.8
46.7
19.1
69.1
29.8
LLL-RGPR-LL LLL-RGPR-OOD
3.9
29.6
13.8
65.8
24.2
63.0

---

Ensemble [DE, 31], which is strong baseline in this regime [28]. We denote the RGPR tuned via Cu. and LOOD with the suffixes "-LL" and "-OOD", respectively. More results are in Appendix D.
On the rotAted-MNIST benchmark, we observe in Fig. that RGPR consistently improves the base LLI.,, especially when tuned with LOOD. while still preserving the calibration of L.L. on the clean data. LLL-RGPR attains better results than GP-DSCS, which confirms that applying GP on top of a trained BNN is more effective than on top of MAP-trained nets. Some improvements, albeit less pronounced (see Table 3 in the appendix for the complementary numerical values), are also observed in CIFARIO-C. For OOD detection (Table 1) we find that LLL is already competitive with all baselines, but RGPR can still improve it further, making it better than Deep Ensemble. Further results comparing RGPR To recent non- -on-Bayesi baselines [32, 33] are Appendix D.
Finally, we discuss the limitation of LOOD- While the use of additional OOD data in tuning o² improves both dataset-shift and OOD detection results, it is not without drawback: LOOD induces slightly worse calibration in terms of ECE (Table 6 in Appendix D). This implies that one can somewhat trade the exactness of RGPR (as assumed by Proposition 2) off with better OOD detection. This trade-off' is expected to a degree since OOD data are often close to the training data. Hence, the single multiplicative hyperparameter of of each the DSCS kernel in (10) cannot simultaneously induce high variance on outliers and low variance on the nearby training data. Table 10 (Appendix D) corroborates this: When a Dour "closer" to the training data (the x2x32 ImageNet dataset [34]) is used, the ECE values induced by LOOD become worse (but the OOD performance improves further). Note that this negative correlation between ECE and OOD detection performance also presents in ofte-offtthe-at OOD detectors (Section D.3.5). So, if the in-distribution calibration performance is more crucial in applications of interest, LL a better choice for tuning σ² since it still gives benefits on non-asymptotic outliers, but preserves calibration better than LOOD-
6 Conclusion
Extending finite ReLU BNNs with an infinite set of additional, carcfully placed ReLU features fixes their asymptotic overconfidence. We do by generalizing the classic cubic spline kernel, which, when used in a GP prior, yields a marginal variance growing cubically in the distance between a test point and the training data. The simplicity of our method is its main strength: RGPR causes no additional overhead during BNNs' training, but nevertheless meaningfully approximates a full GP posterior, because the proposed kernel contributes only negligible prior variance near the training data. RGPR can thus be applied post-hoc to any pre-trained ReLU BNN and causes only a small overhead during prediction. We also showed how RGPR can be extended further-again in a post-hoc manner-to also correct the BNN's uncertainty near the training data, by modeling residuals in the higher layers of the network. The intuition behind RGPR relatively simple, but it bridges the domains of deep learning and non- parametric/kernel models: Correctly modeling uncertainty across the input domain requires non- non-parametric model of infinitely many ReLU features, but only finitely many such features need to be trained to make good point predictions.
Acknowledgments and Disclosure of Funding
The authors gratefully acknowledge financial support by the European Research Council through ERC StG Action 757275 / PANAMA; the DFG Cluster of Excellence "Machine Learning New Perspectives for Science", EXC 2064/1, project number 390727645; the German Federal Ministry of Education and Research (BMBF) through the Tübingen AI Center (FKZ: 01IS 18039A); and funds from the Ministry of Science, Research and Arts of the State of Baden-Württenn Württemberg, AK is grateful to the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for support. AK is also grateful to Felix Dangel, Jonathan Wenger, Nathanael Bosch, Runa Eschenhagen, Christian Fröhlich, and other members of the Methods of Machine Learning group for feedback.
10

---

References
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep Neural Networks Are Easily Fooled: High Confidence Predictions for Unrecognizable Images. In CVPR, 2015.
[2] Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being Bayesian, Even Just Bit, Fixes Overconfi- dence ReLU Networks. ICML, 2020.
[3] Matthias Hein. Maksym Andriushchenko, and Julian Bitterwolf. Why ReLU Networks Yield High- Confidence Predictions Far Away from the Training Data and How to Mitigate the Problem. In CVPR, 2019.
[4] Grace Wahha. Spline Models for Observational Data. SIAM, 1990.
[5] BJN Blight and L Ott. A Bayesian Approach to Model Inadequacy for Polynomial Regression. Biometrika, 62, 1975.
[6] Anthony O'Hagan. Curve Fitting and Optimal Design for Prediction. Journal of the Royal Statistical Society: Series B (Methodological), 40, 1978.
[7] Xin Qiu, Elliot Meyerson, and Risto Müikkulainen. Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with I/O Kernel. In ICLR, 2020.
[8] Alexander Meinke and Matthias Hein. Towards Neural Networks That Provably Know When They Don't Know. In ICIRR 2020.
[9] Mark N Gibbs. Bayesian Gaussian Processes for Regression and Classification. Ph. D. Thesis, Department of Physics, University of Cambridge, 1997.
[10] David JC MacKay, The Evidence Framework Applied to Classification Networks, Neural computation. 1992.
[11] Andrew YK Foong, Yingzhen Li, José Miguel Hernández-Lobato, and Richard E Turner. 'In-Between' Uncertainty in Bayesian Neural Networks. arXiv,, 2019.
[12] Alexander Immer, Maciej Korzepa, and Matthias Bauer. Improving Predictions of Bayesian Neural Networks Via Local Lincarization. In AISTATS, 2020.
[13] Zhiyun Lu, Eugene Ie, and Fei Sha. Uncertainty Estimation with Infinitesimal Jackknife, Its Distribution and Mean-Field Approximation. arXiv preprint 2006::0006075884 2020.
[14] Vinod Nair and Geoffrey E Hinton. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML, 2010.
[15] Grace Wahba. Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression. Journal of the Royal Statistical Society: Series B (Methodological), 40, 1978.
[16] James Hensman, Alexander Matthews, and Zoubin Ghahramani. Scalable Variational Gaussian Process Classification. AISTATS, 2015.
[17] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
[18] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep Anomaly Detection with Outlier Exposure. In ICLR, 2019.
[19] Youngmin Cho and Lawrence Saul. Kernel Methods for Deep Learning. NIPS, 2009.
[20] Jaehoon Lee. Yasaman Bahri, Roman Novak, Samuel Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein. Deep Neural Networks as Gaussian Processes. In ICLR, 2018.
[21] Mohammad Emtiyaz E Khan, Alexander Immer, Ehsan Abedi, and Maciej Korzepa. Approximate Inference Turns Deep Networks Into Gaussian Processes. In NeurlPS, 2019.
[22] James Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Mare Peter Deisenroth. Efficiently Sampling Functions from Gaussian Process Posteriors. In ICML, 2020.
[23] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11), 1998.
11

---

[24] Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In CVPR. 2016.
[25] Hippolyt Ritter, Aleksandar Botev, and David Barber. A Scalable Laplace Approximation for Neural Networks. In ICLR, 2018.
[26] Wesley J Maddox. Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A Simple Baseline for Bayesian Uncertainty Deep Learning. In NeurlPS, 2019.
[27] Andrew G Wilson, Zhiting Hu, Russ R Salakhundinov, and Eric P Xing. Stochastic Variational Deep Kemel Learning. In NIPS, 2016.
[28] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty under Dataset Shift. In NeurfPS, 2019.
[29] Dan Hendrycks and Thomas Dietterich. Benchmarking Neural Network Robustness Common Corrup- tions and Perturbations. ICLR, 2019.
[30] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. In JCML, 2017.
[31] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In NIPS, 2017.
[32] Kimin Lee, Kihok Lee, Honglak Lee, and Jinwoo Shin. A Simple Unified Framework for Detecting cittrf-Distributionn Samples and Adversarial Attacks, NIPS, 2018.
[33] Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty Estimation Using Single Deep Deterministic Neural Network. In JCML,, 2020.
[34] Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. A Downsampled Variant of ImageNet as an Alternative the CIFAR Datasets. arXiv preprint arXiv:1707.088// 2017.
[35] Nicholas Higham. A Survey of Componentwise Perturbation Theory, volume 48. American Mathematical Society, 1994.
[36] Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
[37] Raman Arora, Amitabh Basu, Poorya Mianjy. and Anirbit Mukherjee. Understanding Deep Neural Networks with Rectified Linear Units. ICLR, 2018.
[38] Dan Hendrycks and Kevin Gimpel. A Baseline for Detecting Misclassified and Examples in Neural Networks. In ICLR, 2017.
[39] Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional Variational Bayesian Neural Networks. In ICLR, 2019.
12


Reviews:
[{"review_id": "XkGVWTX7UCB", "reviewer": "Reviewer_h4G7", "summary": "This method analyses overconfidence in ReLU BNNs and introduces an 'infinite-feature' extension that corrects overconfidence far away from training data. The authors propose a method to post-hoc correct trained overconfident ReLU BNNs by adding a mean zero GP to the trained BNN, which has approximately zero variance on the training data, but has variance that increase cubically with distance from the training data, which is enough to guarantee uniform confidence predictions. The method is also adds minimal computational cost to standard evaluations of BNN predictive variance (which require multiple forward passes). Experimental evidence is provided both on 2d visualisations and standard image classification tasks which demonstrate improved performance over standard baselines.\n\n*Update after rebuttal*\nI am raising my score to a 7 due to the additional comparisons to baselines that the authors are including", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*originality* This work builds on previous works looking at correcting overconfidence in ReLU BNNs. Whereas other works were only able to introduce methods that show that asymptotically away from the training data they will not have certain confidence, this submission provable has uniform confidence (theorem 3), which is more desirable.\n\n*quality* The paper is well written and proposes a sensible idea for an important question (overconfidence on ood in BNNs), which is both theoretically and experimentally shown to improve performance. It's also relatively cheap to implement, which is a plus.\n\n*clarity* The paper is well written and clear, especially with regards to flow of its arguments and motivation for the problem they consider.\n\n*significance* Overconfidence and poor uncertainty estimates in BNNs is certainly an issue which I believe is of interest to the wider NeurIPS community. This paper introduces a solid contribution to correct this issue. I have doubts whether this method would become a commonly used post-hoc technique, especially with regards to the difficulty in tuning the introduced $\\sigma$ parameter.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- As unavoidable in a lot of these uncertainty methods is the problem of tuning hyperparameters and indeed it seems like this method also has such difficulties as described in sec 3.4 (even the proposed L_OOD in fig 4d doesn't look great). I appreciate the authors for their honesty and clarity in this limitation.\n- It's not clear that the choices of using the hidden layer activations in section 3.3 to better model uncertainty in deep networks is well motivated. You could pick a lot of things here, e.g. the neural tangent kernel features or restrict to the later layers where we have 'richer' features. It would be good to see some ablations here as to how this choice affects the inference of your methods.\n- It would improve the paper to see some experimental comparison, in e.g. Table 1, to more recent methods in uncertainty quantification for DNNs e.g. DUQ https://arxiv.org/abs/2003.02037 or SNGP https://arxiv.org/abs/2006.10108, especially as they only require a single forward pass at test time.", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "OOZG_wQFh-J", "reviewer": "Reviewer_z1y4", "summary": "The paper proposes addresses the issue of asymptotic over-confidence of Bayesian ReLU neural networks. The idea is to add a Gaussian process with a cubic spline kernel to the GP, which ensures quick (cubic) growth of uncertainty as we move away from the training data. The authors also extend the method by computing the GP on top of the latent features extracted by the BNN, which improves uncertainty near the data. Finally, the authors provide experiments, showing promising results for calibration and OOD detection with the proposed method.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The paper is based on earlier work [1, 2], but uses a novel idea of applying GPs to increase the rate of uncertainty growth in BNNs. The idea of applying GPs on top of intermediate layers for OOD detection is novel and interesting, but related to the paper [3] which I suggest the authors include in the related work.\n\n\nQuality: The paper overall high quality. The theory is well-supported by empirical experiments. The methods are simple, but do their job reasonably well.\n\nExperiments in the paper are sufficient, but not outstanding. In particular, the authors do not compare to any OOD detection baselines, such as [3]. If this paper was presented as a practical method for OOD detection, I would argue that the experiments are insufficient, but I think they are reasonable given the positioning of the paper.\n\nOne downside of the paper (shared with [2]) is that the term BNN seems to be equated with a neural network with a Gaussian distribution over the parameters. In particular, I don't think anything in the paper depends on the distribution being related to the actual BNN posterior in any way. It would also be interesting to see experiments with more precise BNN baselines than last-layer Laplace approximation, e.g. [4]. \n\nCould you please comment on whether or not everything in the paper applies to regular ReLU NNs and not just BNNs?\n\n\nClarity: The paper is well-written.\n\n\nSignificance: I think the significance of this work is somewhat limited, it is unclear how important the asymptotic overconfidence of BNNs is, and how pronounced it is in actual BNNs (i.e. with more precise posterior approximation). The paper is nonetheless interesting and thought provoking. The idea of training a GP on the latent representations seems interesting and useful.\n\n\n[1] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why ReLU Networks Yield High confidence Predictions Far Away from the Training Data and How to Mitigate the Problem\n\n[2] Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks.\n\n[3] A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks\nKimin Lee, Kibok Lee, Honglak Lee, Jinwoo Shin\n\n[4] What Are Bayesian Neural Network Posteriors Really Like?\nPavel Izmailov, Sharad Vikram, Matthew D. Hoffman, Andrew Gordon Wilson", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discussed relevant limitations of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "H4tl5D6Rhpm", "reviewer": "Reviewer_aGgD", "summary": "The author(s) achieve cubic growth (as a function of distance from the training data) in uncertainty for ReLU BNNs. They construct the double-sided cubic spline (DSCS) kernel which sums over 1-D input kernels to ensure that departure for any dimension increases uncertainty. The authors then use a Gaussian Process with this DCSC kernel to model ReLU BNN residuals. This treatment, for well-trained BNNs, results in a GP posterior that is approximately the the ReLU BNN's posterior plus the residual's prior. This observation provides the ability to apply the proposed after training the BNN as usual. Theorem 3 proves that their method in the limit produces a uniform distribution over class labels (maximum uncertainty).", "questions": "", "limitations": "", "rating": 7, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality:\n- I am unfamiliar with Meinke and Hein (2020), but this work does seem differentiated after a cursory glance. However, Meinke and Hein (2020) seems to be more generally applicable.\n\n### Quality:\n- I did not carefully review the proofs, but propositions and theorem reasoning make intuitive sense.\n- Figure 6 does well at showing the proposed methods achieve the lowest confidence without a noticeable drop in accuracy.\n\n### Clarity: \n- Submission is well written.\n- Figures are very small and without captions.\n\n### Significance: \n- Reducing over-confident predictions by classification networks is a very important field of study and this work address such for BNN. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discuss that their methods are limited to ReLU BNNs and not non-Bayesian ReLU NN. I agree with the author's assessment that societal impact is minimal.  ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "8IpWNa8oPXz", "reviewer": "Reviewer_FpoL", "summary": "The paper proposes a method to overcome the issue of neural network overconfidence, particularly focussing on the area far away from the training data. The motivation is that Bayesian neural networks (BNNs) can deal with overconfidence near the training data, but even they struggle in the areas far away. The proposed method is based on enriching a BNN with a GP that is obtained as an approximation of a NN with infinite number of ReLUs units. Although the method is justified with this theoretical approach, in practice it means a simple post-hoc procedure with a low computational overhead. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Update after rebuttal: I have read the authors' response and other review. As per discussion below, I would like to thank the authors for their clarification of my concerns and I am increasing the score of the paper. \n\n/------------------------------------------------------\n\n\nOriginality: The proposed idea is interesting and appears to be novel. The related work is adequately cited and it is clearly indicated how the proposed work differs from it.\n\nQuality: The work is technically sound. The proposed method is theoretically grounded and theoretical analysis is provided. The method is also empirically tested rather extensively.\nSome claims though are not justified. For example, the main motivation that the quadratic variance growth is not enough to give appropriate confidence far away from the data, while the cubic growth is enough is never justified. \nTheoretical derivations are rather loose, such as approximations are specified only as $\\approx$ without any further details.\n\nClarity: The paper is mostly well written and easy to follow. There are some not major issues listed below.\n\nSignificance: The paper addresses an important issue of overconfidence of neural networks and proposes an interesting approach for this. I believe the idea may be interesting for a wide audience and can be used both in practice and as a base to build upon especially in the area of NN and GP connections.\n\nConclusion: Despite some weak points mentioned the paper seems to be a valid contribution to the community, therefore I am voting for acceptance of the paper.\n\nSpecific comments/suggestions:\n1.\tLines 3-8. “This issue arises since the output variance of a BNN with finitely many features is quadratic in the distance from the data region. Meanwhile, Bayesian linear models with ReLU features converge, in the infinite-width limit, to a particular Gaussian process (GP) with a variance that grows cubically so that no asymptotic overconfidence can occur.” – it is not obvious why quadratic is not enough, and cubic is enough.\n2.\tLine 9. “it can be used concretely” – “can be used in practice”?\n3.\tLines 19-20. “the confidence of BNNs can be bounded away from one” – unclear\n4.\tLines 41-42. “And due to the second property, the extended BNNs exhibit super-quadratic output variance growth, and thus are guaranteed to predict with uniform confidence away from the data.” – the idea is still not obvious. At least a reference to prove it is missing.\n5.\tLines 62-64. “maximum a posteriori (MAP) estimate \\theta_{MAP} = arg max_{\\theta} log p(\\theta| D). One can also treat \\theta as a random variable”. If you call it MAP, you already treat \\theta as a random variable.\n6.\tIntegral in line 67. One does not have to use softmax. \n7.\tLine 123-125. “An infinite-ReLU GP with the cubic spline kernel has cubic variance growth, which, combined with the probit approximation, yields the uniform confidence in the limit.” – The third time – why?\n\nMinor points:\n1.\tLine 1. “ReLU nets” -> “ReLU neural networks”\n2.\tLine 17. “overconfident issue” -> “overconfidence issue”\n3.\tE.g., line 17. “Kristiadi et al. [2]” – a mix of reference styles\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed some limitations of the work such as the trade off between ECE and OOD detection. The paper does not seem to have any straightforward potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {"ethical_concerns": "The paper does not seem to raise any immediate ethical concerns. If anything the main problem the paper addresses if solved aims to resolve the current ethical concerns of the existing widely acceptance of NNs despite their overconfidence."}}]

Author rebuttal:

Meta-review: As pointed out by all the reviewers, this paper seems to be addressing an important problem of over-confidence in Bayesian ReLU neural networks. The paper is well-written, the problem is well motivated and the proposed solution is well-supported with theoretical and empirical work. 
Decision: Accept (Spotlight)