Uncertainty-based out-of-distribution detection requires suitable function space priors
Authors: Francesco D'Angelo
Christian Henning
Venue: ICLR 2022 Submitted (2022)
Keywords: Bayesian Statistics
Out-of-distribution Detection
Machine Learning
Neural Networks
Epistemic Uncertainty
Gaussian Process

Abstract: The need to avoid confident predictions on unfamiliar data has sparked interest in out-of-distribution (OOD) detection. It is widely assumed that Bayesian neural networks (BNNs) are well suited for this task, as the endowed epistemic uncertainty should lead to disagreement in predictions on outliers. In this paper, we question this assumption and show that proper Bayesian inference with function space priors induced by neural networks does not necessarily lead to good OOD detection. To circumvent the use of approximate inference, we start by studying the infinite-width case, where Bayesian inference can be exact due to the correspondence with Gaussian processes. Strikingly, the kernels induced under common architectural choices lead to uncertainties that do not reflect the underlying data generating process and are therefore unsuited for OOD detection. Importantly, we find this OOD behavior to be consistent with the corresponding finite-width networks. Desirable function space properties can be encoded in the prior in weight space, however, this currently only applies to a specified subset of the domain and thus does not inherently extend to OOD data.  Finally, we argue that a trade-off between generalization and OOD capabilities might render the application of BNNs for OOD detection undesirable in practice. Overall, our study discloses fundamental problems when naively using BNNs for OOD detection and opens interesting avenues for future research.

Under review as a conference paper at ICLR 2022
UNCERTAINTY-BASED OUT-OF-DISTRIBUTION DETEC- TION REQUIRES SUITABLE FUNCTION SPACE PRIORS
Anonymous authors Paper under double-blind review
ABSTRACT
The need to avoid confident predictions on unfamiliar data has sparked interest -ut-of-distributiin (OOD) detection. It is widely assumed that Bayesian neural networks (BNNss are well suited for this task, as the endowed epistemic uncertainty should lead to disagreement in predictions this assumption and show that proper Bayesian inference with function space priors induced by neural networks does not necessarily Iead to good OOD detection. To circumvent the use of approximate inference, we start by studying the infinite- width case, where Bayesian inference can be exact due to the correspondence with Gaussian processes. Strikingly, the kernels induced under common architectural choices lead to distributions over functions which cause predictive uncertainties that do not reflect the underlying data generating process and are therefore unsuited for OOD detection. Importantly, we find this OOD behavior to be consistent with the corresponding finite-width networks. To overcome this limitation, useful function space properties can also be encoded in the prior in weight space, however, this can currently only be applied to a specified subset of the domain and thus does not inherently extend OOD data. Finally, we argue that a trade-off between generalization and OOD capabilities might render the application of BNNs for OOD detection undesirable in practice. Overall, our study discloses fundamental problems when naively using BNNs for OOD detection and opens interesting avenues for future research.
outliers. In this paper, we question
INTRODUCTION
One of the challenges that the modern machine learning community is striving tackle is the detection of unseen inputs for which predictions should not be trusted. This problem is also known as out-of-distribution (OOD) detection. The challenging nature of this task is partly rooted in the fact that there is no universal mathematical definition that characterizes an unseen input x as OOD as discussed in Sec. 2 Without such a definition, there is no foundation for deriving detection guarantees under verifiable assumptions. In this work, we consider OOD points as points that are unlikely under the distribution the data p(æ)) and points are outside the support of p(æ)). Importantly, we do not claim to provide meaningful definition of OOD, but rather argue that the justification of an OOD method can only be assessed theoretically once such a definition is provided. While it appears natural to tackle the OOD detection problem from generative perspective by explicitly modelling p(æ). this paper is solely concerned with the question of how justified it is to deploy the predictive uncertainty of Bayesian neural network (BNN) for OOD detection. As recent developments forecast an increasing integration of deep learning methods into industrial applications, it becomes essential to provide the theoretical groundings that justify the use of uncertainty for OOD detection, a task that is crucial for safety-critical applications of Al and reliable prediction-making.
When BNNs are used in supervised learning, the dataset is composed of inputs X' and targets
y which are assumed to be generated according to some unknown process: D s.i.d. p(æ))(y)) x) The goal of learning is to infer from D alone the distribution p(y x)) in order to make predictions on unseen inputs x". In the case of deep learning, this problem is approached by choosing neural network f(; w) parametrized by w, and predictions are made via the conditional p(y f(:; w)) (also called the likelihood of w for given x, y). Assuming that the induced class of hypotheses contains some to such that p(v (y f(: u)) = p(u x)) almost everywhere on the support of p(z), Bayesian statistics can be used to infer plausible models p(w (D) under the observed data given some prior knowledge p(w) (see MacKay (203)). This parametric description together with the network implicitly induces a prior over functions p(f) (Williams 1997: Fortuin 2021). Believing
in the validity of subjective choice of prior (O' O'Hagan 2004), Bayesian inference comes with

---

Under review as a conference paper at ICLR 2022
multitude of benefits it less susceptible to overfitting, allows to incorporate evidence without requiring access to past data (Farquhar & Gal 2018) and provides interpretable uncertainties.
The uncertainty captured by BNN be coarsely categorized into aleatoric and epistemic uncertainty. Aleatoric uncertainty is irreducible and intrinsic to the data pyy z)) For instance, blurry image might not contain enough infor- mation to identify unique object classes. On the other hand, BNN also models epistemic un- certainty by maintaining distribution over pa- rameters p(w B)D This distribution reflects the uncertainty about which hypothesis explains the data and can be reduced by observing more data. Arguably, aleatoric uncertainty is of little interest for detecting OOD inputs. However, the recent advent of overparametrized models which further extend the hypothesis class, deceptively motivated the idea that the epistemic uncertainty under a Bayesian framework might be intrinsi- cally suitable to detect unfamiliar inputs, and therewith implying, that BNNs can be used for OOD detection. This conjecture is intuitively true if the following assumption holds: the hypotheses captured by p(w D) must agree on their predictions for in nn-distribution samples but disagree for OOD samples (Fig. Ta). Certain Bayesian methods satisfy this assumption, e.g. a Gaussian process regression with an RBF kernel (ef. Sec. 4 However, the uncertainty induced by Bayesian inference does not in general give rise to OOD capabilities. This can easily be verified by considering the following thought experiment (Fig. [b): Assume a model class with only two hypotheses g(x)) and h(x) such that g(x) # h(x) Vr and data being generated according to g(z) on a restricted domain. Once data is observed, we can commit to the ground-truth hypothesis g(r) and thus lose epistemic uncertainty in- and out-of-distribution alike. Finite-width neural networks, by contrast, form a powerful class of models, and are often put into context with universal function approximators (Hornik 1991). But is this fact in combination with Bayesian statistics enough to attribute them with good OOD capabilities? The literature often seems to imply that BNN is intrinsically good OOD detection. For instance, the use of OOD benchmarks when introducing new methods for approximate inference creates the false impression that the true posterior good OOD detector (Louizos & Welling 2017 Pawlowski et al.) 2017 Krueger et al. 2017]] Henning et al. 2018; Maddox et al., 2019 Closek et al. 2020 D' Angelo & Fortuin 2021). Our work meant to start a discussion among researchers about the properties of OOD uncertainty of the exact
(2003)
(4)
(b)
Figure 1: (a) GP regression with an RBF ker- nel illustrates uncertainty- based OOD detection. The prior variance over function values only squeezed around training points, which leaves epistemic uncertainty high in OOD regions. (b) Conceptual illustration on why epistemic uncer- tainty is not necessarily linked to OOD detection (sce main text for more details). Note, that the ground-trutt p(y x)) is only defined within the support of p(a:).
Bayesian posterior of neural networks. To initiate this, we contribute as follows:
We emphasize the importance of the prior in function space for OOD detection, which is induced
by the choice of architecture (Sec. 4 and weight space prior (Sec. 5)
We empirically show that exact inference in infinite-wine width networks under common architectural choices does not necessarily lead to desirable OOD behavior, and that these observations are consistent with results obtained via approximate inference in their finite-width counterparts.
We furthermore study the OOD behavior in infinite-width networks by analysing the properties of the induced kernels. Moreover, we discuss desirable kernel features for OOD detection and show that also neural networks can approximately carry these features.
We emphasize that the choice of weight-space prior has a strong effect on OOD performance, and that encoding desirable function space properties within unknown OOD domains into such prior is challenging.
We argue that there is a trade-off between good generalization and having high uncertainty on OOD data. Indeed, improving generalization by incorporating prior knowledge (which is usually encoded in an input-domain agnostic manner) can negatively impact OOD uncertainties.
Note, that the Bayesian treatment of network parameters does not account for all types of epistemic uncertainty such as the uncertainty stemming from model misspecification (Hällermeier & Waegeman 2021 We, however, assume the model to be correctly specified.
2

---

Under review as a conference paper at ICLR 2022
2 ON THE DIFFICULTY OF DEFINING OUT-OF-DISTRIUUTIO INPUTS
While the intuitive notion of OOD sample points (or outliers) is commonly agreed upon, for instance as "an observation (or subset of observations) which appears to be inconsistent with the remainder of that set of data" (Barnett & Lewis 1984), mathematical formalization of the essence of outlier is difficult (cf. SM(A) and requires subjective characterizations (Zimek & Filzmoser (018). Often, methods for outlier detection are designed based on an intuitive notion (see Pimentel et al. (2014) for a review), and therewith only implicitly define a method-specific definition of OOD points. This also accounts for uncertainty- -bsed OOD detection with neural networks, where a statistic of the predictive distribution that quantifies uncertainty (c.g., the entropy) is used to decide whether an input is considered OOD (Hendrycks & Gimpel 2017). Commonly, the parameters w of neural network are trained using an objective derived from Ep(a)) KLL (p(y x)) [p y {æ; w) ) (i.e., loss functions based on the negative log- likelihood such as the cross-entropy or mean-squared error loss). Therefore, these networks only have calibrated uncertainties in-distribution with OOD uncertainties not being controlled for unless explicit training on OOD data is performed (eg., Hendrycks et al.) 2019). The question this study is concerned with is therefore: does the explicit treatment of parameter uncertainty via the posterior parameter distribution p(w D) elicit provably high uncertainty OOD?
Note, that we do not consider the problem of outlier detection within the training set (Zimek & Filzmoser 2018), but rather ask whether a deployed model is able to know what it does not know.
BACKGROUND
In this section, we briefly introduce the concepts on which we base our argumentation in the coming sections. We start by introducing BNNs, which rely on approximate inference. We then discuss that in the non- parameeattticc limit and under a certain choice of prior, a BNN converges to Gaussian process, an alternative Bayesian inference framework where exact inference possible. The connection between BNNs and Gaussian processes will later allow us to make interesting conjectures about OOD behavior. Finally, we introduce generalization bounds from the BAC-Bayes framework, which we will later use to argue that generalization and OOD detection can be confficting objectives.
3.1 BAYESIAN NEURAL NETWORKS
In supervised deep learning, typically construct a likelihood function from the conditional density p (y [x:; w)), parameterized by a neural network f(:; w), and the training data D = {(r;, yb 2=1- In BNNs, this is used to form the posterior distribution of all likely network param- eterizations: p(w (D) x II P (y: f(z;;; w)) p(w), where p(w) is the prior distribution over weights. Crucially, when making a prediction with the Bayesian approach on test point x", we do not only use single parameter configuration vp but we marginalize over the whole posterior, thus taking all possible explanations of the data into account: p(y" | x², D) = p'p Ifxx; w)) p(w) | (D) dw.
3.2 GAUSSIAN PROCESS
Gaussian processes are established Bayesian machine learning models that, despite their strong scalability limitations, can offer a powerful inference framework. Formally (Rasmussen) 2004):
Definition A Gaussian process (GP) is a collection of random variables, any finite number of which have joint Gaussian distribution.
Compared to parametric models, GPs have the advantage of performing inference directly in func- tion space. A GP is defined by its mean m(x) = E ff (=)] and covariance C f = E (f()) m(x)) (f(z') m(x'))]. The latter can be specified using a kernel function k Rdd x R2d + IR and implies a prior distribution over functions:
x(f()= N' ((,)
where K(X, X); = k(2,, x,) is the kernel Gram matrix on the training inputs X, and m(x) has been chosen to be O. When observing the training data, the prior is reshaped to place more mass in the regions of functions that are more likely to have generated them, and this knowledge is then used to make predictions ing the joint Gaussian prior: p(f XX, f). To model the data more realistically, we assume to not have direct access to the function values but to noisy observations: y = f(#)++ +e with (0,0,
unseen inputs X.. In probabilistic terms, this operation corresponds to condition-
3

---

Under review as a conference paper at ICLR 2022
This assumption is formally equivalent to a Gaussian likelihood p(ylf) (yyff o²)
The condi-
tional distribution on the noisy observation can then be written as (Rasmussen 2004):
pff. X,, /dfole. ff N
f. K(X,,x) X)),, -
C(f-) ((X,, X.) x(X.,X))KKI x -
where p(f XX, p(vlf)(((X)) with p(D) being the marginal likelihood. P(D)
RBF kernel. A commonly used kernel function for GPs is the squared exponential (RBF):
1 2/2 | x
h(x, exp
with the length scale being hyperparameter. It is important to notice that the covariance between outputs is written exclusively as a function of the distance between inputs. As a consequence, points that are close in the Euclidean space have unitary covariance that decreases exponentially with the distance. As we will see in Sec. this feature has important implications for OOD detection.
The relation of indt BNNs and GPs. The connection between neural networks and GPs has recently gained significant attention. Neal (199)) showed that a 1-hidden layer BNN converges to GP in the infinite-width limit. More recently, the work by Lee al. (2018) and G. Matthews et al. (2018) extended this result to deeper networks, called neural network Gaussian processes (NNGP). Crucially, the kernel function of the related GP strictly depends on the used activation function. To better understand this connection we consider fully connected network with L layers 1 O.....L L with width H,. For each input we use x" x(xx) represent the post-activation with xD = x and f the pre-activation so that (xr = b/-- + Σドー w/y 3 1-1 with zrj = h((f)) and ((-) a point-wise activation function. Furthermore, the weights and biases are distributed according a to oy ~ (0,o7)) and w/, ~ (O, Hi Given the independence of the weights and biases it follows that the aost-activations are independent as well. Hence, the central limit theorem can be applied, and for Hi-1 + obtain fx ~ GP C²). The covariance C' and therefore the prior over functions is specified by the kernel induced by the network architecture (Lee et al. 2018):
1((x))))
B = [FM() -I(())
(4)
=o + o ((m).s
1) [ (f!-1(a)) (a)) h (f- (x'))
with K3 being the 2 covariance matrix formed by k((.)) using æ and x'. For the input layer, where activation function is applied, the kernel is simply given by 1²0 (x. x') of + o ="= For the remaining layers, instead, the kernel expression determined by the non-linearity. Some activation functions like sigmoid or hyperbolic tangent do not admit known analytical form and therefore require a Monte Carlo estimate of Eq. (cf. Fig. S3). For others, Eq. 4 can be computed in closed form (e.g.. Williams 1997 Cho & Saul 2009 Tsuchida al. 2018 Pang al., 2019 Pearce
et al. 2020). We list some kernel functions that are important for this study below, such as the one
for ReLU networks (Cho & Saul
2009p:
o2
(x, = of
M--1 (x' 1 x²) (sin gi-1 2,00 + +(x -0-11 cos 6i
2n
(5)
kc ("x x')
0 0' cos
( (x',
Interestingly, for some non-lincarities kernel that carries similar properties the RBF in Eq. can be induced by neural networks. This is the case for the cosine activation function which has the
Note, that the Gaussian likelihood assumption is commonly applied neural network regression through the of mean-squared error loss.
4

---

Under review as a conference paper at ICLR 2022
following closed form solution in the single hidden layer case (Pearce et al.
2020
2 2
x
x'12
201 a + x'12
Keos (x, x²) = =
exp
+ exp
203
(6)
2d
2d
Similarly, also the exponential activation function as deployed in RBF networks (Broomhead & Lowe, 1988) leads to similar properties. This particular class of neural networks defines computational
units as linear combinations of radial basis functions f((=) = j=1 Wi exp
2og ||:: 1 ||? + b.
These networks, in the infinite- aafittt-wwdthh case and when assuming a Gaussian prior over the centers Mj ~ (O, 1 and wy ~ (O, ow)). also converges to Gaussian process with an analytical kernel
function:
exp |
x x x'11 20
< 11 2n n
toteer (2, x²)
of + o
De
exp
exp
0p
20
where 1/0 = 2/0 + 1/ of
20 g
+ o / and o
=2 / + og. The equivalence with the
RBF kernel is explicit in the limit of + 0c (Williams 1997).
3.3 PAC-BAYES GENERALIZATION BOUND
Considering the supervised learning framework introduced in Sec. [ the PAC theory (Valiant 1984)
establishes probabilistic bound the generalization error of given predictor. The theory referred to as PAC-Bayes [McAllester] 1999 Catoni (007) when a prior distribution p is defined to incorporate prior domain knowledge. Considering distribution q on the hypothesis h E H. and the space B())) of all conditional distributions p(y h())), we can define a bounded loss function such that (
B(V) x V + [0, 1]. This gives rise to the empirical and true risk RD(g) == Σ Ennog (1((()) h(x;)), y:)l and R((g) == [x, w. )<p((a.y)
(p((y h(xx)), y, )]. respectively, where p(z, y) p(x))(( æ)). Note that when q the Bayesian posterior as, for instance, in the GP case, the Bayes risk can be defined as: (g(g)= : E (... v.)(((a..))) [(En~ng (xxx)))] y.)] and it can be shown that Rg(g) < 2R(g) (Seeger 2003 Therefore, a bound over R also implies a bound over Rg (Seeger 2003). The PAC bound gives a probabilistic upper bound on the true risk R(q) terms of the empirical risk (D(4) for training set 'D as formulated in the following theorem:
Theorem (PAC-Bayes theorem (Catoni, 2007 ) For any distribution p(a, y) over x V, for
any distribution q and prior p hypothesis space H. for any 5 (0, 1] and 3 > 0 the following holds with probability at least 5 over the training set D ~ p(xx, y) of cardinality N:
1 exp ( -3Bpll))
(
1 (KL(all)) + +log ) N
bq R(g) - 1-e-8
(8)
For a fixed p. D. 8, minimizing Eq. 8 is equivalent to minimizing N ND(()q) + KL(g) The requirement of bounded loss function makes the use of the mean-squared error, and thus the negative log-likelihood, not applicable this bound. Nevertheless, as long as the loss function measures the quality of predictions, the bound in Eq. can be used to assess the generalization capabilities of model. For this purpose, in our analysis we use as a surrogate loss the following Reeb et al. (2018):
take the first-order Taylor expansion and recover the MSE lexp (Py [b],, y, (Eylp(y h)] y)² /02.
(pl( h)) y) 1 exp [Bulp(y|A)]
Interestingly, for small deviations
4 THE ARCHITECTURE STRONGLY INFLUENCES OOD UNCERTAINTIES
In the previous section, we recalled the connection between BNNs and GPs, namely that Bayesian inference in an infinite-width neural network can be studied in the GP framework (assuming a proper choice of weight-space prior). This connection allows studying how the kernels induced by architectural choices shape the prior in function space, and how these choices ultimately determine OOD behavior. In this section, we analyze this OOD behavior for traditional as well NNGP- induced kernels. To minimize the impact of the approximations on our results, we exclusively focus on conjugate settings, i.e., regression (see SM C. for classification results). Furthermore, this choice
Note that this distribution does not necessarily need he the Bayesian posterior.
5

---

Under review as a conference paper at ICLR 2022
Dr
-6-
(a) 2-layer ReI.U (00)
(b) 1-layer Cosine (oc)
(c) 1-layer RBF net (00)
16
-0
-0
6
(d) 2-layer ReLU (5)
(c) 1-layer Cosine (100)
I-layer RBF net (500)
Figure 2: Standard deviation of.) of the predictive posterior of BNNs. We perform Bayesian inference on mixture of two Gaussians dataset considering different priors function space induced by different architectural choices, The problem is treated as regression task to allow exact inference in combination with GPs (a, b, Predictive uncertainties for finite-width networks obtained using HMC (d, e, f).
of Gaussian likelihood induces direct correspondence between function values and outcomes up noise corruptions. Hence, prior knowledge about outcomes can be encoded in function space through the choice of meaningful function space prior.
Uncertainty quantification for OOD detection. Uncertainty can be quantified in multiple ways, but often measured as the entropy of the predictive posterior. The predictive posterior, however, captures both aleatoric and epistemic uncertainty, which does not allow a distinction between OOD and ambiguous inputs (Mukhoti et al. 2021). While our choice of likelihood does not permit the modelling of input-dependent uncertainty (Gaussian with fixed variance), softmax classifier can capture heteroscedasticc aleatoric uncertainty arbitrarily well by outputting an input-dependent categorical distribution For this reason, uncertainty should be quantified in way that allows OOD detection to based on epistemic uncertainty only. The function space view of GPs naturally provides such measure of epistemic uncertainty by considering the standard deviation of the posterior over function values (f-) = /diag (C(f.) (illustrated in Fig. (.) Such measure
be naturally translated to BNNs by looking at the disagreement between network outputs when sampling from p(w D). Note that in classification tasks all models drawn from p(w DD might lead high- entropy softmax without disagreement, and therefore only uncertainty measure based on model disagreement prevents one from misjudging ambiguous points as OOD, As OOD detection the focus of this paper, we always quantify uncertainty in terms of model disagreement.
Bayesian statistics and OOD detection have no intrinsic connection. As conceptually illus- trated in Fig. D and demonstrated using exact Bayesian inference in Fig. 3b] predictive uncer- tainties are not necessarily reflective of p(x) (and thus not suited for OOD detection), irrespective of how well the underlying task is solved.
a-
ad
(a) RBF Kerne
(b) Periodic Kernel
GP regression with an RBF kernel. We next examine the predictive posterior of GP with squared exponential (RBF) kernel (Eq. B3. Fig. 3a shows epistemic uncertainty as the standard deviation of p(f. X., X, y). We notice that (ff.) nicely captures the data manifold and is thus well suited for OOD detection. This behav-
Figure 3: Standard deviation a (f.)) of the pre- dictive posterior using GPs with common ker- nel functions. GP regression is performed the same dataset as Fig. 2
ior can be understood by considering Eq. 2 while noting that k( x²) const if æ: = x', and that the
'Tere are also expressive likelihood choices to capture heteroscedastic uncertainty for continuous variables (e.g.. Zieha al. 2020

---

Under review as a conference paper at ICLR 2022
variance of posterior function values can be written as 0² (f..) (cf. SM B where 3 are dataset- and input-dependent. The second term is reminiscent of using kernel density estimation (KDE) to approximate p(z), applying a Gaussian kernel, while the first term is the (constant) prior variance. Hence, the link between Bayesian inference and OOD detection can be made explicit, as the posterior variance is inversely related to the input distribution.
-EE ((xx))((,,.. x;)
Loosely speaking, in GP regression as in Eq. 2 and kernels where the KDE analogy holds, epistemic uncertainty can roughly be described as const p(æ=) {
In this view, one starts with high (prior) uncertainty everywhere, which is only reduced where data is observed. By contrast, learning a normalized generative model (e.g., using normalizing flows (Papamakarios et al. 2019) often requires to start from an arbitrary probability distribution.
The OOD behavior induced by NNGP kernels. Fig. 2a illustrates (ft) for an infinite- 2-layer ReLU network (see Fig. S2 for other common architectural choices). It is already visually apparent, that in this case the kernel is less suited for OOD detection compared to an RBF kernel. Moreover, we cannot justify why OOD detection based on o (f.) would be principled for this kernel as the KDE analogy does not hold. This is due to two reasons: (1) the prior uncertainty k(x,, v.) is not constant (Fig. ST and (2) we empirically do not observe that k(æ, æ') can be related to distance measure (Fig. S4). We therefore argue that more theoretical work is necessary if one aims to justify uncertainty- based OOD detection with common architectural choices from the perspective of NNGP kernels. On this note, maintaining parameter uncertainty and being able to detect OOD samples often considered crucial requirements of systems deployed in safety-criticn applications. Given that Bayesian inference is not intrinsically linked to OOD detection, care should be taken to precisely communicate safety-relevant capabilities of BNNs to practitioners.
However, the NNGP perspective also allows us to choose an architecture such that the BNN's uncertainty resembles the, for these problems. de- sirable OOD behavior of the GP with RBF kernel described above. In particular, the cosine (Eq. 6 and RBF (Eq. (7) network induce kernels that are related to the RBF kernel. The last term in Eq. 6 quickly converges to zero for moderate norms of x x' (or high of)) which explains why the uncertainty behavior of Fig. (b] is qualitatívely identical to Fig. 3a In case of the RBF network,
(a) Width-aware prior
(b) Standard prior
Figure 4: The importance of the choice of weight prior p(w). Here, we perform 1d regres- sion using HMC with either (a) a width-aware prior (O, o2 or (b) a standard prior (N00 (0) H;
the RBF kernel is recovered if ||ae||| and ||a'|| are small compared to o explaining why the un- certainty faints towards the boundaries of Fig. [c]
These examples show that in the non- non-parametric limit and for (ow-diiensionnl) regression tasks, BNNs can be constructed such that uncertainty-basse based OOD detection can be justified through mathe- matical argumentation. We next study whether the observations made in the infinite- width limit are relevant for studying finite-width neural networks.
Infinite-width uncertainty is consistent with the finite-width uncertainty. For finite-width BNNs exact Bayesian inference is intractable. To mitigate the effects of approximate inference we resort to Hamiltonian Monte Carlo (HMC, Duane et al. 1987: Neal et al., 201T). Fig. 2d to 2i show an estimate of a (f.)) for knite-width networks corresponding to the non-parametric limits studied above (illustrations with another dataset can be found Fig. (S5). Already for moderate layer widths, the modelled uncertainty resembles the one of the corresponding NNGP. Given this close correspondence, we conjecture that the tools available for the infinite-widtt case are useful for designing architectural guidelines that enhance OOD detection. We illustrated this on low-dimensional problems by studying desirable function space properties induced by the RBF kernel, which can be translated to BNN architectures.
THE CHOICE OF WEIGHT SPACE PRIOR MATTERS FOR OOD DETECTION
For a neural network, the prior in function space is induced by the architecture and the prior in weight space (Wilson& & Izmailov 2020
Note, the KDE approximation of p(x)) is likely to deteriorate massively if the dimensionality of increases.
7

---

Under review as a conference paper at ICLR 2022
In the previous section, restricted ourselves particular class of weight space priors which allowed us to study the archi- tectural choices in the -withh limit. While, in practice, a wide variety of weight space prior choices might lead to good generalization (with respect to test data from p()))p(y a)) e.g., cf. Izmailov et al. (2021b)), the uncertainty behavior that induced OOD might vary drastically. This is illustrated in Fig. 4 where for the same network two different choices of p(w) lead to vastly different predictive uncertainties despite the fact that both choices fit the data well.
-20
-10
0
(a) GP with RBF Kernel
In Sec. we use the infinite-widhe limit to obtain a function space view that allows to make interesting conjectures about predictive uncertainties on OOD data. Recently, multiple stud- ies suggested ways to either explicitly encode function space properties into the weight space prior p(w) or to use a function space prior when performing approximate inference in neural networks -Ham--hhepherd et al. 2017 Sun et al., 2019: Tran et | et al. 2021). 1). However, all these methods depend on the specification of set Xn C X' on which desired function space properties will be enforced (Fig. 5). For instance. the Ridgelet prior proposed by MMatsubara et al. (2021) provides an asymptotically correct weight space prior construction that induces a given GP prior. Thus, on X'x this method allows guide Bayesian
-20
-10
(b) Network with Ridgelet Prior
Figure OOD challenges when encoding function space proper- ties into weight space prior. (a) Samples from GP prior xff (b) Samples from a I-layer Tanh network (width: 3000) using the Ridgelet prior (Matsubara et al., 2021) corresponding
to meaningfully encode prior knowledge inference. But, an priori specification of a set X'R which can be largely covered by samples æ E Xx to ensure the desired prior specification X'R is practically challenging, and concep- tually related to the idea of training on OOD data to calibrate respective uncertainties (ef. Sec. 2) Note, we do not aim to phrase this OOD effect as a drawback of these methods, as we do not consider Bayesian inference to be intrinsically related to OOD detectors. It is, however, important to keep in mind that function space properties deemed beneficial for OOD detection are straightforward to induce when working in weight space.
the GP in (a). Dotted lines denote the domain XR within which the Ridgelet prior was matched to the target GP.
A TRADE-OFF BETWEEN GENERALIZATION AND OOD DETECTION
An important desideratum that modelers attempt to achieve when applying Bayesian statis- ties is good generalization through the incorporation of rel- evant prior knowledge. Is this desideratum generally in con- flict with having high uncer- tainty on OOD data? In this sec- tion, we provide arguments indicating that the answer this question can be yes. Consider data with known periodic structure inside the support of p(æ) (Fig. 6).
Table 1: Empirical risk, KI. divergence, PAC bound (6 0.01)
and log marginal likelihood for the examples shown in Fig.
Ro(g)
KL p)
PAC
log p(D)
RBF
0.117 008
8.336
0.563 0011
-31.536=t.756
ESS 0.00
7.231
4.003
0.519
-27.135
We can either choose to ignore our prior knowledge by selecting GP prior with RBF kernel (Fig. 6a or we explicitly incorporate domain knowledge using a function space prior that allocates its mass on periodic functions e.g. the exp-sine-squaree OOD uncertainties will be useful for OOD detection (Sec. 4 In the latter case, however, we see that uncertainties do not reflect p(z). By contrast, the roles are reversed when it comes to assessing generalization.
(ESS) kernel In the former case, we know that
'Noee that such incorporation of prior knowledge is often done in way that is agnostic to the unknown support of p(z).
8

---

Under review as a conference paper at ICLR 2022
Here, the choice of periodic ker- nel has clear benefits, as we can
The
2
2
see visually and by comparing the
generalization bound introduced in be tightened by minimizing N RD (p(f D)) + KL (p(f) D) (f)). As reported in T the bound's 's value for the ESS kernel is lower than the RBF. Interest- ingly, the bound is minimized if both terms are minimized, and the second term explicitly asks the posterior to remain close to the prior. Therefore. the second term benefits from restricting the prior function space through the incorporation of prior knowledge, counteracting the need of a rich function space for inducing high OOD uncertainties (Fig. I. Consistent with this, also the log marginal likelihood log (D) in Table iis higher for the ESS kernel. An orthogonal problem often considered in the literature is generalization under dataset (or covariate) shift (Quiñonero- Canooell--- al. 2009 Snoek al., 2019 Izmailov et al. 2021a). In
Sec. [3.]] which
2
-2
0
5
10
15
20
0
5
10
15
20
(a) GP with RBF Kerne
(b) GP with ESS kernel
Figure 6: Generalization and OOD detection. Mean and the first three standard deviations of the predictive posterior for different function space priors.
this case, one deliberately seeks to provide meaningful predictions on test data Prest (x) that might not overlap in support with our training input distribution p(z). Therefore, prior knowledge needs to explicitly encode how to obtain "generalization on OOD data" as the data cannot speak for themselves. Such priors are proposed in Izmailov et al. (2021a), but also Fig. 6p can be viewed as an example of how prior encoding specifies how to generalize to OOD data.
ON THE PRACTICAL VALIDATION OF OOD PROPERTIES
Relating the epistemic uncertainty induced by a BNN to p(æ) opens up interesting new possi- bilities. As we show in SM B epistemic uncertainty of GP with RBF kernel can be related to a KDE approximation of p(.). Therefore, one may consider the epistemic uncertainty as an energy function create a generative model from which to sample from the input space (Fig. Moreover, this approach may be used to empiri- cally validate the OOD capabilities of BNN. Indeed, OOD performance is commonly vali- dated by selecting a specific set of known OOD datasets (Izmailov et al., 2021 However, as the OOD space comprises everything except the in in-distribution data, it is infeasible to gain good coverage on high- figgh-dmenssonall data with such test- ing approach. We therefore suggest reverse ap- proach that checks the consistency in regions of certainty instead of uncertain ones by sampling via the epistemic uncertainty. Indeed, if these samples and thus the generative model based on uncertainty estimates are consistent with the in- distribution data then a strong indication for trustworthiness is provided. Additionally, a discrepancy measure (Liu et al., 2016 Gretton et al. 2012) can be used to compare the generated samples and training data points and quantitatively assess to which extent the epistemic uncertainty related to p(z). Moreover, this approach can open up new research opportunities. For instance, continual learner may use its uncertainty to generate its own replay data to combat forgetting (cf. C.3)
(a) I llayyrr ReLU (oo)
(b) 1-layer Cosine (oo)
Figure 7: Sampling from regions of low uncer- tainty. Here, we treat epistemic uncertainty (mea- sured as (f.) )) as an energy function and perform rejection sampling (black dots). If uncertainty faithfully captures p(z). these samples should re- semble in-distribution data.
CONCLUSION
We challenge the common view that uncertainty-base OOD detection with BNNs is intrinsically justified. Our arguments are all based on low-dimensional problems and cannot easily be generalized to real-world problems, but our observations are consistent with the fact that empirically BNNs are often only marginally superior in OOD detection compared to models that do not maintain epistemic uncertainty (Snoek al. 2019 Henning et al., 202T). Overall, this work highlights fundamental limitations of BNNs for OOD detection that not solely explained by the use of approximate inference.

---

Under review as a conference paper at ICLR 2022
Ethics Statement. When machine learning algorithms enter woal-world applications, they not anymore embedded in a controlled environment. Instead, they have to know what they don 't know, which encloses the notion of OOD detection. It is therefore of utmost importance to discuss and understand the justifications for employing uncertainty-basee paper is to disclose in simple but clear manner that these justifications not obvious, and thus calls for more research in this direction. Our illustrative arguments should, however, not be misinterpreted or generalized, for instance, by assuming OOD properties we derive from an RBF kernel can be applied to nigh-dimensional image manifolds.
OOD detection. 'he purpose of our
Reproducibility Statement.
The source code to reproduce all experiments available at: anony-
mous link.
REFERENCES
Vic Barnett and Toby Lewis. Outliers statistical data. Wiley Series Probability and Mathematical Statistics. Applied Probability and Statistics, 1984.
David S Broomhead and David Lowe. Radial basis functions, murti-variable functional interpolation and adaptive networks. Technical report, Royal Signals and Radar Establishment Malvern (United Kingdom), 1988.
Olivier Catoni. Pac-bayesian supervised classification: The thermodynamiis of statistical learning. Lecture Notes-Monogruph Series, 56:i-163, 2007. ISSN 07492170.
Youngmin Cho and Lawrence Saul. Kernel methods for deep learning. In Y. Bengio, D. Schuurmans, Lafferty, C. Williams, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 22, Curran Associates, Inc., 2009.
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, and Richard Turner. Conserva- tive uncertainty estimation by fitting prior networks. In International Conference on Learning Representations, 2020.
Francesco D' Angelo and Vincent Fortuin. Repulsive deep ensembles are bayesian. arXiv, 2021.
Alexander G. de G. Matthews, Jiri Hron, Mark Rowland, Richard E. Turner, and Zoubin Ghahramani. Gaussian process behaviour wide deep neural networks. International Conference on Learning Representations, 2018.
Simon Duane, Anthony Kennedy, Brian Pendleton, and Duncan Roweth. Hybrid monte carlo. Physics letters B, 195(2):216-222, 1987.
Sebastian Farquhar and Yarin Gal. A unifying bayesian view of continual learning. Bayesian Deep Learning Workshop at NeurIPS, 2018.
Daniel Flam-Shepherd, James Requeima, and David Duvenaud. Mapping gaussian process priors to bayesian neural networks. In NIPS Bayesian deep learning workshop, 2017.
Vincent Fortuin. Priors in bayesian deep learning: A review. arXiv, 2021.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel tao-sample test. Journal of Machine Learning Research, 13(25):723-773, 2012.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-disttibution examples in neural networks. In International Conference on Learning Representations, 2017.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, 2019.
Christian Henning, Johannes Oswald, João Sacramento, Simone Carlo Surace, Jean-Pascal Pfister, and Benjamin Grewe. Approximating the predictive distribution via dddversariallyttrinnee hypernetworks. In NewrIPS Bayesian Deep Learning Workshop, 2018.
10

---

Under review as a conference paper at ICLR 2022
Christian Henning, Maria R. Cervera, Francesco D' Angelo, Johannes Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, João Sacramento, and Bemjamin F. Grewe. Posterior meta-replay for continual learning. In Conference Neural Information Processing Systems, 2021.
Kurt Hornik. Approximation capabilities of multilaaer feedforward networks. Neural Nerworks, 4(2): 251-257, 1991. ISSN 0893-6080. doi: https://ooi.org/ 101 (0089666088088000 6000000800999900 190009-T.
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning 1 1003))457-506, 2021.
Pavel Izmailov, Patrick Nicholson, Sanae Lotfi, and Andrew Gordon Wilson. Dangers of bayesian model averaging under covariate shift. arXiv, 2021a.
Pavel Izmailov, Sharad Vikram, Matthew D Hoffmman, and Andrew Gordon Wilson. What are bayesian neural network posteriors really like? arXiv, 2021b.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske. Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, (4((13)352213522) March 2017.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. Being bayesian, even just a bit, fixes overconfidence in relu networks. In International Conference on Machine Learning, pp. 5436- 5446. PMLR, 2020.
Agustinus Kristiadi, Matthias Hein, and Philipp Hennig. An infinite-feature extension for bayesian reLU nets that fixes their asymptotic overconfidence. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, and Aaron Courville. Bayesian hypernetworks. arXiv, 2017.
Jachoon Lee, Jascha Sohl-dickstein, Jeffrey Pennington, Roman Novak. Sam Schoenholz, and Yasaman Bahri. Deep neural networks gaussian processes. In International Conference on Learning Representations, 2018.
Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learming, 8(3-4):293-33 321, 1992.
Qiang Liu, Jason Lee, and Michael Jordan. kernelized stein discrepancy for goodness-of- tests. In Maria Florina Balcan and Kilian Q Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 Proceedings of Machine Learning Research, pp. 276-284, New York, New York, USA, 20-22 Jun 2016. PMLR.
Christos Louizos and Max Welling, Multiplicative Normalizing Flows for Variational Bayesian Neural Networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 2218 2228222272222 Jorg.org, 2017.
David J. C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press, USA, 2003. ISBN 0521642981..
Wesley Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Aluch--Buc, E. Fox, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Takuo Matsubara, Chris J Oates, and Haunçois-Xavier Briol. The ridgelet prior: A covariance function approach to prior specification for bayesian neural networks. Journal of Machine Learning Research, 22:1-57, 2021.
David A. McAllester. Some pac-bayesian theorems. Machine Learning, 37(3):355-36 1999. doi: 10.1023/A:100761862444
11

---

Under review as a conference paper at ICLR 2022
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty. arXiv, 2021.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In International Conference on Learning Representations, 2019a.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, and Balaji Lakshminarayanan. Detecting out-of-distribution inputs to deep generative models using typicality. arXiv, 2019b.
Radford M. Neal. Bayesian Learning for Neural Networks. Springer- 1996. ISBN 0387947248.
Vpriage-VVerlag, Verlag, Berlin, Heidelberg,
Radford M Neal et al. Mcme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
Anthony O' O'Hagan. Bayesian statistics: principles and benefits. Frontis, pp. 31-45, 2004.
Guofei Pang. Liu Yang, and George Em Karniadakis. Neural-net-induced gaussian process regression for function approximation and pde solution. Journal of Computational Physics, 384:270-288,
2019.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. arXiv, 2019.
German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter, Continual lifelong learning with neural networks: review. Neural Networks, 113:54-71, May 2019. ISSN 0893-6080. doi:
Nick Pawlowski, Andrew Brock, Matthew CH Lee, Martin Rajehl, and Ben Glocker. Implicit weight uncertainty in neural networks. arXiv, 2017.
Tim Pearce, Russell Tsuchida, Mohamed Zaki, Alexandra Brintrup, and Andy Neely. Expressive priors in bayesian neural networks: Kernel combinations and periodic functions. In Uncertainty in Artificial Intelligence, pp. 134-144. PMLR, 2020.
Marco AF Pimentel. David A Clifton, Lei Clifton, and Lionel Tarassenko. A review of novelty detection. Signal Processing, 99:215-249, 2014.
Joaquin Satfonnrro-Cadelaa Masashi Sugiyama, Neil D Lawrence, and Anton Schwaighofer. Dataset shift in machine learning. Mit Press, 2009.
Carl Edward Rasmussen. Gaussian Processes in Machine Learning, pp. 63-71. Springer Berlin Hei- delberg, Berlin, Heidelberg, 2004. ISBN 978-3-540-28650-9 doi: 10. 1001/9097 (0007977----500-86550---0
David Reeb, Andreas Doerr, Sebastian Gerwinn, and Barbara Rakitsch. Learning gaussian processes by minimizing pac-bayesian generalization bounds. S. Bengio, Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
Matthias Seeger. Pac- bay-bay generalisation error bounds for gaussian process classification. J. Mach. Learn. Res., 3:233-266 269, March 2003. ISSN 1532-4435, doi: 10. (0.11662/1332443377620088333
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual Learning with Deep Generative Replay. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 2990-2999, Curran Associates, Inc., 2017.
Jasper Snock, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanann Sebastian Nowozin, D Sculley, Joshua Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, pp. 13969-13980, 2019.
12

---

Under review as a conference paper at ICLR 2022
Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. FUNCTIONAL VARIATIONAL BAYESIAN NEURAL NETWORKS. In International Conference on Learning Representations, 2019.
Ba-Hien Tran, Simone Rossi, Dimitrios Milios, and Maurizio Filippone. All you need is good functional prior for bayesian deep learning, arXiv, 2020.
Russell Tsuchida, Fred Roosta, and Marcus Gallagher. Invariance of weight distributions in rectified mlps. In International Conference on Machine Learning, pp. 4995-5004. PMLR, 2018.
Leslic Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134441144 1984.
Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F. Grewe. Continual learning with hypernetworks. In International Conference on Learning Representations, 2020.
Christopher KI Williams. Computing with infinite networks. Advances in neural information processing systems, pp. 295-301, 1997.
Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. Advances in Neural Information Processing Systems, 2020.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual Learning Through Synaptic Intelligence. In Proceedings of the 34th International Conference Machine Learning- Volume 70, ICML'17, pp. 3987-3995. JMLR.org 2017.
Maciej Zięba, Marcin Przewięžlikowski, Marek Šmieja, Jacek Tabor, Tomasz Trzcinski, and Prze- mystaw Spurek. Regflow: Probabilistic flow-based regression for future prediction. arXiv, 2020.
Arthur Zimek and Peter Filzmoser. There and back again: Outlier detection between statistical rea- soning and data mining algorithms. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, (6):e1280) 2018
13

---

Under review as a conference paper at ICLR 2022
SUPPLEMENTARY MATERIAL: UNCERTAINTY-BASED
DETECTION REQUIRES SUITABLE FUNCTION SPACE PRIORS
Anonymous authors
A
WHAT IS AN OUT-OF-DISTRIBUTION INPUT?
Pimentel (2014) reviews methods for outlier detection, putting them coarsely into five categories: (1) probabilistic, (2) distance-based, (3) reconstruction-bsst based, (4) domain-based, and (5) information- theoretic. Our focus lies on a probabilistic characterization criterion allows to decide whether given input is significantly different from the observed training population. In this section, we discuss pitfalls regarding obvious choices for such a criterion order to highlight the difficulties that arise when attempting agree on a single (or application-deppedemn mathematical definition of outliers.
an OOD point, where statistical
There many possible definitions that could be considered for point to be OOD below. Any of these definitions may change the notion of OOD and will therefore affect how a BNN should be designed such that predictive uncertainty adheres the underlying OOD definition. Considering a generative process p(z), the first question arising is regarding the regions outside the the space outside the manifold where samples æ defined For instance,
will outline
support of p(z) assume the data to be images embedded on lower-dimensional manifold. Are points outside this manifold clearly OOD, given that minor noise corruptions are likely leave the manifold? Even disregarding these topological issues solely focusing on the density p(x). makes the distinction between in- and -iittr--dittiittionn challenging. For instance, a threshold-criterion on the density might cause samples in zero probability region to be considered in-distribution (Nalisnick et al.) 2019a). To overcome such challenges, one may resort to concepts from information theory, such as the notion of typical set (MacKay, 20033 Nalisnick al., 2019b). Unfortunately, OOD criterion based on this notion would require looking at sets rather than individual points. We hope that this short outline highlights the challenges regarding the definition of OOD, but also clarifies that proper definition is relevant when assessing OOD capabilities on nigh-dimensional data, where a visual assessment as in Fig. iis not possible.
B ON THE RELATION BETWEEN GP REGRESSION AND KERNEL DENSITY ESTIMATION
GP regression with RBF kernel allows direct understanding of the OOD capabilities that Bayesian posterior may possess, as the a priori uniform epistemic uncertainty is reduced in direct correspondence density of p(ar).
Below, we rewrite the variance of an input æ, as defined in Eq. 2
o²( = k(x,,, x.) E Ba x1)
(9)
with (xxx)=;; (K(X,) +o21) k(x. xy))
Note, that k(æ,, v.) ) is positive constant for an RBF kernel, and that Bi(xx)) 2 0. Furthermore, as the RBF kernel k( xz,) behaves exponentially inverse to the distance between and æ;,, (xx..) approximately zero for all x, that are far from all training points. In addition, a training point x; only decrease the prior variance if it is close x., This analogy closely resembles the philosophy of KDF, which becomes an exact generative model in the limit of infinite data and bandwidth / - 0.
C ADDITIONAL EXPERIMENTS AND RESULTS
In this section, we report additional experiments and results.

---

Under review as a conference paper at ICLR 2022
Fig. S1 shows the prior's standard deviation function values v (x(x,, I..) for several NNGP kernels. Note, kernel that priori treats locations 2,, differently might not be desirable for OOD detection the data's s influence posterior uncertainties might be hard to interpret (cf. Sec.
Fig. S2 shows GP regression results with the GMM dataset (Sec. D) for several NNGP kernels. The plots show that the underlying task can be solved well with all considered architectures (as indicated by the predictive mean f. that captures the ground-trute truth targets in-distribution), even though the uncertainty behavior OOD vastly different and not reflective of p(æ).
4+
13
1.20
(a) I-layer ReLU
(b) 1-layer Erf
(c) 1-layer Cosine
Figure $1: NNGP kernel values v //(x,, ..) for various architectural choices. Note, that the NNGP kernel value k(x., ..) represents the prior variance of function values under the induced GP prior at the location I. (cf. Eq. D. As emphasized in Sec. 4 k(w., x.) ) is constant for an RBF kernel, which has important implications for OOD detection, such that priori (before seeing any data) all points are treated equally. This is not the case for the ReLU kernel, which has an angular dependence and depends on the input norm (cf. Eq. and its dependence on k (xx x))). The kernel induced by networks using an error function (Erf) or cosine as nonlincarity seems to be more desirable in this respect (note the scale of the colorbars).
-to
(a) f. l-aeyrr ReLU
(b) a (f.) 1-layer ReLU
(c) f- layer ReLU
(d) of-)) 2-layer ReLU
44
(e) f- 1-layer Tanh
(I) o (f.) 1-layer Tanh
(g) 2-layer Tanh
(h) off) 2-layer
-0
5
(i) f. 1-layer Erf
(j) (f=)) 1-layer Erf
(k) f. 2-lay Erf
()) (f.) f.)-)) 2-yay Erf
Figure S2: Mean f. and standard deviation (ffx) of the posterior p(f= Xx, 3) for GP regression with NNGP kernels for various architectural choices, such as number of layers or non- linearity. Note, that Tanh and Erf nonlinearities are quite similar in shape, which is reflected in the similar predictive posterior that is induced by these networks. We use the analytically known kernel expression for the Erf kernel (Williams 1997), and use MC sampling for the Tanh network (cf. Fig. S3).
Fig. S3 (in combination with Fig. S2) highlights that also approximated kernel values be used to study architectures with no known closed-form solution for Eq 4 as the posterior seems to be only marginally effected by the MC estimation.
2

---

Under review as a conference paper at ICLR 2022
Fig. S4 visualizes that NNGP kernels for common architectural choices do not encode for Euclidean distances. Kemels that monotonically decrease with increasing distance are, however, important to apply the KDE interpretation of SM
Finally, in Fig. SS we investigate more challenging dataset with two concentric rings. Note, that the center region well the region between the rings can be considered OOD (not included in the support of p(z)). good uncertainty-basse OOD detector should therefore depict high uncertainties in those regions, which is not the for ReLU networks.
0.15
MC
0.05
10a
10
10f
10a
No. MC samples
24ayer ReLU
B-ayer RetU
2-ayer orf
8-byyer orf
Figure S3: Monte Carlo error when estimating NNGP kernel values. Eq. requires estimation whenever no analytic kernel expression is available (for instance, when using a byberbolic tangent non- linearity as in Fig. [2). Here, we visualize the error caused by this approximation for ReLU and function (erf) networks when computing kernel values k(x.,, x.). Eq. 4 requires recursive estima- tion of expected values, where estimate each of them using N samples (N [102 10 10 10%)). Note, that even small eтгоrs can cause eigenvalues of the kernel matrix to become negative. However, with our chosen likelihood variance of 0.02 experience no numerical instabilities during inference, and obtain consistent results using either analytic or estimated (N 10") kernel matrices.
K0 x,)
0
[o Xella
RBF
ReLU 1-layer
ReLU 2-layer
Tanh 1-layer
Tanh 2-layer
Figure S4: NNGP kernel values do generally reflect Euclidean distances. This figure shows kernel values k( xp)) plotted as function of the Euclidean distance ||æ Z,,l2 (using pairs of training points from two Gaussian mixtures dataset). As outlined in See. [ and SM the interpretation of an RBF kernel as Gaussian kernel that can be used in KDE of p(x: is important to justify implied OOD, at least for low-dimensional problems. Unfortunately, the kernels induced by common architectures do not seem to be distance-aware and are thus not useful for KDE.

---

Under review as a conference paper at ICLR 2022
-10
-15-
-5--0- -3
10
15
15
(a) 2-layer ReLU (5)
(b) 2-layer ReLU (100)
(c) 2-layer ReLU (xc)
15
St
-10-
-15-10-
(d) RBF (500)
a7 (f.) RBF net (oo)
(I) RBF kemel
Figure S5: Standard deviation (f-) ) of the predictive posterior. We perform Bayesian inference on dataset composed by two concentric rings (SM comparing posterior uncertainties of GPs with NNGP kernels and an RBF kernel with those obtained by finite-width neural networks. Only the function space prior induced by an RBF kernel RBF network causes epistemic uncertainties that allow outlier detection in the center or in between the two circles. However, the RBF network's uncertainties decrease with increasing input norm, which can be counteracted by further increasing of (Sec. 3
C.1 2D CLASSIFICATION
In this section, we consider the same dataset as in Fig 2 in classification setting instead of regression. In this setting, exact inference is intractable even when using GPs and approxi- mations are needed. In particular, we study classification with the logistic likelihood function p(y f(; = s( s-yyf(a (2; w)) where s is the sigmoid function. We use HMC to sample from the posterior distribution in the finite-width limit, using a standard normal prior where the variance is inversely scaled by the hidden-layer's width. The results are reported in Fig S6 for ReLU, cosine and RBF networks.
As clearly depicted in the plots, our findings regarding the importance of the prior in function space can be extended also for the classification case. Indeed, the uncertainty captured by the ReLU architecture appears unsuitable for OOD detection given that the data distribution is not captured and the disagreement low also in regions of the 2D plane that do not contain any training data points (also see see| Kristiadi et al. 2020 202I). This pathology appears in both cases: when the disagreement is considered over the functions o of.) or over the sigmoid outputs (fs)))) By contrast, also for this new choice of likelihood, the cosine and RBF architecture exhibit uncertainty that conveys similar useful properties for OOD detection as seen the regression example in Fig 2 Despite this empirical correspondence, is important to note that the direct correspondence with the KDE technique that justifies the usefulness of OOD properties of the RBF kernel in the case of GP regression (see Sec. B does not directly apply for classification. Indeed in the latter, we do not have an analytical form of the posterior as in Eq. due to the non- non-Gausse Gaussianity of the logistic likelihood. Therefore our observations
can not be readily generalized outside the settings of our experiments and further theoretical analysis are needed to asses an analogous justification for non-Gaussian likelihoods.
C.2 SPLITMNIST REGRESSION
In this section, we consider SplitMNIST tasks (Zenke et al., 2017). which are binary decision tasks where the original MNIST dataset is split into five tasks containing two digits each: 0/1, 2/3, 4/5, 6/7, 8/9. To perform exact inference via Gaussian Processes, we consider each binary decision task regression problem with labels 1/1. For computational reasons, the training set of each task is reduced to 1000 samples, but test sets remain at their original size. Results are reported in Table ST]
For the chosen length-scale parameters I, RBF kernels show best generalization. Though, they don't excel at OOD detection compared to other kernel choices. In SMBB we discussed that epistemic uncertainties induced by the RBF kernel can be viewed approximation to p(z) when the
4

---

Under review as a conference paper at ICLR 2022
-0
(a) a (f-) 2-tayer ReLU
(b) a(f-) 1-tayes Cosine
(c) off) 1-layer RBF
(fs())) 2-byer ReLU
(0) (s((f)) 1-layer Cosine
(s(( 1-hyer RBF
(g) (f.)
1-aayer ReLU
(h) (ff.) 1-layer Cosine
s(f)) 1-layer RBF
Figure S6: Standard deviation over the logits fa and mean and standard deviation over the sig- moid outputs sf.)) for 2D classification problem. We perform approximate Bayesian inference with HMC for finite-width networks on a mixture two Gaussians dataset considering different priors in function space induced by different architectural choices. The problem classification task using HMC to sample from the posterior. We show the standard deviation a of..) ) of logits f. in (a, b, c), the standard deviation (fx()) of the predicted probability for the input being the positive class in (d, e, f), and the corresponding mean sff.) (g, h, i). The ReLU network has hidden units, the cosine has 100 hidden units, and the RBF network has 500 hidden units.
treated as
Table S1: GP regression on SplitMNIST tasks. We perform GP regression on a single SplitMNIST task using the kernels reported in the first column (entries show mean and standard deviation when using different SplitMNIST tasks for training). AUROC values are computed when considering the test data from all remaining SplitMNIST tasks as OOD data, or by taking the test data of FashionMNIST OOD.
Acc. Train
Acc. Test
AUROC SplitMNIST
AUROC FashionMNIST
RBF (1 1)
100,0=t.00
99,35 #.77
8497
.8931.062
RBF 5)
100.0#.000
99.45 99.45=.582
89221.057
97991.010
RBF (1 = 10)
t.000
99.4 42
884 884
t900
ReLU l-layer
t.334
98.86
884
+.52
ERF 1 I-layer
49.68
98.76 1.11
884 060
9964.002
Cosine l-layer
99.68
98.88
880#.062 062
-93
5

---

Under review as a conference paper at ICLR 2022
dataset size goes to infinity D oo, and the length-scale goes to zero | - 0. As expected, such approximation to p(z) may deteriorate on digeesinnnii image data, presumably as the Euclidean distance does not capture the geometry of the image manifold.
2
10H
10+
5-
A
0-
(a) IND f.
(b) IND (f.)
(c) IND
2
3
3
0-
(d) 00D - f-
(e) OOD o0D-o(ff)
(f) OOD
Figure S7: 2D visualizations of the SplitMNIST posterior for a GP with RBF kernel (1 = 5). The plotted 2D linear subspaces are determined by 3 images (colored dots, denoted b and c). In the upper row, the yellow dot represents the in-distribution (IND) test sample with the highest uncertainty (see image in subplot (c)), while the two green dots the (in-distribution) training samples with smallest Euclidean distance to image c. In the lower row, the yellow dot represents the OOD test image with lowest uncertainty (see image in subplot (f)). Again, the two green dots are the closest IND training samples. Image d is in both cases a randomly chosen point on the 2D subspace. Subplots (a) and (d) show the posterior mean (f.)) ). and subplots (b) and (e) the posterior's standard deviation of..) Subplots (c) and (f) show the images corresponding to the 4 highlighted points on the 2D subspaces.
To gain a better intuition on why the RBF kernel does not excel in OOD detection for image data, we visualize the uncertainty behavior in Fig. S7 The figure shows 2D linear subspaces of the 784D image space. These subspaces are determined by three images (see caption for details). As can be seen in Fig. SS7p SS7p. a too small length-scale might cause test points to be not included in the low-uncertainty regions. On the other hand, if the length-scale is set too high, OOD points may fall inside low-uncertainty regions. The situations depicted in the figure show that for the given training set there is no trade-off length-scale that prevents such behavior (because some OOD points have less Euclidean distance to training points than some test points). Using RBF kernel with metric that encodes similarities in image space rather than using Euclidean distance might overcome these problems.
C.3 CONTINUAL LEARNING VIA UNCERTAINTY-BASE REPLAY
In Sec. 7 we mention that the desideratum of having low uncertainty only in-distribution inputs opens the possibility of considering uncertainty as an energy function from which we can sample. Thus, the uncertainty landscape can be used to construct generative model.
In this section, we use continual learning PParsi et al. 2019) to demonstrate this conceptual idea. In continual learning, a sequence of tasks D DIT) is learned sequentially such that each task has to be learned without access to data from past future tasks. A common approach continual learning is via replay (Lin) 1992). In this case, the current task t is learned with the available training data ((t) as well as datasets D(T) ..... ((t-1) which are supposed to represent the data distributions of the previous tasks. In the simplest case, Dit) is constructed by storing a subset of D(() (Lin 1992).
Other approaches learn a separate generative model that can be used replay (fake) data D(e) while a
new task is learned (e.g.. [Shin et al., 2017 von Oswald et al. 2020)
We consider this (escudo-rreplay scenario without the need of maintaining a separate generative model. In Fig. S8 consider regression problem split into two tasks. Note, that learning series

---

Under review as a conference paper at ICLR 2022
3
Tue Mean Taining Data Prod. Dist.
Tue Mean
3+
Trining
Training Data Samplec Input Pred. Dist.
Sampled input
0-
D-
Prod. Dist.
D-
0
(a) Data from all tasks
(b) Data from task
(c) Data from task 2 + replay
Figure S8: Continual learning with uncertainty-basee replay. As mentioned in Sec. 7 if uncer-
tainty should only be low on in-distribution data, a generative model can be obtained by sampling from regions of low uncertainty. Here, wc use this idea to realize a replay-based continual learning algorithm (Lin 1992) without the need of storing data or maintaining separate generative model. In this case, we split a polynomial regression dataset into two tasks (denoted by the black vertical bar). The posteriors in this figure are obtained via GP regression using NNGP kernel corresponding to a 1-layer Cosine network.
(a) All data is seen at once (no continual learning). (b) Only data of task is seen obtain the posterior. The orange vertical lines denote input locations sampled from low uncertainty regions via rejection sampling. Task I's posterior is then used to label those input locations (orange dots). (e) The replay samples generated with the model of task 1 (orange dots) are used together with the training data of task 2 to obtain combined posterior over all tasks.
of ID regression problems is challenging for most continual learning algorithms,, especially those relying on the recursive Bayesian update: p(w (w D(U), D(2)) xx p(D((2 w))(w D(U) (Henning al.) 202I). By contrast, (pseudo-)replay methods can solve this task with ease the structure of the data in such low-dimensional problem is easy to capture by a generative model.
In our experiment, two tasks are created by splitting the dataset of ID function into two parts as illustrated in SSp. The left half is considered the first task, and the right half the second task, respectively. The blue posteriors shown in all three subpanels are obtained with GP using the NNGP kernel of 1- 1-layer Cosine network, which should exhibit OOD properties similar to an RBF kernel (cf. Sec. In bSh, the posterior is obtained using the combined datasets from both tasks (no continual learning). In S8., the posterior is obtained using only the training data of the first task. After this (first task's) posterior is obtained, we can use it to perform pscudo-replay as outlined below to generate (11) which can be mixed with the data of the second task D(2) to produce the posterior in Fig. [S8.. The posterior in Fig. SS looks qualitatively similar to the one in Fig. [88 even though it has been obtained without direct access to the first task's training data D)1).
To generate ({1) while learning the second task, we need access to the posterior of the first task. Note, the model from the previous task is often kept in memory by continual learning algorithms, for instance, to use it for regularization purposes (Kirkpatrick et al.) 2017) or to replay data with previously trained generative models (Shin et al. 2017 We generate in-distribution input locations (denoted by orange vertical bars in Fig. (S8)) via rejection sampling as in Fig. 7 For that, we define epistemic uncertainty as an energy function (xx.) = a (z.)) and construct a Boltzmann distribution 5 f(a:.) xx exp (- E(x.)/T) where we set the temperature T = 1 As uncertainty of the posterior of Fig. SSP is only low for in-distribution inputs, can assume obtained input locations similar to those represented in D(1), To construct dataset DD1), we use the posterior of Fig. S8 to sample predictions y. (orange dots in Fig. S8b and Fig. S88).
7 As we use a non-parametric model (a GP) which is represented by the training data ( keeping the model in memory requires store D(1) too, which is violation of continual learning desiderata. However, this just a conceptual example. the same experiment would performed via approximate inference in parametric model (e.g.. HMC on a corresponding finite-width network), then the (approximate) posterior of the first task can stored without storing D) In case exact inference can be performed, the Bayesian recursive update yields a sufficient continual learning algorithm.
'Note, if Fl(m.) x log p(x.), then this process yields exact samples from the input distribution. For instance, considering the limiting case of SM where a (f. (a.)) Rv 1 p(x.), ), reasonable choice of energy could be F(x.) = log (f.
7

---

Under review as a conference paper at ICLR 2022
In summary, this section presents example application that arise if the desideratum of high uncertainty on all OOD inputs (i.e., robust OOD detection) is fulfilled by a Bayesian model. Whether BNNs can be constructed to fulfill this desideratum real-world data is, however, unclear at this point.
D EXPERIMENTAL DETAILS
In this section, we report details on our implementation for the experiments conducted in this work. In all wwo-dimensional experiments: the likelihood is fixed be Gaussian with variance 0.02. Unless noted otherwise, the prior in weight space is always width-aware centered Gaussian with o 1.0 and of 1.0 except for the experiments involving RBF networks where select o 200.0. The RBF kernel bandwidth was fixed to 1.0 in all corresponding experiments.
1D regression. We construct the ID regression example (e.g., Fig Ta) by defining p(æ:) uniformly within the ranges [1.0, 1.3), 3.8] and [5.2, 5.5). and p(y x)) [() with f(z) sin(2z) sin (22x) + sin / (x) and N 0.22). The training set has size 20.
Periodic 1D regression (only Fig. We construct this ID regression example by defining p(æ) uniformly within the range [0.0, 12.5]. and ply x)) as (())+e with f(x) sin(x) and (0,33))
The training set has size 30.
Gaussian Mixture. We created two-dimensional mixture of two Gaussians with means f41 (-2, 22), (2 and covariance Σ = 0.5 I and sampled 20 training data points.
Two rings. We uniformly sampled 50 training data points from two rings centred in (0, 0) with inner and outer radii R.,1 = Ro,1 4, R4,2 8, R.,2 9, respectively.
HMC. We use 5 parallel chains for 5000 steps with each constituting 50 leapfrog steps with stepsize 0.001 for width and 0.0001 for width 100. We considered burn-in phase of 1000 steps and collected a total of 1000 samples.
8


Reviews:
[{"review_id": "krvYjTD_bUA", "reviewer": "Reviewer_g5SX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths \n1. The manuscript includes not only finite-width networks but also infinite-width networks in the analyses. \n2. The manuscript points out potential problems when naively using BNNs for ODD detection. \n\nWeaknesses\n1. The manuscript does not provide any theoretical analyses for BNNs on OOD problems.\n2. All the conclusions are made from toy simulated datasets.", "summary_of_the_paper": "The manuscript challenges the widely believed assumption that Bayesian neural networks (BNNs) are well suit for out-of-distribution (OOD) detection, by showing empirical results obtained using infinite-width (allowing the exact inference, because a network can be equivalently represented by a GP with the included kernel) and finite-width networks (requiring the approximate inference). The manuscript provides several observations in line with this purpose (to disclose potential problems when BNNs are used for ODD detection).  For example, the exact inference in infinite-width networks does not necessarily lead to desirable OOD behavior, e.g., the posterior function standard deviation obtained from an infinite-width 2-layer ReLU does not reflect the underlying data generating process and this uncertainty is not suit for OOD detection. In addition, this observation is consistent with the corresponding finite-width networks. ", "summary_of_the_review": "The manuscript has a certain contribution that it points out potential problems when naively using BNNs for ODD detection. However, I think that this contribution is not enough due to the same reasons I listed as the weaknesses of this study above.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8b1Xh34caF", "reviewer": "Reviewer_fGuy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I believe this is an interesting paper. However, it lacks novelty in the sense that everything that is covered in the paper is more or less already known in the machine learning community. Furthermore, it is natural that the posterior predictive uncertainty is not suitable for OOD detection in some cases. The reason is that the posterior uncertainty can be high simply because there is no data in a particular region.\n\nIt is natural to expect that exact inference in infinite-width networks under common architectural choices does not necessarily lead to desirable OOD behavior. This is especially true if the model assumptions are wrong.\n\nThe idea that weight-space prior has a strong effect on OOD performance is something to be expected, since it will have a big impact in the posterior uncertainty.\n\nI do not see why the use of incorporating prior knowledge, that is usually encoded in an input-domain agnostic manner, can negatively affect OOD uncertainties. It will reduce uncertainty, but that need not be bad for OOD detection. In fact, it can be beneficial.\n\nSumming up, I believe that, although this paper shows some interesting concepts or ideas, it is does not provide a significant break thought in terms of analyzing OOD and the obtained results are more or less already known. This will limit its impact in the community.\n\nI also the paper a bit misleading since in several figures the data presented for training is incompatible with the assumptions implied by the chosen prior. For example, in Figure 6 it seems that the RBF kernel is better for OOD detection since it will lead to higher uncertainties. I do not see this. In fact, it seems to me that it will be better to use the ESS kernel for that. (I mean OOD in the y values no the x values).\n\nI also have the feeling that this paper is using the uncertainty on y to detect OOD in the x domain which is counterintuitive. I believe that if you want to detect OOD in the x domain you should have a model for x, not y given x, which is what BNN and GPs do.\n\nThe paper is more or less clear, however, and well written.\n", "summary_of_the_paper": "This paper carries out an analysis that motivates that the use of the Bayesian predictive distribution and its uncertainty is not appropriate for detecting out of distribution data. The paper focuses on the case of Bayesian Neural networks and its infinite-wide generalization. Namely, a Gaussian process. The paper has no experiments but gives illustrative insights about why the Bayesian posterior is not suitable for detecting out of distribution data. They show that exact inference with GPs (infinitely wide networks) does not lead to desirable OOD detection. They discuss desirable kernel features for OOD. They emphasize that the choice of weight-space prior has a strong effect on OOD performance. They argue that there is a trade-off between good generalization and having high uncertainty on OOD.\n", "summary_of_the_review": "A well written paper showing interesting concepts that are mostly well known in the machine learning community. The paper lack novelty.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "3Q0B10GpMeN", "reviewer": "Reviewer_HkGA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and clear, and I enjoyed reading it.\nI think the topic is extremely timely and I fundamentally agree with the authors that there is not necessarily any good reason to think that BNNs should be good for OOD detection.\nWhile the analysis does not really involve anything new from a technical level, the observations the authors make have important implications for the field and have not, to the best of my knowledge, really been discussed before.\nHowever, there are aspects that I believe could be improved.\nTherefore, I am willing to accept the paper, but only tentatively.\n\nMy main reservation is that the paper only truly considers regression tasks, as the classification task it does consider is done via regression.\nWhile the authors do this to avoid having to use approximate inference, classification with classification likelihoods tends to perform quite differently to classification by regression.\nThis is particularly relevant, as most works assessing BNNs on OOD detection that I am aware of do so on classification, not regression.\nIt would be good if the authors could perform the same HMC using logistic regression on their toy problems.\n\nAnother reservation I have is that practically all the examples are low-dimensional and toy.\nWhile I don't think this is strictly necessary for acceptance, since the paper makes no strong claims about more difficult problems, the paper would be significantly strengthened by using more complicated problems.\n\nFinally, it would be good if the authors could comment more about OOD generalization.\nThis would be particularly relevant, as there has been a recent surge of interest in the performance of methods on OOD datasets such as CIFAR-10-C.\nIn particular, the authors propose a method for validating OOD properties in Sec. 7 as an alternative to these sorts of datasets.\nHowever, the authors do not really provide many details as to how one could achieve this, nor do they provide any experiments where they actually use this proposal.\nI think that an example of actually using this would be very useful in gauging the effectiveness of the authors' proposal.\n\nThere are also some references that it would be good if the authors could add or discuss, although this is more minor.\nFor instance, the NNGP equivalence for deep networks was derived concurrently with Lee et al. (2018) and more rigorously proven in [1].\nThe authors should also discuss the recent work of [2], which proposes a new BNN prior to improve OOD generalization.\nFinally, the authors may find it interesting to discuss [3], which provides an explicit example of how the uncertainty of a BNN may fail in classification settings.\n\n\nMinor points: \n- Have the authors checked how well the HMC converges? \nAccording to the experimental details, the HMC is only run for a relatively short amount of time, even accounting for parallel chains.\n\n\nReferences:\n[1] https://arxiv.org/abs/1804.11271\n[2] https://arxiv.org/abs/2106.11905\n[3] https://arxiv.org/abs/2010.02709", "summary_of_the_paper": "In this work, the authors challenge the assumption that underlies many recent works that Bayesian neural networks should be well-suited to out-of-distribution detection.\nIn order to do this, the authors focus on a function-space view by examining the properties of infinite-width BNNs.\nThey use this analysis to argue that BNNs may not necessarily be well-suited to OOD detection.\nThey further argue that there is a tradeoff between OOD generalization and uncertainty, and finally propose an alternative method of validating OOD properties of models.", "summary_of_the_review": "In summary, I found that this paper represents interesting and timely work that should inform future discussion as to how we think about OOD detection and generalization with BNNs.\nHowever, I found the experimental evaluation somewhat lacking, and therefore am only tentatively recommending accept at the moment.\nI look forward to reading the other reviews and to hearing the author responses.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors question the assumption that the epistemic uncertainty provided by Bayesian neural networks should be useful for out of distribution detection. They start their analysis in the infinite width limit so as to be able to understand how the induced kernels in a Gaussian process behave. The paper also discusses the potential tradeoffs between generalization and detection. Overall, the paper presents some facts that, while not surprising, (Reviewer fGuy), are helpful in questioning the default assumption. Overall, though, the combination of the lack of surprise with the multi-part, somewhat loosely connected message reduces the quality of the submission.
Decision: Reject