Year: 2021

Abstract:
The need to avoid confident predictions on unfamiliar data has sparked interest in out-of-distribution (OOD) detection. It is widely assumed that Bayesian neural networks (BNNs) are well suited for this task, as the endowed epistemic uncertainty should lead to disagreement in predictions on outliers. In this paper, we question this assumption and show that proper Bayesian inference with function space priors induced by neural networks does not necessarily lead to good OOD detection. To circumvent the use of approximate inference, we start by studying the infinitewidth case, where Bayesian inference can be exact due to the correspondence with Gaussian processes. Strikingly, the kernels induced under common architectural choices lead to distributions over functions which cause predictive uncertainties that do not reflect the underlying data generating process and are therefore unsuited for OOD detection. Importantly, we find this OOD behavior to be consistent with the corresponding finite-width networks. To overcome this limitation, useful function space properties can also be encoded in the prior in weight space, however, this can currently only be applied to a specified subset of the domain and thus does not inherently extend to OOD data. Finally, we argue that a trade-off between generalization and OOD capabilities might render the application of BNNs for OOD detection undesirable in practice. Overall, our study discloses fundamental problems when naively using BNNs for OOD detection and opens interesting avenues for future research.

heading: 1 INTRODUCTION
text: One of the challenges that the modern machine learning community is striving to tackle is the detection of unseen inputs for which predictions should not be trusted. This problem is also known as out-of-distribution (OOD) detection. The challenging nature of this task is partly rooted in the fact that there is no universal mathematical definition that characterizes an unseen input x∗ as OOD as discussed in Sec. 2. Without such a definition, there is no foundation for deriving detection guarantees under verifiable assumptions. In this work, we consider OOD points as points that are unlikely under the distribution of the data p(x) and points are outside the support of p(x). Importantly, we do not claim to provide a meaningful definition of OOD, but rather argue that the justification of an OOD method can only be assessed theoretically once such a definition is provided. While it appears natural to tackle the OOD detection problem from a generative perspective by explicitly modelling p(x), this paper is solely concerned with the question of how justified it is to deploy the predictive uncertainty of a Bayesian neural network (BNN) for OOD detection. As recent developments forecast an increasing integration of deep learning methods into industrial applications, it becomes essential to provide the theoretical groundings that justify the use of uncertainty for OOD detection, a task that is crucial for safety-critical applications of AI and reliable prediction-making.
When BNNs are used in supervised learning, the dataset is composed of inputs x ∈ X and targets y ∈ Y which are assumed to be generated according to some unknown process: D i.i.d.∼ p(x)p(y | x).The goal of learning is to infer from D alone the distribution p(y | x) in order to make predictions on unseen inputs x∗. In the case of deep learning, this problem is approached by choosing a neural network f(·;w) parametrized by w, and predictions are made via the conditional p ( y | f(x;w)
) (also called the likelihood of w for given x,y). Assuming that the induced class of hypotheses contains some ŵ such that p ( y | f(x; ŵ) ) = p(y | x) almost everywhere on the support of p(x), Bayesian statistics can be used to infer plausible models p(w | D) under the observed data given some prior knowledge p(w) (see MacKay (2003)). This parametric description together with the network implicitly induces a prior over functions p(f) (Williams, 1997; Fortuin, 2021). Believing in the validity of a subjective choice of prior (O’Hagan, 2004), Bayesian inference comes with a
multitude of benefits as it is less susceptible to overfitting, allows to incorporate new evidence without requiring access to past data (Farquhar & Gal, 2018) and provides interpretable uncertainties.
The uncertainty captured by a BNN can be coarsely categorized into aleatoric and epistemic uncertainty. Aleatoric uncertainty is irreducible and intrinsic to the data p(y | x). For instance, a blurry image might not contain enough information to identify unique object classes. On the other hand, a BNN also models epistemic uncertainty by maintaining a distribution over parameters p(w | D).1 This distribution reflects the uncertainty about which hypothesis explains the data and can be reduced by observing more data. Arguably, aleatoric uncertainty is of little interest for detecting OOD inputs. However, the recent advent of overparametrized models which further extend the hypothesis class, deceptively motivated the idea that the epistemic uncertainty under a Bayesian framework might be intrinsically suitable to detect unfamiliar inputs, and
therewith implying, that BNNs can be used for OOD detection. This conjecture is intuitively true if the following assumption holds: the hypotheses captured by p(w | D) must agree on their predictions for in-distribution samples but disagree for OOD samples (Fig. 1a). Certain Bayesian methods satisfy this assumption, e.g. a Gaussian process regression with an RBF kernel (cf. Sec. 4). However, the uncertainty induced by Bayesian inference does not in general give rise to OOD capabilities. This can easily be verified by considering the following thought experiment (Fig. 1b): Assume a model class with only two hypotheses g(x) and h(x) such that g(x) 6= h(x)∀x and data being generated according to g(x) on a restricted domain. Once data is observed, we can commit to the ground-truth hypothesis g(x) and thus lose epistemic uncertainty in- and out-of-distribution alike. Finite-width neural networks, by contrast, form a powerful class of models, and are often put into context with universal function approximators (Hornik, 1991). But is this fact in combination with Bayesian statistics enough to attribute them with good OOD capabilities? The literature often seems to imply that a BNN is intrinsically good at OOD detection. For instance, the use of OOD benchmarks when introducing new methods for approximate inference creates the false impression that the true posterior is a good OOD detector (Louizos & Welling, 2017; Pawlowski et al., 2017; Krueger et al., 2017; Henning et al., 2018; Maddox et al., 2019; Ciosek et al., 2020; D’Angelo & Fortuin, 2021). Our work is meant to start a discussion among researchers about the properties of OOD uncertainty of the exact Bayesian posterior of neural networks. To initiate this, we contribute as follows:
• We emphasize the importance of the prior in function space for OOD detection, which is induced by the choice of architecture (Sec. 4) and weight space prior (Sec. 5).
• We empirically show that exact inference in infinite-width networks under common architectural choices does not necessarily lead to desirable OOD behavior, and that these observations are consistent with results obtained via approximate inference in their finite-width counterparts.
• We furthermore study the OOD behavior in infinite-width networks by analysing the properties of the induced kernels. Moreover, we discuss desirable kernel features for OOD detection and show that also neural networks can approximately carry these features.
• We emphasize that the choice of weight-space prior has a strong effect on OOD performance, and that encoding desirable function space properties within unknown OOD domains into such prior is challenging.
• We argue that there is a trade-off between good generalization and having high uncertainty on OOD data. Indeed, improving generalization by incorporating prior knowledge (which is usually encoded in an input-domain agnostic manner) can negatively impact OOD uncertainties.
1Note, that the Bayesian treatment of network parameters does not account for all types of epistemic uncertainty such as the uncertainty stemming from model misspecification (Hüllermeier & Waegeman, 2021). We, however, assume the model to be correctly specified.

heading: 2 ON THE DIFFICULTY OF DEFINING OUT-OF-DISTRIBUTION INPUTS
text: While the intuitive notion of OOD sample points (or outliers) is commonly agreed upon, for instance as "an observation (or subset of observations) which appears to be inconsistent with the remainder of that set of data" (Barnett & Lewis, 1984), a mathematical formalization of the essence of an outlier is difficult (cf. SM A) and requires subjective characterizations (Zimek & Filzmoser, 2018). Often, methods for outlier detection are designed based on an intuitive notion (see Pimentel et al. (2014) for a review), and therewith only implicitly define a method-specific definition of OOD points. This also accounts for uncertainty-based OOD detection with neural networks, where a statistic of the predictive distribution that quantifies uncertainty (e.g., the entropy) is used to decide whether an input is considered OOD (Hendrycks & Gimpel, 2017). Commonly, the parameters w of a neural network are trained using an objective derived from Ep(x) [ KL ( p(y | x)||p ( y | f(x;w) ))] (i.e., loss functions based on the negative log-likelihood such as the cross-entropy or mean-squared error loss). Therefore, these networks only have calibrated uncertainties in-distribution with OOD uncertainties not being controlled for unless explicit training on OOD data is performed (eg., Hendrycks et al., 2019). The question this study is concerned with is therefore: does the explicit treatment of parameter uncertainty via the posterior parameter distribution p(w | D) elicit provably high uncertainty OOD? Note, that we do not consider the problem of outlier detection within the training set (Zimek & Filzmoser, 2018), but rather ask whether a deployed model is able to know what it does not know.

heading: 3 BACKGROUND
text: In this section, we briefly introduce the concepts on which we base our argumentation in the coming sections. We start by introducing BNNs, which rely on approximate inference. We then discuss that in the non-parametric limit and under a certain choice of prior, a BNN converges to a Gaussian process, an alternative Bayesian inference framework where exact inference is possible. The connection between BNNs and Gaussian processes will later allow us to make interesting conjectures about OOD behavior. Finally, we introduce generalization bounds from the PAC-Bayes framework, which we will later use to argue that generalization and OOD detection can be conflicting objectives.

heading: 3.1 BAYESIAN NEURAL NETWORKS
text: In supervised deep learning, we typically construct a likelihood function from the conditional density p (y | f(x;w)), parameterized by a neural network f(x;w), and the training data D = {(xi,yi)}ni=1. In BNNs, this is used to form the posterior distribution of all likely network parameterizations: p(w | D) ∝ ∏n i=1 p (yi | f(xi;w)) p(w), where p(w) is the prior distribution over weights. Crucially, when making a prediction with the Bayesian approach on a test point x∗, we do not only use a single parameter configuration ŵ but we marginalize over the whole posterior, thus taking all possible explanations of the data into account: p(y∗ | x∗,D) = ∫ p(y∗ | f(x∗;w)) p(w | D) dw.

heading: 3.2 GAUSSIAN PROCESS
text: Gaussian processes are established Bayesian machine learning models that, despite their strong scalability limitations, can offer a powerful inference framework. Formally (Rasmussen, 2004):
Definition 1 A Gaussian process (GP) is a collection of random variables, any finite number of which have a joint Gaussian distribution.
Compared to parametric models, GPs have the advantage of performing inference directly in function space. A GP is defined by its mean m(x) = E [f(x)] and covariance C (f(x),f(x′)) = E [(f(x)−m(x)) (f(x′)−m(x′))]. The latter can be specified using a kernel function k : Rd × Rd → R and implies a prior distribution over functions:
p(f |X) = N (0,K(X,X)) , (1) where K(X,X)ij = k(xi,xj) is the kernel Gram matrix on the training inputs X , and m(x) has been chosen to be 0. When observing the training data, the prior is reshaped to place more mass in the regions of functions that are more likely to have generated them, and this knowledge is then used to make predictions on unseen inputs X?. In probabilistic terms, this operation corresponds to conditioning the joint Gaussian prior: p(f∗|X∗, X,f). To model the data more realistically, we assume to not have direct access to the function values but to noisy observations: y = f(x)+ with ∼ N (0, σ2Id).
This assumption is formally equivalent to a Gaussian likelihood p(y|f) = N (y|f , σ2).2 The conditional distribution on the noisy observation can then be written as (Rasmussen, 2004):
p(f∗ | X∗, X,y) = ∫ dfp(f∗ | X∗, X,f)p(f | X,y) = N ( f̄∗, C(f∗) ) f̄∗ = K(X∗, X)[K(X,X) + σ
2I]−1y C(f∗) = K(X∗, X∗)−K(X∗, X)[K(X,X) + σ2I]−1K(X,X∗)
(2)
where p(f | X,y) = p(y|f)p(f |X))p(D) with p(D) being the marginal likelihood.
RBF kernel. A commonly used kernel function for GPs is the squared exponential (RBF):
k(x,x′) = exp ( − 1
2l2 ‖x− x′‖22
) , (3)
with the length scale l being a hyperparameter. It is important to notice that the covariance between outputs is written exclusively as a function of the distance between inputs. As a consequence, points that are close in the Euclidean space have unitary covariance that decreases exponentially with the distance. As we will see in Sec. 4, this feature has important implications for OOD detection.
The relation of infinite-width BNNs and GPs. The connection between neural networks and GPs has recently gained significant attention. Neal (1996) showed that a 1-hidden layer BNN converges to a GP in the infinite-width limit. More recently, the work by Lee et al. (2018) and de G. Matthews et al. (2018) extended this result to deeper networks, called neural network Gaussian processes (NNGP). Crucially, the kernel function of the related GP strictly depends on the used activation function. To better understand this connection we consider a fully connected network with L layers l = 0, . . . , L with width Hl. For each input we use xl(x) to represent the post-activation with x0 = x and f l the pre-activation so that f li (x) = b l−1 i + ∑Hl−1 j=1 w l−1 ij x l−1 j with x l j = h(f l j) and h(·) a point-wise activation function. Furthermore, the weights and biases are distributed according to blj ∼ N (0, σ2b ) and wlij ∼ N (0, σ2w Hl
). Given the independence of the weights and biases it follows that the post-activations are independent as well. Hence, the central limit theorem can be applied, and for Hl−1 →∞ we obtain f l(x) ∼ GP(0, Cl). The covariance Cl and therefore the prior over functions is specified by the kernel induced by the network architecture (Lee et al., 2018):
Cl(x,x′) = kl(x,x′)︷ ︸︸ ︷ E [ f li (x)f l i (x ′) ] − =0︷ ︸︸ ︷ E[f li (x)]E[f li (x)]
= σ2b + σ 2 wE(f l−1i (x),f l−1i (x′))∼GP(0,Kl−1)
[ h ( f l−1i (x) ) , h ( f l−1i (x ′) )] , (4)
with Kl being the 2 × 2 covariance matrix formed by kl(·, ·) using x and x′. For the input layer, where no activation function is applied, the kernel is simply given by k0(x,x′) = σ2b + σ 2 w
( xTx′
d
) .
For the remaining layers, instead, the kernel expression is determined by the non-linearity. Some activation functions like sigmoid or hyperbolic tangent do not admit a known analytical form and therefore require a Monte Carlo estimate of Eq. 4 (cf. Fig. S3). For others, Eq. 4 can be computed in closed form (e.g., Williams, 1997; Cho & Saul, 2009; Tsuchida et al., 2018; Pang et al., 2019; Pearce et al., 2020). We list some kernel functions that are important for this study below, such as the one for ReLU networks (Cho & Saul, 2009):
klReLU(x,x ′) = σ2b + σ2w 2π
√ kl−1(x,x)kl−1(x′,x′) ( sin θl−1x,x′ + (π − θ l−1 x,x′) cos θ l−1 x,x′ ) θlx,x′ = cos −1 ( kl(x,x′)√
kl(x,x)kl(x′,x′) ) (5) Interestingly, for some non-linearities a kernel that carries similar properties as the RBF in Eq. 3 can be induced by neural networks. This is the case for the cosine activation function which has the
2Note, that the Gaussian likelihood assumption is commonly applied to neural network regression through the use of a mean-squared error loss.
following closed form solution in the single hidden layer case (Pearce et al., 2020):
k1cos(x,x ′) = σ2b + σ2w 2
( exp ( −σ 2 w‖x− x′‖22
2d
) + exp ( −σ 2 w‖x+ x′‖22
2d − 2σ2b
)) (6)
Similarly, also the exponential activation function as deployed in RBF networks (Broomhead & Lowe, 1988) leads to similar properties. This particular class of neural networks defines computational units as linear combinations of radial basis functions f(x) = ∑H j=1 wj exp ( − 12σ2g ‖x− µj‖ 2 )
+ b. These networks, in the infinite-width case and when assuming a Gaussian prior over the centers µj ∼ N (0, σ2µI) and wj ∼ N (0, σ2w), also converges to a Gaussian process with an analytical kernel function:
k1rbfnet(x,x ′) = σ2b + σ 2 w ( σe σµ )d exp ( − ‖x‖ 2 2σ2m ) exp ( − ‖x− x ′‖2 2σ2s ) exp ( − ‖x ′‖2 2σ2m ) (7)
where 1/σ2e = 2/σ 2 g + 1/σ 2 µ, σ 2 s = 2σ 2 g + σ 4 g/σ 2 µ and σ 2 m = 2σ 2 µ + σ 2 g . The equivalence with the RBF kernel is explicit in the limit σ2µ →∞ (Williams, 1997).

heading: 3.3 PAC-BAYES GENERALIZATION BOUND
text: Considering the supervised learning framework introduced in Sec. 1, the PAC theory (Valiant, 1984) establishes a probabilistic bound on the generalization error of a given predictor. The theory is referred to as PAC-Bayes (McAllester, 1999; Catoni, 2007) when a prior distribution p is defined to incorporate prior domain knowledge. Considering a distribution3 q on the hypothesis h ∈ H and the space B(Y) of all conditional distributions p(y | h(x)), we can define a bounded loss function such that l : B(Y)×Y → [0, 1]. This gives rise to the empirical and true risk as RD(q) := 1N ∑N i=1 Eh∼q[l(p(y | h(xi)),yi)] and R(q) := E(x?,y?)∼p(x,y)Eh∼q[l(p(y | h(x?)),y?)], respectively, where p(x,y) = p(x)p(y | x). Note that when q is the Bayesian posterior as, for instance, in the GP case, the Bayes risk can be defined as: RB(q) := E(x?,y?)∼p(x,y)[l(Eh∼q[p(y | h(x?))],y?)] and it can be shown that RB(q) ≤ 2R(q) (Seeger, 2003). Therefore, a bound over R also implies a bound over RB (Seeger, 2003). The PAC bound gives a probabilistic upper bound on the true risk R(q) in terms of the empirical risk RD(q) for a training set D as formulated in the following theorem:
Theorem 1 (PAC-Bayes theorem (Catoni, 2007)) For any distribution p(x,y) over X × Y , for any distribution q and prior p on a hypothesis spaceH, for any δ ∈ (0, 1] and β > 0 the following holds with probability at least 1− δ over the training set D ∼ p(x,y) of cardinality N :
∀q : R(q) ≤ 1 1− e−β
[ 1− exp ( − βRD(q)− 1
N
( KL(q||p) + log 1
δ
))] . (8)
For a fixed p, D, δ, minimizing Eq. 8 is equivalent to minimizing NRD(q) + KL(q||p). The requirement of a bounded loss function makes the use of the mean-squared error, and thus the negative log-likelihood, not applicable to this bound. Nevertheless, as long as the loss function measures the quality of predictions, the bound in Eq. 8 can be used to assess the generalization capabilities of a model. For this purpose, in our analysis we use as a surrogate loss the following Reeb et al. (2018): lexp(p(y | h),y) = 1− exp [ − (Ey [p(y|h)]−y) 2
σ2
] . Interestingly, for small deviations we can take the
first-order Taylor expansion and recover the MSE lexp(Ey[p(y | h)],y, ) ≈ (Ey[p(y | h)]− y)2/σ2.

heading: 4 THE ARCHITECTURE STRONGLY INFLUENCES OOD UNCERTAINTIES
text: In the previous section, we recalled the connection between BNNs and GPs, namely that Bayesian inference in an infinite-width neural network can be studied in the GP framework (assuming a proper choice of weight-space prior). This connection allows studying how the kernels induced by architectural choices shape the prior in function space, and how these choices ultimately determine OOD behavior. In this section, we analyze this OOD behavior for traditional as well as NNGPinduced kernels. To minimize the impact of the approximations on our results, we exclusively focus on conjugate settings, i.e., regression (see SM C.1 for classification results). Furthermore, this choice
3Note that this distribution does not necessarily need to be the Bayesian posterior.
of Gaussian likelihood induces a direct correspondence between function values and outcomes up to noise corruptions. Hence, prior knowledge about outcomes can be encoded in function space through the choice of a meaningful function space prior.
Uncertainty quantification for OOD detection. Uncertainty can be quantified in multiple ways, but is often measured as the entropy of the predictive posterior. The predictive posterior, however, captures both aleatoric and epistemic uncertainty, which does not allow a distinction between OOD and ambiguous inputs (Mukhoti et al., 2021). While our choice of likelihood does not permit the modelling of input-dependent uncertainty (Gaussian with fixed variance), a softmax classifier can capture heteroscedastic aleatoric uncertainty arbitrarily well by outputting an input-dependent categorical distribution.4 For this reason, uncertainty should be quantified in a way that allows OOD detection to be based on epistemic uncertainty only. The function space view of GPs naturally provides such measure of epistemic uncertainty by considering the standard deviation of the posterior over function values σ(f∗) ≡ √ diag (C(f∗)) (illustrated in Fig. 1a). Such measure can be naturally translated to BNNs by looking at the disagreement between network outputs when sampling from p(w | D). Note that in classification tasks all models drawn from p(w | D) might lead to a highentropy softmax without disagreement, and therefore only an uncertainty measure based on model disagreement prevents one from misjudging ambiguous points as OOD. As OOD detection is the focus of this paper, we always quantify uncertainty in terms of model disagreement.
Bayesian statistics and OOD detection have no intrinsic connection. As conceptually illustrated in Fig. 1b and demonstrated using exact Bayesian inference in Fig. 3b, predictive uncertainties are not necessarily reflective of p(x) (and thus not suited for OOD detection), irrespective of how well the underlying task is solved.
GP regression with an RBF kernel. We next examine the predictive posterior of a GP with squared exponential (RBF) kernel (Eq. 3). Fig. 3a shows epistemic uncertainty as the standard deviation of p(f∗ | X∗, X,y). We can notice that σ(f∗) nicely captures the data manifold and is thus well suited for OOD detection. This behav-
ior can be understood by considering Eq. 2 while noting that k(x,x′) = const if x = x′, and that the
4There are also expressive likelihood choices to capture heteroscedastic uncertainty for continuous variables (e.g., Zięba et al., 2020).
variance of posterior function values can be written as σ2(f∗) = k(x∗,x∗)− ∑n i=1 βi(x∗)k(x∗,xi) (cf. SM B), where βi are dataset- and input-dependent. The second term is reminiscent of using kernel density estimation (KDE) to approximate p(x), applying a Gaussian kernel, while the first term is the (constant) prior variance. Hence, the link between Bayesian inference and OOD detection can be made explicit, as the posterior variance is inversely related to the input distribution.
Loosely speaking, in GP regression as in Eq. 2 and kernels where the KDE analogy holds, epistemic uncertainty can roughly be described as const − p(x).5 In this view, one starts with high (prior) uncertainty everywhere, which is only reduced where data is observed. By contrast, learning a normalized generative model (e.g., using normalizing flows (Papamakarios et al., 2019)) often requires to start from an arbitrary probability distribution.
The OOD behavior induced by NNGP kernels. Fig. 2a illustrates σ(f∗) for an infinite-width 2-layer ReLU network (see Fig. S2 for other common architectural choices). It is already visually apparent, that in this case the kernel is less suited for OOD detection compared to an RBF kernel. Moreover, we cannot justify why OOD detection based on σ(f∗) would be principled for this kernel as the KDE analogy does not hold. This is due to two reasons: (1) the prior uncertainty k(x∗, x∗) is not constant (Fig. S1), and (2) we empirically do not observe that k(x,x′) can be related to a distance measure (Fig. S4). We therefore argue that more theoretical work is necessary if one aims to justify uncertainty-based OOD detection with common architectural choices from the perspective of NNGP kernels. On this note, maintaining parameter uncertainty and being able to detect OOD samples are often considered crucial requirements of systems deployed in safety-critical applications. Given that Bayesian inference is not intrinsically linked to OOD detection, care should be taken to precisely communicate safety-relevant capabilities of BNNs to practitioners.
However, the NNGP perspective also allows us to choose an architecture such that the BNN’s uncertainty resembles the, for these problems, desirable OOD behavior of the GP with RBF kernel described above. In particular, the cosine (Eq. 6) and RBF (Eq. 7) network induce kernels that are related to the RBF kernel. The last term in Eq. 6 quickly converges to zero for moderate norms of x or x′ (or high σ2b ), which explains why the uncertainty behavior of Fig. 2b is qualitatively identical to Fig. 3a. In case of the RBF network, the RBF kernel is recovered if ‖x‖ and ‖x′‖ are small compared to σ2m, explaining why the uncertainty faints towards the boundaries of Fig. 2c.
These examples show that in the non-parametric limit and for (low-dimensional) regression tasks, BNNs can be constructed such that uncertainty-based OOD detection can be justified through mathematical argumentation. We next study whether the observations made in the infinite-width limit are relevant for studying finite-width neural networks.
Infinite-width uncertainty is consistent with the finite-width uncertainty. For finite-width BNNs exact Bayesian inference is intractable. To mitigate the effects of approximate inference we resort to Hamiltonian Monte Carlo (HMC, Duane et al., 1987; Neal et al., 2011). Fig. 2d to 2f show an estimate of σ(f∗) for finite-width networks corresponding to the non-parametric limits studied above (illustrations with another dataset can be found in Fig. S5). Already for moderate layer widths, the modelled uncertainty resembles the one of the corresponding NNGP. Given this close correspondence, we conjecture that the tools available for the infinite-width case are useful for designing architectural guidelines that enhance OOD detection. We illustrated this on low-dimensional problems by studying desirable function space properties induced by the RBF kernel, which can be translated to BNN architectures.

heading: 5 THE CHOICE OF WEIGHT SPACE PRIOR MATTERS FOR OOD DETECTION
text: For a neural network, the prior in function space is induced by the architecture and the prior in weight space (Wilson & Izmailov, 2020).
5Note, the KDE approximation of p(x) is likely to deteriorate massively if the dimensionality of x increases.
In the previous section, we restricted ourselves to a particular class of weight space priors which allowed us to study the architectural choices in the infinite-width limit. While, in practice, a wide variety of weight space prior choices might lead to good generalization (with respect to test data from p(x)p(y | x), e.g., cf. Izmailov et al. (2021b)), the uncertainty behavior that is induced OOD might vary drastically. This is illustrated in Fig. 4, where for the same network two different choices of p(w) lead to vastly different predictive uncertainties despite the fact that both choices fit the data well.
In Sec. 4, we use the infinite-width limit to obtain a function space view that allows to make interesting conjectures about predictive uncertainties on OOD data. Recently, multiple studies suggested ways to either explicitly encode function space properties into the weight space prior p(w) or to use a function space prior when performing approximate inference in neural networks (Flam-Shepherd et al., 2017; Sun et al., 2019; Tran et al., 2020; Matsubara et al., 2021). However, all these methods depend on the specification of a set XR ⊆ X on which desired function space properties will be enforced (Fig. 5). For instance, the Ridgelet prior proposed by Matsubara et al. (2021) provides an asymptotically correct weight space prior construction that induces a given GP prior. Thus, on XR this method allows to meaningfully encode prior knowledge to guide Bayesian inference. But, an a priori specification of a set XR which can be largely covered by samples xR ∈ XR to ensure the desired prior specification onXR is practically challenging, and conceptually related to the idea of training on OOD data to calibrate respective uncertainties (cf. Sec. 2). Note, we do not aim to phrase this OOD effect as a drawback of these methods, as we do not consider Bayesian inference to be intrinsically related
to OOD detectors. It is, however, important to keep in mind that function space properties deemed beneficial for OOD detection are not straightforward to induce when working in weight space.

heading: 6 A TRADE-OFF BETWEEN GENERALIZATION AND OOD DETECTION
text: An important desideratum that modelers attempt to achieve when applying Bayesian statistics is good generalization through the incorporation of relevant prior knowledge. Is this desideratum generally in conflict with having high uncertainty on OOD data? In this sec-
tion, we provide arguments indicating that the answer to this question can be yes. Consider data with known periodic structure inside the support of p(x) (Fig. 6).
We can either choose to ignore our prior knowledge by selecting a GP prior with RBF kernel (Fig. 6a), or we explicitly incorporate domain knowledge using a function space prior that allocates its mass on periodic functions e.g. the exp-sine-squared (ESS) kernel.6 In the former case, we know that OOD uncertainties will be useful for OOD detection (Sec. 4). In the latter case, however, we see that uncertainties do not reflect p(x). By contrast, the roles are reversed when it comes to assessing generalization.
6Note, that such incorporation of prior knowledge is often done in a way that is agnostic to the unknown support of p(x).
Here, the choice of a periodic kernel has clear benefits, as we can see visually and by comparing the generalization bound introduced in Sec. 3.3 which can be tightened by minimizing NRD (p(f | D)) + KL (p(f | D)||p(f)). As reported in Table 1, the bound’s value for the ESS kernel is lower than the RBF. Interestingly, the bound is minimized if both terms are minimized, and the second term explicitly asks the posterior to remain close to the prior. Therefore,
the second term benefits from restricting the prior function space through the incorporation of prior knowledge, counteracting the need of a rich function space for inducing high OOD uncertainties (Fig. 1). Consistent with this, also the log marginal likelihood log p(D) in Table 1 is higher for the ESS kernel. An orthogonal problem often considered in the literature is generalization under dataset (or covariate) shift (Quiñonero-Candela et al., 2009; Snoek et al., 2019; Izmailov et al., 2021a). In this case, one deliberately seeks to provide meaningful predictions on test data ptest(x) that might not overlap in support with our training input distribution p(x). Therefore, prior knowledge needs to explicitly encode how to obtain "generalization on OOD data" as the data cannot speak for themselves. Such priors are proposed in Izmailov et al. (2021a), but also Fig. 6b can be viewed as an example of how prior encoding specifies how to generalize to OOD data.

heading: 7 ON THE PRACTICAL VALIDATION OF OOD PROPERTIES
text: Relating the epistemic uncertainty induced by a BNN to p(x) opens up interesting new possibilities. As we show in SM B, epistemic uncertainty of a GP with RBF kernel can be related to a KDE approximation of p(x). Therefore, one may consider the epistemic uncertainty as an energy function to create a generative model from which to sample from the input space (Fig. 7)
.Moreover, this approach may be used to empirically validate the OOD capabilities of a BNN. Indeed, OOD performance is commonly validated by selecting a specific set of known OOD datasets (Izmailov et al., 2021b). However, as the OOD space comprises everything except the in-distribution data, it is infeasible to gain good coverage on high-dimensional data with such testing approach. We therefore suggest a reverse approach that checks the consistency in regions of certainty instead of uncertain ones by sampling via the epistemic uncertainty. Indeed, if these samples and thus the generative model based on uncertainty estimates are consistent with the in-
distribution data then a strong indication for trustworthiness is provided. Additionally, a discrepancy measure (Liu et al., 2016; Gretton et al., 2012) can be used to compare the generated samples and training data points and quantitatively assess to which extent the epistemic uncertainty is related to p(x). Moreover, this approach can open up new research opportunities. For instance, a continual learner may use its uncertainty to generate its own replay data to combat forgetting (cf. SM C.3).

heading: 8 CONCLUSION
text: We challenge the common view that uncertainty-based OOD detection with BNNs is intrinsically justified. Our arguments are all based on low-dimensional problems and cannot easily be generalized to real-world problems, but our observations are consistent with the fact that empirically BNNs are often only marginally superior in OOD detection compared to models that do not maintain epistemic uncertainty (Snoek et al., 2019; Henning et al., 2021). Overall, this work highlights fundamental limitations of BNNs for OOD detection that are not solely explained by the use of approximate inference.
Ethics Statement. When machine learning algorithms enter real-world applications, they are not anymore embedded in a controlled environment. Instead, they have to know what they don’t know, which encloses the notion of OOD detection. It is therefore of utmost importance to discuss and understand the justifications for employing uncertainty-based OOD detection. The purpose of our paper is to disclose in a simple but clear manner that these justifications are not obvious, and thus calls for more research in this direction. Our illustrative arguments should, however, not be misinterpreted or generalized, for instance, by assuming OOD properties we derive from an RBF kernel can be applied to high-dimensional image manifolds.
Reproducibility Statement. The source code to reproduce all experiments is available at: anonymous link.

heading: A WHAT IS AN OUT-OF-DISTRIBUTION INPUT?
text: Pimentel et al. (2014) reviews methods for outlier detection, putting them coarsely into five categories: (1) probabilistic, (2) distance-based, (3) reconstruction-based, (4) domain-based, and (5) informationtheoretic. Our focus lies on a probabilistic characterization of an OOD point, where a statistical criterion allows to decide whether a given input is significantly different from the observed training population. In this section, we discuss pitfalls regarding obvious choices for such a criterion in order to highlight the difficulties that arise when attempting to agree on a single (or application-dependent) mathematical definition of outliers.
There are many possible definitions that could be considered for a point to be OOD as we will outline below. Any of these definitions may change the notion of OOD and will therefore affect how a BNN should be designed such that predictive uncertainty adheres to the underlying OOD definition. Considering a generative process p(x), the first question arising is regarding the regions outside the support of p(x) or even the space outside the manifold where samples x are defined on. For instance, assume the data to be images embedded on a lower-dimensional manifold. Are points outside this manifold clearly OOD, given that minor noise corruptions are likely to leave the manifold? Even disregarding these topological issues solely focusing on the density p(x), makes the distinction between in- and out-of-distribution challenging. For instance, a threshold-criterion on the density might cause samples in a zero probability region to be considered in-distribution (Nalisnick et al., 2019a). To overcome such challenges, one may resort to concepts from information theory, such as the notion of a typical set (MacKay, 2003; Nalisnick et al., 2019b). Unfortunately, an OOD criterion based on this notion would require looking at sets rather than individual points. We hope that this short outline highlights the challenges regarding the definition of OOD, but also clarifies that a proper definition is relevant when assessing OOD capabilities on high-dimensional data, where a visual assessment as in Fig. 2 is not possible.

heading: B ON THE RELATION BETWEEN GP REGRESSION AND KERNEL DENSITY ESTIMATION
text: GP regression with an RBF kernel allows a direct understanding of the OOD capabilities that a Bayesian posterior may possess, as the a priori uniform epistemic uncertainty is reduced in direct correspondence to density of p(x).
Below, we rewrite the variance of an input x∗ as defined in Eq. 2:
σ2(f∗) = k(x∗,x∗)− n∑ i=1 βi(x∗)k(x∗,xi) (9)
with βi(x∗) = ∑n j=1 ( K(X,X) + σ2I )−1 ij k(x∗,xj).
Note, that k(x∗,x∗) is a positive constant for an RBF kernel, and that βi(x∗) ≥ 0. Furthermore, as the RBF kernel k(x∗,xj) behaves exponentially inverse to the distance between x∗ and xj , βi(x∗) is approximately zero for all x∗ that are far from all training points. In addition, a training point xi can only decrease the prior variance if it is close to x∗. This analogy closely resembles the philosophy of KDE, which becomes an exact generative model in the limit of infinite data and bandwidth l→ 0.

heading: C ADDITIONAL EXPERIMENTS AND RESULTS
text: In this section, we report additional experiments and results.
Fig. S1 shows the prior’s standard deviation over function values √ k(x∗,x∗) for several NNGP kernels. Note, a kernel that a priori treats locations x∗ differently might not be desirable for OOD detection as the data’s influence on posterior uncertainties might be hard to interpret (cf. Sec. 4).
Fig. S2 shows GP regression results with the GMM dataset (Sec. D) for several NNGP kernels. The plots show that the underlying task can be solved well with all considered architectures (as indicated by the predictive mean f̄∗ that captures the ground-truth targets in-distribution), even though the uncertainty behavior OOD is vastly different and not reflective of p(x).
(a) 1-layer ReLU (b) 1-layer Erf (c) 1-layer Cosine Figure S1: NNGP kernel values √ k(x∗,x∗) for various architectural choices. Note, that the NNGP kernel value k(x∗,x∗) represents the prior variance of function values under the induced GP prior at the location x∗ (cf. Eq. 1). As emphasized in Sec. 4, k(x∗,x∗) is constant for an RBF kernel, which has important implications for OOD detection, such as that a priori (before seeing any data) all points are treated equally. This is not the case for the ReLU kernel, which has an angular dependence and depends on the input norm (cf. Eq. 5 and its dependence on k0(x,x′)). The kernel induced by networks using an error function (Erf) or a cosine as nonlinearity seems to be more desirable in this respect (note the scale of the colorbars).
(a) f̄∗ – 1-layer ReLU (b) σ(f∗) – 1-layer ReLU (c) f̄∗ – 2-layer ReLU (d) σ(f∗) – 2-layer ReLU
(e) f̄∗ – 1-layer Tanh (f) σ(f∗) – 1-layer Tanh (g) f̄∗ – 2-layer Tanh (h) σ(f∗) – 2-layer Tanh
(i) f̄∗ – 1-layer Erf (j) σ(f∗) – 1-layer Erf (k) f̄∗ – 2-layer Erf (l) σ(f∗) – 2-layer Erf
Figure S2: Mean f̄∗ and standard deviation σ(f∗) of the posterior p(f∗ | X∗, X,y) for GP regression with NNGP kernels for various architectural choices, such as number of layers or nonlinearity. Note, that Tanh and Erf nonlinearities are quite similar in shape, which is reflected in the similar predictive posterior that is induced by these networks. We use the analytically known kernel expression for the Erf kernel (Williams, 1997), and use MC sampling for the Tanh network (cf. Fig. S3).
Fig. S3 (in combination with Fig. S2) highlights that also approximated kernel values can be used to study architectures with no known closed-form solution for Eq. 4, as the posterior seems to be only marginally effected by the MC estimation.
Fig. S4 visualizes that NNGP kernels for common architectural choices do not encode for Euclidean distances. Kernels that monotonically decrease with increasing distance are, however, important to apply the KDE interpretation of SM B.
Finally, in Fig. S5 we investigate a more challenging dataset with two concentric rings. Note, that the center region as well as the region between the two rings can be considered OOD (not included in the support of p(x)). A good uncertainty-based OOD detector should therefore depict high uncertainties in those regions, which is not the case for ReLU networks.
102 103 104 105 No. of MC samples
0.00
0.05
0.10
0.15
re la
tiv e
M C
er ro
r
2-layer ReLU 8-layer ReLU 2-layer erf 8-layer erf
Figure S3: Monte Carlo error when estimating NNGP kernel values. Eq. 4 requires estimation whenever no analytic kernel expression is available (for instance, when using a hyberbolic tangent nonlinearity as in Fig. S2). Here, we visualize the error caused by this approximation for ReLU and error function (erf) networks when computing kernel values k(x∗,x∗). Eq. 4 requires a recursive estimation of expected values, where we estimate each of them using N samples (N ∈ [102, 103, 104, 105]). Note, that even small errors can cause eigenvalues of the kernel matrix to become negative. However, with our chosen likelihood variance of σ = 0.02 we experience no numerical instabilities during inference, and obtain consistent results using either analytic or estimated (N = 105) kernel matrices.
0 1 2 3 ||xp xq||2
0
1
2
3
k( x p
,x q)
RBF ReLU 1-layer ReLU 2-layer Tanh 1-layer Tanh 2-layer
Figure S4: NNGP kernel values do not generally reflect Euclidean distances. This figure shows kernel values k(xq,xp) plotted as a function of the Euclidean distance ‖xq − xp‖2 (using pairs of training points from the two Gaussian mixtures dataset). As outlined in Sec. 4 and SM B, the interpretation of an RBF kernel as Gaussian kernel that can be used in a KDE of p(x) is important to justify implied OOD, at least for low-dimensional problems. Unfortunately, the kernels induced by common architectures do not seem to be distance-aware and are thus not useful for KDE.
(a) 2-layer ReLU (5) (b) 2-layer ReLU (100) (c) 2-layer ReLU (∞)
(d) RBF net (500) (e) σ(f∗) – RBF net (∞) (f) RBF kernel
Figure S5: Standard deviation σ(f∗) of the predictive posterior. We perform Bayesian inference on a dataset composed by two concentric rings (SM D) comparing posterior uncertainties of GPs with NNGP kernels and an RBF kernel with those obtained by finite-width neural networks. Only the function space prior induced by an RBF kernel or RBF network causes epistemic uncertainties that allow outlier detection in the center or in between the two circles. However, the RBF network’s uncertainties decrease with increasing input norm, which can be counteracted by further increasing σ2µ (Sec. 3).
C.1 2D CLASSIFICATION
In this section, we consider the same dataset as in Fig 2 in a classification setting instead of regression. In this setting, exact inference is intractable even when using GPs and approximations are needed. In particular, we study classification with the logistic likelihood function p(y | f(x;w)) = s(−yf(x;w)) where s is the sigmoid function. We use HMC to sample from the posterior distribution in the finite-width limit, using a standard normal prior where the variance is inversely scaled by the hidden-layer’s width. The results are reported in Fig S6 for ReLU, cosine and RBF networks.
As clearly depicted in the plots, our findings regarding the importance of the prior in function space can be extended also for the classification case. Indeed, the uncertainty captured by the ReLU architecture appears unsuitable for OOD detection given that the data distribution is not captured and the disagreement is low also in regions of the 2D plane that do not contain any training data points (also see Kristiadi et al., 2020; 2021). This pathology appears in both cases: when the disagreement is considered over the functions σ(f∗) or over the sigmoid outputs σ(s(f∗)). By contrast, also for this new choice of likelihood, the cosine and RBF architecture exhibit uncertainty that conveys similar useful properties for OOD detection as seen in the regression example in Fig 2. Despite this empirical correspondence, it is important to note that the direct correspondence with the KDE technique that justifies the usefulness of OOD properties of the RBF kernel in the case of GP regression (see Sec. B) does not directly apply for classification. Indeed in the latter, we do not have an analytical form of the posterior as in Eq. 2 due to the non-Gaussianity of the logistic likelihood. Therefore our observations can not be readily generalized outside the settings of our experiments and further theoretical analysis are needed to asses an analogous justification for non-Gaussian likelihoods.
C.2 SPLITMNIST REGRESSION
In this section, we consider SplitMNIST tasks (Zenke et al., 2017), which are binary decision tasks where the original MNIST dataset is split into five tasks containing two digits each: 0/1, 2/3, 4/5, 6/7, 8/9. To perform exact inference via Gaussian Processes, we consider each binary decision task as regression problem with labels -1/1. For computational reasons, the training set of each task is reduced to 1000 samples, but test sets remain at their original size. Results are reported in Table S1.
For the chosen length-scale parameters l, RBF kernels show best generalization. Though, they don’t excel at OOD detection compared to other kernel choices. In SM B, we discussed that epistemic uncertainties as induced by the RBF kernel can be viewed as an approximation to p(x) when the
(a) σ(f∗) – 2-layer ReLU (b) σ(f∗) – 1-layer Cosine (c) σ(f∗) – 1-layer RBF
(d) σ(s(f∗)) – 2-layer ReLU (e) σ(s(f∗)) – 1-layer Cosine (f) σ(s(f∗))– 1-layer RBF
(g) s(f∗) – 1-layer ReLU (h) s(f∗) – 1-layer Cosine (i) s(f∗) – 1-layer RBF
Figure S6: Standard deviation over the logits f∗ and mean and standard deviation over the sigmoid outputs s(f∗) for a 2D classification problem. We perform approximate Bayesian inference with HMC for finite-width networks on a mixture of two Gaussians dataset considering different priors in function space induced by different architectural choices. The problem is now treated as classification task using HMC to sample from the posterior. We show the standard deviation σ(f∗) of logits f∗ in (a, b, c), the standard deviation σ(s(f∗)) of the predicted probability for the input being in the positive class in (d, e, f), and the corresponding mean s(f∗) (g, h, i). The ReLU network has 5 hidden units, the cosine has 100 hidden units, and the RBF network has 500 hidden units.
Table S1: GP regression on SplitMNIST tasks. We perform GP regression on a single SplitMNIST task using the kernels reported in the first column (entries show mean and standard deviation when using different SplitMNIST tasks for training). AUROC values are computed when considering the test data from all remaining SplitMNIST tasks as OOD data, or by taking the test data of FashionMNIST as OOD.
Acc. Train Acc. Test AUROC SplitMNIST AUROC FashionMNIST
RBF (l = 1) 100.0±.000 99.35±.577 .849±.077 .893±.062 RBF (l = 5) 100.0±.000 99.45±.552 .892±.057 .979±.010 RBF (l = 10) 100.0±.000 99.42±.505 .884±.060 .990±.005 ReLU 1-layer 99.74±.313 98.86±1.01 .884±.060 .995±.002 ERF 1-layer 99.68±.356 98.76±1.11 .884±.060 .996±.002 Cosine 1-layer 99.68±.356 98.88±.985 .880±.062 .993±.003
dataset size goes to infinity D → ∞, and the length-scale goes to zero l → 0. As expected, such approximation to p(x) may deteriorate on high-dimensional image data, presumably as the Euclidean distance does not capture the geometry of the image manifold.
(a) IND – f̄∗ (b) IND – σ(f∗) (c) IND
(d) OOD – f̄∗ (e) OOD – σ(f∗) (f) OOD
Figure S7: 2D visualizations of the SplitMNIST posterior for a GP with RBF kernel (l = 5). The plotted 2D linear subspaces are determined by 3 images (colored dots, denoted a, b and c). In the upper row, the yellow dot represents the in-distribution (IND) test sample with the highest uncertainty (see image c in subplot (c)), while the two green dots are the (in-distribution) training samples with smallest Euclidean distance to image c. In the lower row, the yellow dot represents the OOD test image with lowest uncertainty (see image c in subplot (f)). Again, the two green dots are the closest IND training samples. Image d is in both cases a randomly chosen point on the 2D subspace. Subplots (a) and (d) show the posterior mean σ(f∗), and subplots (b) and (e) the posterior’s standard deviation σ(f∗). Subplots (c) and (f) show the images corresponding to the 4 highlighted points on the 2D subspaces.
To gain a better intuition on why the RBF kernel does not excel in OOD detection for image data, we visualize the uncertainty behavior in Fig. S7. The figure shows 2D linear subspaces of the 784D image space. These subspaces are determined by three images (see caption for details). As can be seen in Fig. S7b and S7e, a too small length-scale might cause test points to be not included in the low-uncertainty regions. On the other hand, if the length-scale is set too high, OOD points may fall inside low-uncertainty regions. The situations depicted in the figure show that for the given training set there is no trade-off length-scale l that prevents such behavior (because some OOD points have less Euclidean distance to training points than some test points). Using an RBF kernel with a metric that encodes similarities in image space rather than using Euclidean distance might overcome these problems.
C.3 CONTINUAL LEARNING VIA UNCERTAINTY-BASED REPLAY
In Sec. 7 we mention that the desideratum of having low uncertainty only on in-distribution inputs opens the possibility of considering uncertainty as an energy function from which we can sample. Thus, the uncertainty landscape can be used to construct a generative model.
In this section, we use continual learning (Parisi et al., 2019) to demonstrate this conceptual idea. In continual learning, a sequence of tasks D(1), . . . ,D(T ) is learned sequentially such that each task has to be learned without access to data from past or future tasks. A common approach to continual learning is via replay (Lin, 1992). In this case, the current task t is learned with the available training data D(t) as well as datasets D̃(1), . . . , D̃(t−1) which are supposed to represent the data distributions of the previous tasks. In the simplest case, D̃(s) is constructed by storing a subset of D(s) (Lin, 1992). Other approaches learn a separate generative model that can be used replay (fake) data D̃(s) while a new task is learned (e.g., Shin et al., 2017; von Oswald et al., 2020)
We consider this (pseudo-)replay scenario without the need of maintaining a separate generative model. In Fig. S8, we consider a regression problem split into two tasks. Note, that learning a series
(a) Data from all tasks (b) Data from task 1 (c) Data from task 2 + replay
Figure S8: Continual learning with uncertainty-based replay. As mentioned in Sec. 7, if uncertainty should only be low on in-distribution data, a generative model can be obtained by sampling from regions of low uncertainty. Here, we use this idea to realize a replay-based continual learning algorithm (Lin, 1992) without the need of storing data or maintaining a separate generative model. In this case, we split a polynomial regression dataset into two tasks (denoted by the black vertical bar). The posteriors in this figure are obtained via GP regression using an NNGP kernel corresponding to a 1-layer Cosine network. (a) All data is seen at once (no continual learning). (b) Only data of task 1 is seen to obtain the posterior. The orange vertical lines denote input locations sampled from low uncertainty regions via rejection sampling. Task 1’s posterior is then used to label those input locations (orange dots). (c) The replay samples generated with the model of task 1 (orange dots) are used together with the training data of task 2 to obtain a combined posterior over all tasks.
of 1D regression problems is challenging for most continual learning algorithms, especially those relying on the recursive Bayesian update: p(w | D(1),D(2)) ∝ p(D(2) | w)p(w | D(1)) (Henning et al., 2021). By contrast, (pseudo-)replay methods can solve this task with ease as the structure of the data in such low-dimensional problem is easy to capture by a generative model.
In our experiment, two tasks are created by splitting the dataset of a 1D function into two parts as illustrated in Fig. S8a. The left half is considered the first task, and the right half the second task, respectively. The blue posteriors shown in all three subpanels are obtained with a GP using the NNGP kernel of a 1-layer Cosine network, which should exhibit OOD properties similar to an RBF kernel (cf. Sec. 4). In Fig. S8a, the posterior is obtained using the combined datasets from both tasks (no continual learning). In Fig. S8b, the posterior is obtained using only the training data of the first task. After this (first task’s) posterior is obtained, we can use it to perform pseudo-replay as outlined below to generate D̃(1) which can be mixed with the data of the second task D(2) to produce the posterior in Fig. S8c. The posterior in Fig. S8c looks qualitatively similar to the one in Fig. S8a even though it has been obtained without direct access to the first task’s training data D(1).
To generate D̃(1) while learning the second task, we need access to the posterior of the first task. Note, the model from the previous task is often kept in memory by continual learning algorithms, for instance, to use it for regularization purposes (Kirkpatrick et al., 2017) or to replay data with previously trained generative models (Shin et al., 2017).7 We generate in-distribution input locations (denoted by orange vertical bars in Fig. S8b) via rejection sampling as in Fig. 7. For that, we define epistemic uncertainty as an energy function E(x∗) ≡ σ ( f∗(x∗) ) and construct a Boltzmann
distribution p̃(x∗) ∝ exp ( − E(x∗)/T ) , where we set the temperature T = 1.8 As uncertainty of the posterior of Fig. S8b is only low for in-distribution inputs, we can assume obtained input locations are similar to those represented in D(1). To construct a dataset D̃(1), we use the posterior of Fig. S8b to sample predictions y∗ (orange dots in Fig. S8b and Fig. S8c).
7As we use a non-parametric model (a GP) which is represented by the training dataD(1), keeping the model in memory requires to store D(1) too, which is a violation of continual learning desiderata. However, this is just a conceptual example. If the same experiment would be performed via approximate inference in a parametric model (e.g., HMC on a corresponding finite-width network), then the (approximate) posterior of the first task can be stored without storing D(1). In the case exact inference can be performed, the Bayesian recursive update yields a sufficient continual learning algorithm.
8Note, if E(x∗) ∝ − log p(x∗), then this process yields exact samples from the input distribution. For instance, considering the limiting case of SM B where σ ( f∗(x∗) )2 ≈ 1− p(x∗), a reasonable choice of energy could be E(x∗) ≡ − log [ 1− σ ( f∗(x∗) )2] ).
In summary, this section presents an example application that can arise if the desideratum of high uncertainty on all OOD inputs (i.e., robust OOD detection) is fulfilled by a Bayesian model. Whether BNNs can be constructed to fulfill this desideratum on real-world data is, however, unclear at this point.

heading: D EXPERIMENTAL DETAILS
text: In this section, we report details on our implementation for the experiments conducted in this work. In all two-dimensional experiments: the likelihood is fixed to be Gaussian with variance 0.02. Unless noted otherwise, the prior in weight space is always a width-aware centered Gaussian with σ2w = 1.0 and σ2b = 1.0 except for the experiments involving RBF networks where we select σ 2 w = 200.0. The RBF kernel bandwidth was fixed to 1.0 in all corresponding experiments.
1D regression. We construct the 1D regression example (e.g., Fig 1a) by defining p(x) uniformly within the ranges [1.0, 1.3], [3.5, 3.8] and [5.2, 5.5], and p(y | x) as f(x) + with f(x) = 2 sin(x) + sin( √ 2x) + sin( √ 3x) and ∼ N (0, 0.22). The training set has size 20.
Periodic 1D regression (only Fig. 6). We construct this 1D regression example by defining p(x) uniformly within the range [0.0, 12.5], and p(y | x) as f(x)+ with f(x) = sin(x) and ∼ N (0, 32). The training set has size 30.
Gaussian Mixture. We created a two-dimensional mixture of two Gaussians with means µ1 = (−2,−2), µ2 = (2, 2) and covariance Σ = 0.5 · I and sampled 20 training data points.
Two rings. We uniformly sampled 50 training data points from two rings centred in (0, 0) with inner and outer radii Ri,1 = 3, Ro,1 = 4, Ri,2 = 8, Ro,2 = 9, respectively.
HMC. We use 5 parallel chains for 5000 steps with each constituting 50 leapfrog steps with stepsize 0.001 for width 5 and 0.0001 for width 100. We considered a burn-in phase of 1000 steps and collected a total of 1000 samples.

Review summaries:
This paper carries out an analysis that motivates that the use of the Bayesian predictive distribution and its uncertainty is not appropriate for detecting out of distribution data. The paper focuses on the case of Bayesian Neural networks and its infinite-wide generalization. Namely, a Gaussian process. The paper has no experiments but gives illustrative insights about why the Bayesian posterior is not suitable for detecting out of distribution data. They show that exact inference with GPs (infinitely wide networks) does not lead to desirable OOD detection. They discuss desirable kernel features for OOD. They emphasize that the choice of weight-space prior has a strong effect on OOD performance. They argue that there is a trade-off between good generalization and having high uncertainty on OOD. 

The manuscript challenges the widely believed assumption that Bayesian neural networks (BNNs) are well suit for out-of-distribution (OOD) detection, by showing empirical results obtained using infinite-width (allowing the exact inference, because a network can be equivalently represented by a GP with the included kernel) and finite-width networks (requiring the approximate inference). The manuscript provides several observations in line with this purpose (to disclose potential problems when BNNs are used for ODD detection).  For example, the exact inference in infinite-width networks does not necessarily lead to desirable OOD behavior, e.g., the posterior function standard deviation obtained from an infinite-width 2-layer ReLU does not reflect the underlying data generating process and this uncertainty is not suit for OOD detection. In addition, this observation is consistent with the corresponding finite-width networks. 

In this work, the authors challenge the assumption that underlies many recent works that Bayesian neural networks should be well-suited to out-of-distribution detection. In order to do this, the authors focus on a function-space view by examining the properties of infinite-width BNNs. They use this analysis to argue that BNNs may not necessarily be well-suited to OOD detection. They further argue that there is a tradeoff between OOD generalization and uncertainty, and finally propose an alternative method of validating OOD properties of models.