Do Bayesian Neural Networks Actually Behave Like Bayesian Models?
Authors: Gábor Pituk
Vik Shirvaikar
Tom Rainforth
Venue: ICML 2025 poster (2025)
Keywords: Bayesian Neural Networks
BNNs
Bayesian Deep Learning
Primary area: probabilistic_methods->bayesian_models_and_methods

Abstract: We empirically investigate how well popular approximate inference algorithms for Bayesian Neural Networks (BNNs) respect the theoretical properties of Bayesian belief updating. We find strong evidence on synthetic regression and real-world image classification tasks that common BNN algorithms such as variational inference, Laplace approximation, SWAG, and SGLD fail to update in a consistent manner, forget about old data under sequential updates, and violate the predictive coherence properties that would be expected of Bayesian methods. These observed behaviors imply that care should be taken when treating BNNs as true Bayesian models, particularly when using them beyond static prediction settings, such as for active, continual, or transfer learning.

Do Bayesian Neural Networks Actually Behave Like Bayesian Models?
Gábor Pituk
Vik Shirvaikar'
Tom Rainforth
Abstract
settings (Izmailov et al., 2021b). However, due to the ap- proximate nature of this inference, is no longer a given that they will still conform to Bayesian idcals.
We empirically investigate how well popular ap- proximate inference algorithms for Bayesian Neu- ral Networks (BNNs) respect the theoretical prop- erties of Bayesian belief updating. We find strong evidence on synthetic regression and real-world image classification tasks that common BNN al- gorithms such as variational inference, Laplace approximation, SWAG, and SGLD fail to update in a consistent manner, forget about old data un- der sequential updates, and violate the predictive coherence properties that would be expected of Bayesian methods. These observed behaviors im- ply that care should be taken when treating BNNs as true Bayesian models, particularly when using them beyond static prediction settings, such as for active, continual, or transfer learning.
Why does it matter if BNNs do not behave like Bayesian models? Firstly, several recent efforts seek to improve BNNs by refining priors (Fortuin, 2022) or proposing methodological or algorithmic improvements for enhancing the fidelity of approximate inference (Papamarkou et al., 2024). However, if BNNs fail to respect Bayesian principles such as consistent prior updating and meaningful posterior uncertainty, then we cannot trust that objectively better pri- ors or more faithful inference will translate to performance gains. This is exacerbated by the fact that it is challeng- ing to even precisely evaluate the accuracy of inference in high-dimensional BNNs, so most research has subsequently focused only on improving final predictive performance and calibration metries of the model- averraed solution (Guo et al., 2017: Wilson & Izmailov, 2020).
Secondly, the adherence of BNNs to Bayesian principles is of direct importance in many contexts. For example, pop- ular active learning acquisition functions such as BALD (Houlsby et al., 2011) and EPIG (Bickford Smith et al., 2023) assume the model will be updated in true Bayesian manner, and so will become misaligned if this fails hold, Meanwhile, in settings such continual and transfer learn- ing, it is essential that sequential updates do not forget about old data (Shwartz-Zii Ziv et al., 2022: Wang et al., 2024). More generally, the conformity of BNNs to Bayesian ideals is important whenever we go beyond static prediction settings, such as when we wish to capture reducible (epistemic) uncer- tainty, make joint predictions, reason about possible future data, or sequentially update our model.
1. Introduction
Bayesian neural networks (BNNs) extend the principles of Bayesian inference (Robins Wasserman, 2000; Jaynes,
2003) to neural networks by placing prior distributions on the model parameters and updating them based observed data (MacKay, 1992b; Hinton & Van Camp, 1993; Neal, 2012). This approach promises benefits such as principled uncertainty quantification (Neal et al., 2011; Kendall & Gal, 2017), robustness to distribution shift (Daxberger et al., 2021b), and coherent belief propagation (Gal et al., 2017), often eited as important for tasks such as out-of-distribution detection, sequential decision-making, and active learning.
Unfortunately, the practical implementation of Bayesian inference in neural networks is fundamentally intractable, necessitating the use of approximate inference methods. De- spite typically being crude approximations of the highly complex BNN posterior in parameter space, these methods generally offer better predictive performance and uncer- tainty calibration than deterministic networks in multiple
In this paper we critically evaluate the behavioral proper- ties of popular BNN algorithms. Whereas previous studies have focused on static predictive performance and inference convergence diagnostics (Izmailov et al., 2021b; Sommer et al., 2024), our study aims to more systematically inves- tigate the adherence of BNNs to the expected behavior of true Bayesian models. We find on synthetic regression tasks and the CIFAR and IMDB image and text classifica- tion settings of Izmailov et al. (2021b) that BNNs fail to preserve key features of Bayesian inference. In particular, we show that, for the tasks considered, (1) approximate
Department of Statistics, University of Oxford, Oxford, UK. Correspondence to: Gâbor Pituk <pitukgabor caclooor@@mmml.oooo.m... comi.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Do BNNs Actually Behave Like Bayesian Models?
posteriors lack the functional variability of true Bayesian posteriors (Section 4); (2) sequential inference using BNNs fails to propagate information coherently, leading to signif- icant degradation in accuracy (Section 5.1); and (3) BNN predictive updates are not well-calibrated, failing to retain a self-consistent amount of uncertainty, and thereby violate fundamental information-theoreti properties of Bayesian methods (Sections 5.2 and 6).
Izmailov et al., 2021b; Papamarkou et al., 2024). The maxi- mum a posteriori (MAP) solution has been observed to be overconfident (Guo et al., 2017), which the BMA improves on (Kendall & Gal, 2017; Izmailov et al., 2021b). This is achieved by explicitly accounting for multiple hypotheses, where the ensemble of multiple viable models produces principled uncertainty estimates and prevents overfitting. In practice, BNNs have seen measurable empirical success in predictive tasks, outperforming the MAP estimate in multi- ple settings (Izmailov et al., 2021b).
Our results, therefore, show that, though BNNs often give powerful static predictive performance, we should be cau- tious in our expectation that they practically adhere to Bayesian principles. We further provide hope for future improvement, by suggesting that using empirical martin- gale posterior predictives (Fong et al., 2023) may improve performance over conventional Bayesian model averages.
3. Related Work
BNNs have faced critique. For example, Bayes can be suboptimal in misspecified models (Wang & Blei (2019), Section K.2). BNNs are also often criticized for using un- informative priors that fail to reflect relevant beliefs (Tran et al., 2022; Fortuin, 2022). Although prior engineering is an active research area, objective priors that place mass on non- meaningful predictive representations are widely used (Iz- mailov et al., 202 10:1b; Papamarkou et al., 2024). In addition, the fidelity of approximate inference in BNNs is routinely questioned, with theoretical pathologies noted by Foong et al. (2020) and Coker et al. (2022). Approximate sampling cannot faithfully capture highly complex BNN posteriors (Wiese al., 2023), and many schemes make inappropriate assumptions resulting in crude posterior approximation. These issues inevitably lead to downstream practical issues, e.g. BNNs have been shown to underperform under covari- ate shift (Izmailov et al., 2021a; Sharma et al., 2023).
2. Background
Our focus will be on the supervised learning setting as this is where BNNs are most commonly used. Namely, let D = {z,, B4 ) input and y the labeling; our goal is learn a conditional predictive distribution Y|X x.
1 be a training set where x denotes the
BNNs. To model the data, we use deep neural network fe(x) with model parameters 6. Instead of training a sin- gle empirical risk minimizer 0SGD. BNNs clicit prior (0) over model parameters, condition on 'D observed through an i.i.d. observation model Y; I3 p(u fo(x;)), and approximate the posterior distribution (0) p(v T; 0) (Savage, 1972). The posterior pre- dictive p(y" 1z", = p(y" x=", (d( (D) can then be used predict on new data by marginalizing over the pos- terior, producing the Bayesian model average (BMA, Sec- tion A), which is the optimal predictive distribution from a -thoorecc perspective if the posterior truly repre- sents our beliefs (Fragoso et al., 2018).
x
Knoblauch et al. (2022) highlight that these key assumptions- model specification, meaningful pri- ors, and computationally feasible undates-underpin the theoretical guarantees of Bayesian inference, such as consis- tency and asymptotic optimality (Bernardo & Smith, 2009), but are fragile in practice, particularly for BNNs. As a result, several generalized Bayes approaches have emerged in the wider literature to retain the useful properties of Bayesian inference without strict adherence to full Bayesian updat- ing. Two notable examples are Gibbs posteriors (Bissiri et al., 2016), which replace the log- likelliodoood with a general loss function, and PAC-Bayes (McAllester, 1999), which balances empirical fit and model complexity.
Approximate inference. The intractability of Bayesian inference in deep learning means that practitioners resort to approximate inference algorithms. Popular practical ap- proaches include localized approximations around sin- gle mode such as variational inference (VT) (Graves, 2011; Blei et al., 2017), Laplace approximations (MacKay. 1992a; Daxberger et al., 2021 2021a), and stochastic weight averaging (SWAG) (Maddox et al., 2019), as well as more expressive but computationally more expensive sampling algorithms (Welling & Teh, 2011; Wenzel al., 2020). Full-batch Hamiltonian Monte Carlo (HMC) (Neal et al., 2011) is typically considered state-of-the-art; however, it is computa- tionally infeasible to run even for smaller networks.
In BNNs, a generalized Bayesian approach of particular relevance is cold posteriors or power posteriors, where the posterior is artificially sharpened to improve BMA perfor- mance across different settings (Grünwald, 2012). Wenzel et al. (2020) consider the family of distributions
Ty (B: x (7) II p(u. l), 1/T
Benefits of BNNs. The Bayesian deep learning litera- ture highlights various attractive theoretical properties of Bayesian inference (Blundell et al., 2015; Wilson, 2020;
for "temperatures" T < 1, and demonstrate that approxi- mate tempered posteriors outperform the Bayes posterior
2

---

Do BNNs Actually Behave Like Bayesian Models?
HMC
Vi
Laplace
SWAG
Posterio
-4
33 -2 1
33
-2
2
-2 -1 0
X
Figure 1: Synthetic regression posterior funnctional variability. Functional samples are drawn from approximate posteriors on the synthetic regression task. The posterior means are similar for all four algorithms. The HMC approximate posterior puts weight on more complex and qualitatively more varied functions compared to the other approximate posteriors. especially in the region x 0, but does not seem to account for the potential of non-monotonic behavior in the region < -1. Despite having somewhat similar point uncertainties, the VI approximation represents less complex and less varied range of functions. Laplace and SWAG fail to capture meaningful uncertainty or functional complexity, especially for > 0.
at T = This suggests approximate samplers may strug- gle to explore enough qualitatively different modes of the posterior, hinting at problems with the model setup. Noci al. (2021) and Izmailov al. (2021b) point to data aug- mentation, curation, and prior issues, but single cause has been identified (Nabarro et al., 2022). Others argue that posterior tempering is not necessarily against Bayesian principles (Kapoor et al., 2022; Zhang et al., 2024).
with observed data. They find through repeated re-training on small-scale tasks that non- Ban-ssa algorithms may bet- ter satisfy this criterion than Bayesian approximations, sug- gesting that Bayesian deep learning uncertainty can be quan- tified without explicit priors or posterior updates.
We now shift the focus away from static predictive perfor- mance and investigate how practical BNNs behave with respect to the theoretical properties of Bayesian models.
Elsewhere, Sharma et al. (2023) showed that fully stochastic BNNs do not typically outperform their partially stochustic counterparts. They argue for holistic view of BNNs as
4. What Do BNN Functional Posterior Predictives Look Like?
probabilistic algorithms with good predictive performance rather than true Bayesian models. Izmailov et al. (2021b) systematically evaluate the predictive performance of ap- proximate inference against hugely expensive HMC base- line. While HMC fails to mix in weight space, they argue it mixes well predictive space, and practical BNN approxi- mate posteriors make similar predictions HMC. However, Sharma al. (2023) note that their HMC baseline still ex- hibits correlations in within-chain predictions, undermining claims that its samples represent the true posterior.
To start our investigation into the adherence of BNNs to Bayesian ideals, we consider BNNs applied to synthetic re- gression tasks, and measure empirically to what extent these methods behave like true Bayesian models. We deliberately focus here regression instead of classification as it more clearly highlights nuances of the approximate posterior pre- dictive, noting that we can always consider regressing to class probabilities as a special case.
We use two fully connected BNN architectures with hid- den layers of size 32, 32, 16, and 128, 256, 128, 64, re- spectively. To capture heteroskedastic noise, we choose the observation model Y xx, A (=)(()),
In terms of applications, Bickford Smith et al. (2024a:b) raise issues with BNNs in active learning, while Kirsch et al. (2022) argue for the applicability of Bayesian deep learning joint predictives in active learning and sampling, but find that approximate BNNs fall short in downstream predictive performance. Mcanwhile, in continual learning, BNNs have been shown to exhibit catastrophic forgetting. where learning a new task worsens performance on previous ones (Chen et al., 2021; Kessler et al., 2023; Wang et al., 2024).
where fo(x) (zx))) but Section I shows that our find- ings also generalize to homoskedastic observation mod- els. For training, we consider the mean-field variational inference (MFVI) (Blei et al.. 2017), (diagonal) Laplace (Daxberger al., 2021a), and SWAG algorithms (Maddox et al., 2019), well as the more computationally inten- sive Hamiltonian Monte Carlo (HMC) sampler (Neal et al., 2011), generally considered state of the art. Section B gives background on the algorithms, Section G details the exper- imental setup, Section H shows the desired ground-truth behavior of analytic Bayesian linear regression inference. and Section assesses the robustness of our findings across network size, input dimension through feature expansion, and hyperr-arametterizatoon.
Alternatives BNNs often aim capture uncertainty di- rectly in predictive space. For example, epistemic neu- ral networks directly learn joint distribution over predic- tions (Osband et al., 2023), while evidential deep learning learns the parameters of higher-order distribution (Sensoy et al., 2018). Mlodozeniec et al. (2024) assess the "implicit Bayesianness" of predictive updates via exchangeability
3

---

Do BNNs Actually Behave Like Bayesian Models?
ever, the function draws are more similar to each other than the HMC approximate posterior. Note that VI's behavior is sensitive to range of parameters, including the choice of 3 in Equation (16): smaller 3, smaller initial variances, and shorter training time all trade off a closer fit against regularization towards the prior.
Prior HMC Vi Laplace
BALD (mats)
SWAG
The Laplace and SWAG function- space posteriors mean- while collapse to very specific MAP-like behaviors. Their function draws appear to be mostly simple parallel transfor- mations of their posterior mean; they fail to even produce increasing point uncertainty in the region I > 0. Both algo- rithms make local Gaussian approximations around a MAP estimate, averaging models in this region of the parameter space. In this instance, this local averaging does not seem to capture qualitatively different models to describe the data,
Figure 2: BALD scores for approximate posteriors shown in Figure 1. The HMC approximate posterior uncertainty appears sensible in this instance, increasing in regions with- out observations. Others attribute less importance to sam- ples far from existing observations, and more importance to observing further data around x = -1
4.2. How Much Uncertainty Reduction Should We Expect from Observing Further Data Points?
4.1. What is the Posterior Function Space Variability of Different Approximate BNN Posteriors?
To gain further insights into Bayesian model variabil- ity and connect epistemic uncertainty with point uncer- tainty, we now consider the information-theorrtic BALD score (Houlsby et al., 2011), often used in Bayesian active learning as an acquisition function. At an input x", it is defined as the mutual information between @ and Y* |2", or equivalently, the expected reduction in weight-space uncer- tainty when observing a data point at x².
We begin by questioning whether the different approxi- mate posteriors can capture epistemic uncertainty by putting weight on all the models likely to have generated the ob- served data. BNNs can be described as distributions over the network parameters 0, which we refer to as the weight-space or parameter-space characterization. Equally, they can be considered as distributions over predictive functions, æ + (p(x)), a (z))) in the case of our regression setup, by pushing forward the weight-space distribution through the network fo(r) at any x, the so-called function-space or predictive- space characterization (Izmailov et al., 2021b). Going beyond the posterior mean (BMA) Ee-=(0 fx²") and point uncertainties p(y" , D) and p(u(foa"") |=*,0). ,0,,
Its values for different approximate BNN posteriors regression task are shown in Figure 2. The HMC approxi- mate posterior attributes more importance to observing data far from existing observations, whereas other approximate posteriors would be more informed by further observations around x = -1, which may be undesirable in learning systems. Laplace and SWAG appear so confident in the generalizing behavior for x >> 0 that they expect negligible information gain from observing data in this region.
we are interested in the functional properties of the approxi- mate BNN posteriors in predictive space.
BNN use cases relying on the full posterior (some of which will be detailed in Section 5.1) require that the posterior approximations behave reasonably in function space to rep- resent a range of likely models to describe the data. In particular, they expect function draws from the posterior to preserve the characteristics of the prior predictive in re- gions without observations. We investigate if this holds for practical BNNs in our regression setting.
Despite the differing posterior functional properties and generalization behavior and shortcomings uncovered, the BMAs of these approximate posteriors are all similar and in line with our expectations. This observation motivates our investigation into function-space behavior of practical BNNs beyond model-averaged static performance.
Figure shows function draws from different approximate posteriors trained on a synthetic regression task. Based on the samples, HMC retains more of the prior function complexity in the high-uncertainty region x > 0 compared to the other schemes which overly favor simpler functions. However, even HMC does not account for qualitatively distinct function behaviors in the region < -1.
5. Is Data Captured in Intermediate Posteriors Weighted Consistently with New Data?
In this section we demonstrate deviations of practical BNNs from Bayesian behavior in settings requiring sequential deci- sion making in between observing data, utilizing epistemic uncertainty directly and using intermediary posteriors in subsequent steps of inference. Our tests will be particu- larly relevant to the setting of active, continual, and transfer learning as discussed below.
Variational inference (VI), when appropriately tuned (see Section B.2), seems to produce somewhat sensible point un- certainty, increasing in regions without observations. How-
4

---

Do BNNs Actually Behave Like Bayesian Models?
Data points 0-20
Data points 20-40
Data points 40-60
Fosterior mean
90% HPDI on mean
HPDI on prediction
Posterior function draws Provioutly points
Data points 60-80
Data points 80-100
HMC data points 80-100
Data points 0-100 tegether
22 1 6
-2
Figure 3: Sequential updating experiment with VI on larger network. Each step's prior is the posterior approximation from the previous step. Point uncertainty is shown through high probability density intervals (HPDI) on the predictive mean and the observation. Data points currently trained on are shown in black, previous ones in red. The last step of inference is repeated with HMC, as well as single-step VI approximation is shown for reference. Catastrophic fitting the new data leads to worse on earlier data, and observed information gradually lost.
occurs:
Active learning and experimental design (Atlas al., 1989; Settles, 2009; Rainforth al., 2024) sequential learning paradigms, where popular acquisition functions such as BALD (Houlsby et al., 2011) or EPIG (Bickford Smith et al., 2023) are based on Bayesian model uncertainty. We might model the data with a BNN (Houlsby et al., 201 I; Gal et al., 2017) and sequentially update beliefs with the fresh observations. This, however, requires that BNNs respect the sequential properties of Bayesian updating.
To set up sequential inference, suppose have datasets (DO) N fi=1 which the learner observes sequentially and needs to make decisions in between observing them. Sup- pose further that we have an imitial belief (0), and a scheme T (() + (0) to update beliefs as a function of observed data y. We use the incremental beliefs ((TD)))) ...)))))).papp (0) to make predictions on held-out dataset. We then test how these beliefs generated from sequential inference compare to conditioning all the data in one go, i.e. I ((D()) (0).
Continual learning focuses on an agent sequentially learn- ing a number of tasks; Bayesian formulations using online Laplace approximations, VI, or function-space VI have been proposed (Wang et al., 2024, see Section 1), where it is crucial for the learner not to forget about old observations.
Synthetic regression. We partition our synthetic regres- sion dataset into N = 5 equal groups based on the x value, and run sequential approximate inference. According to Bayesian coherence, each intermediate posterior should be equal to the single-step posterior trained on all prior data points. We find this not to hold in practice: Figure grad- ually demonstrates catastrophic forgetting. The final VI approximate posterior differs from the single-step approx- imate posterior, not fully respecting the first half of the observations. HMC on the last step produces a worse fit than VI, indicating that the intermediary VI posterior after 80 data points may be overconfident. The practically nec- essary of 8 0.05 overweighting observations 80-100 in the final step of in- ference. Section J shows a similar even worse forgetting behavior for Laplace and different network variants.
In transfer learning, learner captures the knowledge of downstream task. There have been Bayesian models of transfer learning (Shwartz-Ziv et al., 2022; Suder et al., 2023) that propose using an intermediary SWAG posterior learned on the pre- training task. Here, there is an implicit assumption that this approximate posterior is able functionally capture pre- training knowledge to inform training the downstream task. We investigate empirically if such applications of BNNs are justified from a Bayesian modeling perspective.
pretraining task to improve performance
masks this inconsistency by
5.1. Are Sequential Belief Updates Coherent?
Image and text classification. We now scale this exper- iment up to real-world model and image and text classifi- cation benchmarks. Izmailov al. (2021b) demonstrated that Bayesian alternatives typically outperform deterministic
An important aspect of Bayes' rule is the following invari- ance with respect to the order of observations.
Proposition 5.1 (Sequential Coherence of Bayesian Infer- ence, Bernardo & Smith (2009)). D and D' are two sets of observations, then the "iterated" posterior π ((0) D') the same as the full posterior и D DUD''.
Thii experiment requires closed-form approximate posterior, which holds for VI. Laplace, and SWAG. Although practitioners may retrain on all prior observations cach time, we analyze if such algorithms are justified from Bayesian angle in the first place.
5

---

Do BNNs Actually Behave Like Bayesian Models?
VI
CIFAR-10 SGD
IMDB VI
IMDB SGD
intrinsic algorithmic difference instead of estimation error accumulation, in this experiment we take N 2 random splits and do only single step of retraining.
60%
80%
Log-boPPood Aetur00y
80%
87%
80%-
70%
78%
79%
60%
60%
Figure 4 compares the iterated posterior against the single- step posterior in terms of the predictive performance metrics of the BMA. Contrary to Bayesian expectations, iterated VI performs significantly and consistently worse than single- step VI, even though (Tpu: D(a) (0) is still an improvement over TDD: (0). To evaluate this improvement,
-1.0
-1.0
-0.4
-1.25
1.5
1.66
-1.00
-0.5
35%
10%
20%
23%
6%
a "se- quential SGD" baseline where an SGD solution is trained on D using the first-split SGD solution arg max, x( (D(1) as an initializer without any additional training guidance (such that the only influence of D(I) is from the initializa- tion). This deliberately non- on-Baaesiian bias to share beliefs leads similar improvement from the intermediate MAP to the "sequential" biased MAP as the improvement from T (0) to (= D (0). Based on this experiment, we do not find the use of VI to be justified in sequential infer-
300
500
300
500
Epooh
Epoch
Epoch
Epoch
MFVI
Sequentra MFVI
SGD
"Segueriaa SGD
Figure 4: Sequential MFVI on CIFAR-10 and IMDB. We show test accuracy, log-likelihood, and expected calibration error (ECE) for the single step and iterated inference (mean and st. dev. across 6 seeds). Under Bayes the two should make the same predictions, but iterated MFVI significantly underperforms single-step MFVI across all metrics. VI improves over SGD, and fixes the growing overeonfidence of SGD during training. However, in iterated training, the principled Bayesian belief sharing does not improve MFVI more than the heuristic belief sharing improves SGD.
ence contexts by its Bayesian motivations alone.
5.2. How Strong are the Beliefs Captured in an Intermediary Posterior?
We now turn back to synthetic regression and test how strong the knowledge captured in the intermediary posterior is com- pared to subsequent observations. To precisely quantify the amount of information encoded in an intermediary posterior and fresh observations, train an approximate posterior on 50 data points generated from Y; a; (0.5,, 0.05²). The learner then gradually observes data from the shifted dis- tribution Y: I; ~ _N(-0.5, 0.05²) with this approximate posterior used as the prior. Training on n new data points should give a predictive mcan of 50-n 0.5 shrunk by the 50+n
networks in multiple classification and regression settings. We replicate their MFVI experiments on ResNet-20-FRM C0, CIFAR-100 (Section F. and CNN-LSTM IMDB, taking their implementation and hyperparameters.
We randomly split the training sets into two splits D(1) and D)2), We train an intermediary MFVI approximate posterior Te (0) on the first split, and pass this as prior to infer a second MFVI approximate posterior (TDm) (y (0) on the second split. As the number of sequential inference steps grows, we expect estimation error to increase and final approximation quality to deteriorate. To focus on the
original prior towards zero. This means we should need to observe more than 50 new points to shift the posterior pre-
dictive mean negative if points are being weighted equally.
Intermediary Posterior via Laplace
HMC retrained on 1 point
HMC retroined on 2 points
HMC posterior on both datasets
Wypde
HMC retral ned on 5 paints
HMC retrained on 25 points
HMC retra need on 50 points
webk
Ryple
apple
-1
Posterfor mean
90% HPDI predictio
Ruperiorfunction
HPOI
Previousty data
Figure 5: Intermediary Laplace posterior experiment. An intermediary posterior is learned from the original dataset, then subsequent HMC inference is performed with this intermediate posterior as prior, observing data from a shifted distribution. We show the larger network with feature expansion, but the result generalizes; see Section J. After only 25 points, the posterior mean becomes negative: the new observations appear to be weighted around twice as much as the pretraining observations, despite the shrinkage of the intermediary Laplace approximate posterior towards the MAP estimate. Single-step HMC inference on the original and shifted datasets the same time is able to produce sensible posterior mean.
6

---

Do BNNs Actually Behave Like Bayesian Models?
HMC
Vi
the model itself updates when those predictions are fed back into the training as pseudo-data. This tests BNNs against the important property of Bayesian models that they should not hallucinate new information without external data or destroy information from previous data.
tog-lleelhodd
-400
-3200
-1000
Laplace
SWAG
In particular, given an observed dataset, any predictive model somehow manipulates the information contained in the data to produce a predictive distribution (Dawid, 1984). Suppose we then generate pseudo-data from our model, im- pute it into our training set, and perform another update. If the model updates are well-calibrated, then on average, across all possible imputations, the new predictive distri- bution should not change, as no new information has been added. If it does, it implies that information from the initial observed data has been lost or transformed through our up- dating scheme, resulting in miscalibration or "hallucination" in predictive space (Falck et al., 2024).
-1000
-2003
-4
Figure 6: Cross-sectional predictive log-likelihoods. We compute sogllikelihoods at x on the pretraining task observing data centered at y 0.5. The tail behavior of different posterior predictives exhibit significant differences. In particular, VI is significantly heavier-tailed than others despite over-weighting observations due to taking 3 < 1. Laplace and SWAG predictives resemble Gaussian tails, and are lighter tailed than the HMC predictive.
A diagnostic based on predictive consistency therefore tests whether a model update is retaining a consistent amount of uncertainty with respect to the data-generating process, or if neeudo-nfoommtionn being introduced through the model's predictions. This diagnostic evaluates posterior model un- certainty not captured by the BMA, thereby assessing the coherence through more fundamental predictive lens.
We use Laplace approximation approximate the inter- mediary posterior, which we then pass to HMC inference as a prior, observing points from the shifted distribution. Figure 5 shows that catastrophic forgetting is observed: re- cent observations are weighted more heavily compared to carlier observations (Wang et al., 2024). Either the Laplace approximation has failed to properly incorporate the pro- vided data, or HMC has failed to mix, potentially stuck in an ill-conditioned optimum, or some combination of the two.
How does this relate to Bayesian behavior? The martingale posterior framework of Fong et al. (2023) presents a predic- tive view of Bayesian inference, where uncertainty arises from missing data, not unknown parameters. Under this view, the Bayesian task is not update prior on parame- ters, but to assign predictive distribution to missing data conditional on observed data. A result from Doob (1949) shows that this predictive uncertainty is equivalent to para- metric uncertainty: incoherence in one implies incoherence in the other. While this is usually framed parametrically- an incorrect update leads to incorrect predictions we focus on the mirrored view: if predictions are miscalibrated, the parametric update must also be flawed.
For VI, we find even stronger forgetting behavior, under- pinned by the less confident VI intermediary posterior pre- dictive; see Section J.4. Neither of these approximate infer- ence schemes appears to faithfully encode pretraining data in its posteriors for use in downstream tasks, nor to capture functional knowledge from pretraining.
We further examine the tail behavior of the posterior predic- tive distributions in Figure 6, as this can shed light on how flexible the current model is to change with future observa- tions. We see that the Laplace predictive is lighter-tailed than HMC; it is more confident in the pretraining observa- tions. Given that we still observe forgetting when using these lighter-tailed Laplace approximations as priors, this suggests inaccuracies in the HMC inference.
We now provide brief technical exposition on the mar- tingale posterior framework, with further details in Sec- tion C. Let θ ~ T() be a parameter of interest with some prior density (0) before any data is observed. For ob- served data YI.N. the posterior mean of this parameter is 0N E [@ Yi.N]. Note here that we did not need to as- sume some explicit likelihood form p(31.x(6)), we simply need a mechanism to recover E (e) Y1:N) from data. For example, we might simply have a mechanism for training a stochastic neural network given observed data. The follow- ing result from Doob (1949) now shows that parametric and predictive uncertainty are equivalent if 0 is a martingale, that is if its conditional expectation each step is equal to its previous value, meaning that new data points only propagate uncertainty and do not introduce bias.
6. Evaluating the Predictive Coherence of Belief Updates with Martingales
In the previous section, examined whether coherence in predictive beliefs is maintained when sequentially updat- ing the parametric posterior using approximate Bayesian inference with increasing amounts of externally provided data, We now instead investigate whether the distribution of predictions made by BNNs itself self-consistent with how
7

---

Do BNNs Actually Behave Like Bayesian Models?
Theorem 6.1 (Martingale Convergence, Doob (1949)). If NN is martingale, with E. [0x Yi-N-1] 01- then
updating behavior in the predictive model even if our in- ference is poorly representing the true parameter posterior. However, in our tests below we find BNNs breaking this pro- ductive coherence, such that they do not properly maintain information in the data between iterations.
under certain measurability and identifiability conditions, NN + θ almost surely.
In other words, when this martingale property holds, rather than viewing belief updates solely as refining our knowledge about O, we can interpret them as the resolution of uncer- tainty about future observations. Each new data point YN refines our predictive uncertainty, and as we sequentially impute the unseen Yi the remaining uncertainty in é vanishes and our sequence of posterior means ON converges to draw from the prior.
We use the CIFAR-10 dataset for this experiment, and con- sider taking two random subsets of 4080 images each: a labeled split (z, y). and an unlabeled split x². We first use the labeled split to train an initial BNN posterior approxima- tion x (0) = (=x.v) (0). For the unlabeled split x", we use the intermediary x -ppredicttvve to sample labelings through @ ~ r'()) followed by Y |0 ~ p(- 6; x²) We then train a new posterior approximation =() = "(x.y)u(a=,.>> (f). This letel-retrain step repeated over N trials, allowing us to compute an empirical martingale posterior distribution
Under this approach, Bayesian inference is nothing more than a predictive task, where our goal is to generate different possible imputations of the unobserved data. Uncertainty is then indexed over these hypothetical completions of the dataset. Specifically. suppose we have initial belief (f)). belief updating scheme (0) > Tyy that incorporates observed data y into belief's π, and parameterized predic- tive model p(y 0). Let Y be a random sample drawn from the marginal predictive ply") = =f f'('' (d))))) Then we define the martingale posterior distribution (Fong et al., 2023) for this sample as the expected belief update across all possible predictive imputations:
N
1 N
MP (or (-) (0)
[(ax))(x> (0)) f=1
that averages over the N different predictive imputations of the unknown labels y². We then compare the BMA from the initial approximate posterior Ee [So (2test) ) and the BMA from the empirical martingale posterior E MP (+* (=) [fo [R(cs))) Propositions 5.1 and 6. 1 imply that under Bayesian inference and N - oc these two model averages should match.
f (y-(())((d)))
MPy-T((P) = E (TT- (0) =|
With the goal of analyzing function-space predictive co- herency, we apply this uncertainty diagnostic to MFVI and Stochastic Gradient Langevin Dynamics (SGLD) (Welling & Teh (2011), Section B.5). We treat these algorithms as end-to-end belief updating schemes, replicating the setup and their hyperparameters from Izmailov et al. (2021b). Figure 7 shows the result of our martingale posterior ex- periment on CIFAR-10, 10, comparing the performance of the initial posterior distribution x' to that of the martingale pos- terior. Specifically, we examine Bayesian model average (BMA) metrics of accuracy, log-likelihood, and expected calibration error as projected summaries of posterior predic- tive performance (see Section A).
can be estimated with the (consistent) empirical martin- gale posterior distribution:
(N) MPy- (()
N J3 (0)
where 23 ltd. p(y") averaging across N trials.
As exact Bayesian updates do satisfy the martingale prop- erty, we can now introduce another key property of Bayesian inference relating to its predictive belief updating:
Proposition 6.1 (Predictive Coherence of Bayesian Infer- ence, based on Fong et al. (2023)). When the belief updating scheme is exact Bayesian inference, the martingale posterior recovers the initial beliefs.
Under true Bayesian inference the martingale posterior should recover x', and thus their BMA predictions are ex- pected to match. However, Figure 7 shows that the martin- gale posterior metrics deviate significantly from those of T', with deviations exceeding bootstrapped confidence intervals. This divergence indicates that the posteriors obtained from MFVI and SGLD fail to fully capture the correct Bayesian belief updating. Notably, the deviation manifests in im- proved predictive performance, especially in the case of MFVI. This suggests that the martingale posterior approach of generating ensembles by simulating pseudo-data and re- training may be generally beneficial compared to just using the BMA in BNN setups, if the additional computation cost of doing so can be justified.
Proof. In the case, for example, when prior and likelihood densities exist and 0- < p(y") < do for p-almost every v*,
= E(T(( [ = ff T(0 )')((dy") = ग. (0)p(v 10) p(u*) p(dy") (0) / pldo 10) (0))
At least in principle, it is possible for this predictive co- herence to be satisfied even if our approximate inference scheme is itself inaccurate. Thus one might see "Bayesian"
8

---

Do BNNs Actually Behave Like Bayesian Models?
Intal Posterior
Range of Imputed Fosteriers
Martingalc Posterior
SGLD
x
15%
10%
60%-
-1.2
Denstyy
10%
35%
Loo LogUhhhoood
-1.4
Acourscy
576
50%
-1.6
5%
45%
-1.8
0%
076
40%-
.2.0
X
35%
*
Apcurany Confidencs
-22
20%
SGLD
0%
10%
Figure 7: Martingale posterior BMA on CIFAR-10 sub- set. We show the test accuracy, log-likelihood, and ECE for the initial posterior x': the full and inter-quartile ranges of imputed posteriors n"; and the empirical martingale poste- rior with a 95% bootstrap confidence interval with respect to resampling imputed labelings 5"" For both VI and SGLD, individual imputed posteriors underperform the initial BMA, but their martingale ensemble outperforms it. Imputing la- bels should not introduce new information, but here retrain- ing increases performance, falsifying Bayesian coherency.
-20%%
0%
20%
40%
60%
80%
100%
20%
40%
60%
80%
100%
Confidance
Cantidence
-96- Intaaal
Imputed Posteriors (IQF)
Martingale Posterior
Figure 8: Predictive uncertainty calibration of the mar- tingale posterior. Investigating the ECE from Figure fur- ther, for the BMA test-set predictions we show the distribu- tion of confidence, i.e., maximum predicted class probability (top), and calibration curves (Guo et al., 2017), i.c., pre- diction accuracy minus confidence for test samples falling into a given confidence bucket (bottom). The initial VI pos- terior is overconfident while SGLD is underconfident. For both, the individual imputed posteriors x" produce more confident but not more accurate aveer-ede predictions. Their martingale posterior has reduced confidence compared to the initial posterior, leading to under-confident but more accurate predictions. The mismatch is greater for VI.
We further examine the full calibration curves of the model averages Figure 8. While the confidence of the ini- tial trained posteriors increases on synthetically retrained datasets, consistent with being trained on more data, the martingale posterior exhibits reduced predictive confidence despite improved accuracy. This is evident in the calibra- tion plots: the martingale posterior increases accuracy for given confidence level on test data. This behavior parallels the well-established benefits of ensembling: averaging over a diverse set of predictive distributions mitigates overconfi- dence and yields more robust uncertainty quantification.
results do not undermine. We simply argue that practitioners should exercise caution in their expectations and interpreta- tion, especially when applying BNNs in distinetly Bayesian contexts such active, continual, and transfer learning, where these pathologies may limit their utility. This also applies to empirical evaluation, good static performance may not translate to other desired behaviors.
We have thus used this prediction-pove -oweeeed diagnostic to assess whether BNN updating preserves the correct model uncertainty. In this case, both VI and SGLD fall short, leading to suboptimal ensembling over models. Martingale posterior ensembles provide a partial remedy, performing function-space correction using unlabeled data. Section E.2 strengthens these findings on the IMDB dataset.
Overall, we encourage a pragmatic perspective: rather than striving for striet Bayesian adherence, BNNs should be viewed as practical algorithms inspired by Bayesian ideas. This reframing, already acknowledged within the field (e.g. Sharma al. (2023)) emphasizes the importance of factors such as optimizer design, regularization, ease of training, and explicit solutions for coherency where needed. In particular, although the problems we raise are a direct result of inexact inference, improving inference schemes is not necessarily the solution: accurate inference may be un- achievable, and strategies focusing on final behavior could be more effective. Further, entirely non- non-Bayesian methods might in practice give more coherence in some settings.
7. Discussion
We have systematically examined the behavioral properties of common BNN algorithms, highlighting deviations from key characteristics that would be expected of exact Bayesian models. Specifically, we find that approximate posteriors lack functional variability, sequential inference does not propagate prior information coherently, information can be hallucinated or lost when updating, and predictive updates are poorly calibrated.
Going forward, the improvements seen by the empirical martingale posterior predictives hint at an interesting direc- tion. Though requiring additional computation, these led to gains predictive performance without any additional data,
Should we avoid BNNs completely? Definitely not! They have been shown to outperform point estimates and provide effective static uncertainty in multiple settings, which our

---

Do BNNs Actually Behave Like Bayesian Models?
Acknowledgements
Learning, volume 37 of Proceedings of Machine Learn- ing Research, pp. 1613-1622, Lille, France, 07-09 Jul 2015. PMLR.
The authors gratefully acknowledge Mrinank Sharma for his guidance and insightful comments during the development of this work. VS is supported by the UK EPSRC Centre for Doctoral Training in Modern Statisties and Statistical Machine Learning (grant EP/S023151/ 1 and Novo Nordisk. TR is supported by the UK EPSRC grant EP/Y037200/1.
Chen, P.H.., Wei, W., Hsieh, C.-J., and Dai, B. Overcoming catastrophic forgetting by Bayesian generative regulariza- tion. In Proceedings of the 38th International Conference on Machine Learning. 2021.
Coker, B.,, Bruinsma, W. P., Burt, D. R., Pan, W., and Doshi- Velez, Wide mean-fiell Bayesian neural networks ignore the data. In International Conference on Artificial Intelligence and Statistics, pp. 5276-5333. PMLR, 2022.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Dawid, A. P. Present Position and Potential Developments:
Some Personal Views; Statistical Theory; The Prequential Approach. Journal of the Royal Statistical Society. Series A (General), (47(2)278-229) 1984, ISSN 0035-9238.
References
Atlas, I.., Cohn, D., and Ladner, R. Training connectionist networks with queries and selective sampling. Advances in neural information processing systems, 2, 1989.
Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., and Hennig, Laplace redux effortless Bayesian deep learning. In Advances in Neural Informa- tion Processing Systems,, volume 34, pp. 20089-20100 Curran Associates, Inc., 2021a.
Bernardo, J. M. and Smith, A. F. Bayesian theory, volume 405, chapter 2.4.4. John Wiley & Sons, 2009.
Daxberger, E., Nalisnick, E., Allingham, J. U., Antoran, J., and Hernandez-Lobate J. M. Bayesian deep learning via subnetwork inference. In Proceedings of the 38th International Conference on Machine Learning, pp. 2510- 2521, 2021b.
Bickford Smith, F., Kirsch, A., Farquhar, S., Gal, Y., Foster, A., and Rainforth, T. Prediction-oriented Bayesian active learning. In Proceedings of The 26th International Con- ference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 7331-7348, PMLR, 25-27 Apr 2023.
Doob, Application of the theory of martingales. Actes du Colloque International Le Calcul des Probabilités applications, pp. 23 23-27, 1949.
Bickford Smith, F., Foster, A., and Rainforth, T. Making better use of unlabelled data in Bayesian active learning. In Proceedings of The 27th International Conference Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pp. 847-855. PMLR. 02-04 May 2024a.
Falck, F., Wang, Z., and Holmes, C. Is in-context learn- ing in large language models Bayesian? A martingale perspective. In Proceedings of the 41st International Con- ference on Machine Learning, volume 235 of ICML' '2, pp. 12784-12805, Vienna, Austria, July 2024. JMLR.org.
Bickford Smith, F., Kossen, J., Trollope, E., van der Wilk, M., Foster, A., and Rainforth, T. Rethinking aleatoric and epistemic uncertainty. In NeurlPS 2024 Workshop on Bayesian Decision-making and Uncertainty, 2024b.
Fong, E., Holmes, C., and Walker, S. G. Martingale poste- rior distributions. Journal of the Royal Statistical Soci- ety Series B: Statistical Methodology, 8355)1135711391, 2023.
Bissiri, P. G., Holmes, C. C., and Walker, S.G. general framework for updating belief distributions. Journal of the Royal Statistical Society. Series B (Statistical Method- ology), 78(5): 2016.
Foong, A., Burt, D., Li, Y., and Turner, R. On the ex- pressiveness of approximate inference in Bayesian neural networks. Advances in Neural Information Processing Systems, 33:15897-15908, 2020.
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Varia- tional inference: A review for statisticians. Journal of the American statistical Association, 1185118)859-877 2017.
Fortuin, V. Priors in bayesian deep learning: A review.
International Statistical Review, 90(3):563-591, 2022.
Fragoso, T. M., Bertoli, W., and Louzada, F. Bayesian model averaging: A systematic review and conceptual classification. International Statistical Review, 86(1):
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. In Proceed-
ings of the 32nd International Conference on Machine
1-28, 2018
10

---

Do BNNs Actually Behave Like Bayesian Models?
Gal, Y., Islam, R., and Ghahramani, Z. Deep Bayesian active learning with image data. In International conference on machine learning, pp. 1183-1192. PMLR, 2017.
MacKay, J. Bayesian interpolation. Neural computation. 4(3):415-447, 1992a.
MacKay, D. J. practical Bayesian framework for back- propagation networks. Neural computation, 4(3):448- 472, 1992b.
Graves, A. Practical variational inference for neural net- works. Advances in neural information processing sys- tems, 24, 2011.
Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., and Wilson, A. G. A simple baseline for Bayesian uncer- tainty in deep learning. Advances in neural information processing systems, 32, 2019.
Grünwald, P. The Safe Bayesian. In Algorithmic Learning Theory, pp. 169-183, 2012.
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
McAllester, D. A. PAC-Bayesian model averaging. In Proceedings of the twelfth annual conference on Compu- tational learning theory. pp. 164-170, 1999.
Hinton, G. E. and Van Camp, D. Keeping the neural net-
works simple by minimizing the description length of the weights. In Proceedings of the sixth annual conference Computational learning theory, pp. 5-13, 1993.
Mlodozeniec, B., Krueger, D., and Turner, R. Implicitly Bayesian prediction rules in deep learning. In 6th Sym- posium on Advances in Approximate Bayesian Inference, 2024.
Houlsby, N., Huszár, F., Ghahramani, Z., and Lengyel, M. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: / 12.5745, 2011.
Nabarro, S., Ganev, S., Garriga-Alonso, A., Fortuin, V., van der Wilk, M., and Aitchison, L. Data augmentation in Bayesian neural networks and the cold posterior ef- fect. In Proceedings of the Thirty-Eighth Conference on Uncertuinty in Artificial Intelligence, volume 180 of Pro- ceedings of Machine Learning Research, pp. 1434-1444.
Izmailov, P., Nicholson, P., Lotfi, S., and Wilson, A. G. Dan- gers of Bayesian model averaging under covariate shift. Advances in Neural Information Processing Systems, 34: 3309-3322, 2021a.
PMLR, 01-05 Aug 2022.
Izmailov, P., Vikram, S., Hoffman, M. D., and Wilson, A. G.
What are Bayesian Neural Network posteriors really like? In International Conference Machine Learning, pp. 4629 4640. PMLR, 2021b.
Neal, R. M. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Neal, R. M. et al. MCMC using Hamiltonian dynamics. Handbook of Markov chain Monte Carlo, 2(11):2, 2011.
Jaynes, E. T. Probability theory: The logic of science. Cambridge university press, 2003.
Noci, L., Roth, K., Bachmann, G., Nowozin, S., and Hof- mann, T. Disentangling the roles of curation, data- augmentation and the prior in the cold posterior effect. In Advances in Neural Information Processing Systems, volume 34, pp. 12738- 12748. Curran Associates, Inc., 2021.
Kapoor, S., Maddox, W. J.. Izmailov, P., and Wilson, A. G. On uncertainty, tempering, and data augmentation in Bayesian classification. In Advances in Neural Informa- tion Processing Systems, volume 35, pp. 18211-18223 Curran Associates, Inc., 2022.
Kendall, A. and Gal, Y. What uncertainties do we need in Bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
Osband, I., Wen, Z., Asghari, S. M., Dwaracherla, V., Ibrahimi, M., Lu, X., and Van Roy, B. Epistemic neural networks. In Advances in Neural Information Processing Systems, volume 36, 2023.
Kessler, S., Cobb, A., Rudner, T. G., Zohren, S., and Roberts, S. J. On sequential Bayesian inference for continual learning. Entropy, 25(6))884, 2023.
Papamarkou, T., Skoularidou, M., Palla, K., Aitchison, L., Arbel, J., Dunson, D., Filippone, M.. Fortuin, V., Hennig, P., Hernández-Lobato, M., Hubin, A., Immer, A., Kar- aletsos, T., Khan, M. E., Kristiadi, A., Li, Y., Mandt, S., Nemeth, C., Osborne, M. A., Rudner, T. G. J., Rügamer, D.,. Teh, Y. W., Welling, M., Wilson, A. G., and Zhang, R. Position: Bayesian deep learning is needed in the age of large-scale AL. In Proceedings of the 41st International Conference on Machine Learning, 2024.
Kirsch, A., Kossen, J., and Gal. Y. Marginal and joint cross- entropies & predictives for online Bayesian inference. Active Learning, and Active Sampling, 2022.
Knoblauch, J., Jewson, J., and Damoulas, T. An Optimization-centio View on Bayes' Rule: Reviewing and Generalizing Variational Inference. Journal of Ma- chine Learning Research, 23113))11-109, 2022.

---

Do BNNS Actually Behave Like Bayesian Models?
Rainforth, T., Foster, A., Ivanova, D. R., and Smith, F. B. Modern Bayesian Experimental Design. Statistical Sci- ence, 39(1):100- 114, 2024. doi: 10. 10.1214/23-STS115.
international conference on machine learning, pp. 681- 688, 2011.
Wenzel, F., Roth, K., Veeling, B. S., Swiatkowski, J., Tran, L., Mandt, S., Snoek, J., Salimans, T., Jenatton, R., and Nowozin, S. How good is the Bayes posterior in deep neural networks really? In Proceedings of the 37th In- ternttional Conference Machine Learning, pp. 10248- 10259, 2020.
Robins, J. and Wasserman, L. Conditioning, likelihood, and coherence: A review of some foundational concepts. Journal of the American Statistical Association, 95(452): 1340-1346, 2000.
Savage, J. The foundations statistics. Courier Corpo- ration, 1972.
Wiese, J. G., Wimmer, L., Papamarkou, T., Bischl, B., Gün- nemann, S., and Rügamer, D. Towards efficient MCMC sampling in Bayesian neural networks by exploiting sym- metry. In Joint European Conference on Machine Learn- ing and Knowledge Discovery in Databases, pp. 459-474. Springer, 2023.
Sensoy, M., Kaplan, L.,, and Kandemir, M. Evidential Deep Learning to Quantify Classification Uncertainty. In Ad- vances in Neural Information Processing Systems, vol- ume 31, 2018.
Settles, B. Active learning literature survey. Technical report, University of Wisconsin-Madise Department of Computer Sciences, 2009.
Wilson, A. G. The case for Bayesian deep learning. arXiv preprint 10l::2000.10995, 2020.
Wilson, A. G. and Izmailov, P. Bayesian deep learning and
Sharma, M.. Farquhar, S., Nalisnick, E., and Rainforth, T. Do Bayesian neural networks need to be fully stochastic? In International Conference on Artificial Intelligence and Statistics, pp. 7694-7722. PMLR, 2023.
probabilistic perspective of generalization. Advances in neural information processing systems, 33:4697-4708,, 2020.
Zhang, Y., Wu, Y.-S., Ortega, L. A., and Masegosa, A.
Shwartz-Ziv, R., Goldblum, M., Souri, H., Kapoor, S., Zhu, C., LeCun, Y., and Wilson, A. G. Pre-train your loss: Easy Bayesian transfer learning with informative priors. In Advances in Neural Information Processing Systems, Curran Associates, Inc.,
The cold posterior effect indicates underfitting, and cold posteriors represent fully Bayesian method to mitigate it. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
volume 35, pp. 27706-27715 2022.
Sommer, E., Wimmer, L., Papamarkou, T., Bothmann, L., Bischl, B., and Rügamer, D. Connecting the dots: Is mode-connectedness the key feasible sample-based inference in Bayesian neural networks? In Forty-first International Conference Machine Learning, 2024.
Suder, P. M., Xu, J., and Dunson, D. B. Bayesian transfer learning. arXiv preprint 2rXiv:2312.13488 2023.
Tran, B.-H., Rossi, S., Milios, D., and Filippone, M. All you need is good functional prior for Bayesian deep learning. The Journal of Machine Learning Research, 23 (1):3210-3265, 2022.
Wang, L., Zhang, X., Su, H., and Zhu, J. A comprehen- sive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Wang, Y. and Blei, D. Variational Bayes under model mis- specification. Advances in Neural Information Processing Systems, 32, 2019.
Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient Langevin dynamics. Proceedings of the 28th
12

---

Do BNNs Actually Behave Like Bayesian Models?
A. Bayesian Model Averaging
In this section, we revise and introduce terminology for the Bayesian model average (BMA), and its estimation. We also define the classifier evaluation metrics we use throughout the paper to compare different BMA or point predictions.
A.1. BMA for Regression and Classification
In the context of Section 2, the decision-theoretiinn optimal Bayes predictor at fresh input x= with respect to loss function can be constructed the minimizer of the expected posterior loss
y" (x") arg min [[(Y", "y""""""">""""
1²)]""" x., DJ = arg min Eo ( Dy [((Y", ,y") |,",0]=
(1)
e.g. (Hirshleifer & Riley, 1992). For regression under the L loss this corresponds to the posterior mean
(x") = { ())([[(("))))>>))))))))))))>)))) (=(0(=)))=(00 D),
(2)
and for classification under the 0-1 loss this corresponds to the a-posteriori most likely class
arg max ply" |x",D) arg max [ p(y" 1x" r) DD).
(3)
y"
Both predictors rely marginalization, i.e., posterior-weighted averages over models.
A.2. Approximating the BMA From Samples
Having access posterior samples . NN π (0 D), we can approximate the BMA with the ensemble T00 D)
N Σ j=1 50, (6), leading to the following finite mixture posterior predictive approximations:
- T(d0 DD)
N
p("(=")\=",D)
~
8ulfo (z))((()))))((
(4)
N
j=1
for the model mean, and
N
-f p(y" (e,,))((0 D) ~
ply" 1x, D) =
N Σ
ply" 1x"
(5)
for the observed Y*. Under some conditions, including 0, being independent samples from an ergodic Markov chain, these likelihood estimators consistent (Robert et al., 1999; Roberts & Rosenthal, 2004).
This leads to the following (consistent) predictive ligeliiel estimator:
N Σ
logp(y" |,",D) x log
plu" |0", |0,,0,) ,0)) - -log N.
(6)
A.3. Evaluating BNNs
When evaluating BNNs on a test set, for regression, we use the term BMA to refer to the predictive mean estimates
S (x:) ); for classification, we refer the full predictive averaged class probability vector (p(y" k D)) J-1 E/ fü (x;) E/ softmax(gj, (Iz)).
In classification, BNNs often evaluated along the following metries. Let (x", y²) be random sample and its label picked uniformly from the test data distribution.
1. The predictive accuracy ( w-) 1 arg max, ply [0, N-1 ΣN 1(87 arg max, may ply| x D)}:
13

---

Do BNNS Actually Behave Like Bayesian Models?
N 2. (normalized) log-likelihood is Exee.yy Ex*",yr) logp(y" |x" D) Niess Σx=1 log p(y x; D);
3. and finally, to introduce calibration and expected calibration error (ECE) as defined by Guo et al. (2017), let us define
confulence as the maximum predicted class probability C(x*) maxy softmax(go(e) D)y;
calibration error as absolute deviation of calibration from confidence at given confidence level
Calibr(x")
(e wr)) 1yy arg
, D)} C(**) C(x*)
and the expected calibration error Eg- Calibr(x*).
The lower the ECE, the more "reliable" the output softmax probabilities are; the SGD solution has been often found to be overconfident its predictive softmax probabilities (Guo al., 2017; Izmailov et al., 2021b).
B. Approximate Inference Algorithms
In this section we introduce some of the popular approximate Bayesian inference algorithms for BNNs.
B.I. Hamiltonian Monte Carlo
Hamiltonian Monte Carlo (HMC) (Neal et al., 2011; Betancourt, 2017) is Markov-chain Monte Carlo (MCMC) algorithm that is the state of the art for sampling from BNN posteriors (Wenzel et al., 2020; Izmailov et al., 2021b). In high-dimensional distributions, such as BNN posteriors, there are many counter-intuitive phenomena not found in low dimensions, such as the concentration of mass to a thin shell some distance away from the mean. An effective sampler needs to explore this rypical set of points of high density efficiently. Classical algorithms such as random-walk Metropolis or Gibbs are not suitable for this: because the shell is thin, random-walk proposals are rejected or stay local, and Gibbs does not mix well because the level sets of the density are non-convex.
HMC operates on the state space extended by the momentum vector of a particle whose movement is governed by Hamiltonian dynamics. The proposal is computed by numerically integrating the path of the particle, using first-order gradient information. This dynamics allows the particle to travel along the typical set so that the proposal is far away, but still has a high probability of acceptance.
More concretely, assume we wish to sample from (q).. We extend the target by a momentum vector p of the same dimension as q: T (p, 4) (q) (pl 4) for some suitable distribution (p q)- Let us define (p, q) log (p) g) log क ((4) K ((p.g) q) (4) to be the Hamiltonian, K(p, q) the kinetic energy, and (g) log π T()) the potential energy of particle.
Motivated by Hamiltonian dynamics, we consider simulating the state of a particle over time governed by the differential equation
dg 0H 0K
dt Öp 0p
=
(7)
dp OH
aK
0V
(8)
dt 0g
3q 0g
for some period to get and q". By change of variables, it is easy to check that H/ stays constant, T(p, q) = T ,²)), Thus, if (p. q) is point in the typical set, so is 1p",9)). Note that WV
can be computed from an unnormalized density म (q))
By
which is crucial for sampling from the Bayesian posterior where the normalizing constant is intractable.
To build a MCMC algorithm based on this idea we construct a random- walk Metropolis proposal by numerically simulating the dynamics using step size for L iterations, with additional tricks to ensure irreducibilitty and symmetry of the proposal distribution. Note that under perfect simulation the densities are equal, hence the acceptance probability is always one. The accept-reject step corrects for the integration error. Marginally, the q state of the particle will efficiently explore the typical set of the target distribution.
The No-U-Turn sampler (NUTS) (Hoffmman et al., 2014) is an adaptive variant of HMC, adjusting the step size e and number of steps L, with the goal of maximizing the distance between the current point and the proposal without wasting computation. This allows NUTS to discover the high- nigh-dimensional state space faster and avoid the need to tune these hyperparameters manually.
14

---

Do BNNS Actually Behave Like Bayesian Models?
HMC is computationalty feasible for sampling from small BNN posteriors, but is prohibitively expensive for networks of size found in modern deep learning applications. For example, Izmailov et al. (2021b) deploy hundreds of Tensor Processor Units to study the mixing and performance of HMC on real-world nets, and find that they mix well in predictive space, although mixing in weight space is worse. For smaller networks, HMC is widely considered the gold standard for representing the Bayes posterior.
B.2. Variational Inference
Variational inference (VI) (Hinton & Van Camp, 1993; Graves, 2011; Blundell et al., 2015; Blei et al., 2017) considers variational distributions from the family Q = {a(0; vb) v e V to approximate the posterior T(0 (D). The metric used is often the "reverse" KL divergence
g(0; v)) T(0 D)
(Kr,(g(0;v)) (0 ()) = BBwg(-ww log
VI casts the (approximate) inference problem as optimization, finding the variational distribution minimizing the divergence from the posterior. The objective is equivalent to the objective maximizing the evidence lower bound (ELBO)
R/² E arg min (KL(A(0:)))
())
=
ai² E arg max (1(;; D),
(10)
where
Co(D; Eg log T()) 11 plyA I4, 0)
(11)
((0:v))
which no longer depends on the normalized posterior density =00 x))
Note that the ELBO can be rearranged as
(1 D) = Σ Ea p(ys x4 0) - DKL(4(0:))) x(0)),
(12)
providing insight into the optimization objective: the first log- log-likelihood term encourages the variational distribution to assign high likelihood to the observations, and the second term acts as a regularizer, drawing the variational distribution towards the prior.
For BNNs we need to resort to stochastic gradient-based optimization (Hoffman et al., 2014; Wingate & Weber, 2013). Assuming our variational distribution can be reparameterized, i.e., 0 ~ q(: v) can be drawn as e A o(-),
(c) for some Ta. differentiable transform with non-singular Jacobian, parameterized by variational parameters tb. Then
the variational density can be computed via change of variable as q(;; (b) slE) det dTutel de
The stochastic ELBO
gradient estimate becomes
1 N
V log ((To(E;)) IT p(U: 2sy (e)))) o(E,) det dTo(rs)
V. CoLDV:
(13)
lid. where ej ~ NO I). This can conveniently be computed using automatic differentiation engines.
For BNNs the variational family Q needs to be simple enough to be able to store the variational parameters ve memory,
as even quadratic memory requirement in the number of model parameters prohibitive. The mean-field assumption commonly employed (Blei et al., 2017: Farquhar et al., 2020b) where the variational distribution factorizes q(0; v)
II, q(,,; 22, In particular, for BNNs the most commonly used variational family is the mean-field Gaussian q(0;; v/1) (0;;; (4a of?) where v consists of vector of means vم(9) and vector of scales (())
"Even though this seems like restrictive family, enforcing independence, it has been shown to expressive in deep networks (Farquhar al., 2020a)
15

---

Do BNNs Actually Behave Like Bayesian Models?
In many cases, the KL term of Equation (12) is available in closed form, allowing one reduce the variance of the gradient estimator:
1 VoLo(V: D) = N
N
" Ve log Fu, To(e)))))))))))))))))))))))))T - V DxL(((0;;))) T())).
(14)
For example, in the mean-field Gaussian case with mean-field Gaussian prior ((0) = (N((mm (2((1))
+ Lass + (1()) Mo), 1 2
(1))) = Σ
log (m)a (1).s
(15)
20 (m).
In practice, one needs many ad-hoe tricks and tweaks to make VI work on larger models (Farquhar et al., 2020a). Without these, optimization fails due to the high variance of the stochastic gradients. To reduce this, practitioners have initialized the variational variances to very small values aid the first stage of optimization, used early stopping criteria, or used the 3-ELBO* with B < 1:
6 (1b;
(1): D)
Σ
Es Bo- log x., 0) - (exx(((00))) (0)),
(16)
1=1
over-weighting the observation likelihood compared to the regularization term, allowing tighter fit to the data at the cost of less regularization towards the prior.
B.3. Laplace Approximation
The Laplace approximation (Laplace, 1774) provides a local Gaussian approximation of a distribution around its MAP. It has been widely used as an approximate Bayesian inference algorithm (MacKay, 1992a) both classically and recently in the context of Bayesian deep learning (Daxberger et al., 2021; Schnaus et al., 2023).
Laplace approximation makes a second-order approximation to the log-density of the distribution: given probability density f00 and MAP solution MAPP.
1 20 @MAP) T log ((MAAP 0MAP)
( + log f(0) log f(0MAP) + V log f(6MAP) 0MAP)
(17)
1
log f(0MAP) + =00 - 0MAP) T log f(0MAp) (0 - 0MAP)
(18)
2
assuming the gradient vanishes at the MAP, resulting in the approximation
f(-) N(:: 1-v2 log [(MAA))
(19)
once the density is normalized.
While Laplace and Gaussian VI both produce approximate Gaussian posteriors, it is important note that VI is global approximation whereas Laplace is local approximation. We will see in experiments how this distinction defines the behaviors of their approximate posteriors.
In the context of Bayesian inference, the likelihood factorizes log (0) y) = log (0) + log p(y æz, 0) + const. Alternatielly to the Hessian of the negative log-like log-likelihood, it is common to use the Fisher information matrix
T(0) = - V logp(Y Y xr, xx, (,00)10 00] EE [VV log |,, (V) lYg |2, ()) [0]
(20)
the empirical Fisher
i(0) (V logp(y) x, () logp(y) x. 8))
(21)
at 0MAP. further widely used alternative is the generalized Gauss-Newton (GGN) matrix, which is a positive semi-defnite approximation the Hessian, coinciding with for common observation models (Daxberger al., 2021; Murphy, 2012).
'Analogously to the 3-VAE (Higgins al., 2016).
16

---

Do BNNS Actually Behave Like Bayesian Models?
The size of the Hessian is quadratically large in the number of model parameters, which is prohibitive for real-world BNNs of thousands millions of parameters. This necessitates further approximations. The simplest is diagonal approximation to the Hessian or the Fisher information matrix (MacKay, 1992b). Even though it is restrictive approximation enforcing independence in the posterior, recent work by Farquhar et al. (2020b) found it to remain expressive for deep networks.
Other approximations produce low-rank or block-diagonal Hessian or Fisher matrices. In particular, the popular Kronecker- factored approximate curvature (K-FAC) (Martens & Grosse, 2015; Daxberger et al., 2021; Schnaus et al., 2023) block- diagonal upproach further factorizes the per-layer Fishers, which are themselves infeasible to store for large networks.
We note that the diagonal Laplace approximation we evaluate is not the most expressive. However, having run full-rank Laplace experiments on smaller networks where it is feasible, we have seen no reason to assume these qualitative behaviors change for more expressive Laplace approximations, including K-FAC.
B.4. SWAG
Stochastic weight averaging (SWA)-Gaussin) (SWAG) (Maddox et al., 2019; Wilson & Izmailov, 2020) is heuristic post-hoc method of obtaining approximate posteriors of trained deep learning models. Interpreting the loss as the negative log-likelihood, the trained solution is OMAP. SWAG performs further gradient descent updates on @ with high learning rate exploring the region of the weight space near OMAP. We record the trajectory of B and fit a low-rank Gaussian to these samples to get the approximate posterior.
Its motivations are arguably Bayesian. These include the local quadratic nature of the loss function around MAP,, the same fact that is exploited by Laplace. The hyperparameter controls the posterior variance: the larger the learning rate is, the higher-variance the samples from the trajectory. It has been used by Shwartz-Ziv et al. (2022) in the context of Bayesian deep learning to pretrain prior for transfer learning task.
B.5. SGLD
Stochastic Gradient Langevin Dynamics (Welling & Teh, 2011) is a popular practical Markov Chain-based sampling scheme for BNNs. The continuous-timg Langevin diffusion with stationary measure [() D) is
d0.- log D)dt vBB.
where B, is a Brownian motion. The score log (0 (D) makes it tractable to deal with the posterior because the normalizing constant cancels out. To simulate this continuous-timg dynamics, one traditionally uses Euler discretization schemes
0-1 0, ez log n (0. (D) + Verk.
where e is the learning schedule and & it NO, 1). For further computational efficiency, one would take stochastic estimates of the gradient by estimating it in batches of size b.
D B.
0c+1 = 0, Ex V log (l,)
Vlogply ( (x.8)EB.
Σ
0;ar) + 22rEe.
For large batch sizes and small e, we expect the chain to be close to the stationary distribution; however, the discretization and the stochastic gradient estimate both introduce some bias. It is possible to add a Satropolis-Hssings-syy// accept-reject step to the algorithm to unbias it, but most practical algorithms choose to skip this step for computational efficiency, arguing that the bias is compensated for by decreased variance.
B.6. Further Bayesian and Non-Bayesian Alternatives
Other Bayesian approximate inference algorithms include Monte Carlo dropout (Gal & Ghahramani, 2016), and further variants of stochastic- trchastic-grrdient Monte Carlo (Chen et al., 2014; Zhang et al., 2020; Wenzel al., 2020).
There are various further methods for averaging over different models producing uncertainty estimates that traditionally considered Bayesian in their motivation. Deep ensembles (Lakshminarayanan et al., 2017) are state of the art for accuracy and out-ol-sample detection on many tasks. They are an ensemble of K different MAP solutions MAP. often
17

---

Do BNNS Actually Behave Like Bayesian Models?
trained with added repulsive force between them to ensure they converge to different MAP solutions in terms of their generalization behavior. Even though they are not widely considered Bayesian approach, there have been arguments for considering these discrete model averages an approximation to the BMA (Wilson & Izmailov, 2020; D A'Angllo & Fortuin, 2021)
K Σ ply x. (MAP) ~ S p(y x., ()0 (D).
(22)
K
For modern large-scale models, training multiple MAPs is not feasible, offering a future research direction of more scalable ensemble-based methods.
Other non- -an-aaaeiinn approaches include epistemic neural networks (Osband et al., 2023) that incorporate small amount of randomness as input to the network, which allows marginalization of joint predictions.
C. Martingale Posteriors: A Review
In this section we briefly discuss the martingale posterior framework of Fong al. (2023), and provide more context on how we apply its ideas test for Bayesian behavior.
The traditional Bayesian approach is to conduct a prior- posterior calculation some parameter of interest. Specifically, in the i.i.d. setting, with an observed sample Yim we specify prior density π rf() and sampling density fo()) to derive the posterior (0) V1:n). This posterior is then used to compute the predictive density
/ fo(y))(() |
ply y1m) =
The fundamental premise of Fong et al. (2023), however, is that statistical uncertainty arises due to missing observations. If a complete population could be observed, any identifiable parameter of interest would be known precisely. Under this viewpoint, the foundation of Bayesian inference is not to update a prior distribution, but to assign predictive distribution on the missing observations conditional on what has been observed. Uncertainty is then indexed over different possible imputations of the complete dataset.
More specifically, Fong et al. (2023) partition inference into two tasks: simulating the missing future data Yn+1:00 (or, in practice, Yu+I=N for some sufficiently large N), and then recovering the parameter of interest from this complete data. Given the observed yi.., the key component for this pipeline is sequence of one-step-ahead predictive distributions {p(- (1s)) which simulate an infinite future dataset 1=+1:00 through sequential process of predictive resampling:
Yx+1 p(- Vim), Yu42 ~ p( 111m+1).......... ...,, Yn p(-111.N-1) for N +
The parameter of interest, denoted 8z. is then computed from the combined observed and imputed data (yi:ns Yut1: The distribution of this computed 0x is defined as the martingale posterior distribution.
The underlying justification for this approach applies result from Doob (1949), which demonstrates an equivalence between Bayesian parametric and predictive uncertainty. In the above example, the authors demonstrate that choosing the standard Bayesian posterior predictive distribution would indeed recover the conventional posterior distribution of the parameter.
C.1. Predictive Evaluations of Bayesian Updating
The predictive representation of Bayesian inference presented above suggests the possibility of predictive diagnostics to evaluate whether models are respecting the properties of Bayesian inference. For instance, Falck et al. (2024) use this one-step-ahead characterization to probe whether large language models are Bayesian in their in-context learning abilities, answering negatively.
Mlodozeniec et al. (2024) take a similar approach to probe small-scale Bayesian deep learning models via repeated re- training for sequential inference. They evaluate whether various predictive rules are "implicitly Bayesian" by testing whether new predictions are exchangeable with observed data, finding that Bayesian algorithms may fall short on this test in practice.
In the case of BNNs, the martingale posterior framework mirrors existing trends that decouple Bayesian inference from the standard parametric interpretation, and instead takes a functional perspective focused on behavior in predictive space.
18

---

Do BNNS Actually Behave Like Bayesian Models?
However, one-step-ahead predictives correspond re-training the Bayesian model with extra (synthetic) observation repeatedly, which is infeasible for at-scale neural network models.
C.2. Our Adaptation for BNNs
To test the extent of adherence Bayesian principles in real-world BNN settings, therefore adapt this framework into single-step predictive resampling procedure, presented in Section 6.
Broadly speaking, our resampling approach corresponds to jointly generating Y=+I:N p(- (1:) as N - (corre- sponding to the synthetic labeling procedure) instead of the one-step-ahead decomposition. In the asymptotic case, Doob's s result implies that the resampled dataset recovers the true setting of the parameters 0. In the finite case, however, we leverage Proposition I''s coherence property to see that we recover the initial setting of the parameters in distribution. This finite-case predictive coherence result allows us to probe st-le BNNs by using test functions (the Bayesian model average) investigate the closeness of the finitely-resmppled martingale posterior distribution to the initial distribution.
C.3. Conditions for the Martingale Posterior Well Defined
In the context of Section 6, most generally, let us consider samples imputed from some predictive p' (dy"). Assume that for (p'-almost) every y" the updated belief density exists TT- (0). Then, by Fubini's theorem, MP is well-defined density.
Further, we often care about quantity of interest, MP - ( h(0), for example, the BMA of Section A in the case of our experiment. For these martingale posterior predictions to be well defined, we also require that
|f [1)) (0)y' (dy" dd < oo.
This is satisfied in our classification experiments because the BMA is bounded.
19

---

Do BNNs Actually Behave Like Bayesian Models?
D. CIFAR and IMDB Experimental Setup
The setups of the CIFAR and IMDB experiments are entirely based on the work of Izmailov et al. (2021b). We de- liberately choose a working codebase and replicate the exact settings from one of the prominent papers in the field where Bayesian algorithms are shown significantly outperform point predictions. This is ensure that our experi- mental findings around shortcomings not merely an artifact of suboptimal pareepparamtttiizaaionn. Specifically, we follow the papee--prameters from Table 4 of Izmailov et al. (2021b). We make our fork of their codebase available at gi github. wwt. pwtppptttpp///eeppm saan_sse_vi_ee aeeeee/mstter/bnnne bmc_
E. Martingale Posterior Experiment: Further Details and Results
E.1. Estimation
We described the estimation of the empirical martingale posterior distribution in the concrete setting of the CIFAR-10 experiment in Section 6. This section elaborates further on the methodology to sample synthetic labels, and calculate bootstrapped confidence intervals for reproducibility.
Recall that we wish to do predictive resampling of the set of observations Y at x² given the posterior predictive ((-10;zz)) (d0). Given the conditional independence of the Bayesian setup, the observation model factorizes.
This gives us the following procedure: use the T posterior to sample a setting of the parameters 8, then noise the predictions fo(=") through the observation model Y p(- ()) = p(- fo()))) In the classification setting, this "noising" means calculating the logits from the sampled model and drawing independent categorical variables according those logits for each unlabeled observation. For regression, it similarly means noising the predictions with independent variables according to the observation model.
Given access to n i.i.d. synthetic labelings {0r Hiz we approximate a retrained posterior (,) - " "(a.a))(tt . (0) for each. Then, we want to ensemble these posteriors to produce the empirical martingale posterior. A group of ls samples is drawn from the individually retrained synthetic posteriors each (0) ~ ( Under Bayesian inference, according to Section 6, each 0 is distributed as n², although they are not independent within groups. Specifically, any integrable test function (0) is consistently approximated by our empirical martingale posterior
(O
h(!)).
We could pick any (integrable) test function to check the equivalence of the two distributions by comparing its expectation under both. We pick the Bayesian model averaged predictions from Section A. The model-averaged logits of the individually retrained posteriors are averaged to get empirical martingale posterior BMA.
For VI, we simply sample = 20 settings of the parameters corresponding to ordinal seeds, and k 20 ensembling samples. For SGLD, we take n 18 equally spaced checkpoints post-warmup, from 1450 to 9950, using all k 900 correlated Markov chain samples to ensemble the posteriors. The seeds used to draw the noising variables for labeling images match the seeds for the n' samples (for VI), or the checkpoint indices (for SGLD).
To get confidence intervals on our martingale posterior distribution test metries, we resample and ensemble these individually- retrained synthetic posterior model averages with replacement, yielding bootstrap confidence interval around the empirical martingale posterior BMA. Concretely, let ( [B ~ (111, n] } be random bootstrap indices resampling the individually retrained synthetic posteriors with replacement. The bootstrapped value of a test statistic h()) is given by
h (B)
1 nk
B.
whose quantiles give an asymptotic confidence interval for our empirical martingale posterior mean estimator above, because between groups, the 0 are independent.
20

---

Do BNNs Actually Behave Like Bayesian Models?
E.2. IMDB Experiment In addition the CIFAR-10 result of Section perform the martingale posterior on the IMDB text classification dataset.
Initial Posterior
Range of Imputed P Posteriors
Martingale P Posterior
-0.45
80%
12%
0
Log-Likellhodd
10%
x
78%
-0.5
8%
ECE
76%
-0.60
6%
4%
74%
2%
-0.70
VI
SGLD
Vi.
SGLD
(3) IMDB martingale posterior evaluation.
VI
SGLD
100%
30%
Dersity
20%
50%
10%
Accuracy Confidence
15%
0%
+e- Ke
-10%
0%
60%
80%
100%
60%
80%
100%
Confidence
Confidence
Initial Posterior
Imputed Posteriors (IQR)
Martingale Posterior
(b) Confidence and calibration of IMDB martingale posterior
Figure 9: Martingale posterior Bayesian model averages IMDB splits of size 4000 each. These plots correspond to Figures 7 and 8, respectively. The VI results are almost identical, strengthening the argument that the martingale posterior can provide performance gains in this setting. For SGLD the performance comparison of the martingale posterior is inconclusive. In this binary classification case SGLD already very underconfident, and while more ensembling leads to comparable or slightly better accuracy, this does not translate to improved likelihood under this mis-calibrated setting. However, we still observe significant incoherence in uncertainty calibration, which this experiment was originally set up to test.
21

---

Do BNNS Actually Behave Like Bayesian Models?
F. Sequential Inference Experiment: Additional Results
E.I CIFAR-100 Results
CIFAR-100 VI
CIF AF--000 SGD
Accuracy
40%
20%
20%
Log Iathood
ECB
5%
20%
2%
0%
200
200
Epoch
Epoch
MFVI
Sequential
SGD
"Sequental" SGD
Figure 10: BMA comparison of single-step and sequentially trained CIFAR-100 MEVI posteriors. Mean and standard deviation of test metries from Section A are shown across seeds. The single-step posterior significantly outperforms the two-step iterated posteriors across three metrics. Unlike CIFAR-10 and IMDB, here the SGD baseline does not show improvement. Although the informative initialization seems to helps at the beginning of training on the second split, the different training hyperparameters may mean this information is lost by the end of its training. Regardless, the performance drop from iterated VI training is even more pronounced than in Figure 4.
F.2. CIFAR-10 Experiment with PCA Split
We perform the experiment of Section 5.1 on CIFAR-10 a second time, on a non-randomized split along the first PCA direction of the training set. This analogous the regression analysis in Section 5.1 where we split the dataset along the axis. Classical Bayesian learning methods such Gaussian Processes able preserve uncertainty in regions with no observations by falling back to the prior, and we have collected evidence that some Bayesian deep learning methods may not do so. This experiment may suggest potential further inconsistencies on top of the results from Section 5.1, but conclusive explanation would require more evidence.
85%
85%
80%
Acouray
75%
70%
70%
MFVI Sequential MFVI
MFVI
65%
Secuential MFVI
Sequeetial MFVI (PCA)
Sequential MFVI (PCA)
60%
50
100
150
200
250
300
50
100
150
200
250
300
Epoch
Epoch
(a) Split
(b) Split 2
Figure 11: Comparing random and PCA-split CIFAR-10 sequential MFVI experiments. We perform the experiment in two ways: training on the PCA split in the two possible permutations (mean and std across 20 seeds). In the first training step the PCA split underperforms the randomized training, as one might expect. On the second split, however, the results are inconclusive: one the runs PCA looks worse than random, but in the other run it looks equally as good.
22

---

Do BNNs Actually Behave Like Bayesian Models?
G. Synthetic Regression Experimental Setup
We make our codebase for the synthetic regression experiments available at github. http://cco//pttto/oo/ee mac. mac.
G.1. Tasks
In our experiments we consider synthetic regression tasks. Even though real-world BNNs are almost exclusively used for classification tasks, there high relevance of BNN's performance and behavior on regression tasks. The pre-activation of the last layer of classifier architectures (usually a softmax layer) is -ww-dimessional (unconstrained) vector that behaves like the output of a regressor network. If BNN regressors fail respect properties of Bayes, that is an indication of failures for classifiers too, even though they might manifest more gracefully because of the cruder observation model.
The posterior predictive is difficult to visualize if the input is high-dimensional. Therefore, we mainly look at one- dimensional tasks. In the real world, deep learning is used on very high- dith-dimmnsional tasks, and there is possibility that BNNs on low and nigh-dimensional tasks behave differently. We also consider tasks where the one-dimensional underlying input is feature expanded using 100 random Fourier features under a radial basis function kernel (Williams Rasmussen, 2006). This may provide insight into any differences between BNNs on low-dimensional and medium- dimensional inputs.
We generate toy datasets of the form
1 ( Bax + 2 ) sin(4x) + 8~ ~(0,0)
Y Σ
(23)
for some choice of and the 3, which is complex enough to behave differently in different regions around zero. We observe data in two different groups separated by gap to assess both interpolation and extrapolation uncertainty of posterior predictives. Because common weight-space priors are not translation nor scale invariant, we consider covariates near zero, and standardize the observations according to standard practice.
G.2. A Architectures
We consider two fully-com conlecconnected architectures: a small and a larger network. The small network has three hidden layers with 32, 32, and 16 hidden units each, and has 1,714 14 model parameters. The larger network has hidden layers with 128, 256, 128, and 64 hidden units each, and has 74,690 model parameters, still short of the number of parameters of many modern networks.
The non- -incariyyy used is SiLU ("softplus", "swish", + 2/(1+e +*))), standard in Bayesian deep learning.
1), which gives smoother posterior than ReLU and is
The observation model, unless otherwise specified, is heteroskedasttic: the network outputs a mean no and a scale The last non-linearity of (x) is scaled by 0.2 to control the prior observation noise levels compared to the predictive mean, and a small number 10-2 is added it, which helped control the variance of the stochastic estimators for ELBO gradients and predictive log- likelihood by avoiding vanishing observation likelihoods.
value (2) parameterized by network parameters 0, and the output is observed as Y | A ~N(Hu (z)
G.3. Priors
In line with standard practice (Fortuin, 2022) we use an isotropic Gaussian prior on weights and biases of the network unless otherwise stated. We specify the variances according to the He initialization (He et al., 2015): w/y. by NNO0, B,) where De is the dimension of the activation into layer t. This ensures that the variance of activations stays constant across the layers.
Note that the isotropic Gaussian prior is not translation invariant in predictive space, in contrast with e.g., Gaussian process priors. Away from zero, their predictives tend to flatten off. Therefore, in line with standard practice, we standardize the input features.
G.4. Inference Algorithms
To carry out our experiments we use NumPyro (Phan et al., 2019; Bingham et al., 2019), a probabilistic programming library in Python built on JAX (Bradbury et al., 2018). NumPyro readily supports efficient HMC and stochastic variational inference on probabilistic programs.
23

---

Do BNNS Actually Behave Like Bayesian Models?
HMC. For HMC inference, we use the adaptive No-U-Tums (NUTS) sampler (Section B.1), running four chains in parallel, each initialized to independent MAP solutions. For larger networks, this choice of initialization proved crucial, often not mixing when initialized randomly, hinting at the fact that certain parts of the weight space are ill-conditioned. and HMC does not explore the whole space.
After a burn-in, we use 150 samples from each chain to make predictions, i.e., 600 samples in total.
VI. We use mean-field Gaussian VI, as Section B.2. our the KL term of Equation (12) between the prior and variational distribution is analytic, as both are Gaussian, so NumPyro can use the stochastic gradient estimate of Equation (14).
To deal with the high variance of the gradient estimates, in line with standard practice (Section B.2) we use a number of tweaks:
We initialize the variational variances to small values (10-5) so that Var V 22 (vh; D)) is small initially,
Intuitively, Var(V (()) (vi; D)) is small if (ar (v) D)) Varg~g(:)) im log p(y [ 0)) is, which in turn is small when Varg-a(-wa wi) (f) is small.
We also use momentum through the Adam (Kingma Ba, 2014) optimizer, and gradient clipping to smooth out the optimization trajectory.
Lastly, we use the 3-ELBO (Equation (16)) with a suitable 3 < 1 as the maximization objective. This is necessary for more stability during optimization, and better posterior fit (Farquhar et al., 2020a).
We pick 3 = 0.325 for the small network and 3 = 0.05 for the larger network, based on grid search comparing the posterior predictive uncertainty the resulting posterior to the HMC approximation.
Laplace. We use the diagonal approximation to the empirical Fisher information matrix [(OMA)) from Section B.3. Although many in practice use the more expressive K-FAC approximation, there are cases to be made for the diagonal approximation (Farquhar et al., 2020b), and we have not found even the full-rank Hessian or empirical Fisher to give qualitatively different posteriors.
To increase the stability of the curvature, we need to add a shrinkage factor À to the precisions. The diagonal of the curvature of the loss function (0) is
(I(O)
2
al(O)
09,
08,-
Adding the shrinkage term and inverting eles we get the scales of the isotropic Gaussian posterior:
01(0)² 09;
1/
+ +
The added shrinkage decreases the isotropic Gaussian parameter scales, making the posterior more confident in the MAP solution.
It also related the cold posterior phenomenon, artificially making the posterior approximation sharper by decreasing the variance of the Gaussian in each direction. We pick À 300 for the small network and X 2000 for the large network. based on a grid search comparing the posterior predictive uncertainty of the resulting posterior to the HMC approximation.
SWAG. Our SWAG implementation relies on the Optax SWAG library (activatedgeek, 2023). 400 samples are collected during high learning-rate gradient descent, and rank-20 Gaussian fitted them. Based on similar grid search tuning, 77 0.02 for the small network and 77 0.01 for the larger network.
24

---

Do BNNs Actually Behave Like Bayesian Models?
H. Synthetic Regression: Tractable Setting
As sanity check, we compare the approximate posteriors produced by our BNN algorithms to the true analytic posterior in Bayesian linear regression setting. To keep higher imput dimensionality, we expand into 20 random Fourier features with squared exponential kernel, approximating (sinusoidal) Gaussian process regression.
Recall that Bayesian linear regression with known observation noise admits closed-form posterior updates under Gaussian prior. Let (X) E RNDD denote the random Fourier feature matrix for inputs X, with (x = 2 cos(w) (wj x + bj). where w 1O II) and by Uniform(O, 2x). We assume the generative model:
Prior: B~ N Eo)
Likelihood: y X, (
where o² is fixed. By conjugacy, the posterior distribution weights B3 is Gaussian:
y. N (M En),
1
En Eg" + σ2 xX)) [(()
1 Nn En Eo Mo + (xx) y
This analytic solution provides grounddtrutte baseline for evaluating approximate inference methods. In experiments, we set 140 0 and En I.
We conclude that, unlike the at-scale experiments throughout the paper, in this small classical Bayesian problem, even our approximate inference algorithms such as MFVI adhere to Bayesian properties, even if their approximate posteriors are not faithful (Figure 13). This further underlines that mayesian-inspired inference large-scale Bayesian deep learning models leads to different behaviors compared to classical settings, and the benefits of Bayesian approach do transfer by default.
rior
True Posterior
HMC
SGLD
-2
Full-rank VI
Mean-field VI
Full-rank Laplace
Diagonal Laplace
-2
5.00
2.5
0.0
2.5
5.0
-500
-25
0.0
2.5
5.0
.5.0
-255
0.0
2.5
5.0
3.0
-2.5
0.0
2.5
5.0
Figure 12: Comparison approximate BNN posteriors to the true posterior in Bayesian linear regression. this scale HMC seems match the true posterior. SGLD captures increased uncertainty far away from data. Full-rank Gaussian approximations match the posterior, itself a (full-rank) Gaussian, but diagonal approximations are qualitatively very different.
25

---

Do BNNs Actually Behave Like Bayesian Models?
Data points 0-20
Data points 20-40
Data points 40-60
-2
44
Data points 60-80
Data points 80-100
HMC on data points 80-100
-2
-4-
-6
-2
-6
-2
2
6
-6
-2
2
6
90% HP DI on mean
90% HPDI on prediction
Posterior functonn draws
Fosterfor mean
Previously seen data points
(a) True posterior
Data points 0-20
Data points 20-40
Data points 40-60
1.5
1.0
0.5
-100
Data points 60-80
Data points 80-100
HMC data points 80-100
1.5-
0.5
-1.5
90% HP DI on mean
90% DI on prediction
Posterior functio draws
Posterior meen
Previously seen deta points
(b) MFVI inference
Figure 13: Comparing MFVI analytic posterior in sequential inference. The setting replicates that of Figure 3. Exact inference demonstrates the desired coherent behavior. MFVI updating still shows coherence, despite its apparent lack of faithful posterior representation. For this classical Bayesian setup, Bayesian properties. This contrasts the BNN behavior we find at scale.
inexact approximate inference schemes exhibit
26

---

Do BNNs Actually Behave Like Bayesian Models?
I. Synthetic Regression: Homoskedastic Observation Models
For the purposes of the synthetic regression experiments presented elsewhere the paper, we made the choice to use a heteroskedastic observation model I, 9 (z) where fo(x) (1)( ( (x)).. This choice was made increase model complexity to get closer to large-scale problems that Bayesian deep learning solutions will encounter in practice.
We recognize that estimating variance is inherently more difficult than estimating the mean, e.g. Detlefsen et al. (2019). Therefore, in Figure 14 we repeat our sequential experiment from Section 5,1 under homoskedastic observation models.
Data points 0-20
Data points 20-40
Data points 40-60
2-
2-
Data points 60-80
Data points 80-100
HM on data points 80-100
4-
2-
2-
-5
00% HPO1 on mean
90% HPDI on predicton
Posterior function draws
Posterior meon
Previcusly seen data points
(a) Fixed true noise scalc.
Data points 0-20
Data points 20-40
Data points 40-60
2
2-
Data points 60-80
Data points 80-100
HMC on data points 80-100
4-
6-
90% HIP mean
90% IiPDI on prediction
Posterior function drows
P Posterior mesn Previoualy seen data points
(b) Inverse Gamma prior on noise scale.
Figure 14: MFVI sequential inference under homoskedastic observation models, both cases, the results agree with the conclusions of Figure 3.
27

---

Do BNNs Actually Behave Like Bayesian Models?
J. Synthetic Regression: Further Results
In this section we present further results regarding our synthetic regressions experiment for completeness. In particular, we give evidence for the robustness of our regression results scaling the network architecture via larger hidden layers or the input dimension random Fourier feature (RFF) expansion.
J.I. Posterior Uncertainty for Larger Networks
Recall from Section G that we perform our experiments on two network architectures with a smaller and a larger network size, and also on a Fourier crature-expanded version of those architectures. Here we begin presenting these complementary results.
In Figure 15 HMC appears to produce uncertainties in line with our expectation, reducing near the data, and increasing where there have been no observations, including in- between the two groups of observations. VI and Laplace also produce somewhat acceptable uncertainty, but their BALD scores highlight differences their uncertainty from HMC. SWAG fails to produce sensible uncertainty, and its BMA even fails to fit the data well.
Prior
HMC
Function draws (mean+-std)
4-
2-
0-
>
-2
-2
-4
-4
-6
-6
-2
-2
X SWAG
Laplace
BALD scores
90%
90% HPDI en prediction
4-
HMC
Pasterior mean
2-
2.0
Laplace
SWAG
0
-2
1.0
-4
-3 2
-1
3
-1
1
-2
3
Figure 15: Larger network approximate posteriors trained on one-dimensional toy regression task. They are represented by the posterior predictive means, 90% credible sets for mean representing model uncertainty, 90% credible sets for the observation representing full predictive uncertainty, and their BALD scores (measured in natural units of information) giving insight into underlying weight-space uncertainty.
28

---

Do BNNs Actually Behave Like Bayesian Models?
HMC
VI
log -i-leehhhodd
-30
-100
115
-200
-250
Laplace
SWAG
[og -iikelihood
-40
-200
100
-400
-6
-4
-2
Log-likelihood estimator (3200-16000 samples)
90% bootstrap
Figure 16: Cross-sectional predictive log-likelihoods of approximate posteriors from Figure 15. We use the estimator from Equation (6) to compute predictive log-likelihoods x The results strengthen the finding of Figure 6: the VI posterior being much heavier tailed than HMC, and Laplace and SWAG posteriors being slightly lighter tailed.
Prior
HMC
(mean std)
4
4-
2
2-
0
X
Laplace
SWAG
BALD scores
90% HPDI on mean
2.0
Posterior mean
2
1.5
Prior
>
0-
nats
1.0
NT
Laplace
-2-
-2
SWAG
0.0
3
-2
-1
3
2
3
2
Figure 17: Feature-expanded smaller network approximate posteriors trained on one-dimensional toy regression task. The input expanded into 100 random Fourier features (RFF) from the RBF kernel, inducing Gaussian Process-like predictives. Again, the HMC approximate posterior uncertainty looks sensible; VI uncertainty is somewhat reasonable but attributes more importance to observing SWAG fails to capture pointwise uncertainty in sensible way.
1. The Laplace approximation generalizes very differently from others, and
29

---

Do BNNs Actually Behave Like Bayesian Models?
J.2. Sequential Inference
Data points 0-20
Data points 20-40
Data points 40-60
-3
11
-3
-2
-1
Data points 60-80
Data points 80-100
HMC data paints 80-100
2
-2
-6
prediction
Paszerior
mean
Previously points
Figure 18: Smaller network sequential VI inference. The first step of inference seems to over-weight flat functions and not put enough weight on tighter-fitting functions, causing later inference steps (even HMC) not to fit the first group of observations.
Data points 0-20
Data points 20-40
Data points 40-60
-6
X
X
Data points 60-80
Data points 80-100
HMC on data points 80- 80-100
-3
-3
-2
-2
-1
90% HPDI mean
90% HPDI prediction
Pasterior function draws
Posterior mean
Previously seen data points
Figure Larger network sequential VT inference with RFF expansion. Similarly to Figure 18, the first step of inference seems introduce much inaccuracy for later steps fit the first half of the data particularly well.
30

---

Do BNNs Actually Behave Like Bayesian Models?
Data points 0-20
points
Data
0
-2
2
0
Data points 60-80
Data points 80-100
HMC on data points 80-100
4
-4
Posterlor
Previously
Figure 20: Small network Laplace sequential inference. The MAP is flat, and Laplace puts weights on models close to the MAP. The prior after the first step of inference is so restrictive that the subsequent MAPs stay flat too, and even HMC cannot put mass on non-flat functions. This shows complete failure of Laplace represent the posterior faithfully in this particular case.
Data points 0-20
Data points 20-40
Data points 40-60
0
2-
-3
-2
-3
-2
-2
Data points 60-80
Data points 80-100
HMC on data points 80-100
6
4
-4
HPDI
90% prediction
Posterfor function
Posterior
Previously
Figure 21: Small network Laplace sequential inference with RFF expansion. The feature expansion allows the functions be richer, and aims to generalize this demonstration to higher-dimensional settings. The small network chosen so that the MAP no longer flat under the prior induced by the network. The random Fourier features taken from the RBF kernel, hence the similarity to kernel ridge with the return-to-zero behavior. Uncertainties do not respect presence of data, and catastrophic forgetting still occurs, even though weaker than without feature expansion.
J.3. Dataset Admitting a Flat MAP
Here we demonstrate a further pathology approximate inference in our synthetic regression setup. We pick a range values from our regression data that are clustered around zero, where we may reasonably expect our posterior to mix over
31

---

Do BNNS Actually Behave Like Bayesian Models?
two classes of functions, but some of our algorithms fail to do
Prior
HMC
VI
-5
Laplace
SWAG
90%
90% HPOI en prediction
Posterior
Function (meane 5td)
-2
2
1
Figure 22: Posterior function draws on dataset admitting both flat and nonlinear fits consistent with the prior. One might reasonably expect the posterior to mix over flat functions with higher observation scale and non-linear functions with smaller observation scale. The HMC approximate posterior is mostly line with this expectation, so VI although with smaller point uncertainty, however, Laplace and SWAG approximate posteriors fail represent any non-linear fits, the local approximation potentially breaking down in ill-conditioned flat region of the parameter space.
J.4. Intermediate Posterior Forgetting Experiment with VI, and Its Dependence on
Prior
retrained
retraines
posterior detasets
S
S
HMC retrained 5 points
HMC retrained 25 points
HMC retrained 50 points
Akl
-1
22
90% mean
function draws
Previously seen points
Figure 23: Pretraining larger network with mean-field VI with tuned B 0.05. The experiment in Section 5.2 is repeated using mean-field VI to pretrain the intermediary posterior. Subsequent HMC inference is performed with this intermediate posterior as prior, observing data from shifted distribution. The initial VI posterior appears somewhat reasonable here: it puts weights on functions that match the pretraining data, but keeps some amount of uncertainty elsewhere, even though there is no substantial increase in uncertainty in between observations. As we observe new points with the intermediate posterior, observation noise variance of the likely models increases. However, after only 25 points original data fully forgotten, inconsistent with the fact that should take 50 fresh data points to shift the posterior mean zero. The single-step HMC approximation is able produce sensible posterior mean around zero.
32

---

Do BNNS Actually Behave Like Bayesian Models?
The 3 hyperparameter (see Section B.2, Equation (16)) affects the weighting of information in the VI intermediary posterior: a < 1 overweights the likelihood term compared to prior regularization through the scaled KL divergence. In the intermediary posterior, this means a / < 1 is supposed to over weighweight pretraining samples compared to the initial uninformative prior. When HMC inference is performed on this distribution as prior, one might expect the HMC posterior to favor the original pretraining observations over new data. However, in this case, they are still weighted less compared to fresh observations after HMC inference: forgetting still happens!
An even smaller value of / here corresponds to encoding the pretraining observations even more strongly into the intermediary posterior; therefore, we expect forgetting to manifest to a lesser extent. While we do observe smaller values of 3 to weight pretraining observations more, the setting of B 0.001 where the weighting is correct leads to otherwise qualitatively inaccurate, ill-regularized posterior. We have found the four HMC chains not to mix for such small values of B, hinting at ill-conditioning in the resulting VI posterior.
We saw a similar result for Laplace in Section 5.2, where despite the shrinkage of the Laplace approximation towards the MAP model (making the approximate distribution more confident on the MAP behavior, albeit without nisweighting observations and priors), forgetting also occurred. This surprising observation further demonstrates that practical VI does not respect information Bayesian model supposed to.
Prior bela=1.0
retrained on point
HMC retrained points
-1
-2
-3
-2
11
1
-3
-2
-1
2
-2
11
2
HMC retrained points
HMC retrained 25 points
HMC retroined 50 points
-2
-3
-2
-1
-2
-1
50% HPOI mean
90% HPDI on prediction
Pesterlor function draws
Posterior mean
Previsusly soen data points
Prior learnt via VI with beta=0.01
HMC retrained on point
HMC retrained on 2 points
-2
HMC retrained on 5 points
HMC retrained on 25 points
HMC retrained on 50 points
-2
22
50%
pradiction
Posterfor function
Postarior
33

---

Do BNNs Actually Behave Like Bayesian Models?
Prior learnt with beta=0.001
HMC retrained on point
HMC retrained 2 points
-1
-3
-2
-3
-2
-2
HMC retrained 5 points
HMC retrained on 25 pointss
HMC retrained on 50 points
50% HPOI on mean
90% HPOI on prodiction
Posterfor for function drows
Posterior mean
Previously seen data points
K. Model Misspecification
To underpin some of the issues in the classical Bayesian literature with suboptimal behavior and pathologies of misspecified Bayesian models, we give short review and demonstrate empirically that such pathologies can occur for BNNs, too.
Bayes is a principled scheme to update beliefs, which is optimal in the sense defined by Bissiri et al. (2016), provided the model is specified correctly. However, "all models are wrong" (Box, 1976), and in general we do not have any guarantees about what Bayes may do under misspecificatioo. In classical statistics there are many theoretical and empirical results about inconsistency and the sub-optimality of Bayes on misspecified models (Masegosa, 2020; Grünwald & van Ommen, 2017).
BNNs may also be misspecified. As Bayesian models, BNNs consist of the neural network outputs and the observation model. Even though the underlying neural network is very expressive, it will not be able to perfectly represent real- world data-generating processes, For classification, the observation model is given; therefore, we do not expect misspecification to affect classifiers much, but problems may arise, e.g., when making joint predictions. Regression settings are sensitive to the observation model, and in general, we cannot specify the true observation noise distribution, which may lead to inconsistency (Zhang & Nalisnick, 2021).
Section K.2 demonstrates a concrete example where Bayesian inference is inconsistent under misspecification.
K.1. Robustness Covariate Shift
Covariate shift in a dataset means the distribution of the covariates differs train and test sets. Even though the observation model conditionally on the inputs might be correctly specified (e.g., classification), it violates the implicit modeling assumption that train and test come from the same distribution. On real-world datasets, covariate shifts occur frequently, as the train set is often carefully curated and augmented, inputs may be noisier inference time, usage patterns of an inference system may change over time.
1.1.1. IS THE MAP MORE REGULARIZED THAN THE MODEL MODEL-AVERAGEE AVERAGED PREDICTOR?
The massive empirical success of deep learning suggests that the MAP solution in deep neural nets found by stochastic gradient methods has good generalization performance. This implicit form of regularization has been demonstrated in classical statistics too, e.g., gradient descent on linear regression leads solutions with small Euclidean norm (Gunasekar al., 2018).
As we saw in Section 2 the true BNN posterior is marginalized model through BMA. It has been argued in Wilson (2020); Wilson & Izmailov (2020) that this marginalization property is itself the case for Bayesian deep learning, making BNNs generalize well in a data-efficient way. But given the well-regularizee behavior of the MAP region of the parameter space, the question arises if averaging over models in different regions of the parameter space can lead to loss of robustness compared to the MAP.
34

---

Do BNNs Actually Behave Like Bayesian Models?
Deep ensembles, averaging over multiple MAP solutions, have often outperformed the BMA. In fact, perhaps their success could also be attributed to the favorable generalization behavior of MAPs, when compared non-MAP models with posterior mass.
1.1.2. EXISTING EVIDENCE
has been widely observed, e.g. in Izmailov et al. (2021a;b), that the Bayesian model-averaged predictive is less robust to covariate shift than the MAP. HMC is shown to significantly underperform the MAP solution on datasets like MNIST or CIFAR-I0 with corruptions introduced to the test set, even cases where HMC outperforms MAP on the task without shift.
concrete example for robustness of MAP is described in Izmailov al. (2021a). In the simplest case, consider "dead" pixels where an input pixel is always set to zero the train set. Under most current BNN priors (weights independent with mode at zero) the MAP solution will set the weights associated with this pixel to zero. However, in the Bayes posterior those weights are distributed according to their prior, therefore the BMA solution depends on the value of this pixel in test. This leads to higher variance the estimator and may lead to worse generalization.
K.1.3. SPURIOUS CORRELATIONS EXPERIMENT
To further demonstrate Bayesian model averaging leading worse robustness, we consider a regression task with spurious correlations. We consider task with standard Gaussian covariates with pairwise correlation p in train, but independent in test. The response is generated according to Y (2x,, We train BNN on this model.
In this case, all features are informative in train, but the first feature is slightly more informative than the rest. We may expect MAP to put predictive power on the most informative feature, and the Bayes posterior to average over models picking different predictive features too.
Table 1: Mean squared errors on spurious correlation task. The use of Laplace prior, equivalent to lasso regularization in linear models, induces sparsity: MAP selects the optimal model, picking the first feature only, whereas HMC averages over the optimal and suboptimal models. Under Gaussian prior, both solutions pick many features instead sparsity the solutions. This also demonstrates the strong, and sometimes counter-intuitive effects that changing to a seemingly similar prior can have on the inference process, highlighting that simple weight-space priors do not capture our prior beliefs well. These results generalize to Bayesian linear regression and other BNN architectures.
LAPLACE PRIOR
GAUSSIAN PRIOR
MAP HMC
0.0046
0.9059
0.1479
0.9045
This experiment demonstrates that the MAP may indeed be more regular, leading to better generalization performance, and converging the optimal model faster. On the other hand, one might argue that the BNN prior was capture our beliefs, the Bayes estimator is decision theoretically optimal under the L loss, so its behavior is reasonable. However, the MSE on the shifted dataset is in effect different loss function. We argue, with today's BNN priors that cannot be said to capture beliefs well, one should ask which behavior leads to better practical performance in real use cases.
K.2. Classical Misspecification Experiment: Does Posterior Get Better with More Data?
The Bernstein-von Mises theorem states that the (well-speciffed) Bayesian posterior asymptotically collapses to the true model. If the true model is, in fact, in the range of candidate models, this consistency property is, of course, desirable.
If the observation model is misspecified, under some conditions the Bernstein-von Mises theorem still holds (Bochkina, 2019), but this collapse leads to inconsistency, preventing the inference scheme from converging to a predictive distribution that matches the true distribution the closest (e.g., KI. divergence).
35

---

Do BNNs Actually Behave Like Bayesian Models?
1.4
KL Divergence
1.2
1.0
posterior predictive from true distribution Asymptotic posterior KL
0.8
12
Number of training datapoints
Figure 24: Divergence of misspecified Bayesian posterior predictive from true generating distribution versus number of observations. Performance of Bayesian inference degrades as more data is observed under a misspecified model. As the Bayes posterior collapses to single model, the posterior predictive gets further and further away from the true -ata-genn distribution as measured by KL divergence.
To demonstrate the inconsistency of Bayesian inference on a misspecified model, we take the following task. The true data generating distribution is coin flip between and 1, and we model it as Y IIu~ [i,, o²) with fixed and μ ~ N0, 1).
In this case, the distribution j Bern([-1, 1}) minimizes the KL divergence between the data generating distribution and the posterior predictive distribution. However, simple caleulation shows that the Bayes posterior collapses to μ 0, supported by the simulation in Figure 24 too.
1
This simple experiment demonstrates that Bayesian inference does not find the optimal distribution over parameters that captures the true distribution the best. the case misspecification, we might become overconfident in a badly fitting model. This is worth keeping in mind BNNs too, especially for regression, where inference is sensitive to the choice of observation model.
References
activatedgeek. Optax SWAG, 2023. URL
Betancourt, M. A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint arXiv: 2017.
Bingham, E., Chen, J. P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T., Singh, R., Szerlip. P. A., Horsfall, P., and Goodman, N. D. Pyro: Deep universal probabilistic programming. J. Mach. Learn. Res., 20:28:1-28:6. 2019.
Bissiri, P. G., Holmes, C. C.,, and Walker, S. G. general framework for updating belief distributions. Journal the Royal
Statistical Society. Series B (Statistical Methodology), 78(5):1 2016.
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American 2017.
statistical Association,
Blundell, C., Comebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1622, Lille, France, 07-09 Jul 2015. PMLR.
Bochkina, N. Bernstein-von Mises theorem and misspecified models: review. Foundations of Modern Statistics, pp.
385- 2019.
Box, G. E. Science and statistics. Journal of the American Statistical Association, 71(356);791-799, 1976.
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne S., and Zhang, Q. JAX: composable transformations of Python+NumPP programs, 2018. URL http:://ittin whw.b. com/ www/googll/ooooooooooooooooooooo google/////oooooooooooooooooooeo / jax.
36

---

Do BNNS Actually Behave Like Bayesian Models?
Chen, T., Fox, E., and Guestrin, C. Stochastic gradient Hamiltonian Monte Carlo. In International Conference on Machine Learning, pp. 1683-1691. PMLR, 2014.
D'Angelo, F. and Fortuin, Repulsive deep ensembles are Bayesian. Advances in Neural Information Processing Systems, 34:3451-3465, 2021.
Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., and Hennig, P. Laplace redux effortless Bayesian deep learning. In Advances in Neural Information Processing Systems, volume 34, pp. 20089-20103, Curran Associates, Inc., 2021.
Detlefsen, N. S., Jørgensen, M., and Hauberg, S. Reliable training and estimation of variance networks. In Advances in Neural Information Processing Systems, pp. 6326-6336, 2019.
Doob, J. Application of the theory of martingales. Actes du Colloque International Le Calcul des Probabilités et ses applications, pp. 23-27, 1949.
Falck, F., Wang, Z., and Holmes, C. in-context learning in large language models Bayesian? A martingale perspective. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of ICML'224 24, pp. 12784-12805, Vienna, Austria, July 2024. JMLR.org.
Farquhar, S., Osborne, M. A., and Gal, Y. Radial Bayesian neural networks: Beyond discrete support in large-scale Bayesian deep learning. In International Conference on Artificial Intelligence and Statistics, pp. 1352-1362. PMLR, 2020a.
Farguhar, S., Smith, L., and Gal, Y. Liberty depth: Deep Bayesian neural nets do not need complex weight posterior approximations, Advances in Neural Information Processing Systems, 33:4346-4357, 2020b.
Fong, E., Holmes, C., and Walker, S. G. Martingale posterior distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology. (355):335-11355 2023.
Fortuin, V. Priors in bayesian deep learning: A review. International Statistical Review, 90(3):563-591, 2022.
Gal, Y. and Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMIRR 2016.
Graves, A. Practical variational inference for neural networks. Advances in neural information processing systems, 24, 201
Grünwald, P. and van Ommen, T. Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing Bayesian Analysis, 12(4):1069- 1103, 2017. doi: 10. (0.1214/17-BA185.
Gunasekar, S., Lee, J., Soudry, D., and Srebro, N. Characterizing implicit bias in terms of optimization geometry. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1832-1841, PMLR, 10-15 Jul 2018.
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International conference machine learning, pp. 1321-1330. PMLR, 2017.
He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034 2015.
Higgins, I., Matthey, I.., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta-vae: Learning basic visual concepts with constrained variational framework. In International conference learning representations, 2016.
Hinton, E. and Van Camp, D. Keeping the neural networks simple by minimizing the description length of the weights.
In Proceedings of the sixth annual conference on Computational learning theory. pp. 5-13, 1993.
Hirshleifer, J. and Riley, J. G. The analytics of uncertainty and information. Cambridge University Press, 1992.
Hoffman, M. D., Gelman, A., et al. The no-Unoo-n U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 151):119331163, 2014.
37

---

Do BNNS Actually Behave Like Bayesian Models?
Izmailov, P., Nicholson, P., Lotfi, S., and Wilson, A. G. Dangers of Bayesian model averaging under covariate shift, Advances in Neural Information Processing Systems, 34:3309-3322 2021a.
Izmailov, P., Vikram, S., Hoffman, M. D., and Wilson, A. G. What are Bayesian Neural Network posteriors really like? In International Conference on Machine Learning, pp. 4629-4640, PMLR, 2021b.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
Lakshminarayanan. B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep
ensembles. Advances neural information processing systems, 30, 2017.
Laplace, P. S. Memoir on the probability of the causes of events. tome sixième. Mémoires de Mathématique et de Physique (English translation by SM Stigler 1986. Statist. Sci., 1 (19): 364-378), 1774.
MacKay, D. Bayesian interpolation. Neural computation, 4(3):415-447, 1992a.
MacKay, J. A practical Bayesian framework for backpropagation networks. Neural computation, 4(3):448477,, 1992b.
Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., and Wilson, A. G. A simple baseline for Bayesian uncertainty in deep learning. Advances in neural information processing systems, 32, 2019.
Martens, and Grosse, R. Optimizing neural networks with Kronecker- approximate curvature. In International conference on machine learning, pp. 2408-2417. PMLR, 2015.
Masegosa, A. Learning under model misspecification: Applications to variational and ensemble methods. In Advances in Neural Information Processing Systems, volume 33, pp. 5479-5491. Curran Associates, Inc., 2020.
Mlodozeniec, B., Krueger, D., and Turner, Implicitly Bayesian prediction rules in deep learning. 6th Symposium on Advances in Approximate Bayesian Inference, 2024.
Murphy, K. P. Machine learning: a probabillisiic perspective. MIT press, 2012.
Neal, R. M. al. MCMC using Hamiltonian dynamics. Handbook of Markov chain Monte Carlo, 2(11):2, 2011.
Osband, I,, Wen, Z.,, Asghari, S. M., Dwaracherla, V., Ibrahimi, M., Lu, X., and Van Roy, B. Epistemic neural networks. In Advances in Neural Information Processing Systems, volume 36, 2023.
Phan, D., Pradhan, N., and Jankowiak, M. Composable effects for flexible and accelerated probabilistic programming in numpyro. arXiv preprint arXiv:1912.11553 2019.
Robert, C. P., Casella, G., and Casella, G. Monte Carlo statistical methods, volume 2. Springer, 1999.
Roberts, G. O. and Rosenthal, J. S. General state space Markov chains and MCMC algorithms. Probability Surveys, 1:20 71, 2004.
Schnaus, D., Lee, J., Cremers, D., and Triebel, R. Learning expressive priors for generalization and uncertainty estimation in neural networks. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 50252-30284 PMLR, 7 2023.
Shwartz-Ziv, R., Goldblum, M., Souri, H., Kapoor, S., Zhu, C., LeCun, Y., and Wilson, A. G. Pre-train your loss: Easy Bayesian transfer learning with informative priors. Advances Neural Information Processing Systems, volume 35, pp. 27706-27715. Curran Associates, Inc., 2022.
Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning, pp. 681-688, 2011.
Wenzel, F. Roth, K., Veeling, B. S., Swiatkowski, J., Tran, L., Mandt, S., Snoek, J., Salimans, T., Jenatton, R., and Nowozin. S. How good is the Bayes posterior in deep neural networks really? In Proceedings of the 37th International Conference on Machine Learning, pp. (0248-10255) 2020.
Williams, C. K. and Rasmussen, C. E. Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
38

---

Do BNNS Actually Behave Like Bayesian Models?
Wilson, A. G. The case for Bayesian deep learning. arXiv preprint arXiv:2001.10999, 2020.
Wilson, A. G. and Izmailov, P. Bayesian deep learning and probabilistic perspective of generalization. Advances in neural information processing systems, 33:4697-4708, 2020.
Wingate, D. and Weber, T. Automated variational inference in probabilistic programming. arXiv preprint 13Xiv 2013.
Zhang, R., Li, C., Zhang, J., Chen, C., and Wilson, A. G. Cyclical stochastic gradient MCMC for Bayesian deep learning. In International Conference on Learning Representations, 2020.
Zhang, Y. and Nalisnick, On the inconsistency of Bayesian inference for misspecified neural networks. In Third Symposium on Advances Approximate Bayesian Inference, 2021.
39


Reviews:
[{"review_id": "SdK3W5FAoq", "reviewer": "Reviewer_Tqfi", "summary": "The paper empirically investigates how well popular approximate inference algorithms for BNNs respect the theoretical properties of Bayesian belief updating. \n\nThe study tries to examine whether different Bayesian neural network (BNN) posterior approximations adequately capture epistemic uncertainty by analyzing their behavior in both parameter-space i.e. weights and function-space i.e. predictive distributions. Experiments reveal HMC preserves richer functional diversity in high-uncertainty zones, while variational and other approximate methods oversimplify predictions despite training data constraints.\n\nThe work is good summary of the insights into the Bayesian inference algorithms. But the studies presented here are broadly understood by the community. The tradeoffs of the different approximate Bayesian inference approaches are well studied.  \n\nLacks novelty.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your work in reviewing our paper. However, we believe there may have been some significant misunderstandings about our work and we are a little puzzled by both your summary of our work and the conclusions of your review, neither of which reflect our actual contributions.\n\nFor instance, the review claims the paper studies whether “different BNN posterior approximations adequately capture epistemic uncertainty by analyzing their behavior in both parameter-space i.e. weights and function-space i.e. predictive distributions”. However, our work is not about comparing how different posterior approximations capture epistemic uncertainty and we do not have any analysis or experimentation at all about parameter-space behavior.\n\nThe tradeoff between various approximate inference techniques in terms of their static predictive performance is indeed already well understood, as we acknowledge in the paper itself.  As explained in, for example, the penultimate paragraph of our introduction, our work is instead about critically examining how well these BNN algorithms adhere to various important theoretical principles of Bayesian belief updating in dynamic settings, which are not in themselves often used to evaluate inference schemes. This goes beyond simply assessing predictive performance or understanding computational and expressivity trade-offs between different approximate inference schemes. The work shifts the focus to systematically examining the behavior of Bayesian updates, looking at functional consistency, coherency of sequential belief updates, and the self-consistency of predictive uncertainty. \n\nOther reviewers recognize this perspective as significant empirical contributions not yet widely established within the community.  We would therefore like to ask you to please reconsider your assessment.", "claims_and_evidence": "-", "methods_and_evaluation_criteria": "-", "theoretical_claims": "-", "experimental_designs_or_analyses": "-", "relation_to_broader_scientific_literature": "These findings are broadly understood. Lacks novelty.", "essential_references_not_discussed": "-", "other_strengths_and_weaknesses": "-", "other_comments_or_suggestions": "-", "questions_for_authors": "-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AX3NamSHfo", "reviewer": "Reviewer_8qCC", "summary": "The paper investigates the alignment of Bayesian neural networks wrt rigorous Bayesian principles/ideals. To do so, tasks like synthetic regression and classification on CIFAR datasets are considered. The main claimed findings are focused on 1) the lack of \"functional consistency\" shown by approximate posteriors (i.e. MFVI or Laplace approximations for BNNs), 2) the issues to sequentially propagate uncertainty in a rigorous way and 3) the ill-calibrated uncertainty of the posterior predictive density models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review and helpful feedback! We are glad you found our paper interesting, and hope that the responses and new results presented below help increase your confidence in backing acceptance of the paper.\n> I don't really understand why references on Bayesian inference […] are focused on Robins et al. and other citations of this sort [...]\n\nThank you for pointing this out. We agree and will update the references to some of the originals (e.g. [1,2]).\n> The BMA in Appendix A looks a bit trivial to me [...]\n\nWe appreciate this appendix indeed presents background that might appear elementary. It introduces terminology and makes our work accessible by detailing common concepts “often underexplained in Bayesian deep learning papers”, quoting reviewer KcGY. We will add a paragraph to better position this appendix. Thanks for highlighting it!\n> Why in Section 4, does it come back to the functional posterior predictive perspective? I don't see the point/utility here.\n\nWe think this is worth reiterating here. Pointwise uncertainty quantification (UQ) and BNNs improving UQ over SGD are of course well studied; the novel investigation here is to evaluate BNNs in dynamic settings. Indeed, VI, for example, produces good static results and uncertainty but its functional and uncertainty calibration properties deviate from Bayesian behaviors.\n> I am concerned about the assumption of heteroscedastic noise modelling in Sec. 4 [...]  I'm a bit afraid that the use of such networks is somehow deteriorating the correctness of conclusions obtained from the empirical results there.\n\nThank you for pointing this out! It is true that the choice of observation model may have a significant impact on the performance and behavior of Bayesian models, and agree this is important to investigate. We highlight that there are relevant ablations already in our appendix (in Appendix F we examine model misspecification and in Appendix D we do a number of ablations on network architecture).\nTo show, however, that our empirical findings are not merely an artefact of the difficulty in heteroskedastic noise calibration, __we have repeated our experiment from Figure 3 with homoskedastic noise model__: [see here](https://raw.githubusercontent.com/anon17438/anon-rebuttal-material/main/rebuttal_homoscedastic_figure_3.pdf). We run inference with both a fixed noise variance, and placing an inverse Gamma hyper-prior on it for a fully Bayesian treatment. In both cases, the results agree with the conclusions of Figure 3.\n> How difficult/easy is satisfying that $\\theta_N$ is a martingale? \n\n$\\theta_N$ is always a martingale under proper Bayesian updating regardless of architecture (subject to some very weak assumptions about the posterior existing). It will typically no longer be a perfect martingale under approximate updates, which is why we are using this as a test for adherence to Bayesian principles. We will add more high level discussion on this to the section to make it clearer.\n>...[Section 6] becomes somehow unclear due to the lack of space. The analysis for Fig 6 and 7 is super important, but results are presented in a quick manner with many details poorly discussed…\n\nThank you for this important feedback. We do agree the clarity has suffered here from needing to get things in length. We plan to use most of the extra page available for the camera ready to expand on and improve the clarity of this section. In particular, we plan to add more high-level intuition and explanation detail throughout, further interpret the results, and add more insights on the MP behavior itself. We have also repeated the experiments from Figures 7 and 8 with additional dataset, as discussed in the response to Reviewer KcGY. Please let us know if there is anything further you feel we should add.\n> The way the experiment is designed looks correct to me, but for instance, the intermediary pi’-predictive densities used to sample are also unclear (or I don't have a taste on how to obtain these in a reproducible way), and how the empirical MP is computed in L412 also doesn't give me a good feeling if applied to different BNN models.\n\nThe $\\pi’$ distributions correspond to an initial trained BNN posterior. Our imputation follows the predictive resampling step from the original MP paper. We first draw a random setting of weights from the posterior $\\Theta\\sim\\pi’(\\theta)$, then generate a label from the predicted class probabilities $f_\\Theta(x^*)$ in the case of classification (but it generalizes). For reproducibility, in our experiments we have taken sequentially increasing seeds. We will make updates to ensure this is clear.\n\nThank you again for your insightful review which has really helped us to improve the paper.  Let us know if you have any further questions or concerns.\n\n-----------\n\n[1] Jeffreys, H. (1939). _Theory of Probability_. Clarendon Press.\n\n[2] Savage, L. J. (1972). _The foundations of statistics_. Courier Corporation.", "claims_and_evidence": "The vast majority of claims and ideas introduced are well-supported with evidence from empirical results, technical derivations or well-referenced context. However, I would like to raise some points here that I thought could be imprecise or just odd from my point of view:\n\n- I don't really understand why references on Bayesian inference (for example @ L85) are focused on Robins et al. and other citations of this sort. Weren't the Bayesian ideals mentioned a contribution of other authors and papers produced long before those ones?\n- The BMA in Appendix A looks a bit trivial to me, or at least another way of technically rewriting standard Bayesian quantities over probabilistic densities. I am aware of the existence of BMA and its use, but here, it didn't give me the right feeling. I could be wrong on this, but that was what came to my mind when reading it..", "methods_and_evaluation_criteria": "Similar thoughts as in the previous section on claims and evidence. I do think the paper is on a good direction, exploring a super interesting topic (which, more or less, many in the probabilistic/Bayesian community already suspected for a long long time). The methods are in general well developed, thorough and supported by good evaluation criteria, all mixed with a high scientific spirit imo. \n\nSome additional comments and concerns from the methodology part:\n- Why in Section 4, does it come back to the functional posterior predictive perspective? I don't see the point/utility here.\n- I am concerned about the assumption of heteroscedastic noise modelling in Sec. 4 being modelled with a $\\sigma^{2}_{\\theta}(x)$ NN. I think there is more or less a consensus in the probabilistic ML community that this is not a great idea, as it fails to really produce well-calibrated uncertainties out of the regions with high density of datapoints. (See for instance Detlefsen et al. NeurIPS 2019). Therefore, I'm a bit afraid that the use of such networks is somehow deteriorating the correctness of conclusions obtained from the empirical results there..\n\n---\nDetlefsen et al. NeurIPS (2019) /  \"Reliable training and estimation of variance networks\"", "theoretical_claims": "Just one question on the Martingale assumption from Theorem 6.1, used also later in Proposition 6.1. How difficult/easy is satisfying that $\\theta_{N}$ is a martingale? I don't really have a feeling here from the NN parameters of different architectures. Could the authors add a bit more on this?", "experimental_designs_or_analyses": "I do like the way experiments support the main 3 points/parts of the findings, that's great! However, to me the most interesting one of the three parts is for sure the last one from Section 6. To me, that becomes somehow unclear due to the lack of space. The analysis for Fig 6 and 7 is super important, but results are presented in a quick manner with many details poorly discussed... \n\nThe way the experiment is designed looks correct to me, but for instance, the intermediary $\\pi'$-predictive densities used to sample are also unclear (or I don't have a taste on how to obtain these in a reproducible way), and how the empirical MP is computed in L412 also doesn't give me a good feeling if applied to different BNN models.", "relation_to_broader_scientific_literature": "Apart from my comments made in the first sections, I don't miss anything special here or at least I'm not aware of missing literature/references. The relation and connection with the ideas discussed around in the BNN community are also right to me.", "essential_references_not_discussed": "Idem to prev. review section", "other_strengths_and_weaknesses": "NC", "other_comments_or_suggestions": "NC", "questions_for_authors": "I do think the paper is in a great direction and it has key values to be worth for acceptance. Due to I still have my doubts and concerns about some parts and points, I do recommend weak acceptance at this point, but I would be also happy to vote for clear acceptance of the manuscript if the rest of the reviewers agree and there are no missing problems that I accidentally ignored.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6JHZ8KHpBI", "reviewer": "Reviewer_yeeB", "summary": "This paper investigates the properties of several popular algorithms for approximate posterior inference in Bayesian neural networks (BNNs). The main experimental findings are that common approximate inference algorithms in BNNs: (a) do not exhibit functional consistency of posteriors, (b) do not propgate information coherently when performing sequential inference, and (c) do not exhibit predictive coherence.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review and helpful feedback! We hope that the responses and new results below answer your questions and alleviate any remaining concerns.\n> In a set of experiments like this, it would be nice to have a small version where exact inference is possible. This would help clarify which effects are due to HMC being treated as the gold standard and what is inherent to Bayesian inference itself.\n\nGreat suggestion! To provide this, __we have performed an extra experiment with Bayesian linear regression__: [see here](https://raw.githubusercontent.com/anon17438/anon-rebuttal-material/main/rebuttal_exact_inference_experiment.pdf). To keep higher input dimensionality, we expand $x$ into 20 random Fourier features with a squared exponential kernel, approximating a (sinusoidal) Gaussian process regression.\nExact inference demonstrates the desired coherent behavior. MFVI updating still shows coherence, despite its apparent lack of faithful posterior representation. For completeness, we have run all algorithms in the single-step setting and confirmed that HMC matches the true posterior. For this classical Bayesian setup, even our inexact approximate inference schemes exhibit Bayesian properties. This contrasts the BNN behavior we find at scale. We will expand on this running example to demonstrate the expected behavior with exact Bayesian updating through the paper.\n> The paper demonstrates several issues with approximate inference algorithms for BNNs but does not attempt to provide theoretical understanding of why these issues occur or what can be done to remedy them.\n\nThe primary objective of our paper is to highlight the discrepancy between the theoretical expectations and the actual behavior of BNNs. We thus believe it is natural that we focus on experimental evaluation rather than new theory. The theoretical reason for the issues is arguably quite straightforward: inexact inference causes substantial discrepancy in behavior from the true posterior. One could try to do theoretical analysis on this gap itself, but it would likely be challenging and prone to the same theory-practice gap we are investigating in the first place; we feel this is beyond the scope of what can be achieved in the current work.\n\nIn terms of remedies, these extremely challenging and fundamental issues at the core of the BNN methodology cannot feasibly be solved in a single paper. We see our work as the first step on this journey, highlighting and carefully documenting the issues, but we expect it to be many years before the community has managed to remedy them, if indeed they ever do.\n> Discussion about key takeaways is also relatively light…\n\nThank you for this feedback. We will exploit the extra page available for the camera ready to add more discussion on key takeaways. On the specific questions you raise:\n> Should practitioners avoid BNNs completely? Should they only be used in certain settings?\n\nDefinitely not! They have been shown to outperform point estimates and provide effective static uncertainty in multiple settings, which our results do not undermine. We simply argue that care is needed in our expectations and interpretation, especially outside these prediction settings where the pathologies we discuss might occur. In particular, we need to be very careful about empirical evaluation, as good static performance may not translate to other desired behaviors.\n> Is more research needed to develop inference schemes that respect e.g. coherence in sequential inference?\n\nWe would indeed argue that more research is needed in this direction. Moreover, in our opinion this should not be thought of as simply an inference scheme problem: entirely not Bayesian methods might in practice give more coherence under sequential inference. Even though the problems we raised are a direct result of inexact inference, improving inference schemes is not necessarily the solution: accurate inference may be unachievable, and strategies focusing on final behavior could be more effective. In short, this remains a highly unsolved problem with many possible future research directions.\n\nOther questions we will discuss are _Why does it matter that BNNs are not truly Bayesian?_ and _If BNNs are not Bayesian, what are they?_.  Let us know if there are any other suggestions.\n> p. 3, l. 156: Are the sizes listed correctly here? 26 seems like a typo.\n\nWell caught! Indeed this should say 256.\n> Are there any cases where the uncovered pathologies are actually beneficial?\n\nThere is never likely to be any direct benefit in the inconsistent behavior itself, but the deviations from the “correct” posterior may potentially sometimes be beneficial from a static prediction perspective. For instance, [1] found the calibration of VI to be consistently better than more faithful Bayesian approximations such as HMC.\n\nThank you again for your review and helpful suggestions!\n\n[1] Izmailov et. al. (2021). _What are Bayesian Neural Network posteriors really like?_ ICML", "claims_and_evidence": "Each of the three experimental findings are supported by evidence.", "methods_and_evaluation_criteria": "There are a mix of toy and larger-scale experiments, which is sensible to investigate the questions being asked.", "theoretical_claims": "There are no novel theoretical claims made in the paper.", "experimental_designs_or_analyses": "The experimental analyses seem reasonable, although I did not fully understand the martingale posterior experiment.", "relation_to_broader_scientific_literature": "The contributions are positioned to explain some folklore knowledge within the Bayesian deep learning community. It is well known that Bayesian methods are sound for models with small numbers of parameters. However, direct application of Bayesian methods to high-dimensional deep neural networks is fraught with practical issues. These issues are known well enough by the community to be explored in the NeurIPS workshop \"I Can't Believe It's Not Better (ICBINB)\". This paper provides a more systematic study of some of these issues, and offers concrete evidence of the shortcomings of current approximate posterior inference methods.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "I have already described above the main strength, which is to shed light on some problematic issues with approximate posterior inference in BNNs. Soundness and support for claims are also discussed above, and the writing is largely quite coherent.\n\nThere are also few weaknesses that deserve mention:\n- In a set of experiments like this, it would be nice to have a small version where exact inference is possible. This would help clarify which effects are due to HMC being treated as the gold standard and what is inherent to Bayesian inference itself.\n- The paper demonstrates several issues with approximate inference algorithms for BNNs but does not attempt to provide theoretical understanding of why these issues occur or what can be done to remedy them.\n- Discussion about key takeaways is also relatively light: should practitioners avoid BNNs completely? Should they be used only in certain contexts? Is more research needed to develop inference schemes that respect e.g. coherence in sequential inference?", "other_comments_or_suggestions": "- p. 2, ll. 58-62: Section 5.2 is listed twice.\n- p. 3, l. 145 (second column): What does BMA stand for?\n- p. 3, l. 151 (second column): It may help the flow to discuss applications of BNNs earlier, in Section 2.\n- p. 3, l. 156: Are the sizes listed correctly here? 26 seems like a typo.\n- p. 4, l. 190: Has $\\beta$ been defined at this point in the paper?", "questions_for_authors": "In addition to the weaknesses raised above: are there any cases where the uncovered pathologies are actually beneficial? One person's incorrect posterior is another person's correct posterior with a different prior. Might there be a case where narrowing down the functions as done by VI, Laplace, and SWAG in Figure 1 can actually help?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QQjPPTnO3E", "reviewer": "Reviewer_KcGY", "summary": "The paper **\"Do Bayesian Neural Networks Actually Behave Like Bayesian Models?\"** investigates whether common **approximate inference algorithms** for Bayesian Neural Networks (BNNs) adhere to the theoretical principles of Bayesian belief updating. It empirically evaluates methods such as **Variational Inference (VI), Laplace Approximation, SWAG, and Stochastic Gradient Langevin Dynamics (SGLD)** on synthetic regression and real-world classification tasks.\n\nThe key findings indicate that:\n\n### **1. Common BNN Algorithms Lack Functional Variability**\nLet $ \\theta_s, s=1,\\ldots, S $ be a collection of posterior samples from a common BNN algorithm such as Laplace, VI, HMC, etc. We can visualize the function draws $ f_{\\theta_s}(x) $ in simple regression settings (e.g., scalar $ x $ and $ y $) by plotting them. Section 4.1 presents evidence that many common BNN methods fail to capture **meaningful uncertainty**, particularly in regions of $ x $ with **few observations**. This suggests that approximate BNN posteriors tend to **over-collapse**, producing function samples that lack diversity and fail to reflect the full range of plausible models. This phenomenon is particularly problematic in **extrapolation regions**, where true Bayesian inference should exhibit greater uncertainty. \n\n### **2. Common BNN Algorithms Forget Past Information in Sequential Learning**\nIn sequential Bayesian updating, the posterior after seeing two datasets sequentially should match the single-step posterior:\n\n$$\n\\pi(\\theta | D_1) \\quad \\text{then updating with} \\quad \\pi(\\theta | D_2) \\quad \\text{should yield} \\quad \\pi(\\theta | D_1 \\cup D_2)\n$$\n\nInstead, common approximate inference methods exhibit **forgetting** as shown in Section 5.\n\n### **3. Common BNN Algorithms Violate Bayesian Predictive Coherence**\nAgain let $ \\theta_s, s=1,\\ldots, S $ be a collection of posterior samples from a common BNN algorithm such as Laplace, VI, HMC, etc. We can use this collection to form a predictive density with which we impute future data. Then on this augmented dataset (original plus imputed) we can reapply the same BNN algorithm to get a new posterior sample. Repeating this imputation $ N $ times, we can get $ N $ new posterior samples. These new samples should remain consistent with the original ones in the sense that metrics such as test accuracy, log-likelihood, and expected calibration error (ECE) should be preserved. However, Section 6 presents evidence that these metrics shift significantly after imputation, which the authors consider to be evidence for predictive incoherence.\n\n## update after rebuttal\nI increased my score from 3 to 4 and maintain a positive impression of the paper. The rebuttal was handled well by the authors. I am not convinced by the review that gave a 1 and have decided to increase my score to help the paper's chance of acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review and helpful feedback! We are glad you found our paper “engaging” and “well argued”. We hope our responses and new experiments below answer your questions and alleviate the concerns you raised.\n> The proposed alternative (martingale posteriors) is presented in a somewhat preliminary manner ...it remains unclear whether it truly resolves the issues identified with standard BNN algorithms.\n\nWhile we agree that the paper could be improved by expanding on martingale posterior (MP) evaluations and will present new results on this shortly, we first wanted to discuss its role within our overall contributions. Our primary aim, as you eloquently explain, is to investigate the adherence of BNNs to the theoretical principles of Bayesian updating. Predictive coherence (Section 6) is one of these key principles. The MP framework is a novel tool we propose for this evaluation, which then naturally leads to our experiments in Figures 7 and 8 intended to identify the lack of coherence in existing approaches.\n\nIt is only after we have conducted these tests that the avenue of using MPs as a mechanism itself to potentially combat the problem becomes apparent. Given our original aims and the large amount of other content, we thus feel that fully exploring this new idea is not feasible within the scope of the current paper. Instead, we see this as a very exciting avenue for future investigation, adding significance to our work.\n\nNonetheless, we do plan to significantly expand on our discussion of the MP using the extra page in the camera ready (see also response to Reviewer 8qCC) and an exposition in the appendix for ML audiences, as you very rightly suggest.\n> The claim that [MPs] could be a worthwhile pursuit could be further substantiated …, further experiments for the [MP] could strengthen the paper’s concluding claim that MPs are worthy to pursue in Bayesian deep learning…\n\nTo provide more evidence for the generality of the MP findings, __we have repeated our CIFAR experiments on the IMDB dataset__: [see here](https://raw.githubusercontent.com/anon17438/anon-rebuttal-material/main/rebuttal_martingale_posterior_imdb.pdf). The VI results are almost identical, strengthening the argument that the MP can provide performance gains in this setting. For SGLD, while the performance comparison of the MP is inconclusive, we still observe significant incoherence in uncertainty calibration, which this experiment was originally set up to test.\n> It would be valuable to compare [MPs] to the broader literature on alternative Bayesian approximations and uncertainty estimation techniques.\n\nThank you for raising this. We will add more discussions and references to the broader literature in Section 3 including non-standard “generalized” updates to Bayes’ rule (we assume this is what you mean by “alternative Bayesian approximations\"), Gibbs posteriors, PAC-Bayesian methods, and non-Bayesian approaches like epistemic neural networks. Please let us know anything else you think would be important to discuss here.\n## Questions For Authors\n> 1. The introduction states that BNNs “offer better predictive performance…than deterministic networks in multiple settings.” How do these claims hold up against deep ensembles?\n\nDeep ensembles (DE) have also indeed been shown to offer better performance than deterministic networks, and we actively avoid taking a position on the relative superiority of BNNs and DE. Assessing non-Bayesian uncertainty quantification methods like DE under our Bayesian lens would certainly be an interesting future investigation.\n> 2. The [MP] is defined without any conditions on the belief updating scheme … does this framework require specific assumptions to guarantee desirable properties?\n\nWe refer back to our initial discussion that the desirable property of MPs for the purpose of our experiment is to recover the initial posterior. Typically, they will not do so under approximate inference.\n> 3. In the CIFAR-10 experiment, why does using the intermediary $\\pi’$-predictive to impute labels lead to a well-defined [MP]?\n\nThe regularity assumptions needed for well-defined MPs are generally weak, requiring that the prior and updated belief densities exist, and BMA has finite expectation, clearly satisfied for classification where it is bounded. We will expand on this.\n> 4. I’m not sure my reading of the CIFAR-10 experiment is correct.\n\nThe summary is correct. We exploit the property that sampling from $\\Theta\\sim\\pi(\\theta|y)$ is equivalent to imputing a random observation $Y'\\sim p(y’|y)$ and then drawing the posterior sample $\\Theta\\sim\\pi(\\theta|y,y')$. Our experiments find that the samples obtained this way deviate from the original distribution in their BMA, as captured by our evaluation metrics.\n\nWe apologize we do not have the space to address all minor comments, but we will incorporate them into the paper.\n\nAgain, thank you for the review. Please let us know of any further questions or concerns.", "claims_and_evidence": "Yes. The main claims—that common BNN algorithms fail to behave in a strictly Bayesian manner—are supported by well-designed empirical analysis. \n\nThe claim that martingale posteriors could be a worthwhile pursuit could be further substantiated however.", "methods_and_evaluation_criteria": "There are multiple ways in which BNN algorithms may fail to be strictly Bayesian. This paper examines a subset of these properties, focusing on functional variability, forgetting past data, and predictive coherence. The choices are reasonable, as they directly impact certain applications. While the experiments use only synthetic data for regression and CIFAR-10 for classification, this is sufficient to support the claims, as the findings are expected to generalize.", "theoretical_claims": "The paper primarily focuses on empirical analysis, with limited theoretical contributions. Proposition 6.1 formalizes the predictive coherence of Bayesian inference using martingale posteriors. The proof appears correct.", "experimental_designs_or_analyses": "The experiments span synthetic regression and CIFAR-10 classification to evaluate whether common BNN algorithms exhibit Bayesian properties. They convincingly demonstrate that approximate inference algorithms fail certain Bayesian properties. The empirical design is sound, and additional experiments would likely reinforce rather than alter the conclusions.\n\nHowever, further experiments  for the martingale posterior could strengthen the paper’s concluding claim that MPs are worthy to pursue in Bayesian deep learning.", "relation_to_broader_scientific_literature": "The paper contributes to ongoing discussions in Bayesian deep learning, where much of the focus has been on improving inference algorithms for better predictive performance and uncertainty quantification. While prior works have acknowledged that approximate Bayesian inference does not strictly adhere to Bayesian principles, this paper systematically documents key deviations. The use of martingale posteriors as a potential fix is intriguing but underdeveloped. It would be valuable to compare this to the broader literature on alternative Bayesian approximations and uncertainty estimation techniques.", "essential_references_not_discussed": "I did not identify any crucial references missing from the discussion. The paper effectively situates its contributions within the existing Bayesian deep learning literature.", "other_strengths_and_weaknesses": "I found this paper well-argued and engaging. It articulates a viewpoint that aligns with my own perspective—that we should not expect BNNs to be strictly Bayesian but instead view them as practical probabilistic models. However, while the paper critiques existing BNN methods effectively, the proposed alternative (martingale posteriors) is presented in a somewhat preliminary manner. The conditions required for the martingale posterior to be well-defined are not fully discussed, and it remains unclear whether it truly resolves the issues identified with standard BNN algorithms.", "other_comments_or_suggestions": "### Minor comments:\n- This paper honestly reads more like a position piece than a purely technical contribution. The only technical contribution is the MP part which is far too brief and preliminary.\n- Figure 8 caption: \"Their martingale posterior over-reduces confidence\"—the phrase \"over-reduces\" is awkward. Consider rewording for clarity.\n- Line 352: \"a belief updating scheme $ P(\\theta) \\mapsto P_y(\\theta) $.\" Since this convention is only used in the context of $ \\pi $, why not define it directly in terms of $ \\pi $ rather than $ P $?\n- The notation for the equation in Line 411 confuses me. On the left-hand side, I don’t understand why there is a hat on $ \\pi $.\n- Would it be possible to explicitly write out reason for the last equality in the proof of Proposition 6.1?\n- In the abstract, the key finding is written as “common BNN algorithms…fail to update in a consistent manner, forget about old data under sequential updates, and violate the predictive coherence properties…” Then later, starting around the last sentence on page 1, it is written that these common BNN algorithms 1) “lack the functional consistency of the Bayesian posterior”, 2) “sequential inference using BNNs fails to propagate information coherently” and 3) “BNN predictive updates are not well-calibrated”. Are these two sets of claims meant to match up one to one and in that order? I fail to see how predictive coherence is the same as well-calibration. Also, I don’t understand the term “functional consistency.” Reading 4.1, it seems that common BNN algorithms lack the true functional “variability”.\n- I find the notation used in the CIFAR-10 experiment a bit confusing. Is $ x^* $ a single unlabeled image or a set of unlabeled images?", "questions_for_authors": "1. The introduction states that BNNs “offer better predictive performance…than deterministic networks in multiple settings.” How do these claims hold up against deep ensembles?\n\n2. The martingale posterior is defined without any conditions on the belief updating scheme. Is this truly appropriate? A belief update could take many forms—does this framework require specific assumptions to guarantee desirable properties?\n\n3. In the CIFAR-10 experiment, why does using the intermediary \\( \\pi’ \\)-predictive to impute labels lead to a well-defined martingale posterior?\n\n4. I’m not sure my reading of the CIFAR-10 experiment is correct. Could you please check my summary of your third claim under Summary and correct any misconceptions I have?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper investigates the properties of several popular algorithms for approximate posterior inference in Bayesian neural networks (BNNs). The main experimental findings are that common approximate inference algorithms in BNNs: (a) do not exhibit functional consistency of posteriors, (b) do not propgate information coherently when performing sequential inference, and (c) do not exhibit predictive coherence.  This paper received 3 high-quality reviews, but I have largely ignored the review of Tqfi as it did not provide helpful insights.  Reviewer 8qCC raised some skepticism about reliance on the heteroskedastic noise model, to which the authors provided homoskedastic results (remaining concerns are mostly minor).  Reviewer yeeB is supportive of the paper on balance, but raises concerns about the lack of theoretical analysis.  Reviewer KcGY found the paper "well-argued and engaging" but requested additional experiments, to which the authors provided experiments on the IMDB dataset.  Discounting reviewer Tqfi there is strong consensus that this paper be accepted.
Decision: Accept (poster)