Evaluating Predictive Distributions: Does Bayesian Deep Learning Work?
Authors: Ian Osband
Zheng Wen
Seyed Mohammad Asghari
Xiuyuan Lu
Morteza Ibrahimi
Vikranth Dwaracherla
Dieterich Lawson
Brendan O'Donoghue
Botao Hao
Benjamin Van Roy
Venue: ICLR 2022 Submitted (2022)
Keywords: Deep Learning
Bayesian
Uncertainty
Testbed
Opensource Code

Abstract: Posterior predictive distributions quantify uncertainties ignored by point estimates.
This paper introduces \textit{The Neural Testbed}, which provides tools for the systematic evaluation of agents that generate such predictions.
Crucially, these tools assess not only the quality of marginal predictions per input, but also joint predictions given many inputs.
Joint distributions are often critical for useful uncertainty quantification, but they have been largely overlooked by the Bayesian deep learning community.
We benchmark several approaches to uncertainty estimation using a neural-network-based data generating process.
Our results reveal the importance of evaluation beyond marginal predictions.
Further, they reconcile sources of confusion in the field, such as why Bayesian deep learning approaches that generate accurate marginal predictions perform poorly in sequential decision tasks, how incorporating priors can be helpful, and what roles epistemic versus aleatoric uncertainty play when evaluating performance.
We also present experiments on real-world challenge datasets, which show a high correlation with testbed results, and that the importance of evaluating joint predictive distributions carries over to real data.
As part of this effort, we opensource The Neural Testbed, including all implementations from this paper.

Under review as a conference paper at ICLR 2022
EVALUATING PREDICTIVE DISTRIBUTIONS: DOES BAYESIAN DEEP LEARNING WORK?
A Anonymous authors Paper under double-blind review
ABSTRACT
Posterior predictive distributions quantify uncertainties ignored by point estimates. This paper introduces The Neural Testbed, which provides tools for the systematic evaluation of agents that generate such predictions. Cru- cially, these tools assess not only the quality of marginal predictions per in- put, but also joint predictions given many inputs. Joint distributions are of- ten critical for useful uncertainty quantification, but they have been largely overlooked by the Bayesian deep learning community. We benchmark sev- oral approaches to uncertainty estimation using a neural-networks basod data generating process. Our results reveal the importance of evaluation beyond marginal predictions. Further, they reconcile sources of confusion in the field, such why Bayesian deep learning approaches that generate accurate marginal predictions perform poorly in sequential decision tasks, how incorporating priors can helpful, and what roles epistemic versus aleatoric uncertainty play when evaluating performance. We also present experiments on real-world challenge datasots, which show a high correlation with testbed results, and that the importance of evaluating joint predictive distributions carries over to real data. As part of this effort, we opensource The Neural Testbed, including all implementations from this paper.
INTRODUCTION
Deep learning has emerged as the state-of-the-ar approach across number of application domains in which agents learn from large amounts of data (LeCun et al., 2015). Neural networks are increasingly used not only to predict outcomes but also to inform decisions. Common approaches in deep learning produce point estimates but not uncertainty estimates, which are often required for effective decision-making Bayesian deep learning extends the methodology to produce such uncertainty estimates (MacKay, 1992; Neal, 2012).
We consider agents that are trained on data pairs ((X,, YL-1) : = 0, 1,,,,.., T - 1) and subsequently generate predictions given new inputs. When presented with input Xr, Bayesian neural network can generate a predictive distribution of the outcome Y141 that is yet to be observed. This distribution characterizes the agent's uncertainty about Y We refer to such a prediction as maryinal to distinguish it from a joint predictive distribution over list (Yr+ Yr++) of prospective outcomes corresponding inputs NT....,,ttra..
The importance of uncertainty estimation has motivated a great deal of research over recent years (Kendall & Gal, 2017). This research has produced a variety of agents that learn to generate predictive distributions. With this proliferation of alternatives, it is increasingly important to analyze and compare their performance (Filos et al., 2019; Nado et al., 2021). In this paper, we introduce new tools for systematic evaluation of such agents.
Our tools overcome several limitations faced by previous methods of evaluation. First, by focusing purely on predictive distributions, we allow for unified trcatment of approaches developed within the Bayesian neural network community and beyond. This sidesteps the
Open source code available httpa: http:///aaaoonooo.ooe.eeeeeeeee httpaaaceeeeeeeaera-eeeeeeeeeeee -esttedd-atae

---

Under review as a conference paper at ICLR 2022
question of whether any approach 'is really Bayesian' (Wilson & Izmailov, 2020). Second, our tools evaluate the quality of higher-order joint predictions (T > 1). Until now, the Bayesian deep learning literature has focused almost exclusively on evaluating marginal predictions (Wang et al., 2021). Finally, we develop a neurra-network-bbased data generating process for Bayesian deep learning that can be used to drive insight and algorithm development. Where research has focused on a small set of challenge datasets, this might introduce bias through overfitting via multiple iterations of algorithm development. We use these tools to compare hundreds of agent variants. Further, we show that performance on our synthetic data generating process data is highly correlated with performance on real-world challenge datasels. We opensource all code used in this paper as The Neural Testbed.
Our results reconcile several sources of confusion the field. One concerns why particular approaches developed by the Bayesian deep learning community, such as Bayes-by-backprog dropout, and deep ensembles, perform poorly sequential decision tasks despite faring well based on evaluation metries of that community (Osband et al., 2018). Our results demonstrate that, while such methods produce accurate marginal predictions, they are no longer competitive when it comes to high-order joint predictions. Joint predictions play critical role in sequential decision-making (Lu et al., 2021).
Another puzzling issue is that state-of- of-the-arttttt methods do not employ domain-specifie priors. Whether Bayesian deep learning approaches should at all is a subject of controversy (Wenzel et al., 2020)- We show that the benefits of domain-specific priors can be pronounced when evaluating high-order joint predictions, even where they are negligible for marginals.
We also help resolve a point of philosophical debate within the deep learning community: the importance of epistemic versus aleatoric uncertainly". The strangeness of this distinc- tion has even made its way into wider popular culture, as satirized in the XKCD comic of Figure (Munroe, 2021). For a given parametric model, we can clearly distinguish param- eter uncertainty from noise, or reducible from irreducible uncertainty. However, from the perspective of a learning agent, the choice of model is subjective; different models can lead to the same marginal predictions. Our formulation provides clear and objective way to assess the quality of predictive distributions, without reliance this subjective distinction between knowledge and chance. Crucially, we show that this can be judged via the quality of joint predictions, but that marginals are not sufficient.
REQULAR UNCERPIITY
EPISTEMIC UNCERTANTYY
OUR STUDY FOOND THE DRUG WAS EFFECTIVE, UMH CONFIDENCE. INTERMAL
STUDY FOUND THE DRUS BE 7:% EFFECTIV.
POWEVER THERE
CHANCE THPT OUR STUDY
RRoM 63x 81%.
LPS MODFED ED BY GEORGE TAMPERER UHOSE ARE UNFREDNCTABLE.
73 74?? '7 -7777 PiP
74%
Figure 1: Epistemic or alcatoric? Does it matter?
It is worth mentioning another notable contribution of this work. The quality of a predictive distribution is commonly assessed in terms of cross-entropy loss. While this measure is well- defined for both marginal and joint predictions, to the best of our knowledge, the literature has only addressed computation in the former case. For high-order joint predictions, the straightforward approach would require computing sums over exponentially many values. To render this computationally tractable, we developed a novel approximation algorithm that leverages random partitioning operation and Monte Carlo simulation. While this approach motivated by concepts from high-dimensional geometry (Kaski, 1998; Donoho, 2006), we leave its analysis as a topic for future theoretical research.
Eppitemicc uncertainty relates to knowledge (ancient Greek epistemee+ opposed to aleatoric uncertainty relating chance (Latin aleat-dicc) (Der Kinreghian & Ditlevsen, 2009).
2

---

Under review as a conference paper at ICLR 2022
2 EVALUATING PREDICTIVE DISTRIBUTIONS
In this section, we introduce notation for the standard supervised learning framework we will consider (classification) well as our evaluation metric (the KT--los). We also explain how we cstimate the KL-loss for high-order joint predictions where exact computation is infeasible, using random partitions and Monte Carlo simulation.
2.1 KULLBACK-LEIREE LOSS
Consider a sequence of pairs ((X,, :t 2,,,,..., where each X, is a feature vector and each Y++1 is its target label. This sequence is ii.d. conditioned on the environment E, which produces the data, and which we view as a latent random variable. We consider an agent that is uncertain about the environment and predicts class labels +I+1:T+r (Yr+1...., Yr+r) given training data pairs Dr = ( :: : 0, 0.1.2.... unlabelled feature vectors XT:T+T-1 = (x+:...,x++7] Y411
1) and From the agent's perspective, each feature vector X₁ is generated i.i.d from fixed distribution P(X, .), and each class label V21 then drawn from P(Y++1 E -|E,,
We describe the agent's predictions in terms of generative model, parameterized by vector Or that the agent learns from the training data Dr. For any inputs XT.T4r-1, Or determines a predictive distribution, which could be used to sample imagined outcomes 4T+1T++ We define the 7 -order expected KT-loss by
dir. [[[K (P (Yr+1.++ E E,,, (1:4t11))))) P(Y++17+++ E 101, X
(1)
environment likelihood
agent likelihood
= log (P (YT+IT++ = YT41T+7 Or, -
+
cross-entropy loss negative log-likelihodd
where E [log (P (Y1+17++|E, Xy:r-r-))) is independent of 0r. The expectation is taken over all random variables, including the environment E, the parameters 0r, Xr.T+1-1 and Y-+----- Note that dkk,, is equivalent to the widely used notion of cross-entropy loss, though offsct by quantity that is independent of 0r (Kullback & Leibler, 1951). For 1, dkL assesses joint rather than the marginal predictions.
2.2 MARGINAL VERSUS JOINT PREDICTIONS
Evaluating an agent's ability estimate uncertainty on joint instead of marginal predictions can result in very different answers. We provide a simple example that illustrates the point. Suppose the data ((X,, Y(+1) : t 0, 2, .). generated by repeated tosses of a possibly biased coin with unknown probability of heads. 2 Lot X, = 0, to indicate that there is no input, and let cach outcome Y++1 be 0 or indicate tails or heads, respectively. Consider two agents that, without any training, predict outcomes. Agent assumes p 2/3 and models the oulcome of each flip as pure chance. Agent. assumes that the coin fully biased, meaning that E (0, 1}, but assigns probabilities 1/3 and 2/3 to 0 and 1.
Let 8/4 and 1 denote the outcomes imagined by the two agents. Despite their differing assumptions, the two agents generate identical marginal predictive distributions: P(YL = 0) = PP(22 = 0) 1/3. On the other hand, joint predictions greatly differ for large
P( O..., 8 0) 1/37 < 1/3 = = 0,
= 8 =).
We can say that agent attributes all uncertainty to aleatoric sources and agent 2, epistemic sources (although as Figure alludes, there are many ways an agent can attribute sources of uncertainty). Evaluating marginal predictions cannot distinguish between the two pos- sibilities, though for a specific prior distribution over one agent could be right and the other wrong. One must evaluate joint predictions to make this distinction.
We consider this coin illustrative model of more complex binary outcomes, such as whether will click on ad, or whether given mortgage will default payments.
3

---

Under review as a conference paper at ICLR 2022
When it comes to decision-making this distinction can be critical (Lu et al., 2021). In casino, under the first agent's s assumption, there is large upside and little risk on repeatedly betting on heads in the long run. However, if there is 1/3 chance the coin will always land tails, as is the case in the second agent's prediction, there is a ruinous risk to repeatedly betting heads. Evaluating joint predictions beyond marginals distinguishes these cases.
2.3 COMPUTATION OF KULLBACK LIILBERR-LEIBLERR LOSS
In contexts we will consider, it is not possible to compute dii, exactly. As such, we will approximate di via Monte Carlo simulation. This section provides a high level overview of our approach, we push the full details to Appendix A. Algorithm outlines a basic approach to estimating dk with respect to a synthetic data generating process. The algorithm samples set of environments and training dataset for each environment. For each of these pairs, the agent is re-initialized, trained, and then lested on N independent lest data r-samples. Note that each test data T-sample includes т data pairs. For each test data T-sample, the likelihood of the environment is computed exactly, but that of the agent's belief distribution approximated. The estimate of dik is taken to be the sample mean of the log- ong-likelihood-ratiie (Algorithm 2).
Algorithm 1 KL-Loss Computation
1: for j= 1, ..,..JJJ do
2: sample environment and training dataset, and train agent
3: for n = 1.. N do
4: sample test data r-sample with feature-label pairs b. likelihood of environment estimated likelihood of agent's belief distribution D estimated log-likelihood-ratio
5: compute Pj.n
6: compute Pin return J
7:
log 10j.n/jj..
While the likelihood of an environment can be efficiently computed, that of an agent's 's belief distribution poses computational challenge. One approach is to estimate this likelihood via Monte Carlo simulation (Algorithm 3). This produces unbiased estimates, which can be accurate when is small. However, maintaining accuracy requires the number of samples to grow exponentially with 7, as discussed in Appendix A.1. To overcome this challenge, we propose novel approach that estimates the likelihood of the agent's beliefs via combina- tion of randomized partitioning and Monte Carlo simulation (Algorithm 4) (Kaski, 1998). We conjecture that, under suitable regularity conditions, this novel approach produces accu- rate estimates even when 7 is large, but leave a formal analysis to future work. Even though Algorithm 1 is developed for a synthetic data generating process, it is straighttorward to extend it to evaluate agents on real data. We outline our approach to real data in Section 5.1, with full details in Appendix A.2.
3 BENCHMARK AGENTS
In this section we outline the baseline agents that we use to benchmark canonical approaches to uncertainty estimation in deep learning. Table links to papers that introduce these agents, as well as the hyperparanters that we tuned to optimize their performance via gridsearch. In each case, we attempt match 'canonical' implementations, which we open source al https: // anonymous 4open. scwenceer
In addition these agent implementations, our opensource project contains all the evalua- tion code to reproduce the results of this paper. Our code is written in Python and makes use of Jax internally (Bradbury et al., 2018). However, our evaluation procedure is framework agnostic, and can equally be used with any Python package including Tensorflow, Pytorch or even SKlearn. Over the course of this paper, we have made extensive use of parallel computation to facilitate large hyperparameter sweeps over many problems. Nevertheless, the overall computational cost is relatively low by modern deep learning standards and relies only standard CPU. For reference, evaluating the mlp agent across all the problems in
4

---

Under review as a conference paper at ICLR 2022
agent
description
hyperparameters
mlp
Vanilla MLP
La decay
ensemble
DDeep Ensemble' (Lakshminarayanan et al., 2017)
Lg decay, ensemble size
dropout
Dropout (Gal & Chahramani, 2016)
L2 decay, network, dropout rate
bbb
Bayes by Backprop (Blundell et al., 2015)
prior mixture, network, early stopping
sgmcmc
Stochastic Langevin MCMC (Welling & Teh, 2011)
learning rate, prior, momentum
ensemble+
Ensemble + prior functions (Osband et al., 2018)
Le decay, ensemble size, prior scale, bootstrap Le decay, prior, bootstrap, index dimension
hypermodel
Hypermodel (Dwaracherla et al., 2020)
Table 1:
Summary of benchmark agents, full details in Appendix B.
our testbed and real data requires less than 3 CPU-hours. We view our opensource effort as one of the major contributions this work. We provide clear and strong baselines, to- gether with objective and accessible method for assessing uncertainty estimates beyond marginal distributions.
THE NEURAL TESTBED
In this section we introduce the Neural Testbed, a system for assessing and comparing agent performance. The Testbed implements synthetic ic data generating processes and streamlines the process of sampling data, training agents, and evaluating test performance by estimat- ing KL-loss for marginal and high-order joint predictions. Since independent data can be generated for each execution, the Testbed can drive insight and multiple iterations of algo- rithm development without risk of overfitting to a fixed datasct. We begin by describing the simple gencrative model basod around random 2-layer MLP. We then apply this testbed to evaluate comprehensive set of benchmark agents.
4.1 SYNTHETIC DATA GENERATING PROCESSES
By data generating process, we do not mean only the conditional distribution of data pairs Y--)EE but also the distribution of the environment E. The Testbed considers 2- dimensional inputs and binary classification problems, although the generating processes can be easily extended to any input dimension and number of classes. The Testbed offers three data generating processes distinguished by a "temperature" setting, which signifies the signal-to-noise ratio (SNR) regime of the generated data. The agent can be tuned separately for each setting. This reflects prior knowledge whether the agent is operating in a high SNR regime such as image recognition or low SNR regime such weather forecasting.
To generate model, the Testbed samples a 2-hidden-layer ReLU MLP with 2 output units, which are scaled by 1/ 1/temperature and passed through softmax function to produce class probabilities. The MLP is sampled according to standard Xavier initialization (Glorot & Bengio, 2010), with the exception that biases the first layer are drawn from (0, 2). The inputs (X, tt = 0, 1,..... are drawn i.idd from N(0, I). The agent is provided with the data generating process as prior knowledge.
In Section 2.1. we described KL-loss as a metric for evaluating performance of an agent. The Neural Testbed estimates KL-loss, with T E (1, 100), for three temperature scttings and several training dataset sizes. For each value of r, the KL-losses are averaged to produce an aggregate performance measure. Further details concerning data generation and agent evaluation are offered in Appendix A.
4.2 PERFORMANCE IN MARGINAL PREDICTIONS
We begin our evaluation of benchmark approaches to Bayesian deep learning in marginal predictions (r - 1). This setting has been the main focus of the Bayesian deep learning literature. Despite this focus, it is surprising to see in Figure 2 that none of the bench- mark methods significantly outperform well-tuned MLP baseline according to dkl. Of course, there are many other metrics one might consider, but in this fundamental metric of prediction quality, the mlp agent presents baseline that is difficult to outperform.
5

---

Under review as a conference paper at ICLR 2022
normalized KL estimate
0.8
ensemble
dropout
bbb
hypermodel
ensemble+
sgmamc
agent
Figure 2: Most Bayesian deep learning approaches do not significantly outperform single MLP in marginal predictions (r =). Once we examine predictive distributions beyond
marginals we see a clear difference in performance between our benchmark agents (T = 100). For each T, the KL estimates are normalized by the KL of the MLP agent.
tau:
100
average KL estimate
nermaltaedd estimatee
0.5
agent
ensemble
2.25
0.2
ensemblet
0.1
0-
1.0
100
1000
10
100
1000
agent heperparmmtees
number of training points
Figure Agent robustness im- proves with bootstrapping.
Figure The benefits of additive prior functions are clear in the high tau, low data regime.
One of the keys to this result is that all of the agents are able to tune their hyperparametens, such as L2 weight decay, to the SNR regime and number of training points. This matches the way deep learning systems are typically implemented in practice, with extensive hyperpa- rameter tuning on validation data. This methodology has led many practitioners to doubt the usefulness of automatic tuning procedures such as bootstrap sampling (Nixon et al., 2020). In Figure 3, we compare the performance of an ensemble+ agent that uses boot- strapping with and without the ability to tune the hyperparameters per problem setting. We see that bootstrap sampling is beneficial when the agent is expected to work robustly over wide range of problem settings. However, the benefits are no longer apparent when the agent is allowed to tune its hyperparameters to individual tasks.
4.3 PERFORMANCE BEYOND MARGINALS
One of the key contributions of this paper is to evaluate predictive distribations beyond marginals. In Figure 2, the red bars show the results of benchmark agents evaluated on joint predictive distributions with - 100. Unlike when evaluating on marginal predic- tions, where method significantly outperforms well-tuned MLP, the potential benefits afforded by Bayesian deep learning become clear when examining higher-order predictive distributions. Our results refute prior works' claims that examining diu, beyond marginals provides little new information (Wang el al., 2021).
Figure shows that sgmeme is the top-performing agent overall. This should be reassuring to the Bayesian deep learning community and beyond. In the limit of large compute this agent should recover the 'gold-standard" of Bayesian inference, and does indeed perform best (Welling & Teh, 2011). However, some of the most popular approaches in this field (ensemble, dropout) do not actually provide good approximations to the predictive distri- bution in 100. In fact, see that even though Bayesian purists may deride ensemble* and hypermodels as 'not really Bayesian', these methods actually provide much better ap- proximations the Bayesian posterior than ffully Bayesian' VT approaches like bbb. We

---

Under review as a conference paper at ICLR 2022
note too that while sgmcmc performs best, it also requires orders of magnitude more com- putation than competitive methods even in this toy setting (see Appendix C.2). As we scale to complex environments, it may therefore be worthwhile to consider alternative approaches to approximate Bayesian inference.
For insight into where top agents able to outperform, we compare ensemble and ensemble+ under the medium SNR regime in Figures and 5. These methods are identical, except for the addition of a randomized prior function (Osband al., 2018). Figure shows that, although these methods perform similarly in the quality of their marginal predictions (T=1) 1), the addition of prior function greatly improves the quality of joint predictive distributions (r= 100) in the low data regime. Figure 5 provides additional intuition into how the randomized prior functions are able to drive improved performance. Figure 5a shows sampled generative model from our Testbed, with the training data shown in red and blue circles. Figure 5b shows the mean predictions and randomly sampled ensemble members from each agent (top=ensemble, bottom=ensemblee agents mostly agree in their mean predictions, ensemblet produces more diverse sampled outcomes enabled by the addition of randomized prior functions. In contrast, ensemble produces similar samples, which may explain why its performance close to bascline mlp.
We see that, although the
true Sunction
0.50
2-
8
0.25
(a) True model.
(b) Agent samples: only ensemble+ produces diverse decision boundaries.
Figure Visualization of the predictions of ensemble and ensemble+ agents.
PERFORMANCE ON REAL DATA
Section 4 provides a simple, sanitized testbed for clear insight to the efficacy of Bayesian deep learning techniques. However, most deep learning research not driven by these sorts of synthetic generative models, but the ultimate goal of performing well real datasets. In this section, we apply the same benchmark agents to a selection of small challenge datasets. We find that, on average, tuning agents for the synthetic problems leads to better performance on real data. We also find that, just as the synthetic testbed, agents that perform similarly in marginal predictions may be distinguished in the quality of their joint predictions.
5.1 DATASETS
We focus on 10 benchmark datasets (3 deature-based, image from pixels) drawn from the literature including Iris, MNIST, and CIFAR-10 (TFD). This collection is not intended to be comprehensive, or to include the most challenging large-scale problems, but instead to represent some canonical real-world data that might reasonably be addressed with the MLP models of Sention 4.1. We apply basic pre-processing step to each dataset, normalizing input features and flattening observations. We push full details to Appendix D.1.
To assess performance in real datasets, follow similar procedure Algorithm 1. The only difference that since it impossible to compute the likelihood of environment for real datasets, compute the negative log- likeiiilloood (NLL) rather than dīr. Appendix A.2 provides further details. Note that NLL and dig,. are equivalent for agent comparison since they differ by constant (see Equation 1). Furthermore, to allow for more direct comparison with the synthetic testbed, we also consider variants of each dataset where the number of training pairs is limited to less than the 'ull' dataset size.

---

Under review as a conference paper at ICLR 2022
5.2 SYNTHETIC DATA IS PREDICTIVE OF REAL DATA
Recall that Figure 2 compares performance across an array of agents, assessed using our syn- thetic data generating process. Each agent's hyperparameters were tuned by first enumerat- ing list of plausibly near-optinal choices and selecting the one that optimizes performance. Each of our real-world datasets can be viewed as generated by an environment sampled from alternative data generating process. A natural question is whether performance on the synthetic data correlates with performance on the real-world data.
The table of Figure 6a displays results pertaining to each of our agents. For each agent, performance for each candidate hyperparameter setting was assessed on synthetic and real data, and the correlation across these pairs is reported. The left and right columns restrict attention to datasets with low and high volumes training data, respectively. If a cor- relation
equal to 1, the hyperparameter setting that optimizes agent performance real data would be identical to that on synthetic data. It is reassuring that the correla- tions are high, reflecting a strong degree of alignment, with the exception of bbb in low data regime, for which there appear to be pathological outcomes distorting performance for small training sets. The values parentheses express 5th and 95th percentile confidence bounds, measured via the statistical bootstrap.
Figure 6b plots performance on real versus synthetic data for the high data regime. Each data point represents one sgent-hyperparanettr combination. If the correlation were equal to 1, the combination that performs best on the synthetic data would also perform best on the real data. It is reassuring that the correlation is large, and the confidence interval be- tween the 5th and 95th percentiles small. better the testbed tend to perform better on real data as well.
combinations that perform
agent
low data
high data
mlp
0.74 (0.57,0.85)
0.68 (0.38.0.99)
NLL on real data
correlation=0.76 (0.67,0.82)
ensemble
0.72 (0.52.0.85)
0.63 (0.31.0.96)
dropout
0.77 (0.68.0.86)
0.78 (0.66.0.87)
bbb
-0.48 (-0.6,-0.35)
0.76 (0.68,0.83)
sgmeme
0.72 (0.53,0.85)
0.86 (0.79.0.92)
0.5
ensemble+
0.85 (0.63,0.98)
0.74 (0.3,0.97)
hypermodel
0.52 (0.17,0.76)
0.33 (0.03.0.59)
0.3
0.03
0.3
KL on testbed
(a) Correlation by agent by data regime.
(b) Correlation in high data regime.
Figure 6: Performance on the Testbed correlates with performance on real datasels.
5.3 HIGHER ORDER PREDICTIONS AND INFORMATIVE PRIORS
Our synthetic testbed can be helpful in driving innovations that carry over to real data. Section 5.2 indicated that performance on the Testbed correlated with that on real- world data. We now repeat the observation from Figure 4 on real data; additive prior functions can significantly improve the accuracy joint predictive distributions generated by ensembles. We show this by comparing the performance of ensemble+ with different forms of prior functions on benchmark datasets. We evaluate an ensemble with no prior function (none), random MLP prior (MLP), and random lincar function of 2-dimensional latent representation as the prior, trained via variational autocncoder (VAE) (Kingma & Welling, 2014). We provide full details in Appendix D.3.
Figure/ plots the improvement in NLL for the ensemble agent relative to a baseline MLP (lower is better), and breaks out the result. for dadasets=MNNST, Tris and т 1, 100. We
8

---

Under review as a conference paper at ICLR 2022
can that the results for Iris mirror our syuthetic data almost exactly. The results for MNIST share some qualitative insights, but also reveal some important differences. For Iris 1 none of the methods outperform the MLP baseline, but for 100 see significant benefits to an additive ML.P prior in the low data regime. For MNIST -= 1 we actually see benefits to ensembles, even without prior functions and even in the high data regime. This reveals some aspects of this real data that are not captured by our synthetic model, where we did not see this behaviour. For T 100 the random MLP prior gives a slight advantage, but the effect is much less pronounced. We hypothesize this is because, unlike the testbed, the MLP prior is not well-matched to the input image data. However, the VAE prior able to provide significant benefit in the low data regime.* 3 These benefits also carry over to Iris, even where random MLPs already provided diglicant. value. Designing archilectures that. offer useful priors for learning agents is an exciting area for future work.
dataset: iris muu
datasel: tau: 100
datasel: minist
dataset: tau: 100
NLL ensemble NLL MLP
tau: 1
0
ensemble prior
MLP
-0.2
none
10
100
10
100
1e1
1d2
163
1e4
lel
1e2
l63
1ed
number of training points
Figure 7: Prior functions provide significant benefit in the high tau, low data regime, just like the lestbed. However, for image datasets random MLP priors provide relatively little benefit. Unsupervised pretraining can help to design useful priors in high dimensional data.
6 CONCLUSION
This paper highlights the need to evaluate predictive distributions beyond marginals. In addition to this conceptual contribution, we develop a suite of practical computational tools that can evaluate diverse approaches to uncertainty estimation. Together with these tools, we provide a neural-netwwrrk-bssel data generating process that facilitates research and iteration beyond small set of challenge datasets. We package these together as The Neural Testbed, including a variety of baseline agent implementations. We believe that this represents an exciting and valuable new benchmark for Bayesian deep learning and beyond.
We have already used this testbed to generate several new insights in this paper. We have shown many popular Bayesian deep learning approaches perform similarly in marginal pre- dictions but quite differently in joint predictions. We reveal the importance of bootstrapping for parameter robustness, and also help reconcile the observed lack of improvement when tuned to specific datascts. We have shown that these insights from synthetic data can carry over to real datasets; that performance in these settings correlated, that agents with sim- ilar marginal predictions can be distinguished by their joint predictions, and that suitable prior functions can play an important role in driving good performance.
The results in this paper are in some sense preliminary. The grand challenge for Bayesian deep learning to provide effective uncertainty estimates in large, rich datasets. While we have demonstrated benefits to predictive evaluation beyond marginals only in the 'low data' regime and small-scale problems, we believe that they will extend more broadly situations where new test inputs appear novel relative to training data. As such, we believe our core insights should carry over to the related problems nonstationarity and covariate shift that plague modern deep learning systems. As agent takes on more and more complex lasks, will continue to run into new and unfamiliar settings and uncertain outcomes; as such, effective predictive distributions will be more important than ever.
We hypothesize that appropriately initialized convnet architectures may be able to leverage image structure as noted in prior work (Ulyanov et al., 2018).
9

---

Under review as a conference paper at ICLR 2022
REFERENCES
TensorFlon Datasets, a collection of ready-to-use datasets. www.::/wwwwwoooo....oooo
datasets.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight un- certainty in neural network. In International Conference Machine Learning, pp. 1613- 1622. PMLR, 2015.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+ programs, 2018. URL http://gt /etp:///ithub.... thub. com/ google/jax.
Thomas Cover and Peter Hart. Nearest neighbor pattern classification. TEEE transactions on information theory, (3:1 27, 1967.
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter? Structural safety, (2):1055-1 2009.
David L Donoho. Compressed sensing. TEEE Transactions on information theory, 52(4):
1289 1306, 2006.
Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi, Ian Osband, Zheng Wen, and Ben- jamin Van Roy. Hypermodels for exploration. In Indernational Conference Learning Representations, 2020. URL https://openreviie peet/forum?ideryxxiiii
Angelos Filos, Sebastian Farquhar, Aidan N Gomez, Tim GJ Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud De Kroon, and Yarin Gal. A systematic compar- ison of Bayesian deep learning robustness in diabetic retinopathy tasks. orXiv preprint arXin: 1912. 10481, 2019.
Jerome H Friedman. The elements of statistical learning: Data mining, inference, and prediction. springer open, 2017.
Yarin Gal and Zoubin Ghahramani. Dropout as Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, 2016.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedfor- ward neural networks. In Proceedings of the 13th international conference on artificial intelligence and statistics, pp. 249 256, 2010.
Bobby He, Balaji Lakshminarayanan, and Yee whye Teh. Bayesian deep ensembles via the neural tangent. kernel. arXiv preprint ar arXi::2007.0886, 2020.
Matthew Hoffman, Andrew Gelman, et al. The no-u-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15(1):1593 16208,163,,,1 2014.
Samuel Kaski. Dimensionality reduction by random mapping: fast similarity computation for clustering. 1998 IEFE International Joint Conference on Newral Networks Proceedings. TEEE World Congress on Computational Intelligence (Cat. No.98CH36227), 1:413 418 vol.1, 1998.
Alex Kendall and Yarin Gal. What uncertainties do we need in Bayesian deep learning for computer vision? Tn Advances in Neural Information Processing Systems, volume 30, 2017.
Diederik P Kingma and Max Welling, Auto-encoding variational Bayes International Conference on Learning Representations, 2014.
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathemalical statistics, 22(1):79 86, 1951.
10

---

Under review as a conference paper at ICLR 2022
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable pre- dictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pp. 6405-6416, 2017.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521((553)::33; 2015.
Xiuyuan Lu, lan Osband, Benjamin Van Roy, and Zheng Wen. Evaluating probabilistic inference in deep learning: Beyond marginal predictions. CoRR, abs/ abs/2107.092244, 2021. URL attps://arxas org/oogg//a httgaas//1000 09224.
David JC MacKay. A practical Bayesian framework for backpropagation networks. Neural computation, 4(3):448-477) 1992.
Randall Munroe. Xked webcomic, 2021. URL https: xwtp::///mxxccdcooom cwm/2440/.
Kevin P Murphy. Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
Zachary Nado, Neil Band, Mark Collier, Josip Djolonga, Michael Dusenberry, Sebastian Farquhar, Angelos Filos, Marton Havasi, Rodolphe Jenatton, Ghassen Jerfel, Jeremiah Liu, Zelda Mariet, Jeremy Nixon, Shreyas Padhy, Jie Ren, Tim Rudner, Yeming Wen, Florian Wenzel, Kevin Murphy, D. Sculley, Balaji Takshminarayanan Jasper Snoek, Yarin Gal, and Dustin Tran. Uncertainty Baselines: Benchmarks for uncertainty robustness in deep learning. arXiv preprint arXie:2106. 100.:2206604415, 2021.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Jeremy Nixon, Balaji Lakshminarayanan, and Dustin Tran. Why are bootstrapped deep ensembles not better? In "I Can't Believe It's Not Better!"NeurIPP 2020 workshop, 2020.
lan Osband and Benjamin Van Roy. Bootstrapped Thompson sampling and deep explo- ration. Xix preprint 3rXiv:1507.00300 2015.
Ian Osband, John Aslanides, and Albin Cassirer. Randomized prior functions for deep reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Newral Information Processing Systems 31, pp. 8617 8629. Curran Associates, Inc., 2018. URL http::/ mips. cc/ppper/ 8080-randomized wwwpamiiiidddddiddeeeeee frrioorfunciions-oe fer- reinforcement-
pdf.
Ian Osband, Zheng Wen, Mohammad Asghari, Morteza Ibrahimi, Xiyuan Lu, and Benjamin Van Roy. Epistemic neural networks. ar preprint ar arXiv:2107.0890 2021.
F. Podregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Griscl, M. Blon- del, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer school on machine learning, pp. 63-71. Springer, 2003.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference machine learning, pp. 1530-1538. PMLR, 2015.
Bernhard Schölkopf and Alexander J Smola. Learning with kernels: Support vector ma- chines, regularization, optimization, and beyond. MIT press, 2018.
Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse. Functional variational Bayesian neural networks. arXiv preprint 1xXxv:1903.077 2019.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 9446-9444 2018.
11

---

Under review as a conference paper at ICLR 2022
Chaoqi Wang, Shengyang Sun, and Roger Grosse. Beyond marginal uncertainty: How accurately can Bayesian regression models estimate posterior predictive correlations? In International Conference on Artificial Intelligence and Statistics, pp. 2476 2484. PMLR, 2021.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), pp. 681-688. Citeseer, 2011.
Florian Wenzel, Kevin Roth, Bastiaan S Veeling, Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, and Sebastian Nowozin. How good the Bayes posterior in deep neural networks really? arXiv preprint arXi::2002. 02405, 2020.
Andrew Gordon Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. arXiv preprint "XXi:2002.08790 2020.
Max A Woodbury. Inverting modified matrices. Statistical Research Group, 1950.
12

---

4. number of sampled problems J, number of test data samples N Under review as a conference paper at ICLR 2022
A TESTBED PSEUDOCODE
We present the testbed pseudocode in this section. Specifically, Algorithm 2 is the pseu- docode for our neural testbed, and Algorithm 3 and Algorithm 4 are two different ap- proaches to estimate the likelihood of test data r-sample conditioned lief. Algorithm 3 based on the standard Monte-Carlo estimation, while Algorithm adopts random partitioning approach. The presented testbed pseudocode works for any prior P(E E :) over the environment and any input distribution Px, including the ones described in Section 4.1. We also release full code and implementations at 4open.
agent's be-
paaeceer/neural-teeeieeee B839.
In addition presenting the testbed pseudocode, we also discuss some core technical issues in the neural testbed design. Specifically, Appendix A.1 discusses how to estimate the likelihood of an agent's belief distribution; Appendix A.2 discusses how extend the testbed to agent evaluation real data; finally, Appendix A.3 explains our choices of experiment parameters.
Algorithm 2 Neural Testbed
Require: the testbed requires the following inputs
1. prior distribution over the environment P(E E +), input distribution Px
2. agent fo
3. number of training data T, test distribution order
5. parameters for agent likelihood estimation, as is specified in Algorithm 3 and 4
for j = 1. do
Step 1: sample environment and training data
1. sample environment & e))
2. sample inputs Xu, 1,,..., Xx-1 i.i.d. from Px
3. sample the training labels YI...., ... Yr conditionally i..d. as
Y41 ~ P E 1 X,) 0, ,.. T
4. choose the training dataset Dr {(X,, M(+1), t= =0 ......"" T -11
Step 2: train agent
train agent fer based on training datasct Dr
Step 3: compute likelihoods
for n = 1.2.....N do
1. sample (x)
(n) T4r-1 i.idd. from Px
2. generate (n) (()
T+1... T4r conditionally independently as
y(mi Y E x 4I = T
3. compute the likelihood under the environment as
- Pr (rim) e, xxx( (m)
Pia- = P (YTHIT++
4. estimate the likelihood conditioned on the agent's belief
Pjn s P (TTITT = ((n) + Or. (()) (()
1-7+7-19 T+1:T++)
based on Algorithm or with test data r-sample (xfe) . x(m) x
return JN ΣA log (p.n/Pp,m))
13

---

Under review as a conference paper at ICLR 2022
Algorithm 3 Monte Carlo Estimation of Likelihood of Agent's Belief
Require:
1. trained agent for and number of Monte Carlo samples M
2. lest data 7-sample (X YT+1.++))
Step 1: sample M models E,,...,& Ess conditionally i.d. from P (
Step estimate p as
Σ P M
1
(PI+I:++ Yr+1T++ XT-T++-1 Y(+IT++)
return P
Algorithm Estimation of Likelihood of Agent's Belief via Random Partitioning
Require:
1. trained agent for
2. number of Monte Carlo samples M
3. number of hyperplanes d
4. test data 7-sample (XT:T+7-1, YT+1TT+)
Step 1: sample M models Ei...., EN conditionally i.i.d. from P(Ê -(Sop); for each model m = 1...., M,, class k; and 1 = T..... T r 1, define
Pm. = P(P.(m) 6+1 = km,X))
and b,,t = --1 (pm,a.)), where ((-) is the CDF of the standard normal function. For each model m, define vector
fm 1m,,,,1, 1m..:..........4.111..
Step 2: sample d (Kr) matrix A and d-dimensional vector b, with each ele- ment cont/ccopp component sampled i.idd from N(0,1), 1). For each m = I,...,,, M, compute
4m |A + b 2 0J E 10.
Step 3: partition the sampled models, with each cell indexed by ei E 10, and defined by
M₂ = (m : vm = +}}
and assign a probability to each cell:
MWl
qo M
Step 4: Vú E 10,1] 1] and Vt = T.T + 1,..., T + - estimate the probability of predicting Ye+1 l conditioned on the cell:
M Emem, Pm,t. if | > 0 M =0
Pelck
Step 5: estimate Pr(G
Y++1T++107 XET Y++1T++) as
Σ
do II
PoaYY
WE (0.10
t=r
return $
14

---

Under review as a conference paper at ICLR 2022
A.1 ESTIMATING LIKELIHOOD OF AGENT'S BELIEF DISTRIBUTION
We have presented two algorithms to estimate the likelihood of a test data 7-sample condi- tioned on a trained agent: Algorithm while Algorithm 4 adopts approach combining random partitioning and Monte Carlo estimation. In this subsection, we briefly discuss the pros and cons between these two algorithms,, and provide some general guidelines how to choose between them.
based on the standard Monte Carlo estimaliinn,
Algorithm produces unbiased estimates of the likelihoods, which is usually accurate when is small (e.g. for < 10). However, maintaining accuracy might require the number of samples M to grow exponentially with т. The following is an illustrative example.
Example 1 (Uniform belief over deterministic models): Consider scenario where the number of class labels is K 2. We say model & deterministic if for any feature vector Xt:
P( = 11 E (0,1}.
Obviously, for any Lest data r-sample (XT:T+=11, with YT+IT44 E {0, 1)", under a deterministic model E, we have
(YI+11T+7 = YT+ITT+ XT YT+IT++) E {0,
When restricted the inputs XT.T+r-1, there are 2 distinguishable deterministic models. Assume the agent's belief distribution is uniform over these 2" distinguishable deterministic models, then for any Yr+IT++ E {0, 13", the likelihood of the agent's belief distribution
P ( = YTHIT+ Or, N
VI+LT-1) V 2
Now let's consider Algorithm 3. When model &m is sampled from the agent's belief distribution, with probability 2
P P -YTII++++ Em, XTTTTT-I,Y--111
and with probability 1
( = YTHIT+T &m, 1I =0.
Consequently, in expectation, need the number of Monte Carlo samples M (((2)) to ensure that the estimate á returned by Algorithm is non-zero.
To overcome this challenge, also propose novel approach to estimate the likelihood of agent's belief via a combination of randomized partitioning and Monte Carlo simulation, as is presented in Algorithm 4. This approach proceeds follows. First, M models sampled from the agent's 's belief distribution. For each sampled model, each test data input X₁, and each class label a predictive probability Pm,t,k and its probit L,,t, = & (Pm,,,)) are computed, where (--) is the CDF of the standard normal distribution. For each sampled model, we also stack its probits into a probit vector f.,. E Kr Then, random hyperplanes a are sampled and used to partition kr into 2 cells. Stacked probit vectors place models in cells. Predictive distributions of models in each cell are averaged, and the likelihood is calculated based on these averages, with each cell weighted according to the number of models contains.
The Neural Testbed applies Algorithm 4 with 2d < M. Hence, some cells are assigned many models. We conjecture that, under suitable regularity conditions, models assigned to the same ccll tend generate similar predictions. If this is the case, this algorithm produces accurate estimates even when t is large. We leave a formal analysis to future work.
Finally, briefly discuss how to choose between Algorithm 3 and Algorithm 4. As rule of thumb, we recommend to choose Algorithm for т < 10 and Algorithm 4 with the number of hyperplanes d bet 5 and 10 for 7 2 10.
15

---

Under review as a conference paper at ICLR 2022
A.2 AGENT EVALUATION ON REAL DATA
Algorithm (and its simplified version Algorithm 1) is developed for a synthetic data gener- ating processes. We now discuss how to extend it to agent evaluation on real data. Consider a scenario with J real datasets, and each dataset. is further partitioned into a training dataset. and test dataset. The main difference between this scenario and synthetie data gener- ating process is that we cannot compute the likelihood of environment for real data. Thus, we compute the cross-entropy loss instead (see Equation 1). The computational approach is similar to Algorithm 1: for each real dataset, we use its training dataset to train an agent. Then, we sample N test data r-samples from the test datasct, and estimate the likelihoods of the agent's 's belief distribution. The estimate of the cross-entropy loss is taken to be the sample mean of the negalive log-likelihoods.
Note that when ranking agents, the cross-entropy loss and diL will lead to the same order of agents, since these two losses differ by a constant independent of the agent (see Equation 1).
A.3 CHOICES OP EXPERIMENT PARAMETERS
To apply Algorithm 2, need to specify an input distribution Px and prior distribution on the environment P(E .). Recall from Section 4.1 that we consider binary classification problems with input dimension 2. We choose Px N(0, I), and consider three envi- ronment priors distinguished by a temperature parameter that controls the signal-to-noise ratio (SNR) regime. We sweep P(E .) is induced by a distribution over MLPs with 2 hidden layers and ReLU activation. The MLP is distributed according to standard Xavier initialization, except that biases in the first layer are drawn from NO, 2). The MLP outputs two units, which are divided by the temperature parameter and passed through the softmax function to produce class probabilities. The implemennattinn of this generative model is in our open source code under the path /generative/factoraae
temperatures in {0.01, 0.1, 0.5). The prior distribution
We now describe the other parameters we use in the Testbed. In Algorithm 2, we pick the order of predictive distributions т {1, 100), training dataset size T e (1,3, 10, 30, 100, 1000}, number of sampled problems J = 10, and number of testing data r-samples
1000. We apply Algorithm 3 for evaluation of dk and Algorithm for evaluation of 0 In both Algorithms and 4, sample M 1000 models from the agent. In Algorithm 4, we set the number hyperplanes d = 7. The specification the testbed parameters in our open soucre code under the path /1 /leaderboard www.rrbbarddwwewe...eeoo py-
On real datasets, we apply the same 7 E (001, = 1000, and M = 1000. We set the number of hyperplanes d = 10 in Algorithm 4.
16

---

Under review as a conference paper at ICLR 2022
B AGENTS
In this section, we describe the benchmark agents in Section and the choice of vari- hyperparameters used in the implementation of these agents. The list of agents in- clude MLP, ensemble, dropout, Bayes by backprop, stochastic Langevin MCMC, ensem- ble+ and hypermodel. We will also include other agents such KNN, random forest, and deep kernel, but the performance of these agents was worse than the other benchmark agents, so we chose not include them in the comparison in Section 4. In each case, attempt to match the "canonical" implementation. The complete implementation of these agents including the hyperparameter sweeps used for the Testbed are available at https:/ hatp:://nnnnouuueee 4open.
We make use of the Epis- temic Neural Networks notation from (Osband et al., 2021) in our code. We set the de- fault hyperparameters of each agent to be the ones that minimize the aggregated KL score qass- = dkl + 100 /100.
B.1 MLP
The mlp agent learns 2-layer MLP with 50 hidden units in each layer by minimiz- ing the cross-cntropy loss with L2 weight regularization. The L2 weight decay scale is chosen either to be t X d where d is the input dimension, 3 is the tempera- ture of the generative process and T is the size the training dataset. We sweep over {10-1, 10-3, 10-2, 10-1 10, 100]. We implement the MLP agent as special case of a deep ensemble (B.2). The implementation and hyperparameter sweeps for the mlp agent can be found in our open source code, as a special case of the ensemble agent, under the path /wgents/factorieneeeeeeee http://aattorrreeeeemmeeeeee wns.mble..mm.eemee
B.2 ENSEMBLE
We implement the basic "deep ensembles" approach for posterior approximation (Takshmi- narayanan et al., 2017). The ensemble agent learns ensemble of MLPs by minimizing the cross-entropy loss with L2 weight regularization. The only difference between the ensemble members is their independently initialized network weights. We chose the L2 weight scale
to be either 1 MT or X d/3 MT where M is the ensemble size, d is the input dimension, B is the temperature of the generative process, and T is the size of the training dataset. We sweep over ensemble size E (1,3, 10,30, 30, 100} and À E {10-4, 10-3, 10-2,10-1, 10, 100). We find that larger ensembles work better, but this effect is within margin of error after 10 elements. The implementation and hyperparameter sweeps for the ensemble agent can be found in open source code under the path Vagents/factories//eeee
B.3 DROPOUT
We follow Gal & Ghahramani (2016) to build a droput agent for posterior approxima- tion. The agent applies dropout on each layer of fully connected MLP with ReLU ac- tivation and optimizes the network using the cross-entropy loss combined with La weight
decay. The L2 weight decay scale is chosen to be either 1 (1 Pdrap) dvøt where Parop 27 is the dropping probability, d is the input dimension, 3 is the temperature of the data generating process, and T' is the size of the training dataset. We sweep over dropout rate Pärop E 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8,, length scale (used for L2 weight decay) I E (0.01, 0.1, 0.3, 1,3, 10), number of neural network layers (2,3), and hidden layer size E (50, 100}. The implementation and hyperparameter sweeps for the dropout agent can be found in our open source code under the path
eessrrrout..ppp../ py.
B.4 BAYES-BY-BACKPROR
We follow Bluddll et al. (2015) to build bbb agent for posterior approximation. We con- sider a scale mixture two zero-mean Gaussian densities as the prior. The Gaussian densi- ties have standard deviations dn and Ø2, and they mixed with probabilities p and p,
17

---

Under review as a conference paper at ICLR 2022
respectively. We sweep over 01 E (1,2.4), 02 0..2 p {0, 0.2., 0.75, 1}, learning rate E 110-3,3 x 10 -3,, number of training steps E (500, 1000, 10000), mumber of neural network layers E {2, (2,3). hidden layer size E {50, 100}, and the ratio of the complexity cost to the likelihood cost E (1,d/B] where d is the input dimension and / is the tempera- ture of the data generating process. The implementation and hyperparameter sweeps for the bbb agent can be found in our open source code under the path /wtentss/aattoriss/////// py.
B.5 STOCHASTIC GRADIENT LANGEVIN DYNAMICS
We follow Welling & Teh (2011) to implement sgmemc agent using stochastic gradient Langevin dynamics (SGLD). We consider two versions of SGLD, one with momentum and other without the momentum. We consider independent Gaussian prior the neural net- work parameters where the prior variance is set to be
T d3
0² =
where À is a hyperparameter that is swept over {0.01, 0.1, 0.5, d is the input dimension, is the temperature of the data gencrating process, and T the size of the training dataset. We consider a constant learning rate that is swept over {10-5,5 x 10 10100.55 10-4,10-3,5 10-3, 10-3). For SGLD with momentum, the momentum decay term is always set to be 0.9. The number of training batches is 5 x 10 with burn-in time of 10 training batches. We save a model every 1000 steps after the burn-in time and use these models as ensemble during the evaluation. The implementation and hyperparameter sweeps for the sgmeme agent can be found in our open source code under the path /agents/ factories /wwps///gmmm/////////////sssesss py-
B.6 ENSEMBLE+
We implement the ensemble+ agent using deep ensembles with randomized prior func- lions (Osband et al., 2018) and boolstrap sampling (Osband & Van Roy, 2015). Sim- ilar to the vanilla ensemble agent in Section B.2, we consider Le weight scale to be either A A (10-4.107 1002.100 1,,01,1,,00 100). The randomized prior functions are sampled exactly addition,
MT or X MT. We sweep over ensemble size M E {1,3, 10, 30, 100} and A e
from the data generating process, and we sweep over prior scaling E {0, /0,11,1, sweep over bootstrap lype (none, exponential, bernoulli). We find that the addition of randomized prior functions is crucial for improvement in performance over vanilla deep ensembles in terms of the quality of joint predictions. We also find that bootstrap sampling improves agent robustness, although the advantage is less apparent when one is allowed to tune the L2 weight decay for each task (see Figure 3). The implementation and hyperpa- rameter sweeps for the ensemble+ agent can be found our open source code under the path /ttg/////aaatoo//////a/// weweebeeeeeeeeeeeeeeeeeeeeeeeeee Bateee_pu_______eee .y. py.
B.7 HYPERMODEL
We follow Dwaracherla et al. (2020) to build a hypermodel agent for posterior approxima- tion. We consider a linear hypermodel over 2-layer MLP base model. We sweep over
index dimension E (1,3,5, 7). The L2 weight decay chosen to be either d or X dyā with À E (0.1,0.3, 1,3, 10), where d is the input dimension, 33 is the temperature of the data generating process, and T the size of the training dataset. We chose three different bootstrapping methods of none, exponential, bernoulli. We use additive prior which is linear hypermodel prior over an MLP base model, which is similar to the generating process, with number of hidden layers in (1, 2), 10 hidden units in each layer, and prior scale from {0, /B, 13. The implementation and hyperparameter sweeps for the hypermodel agent can be found in our open source code under the path aageatts/aactorree///peaddd
18

---

Under review as a conference paper at ICLR 2022
B.8 NON-PARAMETRIC CLASSIFIERS
K-ncarest neighbors (k-NN) (Cover & Hart, 1967) and random forest classifiers (Friedman, 2017) are simple and cheap off-the-shelf non-paramelric baselines (Murphy, 2012; Pedregosa et al., 2011). The "uncertainly" in these classifiers arises merely from the fact that they pro- duce distributions over the labels and such we do not expect them to perform well relative to more principled approaches. Moreover, these methods have no capacity to model di for 1. For the knn agent we swept over the number of neighbors E {1,5, 10, 30, 50, 1003 and the weighting of the contribution of each neighbor as cither uniform or based on distance. For the forest agent we swept over the number of trees in the forest (10, 100, 1000), and the splitting criterion which was either the Gini impurity coefficient or the information gain. To prevent infinite values in the KL we truncate the probabilities produced by these classifiers to be in the interval [0.01, 0.99]. The implementation and hyperparameter sweeps for the knn and random_foree agents can be found our open source code under the paths agents/factories waaeee////nnnpp.e knn.py and 'agenns/factorre wwreiiiesrraddnnnoe forest. py.
B.9 GAUSSIAN PROCESS WITH LEARNED KERNEL
neural network takes input X, E X and produces output Ze+1 W Woa(X,) + R where W E R is matrix, b E RK is bias vector, and Φu X'' + R is the outpul of the penultimate layer of the neural network. In the case of classification the output Z++1 corresponds to the logits over the class labels, ú.e., Yc-1 x exp(Z++1). The neural network should learn function that maps the input into space where the classes are lincarly distinguishable. can be considered a form of kernel (Schölkopf & Smola, 2018), where the kernel function k X' x X' + R is simply k(X, X') (x) éx(X'). With this in mind, we can lake a trained neural network and consider the learned mapping to be the kernel in a Gaussian process (GP) (Rasmussen, 2003), from which we can obtain approximate uncertainty estimates. Concretely, let P0-T-1 E R Txm be the matrix corresponding to the (x()),, t O..... 7..7-1. vectors stacked row-wise and let PT-T+T-1 E R denote the same quantity for the test sct. Fix index i E {O....,K K 1} to be a particular class index. GP models the joint distribution over the datasct to be a multi-variate Gaussian, i.e.,
other words, the mapping that the neural network is learning
...
Z
L-T
-N( [T++T++7]
(
LT
2 + 1 P0:T-1 10 I-T+r-1
PT 1 Por--
@T.T+- TT
T
where models the noise in the training data measurement and MIT: PT+1T+4 are the means under the GP. The conditional distribution is given by
(i)
(
P(Z 2 XOTT+-1)== = (u (UTHIT++1TTT ET+1T441111
where
ET41:44/111 = - -
t1 (021 00 (1)
and rather than use the GP to compute 17 #T+1:T+70077 (which would not be possible since we do not oberve the true logits) we just take it to be the output of the neural network when evaluated on the test dataset. The matrix being inverted in the expression for ET+1:T+700r Morraioo-ooroooooonnonao
has dimension 7' x T, which may be quite large. We use the identity to rewrite it as follows (Woodbury, 1950)
ET41T+770.T = @ T DO - FT.
= (0 - - -1:++-1
which instead involves the inverse of an m x m matrix, which may be much smaller. we perform Cholesky factorization of positive definite matrix (a²I + TT po:r-1) LL
then the samples for all logits simultaneously can be drawn by first sampling Ç m with each entry drawn IID from (0, 1), then forming
Yr+IT++ exp(pr+1.4+7117 + -
The implementation and hyperparameter sweeps for the deep kernel agent can be found in open source code under the path /agents/factories//eee kernel.py.
19

---

Under review as a conference paper at ICLR 2022
B.10 OTHER AGENTS
In our paper we have made a concerted effort to include representative and canonical agents across different families of Bayesian deep learning and adjacent research. In addition to these implementations. we performed exlensive luning to make sure that each agent was given fair shot. However, with the proliferation of research in this area, it not possible for us to evaluate all competiting approaches, We hope that, by opensourcing the Neural Testbed, we can allow researchers in the field easily assess and compare their agents to these basclines.
For example, we highlight few recent pieces of research that might be interesting evaluate in our setting. Of course, there are many more methods to compare and benchmark. We leave this open as an exciting area for future research.
Neural Tangent Kernel Prior Functions (He et al., 2020). Proposes a specific type of prior function in ensemble+ inspired by connections to the neural tangent. kernel.
Functional Variational Bayesian Neural Networks (Sun et al., 2019). Applies variational inference directly to the function outputs, rather than weights like bbb.
Variational normalizing flows (Rezende & Mohamed, 2015). Applies variational in- ference over a more expressive family than bbb.
No U-Turn Sampler (Hoffman al., 2014). Another approach to sgnome that attempts to compute the posterior directly, computational costs can grow large.
C TESTBED RESULTS
In this section, we provide the complete results of the performance of benchmark agents on the Testbed, broken down by the temperature setting, which controls the SNR, and the size of the training dataset. We select the best performing agent within each agent family and plot dk.. and d with the performance of MLP agent a reference. We also provide a plot comparing the training time of different agents.
C.1 PERFORMANCE BREAKDOWN
Figures 8 and show the KL estimates evaluated on r and 100, respectively. For each agent, for each SNR regime, for each number of training points we plot the average KL estimate from the Testbed. cach plot, we include the "baseline" mlp agent as a black dashed line to allow for easy comparison across agents. A detailed description of these beuchmark agents can found Appendix B.
C.2 TRAINING TIME
Figure 10 shows plot comparing the d and training time of different agents normalized with that of an MLP. We can see that sgmemc agent has the best performance, but at the cost of more training time (computation). Both ensemble+ and hypermodel agents have similar performance as sgmcmc with lower training time. We trained our agents on CPU only systems.
D REAL DATA
This section provides supplementary details regarding the experiments in Section 5. As before, we include full implementation and source code at https://anoni //w////nnonous.aa//// 4open. siiemee//nneural-tttteeee-e-ee
D.1 DATASETS
Table 2 outlines the datasets inchuded in our experiments. Unlike to the synthetic testbed, which evaluates agents over range of SNR regimes, these datasets generally all high
20

---

Under review as a conference paper at ICLR 2022
SNR regime. We can this since the top-performing agents in the literature able to obtain high levels of classification accuracy on held out data; something that impossible if the underlying system has high levels of noise.
dataset
type
classes
input dimension
# training pairs
iris
structured
3
4
120
wine quality
structured
11
11
3,918
german credit numeric
structured
800
mnist
image
10
784
60,000
fashion-mnist
image
10
784
60,000
auiss--orruppee//hotttnno
image
10
784
60,000
emnist/letters
image
37
784
88,800
emnist/digge cmaterdb
image
10
784
240,000
image
10
3,072
5,000
cifar10
image
10
3,072
50,000
Table 2: Summary of benchmark datasets used in the paper.
Each of these datasets is provided with canonical training/ter /test set of specific sizes. In order examine performance different data regimes we augment the default settings of Table 2 by also examining the performance of agents these datasets with reduced training data. a way that mirrors the testbed sweep of Section 4.1, we also look at settings where the training data is restricted to T 1, 10, 100, 1000, 10000 data points respectively.
D.2 CORRELATION
Figure breaks down the correlation in performance between testbeds and real data. For the purposes of Table 6a. say that 7 = is the 'low data' regime, and the maximum training dataset size is the 'high data' regime. Our results show that, for each agent, for each data regime, performance of hyperparameters is correlated across settings.
One concern might be that while performance on real data overall is highly correlated, that this might not necessarily be the for any individual dataset. Or, alternatively, that this correlation is driven by extremely strong relationships in one dataset that are not present in others. Figure 11 shows that this not the case. In fact, for each of the datasets considered we have strong and positive correlation over agent- peeuuthyperparmeter pairs. This gives us confidence that the results of Figure 6b are robust not only choice of agent, but also to some reasonable choice of datasets.
D.3 PRIOR FUNCTIONS
We consider two different forms of prior functions for ensemblet: random MLP of the input data and a random linear function of 2-dimensional latent trained via variational autoencoder (VAE) (Kingma & Welling, 2014). For the MLP prior, we tried both linear (MLP with hidden layer) and MLP with hidden layers, and observed that the linear prior works better. To train the 2-dimensional latent, we considcred 2-layer (128, 64) MLP for the Gaussian encoder and 2-layer (64, 128) MLP for the Bernoulli decoder. We trained the VAE using all unsupervised training data available for each dataset. After training the VAE for 10,000 steps, we used the output mean of the Gaussian encoder as the latent.
21

---

Under review as a conference paper at ICLR 2022
temperature 0.01
temperature 0.1
temperature 0.5
0.6
0.4
ensemble
0.6
ensemble+
0.4
0.2
0.6
hypermodel
0.4
0.2
0.6
0.4
dropout
0.2
estimate on tau=1
0.6
0.4
sgm
0.2
0.6
0.4
bbb
0.2
0
0.6
deep kemel
0.2
0
0.6
knn
0.2
0
0.6
random forest
0.4
0.2
1000
100
10
1000
Number of training points
Figure 8: Performance of benchmark agents on the Testbed evaluated on compared against the MLP basclinc.
22

---

Under review as a conference paper at ICLR 2022
temperature = 0.01
temperature 0.1
temperature 0.5
60
ensemble
40
20
60
ensemble+
40
60
hypermodel
20
40
dropout
20
KL estimate on tau=100
60
5gmcmc
40
20
60
40
bbb
20
60
deep kernel
40
knn
20
60
random forest
40
20
100
10
1000
100
1000
Number of training points
Figure Performance of benchmark agents on the Testbed evaluated on J 100, compared against the MLP baseline.
23

---

Under review as a conference paper at ICLR 2022
agent
Normalized aize KL estimate
bbb
0.9
dropout
ensemble
0.8
0.7
hypermodel
milp
0.6
sgmomc
10
30
Average training time MLP training time)
Figure 10: Normalized KL vs training time of different agents
dicand: 200r
03
62
0.00
3.01
obbel nureet
cormiation 0as
combacen Cal
3010c Wes gealy
convidionn
03
01
021
0.0
321
21
001
tecteed
Figure 11: Correlation in high data regime for different datasets.
24


Reviews:
[{"review_id": "mpoDN5Rsr97", "reviewer": "Reviewer_QqqV", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I appreciate that benchmark emphasizes joint predictions as an important component in evalauting uncertainty estimates. I agree that these joint predictions are very important for certain applications of uncertainty estimation, especially in sequential d\necision making settings. However, I am not convinced this benchmark will be a significant and useful contribution to the community in its current state, for the reasons below.\n\n**Existing benchmarks**: Wang et al 2021 [1] already emphasize the differences between joint and marginal uncertainties and propose various ways of evaluating joint predictions, including a downstream transductive active learning task. Interestingly, Wang et al find that directly evaluating joint likelihoods gave little more information than simply marginals, while this paper concludes that different approximate inference methods can produce very different joint likelihoods with similar marginal performance. I suspect the difference here is in the sequence length $\\tau$ used to evaluate joint predictions; Wang et al appear to only evaluate over 5 data points, while this paper uses 100 to show major performance differences. It would be good to include additional results at different $\\tau$'s between 1 and 100 to see how the difference between joint and marginal predictions grows.\n\nAnother benchmark proposed by Wilson et al 2021 [2] also focus on evaluating the faithfulness of Bayesian posterior approximations, though they only evaluate marginal uncertainties. Nontheless, I find their benchmark has several advantanges: they study much larger scale networks (albeit this implies much higher computational cost) and also consider real world input distributions instead of only synthetic ones.\n\n**Regarding likelihoods as the evaluation metric**: While (joint) likelihoods are certainly a very natural choice for comparing predictions, what ultimately matters is how useful the predictions are in making downstream decisions. In addition to just measuring likelihoods, I think the benchmark would be a much stronger contribution if it also emphasized comparisons on downstream tasks, which can be synthetically generated similarly to the existing settings.\nExamples of possible downstream tasks that really test the quality of joint predictions could include active learning, contextual bandit, and Bayesian optimization tasks, plugging different posterior approximations into standard representative algorithms for each task (though Wang et al 2021 already do propose transductive active learning as a benchmark task for joint uncertainties). \nAnother set of potential tasks focusing more on the quality of marginal predictions could include tasks like selective classification, possibly in conjuction with different levels of synethetic covariate shift. Being able to accurately gauge the (relative) trustworthiness of predictions (especially with distribution shift) is certainly an important task for uncertainty estimation, even if it does not rely on the quality of joint predictions. \nWithin the scope of synethetically generated tasks, I think the inclusion of additional evaluations on a variety downstream tasks into a single consolidated benchmark would be a much stronger contribution, and emphasize how different applications of uncertainty estimation can have different needs and be more amenable to different algorithms.\n\n**Regarding trying to approximate true Bayesian posteriors:** Another hesitation I have is that the focus is that the target is given by quality is being measured against a ground truth consisting of random MLPs, whlie it is hard to say whether these pri\nors actually perform well when faced with real world data and tasks. For example, in Izmailov et al [3], they find that (in larger scale settings), exhaustively running HMC, with presumably better approximations of the true posterior, actually performs worse than approximate methods like ensembling when faced with distribution shift, suggesting simply trying to faithfully approximate exact inference is not necessarily the right (or at least only) goal we consider. \n\n**Scale and synthetic nature of the benchmark**: I also have reservations about the scale of the proposed benchmark, as it is focused on a small data regime with very simple models (MLPs). While the authors state that the synthetic results are predictive\n of real world data, the real world datasets evaluated are also quite toy in nature, and extending insights to larger scale models and datasets like in [2] is quite important.\n\n**Regarding likelihood evaluation in Algo 4**: I believe it would be helpful provide more explanation of Algorithm 4 in appendix A.1. Currently, there is little motivation for the different steps in the algorithm, and I would appreciate greatly if the authors could summarize the relevant concepts from high-dimensional geometry as applied here. Given the paper is proposing a new set of evaluations, it would be especially important to make sure that users have confidence that the metrics used are meaningful. If no formal analysis of the accuracy of the approximation is feasible, there should at least experimental comparisons with the direct Monte Carlo estimate (run with exhaustively many samples to estimate the ground truth) to compare the sample complexities of the proposed estimate at different sequence lengths, as well as documentation of how hyperparameters of the estimator were or should be determined.\n\nCitations:\n\n[1] Wang, Chaoqi, Shengyang Sun, and Roger Grosse. \"Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2021.\n\n[2] Andrew Gordon Wilson, Pavel Izmailov, Matthew D Hoffman, Yarin Gal, Yingzhen Li, Melanie F Pradier, Sharad Vikram, Andrew Foong, Sanae Lotfi, Sebastian Farquhar. \"Evaluating Approximate Inference in Bayesian Deep Learning.\" https://izmailovpavel.github.io/neurips_bdl_competition/.\n\n[3] Izmailov, Pavel, et al. \"What Are Bayesian Neural Network Posteriors Really Like?.\" arXiv preprint arXiv:2104.14421 (2021).\n~                                                                                                                                   ", "summary_of_the_paper": "The paper proposes a new benchmark for approximate inference methods, with emphasis on comparing the quality of joint predictions instead of marginals. The proposed testbed is comprised of synethetic test functions generated by simple MLPs, and authors show results on the synthetic benchmark are well correlated with results on small scale real world datasets as well.", "summary_of_the_review": "I believe the scope of the benchmark is quite limited, and the paper does not show any strong benefits provided by this benchmark over other already existing benchmarks.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ggyOzAKlK4g", "reviewer": "Reviewer_E6jF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "_Edit: Thank you for the rebuttal and detailed discussion. I consider the results an interesting contribution that is relevant to the field, which is why I vote for acceptance. The limited score is due to the presentation and discussion of topics in the paper not directly related to the theoretical contribution (see discussion below for details). A proper rewrite of the related sections to guide the reader to focus on the main topic marginal vs joint posterior predictive would strengthen the paper a lot._ \n\n___________\n\n## Strengths\n- The paper is overall well written and structured\n- The experiments demonstrate the difference between marginal and joint posterior predictive the authors ask the community to focus on\n- An extensive library with code is provided, which allows for the implementation and test of further methods\n\n## Weaknesses\n- The title reads a little bit too attention-grabbing, especially given that the paper itself does not actually target the question of whether Bayesian deep learning (BDL) works, but rather whether evaluating a marginal posterior predictive is sufficient, of whether one should consider joint posterior predictive. \n- The abstract claims to solve \"what roles epistemic versus aleatoric uncertainty play\", and the end of the introduction also claims to \"resolve a point of philosophical debate\" on this topic. However, can the authors comment to in which respect they do that? As far as I read it, the authors claim that epistemic and aleatoric uncertainty are highly dependent on the model (e.g. they differ depending on the model, yet can lead to the same predictive distributions for distinct models), and that to compare models, one should rely on the posterior predictive. (To be more precise in the rest of the paper, the joint posterior predictive.) But is that really a point that is being debated and needs to be resolved? That what is reducible and irreducible uncertainty is always conditioned on the model, and that we, therefore, need to focus on the posterior predictive to compare different models with each other properly seems to be completely obvious and not challenged by anybody to my knowledge. (Unfortunately, the supposed philosophical debate lacks all references.)  \n    The realization that we cannot usefully compare ir/reducible uncertainties between models does, of course, not at all tackle the question of whether it can be useful to distinguish between them within, i.e. conditioned on a specific model, to gain a deeper understanding of why the current model performs as it does, or how one can improve it (taking e.g. Depeweg et al. (2018) as an example of an active learning context). \n- Section 2.1 and 2.2 read as if they are due to the paper. However, both the $\\tau$th-order KL-loss as well as a minor variation of the example are due to Lu et al. (2021), which are only cited in passing without highlighting this fact.\n- Given the small nature of the nets further comparisons against an HMC giving access to  the _true_ posterior seems feasible and relevant in the set of models discussed, and if it is infeasible after all an SGHMC (Chen et al., 2014) could give a second point similar to the Langevin `sgmcmc` to have a richer exploration of the posterior. \n- Given the topic, Izmailov et al. (2021) seem highly relevant with similar results concerning ensembles and `sgmcmc` performances (abbreviated SGLD in their case). This prior work is missing completely in the discussion.\n\n\n## Specific additional questions to the authors\n- Can the authors clarify the differences/similarities with Lu et al., 2021?\n- Can the authors comment in greater detail on the claimed different observations between the current work and Wang et al. (2021)?\n- The nets used in the experiments are rather small. Do the authors have any indication that the findings remain stable with deeper nets?\n\n\n## Minor\n- Several parts of the paper are \"confusing\", \"puzzling\", \"surprising\", unclear as to what is \"really Bayesian\", and seem to be formulated with the primary goal of teasing \"Bayesian purists\". While this is a nice structure for a poster/conference talk to provoke some good discussions in the next break, it feels somewhat unnecessary in a paper. _(Feel free to ignore this complaint, as it is only my highly subjective prior)_\n- Similar to the comment on the teasing structure of the paper and the earlier comments on aleatoric/epistemic, the XKCD comic seems somewhat out of place. Especially given that it is introduced as if the deep learning community is its cause. While I do not know the original story behind Munroe's comic, the discussion of different types of sources of uncertainty is a lot older than our current deep learning popularization.\n- \"We see that ensemble ... actually provide much better approximations to the Bayesian posterior than 'fully Bayesian' VI approaches like `bbb`\". I would claim that this finding is not too surprising. Given that most VI approaches take a unimodal, mean-field approximation as their starting point, it is directly clear that the approximation to a highly multimodal posterior is terrible. It seems reasonable that an ensemble approach gives a richer signal as long as its members do not collapse to the same optimum. This has also been extensively evaluated and demonstrated by Izmailov et al. (2021). \n- ~~I might have overlooked it, but the hyperparameter details in the appendix seem to miss the number of samples from the posterior for the BNNs.~~ _Edit: All details are provided, I had just overlooked them. See the author response below_\n\n### Typos\n- The datasets in Sec 5.1 give (TFD) as a reference, while the references only contain the long-form TensorFlow Datasets.\n\n\n\n______\nDepeweg et al. Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning, ICML 2018  \nIzmailov et al. What Are Bayesian Neural Network Posteriors Really Like?, ICML 2021", "summary_of_the_paper": "The authors discuss whether it is sufficient to consider the marginal posterior predictive vs considering a joint posterior predictive when evaluating Bayesian deep learning approaches. Introducing a set of experiments (_The Neural Testbed_), they demonstrate that the performance of common approaches can differ greatly depending on which of these predictive distributions are evaluated. Additionally, extensive code is provided for efficient implementation and evaluation of new models.\n", "summary_of_the_review": "An overall interesting paper in a new direction (after the contribution of Lu et al. is clarified) to evaluate BDL models that tries a bit too hard to be controversial. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "W6vghH1jSzO", "reviewer": "Reviewer_Etjg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\nBayesian deep learning is an important research area but the evaluations of the progress made in the field has not been consistent so far which makes it difficult to evaluate different techniques. Furthermore some of the common practice on evaluations on out of distribution data provides insight but lacks providing a quantitative benchmark. The paper addresses an important problem in Bayesian deep learning and provides a framework and tools to evaluate different techniques. The framework is based on simulations which helps provides control over variables and enables gathering insights about various techniques. Furthermore, the paper extend the evaluations to sequential decisions which makes the contributions unique.\n\nOverall I found the contributions of the paper very important for the field. I also think the paper is very written and organized.\n\nCons:\n\nEven though in the abstract the authors mention that the proposed method provides insights into aleatory and epistemic uncertainty, the manuscript does not elaborate on this point. Even without this point, I think the paper is interesting and removing this point will not hurt its reach.\n\nBayesian deep learning researchers are in general interested in out of distribution generalization. I found the discussion on OOD a bit limited. I believe the proposed framework can also enable evaluations with OOD and I would recommend authors adding a discussion on this.\n\nMinor comments:\n- Abstract: Please explicitly define what you mean by \"joint predictions\".\n- Figure 6. Would be interesting to color code the agents in the Figure.", "summary_of_the_paper": "The paper proposes a simulation based framework to evaluate different techniques proposed for uncertainty estimation of predictive models. The approach relies on simulated data to control effects such as environments where data is collected, data and model uncertainty. This control enables generating interesting insights about various techniques. \n\n The paper also go beyond evaluating marginal posterior predictive distributions and extend their benchmarking work to joint distributions capturing sequential decisions that can be made with such models. Some of the highlights from the results: 1) Their results show that Bayesian deep learning is impactful for capturing the joint predictive distributions. 2) Priors used in ensemble+ help with diversity and therefore enable better predictive distributions. 3) Bootstrapping help with robustness of predictions if the models hyperparameters are not tuned.", "summary_of_the_review": "Overall, I vote for accepting. I like the idea of using simulated settings with explicit control on variables to perform benchmark analysis and this paper enables this through a novel framework as well as open source tools. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "HccIltXv0hx", "reviewer": "Reviewer_4oj5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- This paper is generally well-written and clear. Evaluating predictive distributions is an important and relevant topic to the general UQ community.\n- As far as I can tell, the idea of evaluating predictive distributions via joint distributions over the numerous predictions is quite novel, as well as the proposed algorithm.\n- The experiments section does a good job in demonstrating the discrepancies that can arise between marginal and joint predictions during evaluation, and these findings are interesting.\n\nWeaknesses:\n\n- As this work propose an evaluation metric, I believe it's important to have a discussion on the suite of metrics that are currently used in UQ and how the proposed metric provides an advantage over existing ones. This paper only touches upon KL-divergence (equivalently, cross-entropy, or likelihood), and doesn't mention other widely used metrics, such as calibration, or other proper scoring rules.\n- In fact, would calibration (+multiples notions exist, e.g. ECE, classwise-ECE, adaptive ECE, ...)  of the marginal distributions also fail to differentiate between the different methods in Figure 2?\n- What is the significance of using $\\tau=100$ for the joint predictions? Would there be discrepancies in the ranking of methods based on other $\\tau$'s? In practice, if this metric was used for model selection, which $\\tau$ should the practitioner base the metric on to choose a model?\n- On the same note, I believe there needs to be more demonstration/elaboration that a model with better joint predictions is \"necessarily better\" than another model that has worse joint predictions, but possibly as good or better marginal predictions. In the second paragraph of page 2, the authors hint that some Bayesian models are not performant in sequential decision tasks because they have poor joint predictions. The proposed metric would be more convincing if this point was followed up with, or if other downstream tasks were demonstrated in the experiments.\n\nQuestions on content:\n\n- In Section 4.1, what does the environment variable $\\varepsilon$ represent, concretely? Is it the choice of SNR and the initialization distribution over the data generating NN? Let me know if I am missing something here.\n- In the last sentence of the second paragraph of Section 4.1, I don't understand what is meant by \"agent is provided with the data generating process\"? How is this done, concretely?\n- In Step 1 of Algorithm 4, what exactly is the distribution, $\\mathbb{P}(\\hat{\\varepsilon} | f_{\\theta_{T}})$? How is this distribution produced?\n\nOther points:\n\n- The font and spaces between lines for this submission are quite different from the default style. However, as far as I can tell, it seems to reduce space for content, so I did not flag it.\n- Table 1 protrudes beyond the side margins — I'm not sure if this isn't a formatting violation.\n- In Figure 6(b), which $\\tau$ setting was chosen each of the blue points?", "summary_of_the_paper": "This work advocates evaluating predictive distributions via joint predictions (rather than the standard practice of evaluating marginal predictions) and introduces The Neural Testbed, an open-source software which includes the testing suite, along with implementations of a handful of methods in uncertainty quantification (UQ). The core evaluation metric used is KL divergence (cross-entropy loss) between the predictive distribution and the true likelihood of the data-generating process, and this work proposes an algorithm to compute this metric with joint distributions. The empirical evaluations compare numerous standard UQ methods with the Testbed, with both synthetic and real datasets.", "summary_of_the_review": "As noted in the section above, while I do think this is interesting work with interesting findings, I am not fully convinced of the merits of the metrics proposed. Further comparisons and discussion of other existing metrics, along with demonstrations of the benefits of better joint predictions in a downstream task would be helpful in making a stronger case for this work. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper describes a new testbed to evaluate Bayesian techniques in the context of joint predictive distribution.  Since this is not the first paper that considers marginal vs joint distribution evaluation, the paper should include a thorough discussion of the differences with prior work.  The paper simply states that it refutes Wang et al.'s previous observation that joint distributions do not distinguish techniques much more than marginals.  However, the paper does not really explain why their observation is correct and Wang's observation should be discarded.  Since this is the core contribution of the paper and it is doubtful, this is problematic.  The discussion of epistemic/aleatoric uncertainty also seems superfluous and therefore distract the reader.
Decision: Reject