Evaluating Predictive Distributions: Does Bayesian Deep Learning Work?

1 Introduction . Deep learning has emerged as the state-of-the-art approach across a number of application domains in which agents learn from large amounts of data ( LeCun et al. , 2015 ) . Neural networks are increasingly used not only to predict outcomes but also to inform decisions . Common approaches in deep learning produce point estimates but not uncertainty estimates , which are often required for effective decision-making . Bayesian deep learning extends the methodology to produce such uncertainty estimates ( MacKay , 1992 ; Neal , 2012 ) . We consider agents that are trained on data pairs ( ( Xt , Yt+1 ) : t = 0 , 1 , . . . , T − 1 ) and subsequently generate predictions given new inputs . When presented with an input XT , a Bayesian neural network can generate a predictive distribution of the outcome YT+1 that is yet to be observed . This distribution characterizes the agent ’ s uncertainty about YT+1 . We refer to such a prediction as marginal to distinguish it from a joint predictive distribution over a list ( YT+1 , . . . , YT+τ ) of prospective outcomes corresponding to inputs ( XT , . . . , XT+τ−1 ) . The importance of uncertainty estimation has motivated a great deal of research over recent years ( Kendall & Gal , 2017 ) . This research has produced a variety of agents that learn to generate predictive distributions . With this proliferation of alternatives , it is increasingly important to analyze and compare their performance ( Filos et al. , 2019 ; Nado et al. , 2021 ) . In this paper , we introduce new tools for systematic evaluation of such agents . Our tools overcome several limitations faced by previous methods of evaluation . First , by focusing purely on predictive distributions , we allow for a unified treatment of approaches developed within the Bayesian neural network community and beyond . This sidesteps the Open source code available at https : //anonymous.4open.science/r/neural-testbed-B839 . question of whether any approach ‘ is really Bayesian ’ ( Wilson & Izmailov , 2020 ) . Second , our tools evaluate the quality of higher-order joint predictions ( τ > 1 ) . Until now , the Bayesian deep learning literature has focused almost exclusively on evaluating marginal predictions ( Wang et al. , 2021 ) . Finally , we develop a neural-network-based data generating process for Bayesian deep learning that can be used to drive insight and algorithm development . Where research has focused on a small set of challenge datasets , this might introduce bias through overfitting via multiple iterations of algorithm development . We use these tools to compare hundreds of agent variants . Further , we show that performance on our synthetic data generating process data is highly correlated with performance on real-world challenge datasets . We opensource all code used in this paper as The Neural Testbed . Our results reconcile several sources of confusion in the field . One concerns why particular approaches developed by the Bayesian deep learning community , such as Bayes-by-backprop , dropout , and deep ensembles , perform poorly in sequential decision tasks despite faring well based on evaluation metrics of that community ( Osband et al. , 2018 ) . Our results demonstrate that , while such methods produce accurate marginal predictions , they are no longer competitive when it comes to high-order joint predictions . Joint predictions play a critical role in sequential decision-making ( Lu et al. , 2021 ) . Another puzzling issue is that state-of-the-art methods do not employ domain-specific priors . Whether Bayesian deep learning approaches should at all is a subject of controversy ( Wenzel et al. , 2020 ) . We show that the benefits of domain-specific priors can be pronounced when evaluating high-order joint predictions , even where they are negligible for marginals . We also help to resolve a point of philosophical debate within the deep learning community : the importance of epistemic versus aleatoric uncertainty1 . The strangeness of this distinction has even made its way into wider popular culture , as satirized in the XKCD comic of Figure 1 ( Munroe , 2021 ) . For a given parametric model , we can clearly distinguish parameter uncertainty from noise , or reducible from irreducible uncertainty . However , from the perspective of a learning agent , the choice of model is subjective ; different models can lead to the same marginal predictions . Our formulation provides a clear and objective way to assess the quality of predictive distributions , without reliance on this subjective distinction between knowledge and chance . Crucially , we show that this can be judged via the quality of joint predictions , but that marginals are not sufficient . It is worth mentioning another notable contribution of this work . The quality of a predictive distribution is commonly assessed in terms of cross-entropy loss . While this measure is welldefined for both marginal and joint predictions , to the best of our knowledge , the literature has only addressed computation in the former case . For high-order joint predictions , the straightforward approach would require computing sums over exponentially many values . To render this computationally tractable , we developed a novel approximation algorithm that leverages a random partitioning operation and Monte Carlo simulation . While this approach is motivated by concepts from high-dimensional geometry ( Kaski , 1998 ; Donoho , 2006 ) , we leave its analysis as a topic for future theoretical research . 1Epistemic uncertainty relates to knowledge ( ancient Greek episteme↔knowledge ) , as opposed to aleatoric uncertainty relating to chance ( Latin alea↔dice ) ( Der Kiureghian & Ditlevsen , 2009 ) . 2 Evaluating predictive distributions . In this section , we introduce notation for the standard supervised learning framework we will consider ( classification ) as well as our evaluation metric ( the KL-loss ) . We also explain how we estimate the KL-loss for high-order joint predictions where exact computation is infeasible , using random partitions and Monte Carlo simulation . 2.1 Kullback–Leibler loss . Consider a sequence of pairs ( ( Xt , Yt+1 ) : t = 0 , 1 , 2 , . . . ) , where each Xt is a feature vector and each Yt+1 is its target label . This sequence is i.i.d . conditioned on the environment E , which produces the data , and which we view as a latent random variable . We consider an agent that is uncertain about the environment and predicts class labels YT+1 : T+τ ≡ ( YT+1 , . . . , YT+τ ) given training data pairs DT ≡ ( ( Xt , Yt+1 ) : t = 0 , 1 , 2 , . . . , T − 1 ) and unlabelled feature vectors XT : T+τ−1 ≡ ( XT , . . . , XT+τ−1 ) . From the agent ’ s perspective , each feature vector Xt is generated i.i.d from a fixed distribution P ( Xt ∈ · ) , and each class label Yt+1 is then drawn from P ( Yt+1 ∈ ·|E , Xt ) . We describe the agent ’ s predictions in terms of a generative model , parameterized by a vector θT that the agent learns from the training data DT . For any inputs XT : T+τ−1 , θT determines a predictive distribution , which could be used to sample imagined outcomes ŶT+1 : T+τ . We define the τ th-order expected KL-loss by dτKL =E [ dKL ( P ( YT+1 : T+τ ∈ ·|E , XT : T+τ−1 ) ︸ ︷︷ ︸ environment likelihood ∥∥P ( ŶT+1 : T+τ ∈ ·|θT , XT : T+τ−1 ) ︸ ︷︷ ︸ agent likelihood ) ] ( 1 ) =−E [ log ( P ( ŶT+1 : T+τ = YT+1 : T+τ ∣∣∣θT , XT : T+τ−1 , YT+1 : T+τ ) ) ] ︸ ︷︷ ︸ cross-entropy loss ≡ negative log-likelihood + C , where C = E [ log ( P ( YT+1 : T+τ |E , XT : T+τ−1 ) ) ] is independent of θT . The expectation is taken over all random variables , including the environment E , the parameters θT , XT : T+τ−1 , and YT+1 : T+τ . Note that dτKL is equivalent to the widely used notion of cross-entropy loss , though offset by a quantity that is independent of θT ( Kullback & Leibler , 1951 ) . For τ > 1 , dτKL assesses joint rather than the marginal predictions . 2.2 Marginal Versus Joint Predictions . Evaluating an agent ’ s ability to estimate uncertainty on joint instead of marginal predictions can result in very different answers . We provide a simple example that illustrates the point . Suppose the data ( ( Xt , Yt+1 ) : t = 0 , 1 , 2 , . . . ) is generated by repeated tosses of a possibly biased coin with unknown probability p of heads.2 Let Xt = 0 , to indicate that there is no input , and let each outcome Yt+1 be 0 or 1 to indicate tails or heads , respectively . Consider two agents that , without any training , predict outcomes . Agent 1 assumes p = 2/3 and models the outcome of each flip as pure chance . Agent 2 assumes that the coin is fully biased , meaning that p ∈ { 0 , 1 } , but assigns probabilities 1/3 and 2/3 to 0 and 1 . Let Ŷ 1t+1 and Ŷ 2t+1 denote the outcomes imagined by the two agents . Despite their differing assumptions , the two agents generate identical marginal predictive distributions : P ( Ŷ 1t+1 = 0 ) = P ( Ŷ 2t+1 = 0 ) = 1/3 . On the other hand , joint predictions greatly differ for large τ : P ( Ŷ 11 = 0 , .. , Ŷ 1τ = 0 ) = 1/3τ 1/3 = P ( Ŷ 21 = 0 , . . . , Ŷ 2τ = 0 ) . We can say that agent 1 attributes all uncertainty to aleatoric sources and agent 2 , epistemic sources ( although as Figure 1 alludes , there are many ways an agent can attribute sources of uncertainty ) . Evaluating marginal predictions can not distinguish between the two possibilities , though for a specific prior distribution over p , one agent could be right and the other wrong . One must evaluate joint predictions to make this distinction . 2We consider this coin as an illustrative model of more complex binary outcomes , such as whether a user will click on an ad , or whether a given mortgage will default on payments . When it comes to decision-making , this distinction can be critical ( Lu et al. , 2021 ) . In a casino , under the first agent ’ s assumption , there is large upside and little risk on repeatedly betting on heads in the long run . However , if there is a 1/3 chance the coin will always land tails , as is the case in the second agent ’ s prediction , there is a ruinous risk to repeatedly betting heads . Evaluating joint predictions beyond marginals distinguishes these cases . 2.3 Computation of Kullback–Leibler loss . In contexts we will consider , it is not possible to compute dτKL exactly . As such , we will approximate dτKL via Monte Carlo simulation . This section provides a high level overview of our approach , we push the full details to Appendix A. Algorithm 1 outlines a basic approach to estimating dτKL with respect to a synthetic data generating process . The algorithm samples a set of environments and a training dataset for each environment . For each of these pairs , the agent is re-initialized , trained , and then tested on N independent test data τ -samples . Note that each test data τ -sample includes τ data pairs . For each test data τ -sample , the likelihood of the environment is computed exactly , but that of the agent ’ s belief distribution is approximated . The estimate of dτKL is taken to be the sample mean of the log-likelihood-ratios ( Algorithm 2 ) . Algorithm 1 KL-Loss Computation 1 : for j = 1 , 2 , . . . , J do 2 : sample environment and training dataset , and train agent 3 : for n = 1 , 2 , . . . , N do 4 : sample a test data τ -sample with τ feature-label pairs 5 : compute pj , n . likelihood of environment 6 : compute p̂j , n . estimated likelihood of agent ’ s belief distribution 7 : return 1JN ∑J j=1 ∑N n=1 log ( pj , n/p̂j , n ) . estimated log-likelihood-ratio While the likelihood of an environment can be efficiently computed , that of an agent ’ s belief distribution poses a computational challenge . One approach is to estimate this likelihood via Monte Carlo simulation ( Algorithm 3 ) . This produces unbiased estimates , which can be accurate when τ is small . However , maintaining accuracy requires the number of samples to grow exponentially with τ , as discussed in Appendix A.1 . To overcome this challenge , we propose a novel approach that estimates the likelihood of the agent ’ s beliefs via a combination of randomized partitioning and Monte Carlo simulation ( Algorithm 4 ) ( Kaski , 1998 ) . We conjecture that , under suitable regularity conditions , this novel approach produces accurate estimates even when τ is large , but leave a formal analysis to future work . Even though Algorithm 1 is developed for a synthetic data generating process , it is straightforward to extend it to evaluate agents on real data . We outline our approach to real data in Section 5.1 , with full details in Appendix A.2 .

Summary: The paper proposes a simulation based framework to evaluate different techniques proposed for uncertainty estimation of predictive models. The approach relies on simulated data to control effects such as environments where data is collected, data and model uncertainty. This control enables generating interesting insights about various techniques.    The paper also go beyond evaluating marginal posterior predictive distributions and extend their benchmarking work to joint distributions capturing sequential decisions that can be made with such models. Some of the highlights from the results: 1) Their results show that Bayesian deep learning is impactful for capturing the joint predictive distributions. 2) Priors used in ensemble+ help with diversity and therefore enable better predictive distributions. 3) Bootstrapping help with robustness of predictions if the models hyperparameters are not tuned.