Generative Posterior Networks for Approximately Bayesian Epistemic Uncertainty Estimation

1 INTRODUCTION . Deep learning systems are becoming ubiquitous in many industries , specifically in applications with large datasets and where prediction mistakes are inexpensive . However , for many other industries , such as autonomous manufacturing , where datasets are smaller and mistakes can be catastrophic , the adoption of deep learning techniques has been slow . One of the critical challenges to applying deep learning models in these safety-critical domains is the lack of uncertainty estimation . If a model makes a prediction in a safety critical environment , we would like that model to be highly confident that prediction is correct . Uncertainty prediction can also provide significant benefit in data-poor domains by guiding exploration towards areas of uncertainty . Most deep learning models , however , are not able to estimate epistemic uncertainty – uncertainty deriving from lack of data samples . In this work , we introduce a new method for modeling epistemic uncertainty in neural networks . We formulate this problem as a Bayesian Inference problem : given a prior distribution over functions and a set of training data , we want to model the posterior function distribution . There are many existing approaches to constructing such a posterior , such as Gaussian Processes ( GPs ) or Variational Inference ( VI ) methods , but these methods can require significant engineering to work in practice and can be expensive in high-dimensional settings . For this reason , many practitioners turn to using ensembles of neural networks to approximate epistemic uncertainty for their ease of use in highdimensional settings . Pearce et al . ( 2020 ) illustrated that , given the correct regularization , these ensemble members approximate samples from the posterior distribution of functions . However , using this technique , every new sample of the posterior requires training a new neural network from scratch , which can be prohibitively expensive for large problems . To address this challenge , we introduce Generative Posterior Networks ( GPNs ) , a generative neural network model that directly approximates the posterior distribution by regularizing the output of the network towards samples from the prior distribution . By learning a low-dimensional latent representation of the posterior , our method can quickly sample from the posterior and construct confidence intervals . We prove that our method approximates the Bayesian posterior over functions and show empirically that our method can improves uncertainty estimation over competing methods . 2 RELATED WORK . There are two categories of uncertainty in the context of probabilistic modeling : aleatoric uncertainty and epistemic uncertainty . Aleatoric uncertainty refers to uncertainty inherent in a system , even when the true parameters of the system are known , while epistemic uncertainty refers to modeling uncertainty that can be reduced through collecting more data ( Der Kiureghian & Ditlevsen , 2009 ; Gal , 2016 ) . Deep learning models have shown impressive performance in quantifying aleatoric uncertainty when given enough data . On the other hand , despite many methods seeking to estimate epistemic uncertainty due to its well-motivated applications in Bayesian Optimization ( Snoek et al. , 2012 ; Springenberg et al. , 2016 ) , Reinforcement Learning ( Curi * et al. , 2020 ) , and adversarial robustness ( Stutz et al. , 2020 ) , quantifying it remains a challenging problem for deep learning models . Gaussian Processes ( GPs ) remain one of the most effective methods for epistemic uncertainty estimation in low-dimensional , low-data settings . While scalability can be improved with interpolation methods ( Hensman et al. , 2013 ; Titsias , 2009 ; Wilson & Nickisch , 2015 ) and kernel learning methods ( Wilson et al. , 2016 ) , GPs tend to perform worse than neural network-based methods on high-dimensional problems . Another way to formulate epistemic uncertainty estimation is as a Bayesian Inference problem . Specifically , given a prior distribution over functions and some labeled data , the goal is to construct a posterior distribution in the Bayesian sense . Because this posterior distribution is usually intractable to compute exactly , approximate sampling methods are used instead . Markov-Chain Monte Carlo ( MCMC ) ( Tanner & Wong , 1987 ) and Variational Inference ( VI ) ( Blei et al. , 2017 ) are some of the most common methods for posterior sampling . These methods , however , tend to struggle in high dimensional problems . The computational complexity of MCMC scales poorly with the dimension of the data and VI methods tend to require restrictive function families with strong independence assumptions . Bayesian Neural Networks ( BNNs ) ( Blundell et al. , 2015 ) are another way to perform Bayesian Inference a neural networks by modeling each parameter in the network with an independent Gaussian distribution . While BNNs perform well in low dimensional settings , their performance tends to degrade in high-dimensional settings that require large networks . Gal & Ghahramani ( 2016 ) proved that dropout can also be used as a form of Bayesian Inference . Specifically , if a network is trained using dropout and L2 regularization , posterior samples can be collected by applying dropout at test-time . This method has surprisingly good out-of-distribution detection and can be applied to any network . However , it can be hard to tune the dropout parameter to have good uncertainty prediction without significantly degrading performance . Neural network ensembling is another commonly used epistemic uncertainty estimation technique , which is closely related to our proposed method in this work . This technique involves training a number of completely separate neural networks on the same data to form an ensemble . Because of the stochasticity of initialization and the training process , each member of the ensemble should be a slightly different function . Specifically , the ensemble members should produce similar outputs in data-dense regions of the state-space and dissimilar outputs in data-sparse regions . Pearce et al . ( 2020 ) proved that , by using the correct regularization , each ensemble member becomes an approximate sample from the function posterior . Ensembling methods are particularly attractive to practitioners because they require very little fine-tuning and work surprisingly well in practice . However , these methods have one main drawback , namely that each new sample from the posterior requires training a new network from scratch . Our method , on the other hand , seeks to construct a generative posterior model using similar regularization techniques to Pearce et al . ( 2020 ) , allowing for quick sampling from the posterior . 3 PROBLEM STATEMENT AND BACKGROUND . We will focus on the problem of Bayesian Inference on the parameters of θ of some function f ( · ; θ ) . We assume we have some known prior over parameters Pprior ( θ ) ∼ N ( µprior , Σprior ) as well as access to a noisy dataset ( xobs , yobs ) = { ( x1obs , y1obs ) . . . , ( xNobs , yNobs ) } , where yiobs = f ( xiobs ; θ ) + , ∼ N ( 0 , σ ) for some unknown parameter θ . We can then define the data-likelihood as Plike ( yobs |θ , xobs ) = ∏ i N ( yiobs | f ( xiobs ; θ ) , σ ) . ( 1 ) The goal is to approximate samples from the posterior , Ppost ( θ |yobs , xobs ) ∝ Plike ( yobs |θ , xobs ) Pprior ( θ ) . ( 2 ) For ease of notation , we will drop the xobs from the likelihood . For our analysis , it is convenient if the posterior distribution is Gaussian . To show this , we will use the fact that the product of two Gaussian PDFs is proportional to a Gaussian PDF . We make a distinction here between the data-likelihood , a function of the observed data yobs , and the parameterlikelihood , a function of the parameters θ , defined as follows : Pparam-like ( θ ; yobs ) = Plike ( yobs |θ ) . ( 3 ) The parameter-likelihood PDF is equivalent to the data-likelihood PDF for all values of yobs and θ , but is notably a function of the parameters θ . For a more detailed explanation of the distinction between the parameter and data-likelihoods , see Definition 1 from Pearce et al . ( 2020 ) . When the function f is linear , it can be shown that the parameter-likelihood is also Gaussian , Pparam-like ( θ ; yobs ) ∝ N ( µlike , Σlike ) . ( 4 ) While this is not true in general , Pearce et al . ( 2020 ) assume that the parameter likelihood is Gaussian so that the posterior also becomes Gaussian , Ppost ( θ |yobs ) ∝ Pparam-like ( θ ; yobs ) Pprior ( θ ) ∝ N ( µlike , Σlike ) N ( µprior , Σprior ) = N ( µpost , Σpost ) , ( 5 ) where the posterior covariance and mean are given by Σpost = ( Σ−1like + Σ −1 prior ) −1 , µpost = ΣpostΣ −1 likeµlike + ΣpostΣ −1 priorµprior . ( 6 ) Note , however , that after the reformulation of the problem in Section 4.1 , we no longer require this assumption for our theoretical analysis . 3.1 RANDOMIZED MAP SAMPLING ( RMS ) . Since the values µlike and Σlike are intractible to compute in practice , the challenge of computing the posterior distribution remains . However , Pearce et al . ( 2020 ) showed how we can use a method called Randomized MAP sampling ( RMS ) to approximately sample from the posterior distribution . With RMS we first assume access to a mechanism to compute the maximum a posteriori ( MAP ) solution for our posterior . Then , by randomly shifting the prior distribution we use in our MAP solver , we can estimate samples from the posterior . Specifically , RMS samples “ anchor points ” θanc from some distribution θanc ∼ N ( µanc , Σanc ) . Each anchor point defines a shifted prior distribution Panc ( θanc ) = N ( θanc , Σprior ) . Now we define the MAP function which finds the MAP solution using this shifted prior : θMAP ( θanc ) = arg max θ Plike ( yobs |θ ) Panc ( θ ) . ( 7 ) By sampling different anchor points , we can then construct a probability distribution over MAP solutions P ( θMAP ( θanc ) ) , which can be shown to be Gaussian . Using moment matching , Pearce et al . ( 2020 ) proved , in Theorem 1 , that if we choose µanc = µprior and Σanc = Σprior + ΣpriorΣ−1likeΣprior , then P ( θMAP ( θanc ) ) = Ppost ( θ |yobs ) . Using the approximation Σanc ≈ Σprior , a sample from posterior , θpost , can be collected by sampling θanc ∼ N ( µanc , Σanc ) and optimizing the following : θpost = arg max θ logPparam-like ( θ ; yobs ) + logPanc ( θ ) . ( 8 ) 4 GENERATIVE POSTERIOR NETWORKS . While RMS can be used to construct samples from the posterior , every new sample requires solving a new optimization problem . Instead , we propose learning the MAP function itself . Let g be a neural network parameterized by some vector , φ , that takes as input a sample from the anchor distribution , θanc ∼ Panc ( θanc ) , and outputs θMAP ( θanc ) . If we assume that g has enough expressive power to represent the MAP function , then g ( θanc ; φ ) = θMAP ( θanc ) if : φ = arg max φ Eθanc∼Panc ( θanc ) logPlike ( yobs | g ( θanc ; φ ) ) + logPanc ( g ( θanc ; φ ) ) . ( 9 ) In practice , however , this is a very hard optimization problem for two main reasons . ( 1 ) Optimizing in parameter space tends to be challenging for gradient-descent optimization . ( 2 ) The space of all parameters θanc is not only very large , but there are also many parameters θanc that map to the same outputs for all inputs . Moreover , we would expect the posterior parameters to be highly correlated and , therefore , need less expressive power to be represented . To address these two problems , we will first change the problem definition slightly to find the posterior in output space as opposed to parameter space . We prove in the next section that , by changing our loss function , we can still use RMS to approximate the posterior in output space . Next , we show practically how we construct a low-dimensional representation of the anchor distribution .

Summary: This paper built upon Pearce et al 2020 that shows regularizing parameters to prior distributions in ensemble of NNs results in the approximate Bayesian inference, aiming to improve computational efficiency of the previous method. Specifically, in the method in Pearce et al, sampling one parameter from posterior requires to find a MAP solution for some anchor point, which can be computationally demanding. Therefore, the authors propose to learn MAP function from an anchor point to MAP solution, which removes the required MAP optimization procedure.