Quantifying uncertainty with GAN-based priors

1 INTRODUCTION . Quantifying uncertainty in an inference problem amounts to making a prediction and quantifying the confidence in that prediction . In the context of an image recovery problem , this may be understood as follows . A typical computer vision algorithm uses a noisy version of an image and prior knowledge to produce the recovered image which can be interpreted as the “ best guess ” of the original image . Quantifying uncertainty in this context involves generating an estimate of the level of confidence in the best guess , in addition to the guess itself . Bayesian inference provides a principled approach for quantifying uncertainty . As shown in the following section , it treats the inferred vector as a multivariate stochastic vector and leads to an expression for its distribution . This expression can be used to estimate the most likely solution ( the maximum a-posteriori estimate , or the MAP ) , the mean , the variance , or any other population parameter of interest . Thus providing a recipe for thoroughly quantifying the uncertainty in an inference problem . For the image recovery problems considered in this paper , Bayesian inference not only provides the best guess of the true image , but also a means to estimate measures of uncertainty such as the pixel-wise variance . The knowledge of uncertainty in a prediction can directly influence the downstream action that depends on the inference . Consider an image recovery problem where two distinct inputs lead to similar recovered images : those of a traffic sign with a high speed limit . However , for the first input the predicted variance is small , while for the second input it is large . Further , the set of likely images in the second set also includes images of a Stop Sign . Then the appropriate action for the two inputs , determined after solving the inference problem and quantifying uncertainty , is very different . For the first input , the appropriate action is one of continued motion , whereas for the second input it is to slow down . Similar examples can be drawn from other areas as well , like medical imaging , high frequency trading , and autonomous systems where critical decisions are made based on the output of AI system ( Gal ( 2016 ) ; Begoli et al . ( 2019 ) ) . The knowledge of uncertainty can also be useful in determining the optimal location of a sensor . Consider an image recovery problem , where the goal is to infer the signal , and associated uncertainty , using limited amount of measurement data . In this problem a user can leverage information about the spatial distribution of uncertainty to choose the location with maximum uncertainty as next measurement location . This task falls within the fields of active learning and/or design of ex- periments ( DeGroot et al . ( 1962 ) ; Houlsby et al . ( 2011 ) ) and is particularly useful in applications like satellite imaging , where each measurement requires significant time and/or resources . In Figure 1 , we demonstrate how the proposed GAN-based Bayesian inference algorithm is useful in both scenarios explained above : ( i ) in quantifying uncertainty - “ pixel-wise variance ” - our quantitative measure of uncertainty in the inferred field , and ( ii ) in determining the optimal sensor placement location in an iterative fashion . We return to these applications in greater detail in Section 4 . 1.1 BAYESIAN INFERENCE . Bayesian inference is a well-established technique for quantifying uncertainties in inference problems ( Kaipio & Somersalo ( 2006 ) ; Dashti & Stuart ( 2016 ) ; Polpo et al . ( 2018 ) ) . It has found applications in diverse fields such as geophysics ( Gouveia & Scales ( 1997 ) ; Malinverno ( 2002 ) ) , climate modeling ( Jackson et al . ( 2004 ) ) , chemical kinetics ( Najm et al . ( 2009 ) ) , heat conduction ( Wang & Zabaras ( 2004 ) ) , and the detection and diagnosis of disease ( Siltanen et al . ( 2003 ) ; Kolehmainen et al . ( 2006 ) ) . The two critical ingredients of a Bayesian inference problem are - an informative prior representing the prior belief about the parameters to be inferred and an efficient method for sampling from the posterior distribution . In this manuscript we describe how deep generative adversarial networks ( GANs ) can be effectively used in these roles . Consider the setting where we wish to infer a vector of parameters x ∈ RN from the measurement of a related vector y ∈ RP , where the two are related through a forward model y = f ( x ) . A noisy measurement of y is denoted by ŷ = f ( x ) + η , where the vector η ∈ RP represents noise . While the forward map f is typically well-posed , its inverse is not , and hence to infer x from the measurement ŷ requires techniques that account for this ill-posedness . Classical techniques based on regularization tackle this ill-posedness by using additional information about the sought solution field explicitly or implicitly ( Tarantola ( 2005 ) ) . Bayesian inference offers a different approach to this problem by modeling the unknown solution as well as the measurements as random variables . This framework addresses the ill-posedness of the inverse problem , and allows for the characterization of the uncertainty in the inferred solution . The notion of a prior distribution plays a key role in Bayesian inference . Through multiple observations of the field x , denoted by the set S = { x ( 1 ) , · · · , x ( S ) } , we have some prior knowledge of x that can be utilized when inferring x from ŷ . This is used to build , or intuit , a prior distribution for x , denoted by ppriorX ( x ) . Some typical examples include Gaussian process prior with specified co-variance kernels , Gaussian Markov random fields ( Fahrmeir & Lang ( 2001 ) ) , Gaussian priors defined through differential operators ( Stuart ( 2010 ) ) , and hierarchical Gaussian priors ( Marzouk & Najm ( 2009 ) ; Calvetti & Somersalo ( 2008 ) ) . These priors promote smoothness and/or structure in the inferred solution and importantly , can be expressed explicitly in an analytical form . Another key component of Bayesian inference is a distribution that represents the likelihood of y given an instance of x , denoted by pl ( y|x ) . This is often determined by the distribution of the error in the model , denoted by pη , which captures both model and measurement errors . Given this , and an additive model for noise , the posterior distribution of x , determined using Bayes ’ theorem after accounting for the observation ŷ is given by , ppostX ( x|y ) = 1 Z pl ( y|x ) ppriorX ( x ) = 1 Z pη ( ŷ − f ( x ) ) ppriorX ( x ) . ( 1 ) Here , Z is the prior-predictive distribution of y and ensures that the posterior integrates to one . The posterior distribution characterizes the uncertainty in x ; however for vectors of large dimension characterizing this distribution explicitly is a challenging task . Consequently the expression above is used to perform tasks that are more manageable . These include determining estimates such as the maximum a-posteriori estimate ( MAP ) , expanding the posterior distribution in terms of other distributions that are simpler to work with ( Bui-Thanh et al . ( 2012 ) ) , or using techniques like Markov Chain Monte-Carlo ( MCMC ) to generate samples that are “ close ” to the samples generated by the true posterior distribution ( Han & Carlin ( 2001 ) ; Parno & Marzouk ( 2018 ) ) . Despite its numerous applications in solving inverse problems , Bayesian inference faces significant challenges . These include defining a reliable and informative prior distribution for x when the set S = { x ( 1 ) , · · · , x ( S ) } is difficult to characterize analytically , and efficiently sampling from the posterior distribution when the dimension of x is large ; a typical situation in many practical science and engineering applications . 1.2 RELATED WORK . The main idea developed in this paper tackles the above mentioned challenges by training a generative adversarial network ( GAN ) using the sample set S , and then using the distribution learned by the GAN as the prior distribution in Bayesian inference . The use of sample-based priors for solving an inverse problem has a rich history ( Vauhkonen et al . ( 1997 ) ; Calvetti & Somersalo ( 2005 ) ) . As does the idea of reducing the dimension of the parameter space by mapping it to a lower-dimensional space ( Marzouk & Najm ( 2009 ) ; Lieberman et al . ( 2010 ) ) . However , the use of learning-based deep generative models like GANs in these tasks is novel . Recently , several authors have considered the use of learning-based methods for solving inverse problems arising in different domains . These include the use of deep convolutional neural networks ( CNNs ) , recurrent neural networks ( RNNs ) or variational autoencoders ( VAEs ) to solve physicsdriven inverse problems ( Adler & Öktem ( 2017 ) ; Patel et al . ( 2019 ) ; Pesah et al . ( 2018 ) ) . VAEs and GANs have also been used to solve inverse problems in computer vision ( Yeh et al . ( 2016 ) ; Ham et al . ; Chang et al . ; Kupyn et al . ( 2018 ) ; Ledig et al . ; Zhu et al . ( 2017 ) ) . There is also a growing body of work dedicated to using GANs to learn regularizers in solving inverse problems ( Lunz et al . ( 2018 ) and in compressed sensing ( Bora et al . ( 2017 ; 2018 ) ; Kabkab et al . ( 2018 ) ; Wu et al . ( 2019 ) ; Shah & Hegde ( 2018 ) ) . However , these approaches differ from ours in that they solve the inverse problem as an optimization problem and do not rely on Bayesian inference ; as a result , they add regularization in an ad-hoc manner and do not attempt to quantify the uncertainty in the inferred field . More recently , the approach described in ( Adler & Öktem ( 2018 ) ) utilizes GANs in a Bayesian setting ; however the GAN is trained to approximate the posterior distribution ( and not the prior , as in our case ) , and training is done in a supervised fashion . That is , paired samples of the measurement ŷ and the corresponding true solution x are required . In ( McCarthy et al . ( 2017 ) ) , authors perform variational approximation of the posterior using VAEs with simulator-based decoder to solve physics-based inference problem . We note that deep learning based Bayesian networks , where the network weights are stochastic parameters that are determined using Bayesian inference , are another means of quantifying uncertainty ( MacKay ( 1992 ) ; Kingma & Welling ( 2013 ) ; Gal & Ghahramani ( 2016 ) ) , and have recently been applied to semantic image-segmentation and superresolution ( Kendall & Gal ( 2017 ) ; Kendall et al . ( 2019 ) ; Kohl et al . ( 2018 ) ; Hu et al . ( 2019 ) ; Tanno et al . ( 2019 ) ) . However , these method also rely on supervised learning , whereas in contrast , our approach is unsupervised , and requires only samples of the true solution x to train the GAN prior .

Summary: The paper studies the Bayesian inferences with the generative adversarial network (GAN). In the first half of the paper, the general framework of the Bayes estimation is introduced. Then, The authors proposed how to incorporate GAN to the Bayesian inference. Some computational methods for calculating the mean of the statistic under the posterior distribution are described. Then, numerical experiments using MNIST and Celeb-A datasets are presented.