Year: 2019

Abstract:
Bayesian inference is used extensively to quantify the uncertainty in an inferred field given the measurement of a related field when the two are linked by a mathematical model. Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically. In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges. We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in inference problems arising in computer vision and physics-based applications. In both instances we highlight the role of computing uncertainty in providing a measure of confidence in the solution, and in designing successive measurements to improve this confidence.

heading: 1 INTRODUCTION
text: Quantifying uncertainty in an inference problem amounts to making a prediction and quantifying the confidence in that prediction. In the context of an image recovery problem, this may be understood as follows. A typical computer vision algorithm uses a noisy version of an image and prior knowledge to produce the recovered image which can be interpreted as the “best guess” of the original image. Quantifying uncertainty in this context involves generating an estimate of the level of confidence in the best guess, in addition to the guess itself.
Bayesian inference provides a principled approach for quantifying uncertainty. As shown in the following section, it treats the inferred vector as a multivariate stochastic vector and leads to an expression for its distribution. This expression can be used to estimate the most likely solution (the maximum a-posteriori estimate, or the MAP), the mean, the variance, or any other population parameter of interest. Thus providing a recipe for thoroughly quantifying the uncertainty in an inference problem. For the image recovery problems considered in this paper, Bayesian inference not only provides the best guess of the true image, but also a means to estimate measures of uncertainty such as the pixel-wise variance.
The knowledge of uncertainty in a prediction can directly influence the downstream action that depends on the inference. Consider an image recovery problem where two distinct inputs lead to similar recovered images: those of a traffic sign with a high speed limit. However, for the first input the predicted variance is small, while for the second input it is large. Further, the set of likely images in the second set also includes images of a Stop Sign. Then the appropriate action for the two inputs, determined after solving the inference problem and quantifying uncertainty, is very different. For the first input, the appropriate action is one of continued motion, whereas for the second input it is to slow down. Similar examples can be drawn from other areas as well, like medical imaging, high frequency trading, and autonomous systems where critical decisions are made based on the output of AI system (Gal (2016); Begoli et al. (2019)).
The knowledge of uncertainty can also be useful in determining the optimal location of a sensor. Consider an image recovery problem, where the goal is to infer the signal, and associated uncertainty, using limited amount of measurement data. In this problem a user can leverage information about the spatial distribution of uncertainty to choose the location with maximum uncertainty as next measurement location. This task falls within the fields of active learning and/or design of ex-
periments (DeGroot et al. (1962); Houlsby et al. (2011)) and is particularly useful in applications like satellite imaging, where each measurement requires significant time and/or resources.
In Figure 1, we demonstrate how the proposed GAN-based Bayesian inference algorithm is useful in both scenarios explained above: (i) in quantifying uncertainty - “pixel-wise variance” - our quantitative measure of uncertainty in the inferred field, and (ii) in determining the optimal sensor placement location in an iterative fashion. We return to these applications in greater detail in Section 4.

heading: 1.1 BAYESIAN INFERENCE
text: Bayesian inference is a well-established technique for quantifying uncertainties in inference problems (Kaipio & Somersalo (2006); Dashti & Stuart (2016); Polpo et al. (2018)). It has found applications in diverse fields such as geophysics (Gouveia & Scales (1997); Malinverno (2002)), climate modeling (Jackson et al. (2004)), chemical kinetics (Najm et al. (2009)), heat conduction (Wang & Zabaras (2004)), and the detection and diagnosis of disease (Siltanen et al. (2003); Kolehmainen et al. (2006)). The two critical ingredients of a Bayesian inference problem are - an informative prior representing the prior belief about the parameters to be inferred and an efficient method for sampling from the posterior distribution. In this manuscript we describe how deep generative adversarial networks (GANs) can be effectively used in these roles.
Consider the setting where we wish to infer a vector of parameters x ∈ RN from the measurement of a related vector y ∈ RP , where the two are related through a forward model y = f(x). A noisy measurement of y is denoted by ŷ = f(x) + η, where the vector η ∈ RP represents noise. While the forward map f is typically well-posed, its inverse is not, and hence to infer x from the measurement ŷ requires techniques that account for this ill-posedness. Classical techniques based on regularization tackle this ill-posedness by using additional information about the sought solution field explicitly or implicitly (Tarantola (2005)). Bayesian inference offers a different approach to this problem by modeling the unknown solution as well as the measurements as random variables. This framework addresses the ill-posedness of the inverse problem, and allows for the characterization of the uncertainty in the inferred solution.
The notion of a prior distribution plays a key role in Bayesian inference. Through multiple observations of the field x, denoted by the set S = {x(1), · · · ,x(S)}, we have some prior knowledge of x that can be utilized when inferring x from ŷ. This is used to build, or intuit, a prior distribution for x, denoted by ppriorX (x). Some typical examples include Gaussian process prior with specified co-variance kernels, Gaussian Markov random fields (Fahrmeir & Lang (2001)), Gaussian priors defined through differential operators (Stuart (2010)), and hierarchical Gaussian priors (Marzouk & Najm (2009); Calvetti & Somersalo (2008)). These priors promote smoothness and/or structure in the inferred solution and importantly, can be expressed explicitly in an analytical form.
Another key component of Bayesian inference is a distribution that represents the likelihood of y given an instance of x, denoted by pl(y|x). This is often determined by the distribution of the error in the model, denoted by pη , which captures both model and measurement errors. Given this, and an additive model for noise, the posterior distribution of x, determined using Bayes’ theorem after accounting for the observation ŷ is given by,
ppostX (x|y) = 1
Z pl(y|x)ppriorX (x) =
1 Z pη(ŷ − f(x))ppriorX (x). (1)
Here, Z is the prior-predictive distribution of y and ensures that the posterior integrates to one.
The posterior distribution characterizes the uncertainty in x; however for vectors of large dimension characterizing this distribution explicitly is a challenging task. Consequently the expression above is used to perform tasks that are more manageable. These include determining estimates such as the maximum a-posteriori estimate (MAP), expanding the posterior distribution in terms of other distributions that are simpler to work with (Bui-Thanh et al. (2012)), or using techniques like Markov Chain Monte-Carlo (MCMC) to generate samples that are “close” to the samples generated by the true posterior distribution (Han & Carlin (2001); Parno & Marzouk (2018)).
Despite its numerous applications in solving inverse problems, Bayesian inference faces significant challenges. These include defining a reliable and informative prior distribution for x when the set S = {x(1), · · · ,x(S)} is difficult to characterize analytically, and efficiently sampling from the posterior distribution when the dimension of x is large; a typical situation in many practical science and engineering applications.

heading: 1.2 RELATED WORK
text: The main idea developed in this paper tackles the above mentioned challenges by training a generative adversarial network (GAN) using the sample set S, and then using the distribution learned by the GAN as the prior distribution in Bayesian inference.
The use of sample-based priors for solving an inverse problem has a rich history (Vauhkonen et al. (1997); Calvetti & Somersalo (2005)). As does the idea of reducing the dimension of the parameter space by mapping it to a lower-dimensional space (Marzouk & Najm (2009); Lieberman et al. (2010)). However, the use of learning-based deep generative models like GANs in these tasks is novel.
Recently, several authors have considered the use of learning-based methods for solving inverse problems arising in different domains. These include the use of deep convolutional neural networks (CNNs), recurrent neural networks (RNNs) or variational autoencoders (VAEs) to solve physicsdriven inverse problems (Adler & Öktem (2017); Patel et al. (2019); Pesah et al. (2018)). VAEs and GANs have also been used to solve inverse problems in computer vision (Yeh et al. (2016); Ham et al.; Chang et al.; Kupyn et al. (2018); Ledig et al.; Zhu et al. (2017)). There is also a growing body of work dedicated to using GANs to learn regularizers in solving inverse problems (Lunz et al. (2018) and in compressed sensing (Bora et al. (2017; 2018); Kabkab et al. (2018); Wu et al. (2019); Shah & Hegde (2018)). However, these approaches differ from ours in that they solve the inverse problem as an optimization problem and do not rely on Bayesian inference; as a result, they add regularization in an ad-hoc manner and do not attempt to quantify the uncertainty in the inferred field.
More recently, the approach described in (Adler & Öktem (2018)) utilizes GANs in a Bayesian setting; however the GAN is trained to approximate the posterior distribution (and not the prior, as in our case), and training is done in a supervised fashion. That is, paired samples of the measurement ŷ and the corresponding true solution x are required. In (McCarthy et al. (2017)), authors perform variational approximation of the posterior using VAEs with simulator-based decoder to solve physics-based inference problem. We note that deep learning based Bayesian networks, where the network weights are stochastic parameters that are determined using Bayesian inference, are another means of quantifying uncertainty (MacKay (1992); Kingma & Welling (2013); Gal & Ghahramani (2016)), and have recently been applied to semantic image-segmentation and superresolution (Kendall & Gal (2017); Kendall et al. (2019); Kohl et al. (2018); Hu et al. (2019); Tanno et al. (2019)) . However, these method also rely on supervised learning, whereas in contrast, our approach is unsupervised, and requires only samples of the true solution x to train the GAN prior.

heading: 1.3 OUR CONTRIBUTION
text: The main contribution of this paper can be summarized as follows:
1. A novel method for performing Bayesian inference involving complex priors and high dimensional posterior. In this method we utilize the distribution learned by a GAN as a surrogate for the prior distribution and reformulate the inference problem in the low-dimensional latent space of the GAN. Furthermore, we provide a theoretical analysis of the weak convergence of the posterior density learned by the proposed method to the true posterior density.
2. Novel unsupervised image denoising and inpainting algorithms with quantitative measures of uncertainty through pixel-wise variance.
3. Application of the proposed method to physics-based inference problems. 4. Demonstration of the utility of quantifying uncertainty to facilitate active learning.

heading: 2 PROBLEM FORMULATION
text: Let S denote the set of instances of vector x sampled from the true distribution, ptrueX (x). Further, let z ∼ pZ(z) characterize the latent vector space and g(z) be the generator of a GAN trained using S. Then according to Goodfellow et al. (2014), with infinite capacity and sufficient data, the generator learns the true distribution. That is,
pgenX (x) = p true X (x). (2)
The distribution pgenX (x) is defined as
x ∼ pgenX (x)⇒ x = g(z), z ∼ pZ(z). (3)
Here pZ is the multivariate distribution of the latent vector whose components are iid and typically conform to a Gaussian or a uniform distribution. The equation above implies that the GAN generates samples of x by sampling z from pZ and then passing these through the generator.
Now consider a measurement ŷ from which we would like to infer the posterior distribution of x. For this we use (1) and set the prior distribution equal to the true distribution, that is ppriorX = p true X . Then from (2) this is the same as ppriorX = p gen X . Therefore,
ppostX (x|y) = 1
Z pη(ŷ − f(x))pgenX (x). (4)
Now for any l(x), we have
E x∼ppostX
[l(x)] = 1
Z E x∼pgenX [l(x)pη(ŷ − f(x))], From (4)
= 1
Z E z∼pZ [l(g(z))pη(ŷ − f(g(z)))], From (3)
= E z∼ppostZ [l(g(z))], (5)
where E is the expectation operator, and
ppostZ (z|y) ≡ 1
Z pη(ŷ − f(g(z)))pZ(z). (6)
The distribution ppostZ is the analog of p post X in the latent vector space. The measurement ŷ updates the prior distribution for x to the posterior distribution. Similarly, it updates the prior distribution for z, pZ , to the posterior distribution, p post Z , defined above.
Equation (5) implies that sampling from the posterior distribution of x is equivalent to sampling from the posterior distribution for z and passing the sample through the generator g. That is,
x ∼ ppostX (x|y)⇒ x = g(z), z ∼ p post Z (z|y). (7)
Since the dimension of z is typically smaller than that of x, this represents an efficient approach to sampling from the posterior of x.
The left hand side of (5) is an expression for a population parameter of the posterior, defined by l(x) ≡ Ex∼ppostX [l(x)]. The right hand sides of the last two lines of this equation describe how this parameter may be evaluated by sampling z (instead of x) from either pZ or p post Z .
The equality in (5) holds for a GAN with an infinite number of weights in the generator and the discriminator. In Appendix A of this manuscript, we consider the case of a Wasserstein GAN with a finite number of weights and prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights is increased.

heading: 2.1 SAMPLING FROM THE POSTERIOR DISTRIBUTION
text: We consider a scenario where we wish to infer and characterize the uncertainty in the vector of parameters x from a noisy measurement of y, denoted by ŷ, where f is a known map that connects x and y. And we have several prior realizations of plausible x, contained in the set S. For this problem we propose the following algorithm that accounts for the prior information in S and the “new” measurement ŷ through a Bayesian update:
1. Train a GAN with a generator g(z) on S.
2. Sample x from ppostX (x|y) given in (7).
With sufficient capacity in the GAN and with sufficient training, the posterior obtained using this algorithm will converge to the true posterior (see eq. (5) above and Appendix A). Since GANs can be used to represent complex distributions efficiently, this algorithm provides a means of including complex priors that are solely defined by samples within a Bayesian update. Further, it provides an efficient approach to sampling from the posterior since the dimension of z is typically much smaller (101 - 102) than that of x (104 - 107). Finally, in contrast to other methods that attempt to quantify uncertainty in image recovery tasks, this algorithm falls within the class of unsupervised learning algorithms, and does not require paired data for training. We now describe two approaches for estimating population parameters of the posterior using this algorithm. Monte-Carlo (MC) approximation The first approach is based on a Monte-Carlo approximation of a population parameter of the posterior distribution. This integral, which is defined in the second line of (5), may be approximated as,
l(x) ≡ E x∼ppostX
[l(x)] ≈ ∑Nsamp n=1 l(g(z))pη(ŷ − f(g(z)))∑Nsamp
n=1 pη(ŷ − f(g(z))) , z ∼ pZ(z). (8)
In the equation above, the numerator is obtained from a MC approximation of the integral in (5), and the denominator is obtained from a MC approximation of the scaling parameter Z. Sampling within this approach is simple since in a typical GAN, the zis belong to a simple distribution like a Gaussian or a uniform distribution. However, we anticipate that in many applications the likelihood will tend to concentrate the distribution of latent vector z to a small region within Ωz and thee sampling described above may be inefficient. Markov-Chain Monte-Carlo (MCMC) approximation A more efficient approach is to generate an MCMC approximation pmcmcZ (z|y) ≈ p post Z (z|y) using the definition in (6), and thereafter sample z from this distribution. Then from the third line of (5), any desired population parameter may be approximated as
l(x) ≡ E x∼ppostX [l(x)] ≈ 1 Nsamp Nsamp∑ n=1 l(g(z)), z ∼ pmcmcZ (z|y). (9)
We note that for all the numerical experiments in this paper we have used MCMC because of its better sample efficiency. Summary We have described two methods for probing the posterior distribution when the prior is defined by a GAN. These include an MC (8) and an MCMC estimate (9) of a given population parameter and a MAP estimate that is applicable to additive Gaussian noise with a Gaussian prior for the latent vector (see Section B in the Appendix). In the following section we apply the MCMC approach to inverse problems drawn from physics-based and computer vision applications.

heading: 3 EXPERIMENTS
text: We evaluate our method empirically on practical probabilistic field inference tasks in the domain of computer vision and physical science. In computer vision, we consider image denoising and inpainting tasks and evaluate our algorithm on MNIST and CelebA datasets. Here we attempt to infer the true image and quantify the uncertainty in this inference. We also demonstrate the utility of quantifying uncertainty by using it in an active learning setup. In the physics-based application, we consider the problem of recovering the initial temperature distribution from a measurement at later time, and use this problem to demonstrate the statistics inferred using our method converge to their “true” values.
In all cases we use a Wasserstein GAN-GP (Gulrajani et al. (2017)) to learn the prior density (architecture described in the Appendix D). We also ensure that the target images are not chosen from the set used to train the GAN. We sample from the posterior using Hamiltonian Monte Carlo (Brooks et al. (2012)) and implement it using Tensorflow-probability (Dillon et al. (2017)) library. We use initial step size of 1.0 for HMC and adapt it following (Andrieu & Thoms (2008)) based on the target acceptance probability. We use 64k samples with burn-in period of 0.5. We select these parameters to ensure convergence of chains. Using the HMC sampler we compute the MAP, which is our “best guess” of the true image, the pixel-wise mean, and the pixel-wise variance, which is our quantitative and spatially-varying estimate of uncertainty.

heading: 3.1 IMAGE RECOVERY USING THE MNIST & CELEBA DATABASES
text: We first consider the MNIST database of hand-written digits and use 55k images to train the GAN. We use a latent vector of dimension 100 with Gaussian distribution. For the image denoising task, we add Gaussian noise with zero mean and specified variance (σy) to the test image and use it as measurement to recover the distribution of likely images using the MCMC approach. For this problem the forward operator is the identity map, and the likelihood distribution is Gaussian. In Figure 2, we have plotted the noisy input image, the MAP estimate, and the pixel-wise mean and variance. We observe that for low and medium noise levels (σy = 0.1, 1), we are able to recover the original image with good accuracy, the pixel-wise variance is small overall, and is largest around the boundary of the recovered digit; this represents the variability in the different realizations of the recovered digit within the GAN prior. For the highest noise level (σy = 10), however, the image recovered by the MAP is incorrect in 2/3 cases, and would be misleading if viewed by itself. However, when viewed in conjunction with the estimated variance, which is large, it is clear that the confidence in the inference is small and the inferred image ought not be trusted for downstream tasks. The dependence of the average per-pixel variance in the recovered image on the variance of noise in the measured image is shown in Figure 4 (a) with 95% confidence interval. As expected, the per-pixel variance increases with increasing noise.
In the right-most panel of Figure 2, we show image inpainting + denoising results for an image of digit 3 with σy = 1 . Here the forward map is the indicator function set to zero on the occluded pixels. We note that for the small and intermediate occluded regions, the MAP solution is close to the true solution. However, when most of the image is occluded, the MAP is incorrect. Once again, the variance image, which is small for the low and medium occluded regions, and large for
the large occlusion, is a reliable indicator of the confidence in the recovered MAP image. Furthermore, the variance is peaked within the occluded region demonstrating lower confidence in the image reconstructed in this region. This is useful in applications like autonomous cars and medical imaging, where partial measurements are common and the spatial distribution of uncertainty can inform downstream tasks. More image denoising and inpainting examples are provided in Appendix C.
In Figure 3, we demonstrate how uncertainty may be used in active learning/design of experiment, where the goal is to determine the optimal location for a measurement. We begin with an input where the entire image is occluded and in every subsequent step, we allow for a small 7×7 pixel window to be revealed. We select the window with the largest average pixel-wise variance. As the iterations progress, the MAP estimate converges to the true digit, and the variance decreases. In about 4 iterations we arrive at a very good guess for the digit. The performance of this approach is quantified in Figure 4(b), where we have plotted reconstruction error versus the number windows for this strategy, and a strategy where the subsequent window is selected randomly. The variancedriven strategy consistently performs better. We note that we are not aware of any other methods for computing uncertainty in recovered images that have been applied to drive an active learning task in image inpainting. While methods based on dropout (Kendall & Gal (2017); Kendall et al. (2019)) or variational inference (Kohl et al. (2018)) could be extended to accomplish this, this has not been done thus far.
Results for the variance-based window selection strategy applied to the CelebA dataset are shown in Figure 5. We observe that the algorithm produces realistic images at each iteration; however, the initial variance is large indicating large uncertainty. As more windows are sampled using the variance driven active learning strategy, the variance reduces and by the 7th iteration a good approximation of the true image is obtained, even though only a small, noisy portion is revealed. Additional results along with implementation details for this dataset are discussed in Appendix C.2.

heading: 3.2 A PHYSICS-DRIVEN INFERENCE PROBLEM
text: We consider the problem of inverse heat conduction, where the goal is to infer the initial temperature distribution (at t = 0) in a domain given a noisy measurement of temperature at later time (t = 1) and the thermal conductivity of the material. The forward map is the solution of the time-dependent heat conduction problem with uniform conductivity, κ = 0.64, and in a square domain of length L = 2π with Dirichlet boundary conditions. This operator maps the initial temperature field to the temperature field at later time. The discrete version of this operator is obtained by discretizing the time-dependent linear heat conduction equation using the central difference scheme in space and backward Euler scheme in time. Much like a blurring kernel, the forward operator smooths the initial temperature distribution, and the extent of smoothing increases with κ× t. We consider a family of initial temperatures where the background is zero, and the temperature on a rectangular sub-domain varies linearly from 2 units on the left edge to 4 units on the right edge. This distribution is parameterized by four parameters, {ξi}4i=1, which are the coordinates of the lower left and upper right corners of the rectangular region. The sample set S is created by sampling each parameter from a uniform distribution (xii ∼ U(0.25, 0.75)×L) and is used to train the GAN prior. The posterior distribution is sampled using the HMC sampler.
In the top two rows of Figure 6, we have plotted the true initial condition, the noise-free temperature at t = 1, and the noisy temperature measurement (σy = 1) used as input in the GAN-based prior approach. The corresponding MAP, mean and pixel-wise variance estimated by the MCMC approximation are shown next. We observe that the MAP is very close to the true initial temperature distribution and the variance is concentrated along the edges of the rectangle where the uncertainty is the largest. In the following columns we have plotted the MAP estimate obtained assuming L2 andH1 Gaussian priors, which are often used to solve these types of problems, and are clearly much less accurate.
For this problem the “true” posterior can be reduced to the 4-dimensional space of parameters, and sampled by generating initial conditions corresponding to the values of these parameters. A simple MC approximation can be performed to compute statistics - the mean and the pixel-wise variance for the true posterior (last two columns of Figure 6). By comparing these with the mean and the pixelwise variance (columns 5 & 6) estimated by the GAN-based prior, we conclude that GAN-based posterior has converged to the true posterior.
In the bottom rows of Figure 6, we plot similar results for initial conditions and GAN-based priors generated from the MNIST database. In this case the measurement is made at t = 0.2. Since the “true” distribution for this set is not known the true mean and variance are not plotted.

heading: 4 CONCLUSIONS
text: The ability to quantify the uncertainty in an inference problem is useful in developing confidence in that inference, and in designing strategies to improve the confidence. In this paper we have described how this may be accomplished when solving a Bayesian inference problem by using GANs as priors. Since GANs can learn complex distributions of a wide variety of fields from their
samples, this approach can be applied to a range of problems in computer vision and physics-driven inference. It derives its efficiency by mapping the posterior distribution to the latent space, whose dimension is often much smaller than that of the inferred field. We have applied this approach to image recovery tasks and demonstrated how the knowledge of uncertainty in the prediction can be used to assess confidence in a prediction, and via active learning to design a strategy to improve it. We have also applied this approach to a physics-based problem, where we have verified its accuracy and robustness. In the results reported in this manuscript we assume knowledge of the forward map; however, we note that the proposed algorithm can easily be extended to a regime where the forward map is also unknown by utilizing likelihood-free inference methods like ABC or meta-learning approaches. We leave this as an interesting direction for future research.

heading: A WEAK CONVERGENCE OF THE POSTERIOR DENSITY
text: In this section we prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights in the generator and discriminator are increased.
Let the generator of the GAN be given by g(z;θ), where z ∈ RM is the latent vector, and θ ∈ RNθ is the vector of weights. The vector z is selected from the distribution pZ(z). Note that g : RM → RN .
Let the discriminator of the GAN be given by d(x;φ), where x ∈ RN , and φ ∈ RNφ be the vector of weights. Note that d : RN → R(0, 1).
The GAN is trained using a set of samples of x, drawn from ppriorX (x). For the loss function, consider
L(θ,φ) = E x∼ppriorX [ρ(1− d(x;φ))] + E z∼pZ [ρ(d(g(z;θ);φ))]. (10)
Here ρ is a monotone real-valued function which defines the GAN family being analyzed. For example, for the Wasserstein GAN, ρ(ξ) = ξ.
The optimal values of the weights are given by
θ∗,φ∗ = argmax θ (argmin φ (L(θ,φ))). (11)
A.1 STATIONARITY CONDITIONS
The necessary conditions for these optimal values are
∂L(θ∗,φ∗)
∂φ = 0 (12)
∂L(θ∗,φ∗)
∂θ = 0. (13)
Using the definition of the loss function (10) in (12), we have
E x∼ppriorX
[ρ′(1− d(x;φ∗)) ∂d ∂φ (x;φ∗)] = E z∼pZ [ρ′(d(g(z;θ∗);φ∗)) ∂d ∂φ (g(z;θ∗);φ∗)]. (14)
Similarly, using (10) in (13), we have
E z∼pZ
[ρ′(d(g(z;θ∗);φ∗)) ∂d ∂x (g(z;θ∗);φ∗) · ∂g ∂θ (z;θ∗)] = 0. (15)
A.2 WASSERSTEIN GAN
For the Wasserstein GAN, ρ(ξ) = ξ and ρ′(ξ) = 1. As a result (14) and (15) reduce to,
E x∼ppriorX
[ ∂d
∂φ (x;φ∗)] = E z∼pZ [ ∂d ∂φ (g(z;θ∗);φ∗)] (16)
E z∼pZ
[ ∂d ∂x (g(z;θ∗);φ∗) · ∂g ∂θ (z;θ∗)] = 0. (17)
Let wa(x) ≡ ∂d∂φa (x;φ ∗), a = 1, · · · , Nφ. Then (16) implies
E x∼ppriorX [wa(x)] = E z∼pZ
[wa(g(z;θ ∗))], a = 1, · · · , Nφ. (18)
As Nφ → ∞, this is a weak statement of the equivalence of ppriorX and p gen X , where the latter is defined in (3). In particular this says that the push forward of the measure in the latent space under
the function g(z) weakly converges to the measure associated with distribution of x. We note with increasing number of weights in the discriminator, the relation above is required to hold for an increasing number of test functions, wa. In addition, we have implicitly assumed that the generator is rich enough, that is it has enough weights/layers, such that this relation can actually be satisfied. To make this clear consider the extreme case of a generator with a single weight; in this case there is no way that (18) will be satisfied for a large number Nφ. Thus in order for this relation to hold for a large Nφ, we must also provide the generator with a large Nθ.
Now consider a sufficiently smooth function m(x) which defines the point estimate we wish to compute. Expand it using wa(x) as basis such that
m(x)− Nφ∑ a=1 αawa(x) = (x), (19)
and the coefficients are selected to minimize
̄ = ‖ (x)‖∞. (20) As we increase Nφ the dimension of the basis used to represent m increases, and therefore ̄ tends to zero.
Given this,
E x∼ppriorX [m(x)] = E x∼ppriorX [ Nφ∑ a=1 αawa(x) + (x)], from(19)
= Nφ∑ a=1 αa E x∼ppriorX [wa(x)] + E x∼ppriorX [ (x)]
= Nφ∑ a=1 αa E z∼pZ [wa(g(z;θ ∗))] + E x∼ppriorX [ (x)], from(18)
= E z∼pZ [ Nφ∑ a=1 αawa(g(z;θ ∗))] + E x∼ppriorX [ (x)]
= E z∼pZ [m(g(z;θ∗))− (g(z;θ∗))] + E x∼ppriorX [ (x)], from(19)
= E z∼pZ [m(g(z;θ∗))]− E z∼pZ [ (g(z;θ∗))] + E x∼ppriorX [ (x)]. (21)
This yields,
| E x∼ppriorX [m(x)]− E z∼pZ [m(g(z;θ∗))]| ≤ | − E z∼pZ [ (g(z;θ∗))] + E x∼ppriorX [ (x)]|
≤ | E z∼pZ [ (g(z;θ∗))]|+ | E x∼ppriorX [ (x)]|
≤ E z∼pZ [‖ (g(z;θ∗))‖∞] + E x∼ppriorX [‖ (x)‖∞]
≤ E z∼pZ [̄] + E x∼pX [̄], from(20)
= 2̄. (22)
Note that in deriving this estimate, in the third line we have assumed that g(z;θ∗) ∈ Ωx. That is, the generator does not land outside the domain of x. The statement above bounds the error in computing a population parameter for the prior using a GAN.
A.3 CONVERGENCE TO THE POSTERIOR DENSITY
In order to turn this estimate into an estimate for the error in approximating a population parameter for the posterior density, we simply choose m(x) = l(x)pη(ŷ−f(x))Z . With this choice,
E x∼ppriorX [m(x)] = E x∼ppostX [l(x)], from (1) (23)
and
E z∼pZ [m(g(z;θ∗))] = E z∼ppostZ [l(g(z;θ∗))], from (6). (24)
Using these in (22) we have the desired result,
| E x∼ppostX [l(x)]− E z∼ppostZ
[l(g(z;θ∗))]| ≤ 2̄. (25)
This result demonstrates that the difference between the true population parameter for the posterior and its approximation obtained after using a GAN as a prior (the method proposed in the paper) reduces as the number of weights in the generator and the discriminator are increased. In the limit of infinite weights, ̄→ 0, and the equation above is a statement of weak convergence of the posterior obtained by using the GAN and the true posterior density.

heading: B EXPRESSION FOR THE MAXIMUM A-POSTERIORI ESTIMATE
text: The techniques described in Section 2.1 focus on sampling from the posterior distribution and computing approximations to population parameters. These techniques can be applied in conjunction with any distribution used to model noise and the latent space vector; that is, any choice of pη (likelihood) and pZ (prior). In this section we consider the special case when Gaussian models are used for noise and the latent vector. In this case, we can derive a simple optimization algorithm to determine the maximum a-posteriori estimate (MAP) for ppostZ (z|y). This point is denoted by zmap in the latent vector space and represents the most likely value of the latent vector in the posterior distribution. It is likely that the operation of the generator on zmap, that is g(zmap), will yield a value that is close to xmap, and may be considered as a likely solution to the inference problem.
We consider the case when the components of the latent vector are iid with a normal distribution with zero mean and unit variance. This is often the case in many typical applications of GANs. Further, we assume that the components of noise vector are defined by a normal distribution with zero mean and a covariance matrix Σ. Using these assumptions in (6), we have
ppostZ (z|y) ∝ exp ( − 1
2
≡r(z)︷ ︸︸ ︷( |Σ−1/2(ŷ − f(g(z)))|2 + |z|2 ) ) . (26)
The MAP estimate for this distribution is obtained by minimizing the negative of the argument of the exponential. That is
zmap = arg min z r(z). (27)
This minimization problem may be solved using any gradient-based optimization algorithm. The input to this algorithm is the gradient of the functional r with respect to z, which is given by
∂r ∂z = HT (z)Σ−1(f(g(z))− ŷ) + z, (28)
where the matrixH is defined as
H ≡ ∂f(g(z)) ∂z = ∂f ∂x ∂g ∂z . (29)
Here ∂f∂x is the derivative of the forward map f with respect to its input x, and ∂g ∂z is the derivative of the generator output with respect to the latent vector. In evaluating the gradient above we need to evaluate the operation of the matrices ∂f∂x and ∂g ∂z on a vector, and not the matrices themselves. The operation of ∂g∂z on a vector can be determined using a back-propagation algorithm with the GAN; while the operation of ∂f∂x can be determined by making use of the adjoint of the linearization of the forward operator.
Once zmap is determined, one may evaluate g(zmap) by using the GAN generator. This represents the value of the field we wish to infer at the most likely value value of latent vector. Note that this is not the same as the MAP estimate of ppostX (x|y).

heading: C ADDITIONAL RESULTS
text: In this section we provide additional results for both MNIST and CelebA dataset for different tasks discussed in the main paper.
C.1 MNIST
First we provide additional examples in Figure 7 for variance-based adaptive measurement window selection procedure described in Section 3.1.
Figure 8 shows additional results for the inpainting + denoising task, where MNIST digits are occluded with masks of different sizes at different locations. Note that the variance is high where the occlusion mask is located indicating lower confidence in reconstructed image in that location.
C.2 CELEBA
For the CelebA dataset, we trained WGAN-GP model using more than 200k celebrity facial images and perform inference using remaining test set images. The input images were cropped to a 64× 64 RGB image and were normalized between [-1, 1].
Once the GAN was trained, the HMC algorithm was used for posterior sampling and inference on a complimentary set of images (not used for training). In Figure 9 we show some additional results for variance-based adaptive measurement window selection procedure for CelebA dataset.
Next, in figure 10 we show some additional results for image recovery task for CelebA dataset. Once again we note that the MAP estimate and the mean is close to the true image. On the other hand, the closest image from the training set (in an L2 sense) is not as accurate. This points to the utility of using the GAN as an interpolant in the latent vector space.

heading: D ARCHITECTURE AND TRAINING DETAILS
text: We use the WGAN-GP model for learning prior density. The tuned value of hyper-parameters is shown in Table 1. We use the same generator and discriminator architecture for the MNIST and the physics-based inference problem; whereas for the CelebA dataset we use a slightly different architecture to accommodate different input image size. The layout of both these architecture is shown in Figure 11. Some notes regarding nomenclature used in Figure 11.

Review summaries:
The paper studies the Bayesian inferences with the generative adversarial network (GAN). In the first half of the paper, the general framework of the Bayes estimation is introduced. Then, The authors proposed how to incorporate GAN to the Bayesian inference. Some computational methods for calculating the mean of the statistic under the posterior distribution are described. Then, numerical experiments using MNIST and Celeb-A datasets are presented.