Title: QUANTIFYING UNCERTAINTY WITH GAN-BASED PRI-ORS

Abstract: Bayesian inference is used extensively to quantify the uncertainty in an inferred field given the measurement of a related field when the two are linked by a mathematical model. Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically. In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges. We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in inference problems arising in computer vision and physics-based applications. In both instances we highlight the role of computing uncertainty in providing a measure of confidence in the solution, and in designing successive measurements to improve this confidence.

1. INTRODUCTION:

Quantifying uncertainty in an inference problem amounts to making a prediction and quantifying the confidence in that prediction. In the context of an image recovery problem, this may be understood as follows. A typical computer vision algorithm uses a noisy version of an image and prior knowledge to produce the recovered image which can be interpreted as the "best guess" of the original image. Quantifying uncertainty in this context involves generating an estimate of the level of confidence in the best guess, in addition to the guess itself.

Bayesian inference provides a principled approach for quantifying uncertainty. As shown in the following section, it treats the inferred vector as a multivariate stochastic vector and leads to an expression for its distribution. This expression can be used to estimate the most likely solution (the maximum a-posteriori estimate, or the MAP), the mean, the variance, or any other population parameter of interest. Thus providing a recipe for thoroughly quantifying the uncertainty in an inference problem. For the image recovery problems considered in this paper, Bayesian inference not only provides the best guess of the true image, but also a means to estimate measures of uncertainty such as the pixel-wise variance.

The knowledge of uncertainty in a prediction can directly influence the downstream action that depends on the inference. Consider an image recovery problem where two distinct inputs lead to similar recovered images: those of a traffic sign with a high speed limit. However, for the first input the predicted variance is small, while for the second input it is large. Further, the set of likely images in the second set also includes images of a Stop Sign. Then the appropriate action for the two inputs, determined after solving the inference problem and quantifying uncertainty, is very different. For the first input, the appropriate action is one of continued motion, whereas for the second input it is to slow down. Similar examples can be drawn from other areas as well, like medical imaging, high frequency trading, and autonomous systems where critical decisions are made based on the output of AI system (Gal (2016) ; Begoli et al. (2019) ).

The knowledge of uncertainty can also be useful in determining the optimal location of a sensor. Consider an image recovery problem, where the goal is to infer the signal, and associated uncertainty, using limited amount of measurement data. In this problem a user can leverage information about the spatial distribution of uncertainty to choose the location with maximum uncertainty as next measurement location. This task falls within the fields of active learning and/or design of ex-periments (DeGroot et al. (1962) ; Houlsby et al. (2011) ) and is particularly useful in applications like satellite imaging, where each measurement requires significant time and/or resources.

In Figure 1 , we demonstrate how the proposed GAN-based Bayesian inference algorithm is useful in both scenarios explained above: (i) in quantifying uncertainty -"pixel-wise variance" -our quantitative measure of uncertainty in the inferred field, and (ii) in determining the optimal sensor placement location in an iterative fashion. We return to these applications in greater detail in Section 4.

Figure 1 : Estimate of the MAP (2nd row) and pixel-wise variance (3rd row) from the limited view of a noisy image (1st row) using the proposed method. The first five columns correspond to a strategy where the next window is selected randomly, while the other columns correspond to a strategy where the next window is selected in the region with maximum estimated variance. For equivalent accuracy, the variance-driven selection strategy uses fewer sampling windows (4 versus 9). In both cases variance reduces with increasing measurement.

1.1. BAYESIAN INFERENCE:

Bayesian inference is a well-established technique for quantifying uncertainties in inference problems (Kaipio & Somersalo (2006) ; Dashti & Stuart (2016); Polpo et al. (2018) ). It has found applications in diverse fields such as geophysics (Gouveia & Scales (1997) ; Malinverno (2002) ), climate modeling (Jackson et al. (2004) ), chemical kinetics (Najm et al. (2009) ), heat conduction (Wang & Zabaras (2004) ), and the detection and diagnosis of disease (Siltanen et al. (2003) ; Kolehmainen et al. (2006) ). The two critical ingredients of a Bayesian inference problem are -an informative prior representing the prior belief about the parameters to be inferred and an efficient method for sampling from the posterior distribution. In this manuscript we describe how deep generative adversarial networks (GANs) can be effectively used in these roles.

Consider the setting where we wish to infer a vector of parameters x ∈ R N from the measurement of a related vector y ∈ R P , where the two are related through a forward model y = f (x). A noisy measurement of y is denoted by ŷ = f (x) + η, where the vector η ∈ R P represents noise. While the forward map f is typically well-posed, its inverse is not, and hence to infer x from the measurement ŷ requires techniques that account for this ill-posedness. Classical techniques based on regularization tackle this ill-posedness by using additional information about the sought solution field explicitly or implicitly (Tarantola (2005) ). Bayesian inference offers a different approach to this problem by modeling the unknown solution as well as the measurements as random variables. This framework addresses the ill-posedness of the inverse problem, and allows for the characterization of the uncertainty in the inferred solution.

The notion of a prior distribution plays a key role in Bayesian inference. Through multiple observations of the field x, denoted by the set S = {x (1) , • • • , x (S) }, we have some prior knowledge of x that can be utilized when inferring x from ŷ. This is used to build, or intuit, a prior distribution for x, denoted by p prior X (x). Some typical examples include Gaussian process prior with specified co-variance kernels, Gaussian Markov random fields (Fahrmeir & Lang (2001) ), Gaussian priors defined through differential operators (Stuart (2010) ), and hierarchical Gaussian priors (Marzouk & Najm (2009) ; Calvetti & Somersalo (2008) ). These priors promote smoothness and/or structure in the inferred solution and importantly, can be expressed explicitly in an analytical form.

Another key component of Bayesian inference is a distribution that represents the likelihood of y given an instance of x, denoted by p l (y|x). This is often determined by the distribution of the error in the model, denoted by p η , which captures both model and measurement errors. Given this, and an additive model for noise, the posterior distribution of x, determined using Bayes' theorem after accounting for the observation ŷ is given by,

EQUATION

Here, Z is the prior-predictive distribution of y and ensures that the posterior integrates to one.

The posterior distribution characterizes the uncertainty in x; however for vectors of large dimension characterizing this distribution explicitly is a challenging task. Consequently the expression above is used to perform tasks that are more manageable. Despite its numerous applications in solving inverse problems, Bayesian inference faces significant challenges. These include defining a reliable and informative prior distribution for x when the set S) } is difficult to characterize analytically, and efficiently sampling from the posterior distribution when the dimension of x is large; a typical situation in many practical science and engineering applications.

EQUATION

1.2. RELATED WORK:

The main idea developed in this paper tackles the above mentioned challenges by training a generative adversarial network (GAN) using the sample set S, and then using the distribution learned by the GAN as the prior distribution in Bayesian inference.

The use of sample-based priors for solving an inverse problem has a rich history (Vauhkonen et al. (1997) ; Calvetti & Somersalo (2005) ). As does the idea of reducing the dimension of the parameter space by mapping it to a lower-dimensional space (Marzouk & Najm (2009) ; Lieberman et al. (2010) ). However, the use of learning-based deep generative models like GANs in these tasks is novel.

Recently, several authors have considered the use of learning-based methods for solving inverse problems arising in different domains. These include the use of deep convolutional neural networks (CNNs), recurrent neural networks (RNNs) or variational autoencoders (VAEs) to solve physicsdriven inverse problems (Adler & Öktem (2017) ; Patel et al. (2019) ; Pesah et al. (2018) ). VAEs and GANs have also been used to solve inverse problems in computer vision (Yeh et al. (2016) ; Ham et al.; Chang et al.; Kupyn et al. (2018) ; Ledig et al.; Zhu et al. (2017) ). There is also a growing body of work dedicated to using GANs to learn regularizers in solving inverse problems (Lunz et al. (2018) and in compressed sensing (Bora et al. (2017; 2018) ; Kabkab et al. (2018) ; Wu et al. (2019) ; Shah & Hegde (2018) ). However, these approaches differ from ours in that they solve the inverse problem as an optimization problem and do not rely on Bayesian inference; as a result, they add regularization in an ad-hoc manner and do not attempt to quantify the uncertainty in the inferred field.

More recently, the approach described in (Adler & Öktem (2018) ) utilizes GANs in a Bayesian setting; however the GAN is trained to approximate the posterior distribution (and not the prior, as in our case), and training is done in a supervised fashion. That is, paired samples of the measurement ŷ and the corresponding true solution x are required. In 2019)) . However, these method also rely on supervised learning, whereas in contrast, our approach is unsupervised, and requires only samples of the true solution x to train the GAN prior.

1.3. OUR CONTRIBUTION:

The main contribution of this paper can be summarized as follows:

1. A novel method for performing Bayesian inference involving complex priors and high dimensional posterior. In this method we utilize the distribution learned by a GAN as a surrogate for the prior distribution and reformulate the inference problem in the low-dimensional latent space of the GAN. Furthermore, we provide a theoretical analysis of the weak convergence of the posterior density learned by the proposed method to the true posterior density. 2. Novel unsupervised image denoising and inpainting algorithms with quantitative measures of uncertainty through pixel-wise variance. 3. Application of the proposed method to physics-based inference problems. 4. Demonstration of the utility of quantifying uncertainty to facilitate active learning.

2. PROBLEM FORMULATION:

Let S denote the set of instances of vector x sampled from the true distribution, p true X (x). Further, let z ∼ p Z (z) characterize the latent vector space and g(z) be the generator of a GAN trained using S. Then according to Goodfellow et al. (2014) , with infinite capacity and sufficient data, the generator learns the true distribution. That is,

EQUATION

The distribution p gen X (x) is defined as

EQUATION

Here p Z is the multivariate distribution of the latent vector whose components are iid and typically conform to a Gaussian or a uniform distribution. The equation above implies that the GAN generates samples of x by sampling z from p Z and then passing these through the generator.

Now consider a measurement ŷ from which we would like to infer the posterior distribution of x.

For this we use (1) and set the prior distribution equal to the true distribution, that is p prior X = p true X . Then from (2) this is the same as p prior X = p gen X . Therefore,

EQUATION

Now for any l(x), we have

EQUATION

where E is the expectation operator, and

EQUATION

The distribution p post Z is the analog of p post X in the latent vector space. The measurement ŷ updates the prior distribution for x to the posterior distribution. Similarly, it updates the prior distribution for z, p Z , to the posterior distribution, p post Z , defined above. Equation ( 5) implies that sampling from the posterior distribution of x is equivalent to sampling from the posterior distribution for z and passing the sample through the generator g. That is,

EQUATION

Since the dimension of z is typically smaller than that of x, this represents an efficient approach to sampling from the posterior of x.

The left hand side of ( 5) is an expression for a population parameter of the posterior, defined by

l(x) ≡ E x∼p post X [l(x)].

The right hand sides of the last two lines of this equation describe how this parameter may be evaluated by sampling z (instead of x) from either p Z or p post Z . The equality in (5) holds for a GAN with an infinite number of weights in the generator and the discriminator. In Appendix A of this manuscript, we consider the case of a Wasserstein GAN with a finite number of weights and prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights is increased.

2.1. SAMPLING FROM THE POSTERIOR DISTRIBUTION:

We consider a scenario where we wish to infer and characterize the uncertainty in the vector of parameters x from a noisy measurement of y, denoted by ŷ, where f is a known map that connects x and y. And we have several prior realizations of plausible x, contained in the set S. For this problem we propose the following algorithm that accounts for the prior information in S and the "new" measurement ŷ through a Bayesian update:

1. Train a GAN with a generator g(z) on S.

2.. Sample x from p post:

X (x|y) given in ( 7). With sufficient capacity in the GAN and with sufficient training, the posterior obtained using this algorithm will converge to the true posterior (see eq. ( 5) above and Appendix A). Since GANs can be used to represent complex distributions efficiently, this algorithm provides a means of including complex priors that are solely defined by samples within a Bayesian update. Further, it provides an efficient approach to sampling from the posterior since the dimension of z is typically much smaller (10 1 -10 2 ) than that of x (10 4 -10 7 ). Finally, in contrast to other methods that attempt to quantify uncertainty in image recovery tasks, this algorithm falls within the class of unsupervised learning algorithms, and does not require paired data for training. We now describe two approaches for estimating population parameters of the posterior using this algorithm.

Monte-Carlo (MC) approximation:

The first approach is based on a Monte-Carlo approximation of a population parameter of the posterior distribution. This integral, which is defined in the second line of (5), may be approximated as,

EQUATION

In the equation above, the numerator is obtained from a MC approximation of the integral in (5), and the denominator is obtained from a MC approximation of the scaling parameter Z. Sampling within this approach is simple since in a typical GAN, the z i s belong to a simple distribution like a Gaussian or a uniform distribution. However, we anticipate that in many applications the likelihood will tend to concentrate the distribution of latent vector z to a small region within Ω z and thee sampling described above may be inefficient.

Markov-Chain Monte-Carlo (MCMC) approximation A more efficient approach is to generate an MCMC approximation p mcmc Z (z|y) ≈ p post Z (z|y) using the definition in (6), and thereafter sample z from this distribution. Then from the third line of (5), any desired population parameter may be approximated as

EQUATION

We note that for all the numerical experiments in this paper we have used MCMC because of its better sample efficiency. Summary We have described two methods for probing the posterior distribution when the prior is defined by a GAN. These include an MC (8) and an MCMC estimate (9) of a given population parameter and a MAP estimate that is applicable to additive Gaussian noise with a Gaussian prior for the latent vector (see Section B in the Appendix). In the following section we apply the MCMC approach to inverse problems drawn from physics-based and computer vision applications.

3. EXPERIMENTS:

We evaluate our method empirically on practical probabilistic field inference tasks in the domain of computer vision and physical science. In computer vision, we consider image denoising and inpainting tasks and evaluate our algorithm on MNIST and CelebA datasets. Here we attempt to infer the true image and quantify the uncertainty in this inference. We also demonstrate the utility of quantifying uncertainty by using it in an active learning setup. In the physics-based application, we consider the problem of recovering the initial temperature distribution from a measurement at later time, and use this problem to demonstrate the statistics inferred using our method converge to their "true" values.

In all cases we use a Wasserstein GAN-GP (Gulrajani et al. (2017) ) to learn the prior density (architecture described in the Appendix D). We also ensure that the target images are not chosen from the set used to train the GAN. We sample from the posterior using Hamiltonian Monte Carlo (Brooks et al. (2012) ) and implement it using Tensorflow-probability (Dillon et al. (2017) ) library. We use initial step size of 1.0 for HMC and adapt it following (Andrieu & Thoms (2008) ) based on the target acceptance probability. We use 64k samples with burn-in period of 0.5. We select these parameters to ensure convergence of chains. Using the HMC sampler we compute the MAP, which is our "best guess" of the true image, the pixel-wise mean, and the pixel-wise variance, which is our quantitative and spatially-varying estimate of uncertainty.

3.1. IMAGE RECOVERY USING THE MNIST & CELEBA DATABASES:

We first consider the MNIST database of hand-written digits and use 55k images to train the GAN. We use a latent vector of dimension 100 with Gaussian distribution. For the image denoising task, we add Gaussian noise with zero mean and specified variance (σ y ) to the test image and use it as measurement to recover the distribution of likely images using the MCMC approach. For this problem the forward operator is the identity map, and the likelihood distribution is Gaussian. In Figure 2 , we have plotted the noisy input image, the MAP estimate, and the pixel-wise mean and variance. We observe that for low and medium noise levels (σ y = 0.1, 1), we are able to recover the original image with good accuracy, the pixel-wise variance is small overall, and is largest around the boundary of the recovered digit; this represents the variability in the different realizations of the recovered digit within the GAN prior. For the highest noise level (σ y = 10), however, the image recovered by the MAP is incorrect in 2/3 cases, and would be misleading if viewed by itself. However, when viewed in conjunction with the estimated variance, which is large, it is clear that the confidence in the inference is small and the inferred image ought not be trusted for downstream tasks. The dependence of the average per-pixel variance in the recovered image on the variance of noise in the measured image is shown in Figure 4 (a) with 95% confidence interval. As expected, the per-pixel variance increases with increasing noise.

Figure 2 : Estimate of the MAP, mean and pixel-wise variance from a noisy image using the proposed method. In the first three panels σ y = 0.1, 1, &10, when moving from left to right. In the fourth panel σ y = 1, and the size of the occluded region is increased.

In the right-most panel of Figure 2 , we show image inpainting + denoising results for an image of digit 3 with σ y = 1 . Here the forward map is the indicator function set to zero on the occluded pixels. We note that for the small and intermediate occluded regions, the MAP solution is close to the true solution. However, when most of the image is occluded, the MAP is incorrect. Once again, the variance image, which is small for the low and medium occluded regions, and large for the large occlusion, is a reliable indicator of the confidence in the recovered MAP image. Furthermore, the variance is peaked within the occluded region demonstrating lower confidence in the image reconstructed in this region. This is useful in applications like autonomous cars and medical imaging, where partial measurements are common and the spatial distribution of uncertainty can inform downstream tasks. More image denoising and inpainting examples are provided in Appendix C.

Figure 3 : Estimate of the MAP, mean and variance from the limited view of a noisy image (2nd row) using the proposed method for the digits 8 & 5 (left and right panels). The window to be revealed at a given iteration (shown in red box) is selected using a variance-driven strategy.

In Figure 3 , we demonstrate how uncertainty may be used in active learning/design of experiment, where the goal is to determine the optimal location for a measurement. We begin with an input where the entire image is occluded and in every subsequent step, we allow for a small 7×7 pixel window to be revealed. We select the window with the largest average pixel-wise variance. As the iterations progress, the MAP estimate converges to the true digit, and the variance decreases. In about 4 iterations we arrive at a very good guess for the digit. The performance of this approach is quantified in Figure 4 (b), where we have plotted reconstruction error versus the number windows for this strategy, and a strategy where the subsequent window is selected randomly. The variancedriven strategy consistently performs better. We note that we are not aware of any other methods for computing uncertainty in recovered images that have been applied to drive an active learning task in image inpainting. While methods based on dropout (Kendall & Gal (2017) ; Kendall et al. (2019) ) or variational inference (Kohl et al. ( 2018)) could be extended to accomplish this, this has not been done thus far. Results for the variance-based window selection strategy applied to the CelebA dataset are shown in Figure 5 . We observe that the algorithm produces realistic images at each iteration; however, the initial variance is large indicating large uncertainty. As more windows are sampled using the variance driven active learning strategy, the variance reduces and by the 7th iteration a good approximation of the true image is obtained, even though only a small, noisy portion is revealed. Additional results along with implementation details for this dataset are discussed in Appendix C.2. 

3.2. A PHYSICS-DRIVEN INFERENCE PROBLEM:

We consider the problem of inverse heat conduction, where the goal is to infer the initial temperature distribution (at t = 0) in a domain given a noisy measurement of temperature at later time (t = 1) and the thermal conductivity of the material. The forward map is the solution of the time-dependent heat conduction problem with uniform conductivity, κ = 0.64, and in a square domain of length L = 2π with Dirichlet boundary conditions. This operator maps the initial temperature field to the temperature field at later time. The discrete version of this operator is obtained by discretizing the time-dependent linear heat conduction equation using the central difference scheme in space and backward Euler scheme in time. Much like a blurring kernel, the forward operator smooths the initial temperature distribution, and the extent of smoothing increases with κ × t.

We consider a family of initial temperatures where the background is zero, and the temperature on a rectangular sub-domain varies linearly from 2 units on the left edge to 4 units on the right edge. This distribution is parameterized by four parameters, {ξ i } 4 i=1 , which are the coordinates of the lower left and upper right corners of the rectangular region. The sample set S is created by sampling each parameter from a uniform distribution (xi i ∼ U (0.25, 0.75) × L) and is used to train the GAN prior. The posterior distribution is sampled using the HMC sampler.

In the top two rows of Figure 6 , we have plotted the true initial condition, the noise-free temperature at t = 1, and the noisy temperature measurement (σ y = 1) used as input in the GAN-based prior approach. The corresponding MAP, mean and pixel-wise variance estimated by the MCMC approximation are shown next. We observe that the MAP is very close to the true initial temperature distribution and the variance is concentrated along the edges of the rectangle where the uncertainty is the largest. In the following columns we have plotted the MAP estimate obtained assuming L 2 and H 1 Gaussian priors, which are often used to solve these types of problems, and are clearly much less accurate.

For this problem the "true" posterior can be reduced to the 4-dimensional space of parameters, and sampled by generating initial conditions corresponding to the values of these parameters. A simple MC approximation can be performed to compute statistics -the mean and the pixel-wise variance for the true posterior (last two columns of Figure 6 ). By comparing these with the mean and the pixelwise variance (columns 5 & 6) estimated by the GAN-based prior, we conclude that GAN-based posterior has converged to the true posterior.

In the bottom rows of Figure 6 , we plot similar results for initial conditions and GAN-based priors generated from the MNIST database. In this case the measurement is made at t = 0.2. Since the "true" distribution for this set is not known the true mean and variance are not plotted.

4. CONCLUSIONS:

The ability to quantify the uncertainty in an inference problem is useful in developing confidence in that inference, and in designing strategies to improve the confidence. In this paper we have described how this may be accomplished when solving a Bayesian inference problem by using GANs as priors. Since GANs can learn complex distributions of a wide variety of fields from their 4), ( 5), ( 6) MAP, mean and pixel-wise variance estimates using GAN priors, ( 7) and ( 8) MAP estimates using L 2 and H 1 Gaussian priors, ( 9) & ( 10) true MAP and variance obtained by sampling over the true parameter space. samples, this approach can be applied to a range of problems in computer vision and physics-driven inference. It derives its efficiency by mapping the posterior distribution to the latent space, whose dimension is often much smaller than that of the inferred field. We have applied this approach to image recovery tasks and demonstrated how the knowledge of uncertainty in the prediction can be used to assess confidence in a prediction, and via active learning to design a strategy to improve it. We have also applied this approach to a physics-based problem, where we have verified its accuracy and robustness. In the results reported in this manuscript we assume knowledge of the forward map; however, we note that the proposed algorithm can easily be extended to a regime where the forward map is also unknown by utilizing likelihood-free inference methods like ABC or meta-learning approaches. We leave this as an interesting direction for future research.

A WEAK CONVERGENCE OF THE POSTERIOR DENSITY:

In this section we prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights in the generator and discriminator are increased.

Let the generator of the GAN be given by g(z; θ), where z ∈ R M is the latent vector, and θ ∈ R N θ is the vector of weights. The vector z is selected from the distribution p Z (z). Note that g : R M → R N .

Let the discriminator of the GAN be given by d(x; φ), where x ∈ R N , and φ ∈ R N φ be the vector of weights. Note that d : R N → R(0, 1).

The GAN is trained using a set of samples of x, drawn from p prior X (x). For the loss function, consider

EQUATION

)

Here ρ is a monotone real-valued function which defines the GAN family being analyzed. For example, for the Wasserstein GAN, ρ(ξ) = ξ.

The optimal values of the weights are given by

EQUATION

A.1 STATIONARITY CONDITIONS

The necessary conditions for these optimal values are

EQUATION

)

Using the definition of the loss function ( 10) in ( 12), we have

EQUATION

Similarly, using (10) in ( 13), we have

EQUATION

A.2 WASSERSTEIN GAN For the Wasserstein GAN, ρ(ξ) = ξ and ρ (ξ) = 1. As a result ( 14) and ( 15) reduce to,

EQUATION

EQUATION

Let w a (x) ≡ ∂d ∂φa (x; φ * ), a = 1, • • • , N φ . Then ( 16) implies

EQUATION

)

As N φ → ∞, this is a weak statement of the equivalence of p prior X and p gen X , where the latter is defined in (3). In particular this says that the push forward of the measure in the latent space under the function g(z) weakly converges to the measure associated with distribution of x. We note with increasing number of weights in the discriminator, the relation above is required to hold for an increasing number of test functions, w a . In addition, we have implicitly assumed that the generator is rich enough, that is it has enough weights/layers, such that this relation can actually be satisfied. To make this clear consider the extreme case of a generator with a single weight; in this case there is no way that (18) will be satisfied for a large number N φ . Thus in order for this relation to hold for a large N φ , we must also provide the generator with a large N θ . Now consider a sufficiently smooth function m(x) which defines the point estimate we wish to compute. Expand it using w a (x) as basis such that

EQUATION

and the coefficients are selected to minimize

EQUATION

As we increase N φ the dimension of the basis used to represent m increases, and therefore ¯ tends to zero.

Given this,

EQUATION

EQUATION

EQUATION

EQUATION

This yields,

EQUATION

EQUATION

Note that in deriving this estimate, in the third line we have assumed that g(z; θ * ) ∈ Ω x . That is, the generator does not land outside the domain of x. The statement above bounds the error in computing a population parameter for the prior using a GAN.

A.3 CONVERGENCE TO THE POSTERIOR DENSITY:

In order to turn this estimate into an estimate for the error in approximating a population parameter for the posterior density, we simply choose m(x) = l(x)pη( ŷ-f (x)) Z

. With this choice,

EQUATION

and

EQUATION

Using these in ( 22) we have the desired result,

EQUATION

This result demonstrates that the difference between the true population parameter for the posterior and its approximation obtained after using a GAN as a prior (the method proposed in the paper) reduces as the number of weights in the generator and the discriminator are increased. In the limit of infinite weights, ¯ → 0, and the equation above is a statement of weak convergence of the posterior obtained by using the GAN and the true posterior density.

B EXPRESSION FOR THE MAXIMUM A-POSTERIORI ESTIMATE:

The techniques described in Section 2.1 focus on sampling from the posterior distribution and computing approximations to population parameters. These techniques can be applied in conjunction with any distribution used to model noise and the latent space vector; that is, any choice of p η (likelihood) and p Z (prior). In this section we consider the special case when Gaussian models are used for noise and the latent vector. In this case, we can derive a simple optimization algorithm to determine the maximum a-posteriori estimate (MAP) for p post Z (z|y). This point is denoted by z map in the latent vector space and represents the most likely value of the latent vector in the posterior distribution. It is likely that the operation of the generator on z map , that is g(z map ), will yield a value that is close to x map , and may be considered as a likely solution to the inference problem.

We consider the case when the components of the latent vector are iid with a normal distribution with zero mean and unit variance. This is often the case in many typical applications of GANs. Further, we assume that the components of noise vector are defined by a normal distribution with zero mean and a covariance matrix Σ. Using these assumptions in (6), we have

EQUATION

)

The MAP estimate for this distribution is obtained by minimizing the negative of the argument of the exponential. That is

EQUATION

This minimization problem may be solved using any gradient-based optimization algorithm. The input to this algorithm is the gradient of the functional r with respect to z, which is given by

EQUATION

where the matrix H is defined as

EQUATION

)

Here ∂f ∂x is the derivative of the forward map f with respect to its input x, and ∂g ∂z is the derivative of the generator output with respect to the latent vector. In evaluating the gradient above we need to evaluate the operation of the matrices ∂f ∂x and ∂g ∂z on a vector, and not the matrices themselves. The operation of ∂g ∂z on a vector can be determined using a back-propagation algorithm with the GAN; while the operation of ∂f ∂x can be determined by making use of the adjoint of the linearization of the forward operator.

Once z map is determined, one may evaluate g(z map ) by using the GAN generator. This represents the value of the field we wish to infer at the most likely value value of latent vector. Note that this is not the same as the MAP estimate of p post X (x|y). 

:



Figures and Tables:

(McCarthy et al. (2017)), authors perform variational approximation of the posterior using VAEs with simulator-based decoder to solve physics-based inference problem. We note that deep learning based Bayesian networks, where the network weights are stochastic parameters that are determined using Bayesian inference, are another means of quantifying uncertainty (MacKay (1992); Kingma & Welling (2013);Gal & Ghahramani (2016)), and have recently been applied to semantic image-segmentation and superresolution(Kendall & Gal (2017);Kendall et al. (2019);Kohl et al. (2018);Hu et al. (2019);Tanno  et al. (

Figure 4: (a) Average variance per pixel in a reconstructed image as a function of variance in noise for 10 digits (along with 95% confidence interval). (b) Average reconstruction error (along with 95% confidence interval) as a function of number of windows for a variance-driven (adaptive learning) and a random sampling strategy.

Figure 5: Estimate of the MAP, mean and variance from the limited view of a noisy image (2nd row) of a true image (1st row) using the variance-driven adaptive learning strategy.

Figure 6: From left to right: (1) true initial temperature, (2) temperature at t = 1, (3) noisy version temperature used as measurement, (4), (5), (6) MAP, mean and pixel-wise variance estimates using GAN priors, (7) and (8) MAP estimates using L 2 and H 1 Gaussian priors, (9) & (10) true MAP and variance obtained by sampling over the true parameter space.

Figure7: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from the limited view of a noisy image (2nd row) using the proposed method. The window to be revealed at a given iteration (shown in red box) is selected using a variance-driven strategy. Top row indicates ground truth. For all digits σ y = 1.18

Figure 8: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from a noisy image (2nd row) using the proposed method. Top row shows ground truth. For all the examples σ y = 1.

Figure9: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from the limited view of a noisy image (2nd row) using the proposed adaptive method. The window to be revealed at a given iteration (shown in red box) is selected using a variance-driven strategy. Top row indicates ground truth. For all images σ y = 1.

Figure 11: Generator and discriminator architectures,

Peer review (iclr2020):
Summary of the paper: The paper proposes a Bayesian approach to make inference about latent variables such as un-corrupted images. The prior distribution plays a key role in this task. The authors use a GAN to estimate this prior distribution. Then, standard Bayesian techniques such a Hamilton Monte Carlo are used to make inference about the latent variables. Detailed comments: Eq. (8) is expected to give very bad results. The reason is that it is very unlikely to sample from the prior configurations for z that are compatible with y. The paper does not address learning any model parameters. e.g. the amount of noise. A more principled approach would be to estimate the prior parameters using maximum likelihood estimation. That has already been done in the case of the variational autoencoder. The variational autoencoder is an already known method that can be used to solve the problem formulated by the authors. It also automatically proposes an inference network that can be used for recognition. If the likelihood is Gaussian and p(x|z) is also Gaussian, one can directly marginalize x and work with p(y|z) and p(z). The authors should at leas discuss the potential use of this method alongside with the BIGAN model which also provides a recognition model. It is not clear how the HMC parameters are fixed. The experiments do not have error bars (Figure 4.) This questions the significance of the results. My overall impression is that there is little novelty in the proposed approach. Namely, using a GAN to learn the prior distribution, and then very well known techniques to infer the original input image. I have missed some references to related work on inverse problems. An example is: https://arxiv.org/pdf/1712.03353.pdf Is the original figure contained in the training set used to infer the GAN. If so that can lead to biased results. I have missed a simple baseline in which one simply finds the training image that is closest to the corrupted observed or partially observed image. My overall impression is that there is not much novelty in the paper as it is simply a combination of well known techniques. E.g. GANs and Bayesian inference with Monte Carlo methods.