Quantifying uncertainty with GAN-based priors
Authors: Dhruv V. Patel
Assad A. Oberai
Venue:  (2020)
Keywords: Bayesian inference
Uncertainty quantification
Generative adversarial networks

Abstract: Bayesian inference is used extensively to quantify the uncertainty in an inferred field given the measurement of a related field when the two are linked by a mathematical model. Despite its many applications, Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically. In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges. We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in inference problems arising in computer vision and  physics-based applications. In both instances we highlight the role of computing uncertainty in providing a measure of confidence in the solution, and in designing successive measurements to improve this confidence. 

Under review as a conference paper at ICLR 2020
QUANTIFYING UNCERTAINTY WITH GAN- PRI- ORS
Anonymous authors Paper under double-blind review
ABSTRACT
Bayesian inference is used extensively quantify the uncertainty in an inferred field given the measurement of a related field when the two are linked by mathe- matical model. Despite its many applications. Bayesian inference faces challenges when inferring fields that have discrete representations of large dimension, and/or have prior distributions that are difficult to characterize mathematically. In this work we demonstrate how the approximate distribution learned by generative adversarial network (GAN) may used as prior in a Bayesian update to address both these challenges. We demonstrate the efficacy of this approach by inferring and quantifying uncertainty in inference problems arising in computer vision and physics- based applications. In both instances highlight the role of computing uncertainty in providing mcasure of confidence in the solution, and in designing successive measurements to improve this confidence.
INTRODUCTION
Quantifying uncertainty in inference problem amounts making prediction and quantifying the confidence in that prediction. In the context of an image recovery problem, this may be understood as follows. A typical computer vision algorithm uses noisy version of an image and prior knowledge to produce the recovered image which can be interpreted as the "best guess" of the original image. Quantifying uncertainty in this context involves generating an estimate of the level of confidence in the best guess, in addition to the guess itself.
Bayesian inference provides principled approach for quantifying uncertainty. shown in the following section, it treats the inferred vector as multivariate stochastic vector and leads to an expression for its distribution. This expression can be used to estimate the most likely solution (the maximum a-posteriori estimate, or the MAP), the mean, the variance, any other population parameter of interest. Thus providing a recipe for thoroughly quantifying the uncertainty in an in- ference problem. For the image recovery problems considered in this paper, Bayesian inference not only provides the best guess of the true image, but also a means to estimate measures of uncertainty such the pixel-wise variance.
The knowledge of uncertainty in a prediction can directly influence the downstream action that depends on the inference. Consider an image recovery problem where two distinct inputs lead to similar recovered images: those of a traffic sign with a high speed limit. However, for the first input the predicted variance is small, while for the second input is large. Further, the of likely images in the second set also includes images of Stop Sign. Then the appropriate action for the two inputs, determined after solving the inference problem and quantifying uncertainty, is very different. For the first input, the appropriate action is one of continued motion, whereas for the second input it is to slow down. Similar examples can be drawn from other areas as well, like medical imaging, high frequency trading, and autonomous systems where critical decisions are made based on the output of AI system (Gal (2016): Begoli et al. (2019)).
The knowledge of uncertainty can also be useful in determining the optimal location of a sensor. Consider an image recovery problem, where the goal is to infer the signal, and associated uncer- tainty, using limited amount of measurement data. In this problem user leverage information about the spatial distribution of uncertainty to choose the location with maximum uncertainty as next measurement location. This task falls within the fields of active learning and/or design of ex-

---

Under review as a conference paper at ICLR 2020
periments (DeGroot et al. (1962); Houlsby et al. (2011) and is particularly useful in applications like satellite imaging, where each measurement requires significant time and/or resources.
In Figure D demonstrate how the proposed based Bayesian inference algorithm is ful in both scenarios explained above: (i) in quantifying uncertainty "pixel-wise variance" our quantitative measure of uncertainty in the inferred field, and (ii) in determining the optimal sensor placement location in an iterative fashion. We return to these applications in greater detail in Section
Ma =3
Measurement
Reconstructed signal
Xmup
Model uncertainty
output
Yeor
Figure 1: Estimate of the MAP (2nd row) and pixel-wise variance (3rd row) from the limited view of noisy image (Ist row) using the proposed method. The first five columns correspond to strategy where the next window is selected randomly, while the other columns correspond to a strategy where the next window selected in the region with maximum estimated variance. For equivalent accuracy, the variance-drivee selection strategy uses fewer sampling windows (4 versus 9). In both cases variance reduces with increasing measurement.
1.1 BAYESIAN INFERENCE
Bayesian inference a well-established technique for quantifying uncertainties in inference prob- lems (Kaipio & Somersalo (200)); Dashti & Staart (2016): Polpo et al. (2018)). It has found appli- cations diverse fields such as geophysics (Gouveia & Scales 1997); Malinverno (002)) ) climate modeling (Jackson et al. (0004)), chemical kinetics (Najm et al. (209 ). heat conduction (Wang & Zabaras (2004)), and the detection and diagnosis disease (Siltanen et al. (2003); Kolehmainen al..:.200)))))))l (2006)). The two critical ingredients of Bayesian inference problem are informative prior representing the prior belief about the parameters to be inferred and an efficient method for sampling from the posterior distribution. In this manuscript we describe how deep generative adver- sarial networks (GANs) can be effectively used in these roles.
Consider the setting where wish to infer vector of parameters R. from the measurement of a related vector y E RR,,, where the two are related through forward model y f(x). A noisy measurement of y is denoted by ÿ f(x) + n). where the vector 7) R2 represents noise. While the forward map ff is typically well-posed, its inverse is not, and hence to infer æ from the measurement ý requires techniques that account for this ill-posedness. Classical techniques based on regularization tackle this ill- poleposessess by using additional information about the sought solution field explicitly or implicitly (Tarantola (2005).. Bayesian inference offers a different approach to this problem by modeling the unknown solution as well as the measurements random variables. This framework addresses the pol-posedeess of the inverse problem, and allows for the characterization of the uncertainty in the inferred solution.
The notion of a prior distribution plays a key role Bayesian inference. Through multiple obser- vations of the field æ, denoted by the set = {1x(1).... ... .s))), we have some prior knowledge of that can be utilized when inferring from ÿ. This is used to build, or intuit, prior distribution for denoted by prrioo (x). Some typical examples include Gaussian process prior with specified co-variance kernels. Gaussian Markov random fields (Fahrmeir & Lang (2001)). Gaussian priors defined through differential operators (Stuart (2010)). and hierarchical Gaussian priors (Marzouk & Najm (2009): Calvetti & Somersalo (2008). These priors promote smoothness and/or structure in the inferred solution and importantly, be expressed explicitly in analytical form.
2

---

Under review as a conference paper at ICLR 2020
Another key component of Bayesian inference is distribution that represents the likelihood of given an instance of æ, denoted by p' (yyy(z). This is often determined by the distribution of the error in the model, denoted by Py, which captures both model and measurement enror.. Given this, and an additive model for noise, the posterior distribution of æ; determined using Bayes' theorem after accounting for the observation is given by,
PX (a y)
1 Z (v)=))p prior (x) (x)= P ()=((()()))33
1
(x prior (x).
(1)
Here, Z is the prior-predietie distribution of y and ensures that the posterior integrates to one.
The posterior distribution characterizes the uncertainty in æ; however for vectors of large dimension characterizing this distribution explicitly is challenging task. Consequently the expression above is used to perform tasks that are more manageable. These include determining estimates such as the maximum a p-ssterioni estimate (MAP), expanding the posterior distribution in terms of other distributions that are simpler to work with Chain Monte-Carlo (MCMC) to generate samples that are "close" the samples generated by the true posterior distribution (Han & Carlin 2001): Parno & Marzouk (2018)).
et al. )2012), ) or using techniques like Markov
Despite its numerous applications in solving inverse problems, Bayesian inference faces significant challenges. These include defining reliable and informative prior distribution for æ when the set = {x(1)) ..... xx)} } is difficult to characterize analytically, and efficiently sampling from the posterior distribution when the dimension of is large; a typical situation in many practical science and engineering applications.
1.2 RELATED WORK
The main idea developed in this paper tackles the above mentioned challenges by training a gener- ative adversarial network (GAN) using the sample set 8, and then using the distribution learned by the GAN as the prior distribution in Bayesian inference.
The of sample-based priors for solving inverse problem has rich history (Vauhkonen et al. [1997]; Calvetti & Somersalo (2005)). As does the idea of reducing the dimension of the parame- ter space by mapping lower-dimensional space (Marzouk & Najm (2009); Lieberman et al. (210)). However, the use of learning- based deep generative models like GANs in these tasks is novel.
Recently, several authors have considered the use of learning- -baii methods for solving inverse problems arising in different domains. These include the use of deep convolutional neural networks (CNNs), recurrent neural networks (RNNs) or variational autoencoders (VAEs) to solve physics- driven inverse problems (Adler Öktem (2017);
: Patel al. 2011);: Pesah et al. (2018)). VAËs and GANs have also been used to solve inverse problems in computer vision (Yeh et al. (2016); Ham et al. Chang et al.; Kupyn et al. (2018); Ledig et al., Zhu et al. (2017)). There is also a growing body of work dedicated to using GANs to learn regula larzerr in solving inverse problems (Lunz al. (2018) and in compressed sensing (Bora et al. (2017) 2018); Kabkab et al. (2018); Wu et al. (2019); Shah & Hegde (0018). However, these approaches differ from ours in that they solve the inverse problem as an optimization problem and do not rely on Bayesian inference; as a result, they add regularization in an ad-hoc manner and do not attempt to quantify the uncertainty in the inferred
field.
More recently, the approach described in (Adler & Öktem (018)) utilizes GANs in Bayesian setting; however the GAN is trained to approximate the posterior distribution (and not the prior, as in our case), and training done in a supervised fashion. That is, paired samples of the mea- surement ý and the corresponding true solution are required. In (McCarthy et al.) (2017)). au- thors perform variational approximation of the posterior using VAES with simulator- based decoder to solve physics-based inference problem. We note that deep learning based Bayesian networks, where the network weights are stochastic parameters that are determined using Bayesian inference, another means of quantifying uncertainty (MacKay (1992); Kingma & Welling (2013); Gal & Ghahramani (0016)), and have recently been applied to semantic image- seagg--eemmetttiion and super- resolution (Kendall& & Gal (2017); Kendall et al. (2019); Kohl et al. 2018); Hu et al. (2019) : Tanno jet al.] (2019) However, these method also rely on supervised learning, whereas in contrast, our approach is unsupervised, and requires only samples of the true solution æ train the GAN prior.
3

---

Under review as a conference paper at ICLR 2020
1.3 OUR CONTRIBUTION
The main contribution of this paper can be summarized as follows:
1. A novel method for performing Bayesian inference involving complex priors and high di- mensional posterior. In this method we utilize the distribution learned by a GAN surro- gate for the prior distribution and reformulate the inference problem in the low-dimensional latent space of the GAN. Furthermore, we provide theoretical analysis of the weak con- vergence of the posterior density learned by the proposed method to the true posterior density.
2. Novel unsupervised image denoising and inpainting algorithms with quantitative measures of uncertainty through pixel-wise variance.
3. Application of the proposed method physics-based -bsed inference problems.
4. Demonstration of the utility of quantifying uncertainty to facilitate active learning.
2 PROBLEM FORMULATION
Let 8 denote the set of instances of vector sampled from the true distribution, (x) Further, let z ~ (z) characterize the latent vector space and g(z) be the generator a GAN trained using S. Then according to Goodiellow et al. (2014), with infinite capacity and sufficient data, the generator learns the true distribution. That is,
PX (x) Px
true (x)).
The distribution Px (x) is defined
x
(x) g ~ pz(z).
Here pz is the multivariate distribution of the latent vector whose components are iid and typically conform to a Gaussian or uniform distribution. The equation above implies that the GAN generates samples of x: by sampling from pz and then passing these through the generator.
Now consider a measurement ý from which we would like to infer the posterior distribution of æ. For this we use (1) and set the prior distribution equal to the true distribution, that is px prior pX Therefore,
PX
Then from this is the same as Px
prior
1 PPo f(x)) (z).
post (xly) p
(4)
Now for any ((x), have
[[xx]]
E
(0 f(x)))
From
Z m-pp
E [(g(()())))))))))))))))))))))))) f(g(2)))
From 3
Z ж~pz
E [(g())))
(5)
2-2
where E is the expectation operator, and
post pz
(x(y)
pr f(g(z)))pz(2).
(6)
The distribution pZ post the analog of post in the latent vector space. The measurement ý updates the prior distribution for the posterior distribution. Similarly, it updates the prior distribution for pz, to the posterior distribution, post defined above.
Equation (5) implics that sampling from the posterior distribution of æ is equivalent to sampling from the posterior distribution for and passing the sample through the generator g. That is,
pX (xly)
g(=), PZ post (zly).
4

---

Under review as a conference paper at ICLR 2020
Since the dimension of z is typically smaller than that of æ, this represents efficient approach to sampling from the posterior of æ.
The left hand side of (5) is an expression for population parameter of the posterior, defined by [(x) B [- The right hand sides of the last two lines of this equation describe how this parameter may be evaluated by sampling (instead of æ:) from either pz or p2
The equality in (5) holds for a GAN with an infinite number of weights in the generator and the discriminator. In Appendix A of this manuscript, consider the case of a Wasserstein GAN with finite number of weights and prove the weak convergence of the posterior density obtained by using GAN prior the true posterior density the numher of weights is increased.
2.1 SAMPLING FROM THE POSTERIOR DISTRIBUTION
We consider a scenario where we wish to infer and characterize the uncertainty in the vector of parameters from a noisy measurement of y, denoted by y. where f is a known map that connects and y. And we have several prior realizations of plausible æ, contained in the set 8. For this problem we propose the following algorithm that accounts for the prior information 8 and the "new" measurement through Bayesian update:
1. Train GAN with a generator g(z) on S.
2. Sample from post (aly) given in 7
With sufficient capacity in the GAN and with sufficient training, the posterior obtained using this algorithm will converge the true posterior (see eq- (5) above and Appendíx A). Since GANs can be used to represent complex distributions efficiently, this algorithm provides a means of including complex priors that are solely defined by samples within Bayesian update. Further, provides an efficient approach to sampling from the posterior since the dimension of = is typically much smaller (10² -02--100 100) than that of æ (10a- 10²). Finally, in contrast to other methods that attempt to quantify uncertainty in image recovery tasks, this algorithm falls within the class of unsupervised learning algorithms,, and does not require paired data for training. We now describe two approaches for estimating population parameters of the posterior using this algorithm.
Monte-Carlo (MC) approximation The first approach is based on a Monte-Catt approximation of population parameter of the posterior distribution. This integral, which is defined in the second line of 5) may be approximated as,
>V (U-
((x)
E Zrup
[[x)]] ~
~~ pz)x.
(8)
ENamp Pn f(g(2)))
In the equation above, the numerator is obtained from a MC approximation of the integral in (5). and the denominator is obtained from MC approximation of the scaling parameter Z. Sampling within this approach is simple since in a typical GAN, the 2,8 belong to a simple distribution like a
Gaussian uniform distribution. However, anticipate that in many applications the likelihood will tend to concentrate the distribution of latent vector z to a small region within Ω2 and thee sampling described above may be inefficient.
Markoy-Chain Monte-Carlo (MCMC) approximation A more efficient approach is to generate an MCMC approximation pm (x|y) = p2 (y) using the definition in (6). and thereafter sam- ple z from this distribution. Then from the third line of (5) any desired population parameter may be approximated as
kamp
(xx)
E x
[(x))
l(g(2)),
Z pz
mcmc
(epemc(((l)) (zly)-
samp
We note that for all the numerical experiments in this paper we have used MCMC because of its better sample efficiency.
Summary We have described two methods for probing the posterior distribution when the prior is defined by a GAN. These include an MC (8) and an MCMC estimate 9 of a given population parameter and MAP estimate that is applicable to additive Gaussian noise with Gaussian prior for the latent vector (see Section B in the Appendix). In the following section we apply the MCMC approach to inverse problems drawn from physics-based and computer vision applications.
5

---

Under review as a conference paper at ICLR 2020
3 EXPERIMENTS
We evaluate our method empirically on practical probabilistic field inference tasks in the domain of computer vision and physical science. In computer vision, we consider image denoising and inpainting tasks and evaluate our algorithm on MNIST and CelebA datasets. Here we attempt to infer the true image and quantify the uncertainty in this inference. We also demonstrate the utility of quantifying uncertainty by using it in an active learning setup. In the physics- based application, we consider the problem of recovering the initial temperature distribution from a measurement later time, and use this problem to demonstrate the statistics inferred using our method converge to their "true" values.
In all cases we use a Wasserstein GAN-GP (Gulrajani et al. (2017) to learn the prior density (archi- tecture described in the Appendix D) We also ensure that the target images are not chosen from the set used to train the GAN. We sample from the posterior using Hamiltonian Monte Carlo (Brooks et al. (2012)) and implement it using Tensorflow probability (Dillon et al. (2017)) library. We use initial step size of 1.0 for HMC and adapt it following (Andrieu & Thoms (2008)) based on the target acceptance probability. We use 64k samples with burn-in period of 0.5. We select these parameters ensure convergence of chains. Using the HMC sampler compute the MAP, which is our "best guess" of the true image, the pixel-wise mean, and the pixel-wise variance, which our quantitative and spatially-varying estimate of uncertainty.
Tensorilowprobbbbliiyy
3.1 IMAGE RECOVERY USING THE MNIST & CELEBA DATABASES
We first consider the MNIST database of hand-written digits and use 55k images to train the GAN. We use latent vector dimension 100 with Gaussian distribution. For the image denoising task, we add Gaussian noise with zero mean and specified variance (o,,) to the test image and use it as measurement to recover the distribution of likely images using the MCMC approach. For this problem the forward operator is the identity map, and the likelihood distribution is Gaussian. In Figure we have plotted the noisy input image, the MAP estimate, and the pixel-wise mean and variance. We observe that for low and medium noise levels (o, 0.1, 1), are able to recover the original image with good accuracy, the pixel-wise variance is small overall, and is largest around the boundary of the recovered digit; this represents the variability in the different realizations of the recovered digit within the GAN prior. For the highest noise level (a, = 10). however, the image recovered by the MAP is incorrect in 2/3 cases, and would be misleading if viewed by itself. However, when viewed in conjunction with the estimated variance, which is large, it is clear that the confidence in the inference is small and the inferred image ought not be trusted for downstream tasks. The dependence of the average per-pixel variance in the recovered image on the variance of noise the measured image is shown in Figure 4 (a) with 95% confidence interval. As expected, the per-pixel variance increases with increasing noise.
Messurement 2
5
2 2
Z
5
5
9
6
6
4
3 3
a
2
2
6
5
5
7
6
6
3 3
9
internei signet
Variance interredriggull
2
6
2
3
Figure 2: Estimate of the MAP, mean and pixel-wise variance from a noisy image using the proposed method. In the first three panels o, = 0.1, 1, &00, when moving from left to right. In the fourth panel og = I. and the size of the occluded region is increased.
In the right-most panel of Figure 2 we show image inpainting denoising results for image of digit 3 with o, = 1 Here the forward map is the indicator function set to zero on the occluded pixels. We note that for the small and intermediate occluded regions, the MAP solution is close to the true solution. However, when most of the image is occluded, the MAP is incorrect. Once again, the variance image, which is small for the low and medium occluded regions, and large for

---

Under review as a conference paper at ICLR 2020
the large occlusion, is reliable indicator of the confidence in the recovered MAP image. Fur- thermore, the variance is peaked within the occluded region demonstrating lower confidence in the image reconstructed in this region. This is useful in applications like autonomous cars and medical imaging, where partial measurements are common and the spatial distribution of uncertainty can in- form downstream tasks. More image denoising and inpainting examples are provided in Appendix
8
8
8
8
8
5555
5
5
Tive
Measurement
1
y
6
3 3
i
O
3 5
6 5
5 6
5 5 5
3
7
8
8
9
Veriance
0
0
-
5
6
Figure 3:
Estimate of the MAP, mean and variance from the limited view noisy image (2nd row) using the proposed method for the digits 8 & (left and right panels). The window to be revealed at a given iteration (shown red box) is selected using a variance-driven strategy.
In Figure D we demonstrate how uncertainty may be used in active learning/design of experiment, where the goal is to determine the optimal location for a measurement. We begin with an input where the entire image is occluded and in every subsequent step, we allow for small 7x7 pixel window to be revealed. We select the window with the largest average pixel-wise variance. As the iterations progress, the MAP estimate converges
the true digit, and the variance decreases. In about 4 iterations we arrive at very good guess for the digit. The performance of this approach is quantified Figure 4b), where have plotted reconstruction error versus the number windows for this strategy, and a strategy where the subsequent window is selected randomly. The variance- driven strategy consistently performs better. We note that we are not aware of any other methods for computing uncertainty in recovered images that have been applied to drive an active learning task in image inpainting. While methods based on dropout (Kendall & Gal (2017): Kendail al. (2019) or variational inference (Kohl et al. (2018) could be extended to accomplish this, this has not been done thus far.
Log(per-pixe) per-ppiee variance)
por pixel rezan, error (422
0.025
stratesy
0.020
0.015
12
2.0
10
Number of wi windows
(a)
(b)
Figure 4: (a) Average variance per pixel in a reconstructed image as a function variance noise for 10 digits (along with 95% confidence interval). (b) Average reconstruction error (along with 95% confidence interval) as a function of number of windows for a variance-driven (adaptive learning) and random sampling strategy.
Results for the variance-beie based window selection strategy applied to the CelebA dataset are shown in Figure We observe that the algorithm produces realistic images at each iteration; however, the ini- tial variance is large indicating large uncertainty. As more windows are sampled using the variance driven active learning strategy, the variance reduces and by the 7th iteration good approximation of the true image is obtained, even though only a small, noisy portion is revealed. Additional results along with implementation details for this dataset are discussed in Appendix C.2
7

---

Under review as a conference paper at ICLR 2020
Measurement
Meen
Variance
Figure 5: Estimate of the MAP, mean and variance from the limited view of a noisy image (2nd row) of true image (1st row) using the variance-driven adaptive learning strategy.
3.2 A PHYSICS-DRIVEN INFERENCE PROBLEM
We consider the problem of inverse heat conduction, where the goal infer the initial temperature distribution (at & = 0) in a domain given a noisy measurement of temperature at later time (t = 1) and the thermal conductivity of the material. The forward map the solution of the time-dependent heat conduction problem with uniform conductivity, к 0.64, and in square domain of length = 2π with Dirichlet boundary conditions. This operator maps the initial temperature field the temperature field at later time. The discrete version of this operator is obtained by discretizing the dime-dependent linear heat conduction equation using the central difference scheme in space and backward Euler scheme in time. Much like blurring kernel, the forward operator smooths the initial temperature distribution, and the extent smoothing increases with x x t.
We consider a family of initial temperatures where the background is zero, and the temperature a rectangular sub-domain varies linearly from 2 units on the left edge to 4 units on the right edge. This distribution is parameterizze by four parameters, {E, ] ) which are the coordinates of the lower left and upper right corners of the rectangular region. The sample set S is created by sampling each parameter from uniform distribution (riq (0.25,, 0.75) x L.) and is used to train the GAN prior. The posterior distribution is sampled using the HMC sampler.
In the top rows of Figure 6 have plotted the true initial condition, the noise-free free tempera- ture at = 1, and the noisy lemperature measurement (a, = 1) used as input in the GAN-based prior approach. The corresponding MAP, mean and pixel-wise variance estimated by the MCMC approximation are shown next. We observe that the MAP is very close to the true initial temperature distribution and the variance is concentrated along the edges of the rectangle where the uncertainty is the largest. In the following columns we have plotted the MAP estimate obtained assuming T and H Gaussian priors, which are often used to solve these types of problems, and are clearly much less accurate.
For this problem the "true" posterior can reduced to the 4-dimensional space of parameters, and sampled by generating initial conditions corresponding to the values of these parameters. A simple MC approximation can be performed to compute statistics the mean and the pixel-wise variance for the true posterior (last two columns of Figure [6). By comparing these with the mean and the pixel- wise variance (columns 5 & 6) estimated by the GAN-based prior, we conclude that GAN-based posterior has converged to the true posterior.
In the bottom rows of Figure 6 we plot similar results for initial conditions and GAN-based priors generated from the MNIST database. In this case the measurement is made 0.2. Since the "true" distribution for this set is not known the true mean and variance are not plotted.
4 CONCLUSIONS
The ability to quantify the uncertainty in an inference problem is useful in developing confidence in that inference, and in designing strategies improve the confidence. In this paper have described how this may be accomplished when solving a Bayesian inference problem by using GANs as priors. Since GANs can learn complex distributions of wide variety of fields from their
8

---

Under review as a conference paper at ICLR 2020
Verlasee frudl
nolo) Inake]
3
3
3
3
4
4
4
4
Figure 6: From left to right: (1) true initial temperature, (2) temperature at = (3) noisy version temperature used as measurement, (4), (5), (6) MAP, mean and pixel-wise variance estimates using GAN priors, (7) and (8) MAP estimates using L2 and H' Gaussian priors, (9) & (10) true MAP and variance obtained by sampling over the true parameter space.
samples, this approach can be applied to range of problems computer vision and physics-driven inference. It derives its efficiency by mapping the posterior distribution to the latent space, whose dimension is often much smaller than that of the inferred field. We have applied this approach to image recovery tasks and demonstrated how the knowledge of uncertainty in the prediction can be used assess confidence a prediction, and via active learning to design a strategy to improve it. We have also applied this approach physics-based problem, where have verified its accuracy and robustness. the results reported in this manuscript we assume knowledge of the forward map; however, note that the proposed algorithm can easily be extended to regime where the forward map is also unknown by utilizing likelihood-free inference methods like ABC or meta-learning approaches. We leave this as interesting direction for future research.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Jonas Adler and Ozan Öktem. Solving ill-posed inverse problems using iterative deep neural net- works. Inverse Problems, 33(12):124007, dec 2017. ISSN 0266-5611. doi: 10000//66110000 aa9581. /ttp://////.aeaoooo....o//0////.
crossref. TTo.:.e..6...aaaaa11777777770000
Jonas Adler and Ozan Öktem. Deep bayesian inversion. arXiv preprint wrXiv:/8/.009// 2018.
Christophe Andrieu and Johannes Thoms. A tutorial on adaptive MCMC. Statistics and Computing, 18(4):343-373, dec 2008. ISSN 09603174, doi: 10. 10.1007/s1 11tp://11122000888 000-220000---------- 9110-y.
Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantifica- tion in machine-assisted assisted medical decision making, Nature Machine Intelligence, 1(1):20, 2019.
Ashish Bora, Ajil Jalal, Eric Price, and Alexandros Dimakis. Compressed sensing using genera- tive models. In Proceedings of the 34th International Conference Machine Learning- Voluue 70, pp. 537-546. JMLR. org, 2017.
Ashish Bora, Eric Price, and Alexandros G Dimakis. Ambientgan: Generative models from lossy measurements. ICLR, 2:5, 2018.
Steve Brooks, Andrew Gelman, Galin Jones, Xiao-Li Meng, and Radford M Neal. MCMC using Hamiltonian dynamics. Technical report, 2012. URL htttps://arxiv.org///oo
1901. pdf
Tan Bui-Thanh, Carsten Burstedde, Omar Ghattas, James Martin, Georg Stadler, and Lucas C. Wilcox. Extreme-scale UQ for Bayesian inverse problems governed by PDEs. In 2012 In- ternational Conference for High Performance Computing, Networking, Storage and Analysis,
Pp. 1-11. IEEE, nov 2012. ISBN 978- 978-1-4673-0805-2. doi: 10.1109/SC.2012.56. URL
http: /itp:://eeeepporeeee wee.. 0233/oocummnt//444422
Daniela Calvetti and Erkki Somersalo.
Priorconditioners for linear systems. In- verse Problems, 11444)139711418 aug 2005. ISSN 0266-5611.
doi: 10.1088/
0266-5611/21/40000 URL
key=crossref. ab4 9f e3 bf
Daniela Calvetti and Erkki Somersalo. Hypermodels the Bayesian imaging framework.
verse Problems, 24(3)0034013, jun 2008. ISSN 0266-5611. doi: 10. 10.1088/0266-500 56111// /tt.aa/k.....por///222//////////
3/034013. URL (22.
crossref (
J H Rick Chang, Chun-LLang Li, Barnab Barnahás, Oczos, B V Vijaya Kumar, and Aswin Sankaranarayanan. One Network to Solve Them All-Solving Linear Inverse Problems using Deep Projection Models. Technical report. URL
pdf
Masoumeh Dashti and Andrew M Stuart. The bayesian approach to inverse problems. Handbook of Uncertainty Quantification, pp. 18, 2016.
Morris H DeGroot et al. Uncertainty, information, and sequential experiments. The Annals of Mathematical Statistics, 33(2):4044419 1962.
Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, and Rif A. Saurous. TensorFlow Distributions. nov 2017. TRLLttp:://aee
org/ abs/1711. 10604
Ludwig Fahrmeir and Stefan Lang. Bayesian inference for generalized additive mixed models based on Markov random field priors. Journal of the Royal Statistical Society: Series C (Applied Statistics), 50(2):201-220 jan 2001. ISSN 0035-9254. doi: 10.1111/1467-9876.00299 URL hittps://onll ine ibrary. wi wiley. com/ do // //a/////1000 / 1111:/...1111111--111111
00229
Yarin Gal. Uncertainty in deep learning. PhD thesis, PhD thesis, University of Cambridge, 2016.
10

---

Under review as a conference paper at ICLR 2020
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059, 2016.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Wences Gouveia and John A Scales. Resolution of seismic waveform inversion: Bayes ver- Occam. Inverse Problems, 13(2):323 349, apr 1997. ISSN 0266-5611. doi: 10.1088/
0266-5611/13///0009 URL http::/c.........ee.eeaaeaa//aa.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. In Advances in neural information processing systems, pp- 5767-5777, 2017.
Cusuh Ham, Amit Raj, Vincent Cartillier, and Irfan Essa. Variational Image Inpainting. Technical report.
Cong Han and Bradley Carlin. Markov chain monte carlo methods for computing bayes factors: A comparative review. Journal of the American Statistical Association, 96(455):1122 122-1122-11111 1132, 2001.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classification and preference learning. arXiv preprint arXiv: / / 2011.
Shi Hu, Daniel Worrall, Stefan Knegt, Bas Veeling, Henkjan Huisman, and Max Welling. Supervised Uncertainty Quantification for Segmentation with Multiple Annotations. jul 2019. URL. httt:t
//arxiv.or//aali /0tp/////90...0 00.0000000 01..1110 01999
Charles Jackson, Mrinal K. Sen, Paul L. Stoffa, Charles Jackson, Mrinal K. Sen, and Paul L. Stoffa. An Efficient Stochastic Bayesian Approach to Optimal Parameter and Uncertainty Estimation for Climate Model Predictions. Journal of Climate, 17(14): doi: 10.175//500-0422000001177773
2828-2841, jul 2004. ISSN 0894-8755. URL B2A.A///......T..00...EAAAAATTTE
ametsoc wttp:///////.////.11//100/ 1520- (S) (S))22004 (900000446220111777 3C2828 3AAESBAT ( 3.2.0.22....0
20..
Maya Kabkab, Pouya Samangouei, and Rama Chellappa. Task-aware compressed sensing with
generative adversarial networks. Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Jari Kaipio and Erkki Somersalo. Statistical and computational inverse problems, volume 160. Springer Science & Business Media, 2006.
Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? mar 2017. URL htttp://axxv..oggg////// wrggabs/////./ 149777
Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. Bayesian SegNet: Model Uncertainty in Deep Convolutional Sncoder-Decoder Architectures for Scene Understanding. British Machine Vision Association and Society for Pattern Recognition, may 2019. doi: 10.5244/c.31.57.
Diederik P Kingma and Max Welling. 1rX1v:1312.614, 2013.
Auto-encoding variational bayes.
arXiv preprint
Simon A. A. Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R. Ledsam, Klaus H. Maier-Hein, S. M. Ali Eslami, Danilo Jimenez Rezende, and Olaf Ron- neberger. A Probabilistic U-Net for Segmentation of Ambiguous Images. jun 2018. URL http::/arxix org/ abs 05034
V. Kolehmainen, A. Vanne, Siltanen, Jarvenpaa, J.P. Kaipio, M. Lassas, and M. Kalke. Par- allelized Bayesian inversion for ptree-dimensione dental X-ray imaging. IEEE Transactions Medical Imaging, 25(2):218-22) feb 2006. ISSN 0278-0062, doi: 10.10090MM....05822666
URL htkplhhtt:///eeeeee ore. leee. www.ee..rg/ddeee /

---

Under review as a conference paper at ICLR 2020
Orest Kupyn,
Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Ji Matas.
DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks, 2018.
URL http://openacee
thecvf. crrecvt.co//coneenttt content( Jovprí 1w011//tttttt Kupyn Debl (_ (_ Mot ( ppappr pappr..tmel
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunning- ham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Ze-
han Wang. and Wenzhe Shi Twitter.
Photo-Realistie Single Image Super-
Resolution Using a Generative Adversarial Network.
Technical report. URL
hwwpeeenaaccss...ccceeeeeo htttp://openaccess/ twwccf..ccccccc com/ content ( cvpr 12011//ap1rrs.///// Image CVPR | pappr..
pdf
Chad Lieberman, Karen Willcox, and Omar Ghattas. Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems. SIAM Journal on Scientific Computing, 32(5):2523- 2542, jan 2010. ISSN 1064-8275. doi: 10.1137/090775622. URL nttp://epubs. siam.
/rq//ol//0. 111.0/00.11117000777007
Sebastian Lunz, Ozan Öktem, and Carola-Bibiane Schönlich. Adversarial regularizers in inverse problems. In Advances Neural Information Processing Systems, pp. 8507-8516, 2018.
David JC MacKay, A practical bayesian framework for backpropagation networks. Neural compu- tation, 4(3):488-444 472, 1992.
A. Malinverno. Parsimonious Bayesian Markov chain Monte Carlo inversion in nonlinear geo- physical problem. Geophysical Journal International, 151(3):675 688, dec 2002. ISSN 0956- 540X. doi: 10. 166.66)61366-2222222200220000000
URL /accpp:://ccaemiic.. oup. com/
git/article- 317 200.2200222222222 2002. 01847.x
Youssef M. Marzouk and Habib N. Najm. Dimensionality reduction and polynomial chaos ac- celeration of Bayesian inference in inverse problems. Journal of Computational Physics, 228 19922, apr 2009. ISSN 0021-9991. doi: 10.101661.CPP2200..III 11.024. URL https:
//www www.eceiiiiiccc.ccooooooooc rect. com wwwpom/scincce/eeee http://ccc.c.ceeeeeaataata//////
Adam McCarthy, Blanca Rodriguez, and Ana Minchole. Variational Inference over Non- differentiable Cardiac Simulators using Bayesian Optimization. dec 2017. URL http://
arxiv. ortp
halgg///b//// 11t12/ 03353
H. N. Najm, B. Debusschere, Y. M. Marzouk, S. Widmer, and O.
P. Le Mae@ure. Uncertainty quantification in chemical systems. International Journal for Numerical Methods in Engineering, 80(6a7):789 814, nov 2009. ISSN 00295981. doi: 10. 10.10002/mm.255 URL http://dol.
wi com/c110000 100 002/ nme. 2551
Matthew D Parno and Youssef M Marzouk. Transport map accelerated markov chain monte carlo. SIAM/ASA Journal on Uncertainty Quantification, 6(2):645-622, 2018
Dhruv Patel, Raghav Tibrewala, Adriana Vega, Li Dong, Nicholas Hugenberg, and Assad A Oberai.
Cireumventing the solution of inverse problems in mechanics through deep learning: Application to elasticity imaging. Computer Methods in Applied Mechanics and Engineering, 353:448 466, 2019.
Arthur Pesah, Antoine Wehenkel, and Gilles Louppe. Recurrent machines for likelihood-frre free infer- ence. nov 2018. URLhttp:: /aattp://rrxvv org/ btlgg/bb//// tabs/1811.. 12932
Adriano Polpo, Julio Stern, Francisco Louzada, Rafael Izbicki, and Hellinton Takada (eds.). Bayesian Inference and Maximum Entropy Methods in Science and Engineering, volume 239 of Springer Proceedings in Mathematics & Statistics. Springer International Publishing, Cham, 2018 ISBN 978-3-319-91142-7.
doi: 00.1007/978-333199911433 URL http:://ink:
spri springer. www.eer.com//e/ 1007/9700007 -- 91143-4
Viraj Shah and Chinmay Hegde. Solving linear inverse problems using gan priors: An algorithm with provable guarantees. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 4609-4613. IEEF, 2018.
12

---

Under review as a conference paper at ICLR 2020
S Siltanen, V Kolehmainen, rvenp, J P Kaipio, P Koistinen, M Lassas, Pirttil, and E Som- ersalo. Statistical inversion for medical x-ray tomography with few radiographs: I. General theory. Physics in Medicine and Biology, 48(10):1437-1463, may 2003. ISSN 0031-9155.
doi: 10. 00.1088/003119955548800044444 URL /tthhtpp///taasss.oo.rr///////
1= 1==10//=3147heeycrrossrr ott.://ccss.......2222oooooooeen
M Stuart. Inverse problems: Bayesian perspective. Acta Numerica, 19:451-559,
2010. doi: 10.1017/00002499910000000 URL http:/ http://joouaaas..amm.aaaaa
ract S..SS006229922 (5009629990000005
Ryutaro Tanno, Daniel Worrall, Enrico Kaden, Aurobrata Ghosh, Francesco Grussu, Alberto Bizzi, Stamatios N. Sotiropoulos, Antonio Criminisi, and Daniel C. Alexander. Uncertainty Quan- tification in Deep Learning for Safer Neuroimage Enhancement. jul 2019. URL http:
//arxiv.org/abst 907.
http://aab/999..11111 13418
Albert Tarantola. Inverse problem theory and methods for model parameter estimation, volume 89.
siam, 2005.
M Vauhkonen, J P Kaipio, E Somersalo, and P A Karjalainen. Electrical impedance tomogra- phy with basis constraints. Inverse Problems, 13(2):523-530, apr 1997. ISSN 0266-5611, doi: 10. 1008800066-51111132222000 ott.aaacce..caeeeeeaaaeaeaaaeeee 5611:/0026661111111222
URL
a= C=0207key=roossre..
Jingbo Wang and Nicholas Zabaras. Hierarchical bayesian models for inverse problems heat
conduction. Inverse Problems, 21(1):183-006, dec 2004. doi: 10. 10.1088/0266-5611/211/0
Yan Wu, Mihaela Rosca, and Timothy Lillicrap. /5tt://9550673.3 2019.
Deep compressed sensing. arXiv preprint
Raymond A. Ych, Chen Chen, Teck-Yiae Lim, Alexander G. Schwing, Mark Hasegawa-Johnson and Minh N. Do. Semantic image inpainting with deep generative models. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6882-6890, 2016.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Computer Vision (ICCV), 2017 IEEE Interna- tional Conference on, 2017.
13

---

Under review as a conference paper at ICLR 2020
A WEAK CONVERGENCE OF THE POSTERIOR DENSITY
In this section we prove the weak convergence of the posterior density obtained by using GAN as prior to the posterior density the number of weights in the generator and discriminator
increased.
Let the generator of the GAN be given by g(z; 0), where RM is the latent vector, and 0 RR is the vector of weights. The vector x is selected from the distribution pz(z). Note that IR 7 R.
Let the discriminator of the GAN be given by d(æ; Φ), where E R and ø E R be the vector of weights. Note that d: RRV + [R, 1).
The GAN is trained using set of samples of æ. drawn from p (a:). For the loss function, consider
L(O, Φ) - E 1e(1 d(æ; +1)
E [)(d(g(2;)); 4)]]-
(10)
x-pp
zmpp
Here p is a monotone real-valued function which defines the GAN family being analyzed. For example, for the Wasserstein GAN, p(E) f.
The optimal values of the weights are given by
0 Ø argmax(argmin 6))).
(1I)
A.1 STATIONARITY CONDITIONS
The necessary conditions for these optimal values
0
(12)
0Φ
OL(O', x¹)
0.
(13)
00
Using the definition of the loss function
10 in
12
we have
Od dΦ
Od
E [p'(1 d(1;;(xx;pp )b))
(x;
E
[dd((((;;0)))
as (g(2; od)]
(14)
z~pz
Similarly, using 10 in 13
we have
0d Ox
0g 00
E (d)g((;;0);;))))
(g( 0");²))
(z; 0*) 0.
(15)
z~pz
A.2 WASSERSTHIN GAN
For the Wasserstein GAN, p(E) = { and p'(E) = 1. As a result
(14) and (15
reduce to,
Od
E od z~pz
(x; Ø²)]
[g(=:0))
6')]
(16)
x prtor
dΦ
ad
Ög
E
[g( z 0 Φ³)
0*)
0.
(17)
dx
30
Let W. (x) =
Ad
( 6") 1,...
No. Then 16 implies
Bon
E
[wa (x))
[ (w 0")) -, No.
(g((;)))))
(18)
zーPz
As No. + 00, this weak statement of the equivalence of p and px where the latter is defined in (3) In particular this says that the push forward of the measure in the latent space under
14

---

Under review as a conference paper at ICLR 2020
the function g(z) weakly converges to the measure associated with distribution of æ. We note with increasing number of weights in the discriminator, the relation above is required to hold for an increasing number of test functions, W. In addition, we have implicitly assumed that the generator is rich enough, that is it has enough weights/layers, such that this relation can actually be satisfied. To make this clear consider the extreme case of generator with single weight; in this case there is no way that (18) will be satisfied for a large number Ng. Thus in order for this relation to hold for a large Ng. must also provide the generator with large Ng.
Now consider a sufficiently smooth function m(x) which defines the point estimate we wish to compute. Expand it using Wa (2) as basis such that
No.
m(x)
Σ as aawalo) c(x);
(19)
and the coefficients are selected to minimize
E= [|<])\>)
(20)
As we increase No the dimension of the basis used to represent increases, and therefore č tends to zero.
Given this,
N aa ( + +x()]],
[m(x)]
from 19
x
prior
N
Oa
E
[wa(x))
E
[e(2r)
wapp
xmupl prios
N₂
E
[Wo (g(2;
+
E
[e(x)],
from 18)
znpp
mmpp ocior
E
Og W/ (2;=000))
[<(x))
g-p2
pxppx
E (m( 0*)) <(g(z;
E
[e(x)],
from 19
æ<pz
mmppi pripor
=
E [m(g(z; 0*)) - E [<(g(2; 0* + E
[e(ar)]
(21)
z~pz
amppl Priar
This yields,
E [m())]
E [m((g(z;0)))
<
E (e(g(2;0")) +
E
[<(x)]
2mp2
[<(g(2; 0*])
[<(x)]
z-p2
Urior
E
[]]{(g(z; 0 [1
+
E
[x))
zupp
m ma
E
[
E
from 20
p-
x--px
2E.
(22)
Note that in deriving this estimate, in the third line we have assumed that g(2; 0*) E Ωz That is, the generator does land outside the domain of æ. The statement above bounds the error in computing a population parameter for the prior using GAN.
A.3 CONVERGENCE TO THE POSTERIOR DENSITY
In order to turn this estimate into an estimate for the error in approximating population parameter for the posterior density, we simply choose m mxx) 1(m),, (g-fl=)) With this choice,
E [m(x))
= E
[(x)],
from
(23)
Tt Priar
15

---

Under review as a conference paper at ICLR 2020
and
E (m(
= E [(1(2;0")),
from 6
(24)
s<p2
gnp5 post
Using these in (2) we have the desired result,
E
[(xx]]
E [z];
22.
(25)
This result demonstrates that the difference between the true population parameter for the posterior and its approximation obtained after using GAN prior (the method proposed in the paper) reduces as the number of weights in the generator and the discriminator are increased. In the limit of infinite weights, é - 0, and the equation above is statement of weak convergence of the posterior obtained by using the GAN and the true posterior density.
B EXPRESSION FOR THE MAXIMUM A-POSTERIORI ESTIMATE
The techniques described in Section 2.1 focus on sampling from the posterior distribution and com- puting approximations to population parameters. These techniques can applied in conjunction with any distribution used to model noise and the latent space vector; that is, any choice of P, (like- lihood) and pz (prior). In this section we consider the special case when Gaussian models are used for noise and the latent vector. In this case, we can derive simple optimization algorithm to de- (zly) This point is denoted by map in the latent vector space and represents the most likely value of the latent vector in the posterior distribution. It is likely that the operation of the generator on 2 map, that is g2= will yield value that is close to map,, and may considered as a likely solution to the inference problem.
termine the maximum a-posteriori estimate (MAP) for (
We consider the case when the components of the latent vector are iid with normal distribution with zero mean and unit variance. This often the case many typical applications of GANs. Further, assume that the components of noise vector defined by normal distribution with zero mean and a covariance matrix E. Using these assumptions in we have
E7(x)
2200 (zly) exp
1 ( (0=()(((=(((()))))
f(g()))) +
1z1²)
(26)
The MAP estimate for this distribution is obtained by minimizing the negative of the argument of the exponential. That is
map arg m())
(27)
This minimization problem may be solved using any gradient-based optimization algorithm. The input to this algorithm is the gradient of the functional r with respect to 2, which is given by
dr
H" (f(g(z)) -9) +
(28)
Oz
where the matrix H is defined as
0f(g(z)) Oz
0 0g
H
(29)
Ox 0z
dg is the derivative Here 8f the derivative of the forward map f with respect its input æ, and oh of the generator output with respect to the latent vector. In evaluating the gradient above we need to evaluate the operation of the matrices of 2g 0z on vector, and not the matrices themselves. The operation of dg on vector can be determined using back-propagation algorithm with the GAN; u5
Pur and
while the operation of af can be determined by making use of the adjoint of the linearization of the Zu
forward operator.
Once z map is determined, one may evaluate map) by using the GAN generator. This represents the value of the field wish to infer at the most likely value value of latent vector. Note that this is not the same the MAP estimate of post (aryy).
16

---

Under review as a conference paper at ICLR 2020
C ADDITIONAL RESULTS
In this section we provide additional results for both MNIST and CelebA dataset for different tasks discussed in the main paper.
C.I MNIST
First we provide additional examples in Figure 7 for variance-based adaptive measurement window selection procedure described in Section 3.1
Figure shows additional results for the inpainting + denoising task, where MNIST digits are cluded with masks of different sizes at different locations. Note that the variance is high where occlusion mask is located indicating lower confidence in reconstructed image in that location.
C.2 CELEBA
For the CelebA dataset, we trained WGAN-GP model using more than 200k celebrity facial images and perform inference using remaining RGB image and were normalized between [-1, 1].
images. The input images were cropped 64 64
Once the GAN was trained, the HMC algorithm was used for posterior sampling and inference on a complimentary set of images (not used for training). In Figure 9 we show some additional results for variance-basee based adaptive measurement window selection procedure for CelebA dataset.
Next, in figure 10 we show some additional results for image recovery task for CelebA dataset. Once again we that the MAP estimate and the mean is close to the true image. On the other hand, the closest image from the training set (in La sense) is not as accurate. This points to the utility of using the GAN as an interpolant in the latent vector space.
D ARCHITECTURE AND TRAINING DETAILS
We use the WGAN-GP model for learning prior density. The tuned value hyper-parameters shown in Table [ We use the same generator and discriminator architecture for the MNIST and the physics-based inference problem; whereas for the CelebA dataset we use a slightly different architecture to accommodate different input image size. The layout of both these architecture is shown in Figure II Some notes regarding nomenclature used in Figure 11
Table 1:
for WGAN-GP model
Parameters
MNIST
CelebA
inverse heat conduction
epochs
1000
500
200
learning rate
0.0002
0.0001
0.0002
optimizer momentum params (31, (2) batch size
Adam
Adam
Adam
0.5, 0.999
0.5, 0.999
0.5, 0.999
64
64
Teriitic/ Mgen
5
5
Conv (HxWxC s=n) indicates convolutional layer with filer size of HxW and number of filters=C with stride=n.
FC (x.y) indicates fully connected layer with x neurons in input layer and neurons in output layer.
BN = Batch norm, LN Layer norm.
TrConv Transposed Convolution.
LReLU Leaky ReLU with a=0.2.
17

---

Under review as a conference paper at ICLR 2020
/
3
7
1
9
7
/
(a) Digit 0
(b) Digit
2
2
2
2
2
3
3
3
3
3
1
6
7
2
9
3
3
3
3
6
2
2
9
1
3
3
3
2
9
6
3
3
(c) Digit 2
(d) Digit
4
4
4
4
4
6
6
6
6
6
F
3
4
4 4
C 4
4
9
(e) Digit 4
(I) Digit
7
7
7
7
7
9
9
9
9
9
H
7
MAN
9
7
7
7
7
3
7
4
4
9
9
7
7
7
7
9
7
9
9
9
0
0
(g) Digit 7
(h) Digit 9
Figure 7: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from the limited view of a noisy image (2nd row) using the proposed method. The window to be revealed at a given iteration (shown red box) is selected using variance-driven strategy. Top row indicates ground truth. For all digits og = 1.
18

---

Under review as a conference paper at ICLR 2020
2
3
4
6
8
9
2
3
4
6
7
9
2
3
4
6
8
9
2
3
4
Figure 8: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from noisy image (2nd row) using the proposed method. Top row shows ground truth. For all the examples o 1.
19

---

Under review as a conference paper at ICLR 2020
(a)
(b)
(c)
(d)
(c)
(f
Figure 9: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from the limited view of noisy image (2nd row) using the proposed adaptive method. The window to be revealed at given iteration (shown in red box) is selected using a variance-driven strategy. Top row indicates ground truth. For all images o,- = 1.
20

---

Under review as a conference paper at ICLR 2020
Xyuse
meees
Xmap
Xmean
Xvar
(a)
Xiruo
Ymess
Xmap
Xinaa
Xyar
Xeloxees
(b)
Figure 10: Estimate of the MAP (3rd row), mean (4th row) and variance (5th row) from a noisy image (2nd row) using the proposed method. Top row shows the ground truth. The last row shows the closest example in training set (by the La measure). For all images 1.
21

---

Under review as a conference paper at ICLR 2020
Generator
Discriminator
g300
[64x8'x3]
FC 1200, 51921
BN-RoLU
Conv (8x5a64 1- UUeeUU
Generator
Discriminator
Reshape [Batch_stae
2300
Input [28x28xt]
Com: E3xbx128 5=2)
LN LRRLU
[100, 1024) BN+RRLU
TeConv (5x5x256 s=2)
Conv (5x5x1 s=22
Conv (Sx5x128 s=2) LN a LR
1102, 6272) BN+RRLU
IrConv (5x5x256
Conw (habadi s==2
Conv (Se52256 sm)) LRt
Reshape
TrConv (5x5x128 DN ReiU
[Batch 128]
FC (3136, 1024) LN
Conv (Sx555 92 s=2)
+ LitLU
(x2) Conv (bc3644 |
(5x5x54 BN ReLU
==1) ReLU
(1024,1)
FC (2048,1)
BI (x2) + Conv (br3d tanh
TrCorv [5xSx3 5=2) tanth
Output [1]
Output [1]
Output
Output (64x6Ax3)
[28x28x1]
(a) Architecture for MNIST and synthetic dataset
(b) Architecture for CelebA dataset
(used in physics-based inference problem)
Figure 11: Generator and discriminator architectures,
22


Reviews:
[{"review_id": "rJghfrNnKS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies the Bayesian inferences with the generative adversarial network (GAN). In the first half of the paper, the general framework of the Bayes estimation is introduced. Then, The authors proposed how to incorporate GAN to the Bayesian inference. Some computational methods for calculating the mean of the statistic under the posterior distribution are described. Then, numerical experiments using MNIST and Celeb-A datasets are presented. \n\nThough the Bayesian inference using GAN is a natural idea, learning algorithms proposed in this paper are simple and are not intensively developed. In numerical experiments, there is no comparison with major competitors besides random sampling in the active learning setup. Hence, the effectiveness and advantage of the proposed methods are not clear.\n- In active learning, the proposed method should be compared with other methods such as Bayesian DNN using dropout, etc. \n- How does the estimation accuracy of GAN relate to the estimation accuracy of the proposed method? Showing a quantitative description would be nice.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "SJlld01kcS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use a trained GAN model as the prior distribution for Bayesian inference to quantify the uncertainty. As for me, the best application of this paper is to restore a corrupted image, which shares a lot of common properties in image restoration, denoising and image reconstruction. I do like the extension of applying the idea in physics problems. And the results demonstrate at some extent, the proposed method could evaluate some uncertainty. \n\nThe idea is pretty simple and the paper is easy to read.  Nonetheless, there are some issues:\n\nA big issue of this paper is the deviation of purpose and method. As the paper claims to quantify the uncertainty, the paper is supposed to give specific quantitative metric or values to probe the uncertainty. However, the paper demonstrates to us only the ability, not exactly “quantification”. I’d like to see a specific metric of uncertainty that could only be calculated through the proposed method. \n\nThere are some grammar issues in the paper. For example. “…we the MAP…” in the 7th page.\n\nGiven my major issue seems to be quite problematic, I currently would weakly reject this paper. But I don’t have a full picture over this area, I’ll read the rebuttal and see if I could raise the score.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BygwEyIPKB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of the paper:\n  \n        The paper proposes a Bayesian approach to make inference about latent variables such as un-corrupted images. The prior distribution plays a key role in this task. The authors use a GAN to estimate this prior distribution. Then, standard Bayesian techniques such a Hamilton Monte Carlo are used to make inference about the latent variables.\n\nDetailed comments:\n\nEq. (8) is expected to give very bad results. The reason is that it is very unlikely to sample from the prior configurations for z that are compatible with y.\n\nThe paper does not address learning any model parameters. e.g. the amount of noise.\n\nA more principled approach would be to estimate the prior parameters using maximum likelihood estimation. That has already been done in the case of the\nvariational autoencoder.\n\nThe variational autoencoder is an already known method that can be used to solve the problem formulated by the authors. It also automatically proposes\nan inference network that can be used for recognition. If the likelihood is Gaussian and p(x|z) is also Gaussian, one can directly marginalize x and work\nwith p(y|z) and p(z). The authors should at leas discuss the potential use of this method alongside with the BIGAN model which also provides a recognition model.\n\nIt is not clear how the HMC parameters are fixed.\n\nThe experiments do not have error bars (Figure 4.) This questions the significance of the results.\n\nMy overall impression is that there is little novelty in the proposed approach. Namely, using a GAN to learn the prior distribution, and then very well known\ntechniques to infer the original input image.\n\nI have missed some references to related work on inverse problems. An example is:\n\nhttps://arxiv.org/pdf/1712.03353.pdf\n\n\nIs the original figure contained in the training set used to infer the GAN. If so that can lead to biased results.\n\nI have missed a simple baseline in which one simply finds the training image that is closest to the corrupted observed or partially observed image.\n\nMy overall impression is that there is not much novelty in the paper as it is simply a combination of well known techniques. E.g. GANs and Bayesian inference with Monte Carlo methods.\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper suggests a Bayesian approach to make inference about latent variables for image inference tasks. While the idea in the paper seems elegant and simple, reviewers pointed out a few concerns, including lack of comparisons, missing references, and requested for more extensive validations. While a few comments might have been misunderstandings (eg lack of quantification - seems to be resolved by author’s comments), other comments are not (eg equation (8) needs further justification even if the final results don’t use it). We encourage authors to carefully review comments and edit the manuscript (perhaps some appendix items should be in the main to reduce confusion) for resubmitting to future conferences. 
Decision: Reject