Q UANTIFYING U NCERTAINTY WITH GAN- BASED P RIORS
Anonymous authors
Paper under double-blind review
A BSTRACT
Bayesian inference is used extensively to quantify the uncertainty in an inferred ﬁeld given the measurement of a related ﬁeld when the two are linked by a mathematical model.
Despite its many applications, Bayesian inference faces challenges when inferring ﬁelds that have discrete representations of large dimension, and/or have prior distributions that are difﬁcult to characterize mathematically.
In this work we demonstrate how the approximate distribution learned by a generative adversarial network (GAN) may be used as a prior in a Bayesian update to address both these challenges.
We demonstrate the efﬁcacy of this approach by inferring and quantifying uncertainty in inference problems arising in computer vision and physics-based applications.
In both instances we highlight the role of computing uncertainty in providing a measure of conﬁdence in the solution, and in designing successive measurements to improve this conﬁdence.
I NTRODUCTION
Quantifying uncertainty in an inference problem amounts to making a prediction and quantifying the conﬁdence in that prediction.
In the context of an image recovery problem, this may be understood as follows.
A typical computer vision algorithm uses a noisy version of an image and prior knowledge to produce the recovered image which can be interpreted as the “best guess” of the original image.
Quantifying uncertainty in this context involves generating an estimate of the level of conﬁdence in the best guess, in addition to the guess itself.
Bayesian inference provides a principled approach for quantifying uncertainty.
As shown in the following section, it treats the inferred vector as a multivariate stochastic vector and leads to an expression for its distribution.
This expression can be used to estimate the most likely solution (the maximum a-posteriori estimate, or the MAP), the mean, the variance, or any other population parameter of interest.
Thus providing a recipe for thoroughly quantifying the uncertainty in an inference problem.
For the image recovery problems considered in this paper, Bayesian inference not only provides the best guess of the true image, but also a means to estimate measures of uncertainty such as the pixel-wise variance.
The knowledge of uncertainty in a prediction can directly inﬂuence the downstream action that depends on the inference.
Consider an image recovery problem where two distinct inputs lead to similar recovered images: those of a trafﬁc sign with a high speed limit.
However, for the ﬁrst input the predicted variance is small, while for the second input it is large.
Further, the set of likely images in the second set also includes images of a Stop Sign.
Then the appropriate action for the two inputs, determined after solving the inference problem and quantifying uncertainty, is very different.
For the ﬁrst input, the appropriate action is one of continued motion, whereas for the second input it is to slow down.
Similar examples can be drawn from other areas as well, like medical imaging, high frequency trading, and autonomous systems where critical decisions are made based on the output of AI system (Gal (2016); Begoli et al.
The knowledge of uncertainty can also be useful in determining the optimal location of a sensor.
Consider an image recovery problem, where the goal is to infer the signal, and associated uncertainty, using limited amount of measurement data.
In this problem a user can leverage information about the spatial distribution of uncertainty to choose the location with maximum uncertainty as next measurement location.
This task falls within the ﬁelds of active learning and/or design of ex- periments (DeGroot et al.
Houlsby et al.
and is particularly useful in applications like satellite imaging, where each measurement requires signiﬁcant time and/or resources.
In Figure 1, we demonstrate how the proposed GAN-based Bayesian inference algorithm is useful in both scenarios explained above: (i) in quantifying uncertainty - “ pixel-wise variance ” - our quantitative measure of uncertainty in the inferred ﬁeld, and (ii) in determining the optimal sensor placement location in an iterative fashion.
We return to these applications in greater detail in Section 4.
1. B AYESIAN INFERENCE
Bayesian inference is a well-established technique for quantifying uncertainties in inference problems (Kaipio & Somersalo (2006); Dashti & Stuart (2016); Polpo et al.
It has found applications in diverse ﬁelds such as geophysics (Gouveia & Scales (1997);
Malinverno (2002)), climate modeling (Jackson et al.
, chemical kinetics (Najm et al.
(2009)), heat conduction (Wang &
Zabaras (2004)), and the detection and diagnosis of disease (Siltanen et al. (2003); Kolehmainen et al.
The two critical ingredients of a Bayesian inference problem are - an informative prior representing the prior belief about the parameters to be inferred and an efﬁcient method for sampling from the posterior distribution.
In this manuscript we describe how deep generative adversarial networks (GANs) can be effectively used in these roles.
Consider the setting where we wish to infer a vector of parameters x ∈ R N from the measurement of a related vector y ∈ R P , where the two are related through a forward model y =
f ( x ) .
A noisy measurement of y is denoted by ˆ y = f
( x ) + η , where the vector η ∈ R P represents noise.
While the forward map f is typically well-posed, its inverse is not, and hence to infer x from the measurement
ˆ y requires techniques that account for this ill-posedness.
Classical techniques based on regularization tackle this ill-posedness by using additional information about the sought solution ﬁeld explicitly or implicitly (Tarantola (2005)).
Bayesian inference offers a different approach to this problem by modeling the unknown solution as well as the measurements as random variables.
This framework addresses the ill-posedness of the inverse problem, and allows for the characterization of the uncertainty in the inferred solution.
The notion of a prior distribution plays a key role in Bayesian inference.
Through multiple observations of the ﬁeld x , denoted by the set S = { x (1) , · · · , x ( S ) } , we have some prior knowledge of x that can be utilized when inferring x from ˆ y .
This is used to build, or intuit, a prior distribution for x , denoted by p prior X ( x ) .
Some typical examples include Gaussian process prior with speciﬁed co-variance kernels, Gaussian Markov random ﬁelds (Fahrmeir & Lang (2001)), Gaussian priors deﬁned through differential operators (Stuart (2010)), and hierarchical Gaussian priors (Marzouk &
Najm (2009); Calvetti & Somersalo (2008)).
These priors promote smoothness and/or structure in the inferred solution and importantly, can be expressed explicitly in an analytical form.
Another key component of Bayesian inference is a distribution that represents the likelihood of y given an instance of x , denoted by p l ( y | x ) .
This is often determined by the distribution of the error in the model, denoted by p η , which captures both model and measurement errors.
Given this, and an additive model for noise, the posterior distribution of x , determined using Bayes’ theorem after accounting for the observation ˆ y is given by,
[Equation]
Here, Z is the prior-predictive distribution of y and ensures that the posterior integrates to one.
The posterior distribution characterizes the uncertainty in x ; however for vectors of large dimension characterizing this distribution explicitly is a challenging task.
Consequently the expression above is used to perform tasks that are more manageable.
These include determining estimates such as the maximum a-posteriori estimate (MAP), expanding the posterior distribution in terms of other distributions that are simpler to work with (Bui-Thanh et al.
(2012)), or using techniques like Markov
Chain Monte-Carlo (MCMC) to generate samples that are “close” to the samples generated by the true posterior distribution (Han & Carlin (2001); Parno & Marzouk (2018)).
Despite its numerous applications in solving inverse problems, Bayesian inference faces signiﬁcant challenges.
These include deﬁning a reliable and informative prior distribution for x when the set S
= { x (1) , · · · , x ( S ) } is difﬁcult to characterize analytically, and efﬁciently sampling from the posterior distribution when the dimension of x is large; a typical situation in many practical science and engineering applications.
1. RELATED WORK
The main idea developed in this paper tackles the above mentioned challenges by training a generative adversarial network (GAN) using the sample set S , and then using the distribution learned by the GAN as the prior distribution in Bayesian inference.
The use of sample-based priors for solving an inverse problem has a rich history (Vauhkonen et al.
(1997); Calvetti & Somersalo (2005)).
As does the idea of reducing the dimension of the parameter space by mapping it to a lower-dimensional space (Marzouk & Najm (2009); Lieberman et al.
However, the use of learning-based deep generative models like GANs in these tasks is novel.
Recently, several authors have considered the use of learning-based methods for solving inverse problems arising in different domains.
These include the use of deep convolutional neural networks (CNNs), recurrent neural networks (RNNs) or variational autoencoders (VAEs) to solve physicsdriven inverse problems (Adler & ¨ Oktem (2017);
Patel et al.
; Pesah et al.
VAEs and
GANs have also been used to solve inverse problems in computer vision (Yeh et al.
; Ham et al.;
Chang et al.
; Kupyn et al.
; Ledig et al.;
Zhu et al.
There is also a growing body of work dedicated to using GANs to learn regularizers in solving inverse problems (Lunz et al.
(2018) and in compressed sensing (Bora et al.
(2017; 2018); Kabkab et al.
; Wu et al.
Shah & Hegde (2018)).
However, these approaches differ from ours in that they solve the inverse problem as an optimization problem and do not rely on Bayesian inference; as a result, they add regularization in an ad-hoc manner and do not attempt to quantify the uncertainty in the inferred ﬁeld.
More recently, the approach described in (Adler & ¨Oktem (2018)) utilizes GANs in a Bayesian setting; however the GAN is trained to approximate the posterior distribution (and not the prior, as in our case), and training is done in a supervised fashion.
That is, paired samples of the measurement ˆ y and the corresponding true solution x are required.
In (McCarthy et al.
, authors perform variational approximation of the posterior using VAEs with simulator-based decoder to solve physics-based inference problem.
We note that deep learning based Bayesian networks, where the network weights are stochastic parameters that are determined using Bayesian inference, are another means of quantifying uncertainty (MacKay (1992); Kingma & Welling (2013); Gal &
Ghahramani (2016)), and have recently been applied to semantic image-segmentation and superresolution (Kendall & Gal (2017); Kendall et al.
; Kohl et al.
; Hu et al.
Tanno et al.
However, these method also rely on supervised learning, whereas in contrast, our approach is unsupervised, and requires only samples of the true solution x to train the GAN prior.
1. O UR CONTRIBUTION
The main contribution of this paper can be summarized as follows:
A novel method for performing Bayesian inference involving complex priors and high dimensional posterior.
In this method we utilize the distribution learned by a GAN as a surrogate for the prior distribution and reformulate the inference problem in the low-dimensional latent space of the GAN.
Furthermore, we provide a theoretical analysis of the weak convergence of the posterior density learned by the proposed method to the true posterior density.
Novel unsupervised image denoising and inpainting algorithms with quantitative measures of uncertainty through pixel-wise variance.
Application of the proposed method to physics-based inference problems.
Demonstration of the utility of quantifying uncertainty to facilitate active learning.
P ROBLEM F ORMULATION
Let S denote the set of instances of vector x sampled from the true distribution, p true X ( x ) .
Further, let z ∼ p
Z ( z ) characterize the latent vector space and g ( z ) be the generator of a GAN trained using S .
Then according to Goodfellow et al.
(2014), with inﬁnite capacity and sufﬁcient data, the generator learns the true distribution.
That is,
[Equation]
The distribution p gen X ( x ) is deﬁned as
[Equation]
Here p Z is the multivariate distribution of the latent vector whose components are iid and typically conform to a Gaussian or a uniform distribution.
The equation above implies that the GAN generates samples of x by sampling z from p Z and then passing these through the generator.
Now consider a measurement ˆ y from which we would like to infer the posterior distribution of x .
For this we use (1) and set the prior distribution equal to the true distribution, that is p prior X
=
p true X .
Then from (2) this is the same as p prior X = p gen X .
Therefore,
[Equation]
Now for any l ( x ) , we have
[Equation] From [Equation]
[Equation]
where E is the expectation operator, and
[Equation]
The distribution p post Z is the analog of p post X in the latent vector space.
The measurement ˆ y updates the prior distribution for x to the posterior distribution.
Similarly, it updates the prior distribution for z , p Z , to the posterior distribution, p post Z , deﬁned above.
Equation (5) implies that sampling from the posterior distribution of x is equivalent to sampling from the posterior distribution for z and passing the sample through the generator g .
That is,
[Equation]
Since the dimension of z is typically smaller than that of x , this represents an efﬁcient approach to sampling from the posterior of x .
The left hand side of (5) is an expression for a population parameter of the posterior, deﬁned by l ( x ) ≡ E x ∼ p post
X [ l ( x )] .
The right hand sides of the last two lines of this equation describe how this parameter may be evaluated by sampling z (instead of x ) from either p Z or p post Z .
The equality in (5) holds for a GAN with an inﬁnite number of weights in the generator and the discriminator.
In Appendix A of this manuscript, we consider the case of a Wasserstein GAN with a ﬁnite number of weights and prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights is increased.
2. S AMPLING FROM THE POSTERIOR DISTRIBUTION
We consider a scenario where we wish to infer and characterize the uncertainty in the vector of parameters x from a noisy measurement of y , denoted by ˆ y , where f is a known map that connects x and y .
And we have several prior realizations of plausible x , contained in the set S .
For this problem we propose the following algorithm that accounts for the prior information in S and the “new” measurement ˆ y through a Bayesian update:
Train a GAN with a generator g ( z ) on S . 2.
Sample x from p post X ( x | y ) given in (7).
With sufﬁcient capacity in the GAN and with sufﬁcient training, the posterior obtained using this algorithm will converge to the true posterior (see eq.
(5) above and Appendix A).
Since GANs can be used to represent complex distributions efﬁciently, this algorithm provides a means of including complex priors that are solely deﬁned by samples within a Bayesian update.
Further, it provides an efﬁcient approach to sampling from the posterior since the dimension of z is typically much smaller ( 10 1 - 10 2 ) than that of x ( 10 4 - 10 7 ).
Finally, in contrast to other methods that attempt to quantify uncertainty in image recovery tasks, this algorithm falls within the class of unsupervised learning algorithms, and does not require paired data for training.
We now describe two approaches for estimating population parameters of the posterior using this algorithm.
Monte-Carlo (MC) approximation
The ﬁrst approach is based on a Monte-Carlo approximation of a population parameter of the posterior distribution.
This integral, which is deﬁned in the second line of (5), may be approximated as,
[Equation]
In the equation above, the numerator is obtained from a MC approximation of the integral in (5), and the denominator is obtained from a MC approximation of the scaling parameter Z .
Sampling within this approach is simple since in a typical GAN, the z i s belong to a simple distribution like a
Gaussian or a uniform distribution.
However, we anticipate that in many applications the likelihood will tend to concentrate the distribution of latent vector z to a small region within Ω z and thee sampling described above may be inefﬁcient.
Markov-Chain Monte-Carlo (MCMC) approximation
A more efﬁcient approach is to generate an MCMC approximation p mcmc Z ( z | y )
≈ p post Z ( z | y ) using the deﬁnition in (6), and thereafter sample z from this distribution.
Then from the third line of (5), any desired population parameter may be approximated as
[Equation]
We note that for all the numerical experiments in this paper we have used MCMC because of its better sample efﬁciency.
Summary We have described two methods for probing the posterior distribution when the prior is deﬁned by a GAN.
These include an MC (8) and an MCMC estimate (9) of a given population parameter and a MAP estimate that is applicable to additive Gaussian noise with a Gaussian prior for the latent vector (see Section B in the Appendix).
In the following section we apply the MCMC approach to inverse problems drawn from physics-based and computer vision applications.
E XPERIMENTS
We evaluate our method empirically on practical probabilistic ﬁeld inference tasks in the domain of computer vision and physical science.
In computer vision, we consider image denoising and inpainting tasks and evaluate our algorithm on MNIST and CelebA datasets.
Here we attempt to infer the true image and quantify the uncertainty in this inference.
We also demonstrate the utility of quantifying uncertainty by using it in an active learning setup.
In the physics-based application, we consider the problem of recovering the initial temperature distribution from a measurement at later time, and use this problem to demonstrate the statistics inferred using our method converge to their “true” values.
In all cases we use a Wasserstein GAN-GP (Gulrajani et al.
to learn the prior density (architecture described in the Appendix D).
We also ensure that the target images are not chosen from the set used to train the GAN.
We sample from the posterior using Hamiltonian Monte Carlo (Brooks et al.
and implement it using Tensorﬂow-probability (Dillon et al.
(2017)) library.
We use initial step size of 1.0 for HMC and adapt it following (Andrieu & Thoms (2008)) based on the target acceptance probability.
We use 64k samples with burn-in period of 0.5.
We select these parameters to ensure convergence of chains.
Using the HMC sampler we compute the MAP, which is our “best guess” of the true image, the pixel-wise mean, and the pixel-wise variance, which is our quantitative and spatially-varying estimate of uncertainty.
3. I MAGE RECOVERY USING THE MNIST & C ELEB A DATABASES
We ﬁrst consider the MNIST database of hand-written digits and use 55k images to train the GAN.
We use a latent vector of dimension 100 with Gaussian distribution.
For the image denoising task, we add Gaussian noise with zero mean and speciﬁed variance ( σ y ) to the test image and use it as measurement to recover the distribution of likely images using the MCMC approach.
For this problem the forward operator is the identity map, and the likelihood distribution is Gaussian.
In
Figure 2, we have plotted the noisy input image, the MAP estimate, and the pixel-wise mean and variance.
We observe that for low and medium noise levels ( σ y = 0 . 1 , 1 ), we are able to recover the original image with good accuracy, the pixel-wise variance is small overall, and is largest around the boundary of the recovered digit; this represents the variability in the different realizations of the recovered digit within the GAN prior.
For the highest noise level ( σ y = 10 ), however, the image recovered by the MAP is incorrect in 2/3 cases, and would be misleading if viewed by itself.
However, when viewed in conjunction with the estimated variance, which is large, it is clear that the conﬁdence in the inference is small and the inferred image ought not be trusted for downstream tasks.
The dependence of the average per-pixel variance in the recovered image on the variance of noise in the measured image is shown in Figure 4 (a) with 95% conﬁdence interval.
As expected, the per-pixel variance increases with increasing noise.
In the right-most panel of Figure 2, we show image inpainting + denoising results for an image of digit 3 with σ y = 1 .
Here the forward map is the indicator function set to zero on the occluded pixels.
We note that for the small and intermediate occluded regions, the MAP solution is close to the true solution.
However, when most of the image is occluded, the MAP is incorrect.
Once again, the variance image, which is small for the low and medium occluded regions, and large for the large occlusion, is a reliable indicator of the conﬁdence in the recovered MAP image.
Furthermore, the variance is peaked within the occluded region demonstrating lower conﬁdence in the image reconstructed in this region.
This is useful in applications like autonomous cars and medical imaging, where partial measurements are common and the spatial distribution of uncertainty can inform downstream tasks.
More image denoising and inpainting examples are provided in Appendix C.
In Figure 3, we demonstrate how uncertainty may be used in active learning/design of experiment, where the goal is to determine the optimal location for a measurement.
We begin with an input where the entire image is occluded and in every subsequent step, we allow for a small 7 × 7 pixel window to be revealed.
We select the window with the largest average pixel-wise variance.
As the iterations progress, the MAP estimate converges to the true digit, and the variance decreases.
In about 4 iterations we arrive at a very good guess for the digit.
The performance of this approach is quantiﬁed in Figure 4(b), where we have plotted reconstruction error versus the number windows for this strategy, and a strategy where the subsequent window is selected randomly.
The variancedriven strategy consistently performs better.
We note that we are not aware of any other methods for computing uncertainty in recovered images that have been applied to drive an active learning task in image inpainting.
While methods based on dropout (Kendall & Gal (2017)
Kendall et or variational inference (Kohl et al.
(2018)) could be extended to accomplish this, this has not been done thus far.
[Figure]
Results for the variance-based window selection strategy applied to the CelebA dataset are shown in
Figure 5.
We observe that the algorithm produces realistic images at each iteration; however, the initial variance is large indicating large uncertainty.
As more windows are sampled using the variance driven active learning strategy, the variance reduces and by the 7th iteration a good approximation of the true image is obtained, even though only a small, noisy portion is revealed.
Additional results along with implementation details for this dataset are discussed in Appendix C.2.
3. A PHYSICS - DRIVEN INFERENCE PROBLEM
We consider the problem of inverse heat conduction, where the goal is to infer the initial temperature distribution (at t = 0 ) in a domain given a noisy measurement of temperature at later time ( t = 1 ) and the thermal conductivity of the material.
The forward map is the solution of the time-dependent heat conduction problem with uniform conductivity, κ = 0 .
64 , and in a square domain of length
L = 2 π with Dirichlet boundary conditions.
This operator maps the initial temperature ﬁeld to the temperature ﬁeld at later time.
The discrete version of this operator is obtained by discretizing the time-dependent linear heat conduction equation using the central difference scheme in space and backward Euler scheme in time.
Much like a blurring kernel, the forward operator smooths the initial temperature distribution, and the extent of smoothing increases with κ × t .
We consider a family of initial temperatures where the background is zero, and the temperature on a rectangular sub-domain varies linearly from 2 units on the left edge to 4 units on the right edge.
This distribution is parameterized by four parameters, { ξ i } 4 i =1 , which are the coordinates of the lower left and upper right corners of the rectangular region.
The sample set S is created by sampling each parameter from a uniform distribution ( xi i ∼ U (0 . 25 , 0 . 75) × L ) and is used to train the GAN prior.
The posterior distribution is sampled using the HMC sampler.
In the top two rows of Figure 6, we have plotted the true initial condition, the noise-free temperature at t = 1 , and the noisy temperature measurement ( σ y = 1 ) used as input in the GAN-based prior approach.
The corresponding MAP, mean and pixel-wise variance estimated by the MCMC approximation are shown next.
We observe that the MAP is very close to the true initial temperature distribution and the variance is concentrated along the edges of the rectangle where the uncertainty is the largest.
In the following columns we have plotted the MAP estimate obtained assuming Land H 1 Gaussian priors, which are often used to solve these types of problems, and are clearly much less accurate.
For this problem the “true” posterior can be reduced to the 4-dimensional space of parameters, and sampled by generating initial conditions corresponding to the values of these parameters.
A simple
MC approximation can be performed to compute statistics - the mean and the pixel-wise variance for the true posterior (last two columns of Figure 6).
By comparing these with the mean and the pixelwise variance (columns 5 & 6) estimated by the GAN-based prior, we conclude that GAN-based posterior has converged to the true posterior.
In the bottom rows of Figure 6, we plot similar results for initial conditions and GAN-based priors generated from the MNIST database. In this case the measurement is made at t = 0 .
Since the “true” distribution for this set is not known the true mean and variance are not plotted.
C ONCLUSIONS
The ability to quantify the uncertainty in an inference problem is useful in developing conﬁdence in that inference, and in designing strategies to improve the conﬁdence.
In this paper we have described how this may be accomplished when solving a Bayesian inference problem by using GANs as priors.
Since GANs can learn complex distributions of a wide variety of ﬁelds from their samples, this approach can be applied to a range of problems in computer vision and physics-driven inference.
It derives its efﬁciency by mapping the posterior distribution to the latent space, whose dimension is often much smaller than that of the inferred ﬁeld.
We have applied this approach to image recovery tasks and demonstrated how the knowledge of uncertainty in the prediction can be used to assess conﬁdence in a prediction, and via active learning to design a strategy to improve it.
We have also applied this approach to a physics-based problem, where we have veriﬁed its accuracy and robustness.
In the results reported in this manuscript we assume knowledge of the forward map; however, we note that the proposed algorithm can easily be extended to a regime where the forward map is also unknown by utilizing likelihood-free inference methods like ABC or meta-learning approaches.
We leave this as an interesting direction for future research.
A W EAK CONVERGENCE OF THE POSTERIOR DENSITY
In this section we prove the weak convergence of the posterior density obtained by using a GAN as a prior to the true posterior density as the number of weights in the generator and discriminator are increased.
Let the generator of the GAN be given by g ( z ; θ ) , where z ∈ R M is the latent vector, and θ ∈ R N θ is the vector of weights.
The vector z is selected from the distribution p Z ( z ) .
Note that g : R M → R N .
Let the discriminator of the GAN be given by d ( x ; φ ) , where x ∈ R N , and φ ∈ R N φ be the vector of weights.
Note that d : R N → R (0 , 1) .
The GAN is trained using a set of samples of x , drawn from p prior X ( x ) .
For the loss function, consider
[Equation]
Here ρ is a monotone real-valued function which deﬁnes the GAN family being analyzed.
For example, for the Wasserstein GAN, ρ ( ξ ) = ξ .
The optimal values of the weights are given by
[Equation]
A. S TATIONARITY CONDITIONS
The necessary conditions for these optimal values are
[Equation] [Equation]
Using the deﬁnition of the loss function (10) in (12), we have
E
[Equation]
Similarly, using (10) in (13), we have
[Equation] [Equation]
A. W ASSERSTEIN GAN
For the Wasserstein GAN, ρ ( ξ ) = ξ and ρ (cid:48) ( ξ ) = 1 .
As a result (14) and (15) reduce to,
[Equation]
[Equation]
Let w a ( x ) ≡ ∂d ∂φ a ( x ; φ ∗ ) , a = 1 , · · · , N φ .
Then (16) implies
[Equation]
As N φ → ∞ , this is a weak statement of the equivalence of p prior X and p gen X , where the latter is deﬁned in (3).
In particular this says that the push forward of the measure in the latent space under the function g ( z ) weakly converges to the measure associated with distribution of x .
We note with increasing number of weights in the discriminator, the relation above is required to hold for an increasing number of test functions, w a .
In addition, we have implicitly assumed that the generator is rich enough, that is it has enough weights/layers, such that this relation can actually be satisﬁed.
To make this clear consider the extreme case of a generator with a single weight; in this case there is no way that (18) will be satisﬁed for a large number N φ .
Thus in order for this relation to hold for a large N φ , we must also provide the generator with a large N θ .
Now consider a sufﬁciently smooth function m ( x ) which deﬁnes the point estimate we wish to compute.
Expand it using w a ( x ) as basis such that
[Equation]
and the coefﬁcients are selected to minimize
[Equation]
As we increase N φ the dimension of the basis used to represent m increases, and therefore ¯ (cid:15) tends to zero.
Given this,
E
[Equation]
from
[Equation] [Equation]
[Equation]
This yields, |
E
[Equation] [Equation]
Note that in deriving this estimate, in the third line we have assumed that g ( z ; θ ∗ )
∈
Ω x .
That is, the generator does not land outside the domain of x .
The statement above bounds the error in computing a population parameter for the prior using a GAN.
A. C ONVERGENCE TO THE POSTERIOR DENSITY
In order to turn this estimate into an estimate for the error in approximating a population parameter for the posterior density, we simply choose m ( x )
=
l ( x ) p η (ˆ y − f ( x ))
Z .
With this choice,
E
[Equation]
E
[Equation]
from (1) (23) and
[Equation]
from
[Equation] [Equation]
Using these in (22) we have the desired result,
[Equation]
This result demonstrates that the difference between the true population parameter for the posterior and its approximation obtained after using a GAN as a prior (the method proposed in the paper) reduces as the number of weights in the generator and the discriminator are increased.
In the limit of inﬁnite weights, ¯ (cid:15) → 0 , and the equation above is a statement of weak convergence of the posterior obtained by using the GAN and the true posterior density.
B E XPRESSION FOR THE MAXIMUM A - POSTERIORI ESTIMATE
The techniques described in Section 2.1 focus on sampling from the posterior distribution and computing approximations to population parameters.
These techniques can be applied in conjunction with any distribution used to model noise and the latent space vector; that is, any choice of p η (likelihood) and p Z (prior).
In this section we consider the special case when Gaussian models are used for noise and the latent vector.
In this case, we can derive a simple optimization algorithm to determine the maximum a-posteriori estimate (MAP) for p post
Z ( z | y ) .
This point is denoted by z map in the latent vector space and represents the most likely value of the latent vector in the posterior distribution.
It is likely that the operation of the generator on z map , that is g ( z map ) , will yield a value that is close to x map , and may be considered as a likely solution to the inference problem.
We consider the case when the components of the latent vector are iid with a normal distribution with zero mean and unit variance.
This is often the case in many typical applications of GANs.
Further, we assume that the components of noise vector are deﬁned by a normal distribution with zero mean and a covariance matrix Σ .
Using these assumptions in (6), we have
[Equation] [Equation]
The MAP estimate for this distribution is obtained by minimizing the negative of the argument of the exponential.
That is
[Equation]
This minimization problem may be solved using any gradient-based optimization algorithm.
The input to this algorithm is the gradient of the functional r with respect to z , which is given by
[Equation]
where the matrix H is deﬁned as
[Equation]
Here ∂ f ∂ x is the derivative of the forward map f with respect to its input x , and ∂ g ∂ z is the derivative of the generator output with respect to the latent vector.
In evaluating the gradient above we need to evaluate the operation of the matrices ∂ f ∂ x and ∂ g ∂ z on a vector, and not the matrices themselves.
The operation of ∂ g ∂ z on a vector can be determined using a back-propagation algorithm with the GAN; while the operation of ∂ f ∂ x can be determined by making use of the adjoint of the linearization of the forward operator.
Once z map is determined, one may evaluate g ( z map ) by using the GAN generator.
This represents the value of the ﬁeld we wish to infer at the most likely value value of latent vector.
Note that this is not the same as the MAP estimate of p post X ( x | y ) .
C A DDITIONAL RESULTS
In this section we provide additional results for both MNIST and CelebA dataset for different tasks discussed in the main paper.
C. MNIST
First we provide additional examples in Figure 7 for variance-based adaptive measurement window selection procedure described in Section 3.1.
Figure 8 shows additional results for the inpainting + denoising task, where MNIST digits are occluded with masks of different sizes at different locations.
Note that the variance is high where the occlusion mask is located indicating lower conﬁdence in reconstructed image in that location.
C [Equation]
For the CelebA dataset, we trained WGAN-GP model using more than 200k celebrity facial images and perform inference using remaining test set images.
The input images were cropped to a 64 × RGB image and were normalized between [-1, 1].
Once the GAN was trained, the HMC algorithm was used for posterior sampling and inference on a complimentary set of images (not used for training).
In Figure 9 we show some additional results for variance-based adaptive measurement window selection procedure for CelebA dataset.
Next, in ﬁgure 10 we show some additional results for image recovery task for CelebA dataset.
Once again we note that the MAP estimate and the mean is close to the true image.
On the other hand, the closest image from the training set (in an L 2 sense) is not as accurate.
This points to the utility of using the GAN as an interpolant in the latent vector space.
D A RCHITECTURE AND TRAINING DETAILS
We use the WGAN-GP model for learning prior density.
The tuned value of hyper-parameters is shown in Table 1.
We use the same generator and discriminator architecture for the MNIST and the physics-based inference problem; whereas for the CelebA dataset we use a slightly different architecture to accommodate different input image size.
The layout of both these architecture is shown in Figure 11.
Some notes regarding nomenclature used in Figure 11. [Table]
• Conv (HxWxC s=n) indicates convolutional layer with ﬁler size of HxW and number of ﬁlters=
C with stride=
n. • FC
(x,y) indicates fully connected layer with x neurons in input layer and y neurons in output layer.
• BN = Batch norm,
LN =
Layer norm.
• TrConv = Transposed Convolution.
• LReLU = Leaky ReLU with α =0.2.
[Figure]
[Figure] [Figure] [Figure]