E FFICIENT S AMPLING FOR G ENERATIVE A DVERSARIAL N ETWORKS WITH C OUPLING M ARKOV C HAINS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).
However, in practice, they typically have poor sample efﬁciency because of the independent proposal sampling from the generator.
In this work, we propose CMC-GAN, a novel sampling method with dependent proposals reparametrized through a Coupling Markov Chain (CMC) in the latent space of the generator.
Theoretically, we prove that our CMC proposal can converge to the true data distribution, given a perfect discriminator.
Empirically, extensive experiments on synthetic and real datasets demonstrate that our CMCGAN largely improves the sample efﬁciency and obtains better sample quality simultaneously.
I NTRODUCTION
Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have achieved a great success on generating realistic images in recent years (Karras et al., 2019; Brock et al., 2019).
Unlike previous models that explicitly parameterize the data distribution, GANs rely on an alternative optimization between a generator and a discriminator to learn the data distribution implicitly.
However, in practice, samples generated by GANs still suffer from problems such as mode collapse and bad artifacts.
Recently, sampling methods have shown promising results on enhancing the sample quality of GANs by making use of the information in the discriminator.
Conventionally, after GAN training, the discriminator is discarded, and only the generator is left for image generation.
While the sampling methods utilize the density ratio information (implied by the discriminator) to close the gap between the generative distribution and the data distribution.
For example, DRS (Azadi et al., 2019) applies rejection sampling, and MH-GAN (Turner et al., 2019) adopts Markov chain Monte Carlo (MCMC) sampling for the improved sample quality of GANs.
Nevertheless, these methods still suffer a lot from the sample efﬁciency problem.
For example, as will be shown in Section 5, MH-GAN’s average acceptance ratio on CIFAR10 can be lower than 5% , which makes the Markov chains slow to mix. As
MH-GAN adopts an independent proposal q , i.e., q ( x (cid:48)
| x ) =
q ( x (cid:48) ) , the difference between samples can be so large that the proposal gets rejected easily.
To address this limitation, we propose to generalize the independent proposal to a general dependent proposal q ( x (cid:48) | x ) .
To the end, the proposed sample can be a reﬁnement of the previous one, which leads to a higher acceptance ratio and better sample quality.
We can also balance between the exploration and exploitation of the Markov chains by tuning the step size.
However, it is hard to design a proper dependent proposal in the high dimensional sample space X because the energy landscape could be very complex (Neal et al., 2010).
Nevertheless, we notice that the generative distribution p g ( x ) of GANs is implicitly deﬁned as the push-forward of the latent prior distribution p 0 ( z ) , and designing proposals in the low dimensional latent space is generally much easier.
Hence, GAN’s latent variable structure motivates us to design a structured dependent proposal with two Coupling Markov Chains (CMC), one in the sample space X and the other in the latent space Z .
As shown in Figure 1, given the current coupling samples ( z k , x k ) , we draw the next proposal x (cid:48) in a bottom-to-up way: 1) drawing a latent proposal z (cid:48) following q ( z (cid:48) | z k ) ; 2) pushing it forward through the generator and getting the sample proposal x (cid:48)
{+= G ( z (cid:48) ) ; 3) assigning x k +
 = x (cid:48)+}
if the proposal x {+(cid:48) is accepted, and x k +
+}
= x k otherwise. By utilizing the underlying structure of GANs, the proposed CMC sampling becomes more efﬁcient in the low-dimensional latent space.
We summarize our main contributions as follows: [Table] /TV7ccsjVAaJqjWHc9NjJ9RZTgTOCl2U40JZSM6wI6lkkao/Wx27oScWqVPwljZkobM1N8TGY20HkeB7YyoGepFbyr+53VSE177GZdJalCy+aIwFcTEZPo76XOFzIixJZQpbm8lbEgVZcYmVLQheIsvL5PmRcWrVrz7y3LtJo+jAMdwAmfgwRXU4A7q0AAGI3iGV3hzEufFeXc+5q0rTj5zBH/gfP4ABdePWw==</latexit> ... accept reject accept ...
Latent transition
Generator Selector G G
G G
MHGAN DDLS CMCGAN (ours) • We propose a structured dependent proposal of GANs, which reparameterizes the samplelevel transition x → x (cid:48) into the latent-level z → z (cid:48) with two coupling Markov chains.
We prove that it converges to the true data distribution with a tractable acceptance criterion. • Our proposed method, called CMC-GAN, serves as a uniﬁed framework for the existing sampling methods of GANs. It provides a better balance between exploration and exploitation by the structured dependent proposal, and also guarantees the convergence of the Markov chain by the acceptance-rejection step.
• Empirical results demonstrate that CMC-GAN achieves better image quality and much higher sample efﬁciency than the state-of-the-art methods on both synthetic and real datasets.
R ELATED W ORK
Although GANs are able to synthesize high-quality images, the minimax nature of GANs makes it quite unstable and leads to mode collapse.
A series of works like DCGAN (Radford et al., 2015) and
SNGAN (Miyato et al., 2018) improve the sample quality by designing new architectures.
Methods like Unrolled GAN (Metz et al., 2017) and PACGAN (Lin et al., 2018), try to ﬁx mode collapse with new training mechanisms.
Other methods like WGAN (Arjovsky et al., 2017) and WGAN-GP (Gulrajani et al., 2017) instead propose alternative objectives for better training stability.
Moreover, there is another line of work using sampling methods to improve the sample quality of GANs. DRS (Azadi et al., 2019) ﬁrstly proposes to use rejection sampling.
MH-GAN (Turner et al., 2019) instead uses the Metropolis-Hasting (MH) algorithm with an independent proposal.
DDLS (Che et al., 2020) and DCD (Song et al., 2020) apply gradient-based proposals by viewing GAN as an energy-based model.
Tanaka (2019) proposes a similar gradient-based method named DOT from the perspective of optimal transport.
[Table] Different from them, our CMC-GAN introduces a structured dependent proposal with two coupling
Markov chains, and includes all three effective sampling mechanisms, the Markov chain Monte Carlo method, the acceptance-rejection step, and the latent gradient-based proposal, to further improve the sample efﬁciency.
As shown in Table 1, many existing works are special cases of our CMC-GAN.
B ACKGROUND
In this section, we brieﬂy introduce the notations and backgrounds.
3. GAN
GAN models the data distribution p d ( x ) implicitly with a generator G : Z → X mapping from a low-dimensional latent space Z to a high-dimensional sample space X , x = G ( z ) , z ∼ p 0 ( z ) , (1) where the sample x follows the generative distribution p g ( x ) and the latent variable z follows the prior distribution p 0 ( z ) , e.g., a standard normal distribution N ( 0 , I ) .
In GAN, a discriminator
D : X → [0 , 1] is learned to distinguish samples from p d ( x ) and p g ( x ) in an adversarial way
[Equation]
Goodfellow et al.
(2014) point out that an optimal discriminator D implies the density ratio between the data and generative distributions
[Equation]
3. MCMC
Markov chain Monte Carlo (MCMC) refers to a kind of sampling methods that draws a chain of samples x 1: K ∈ X K from a target distribution p t ( x ) .
We denote the initial distribution as p 0 ( x ) and the proposal distribution as q ( x (cid:48) | x k ) .
With the Metropolis-Hastings (MH) algorithm, we accept the proposal x (cid:48) ∼ q ( x (cid:48) | x k ) with probability α ( x (cid:48) , x k ) =
min (cid:18) 1 , p
[Equation]
If x (cid:48) is accepted, x k +
= x (cid:48) , otherwise x k +1 = x k .
Under mild assumptions, the Markov chain is guaranteed to converge to p t ( x ) as K → ∞ .
In practice, the sample efﬁciency of MCMC crucially depends on the proposal distribution to trade off between exploration and exploitation.
T HE P ROPOSED CMC-GAN
In this section, we ﬁrst review MH-GAN and point out the limitations.
We then propose our structured dependent proposal to overcome these obstacles, and ﬁnally discuss its theoretical properties as well as practical implementation.
4. F ROM I NDEPENDENT P ROPOSAL TO D EPENDENT P ROPOSAL
MH-GAN (Turner et al., 2019)
ﬁrst proposes to improve GAN sampling with MCMC.
Speciﬁcally, given a perfect discriminator D and a descent (but imperfect) generator G after training, they take the data distribution p d ( x ) as the target distribution and use the generator distribution p g ( x ) as an independent proposal
[Equation]
With the MH criterion (Eqn. (4)) and the density ratio (Eqn. (3)), we should accept x (cid:48) with probability α MH ( x (cid:48) , x k )
[Equation]
min [Equation] As a result, the Markov chain is guaranteed to converge to the true data distribution p d ( x ) .
However, to achieve tractability, MH-GAN adopts an independent proposal q ( x (cid:48) ) with poor sample efﬁciency.
As the proposed sample x (cid:48) is independent of the current sample x k , the difference between the two samples can be so large that it results in a very low acceptance probability.
Consequently, samples can be trapped in the same place for a long time, leading to a very slow mixing of the chain.
A natural solution is to take a dependent proposal q ( x (cid:48) | x k ) that will propose a sample x (cid:48) close to the current one x k , which is more likely to be accepted.
Nevertheless, the problem of such a dependent proposal is that its MH acceptance criterion
[Equation]
is generally intractable because the data density p d ( x ) is unknown.
Besides, it is hard to design a proper dependent proposal q ( x (cid:48)
| x k ) in the high dimensional sample space X with complex landscape.
These obstacles prevent us from adopting a dependent proposal that is more suitable for MCMC.
4. A T RACTABLE S TRUCTURED D EPENDENT P ROPOSAL WITH C OUPLING M ARKOV C HAINS
As discussed above, the major difﬁculty of a general dependent proposal q ( x (cid:48) | x k ) is to compute the MH criterion.
We show that it can be made tractable by considering an additional coupling Markov chain in the latent space.
As we know, samples of GANs lie in a low-dimensional manifold induced by the push-forward of the latent variable.
Suppose that at the k -th step of the Markov chain, we have a GAN sample x k with latent z k .
Instead of drawing a sample x (cid:48) directly from a sample-level proposal distribution q ( x (cid:48) | x k ) , we ﬁrst draw a latent proposal z (cid:48) from a dependent latent proposal distribution q ( z (cid:48) | z k ) .
Afterward, we push the latent z (cid:48) forward through the generator and get the output x (cid:48) as our sample proposal.
As illustrated in Figure 1, our bottom-to-up proposal relies on the interplay between two coupling
Markov chains in the sample space X and the latent space Z .
Hence we call it a CMC (Coupling Markov Chain) proposal.
Through a learned generator, we transport the transition x k → x (cid:48) in the high dimensional space X into the low dimensional space Z , z k
→ z (cid:48) , which enjoys a much better landscape and makes it easier to design proposals in MCMC algorithms.
For example, the latent target distribution is nearly standard normal when the generator is nearly perfect.
In fact, under mild conditions, the CMC proposal distribution q CMC ( x (cid:48) | x k ) and the latent proposal distribution q ( z (cid:48) | z k ) are tied with the following change of variables (Gemici et al., 2016; Ben-Israel, 1999)
[Equation]
where J z denotes the Jacobian matrix of the push-forward G at z , i.e., [ J z ] ij = ∂
x i /∂
z j , x = G ( z ) .
Nevertheless, it remains unclear whether we can perform the MH test to decide the acceptance of the proposal x (cid:48) .
Note that a general dependent proposal distribution does not meet a tractable MH acceptance criterion (Eqn. (7)).
Perhaps surprisingly, it can be shown that with our structured CMC proposal, the MH acceptance criterion is tractable for general latent proposals q ( z (cid:48) | z k ) .
Theorem 1.
Consider a Markov chain of GAN samples x 1: K with initial distribution p g ( x ) . For step k + 1 , we accept our CMC proposal x (cid:48) ∼ q CMC ( x (cid:48) | x k ) with probability
[Equation]
i.e. let x k +
= x (cid:48) if x (cid:48) is accepted and x k +1 =
x k otherwise.
Further assume the chain is irreducible and aperiodic.
Then, according to the Metropolis-Hastings algorithm, the stationary distribution of this Markov chain is the data distribution p d ( x ) .
Proof.
Note that similar to Eqn (8), we also have the change of variables between p g ( x ) and p 0 ( z ) ,
[Equation]
We only need to prove that Eqn.
(9) holds.
Together with Eqn. (3), (7) and (8), we have
[Equation]
Hence the proof is completed.
The theorem above demonstrates the following favorable properties of our method:
• The discriminator score ratio is the same as α MH ( x (cid:48) , x k ) , but MH-GAN is restricted to a speciﬁc independent proposal.
Our method instead works for any latent proposal q ( z (cid:48)
| z k ) . When we take q ( z (cid:48) | z k ) =
p 0 ( z (cid:48) ) , our method reduces to MH-GAN.
Compared to α DEP ( x (cid:48) , x k ) of a general dependent proposal (Eqn. (7)), the unknown data distributions terms are successfully cancelled in the reparameterized acceptance criterion.
• The reparameterized MH acceptance criterion becomes tractable as it only involves the latent priors, the latent proposal distributions, and the discriminator scores.
Combining the CMC proposal q CMC ( x (cid:48) | x k ) and its tractable MH criterion α CMC ( x (cid:48) , x k ) , we have developed a novel sampling method for GANs, coined as CMC-GAN.
See Appendix 1 for a detailed description.
4. A P RACTICAL I MPLEMENTATION
CMC-GAN enables us to utilize the vast literature of existing MCMC algorithms (Neal et al., 2010) to design dependent proposals for GANs.
We take Langevin Monte Carlo (LMC) as an example.
As an Euler-Maruyama discretization of the Langevin dynamics, LMC updates the Markov chain with
[Equation]
(12) for a target distribution p t ( x ) .
Compared to MH-GAN, LMC utilizes the gradient information to explore the energy landscape more efﬁciently.
However, if we directly take the (unknown) data distribution p d ( x ) as the target distribution p t ( x ) , LMC does not meet a tractable update rule.
As discussed above, the reparameterization of CMC-GAN makes it easier to design transitions in the low-dimensional latent space.
Hence, we instead propose to use LMC for the latent Markov chain.
We assume that the data distribution also lies in the low-dimensional manifold induced by the generator, i.e., Supp ( p d ) ⊂
Im( G ) .
This implies that the data distribution p d ( x ) also has a coupling distribution in the latent space, denoted as p t ( z ) .
They are tied with the change of variables
[Equation]
Taking p t ( z ) as the (unknown) target distribution of the latent Markov chain, we have the following
Latent LMC (L2MC) proposal
[Equation]
where x k = G ( z k ) .
As we can see, L2MC is made tractable by our structured dependent proposal with coupling Markov chains.
DDLS (Che et al., 2020) proposes a similar Langevin proposal by formalizing GANs as an implicit energy-based model, while here we provide a straightforward derivation through reparameterization.
Our major difference to DDLS is that CMC-GAN also includes a tractable MH correction step (Eqn. (9)), which accounts for the numerical errors introduced by the discretization and ensures that detailed balance holds.
4. E XTENSION TO WGAN
Our method can also be extended to other kinds of GAN, like Wasserstein GAN (WGAN)
(Arjovsky et al., 2017).
The WGAN objective is
[Equation]
(15) where D : X → R is restricted to be a Lipschitz function.
Under certain conditions, WGAN also implies an approximate estimation of the density ratio (Che et al., 2020),
[Equation]
[Equation]
Following the same derivations as in Eqn.
(11) and (14), we will have the WGAN version of CMC-GAN.
Speciﬁcally, with x k = G ( z k ) , the L2MC proposal follows [Equation]
[Equation]
and the MH acceptance criterion is
[Equation]
E XPERIMENTS
We show our empirical results both on synthetic and real image datasets.
5. S YNTHETIC D ATA
Following DOT (Tanaka, 2019) and DDLS (Che et al., 2020), we apply CMC-GAN to the synthetic
Swiss Roll dataset, where data samples lie on a Swiss roll manifold in the two-dimensional space.
We construct the dataset by scikit-learn with 100,000 samples, and train a WGAN as in Tanaka (2019),
[Figure]
[Table]
[Figure]
where both the generator and discriminator are fully connected neural networks with leaky ReLU nonlinearities.
We optimize the model using the Adam optimizer, with α = 0 . 0001 , β 1 = 0 .
5 , β 2 = 0 .
After training, we draw 1 , 000 samples with different sampling methods.
As shown in Figure 2, with appropriate step size ( τ = 0 . 01 ), the gradient-based methods (DDLS and
CMC-GAN) outperform independent proposals (DRS and MH-GAN) by a large margin, while DDLS is more discontinuous on shape compared to CMC-GAN.
In DDLS, when the step size becomes too large ( τ = 0 . 1 , 1 ), the numerical error of the Langevin dynamics becomes so large that the chain either collapses or diverges.
In contrast, those bad proposals are rejected by the MH correction steps of CMC-GAN, which prevents the misbehavior of the Markov chain.
5. R EAL I MAGE D ATA
Following MH-GAN (Turner et al., 2019), we conduct experiments on two real-world image datasets, CIFAR-10 and CelebA, for DCGAN (Radford et al., 2015) and WGAN (Arjovsky et al., 2017).
For a fair comparison of MCMC methods, each Markov chain starts from a GAN sample and runssteps, and we take the last sample for evaluation.
The step size τ of our L2MC proposal is 0 . 01 on CIFAR-10 and 0 .
on CelebA. We calibrate the discriminator with Logistic Regression as in Turner et al.
(2019), and draw 50 , 000 samples to compute the Inception Score (Salimans et al., 2016).
From Table 2, we can see our method outperforms state-of-the-art methods on both datasets. We also plot the Inception Score and acceptance ratio per epoch in Figure 3 based on our re-implementation.
CMC-GAN and MH-GAN outperform DDLS by a large margin, showing that the MH correction steps also matter on real-world datasets. In most cases, CMC-GAN has higher Inception Scores and much higher acceptance ratios than MH-GAN, indicating the superiority of our structured dependent proposal.
The average acceptance ratio of MH-GAN is lower than 0 .
2 in most cases, while CMCGAN has an acceptance ratio of around 0 .
4 to 0 .
8 , which is known to be a good tradeoff for MCMC algorithms. Ablation study.
From Table 3, we can see that without the MH correction step, the Langevin steps often result in worse sample quality. Meanwhile, the acceptance is very small on CIFAR- [Table] without the dependent CMC proposal.
As a result, our CMC-GAN (CMC
MH) is the only setup that consistently improves over the baseline and obtains the best Inception Score on each dataset.
The only exception is DCGAN on CelebA, where the independent proposal outperforms our CMC proposal with a higher acceptance ratio.
We believe that this is because the human face samples of
CelebA are very alike among each other, such that independent samples from the generator can also be easily accepted.
Nevertheless, the acceptance ratio of the independent proposal can be much lower on datasets with diverse sources of images, like CIFAR-10.
Markov chain visualization.
In Figure 4, we demonstrate two Markov chains sampled with different methods.
We can see that MH-GAN is often trapped in the same place because of the independent proposals.
DDLS and CMC-GAN instead gradually reﬁne the samples with gradient steps. In addition, the MH rejection steps of CMC-GAN help avoid some bad artifacts in the images.
C ONCLUSION
In this paper, we have proposed a novel method, CMC-GAN, to improve the sampling of GANs.
We devise a structured dependent proposal that reparameterizes the sample-level transition of GANs into the latent-level transition.
More importantly, we ﬁrst prove that this general proposal meets a tractable MH criterion.
Experiments show our method does not only improve sample efﬁciency but also demonstrate state-of-the-art sample quality on benchmark datasets over existing sampling methods.
We demonstrate two Markov chains on CelebA with different MCMC sampling methods of WGAN in Figure 5.
We can see that on CelebA, the acceptance ratio of MH-GAN becomes much higher than that on CIFAR-10. Nevertheless, the sample quality is still relatively low. In comparison, the gradient-based method can gradually reﬁne the samples with Langevin steps, and our CMC-GAN can alleviate image artifacts with MH correction steps.