Efficient Sampling for Generative Adversarial Networks with Reparameterized Markov Chains

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have achieved a great success on generating realistic images in recent years ( Karras et al. , 2019 ; Brock et al. , 2019 ) . Unlike previous models that explicitly parameterize the data distribution , GANs rely on an alternative optimization between a generator and a discriminator to learn the data distribution implicitly . However , in practice , samples generated by GANs still suffer from problems such as mode collapse and bad artifacts . Recently , sampling methods have shown promising results on enhancing the sample quality of GANs by making use of the information in the discriminator . In the alternative training scheme of GANs , the generator only performs a few updates for the inner loop and has not fully utilized the density ratio information estimated by the discriminator . Thus , after GAN training , the sampling methods propose to further utilize this information to bridge the gap between the generative distribution and the data distribution in a fine-grained manner . For example , DRS ( Azadi et al. , 2019 ) applies rejection sampling , and MH-GAN ( Turner et al. , 2019 ) adopts Markov chain Monte Carlo ( MCMC ) sampling for the improved sample quality of GANs . Nevertheless , these methods still suffer a lot from the sample efficiency problem . For example , as will be shown in Section 5 , MH-GAN ’ s average acceptance ratio on CIFAR10 can be lower than 5 % , which makes the Markov chains slow to mix . As MH-GAN adopts an independent proposal q , i.e. , q ( x′|x ) = q ( x′ ) , the difference between samples can be so large that the proposal gets rejected easily . To address this limitation , we propose to generalize the independent proposal to a general dependent proposal q ( x′|x ) . To the end , the proposed sample can be a refinement of the previous one , which leads to a higher acceptance ratio and better sample quality . We can also balance between the exploration and exploitation of the Markov chains by tuning the step size . However , it is hard to design a proper dependent proposal in the high dimensional sample space X because the energy landscape could be very complex ( Neal et al. , 2010 ) . Nevertheless , we notice that the generative distribution pg ( x ) of GANs is implicitly defined as the push-forward of the latent prior distribution p0 ( z ) , and designing proposals in the low dimensional latent space is generally much easier . Hence , GAN ’ s latent variable structure motivates us to design a structured dependent proposal with two pairing Markov chains , one in the sample space X and the other in the latent space Z . As shown in Figure 1 , given the current pairing samples ( zk , xk ) , we draw the next proposal x′ in a bottom-to-up way : 1 ) drawing a latent proposal z′ following q ( z′|zk ) ; 2 ) pushing it forward through the generator and getting the sample proposal x′ = G ( z′ ) ; 3 ) assigning xk+1 = x′ if the proposal x′ is accepted , otherwise rejected xk+1 = xk . By utilizing the underlying structure of GANs , the proposed reparameterized sampler becomes more efficient in the low-dimensional latent space . We summarize our main contributions as follows : • We propose a structured dependent proposal of GANs , which reparameterizes the samplelevel transition x → x′ into the latent-level z → z′ with two pairing Markov chains . We prove that our reparameterized proposal admits a tractable acceptance criterion . • Our proposed method , called REP-GAN , serves as a unified framework for the existing sampling methods of GANs . It provides a better balance between exploration and exploitation by the structured dependent proposal , and also corrects the bias of Markov chains by the acceptance-rejection step . • Empirical results demonstrate that REP-GAN achieves better image quality and much higher sample efficiency than the state-of-the-art methods on both synthetic and real datasets . 2 RELATED WORK . Although GANs are able to synthesize high-quality images , the minimax nature of GANs makes it quite unstable , which usually results in degraded sample quality . A vast literature has been developed to fix the problems of GANs ever since , including novel network modules ( Miyato et al. , 2018 ) , training mechanism ( Metz et al. , 2017 ) , and alternative objectives ( Arjovsky et al. , 2017 ) . Moreover , there is another line of work using sampling methods to improve the sample quality of GANs . DRS ( Azadi et al. , 2019 ) firstly proposes to use rejection sampling . MH-GAN ( Turner et al. , 2019 ) instead uses the Metropolis-Hasting ( MH ) algorithm with an independent proposal . DDLS ( Che et al. , 2020 ) and DCD ( Song et al. , 2020 ) apply gradient-based proposals by viewing GAN as an energy-based model . Tanaka ( 2019 ) proposes a similar gradient-based method named DOT from the perspective of optimal transport . Different from them , our REP-GAN introduces a structured dependent proposal through latent reparameterization , and includes all three effective sampling mechanisms , the Markov Chain Monte Carlo method , the acceptance-rejection step , and the latent gradient-based proposal , to further improve the sample efficiency . As shown in Table 1 , many existing works are special cases of our REP-GAN . Our method also belongs to the thread of works that combine MCMC and neural networks for better sample quality . Previously , some works combine variational autoencoders ( Kingma & Welling , 2014 ) and MCMC to bridge the amorization gap ( Salimans et al. , 2015 ; Hoffman , 2017 ; Li et al. , 2017 ) , while others directly learn a neural proposal function for MCMC ( Song et al. , 2017 ; Levy et al. , 2018 ; Wang et al. , 2018 ) . Our work instead reparameterizes the high-dimensional sample-level transition into a simpler low-dimensional latent space via the learned generator network . 3 BACKGROUND . 3.1 GAN . GAN models the data distribution pd ( x ) implicitly with a generator G : Z → X mapping from a low-dimensional latent space Z to a high-dimensional sample space X , x = G ( z ) , z ∼ p0 ( z ) , ( 1 ) where the sample x follows the generative distribution pg ( x ) and the latent variable z follows the prior distribution p0 ( z ) , e.g. , a standard normal distribution N ( 0 , I ) . In GAN , a discriminator D : X → [ 0 , 1 ] is learned to distinguish samples from pd ( x ) and pg ( x ) in an adversarial way min G max D Ex∼pd ( x ) log ( D ( x ) ) + Ez∼p0 ( z ) log ( 1−D ( G ( z ) ) ) . ( 2 ) Goodfellow et al . ( 2014 ) point out that an optimal discriminator D implies the density ratio between the data and generative distributions D ( x ) = pd ( x ) pd ( x ) + pg ( x ) ⇒ pd ( x ) pg ( x ) = 1 D ( x ) −1 − 1 . ( 3 ) 3.2 MCMC . Markov Chain Monte Carlo ( MCMC ) refers to a kind of sampling methods that draw a chain of samples x1 : K ∈ XK from a target distribution pt ( x ) . We denote the initial distribution as p0 ( x ) and the proposal distribution as q ( x′|xk ) . With the Metropolis-Hastings ( MH ) algorithm , we accept the proposal x′ ∼ q ( x′|xk ) with probability α ( x′ , xk ) = min ( 1 , pt ( x ′ ) q ( xk|x′ ) pt ( xk ) q ( x′|xk ) ) ∈ [ 0 , 1 ] . ( 4 ) If x′ is accepted , xk+1 = x′ , otherwise xk+1 = xk . Under mild assumptions , the Markov chain is guaranteed to converge to pt ( x ) as K →∞ . In practice , the sample efficiency of MCMC crucially depends on the proposal distribution to trade off between exploration and exploitation . 4 THE PROPOSED REP-GAN . In this section , we first review MH-GAN and point out the limitations . We then propose our structured dependent proposal to overcome these obstacles , and finally discuss its theoretical properties as well as practical implementations . 4.1 FROM INDEPENDENT PROPOSAL TO DEPENDENT PROPOSAL . MH-GAN ( Turner et al. , 2019 ) first proposes to improve GAN sampling with MCMC . Specifically , given a perfect discriminator D and a descent ( but imperfect ) generator G after training , they take the data distribution pd ( x ) as the target distribution and use the generator distribution pg ( x ) as an independent proposal x′ ∼ q ( x′|xk ) = q ( x′ ) = pg ( x′ ) . ( 5 ) With the MH criterion ( Eqn . ( 4 ) ) and the density ratio ( Eqn . ( 3 ) ) , we should accept x′ with probability αMH ( x ′ , xk ) = min ( 1 , pd ( x ′ ) q ( xk ) pd ( xk ) q ( x′ ) ) = min ( 1 , D ( xk ) −1 − 1 D ( x′ ) −1 − 1 ) . ( 6 ) However , to achieve tractability , MH-GAN adopts an independent proposal q ( x′ ) with poor sample efficiency . As the proposed sample x′ is independent of the current sample xk , the difference between the two samples can be so large that it results in a very low acceptance probability . Consequently , samples can be trapped in the same place for a long time , leading to a very slow mixing of the chain . A natural solution is to take a dependent proposal q ( x′|xk ) that will propose a sample x′ close to the current one xk , which is more likely to be accepted . Nevertheless , the problem of such a dependent proposal is that its MH acceptance criterion αDEP ( x ′ , xk ) = min ( 1 , pd ( x ′ ) q ( xk|x′ ) pd ( xk ) q ( x′|xk ) ) , ( 7 ) is generally intractable because the data density pd ( x ) is unknown . Besides , it is hard to design a proper dependent proposal q ( x′|xk ) in the high dimensional sample space X with complex landscape . These obstacles prevent us from adopting a dependent proposal that is more suitable for MCMC . 4.2 A TRACTABLE STRUCTURED DEPENDENT PROPOSAL WITH REPARAMETERIZED MARKOV CHAINS . As discussed above , the major difficulty of a general dependent proposal q ( x′|xk ) is to compute the MH criterion . We show that it can be made tractable by considering an additional pairing Markov chain in the latent space . As we know , samples of GANs lie in a low-dimensional manifold induced by the push-forward of the latent variable . Suppose that at the k-th step of the Markov chain , we have a GAN sample xk with latent zk . Instead of drawing a sample x′ directly from a sample-level proposal distribution q ( x′|xk ) , we first draw a latent proposal z′ from a dependent latent proposal distribution q ( z′|zk ) . Afterward , we push the latent z′ forward through the generator and get the output x′ as our sample proposal . As illustrated in Figure 1 , our bottom-to-up proposal relies on the transition reparameterization with two pairing Markov chains in the sample space X and the latent space Z . Hence we call it a REP ( reparameterized ) proposal . Through a learned generator , we transport the transition xk → x′ in the high dimensional space X into the low dimensional space Z , zk → z′ , which enjoys a much better landscape and makes it easier to design proposals in MCMC algorithms . For example , the latent target distribution is nearly standard normal when the generator is nearly perfect . In fact , under mild conditions , the REP proposal distribution qREP ( x′|xk ) and the latent proposal distribution q ( z′|zk ) are tied with the following change of variables ( Gemici et al. , 2016 ; Ben-Israel , 1999 ) log qREP ( x ′|xk ) = log q ( x′|zk ) = log q ( z′|zk ) − 1 2 log detJ > z′Jz′ , ( 8 ) where Jz denotes the Jacobian matrix of the push-forward G at z , i.e. , [ Jz ] ij = ∂ xi/∂ zj , x = G ( z ) . Nevertheless , it remains unclear whether we can perform the MH test to decide the acceptance of the proposal x′ . Note that a general dependent proposal distribution does not meet a tractable MH acceptance criterion ( Eqn . ( 7 ) ) . Perhaps surprisingly , it can be shown that with our structured REP proposal , the MH acceptance criterion is tractable for general latent proposals q ( z′|zk ) . Theorem 1 . Consider a Markov chain of GAN samples x1 : K with initial distribution pg ( x ) . For step k + 1 , we accept our REP proposal x′ ∼ qREP ( x′|xk ) with probability αREP ( x ′ , xk ) = min ( 1 , p0 ( z ′ ) q ( zk|z′ ) p0 ( zk ) q ( z′|zk ) · D ( xk ) −1 − 1 D ( x′ ) −1 − 1 ) , ( 9 ) i.e . let xk+1 = x′ if x′ is accepted and xk+1 = xk otherwise . Further assume the chain is irreducible , aperiodic and not transient . Then , according to the Metropolis-Hastings algorithm , the stationary distribution of this Markov chain is the data distribution pd ( x ) ( Gelman et al. , 2013 ) . Proof . Note that similar to Eqn ( 8 ) , we also have the change of variables between pg ( x ) and p0 ( z ) , log pg ( x ) |x=G ( z ) = log p0 ( z ) − 1 2 log detJ > z Jz . ( 10 ) According to Gelman et al . ( 2013 ) , the assumptions that the chain is irreducible , aperiodic , and not transient make sure that the chain has a unique stationary distribution , and the MH algorithm ensures that this stationary distribution equals to the target distribution pd ( x ) . Thus we only need to show that the MH criterion in Eqn . ( 9 ) holds . Together with Eqn . ( 3 ) , ( 7 ) and ( 8 ) , we have αREP ( x ′ , xk ) = pd ( x ′ ) q ( xk|x′ ) pd ( xk ) q ( x′|xk ) = pd ( x ′ ) q ( zk|z′ ) ( det J > zkJzk ) − 12 pg ( xk ) pg ( x′ ) pd ( xk ) q ( z′|zk ) ( det J > z′Jz′ ) − 12 pg ( x′ ) pg ( xk ) = q ( zk|z′ ) ( det J > zkJzk ) − 12 p0 ( z′ ) ( det J > z′Jz′ ) − 12 ( D ( xk ) −1 − 1 ) q ( z′|zk ) ( det J > z′Jz′ ) − 12 p0 ( zk ) ( det J > zkJzk ) − 12 ( D ( x′ ) −1 − 1 ) = p0 ( z ′ ) q ( zk|z′ ) ( D ( xk ) −1 − 1 ) p0 ( zk ) q ( z′|zk ) ( D ( x′ ) −1 − 1 ) . ( 11 ) Hence the proof is completed . The theorem above demonstrates the following favorable properties of our method : • The discriminator score ratio is the same as αMH ( x′ , xk ) , but MH-GAN is restricted to a specific independent proposal . Our method instead works for any latent proposal q ( z′|zk ) . When we take q ( z′|zk ) = p0 ( z′ ) , our method reduces to MH-GAN . • Compared to αDEP ( x′ , xk ) of a general dependent proposal ( Eqn . ( 7 ) ) , the unknown data distributions terms are successfully cancelled in the reparameterized acceptance criterion . • The reparameterized MH acceptance criterion becomes tractable as it only involves the latent priors , the latent proposal distributions , and the discriminator scores . Combining the REP proposal qREP ( x′|xk ) and its tractable MH criterion αREP ( x′ , xk ) , we have developed a novel sampling method for GANs , coined as REP-GAN . See Appendix 1 for a detailed description . Moreover , our method can serve as a general approximate inference technique for Bayesian models by bridging MCMC and GANs . Previous works ( Marzouk et al. , 2016 ; Titsias , 2017 ; Hoffman et al. , 2019 ) also propose to avoid the bad geometry of a complex probability measure by reparameterizing the Markov transitions into a simpler measure . However , these methods are limited to explicit invertible mappings without dimensionality reduction . In our work , we first show that it is also tractable to conduct such model-based reparameterization with implicit models like GANs .

Summary: The paper proposes an MCMC sampling strategy for GANs. The idea is clear: for high-dimensional x, making a good proposal is difficult, so they propose to do that in the latent space.  Then they use a similar strategy as MH-GAN to compute a rejection strategy. The difference between the two methods is that proposal in MH-GAN does not depend on x while the proposed method does, and the argument is that it results in a higher acceptance rate.