Efficient Sampling for Generative Adversarial Networks with Reparameterized Markov Chains

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have achieved a great success on generating realistic images in recent years ( Karras et al. , 2019 ; Brock et al. , 2019 ) . Unlike previous models that explicitly parameterize the data distribution , GANs rely on an alternative optimization between a generator and a discriminator to learn the data distribution implicitly . However , in practice , samples generated by GANs still suffer from problems such as mode collapse and bad artifacts . Recently , sampling methods have shown promising results on enhancing the sample quality of GANs by making use of the information in the discriminator . In the alternative training scheme of GANs , the generator only performs a few updates for the inner loop and has not fully utilized the density ratio information estimated by the discriminator . Thus , after GAN training , the sampling methods propose to further utilize this information to bridge the gap between the generative distribution and the data distribution in a fine-grained manner . For example , DRS ( Azadi et al. , 2019 ) applies rejection sampling , and MH-GAN ( Turner et al. , 2019 ) adopts Markov chain Monte Carlo ( MCMC ) sampling for the improved sample quality of GANs . Nevertheless , these methods still suffer a lot from the sample efficiency problem . For example , as will be shown in Section 5 , MH-GAN ’ s average acceptance ratio on CIFAR10 can be lower than 5 % , which makes the Markov chains slow to mix . As MH-GAN adopts an independent proposal q , i.e. , q ( x′|x ) = q ( x′ ) , the difference between samples can be so large that the proposal gets rejected easily . To address this limitation , we propose to generalize the independent proposal to a general dependent proposal q ( x′|x ) . To the end , the proposed sample can be a refinement of the previous one , which leads to a higher acceptance ratio and better sample quality . We can also balance between the exploration and exploitation of the Markov chains by tuning the step size . However , it is hard to design a proper dependent proposal in the high dimensional sample space X because the energy landscape could be very complex ( Neal et al. , 2010 ) . Nevertheless , we notice that the generative distribution pg ( x ) of GANs is implicitly defined as the push-forward of the latent prior distribution p0 ( z ) , and designing proposals in the low dimensional latent space is generally much easier . Hence , GAN ’ s latent variable structure motivates us to design a structured dependent proposal with two pairing Markov chains , one in the sample space X and the other in the latent space Z . As shown in Figure 1 , given the current pairing samples ( zk , xk ) , we draw the next proposal x′ in a bottom-to-up way : 1 ) drawing a latent proposal z′ following q ( z′|zk ) ; 2 ) pushing it forward through the generator and getting the sample proposal x′ = G ( z′ ) ; 3 ) assigning xk+1 = x′ if the proposal x′ is accepted , otherwise rejected xk+1 = xk . By utilizing the underlying structure of GANs , the proposed reparameterized sampler becomes more efficient in the low-dimensional latent space . We summarize our main contributions as follows : • We propose a structured dependent proposal of GANs , which reparameterizes the samplelevel transition x → x′ into the latent-level z → z′ with two pairing Markov chains . We prove that our reparameterized proposal admits a tractable acceptance criterion . • Our proposed method , called REP-GAN , serves as a unified framework for the existing sampling methods of GANs . It provides a better balance between exploration and exploitation by the structured dependent proposal , and also corrects the bias of Markov chains by the acceptance-rejection step . • Empirical results demonstrate that REP-GAN achieves better image quality and much higher sample efficiency than the state-of-the-art methods on both synthetic and real datasets . 2 RELATED WORK . Although GANs are able to synthesize high-quality images , the minimax nature of GANs makes it quite unstable , which usually results in degraded sample quality . A vast literature has been developed to fix the problems of GANs ever since , including novel network modules ( Miyato et al. , 2018 ) , training mechanism ( Metz et al. , 2017 ) , and alternative objectives ( Arjovsky et al. , 2017 ) . Moreover , there is another line of work using sampling methods to improve the sample quality of GANs . DRS ( Azadi et al. , 2019 ) firstly proposes to use rejection sampling . MH-GAN ( Turner et al. , 2019 ) instead uses the Metropolis-Hasting ( MH ) algorithm with an independent proposal . DDLS ( Che et al. , 2020 ) and DCD ( Song et al. , 2020 ) apply gradient-based proposals by viewing GAN as an energy-based model . Tanaka ( 2019 ) proposes a similar gradient-based method named DOT from the perspective of optimal transport . Different from them , our REP-GAN introduces a structured dependent proposal through latent reparameterization , and includes all three effective sampling mechanisms , the Markov Chain Monte Carlo method , the acceptance-rejection step , and the latent gradient-based proposal , to further improve the sample efficiency . As shown in Table 1 , many existing works are special cases of our REP-GAN . Our method also belongs to the thread of works that combine MCMC and neural networks for better sample quality . Previously , some works combine variational autoencoders ( Kingma & Welling , 2014 ) and MCMC to bridge the amorization gap ( Salimans et al. , 2015 ; Hoffman , 2017 ; Li et al. , 2017 ) , while others directly learn a neural proposal function for MCMC ( Song et al. , 2017 ; Levy et al. , 2018 ; Wang et al. , 2018 ) . Our work instead reparameterizes the high-dimensional sample-level transition into a simpler low-dimensional latent space via the learned generator network . 3 BACKGROUND . 3.1 GAN . GAN models the data distribution pd ( x ) implicitly with a generator G : Z → X mapping from a low-dimensional latent space Z to a high-dimensional sample space X , x = G ( z ) , z ∼ p0 ( z ) , ( 1 ) where the sample x follows the generative distribution pg ( x ) and the latent variable z follows the prior distribution p0 ( z ) , e.g. , a standard normal distribution N ( 0 , I ) . In GAN , a discriminator D : X → [ 0 , 1 ] is learned to distinguish samples from pd ( x ) and pg ( x ) in an adversarial way min G max D Ex∼pd ( x ) log ( D ( x ) ) + Ez∼p0 ( z ) log ( 1−D ( G ( z ) ) ) . ( 2 ) Goodfellow et al . ( 2014 ) point out that an optimal discriminator D implies the density ratio between the data and generative distributions D ( x ) = pd ( x ) pd ( x ) + pg ( x ) ⇒ pd ( x ) pg ( x ) = 1 D ( x ) −1 − 1 . ( 3 ) 3.2 MCMC . Markov Chain Monte Carlo ( MCMC ) refers to a kind of sampling methods that draw a chain of samples x1 : K ∈ XK from a target distribution pt ( x ) . We denote the initial distribution as p0 ( x ) and the proposal distribution as q ( x′|xk ) . With the Metropolis-Hastings ( MH ) algorithm , we accept the proposal x′ ∼ q ( x′|xk ) with probability α ( x′ , xk ) = min ( 1 , pt ( x ′ ) q ( xk|x′ ) pt ( xk ) q ( x′|xk ) ) ∈ [ 0 , 1 ] . ( 4 ) If x′ is accepted , xk+1 = x′ , otherwise xk+1 = xk . Under mild assumptions , the Markov chain is guaranteed to converge to pt ( x ) as K →∞ . In practice , the sample efficiency of MCMC crucially depends on the proposal distribution to trade off between exploration and exploitation . 4 THE PROPOSED REP-GAN . In this section , we first review MH-GAN and point out the limitations . We then propose our structured dependent proposal to overcome these obstacles , and finally discuss its theoretical properties as well as practical implementations . 4.1 FROM INDEPENDENT PROPOSAL TO DEPENDENT PROPOSAL . MH-GAN ( Turner et al. , 2019 ) first proposes to improve GAN sampling with MCMC . Specifically , given a perfect discriminator D and a descent ( but imperfect ) generator G after training , they take the data distribution pd ( x ) as the target distribution and use the generator distribution pg ( x ) as an independent proposal x′ ∼ q ( x′|xk ) = q ( x′ ) = pg ( x′ ) . ( 5 ) With the MH criterion ( Eqn . ( 4 ) ) and the density ratio ( Eqn . ( 3 ) ) , we should accept x′ with probability αMH ( x ′ , xk ) = min ( 1 , pd ( x ′ ) q ( xk ) pd ( xk ) q ( x′ ) ) = min ( 1 , D ( xk ) −1 − 1 D ( x′ ) −1 − 1 ) . ( 6 ) However , to achieve tractability , MH-GAN adopts an independent proposal q ( x′ ) with poor sample efficiency . As the proposed sample x′ is independent of the current sample xk , the difference between the two samples can be so large that it results in a very low acceptance probability . Consequently , samples can be trapped in the same place for a long time , leading to a very slow mixing of the chain . A natural solution is to take a dependent proposal q ( x′|xk ) that will propose a sample x′ close to the current one xk , which is more likely to be accepted . Nevertheless , the problem of such a dependent proposal is that its MH acceptance criterion αDEP ( x ′ , xk ) = min ( 1 , pd ( x ′ ) q ( xk|x′ ) pd ( xk ) q ( x′|xk ) ) , ( 7 ) is generally intractable because the data density pd ( x ) is unknown . Besides , it is hard to design a proper dependent proposal q ( x′|xk ) in the high dimensional sample space X with complex landscape . These obstacles prevent us from adopting a dependent proposal that is more suitable for MCMC . 4.2 A TRACTABLE STRUCTURED DEPENDENT PROPOSAL WITH REPARAMETERIZED MARKOV CHAINS . As discussed above , the major difficulty of a general dependent proposal q ( x′|xk ) is to compute the MH criterion . We show that it can be made tractable by considering an additional pairing Markov chain in the latent space . As we know , samples of GANs lie in a low-dimensional manifold induced by the push-forward of the latent variable . Suppose that at the k-th step of the Markov chain , we have a GAN sample xk with latent zk . Instead of drawing a sample x′ directly from a sample-level proposal distribution q ( x′|xk ) , we first draw a latent proposal z′ from a dependent latent proposal distribution q ( z′|zk ) . Afterward , we push the latent z′ forward through the generator and get the output x′ as our sample proposal . As illustrated in Figure 1 , our bottom-to-up proposal relies on the transition reparameterization with two pairing Markov chains in the sample space X and the latent space Z . Hence we call it a REP ( reparameterized ) proposal . Through a learned generator , we transport the transition xk → x′ in the high dimensional space X into the low dimensional space Z , zk → z′ , which enjoys a much better landscape and makes it easier to design proposals in MCMC algorithms . For example , the latent target distribution is nearly standard normal when the generator is nearly perfect . In fact , under mild conditions , the REP proposal distribution qREP ( x′|xk ) and the latent proposal distribution q ( z′|zk ) are tied with the following change of variables ( Gemici et al. , 2016 ; Ben-Israel , 1999 ) log qREP ( x ′|xk ) = log q ( x′|zk ) = log q ( z′|zk ) − 1 2 log detJ > z′Jz′ , ( 8 ) where Jz denotes the Jacobian matrix of the push-forward G at z , i.e. , [ Jz ] ij = ∂ xi/∂ zj , x = G ( z ) . Nevertheless , it remains unclear whether we can perform the MH test to decide the acceptance of the proposal x′ . Note that a general dependent proposal distribution does not meet a tractable MH acceptance criterion ( Eqn . ( 7 ) ) . Perhaps surprisingly , it can be shown that with our structured REP proposal , the MH acceptance criterion is tractable for general latent proposals q ( z′|zk ) . Theorem 1 . Consider a Markov chain of GAN samples x1 : K with initial distribution pg ( x ) . For step k + 1 , we accept our REP proposal x′ ∼ qREP ( x′|xk ) with probability αREP ( x ′ , xk ) = min ( 1 , p0 ( z ′ ) q ( zk|z′ ) p0 ( zk ) q ( z′|zk ) · D ( xk ) −1 − 1 D ( x′ ) −1 − 1 ) , ( 9 ) i.e . let xk+1 = x′ if x′ is accepted and xk+1 = xk otherwise . Further assume the chain is irreducible , aperiodic and not transient . Then , according to the Metropolis-Hastings algorithm , the stationary distribution of this Markov chain is the data distribution pd ( x ) ( Gelman et al. , 2013 ) . Proof . Note that similar to Eqn ( 8 ) , we also have the change of variables between pg ( x ) and p0 ( z ) , log pg ( x ) |x=G ( z ) = log p0 ( z ) − 1 2 log detJ > z Jz . ( 10 ) According to Gelman et al . ( 2013 ) , the assumptions that the chain is irreducible , aperiodic , and not transient make sure that the chain has a unique stationary distribution , and the MH algorithm ensures that this stationary distribution equals to the target distribution pd ( x ) . Thus we only need to show that the MH criterion in Eqn . ( 9 ) holds . Together with Eqn . ( 3 ) , ( 7 ) and ( 8 ) , we have αREP ( x ′ , xk ) = pd ( x ′ ) q ( xk|x′ ) pd ( xk ) q ( x′|xk ) = pd ( x ′ ) q ( zk|z′ ) ( det J > zkJzk ) − 12 pg ( xk ) pg ( x′ ) pd ( xk ) q ( z′|zk ) ( det J > z′Jz′ ) − 12 pg ( x′ ) pg ( xk ) = q ( zk|z′ ) ( det J > zkJzk ) − 12 p0 ( z′ ) ( det J > z′Jz′ ) − 12 ( D ( xk ) −1 − 1 ) q ( z′|zk ) ( det J > z′Jz′ ) − 12 p0 ( zk ) ( det J > zkJzk ) − 12 ( D ( x′ ) −1 − 1 ) = p0 ( z ′ ) q ( zk|z′ ) ( D ( xk ) −1 − 1 ) p0 ( zk ) q ( z′|zk ) ( D ( x′ ) −1 − 1 ) . ( 11 ) Hence the proof is completed . The theorem above demonstrates the following favorable properties of our method : • The discriminator score ratio is the same as αMH ( x′ , xk ) , but MH-GAN is restricted to a specific independent proposal . Our method instead works for any latent proposal q ( z′|zk ) . When we take q ( z′|zk ) = p0 ( z′ ) , our method reduces to MH-GAN . • Compared to αDEP ( x′ , xk ) of a general dependent proposal ( Eqn . ( 7 ) ) , the unknown data distributions terms are successfully cancelled in the reparameterized acceptance criterion . • The reparameterized MH acceptance criterion becomes tractable as it only involves the latent priors , the latent proposal distributions , and the discriminator scores . Combining the REP proposal qREP ( x′|xk ) and its tractable MH criterion αREP ( x′ , xk ) , we have developed a novel sampling method for GANs , coined as REP-GAN . See Appendix 1 for a detailed description . Moreover , our method can serve as a general approximate inference technique for Bayesian models by bridging MCMC and GANs . Previous works ( Marzouk et al. , 2016 ; Titsias , 2017 ; Hoffman et al. , 2019 ) also propose to avoid the bad geometry of a complex probability measure by reparameterizing the Markov transitions into a simpler measure . However , these methods are limited to explicit invertible mappings without dimensionality reduction . In our work , we first show that it is also tractable to conduct such model-based reparameterization with implicit models like GANs .

Summary: The paper proposes an MCMC based sampling mechanism for GANs. In contrast to earlier work, the proposal distribution is conditioning conditioned on the previous state (here in latent space), which is supposed to help sampling efficiency. This is achieved by a clever re-parametrization of intermediate steps of the MCMC chain. As an example, the authors provide a Langevin version (which uses gradient information) of their method.