Generative Ratio Matching Networks

1 INTRODUCTION . Deep generative models ( Kingma & Welling , 2013 ; Goodfellow et al. , 2014 ; Kingma & Dhariwal , 2018 ) have been shown to learn to generate realistic-looking images . These methods train a deep neural network , called a generator , to transform samples from a noise distribution to samples from the data distribution . Most methods use adversarial learning ( Goodfellow et al. , 2014 ) , in which the generator is pitted against a critic function , also called a discriminator , which is trained to distinguish between the samples from the data distribution and from the generator . Upon successful training the two sets of samples become indistinguishable with respect to the critic . Maximum mean discrepancy networks ( MMD-nets ) ( Li et al. , 2015 ; Dziugaite et al. , 2015 ) are a class of generative models that are trained to minimize the MMD ( Gretton et al. , 2012 ) between the true data distribution and the model distribution . MMD-nets are similar in spirit to generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ; Nowozin et al. , 2016 ) , in the sense that the MMD is defined by maximizing over a class of critic functions . However , in contrast to GANs , where finding the right balance between generator and critic is difficult , training is simpler for MMD-nets because using the kernel trick the MMD can be estimated without the need to numerically optimize over the critic function . This avoids the need in GANs to numerically solve a saddlepoint problem . Unfortunately , although MMD-nets work well on low dimensional data , these networks have not on their own matched the generative performance of adversarial methods on higher dimensional datasets , such as natural images ( Dziugaite et al. , 2015 ) . Several authors ( Li et al. , 2017 ; Bińkowski et al. , 2018a ) suggest that a reason is that MMD is sensitive to the choice of kernel . Li et al . ( 2017 ) propose a method called MMD-GAN , in which the critic maps the samples from the generator and the data into a lower-dimensional representation , and MMD is applied in this transformed space . This ∗Equal contributions . †Additional affiliations : The University of Edinburgh and IBM Research . ‡Additional affiliations : The University of Edinburgh and The Alan Turing Institute 1Official implementations are available at https : //github.com/GRAM-nets . can be interpreted as a method for learning the kernel in MMD . The critic is learned adversarially by maximizing the MMD at the same time as it is minimized with respect to the generator . This is much more effective than MMD-nets , but training MMD-GANs can be challenging , because in this saddlepoint optimization problem , the need to balance training of the learned kernel and the generator can create a sensitivity to hyperparameter settings . In practice , it is necessary to introduce several additional penalties to the loss function in order for training to be effective . In this work , we present a novel training method that builds on MMD-nets ’ insight to use kernels as fixed adversaries in order to avoid saddlepoint optimization based training for the critic and the generator . Our goal is for the critic to map the samples into a lower-dimensional space in which the MMD-net estimator will be more effective . Our proposal is that the critic should be trained to preserve density ratios , namely , the ratio of the true density to the model density . If the critic is successful in this , then matching the generator to the true data in the lower dimensional space will also match the distributions in the original space . We call networks that have been trained using this criterion generative ratio matching ( GRAM ) networks , or GRAM-nets2 . We show empirically that our method is not only able to generate high quality images but by virtue of avoiding a zero-sum game ( in critic and generator ) it avoids saddlepoint optimization and hence is more stable to train and robust to the choice of hyperparameters . 1.1 RELATED WORK . Li et al . ( 2015 ) and Dziugaite et al . ( 2015 ) independently proposed MMD-nets , which use the MMD criterion to train a deep generative model . Unlike f -divergences , MMD is well defined even for distributions that do not have overlapping support , which is an important consideration for training generative models ( Arjovsky et al. , 2017 ) . Therefore , MMD-nets use equation ( 2 ) in order to minimize the discrepancy between the distributions qx and px with respect to Gγ . However , the sample quality of MMD-nets generally degrades for higher dimensional or color image datasets ( Li et al. , 2015 ) . To address this problem , Li et al . ( 2017 ) introduce MMD-GANs , which use a critic fθ : RD 7→ RK to map the samples to a lower dimensional space RK , and train the generator to minimize MMD in this reduced space . This can be interpreted as learning the kernel function for MMD , because if fθ is injective and k0 is a kernel in RK , then k ( x , x′ ) = k0 ( fθ ( x ) , fθ ( x′ ) ) is a kernel in RD . This injectivity constraint on fθ is imposed by introducing another deep neural network f ′φ , which is trained to approximately invert fθ using an auto-encoding penalty . Though it has been shown before that inverting fθ is not necessary for the method to work . We also confirm this in experiments ( See appendix ) . The critic fθ is trained using an adversarial criterion ( maximizing equation ( 2 ) that the generator minimizes ) , which requires numerical saddlepoint optimization , and avoiding this was one of the main attractions of MMD in the first place . Due to this , successfully training fθ in practice required a penalty term called feasible set reduction on the class of functions that fθ can learn to represent . Furthermore , f is restricted to be k-Lipschitz continuous by using a low learning rate and explicitly clipping the gradients during update steps of f akin to WGAN ( Arjovsky et al. , 2017 ) . Recently , Bińkowski et al . ( 2018b ) ; Li et al . ( 2019 ) have proposed improvements to stabilize the training of the MMD-GAN method . But these method still rely on solving the same saddle-point problem to train the critic . 2 BACKGROUND . Given data xi ∈ RD for i ∈ { 1 . . . N } from a distribution of interest with density px , the goal of deep generative modeling is to learn a parameterized function Gγ : Rh 7→ RD , called a generator network , that maps samples z ∈ Rh where h < D from a noise distribution pz to samples from the model distribution . Since Gγ defines a new random variable , we denote its density function by qx , and also write xq = Gγ ( z ) , where we suppress the dependency of xq on γ . The parameters γ of the generator are chosen to minimize a loss criterion which encourages qx to match px . 2Interestingly , the training of GRAM-nets heavily relies on the use of kernel Gram matrices . 2.1 MAXIMUM MEAN DISCREPANCY . Maximum mean discrepancy measures the discrepancy between two distributions as the maximum difference between the expectations of a class of functions F , that is , MMDF ( p , q ) = sup f∈F ( Ep [ f ( x ) ] − Eq [ f ( x ) ] ) , ( 1 ) where E denotes expectation . If F is chosen to be a rich enough class , then MMD ( p , q ) = 0 implies that p = q. Gretton et al . ( 2012 ) show that it is sufficient to choose F to be a unit ball in an reproducing kernel Hilbert spaceR with a characteristic kernel k. Given samples x1 . . . xN ∼ p and yi . . . yM ∼ q , we can estimate MMDR as ˆMMD 2 R ( p , q ) = 1 N2 N∑ i=1 N∑ i′=1 k ( xi , xi′ ) − 2 NM N∑ i=1 M∑ j=1 k ( xi , yj ) + 1 M2 M∑ j=1 M∑ j′=1 k ( yj , yj′ ) . ( 2 ) 2.2 DENSITY RATIO ESTIMATION . Sugiyama et al . ( 2012 ) present an elegant MMD-based estimator for the ratio between the densities p and q ; r ( x ) = p ( x ) q ( x ) that only needs samples from p and q . This estimator is derived by optimizing min r∈R ∥∥∥∥∫ k ( x ; . ) p ( x ) dx− ∫ k ( x ; . ) r ( x ) q ( x ) dx∥∥∥∥2 R , ( 3 ) where k is a kernel function . It is easy to see that at the minimum , we have r = p/q . This estimator is not guaranteed to be non-negative . As such , a positivity constraint should be imposed if needed . Sugiyama et al . ( 2011 ) suggest that density ratio estimation for distributions p and q over RD can be more accurately done in lower dimensional sub-spaces RK . They propose to first learn a linear projection to a lower dimensional space by maximizing an f -divergence between the distributions p̄ and q̄ of the projected data and then estimate the ratio of p̄ and q̄ ( using direct density ratio estimation ) . They showed that the projected distributions preserve the original density ratio . 3 METHOD . Our aim will be to enjoy the advantages of MMD-nets , but to improve their sample quality by mapping the data ( RD ) into a lower-dimensional space ( RK ) , using a critic network fθ , before computing the MMD criterion . Because MMD with a fixed kernel performs well for lower-dimensional data ( Li et al. , 2015 ; Dziugaite et al. , 2015 ) , we hope that by choosing K < D , we will improve the performance of the MMD-net . Instead of training fθ using an adversarial criterion like MMD-GAN , we aim at a stable training method that avoids the saddle-point optimization for training the critic . More specifically , unlike the MMD-GAN type methods , instead of maximizing the same MMD criterion that the generator is trained to minimize , we train fθ to minimize the squared ratio difference , that is , the difference between density ratios in the original space and in the low-dimensional space induced by fθ ( Section 3.1 ) . More specifically , let q̄ be the density of the transformed simulated data , i.e. , the density of the random variable fθ ( Gγ ( z ) ) , where z ∼ pz . Similarly let p̄ be the density of the transformed data , i.e. , the density of the random variable fθ ( x ) . The squared ratio difference is minimized when θ is such that px/qx equals p̄/q̄ . The motivation is that if density ratios are preserved by fθ , then matching the generator to the data in the transformed space will also match it in the original space ( Section 3.2 ) . The reduced dimension of fθ should be chosen to strike a trade-off between dimensionality reduction and ability to approximate the ratio . If the data lie on a lower-dimensional manifold in RD , which is the case for e.g . natural images , then it is reasonable to suppose that we can find a critic that strikes a good trade-off . To compute this criterion , we need to estimate density ratio p̄/q̄ , which can be done in closed form using MMD ( Section 2.2 ) . The generator is trained as an MMD-net to match the transformed data { fθ ( xi ) } with transformed outputs from the generator { f ( Gγ ( zi ) } in the lower dimensional space ( Section 3.2 ) . Our method performs stochastic gradient ( SG ) optimization on the critic and the generator jointly ( Section 3.3 ) .

Summary: This paper proposes GRAM-nets using MMD as the critic to train GANs. Similar to MMD-GAN, the MMD is computed on a projected lower dimensional space to prevent the kernel struggle in the high dimensional observed space. On the other hand, contrary to MMD-GAN, GRAM-nets trains the projection f_{\theta} trying to matching the density ratio of p/q between the observed and the latent space. The proposed density ratio matching criterion avoids the adversarial training in MMD-GAN, thus can potentially fix the two-player optimization problem. The paper shows improved FID scores and nice-looking CIFAR10 generations.