Refining Deep Generative Models via Discriminator Gradient Flow
Authors: Abdul Fatir Ansari
Ming Liang Ang
Harold Soh
Venue: ICLR 2021 Poster (2021)
Keywords: gradient flows
generative models
GAN
VAE
Normalizing Flow

Abstract: Deep generative modeling has seen impressive advances in recent years, to the point where it is now commonplace to see simulated samples (e.g., images) that closely resemble real-world data. However, generation quality is generally inconsistent for any given model and can vary dramatically between samples. We introduce Discriminator Gradient $f$low (DG$f$low), a new technique that improves generated samples via the gradient flow of entropy-regularized $f$-divergences between the real and the generated data distributions. The gradient flow takes the form of a non-linear Fokker-Plank equation, which can be easily simulated by sampling from the equivalent McKean-Vlasov process. By refining inferior samples, our technique avoids wasteful sample rejection used by previous methods (DRS & MH-GAN). Compared to existing works that focus on specific GAN variants, we show our refinement approach can be applied to GANs with vector-valued critics and even other deep generative models such as VAEs and Normalizing Flows. Empirical results on multiple synthetic, image, and text datasets demonstrate that DG$f$low leads to significant improvement in the quality of generated samples for a variety of generative models, outperforming the state-of-the-art Discriminator Optimal Transport (DOT) and Discriminator Driven Latent Sampling (DDLS) methods.

Published as a conference paper at ICLR 2021
REFINING DEEP GENERATIVE MODELS VIA DISCRIMINATOR GRADIENT FLOW
Abdul Fatir Ansari, Ming Liang Ang & Harold Soh
Department of Computer Science, School of Computing
National University of Singapore (abdulfatir, ongmingliang @uu nus. nus, harold@comp nus. edu.
ABSTRACT
Deep generative modeling has seen impressive advances in recent years, to the point where is now commonplace to see simulated samples (e.g.. images) that closely resemble real-world data. However, generation quality is generally in- consistent for any given model and can vary dramatically between samples. We introduce Discriminator Gradient flow (DGflow), new technique that improves generated samples via the gradient flow of entropy-regularic tween the real and the generated data distributions. The gradient flow takes the form of a non-linear Fokker-Plank equation, which can be easily simulated by sampling from the equivalent McKean- Vlasov process. By refining inferior sam- ples, our technique avoids wasteful sample rejection used by previous methods (DRS & MH-GAN). Compared to existing works that focus on specific GAN vari- ants, we show our refinement approach can be applied to GANs with vector-valued crities and even other deep generative models such as VAEs and Normalizing Flows. Empirical results strate that DG flow leads to significant improvement in the quality of generated samples for variety of generative models, outperforming the state-of-the-ar Dis- criminator Optimal Transport (DOT) and Discriminator Driven Latent Sampling (DDLS) methods.
f-divergences be-
multiple synthetic, image, and text datasets demon-
INTRODUCTION
Deep generative models (DGMs) have excelled at numerous tasks, from generating realistic im- ages (Brock al. 2019) learning policies reinforcement learning (Ho & Ermon 2016). Among the variety of proposed DGMs, Generative Adversarial Networks (GANs) (Goodiellow et al. 2014) have received widespread popularity for their ability to generate high quality samples that resem- ble real data. Unlike Variational Autoencoders (VAEs) (Kingma & Welling 2014) and Normal- izing Flows (Rezende & Mohamed, 2015) Kingma Dhariwal 2018), GANs are likelihood-Fii methods; training is formulated as minimax optimization problem involving generator and a dis- criminator. The generator seeks to generate samples that are similar to the real data by minimizing measure of discrepancy (between the generated samples and real samples) furnished by the dis- criminator. The discriminator trained to distinguish the generated samples from the real samples. Once trained, the generator is used simulate samples and the discriminator has traditionally been discarded.
However, recent work has shown that discarding the discriminator is wasteful - it actually contains useful information about the underlying data distribution. This insight has led to sample improve- techniques that use this information improve the quality of generated samples (Azadi al.)
2019 Tumerr et al. 2019 Tanaka 2019 Che et al.) 2020). Unfortunately, current methods either
rely on wasteful rejection operations in the data space (Azadi et al.] [2019] Turner al., 2019), or require sensitive diffusion term to ensure sample diversity (Che et al. 2020). Prior work has also focused on improving GANs with scalar-valued discriminators, which excludes a large family of GANs with rector-valued critics, e.g., MMDGAN (Li et al., 2017]; Binkkwwsi et al. 2018) and
OCFGAN AAnari et al. 2020), and likelihood-bast based generative models.

---

Published as a conference paper at ICLR 2021
In this work, we propose Discriminator Gradient flow (DGflow) which formulates sample improvement as re- fining inferior samples using the gradient flow of f- divergences between the generator and the real data dis- tributions (Fig. T flG avoids wasteful rejec- tion operations and can be used in a deterministic set- ting without diffusion term. Existing state-of-the-art methods specifically, Discriminator Optimal Transport (DOT) Tanaka 2019) and Discriminator Driven Latent Sampling (DDLS) (Che et al., 2020) can be viewed as special cases of DG Similar to DDLS, DGJllow re- covers the real data distribution when the gradient flow is simulated exactly.
P2
FI
po
pi
D2
PN
PN
2
2
B
We further present generalized framework that employs existing pre-trained discriminators to refine samples from variety of deep generative models: we demonstrate our method can be applied to GANs with vector-valued critics, and even likelihood-based models such as VAEs and Normalizing Flows. Empirical results on synthetic datasets, and benchmark image (CIFARIO, STL10) and text (Billion Words) datasets demonstrate that our gradi- ent flow-based approach outperforms DOT and DDLS on multiple quantitative evaluation metrics.
N
2 -1
21+1 Nx, (e
Figure I: An illustration of refinement us- ing DG flow, with the gradient flow in the 2- Wassersttin space Pa (top) and the corre- sponding discretized SDE in latent space 2 (bottom). The image samples from the densities along the gradient flow are shown in the middle.
In summary, this paper's key contributions are:
DG flo,, a method to refine deep generative models using the gradient flow of f-divergences;
a framework that extends flow to GANs with vector-valued critics, VAEs, and Nor- malizing Flows;
experiments on a variety of generative models trained on synthetic, image (CIFAR10 & STL10), and text (Billion Words) datasets demonstrating that D is effective in im- proving samples from generative models.
BACKGROUND: GRADIENT FLOWS
The following gives brief introduction gradient flows; refer readers the excellent overview by Santambrogio (001)) for a more thorough introduction.
Let (X, || (12) be Euclidean space and F X + R be a smooth energy function. The gradient flow of F is the smooth curve JXER that follows the direction of steepest descent, i.e.,
x x'(1) (x(()))).
The value of the energy F is minimized along this curve. This idea of steepest descent curves can be characterized in arbitrary metric spaces via the minimizing movement scheme (Jordan et al. 1998). Of particular interest is the metric space of probability measures that is endowed with the Wasser- stein distance (Wp.): the Wasserstein distance
metric and the Wp topology satisfies weak con- vergence of probability measures (Villani 2008 Theorem 6.9). Gradient flows in the 2-Wasserstein space (P2)),, W2) - i.e., the space of probability measures with finite second moments and the 2- 2-Wasserstein metric have been studied extensively. Let {Pt)ER, be the gradient flow of functional F in the 2- Wasserstein space, where Pe is absolutely continuous with respect to the Lebesgue measure. The curve {pthER satisfies the continuity equation (Ambrosio et al. 2008 Theorem 8.3.1),
0.p: + V (pev:) = 0.
The velocity field v, in Eq. is given by
8F v((x) V
8p (p),
(3)
2

---

Published as a conference paper at ICLR 2021
where 6. denotes the first variation of the functional F. 6p
Since the seminal work of Jordan et al. (9988) that showed that the Fokker-Plank equation is the gradient flow of a particular functional in the Wasserstein space, gradient flows in the Wasserstein metric have been a popular tool in the analysis of partial differential equations (PDEs). For ample, they have been applied to the study of the porous-ma porous-medium equation (Otto) 2001), crowd modeling (Maury al. 2010 2011), and mean-field games (Almulla et al. 2017). More recently, gradient flows of various distances used in deep generative modeling literature have been proposed, notably that of the sliced Wasserstein distance (Liutkus et al., 2019), the maximum mean discrep- ancy (Arbel ct al. (019)), the Stein discrepancy (Liu (017), and the Sobolev discrepancy (Mrouch al, 2019). Gradient flows have also been used for learning con-parametric and parametric implicit generative models (Liutkus et al. 2019 Gao et al. 2019 2020). As an example of the latter, Varia- tional Gradient Flow (Gao et al. 2019) learns mapping between latent vectors and samples evolved using the gradient flow of /-divergences. In this work, we present method using gradient flows of entropy mntropy-regularized f-divergences for refining samples from deep generative models employing existing discriminators as density-ratio estimators.
GENERATOR REFINEMENT VIA DISCRIMINATOR GRADIENT FLOW
This section describes our main contribution: Discriminator Gradient flow (DGflow). As an overview, we begin with the construction of the gradient flow of catropy-regularizee f- f-divergences and describe its application to sample refinement. We then discuss how to simulate the gradient flow in the latent space of the generator Finally, we present simple technique that extends our method to generative models that have not yet been studied in the context of refinement. Due to space constraints, we focus on conveying the key concepts and relegate details (e.g., proofs) to the appendix.
procedure more suitable for high-dimensional datasets.
The antropy-regularizee f-divergence functional is defined as
F((e) 2 Dy(ullp) - ->H(p),
(4)
where the f-divergence term (u(l()) ensures that the "distance" between the probability density p and the target density μ decreases along the gradient flow. The differential entropy term H(p) improves diversity and expressiveness when the gradient flow is simulated for finite time-steps. We now construct the gradient flow of Ff-
Lemma 3.1. Define the functional F/ P2( - R as
FL(p) & / [((xx//((x)) u(x)dx + / p(x) log p(x)dxx
(5)
f-divergence
negative entropy
where I is twice-differentaable convex function with f(1) O. The gradient flow of the functional Fd. (p) in the Wasserstein space (P2())), W₂) is given by the following PDE,
Oepe(x) Vx Vxf (o(xx/((x))) -xxxp(( = O,
(6)
where Vx and Axx denote the divergence and the Laplace operators respectively.
The proof is given in Appendix A.I FPES have been studied extensively in the literature of stochastic processes and have Stochastic Differential Equation (SDE) counterpart (Risken) 1996). In the case of Eq. (6). the equivalent SDE is given by
The PDE in Eq. (6) is type of Fokker-Plant equation (FPE).
dx, V V (pt/p) (xt)dt + 22idwe,
diffission
where dw, denotes the standard Wiener process. Eq. defines the evolution particle x, under the influence of drift and diffusion. Specifically, it is McKean- process (Braun & Hepp 1977) which a type of non-linear stochastic process as the drift term at any time & depends on the distribution Pt of the particle x4. Eqs. (6) and (7) are equivalent in the sense that the distribution of the particle x, in Eq. (7) solves the PDE in Eq. (6). Consequently, samples from the density P along the gradient flow can be obtained by first drawing samples Xo Po and then simulating the
3

---

Published as a conference paper at ICLR 2021
SDE in Eq. The SDE can be approximately simulated via the stochastic Euler scheme (also known as the Euler-Mann Malle-ma method) (Beyn & Kruse 2011) given by
11 Xr, Vxl" (Pr,/u) (xx.. + 2
(8)
where& &T. ~ (O, I), the time interval [0, I3 is partitioned into equal intervals of size and To < Ti TN denote the discretized time-steps.
Eq. (8) provides non-parametric procedure to refine samples from a generator where we let be the density of real samples and Pro the density of samples generated from go obtained by first sampling from the prior latent distribution ~ Pz (z) and then feeding z into go- We first generate particles X Pro and then update the particles using Eq. (8) for N time steps.
Given a binary classifier (discriminator) D that has been trained to distinguish between samples from y and (ry. the density-ratio Pro (r)xx/(((x) can be estimated via the well-known density-ratio trick (Sugiyama et al. 2012).
D( = 1/x) exp(-d(x))
Pro (x) = D 1xx
where Dyy = 1 denotes the conditional probability of the sample x being from μ and d(x) denotes the logit output of the classifier D. We term this procedure where samples are refined via gradient flow of f-divergences as Discriminator Gradient flow (DGflow).
3.1 REFINEMENT IN THE LATENT SPACE
Eq. (8) requires a running estimate of the density-ratio Pr. (x)/ which can be approximated using the stale estimate Pr, (x)/p(x) Pry (x) (x)/p(x) for 0 and small N, where the density Pr. will be close to Pr. However, our initial image experiments showed that refining directly in high-dimensional data-spaces with the stale estimate is problematic; error is accumulated each time-step leading to visible degradation in the quality of data samples (e.g., appearance of artifacts in images).
To tackle this problem, we propose refining the latent vectors before mapping them to samples in data-space using Ge. We describe a procedure analogous Eq. (8) but in the latent space for generators go that take a latent vector Z as input and generate a sample E X. We first show Lemma 3.2 that the density-ratio in the latent space between two distributions can be estimated via the density-ratio of corresponding distributions in the data space.
Lemma 3.2. Let 2 + X be sufficiently well-behaue behavee injective function where Z C R" and X 18.T with > n. Let (z)z,, P2 (2) be probability densities on Z and qx (x), 9 (x) be the den- sities of the pushforward measures 2 gtΩ respectively. Assume that (z(2) and p2 (2) have same support, and the Jacobian matrix Jg has full column rank. Then, the density-ratio p2 (u) (u)/pz(Un p////) at the point u E Z is given by
Pz (=(u) Pz (1)
(g(( (x()(n))
(10)
The proof in Appendix A.2 We let P2 (2) be the density of the "correct" latent space distribution induced by a generator go. i.e., P2 (z) is the density of probabil- ity measure whose pushforward under ge approximately equals the target data den- sity p. The density-ratio of the prior latent distribution (z(z) and P²(2) can now be computed by combining Lemma 3.2 with Eq. (9)
Algorithm 1 Refinement in the Latent Space using DG flow.
Require:
First derivative of f (f'), generator (go), dis- criminator (do), number of update steps (N), step- size (1)), noise factor (7).
1: Zu pz pz(z).
Sample from the prior.
2: for i < 0, N do
3: &: (0,
4: 2,+1 2, V (leeeeo(oolzxx))
do(oo(z)))) +
2
Pz (u (90 exp(- ((((((((()))
5: end for
return go (z..)
The refined sample.
Pu) m(go(u))
(11)
4

---

Published as a conference paper at ICLR 2021
Although generator ge parameterized by neural network may not satisfy the conditions of in- jectivity and full column rank Jacobian matrix J9.- Eq. (II) provides an approximation that works well in practice shown by our experiments. Combining Eq. (TI) with Eq. ( provides with an update rule for refining samples in the latent space,
U Ur Vvs' (Pu.,/ P22) (ur,,) V
(12)
where Uno (z) and the density-ratio Pu.,, /P2 is approximated using the stale estimate Pu-g/ /Pz = exp((d(go(u)) We summarize the complete algorithm in Algorithm
3.2 REFINEMENT FOR ALL.
Thus far, prior work (Azadi et al. 2019 Turner et al. 2019 Tanaka 2019 Che et al., 2020) has
focused on improving samples for GANs with scalar-va valued discriminators, which comprises the canonical GAN as well as recent variants, e.g., WGAN (Gulrajani et al. al. (017). and SNGAN (Miyato) et al. 2018). Here, we propose technique that extends our approach to refine samples from larger class of DGMs including GANs with vector-valued critics, VAEs, and Normalizing Flows.
Let Po be the density of the samples generated by generator go and y be the density of the real data distribution. We are interested in refining samples from ge: however, a corresponding density-ratio estimator for pa/ μ is unavailable, as is the case with the aforementioned generative models.
Let Do be a discriminator that has been trained on the same dataset but for a different generative model Js (e.g., let Is and D& be the generator and discriminator of SNGAN respectively). D& can be used to compute the density ratio Po/ μ.. A straightforward technique would be to use the crude approximation Po/ μ Po/ μ.. which could work provided pa and Pa are not too far from each other. Our experiments show that this simple approximation works to a limited extent (see appendix
To improve upon the crude approximation above, propose to correct the density-ratin estimate. Specifically, a discriminator D2 is initialized with the weights from Dg and is fine-tuned on samples from Oo and go. Do and Dx are then used approximate the density-ratio pa p.
(x)
p (x) (x xxx) P (x(
exp(- exp((dx(x)), (x)
(13)
M
where dæ and d, are logits output from D3 and Dx, respectively. We term the network Dy the density ratio corrector, which experiments show produces higher quality samples than using po/ Ro ps/p. The estimate in Eq. (13) is similar to telescoping density-ratio estimation (TRE), technique proposed in very recent independent work (Rhodes et al. 2020). In brief, [Rhodes et al. (2020) show that classifier-based density ratio estimators perform poorly when distributions are "too far apart"; the classifier can easily distinguish between the distributions, even with a poor estimate of the density ratio. TRE expands the standard density ratio into a telescoping product of more difficult- to-distinguish intermediate density ratios. Likewise, in Fq. 13 we treat ps as an intermediate distribution and estimate the final density-ratio as product of two density-ratios.
4 RELATED WORK
Azadi et al. (2019) first proposed the idea of improving samples from a GAN's generator by discrim- inator rejection sampling (DRS), making use of the density-ratio provided by the discriminator to estimate the acceptance probability. Metropolis- Hastiinss GAN MMH-GAN) (Turner al.) 2019) im- proved upon the costly rejection sampling procedure via the Metropolis- HasliisssHtingssss algorithm. Unlike DG/low, both of these methods reject inferior samples instead of refining them.
Our method is closely related to recent state-of-the-an -fte----ttt--tttt sample refinement techniques, specifically Discriminator-Drrvenn Latent Sampling (DDLS) (Che et al. (020) and Discriminator Optimal Trans- port (DOT) (Tanaka 2019). In fact, both these methods can be seen special cases of flow.
DDLS treats a GAN as an energy-based model and uses Langevin dynamics to sample from the energy-based latent distribution pr pz (z) exp(d(gn (z)) induced by performing rejection sam- pling in the latent space. This distribution is the same as p2 which can be seen by rearranging terms in Eq. [T]. If we use the KL-divergence by setting f loggr DG is equivalent to DDLS. However, there are practical differences that make DG DG flow more appealing. DDLS requires estimation of the score function V, (log pz (z) + d(go (z))(z))) perform the update which becomes
5

---

Published as a conference paper at ICLR 2021
undefined if escapes the support of (z),,, e.g., in the case of the uniform prior distribution com- monly used in GANs; handling such cases would require techniques such as projected gradient descent. This problem does not arise in the case of flow since it only uses the density-ratio that is implicitly defined by the discriminator. Moreover, DDLS uses Langevin dynamics which requires the sensitive diffusion term to ensure diversity and to prevent points from collapsing the siiimmum-ikkelhhodd point. In flow, the sample diversity is ensured by the density-ratio term and the diffusion term serves as an enhancement. Note that flow performs well even without the diffusion term (i.e., with y 0, see Tables 13 & 14 in the appendix). This deterministic variant of DGflow a practical alternative with one less hyperparameter to tune.
DOT refines samples by constructing an Optimal Transport (OT) map induced by the WGAN dis- criminator. The OT map realized by means of a deterministic optimization problem in the vicinity of the generated samples. If we further analyze the case of DG DGflow flow with 0 and solve the resulting ordinary differential equation (ODE) using the backward Euler method,
Ur, arg min uER"
{ [ (pu... /P2) (u) +
1 2A
u7 112
(14)
DOT emerges as special case when we consider single update step of Eq. 14 using gradient
descent and set (t() = log(t]) with À = 2.
1 This connection of flow to DOT, an optimal
transport technique, is perhaps unsurprising given the relationship between gradient flows and the dynamical Benamou-Brenier formulation of optimal transport (Santambrogio (017).
Recent work has also sought improve generative models via sample evolution in the train- ing/generation process. In energy-based generative models (Arbel et al. 2021: Deng et al.) 2020). the energy functions can be viewed as a component that improves some base generator. For example, the Generalized Energy-Based Model (GEBM) (Arbel et al. 202T) jointly trains a base generator by minimizing a lower bound of the KL divergence along with an energy function in an alternating fashion. Once trained, the energy function is used to refine samples from the base generator using Langevin dynamics and serves a similar purpose to the discriminator
DDLS and DG flow. The Noise Conditional Score Network (NCSN) (Song & Ermon 2019 2020) a score-based genera- tive model can be seen as a gradient flow that refines sample right from noise to data. Latent Optimization GAN (LOGAN) (Wu et al., 2020) optimizes latent vector via natural gradient de- scent as part of the GAN training process. In contrast to these works, we primarily focus on refining samples from pretrained generative models using the gradient flow of f-divergences
5 EXPERIMENTS
In this section, we present empirical results on various deep generative models trained on multiple synthetic and real world datasets. Our primary goals were to determine if (a) fl is effective in improving the quality of samples from generative models, (b) the proposed extension to other generative models improves their sample quality, and (c) DG is generalizable to different types of data and metrics. Note that we did not seek to achieve state-of-the-an results for the datasets studied but to demonstrate that fl is able to significantly improve samples from the bare generators for different models.
We experimented with three f-divergences, namely the Kullback Lulbbec (KL) divergence, the Jensen-Shannon (IS) divergence, and the log D divergence (Gao et al.) 2019). The specific forms of the functions/ f' and corresponding derivatives are tabulated in Table 7 (appendix). We com- pare Dl/low with two -ate--of-hh--att competing methods: DOT and DDLSS In this section we discuss the main results and relegate details to the appendix. Our code is available online at https://github com/ www.releearnn/FFFI
5.1 2D DATASETS
We first tested DG flo on two synthetic datasets, 25Gaussians and 2DSwissroll, to visually inspect the improvement in the quality of generated samples. We generated 5000 samples from a trained WGAN-GP generator and refined them using DOT, DDLS, and DG flow. We performed refinement in the latent space for DDLS and directly in the data-space for DOT and fl Fig. 2 shows
'This implics that ft) = t flg t- - t ++1 1, which is lwice-differentiable convex function with f(0) = 1.
For further discussion about these techniques, please refer to appendix C

---

Published as a conference paper at ICLR 2021
Real Sample
GAN
DOT
DOLB
DGllow
Figure 2: Qualitative comparison of DG Dl/wowkr)) with DOT and DDLS on synthetic 2D datasets.
the samples generated from the WGAN-GP generator (blue) and the refined samples using different techniques (red) against the real samples from the training dataset (brown). Although the WGAN- GP generator learned the overall structure of the dataset, it also learned a number spurious modes. DOT is able to refine the spurious samples but to limited degree. In contrast, DDLS and DG flow are able to correct almost all spurious samples and are able to recover the correct structure of the data. Visualizations for DG flow with different f-divergences can be found in the appendix (Fig. 4).
We also compared the different methods quan- titatively on two metries: % high quality samples and kernel density estimate (KDE)
Table 1: Quantitative comparison on the 25Gaussians dataset. Higher scores better.
score. A sample is classified as high qual- ity sample if lies within standard de- viations of its nearest Gaussian. The KDE score computed by first estimating the KDE using generated samples and then comput- ing the log-likelihood of the training sam- ples under the KDE estimate. We computed both the metrics 10 times using 5000 sam-
High Quality
KDE Score
26.5
DOT
69.8
.4449
DDLS
89.3 +
-9977
17
DG flowaa.
89.5 1 .4
-2893
07
flowws))
DG fooloooo D)
84.5 + 3
-3036
ples and report the mean in Table [ The quantitative metries reinforce the qualitative anal-
ysis and show that DDLS and DG/low significantly improve the samples from the gen- erator, with DG performing slightly better than DDLS in terms of the KDE score.
5.2 IMAGE EXPERIMENTS
We conducted experiments on the CIFARIO and STL10 datasets to demonstrate the efficacy of flflow
in the real-world setting. We fol- lowed the setup of Tanaka for our image experiments.
(2019)
We
used the Fréchet Inception Distance (FID) (Heusel et al.] (2017) and In- ception Score (IS) (Salimans et al. 2016) metrics to evaluate the quality of generated samples before and after refinement. A high value of IS and low value of FID corresponds to high quality samples, respectively.
We first applied DG to GANs with scalar-valued discriminators (e.g.. WGAN-GP, SNGAN) trained on the CIFAR10 and the STL10 datasets. Table that flol significantly improves the quality of
(a) CIFARIO
(b) STL1O
Figure 3: Improvement in the quality of samples generated from base model (leftmost columns) over the steps DG DGflow for SN ResNet-GAN and SN-DCGAN on the CIFARIO and STL10 datasets respectively.

---

Published as a conference paper at ICLR 2021
Table 2: Comparison of different variants of DG flow with DOT the CIFARIO and STL10 datasets. For BN-DCGAN, (hi) denotes the hinge loss and (ns) denotes the non-saturating loss. Lower scores are better, DG flow's results have been averaged over 5 random runs with the standard deviation in parentheses.
Fréchet Inception Distance
Model
Base Model
DOT
DGflowxu
DG flowas)
DG flooooo D)
WGAN-GP
28.37 (.08)
24.14
24.68 (.09)
23.15 (.07)
24.53 (.11)
STLI0 CIFARIO
SN-DCGAN (hi)
20.70 (.05)
17.12
15.68 (.07)
16,45 (.06)
17.36 (.05)
SN-DCGAN (ns)
20.90 (.11)
15.78
15.30 (.08)
15.90 (.11)
16.42 (.05)
SN-ResNet-GAN
14.10 (.06)
9.62 (.03)
9.79 (.02)
9.73 (.05)
WGAN-GP
51.50 (.15)
44.45
39.07 (.07)
50.83 (.06)
39.71 (.29)
SN-DCGAN (hi)
40.54 (.17)
34.85
34.95 (.06)
36,37 (.12)
36,56 (.08)
SN-DCGAN (ns)
41.86 (.12)
34.84
34.60 (.11)
35.37 (.12)
37.07 (.14)
Table 3: Inception scores of different generative models, DRS, MHI-GAN, DDLS, and flow the CI-
FARIO RR dataset. Higher scores are better.
Model
Inception Score
WGAN-GP Gulrajani et al.
2017
7.86 (.07)
ProgressiveGANN (Karras et al. 201
8.80 (.05)
SN-ResNet-GAN Miyato
2018
8.22 (.05)
NCSN (Song & Ermon 2019)
8.87 (.12)
DCGAN
2.88
DCGAN + DRS (cal) (Azadi et al.
2019
3.07
DCGAN + MH (cal) Turner el al.
2019
3.38
SN-ResNct-GAN (our evaluation)
8.38 (.03)
SN-ResNet-GAN + DDLS (cal) (Che et al.
2020
9.09 (.10)
SN-ResNet-GAN + (lowow
9.35 (.03)
BigGAN
9.22
the samples in terms of the FID score and outperforms DOT on multiple models. The corresponding values of the Inception score can be found in the Appendix (Table [T]. which shows DGflow outperforms DOT on all models. In Table 3] we reproduce previously reported IS results for generative models and other sample improvement methods (DRS, MH-GAN, and DDLS) for completeness. DG flow performs the best in terms of relative improvement from the base score and even outperforms the tfat--of-the-ar BigGAN (Brock et al.) 2019). conditional generative model, without the need for additional labels. Qualitatively, DG improves the vibrance of the samples and corrects deformations in the foreground object. Fig. 3 shows the change in the quality of samples when using DG where the leftmost columns show the image generated form the base models and the successive columns show the refined sample using fl over increments of 5 update steps.
We then evaluated the ability of Dl/low to refine samples from generative models without cor- responding discriminators, namely MMDGAN, OCFGAN-GP, VAES, and Normalizing Flows (Glow). We used the SN-DCGAN (ns) as the surrogate discriminator D& for these models and fine-tuned density ratio correctors D> for each model as described in section 332 Table the FID scores achieved by these models without and with refinement using DG DG/low. We obtain a clear improvement in quality of samples when these generative models are combined with flow.
5.3 CHARACTER-LEVER LANGUAGE MODELING
Finally, we conducted an experiment on the character-level language modeling task proposed by Gulrajani et al. (017)) to show that DG works different types of data. We trained character-leve GAN language model on the Billion Words Dataset (Chelba et al.; 2013), which was pre-processed into 32-character long strings. We evaluated the generated samples using the JS-4 and JS-6 scores which compute the Jensen-Shaee Shannon divergence between the 4-gram and 6-gram proba- bilities of the data generated by the model and the real data. (ab shows that DG f flow leads to
8

---

Published as a conference paper at ICLR 2021
Table 4: Comparison of different variants of flow applied to MMDGAN. OCFGAN-GR. VAE, and Glow models, Lower scores are better. Results have been averaged over random runs with the standard deviation parentheses.
Fréchet Inception Distance
Model
Base Model
DG (lowa))
DG (Gllowas
flolowoog
MMDGAN
41.98 (.12)
36.75 (.09)
38.06 (.14)
37.75 (.10)
CIFARIO
OCFGAN-GP
31.98 (.12)
26.89 (.06)
28.20 (.06)
27.82 (.09)
VAE
129.5 (.13)
116.0 (.21)
128.9 (.13)
115.2 (.06)
Glow
100.5 (.52)
79.02 (.23)
94.61 (.34)
81,12 (.35)
MMDGAN
47.20 (.07)
43.21 (.06)
46.74 (.05)
43.06 (.05)
SIL10
OCFGAN-GP
36.55 (.08)
31.12 (.13)
36.05 (.11)
30.61 (.14)
VAE
150.5 (.09)
130.1 (.18)
149.9 (.08)
132.5 (.28)
Table 5: Results of DG flow on a character-level GAN language model.
JS-4 and JS-6 scores. Lower scores hetter.
Model
JS-4
JS-6
WGAN-GP
0.224 (.0009)
0.574 (.0015)
DGf DllloW(L)
0.212 (.0008)
0.512 (.0012)
flowos
0.186 (.0007)
0.508 (.0011)
DG D)
0.209 (.0005)
0.506 (.0008)
(b) Examples of text samples refined by DG
Generated by WGAN-GP
Refined by DG
In Ruoduce that fhance would pol said thowe toot lind talker Now their rarning injurer hows Police report in B0sbu does off We gine jaid 121 one bub like years in 19mbisuch said he h
In product that chance could rol
said this stood line talked
Now their warning injurer shows
Police report inturner will befe
We gave wall said left out like
In year's in 1900b such said he h
an improvement the JS-4 and JS-6 scores. visibly improves the quality of generated text.
5bb shows example sentences where flow
CONCLUSION
In this paper, we proposed technique to improve samples from deep generative models by refining them using gradient flow of /-divergences between the real and the generator data distributions. We also presented a simple framework that extends the proposed technique to commonly used deep gen- crative models: GANs, VAEs, and Normalizing Flows. Experimental results indicate that gradient flows provide excellent alternative methodology refine generative models. Moving forward, we are considering several technical enhancements to improve DG flo''s performance. At present, DG/low uses stale estimate of the density-ratio, which could adversely affect sample evolution when the gradient flow is simulated for larger number of steps; how we can efficiently update this estimate is open question. Another related question is when the evolution of the samples should be stopped; rumning chains for too long may modify characteristics of the original sample (e.g., ori- entation and color) which may be undesirable. This issue does not just affect DGflow; a method for automatically stopping sample evolution could improve results across refinement techniques.
ACKNOWLEDGEMENTS
This research is supported by the National Research Foundation Singapore under its Al Singapore Programme (Award Number: AISG-RP-2019-01)) to H. Soh. Thank you to J. Scarlett for his comments regarding the proofs.

---

Published as a conference paper at ICLR 2021
REFERENCES
Noha Almulla, Rita Ferreira, and Diogo Gomes. Two numerical approaches to stationary mean-field games. Dynamic Games and Applications, 7(4), 2017.
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008.
Abdul Fatir Ansari, Jonathan Scarlett, and Harold Soh. A characteristic function approach to deep implicit generative modeling. In CVPR, 2020.
Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. In NeurIPS, 2019.
Michael Arbel, Liang Zhou, and Arthur Gretton. Generalized energy based models. In ICLR, 2021.
Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, and Augustus Odena. Discrim- inator rejection sampling. In ICLR, 2019.
Adi Ben-Isracl. The change-of-variable formula using matrix volume. SIAM Journal on Matrix Analysis and Applications, 21( 1999.
Wolf-Jurgen Beyn and Raphael Kruse. Numerical methods for stochastic processes. (in preparation), 2011.
Lecture Book
Mikotaj Binkowski, Dougal J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In ICLR, 2018.
Werner Braun and K Hepp. The Vlasov dynamics and its fluctuations in the 1/n limit of interacting classical particles. Communications in mathematical physics, 56(2), 1977.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In ICLR, 2019.
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, and Yoshua Bengio. Your GAN is secretly an energy-based model and you should use discriminator driven latent sampling. In NeurlPS, 2020.
Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint 13 2013.
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, 2011.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Mare' Aurelio Ranzato. Residual energy-based models for text generation. In ICLR, 2020.
Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang, and Shunkang Zhang. Deep generative learning via variational gradient flow. In ICML, 2019.
Yuan Gao, Jian Huang, Yuling Jiao, and Jin Liu. Learning implicit generative models with theoretical guarantees. arXiv preprint 2rXiv:2002.02862. 2020.
Mevlana C Gemici, Danilo Rezende, and Shakir Mohamed. Normalizing flows on Riemannian manifolds. arXiv preprint urXiv:161..02300 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurlPS, 2014.
Aditya Grover, Ramki Gummadi, Miguel Mazzar-Grredill, Dale Schuurmans, and Stefano Ermon. Variational rejection sampling. In AISTATS, 2018.
Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, and Stefano Ermon. Bias correction of learned generative models using likelihood free importance weighting. In NewrlPS, 2019.
10

---

Published as a conference paper at ICLR 2021
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training Wasserstein gans. In NeurlPS, 2017.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge local nash equilibrium. In NeurIPS, 2017.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In NeurlPS, 2016.
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the Fokker- Planck equation. SIAM journal mathematical analysis, 29(1), 1998.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint 1rXiv:1710.10196 2017.
Diederik P. Kingma and M. Welling.
Auto-encoding variational bayes. arXiv preprint
arXiv:1312.6114, 2014.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolutions. In NeurlPS, 2018.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMDGAN: Towards deeper understanding of moment matching network. In NeurlPS, 2017.
Qiang Liu. Stein variational gradient descent as gradient flow. In NeurIPS, 2017.
Antoine Liutkus. Umut Simsekli, Szymon Majewski, Alain Durmus, and Sabian-Robert Stöter. Sliced-Wassere Wasserstein flows: Nonparametric generative modeling via optimal transport and diffu- sions. In ICML, 2019.
Bertrand Maury, Aude Boudneff-Chuplie and Filippo Santambrogio. A macroscopic crowd motion model of gradient flow type. Mathematical Models and Methods Applied Sciences, 20(10), 2010.
Bertrand Maury, Aude Roudneff- Houddeff-Chhuinn, Filippo Santambrogio, and Juliette Venel. Handling con- gestion in crowd motion modeling. arXiv preprint arXiv: 2011.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.
Youssef Mroueh, Tom Sercu, and Anant Raj. Sobolev descent. In AISTATS, 2019.
Felix Otto. The geometry of dissipative evolution equations: the porous medium equation. 2001.
Danilo Jimenez Rezende and S. Mohamed. Variational inference with normalizing flows. In ICML, 2015.
Benjamin Rhodes, Kai Xu, and Michael U. Gutmann. Telescoping density-ratio estimation. In NeurIPS, 2020.
Hannes Risken. fokker-Planck equation. Springer, 1996.
Tim Salimans. lan Goodfellow, Wojciech Zaremba. Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. NeurIPS, 2016.
Filippo Santambrogio. (Euclidean, metric, and Wasserstein) gradient flows: an overview. Bulletin of Mathematical Sciences. 7(1), 2017.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NewrlPS, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score- score-based generative models. In NeurIPS, 2020.
11

---

Published as a conference paper at ICLR 2021
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
Akinori Tanaka. Discriminator optimal transport. In NeurIPS, 2019.
Ryan Turner, Jane Hung, Eric Frank, Yunus Saatchi, and Jason Yosinski. Metropolis -aastings generative adversarial networks. ICML, 2019.
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. LOGAN: Latent optimisation for generative adversarial networks. arXiv preprint arXiv: 1rXiv:191,.00953, 2020.
12

---

Published as a conference paper at ICLR 2021
A PROOFS
A.1
LEMMA
Proof. Gradient flows in the Wasserstein space are of the form of the continuity equation (see Am- brosto al. (2008), page 281), i.e,
Depe + V. (pev) 0.
(15)
The velocity field v in Eq. 15
is given by
8F v(x) = - Vx (p). 8p
(16)
where (p) 6p
denotes the first variation of the functional F. The first variation is defined
d Fll+
oF 8p (p)x,
(17)
de
c==0
where X - p for some v E P2()).
Let's derive expression for the the first variation of F. In the following, we drop the notation for dependence on x for clarity,
d F(x+ex)
= d.
f ( (etex
d
1 1+7 / (x) +
(18)
de
==0 de
1
c-
= = =s p (D+EX) x+* ( +xx (x0 + +1x
(19)
==0
p + log(p) +7/ x.
(20)
Substituting 6p
52 (p) in Eq. (6) we get,
v(x) Vx l (8) (og(()) +7
(21)
(4)
-xJ (9) -
(22)
14
Substituting v in Eq.
15
we get the gradient flow,
depr -x (DeV>e
P
(23)
11
0xpx(x) Vx Vx (D:Vas
((x)
Axxx(((x) =0.
(24)
[((x)
where Axx and xx denote the Laplace and the divergence operators respectively.
A.2 LEMMA 3.2
Proof. Let be an integrable function on X. If J,, has full column rank and g is an injective function, then we have the following vhariblss-variables equation (Ben-Israel 1999 Gemici et al. 2016).
J f(x)dx
- ())))(z) det JJJ
(25)
13

---

Published as a conference paper at ICLR 2021
This implies that the infinitesimal volumes dx and dz are related as dx = / /et. J and the densities pz (z) and 9x (x) are related as pz (z - (x(()z)) det J. Jg (z). Similarly, P2 (2(2)
ax (x(((2)) det J J.J. (2). Finally, the density-ratio p. (z(uu)/pzu)) at the point Z is given
(u) ax /el Ex
(26)
Pz(u) gx ex(e(u)), /detJJJJJu ex(g(u))
B A DISCUSSION ON DGfLOW FOR WGAN
We apply /l/low to WGAN models by treating the output from their critics as the logit for the estimation density-ratio. However, it is well-known that WGAN critics are not density-ratio estimators as they are trained to maximize the 1-Wasserstein distance with unconstrained out- put. this section, we provide theoretical justification for the good performance of DG flow on WGAN models. We show that flolow is related to the gradient flow of the entropy-regularized Wasserstein functional FW : P22(2)) - R,
dxx) (x))xx - 1 p(x))x p(x)dx +7
(p)
sup Id
p(x) log p(x)dxx,
(27)
Wasserstein distance
negative entropy
where y denotes the target density, ||d||tip denotes the Lipschitz constant of the function d.
Let d' be the function that achieves the supremum in Eq. (7 This results in the functional,
Py (p) = f d' (x) u())xxx / (x) p(x) + + p(x) log p(x)dx.
(28)
Following similar derivation as in Appendix A.1 the gradient flow of FW (p) is given by the following PDE,
0epo(x) VX V (x)) (xxxP:(x) 0.
(29)
If d' is approximated using the critic (dz) of WGAN, we get the following gradient flow,
dxpr(x) Vx (
-
(30)
which is same the gradient flow of entropyy-regularized f-divergence with / = log (i.e., the KL divergence) when dg is treated as density-ratio estimator. The gradient flow of entropy-regullrized f-divergence with logr is simplified below,
(x): Vx (pr V (exp( - do (x)))) -xxx)((x)= =
(31)
(xp((x) Vx (p. Vx
+ 1)) - (x)x =0 (xxxxxxxx 0.
(32)
(xp:(x) + Vx (p.V
(33)
The equality of Eq. (30 and Eq. 33 implics that fl approximates the gradient flow of the 1- Wasserrstein distance when the eritic of WGAN is used for density- estimation.
C FURTHER DISCUSSION ON RELATED WORK
Energy-based & Score-based Generative Models flw is related to recently proposed energy-based generative models (Arbel et al., 2021 Deng et al. 2020) - one can view the en- ergy functions used in these methods as component that improves some base model. For example, the Generalized Energy-Baan -Based Model (GEBM) (Arbel et al.) (02T) jointly trains an implicit gener- ative model with energy function and uses Langevin dynamics to sample from the combination
14

---

Published as a conference paper at ICLR 2021
of the two. Similarly, in Deng et al. (2020), discriminator that estimates the energy function is combined with a language model to train an energy-basee eext-genrration model.
Score-based generative modeling (SBGM) (Song & Ermon 2019 2020 is another active area of research closely-red delate-related to energy-based models. Noise Conditional Score Network (NCSN) (Song & Ermon 2019 2020), SBGM, trains neural network estimate the score function of proba- bility density at various noise levels. Once trained, this score network used to evolve samples from noise the data distribution using Langevin dynamics. NCSN can be viewed refines a sample right from noise to data; however, unlike DG f'low, NCSN is a complete genera- tive models itself and not sample refinement technique that can be applied other generative models.
gradient flow that
Other Related Work Monte Carlo techniques have been used for improving various components in generative models, e.g., .g..Grover et al. (2018) proposed Variational Rejection Sampling which per- forms rejection sampling in the latent space of VAEs improve the variational posterior and Grover et al. (2019) used likelihood-free importance sampling for bias correction in generative models Wu al. (2020) proposed Latent Optimization GAN (LOGAN) which optimizes the latent vector as part of the training process unlike D fow that refines the latent vector post training.
D IMPLEMENTATION DETAILS
D.1 2D DATASETS
Datasets The 25 Gaussians dataset was constructed by generating 100000 samples from a mixture of 25 equally likely 2D isotropic Gaussians with means {-4, -2 0, 2, 2] x (-4, 0, c R.2 and standard deviation 0.05, Once generated, the data-points were normalized by 2v2 following Tanakaa (2019). The 2DSwissroll dataset was constructed by first generating 100000 samples of the 3D swissroll dataset using make_swiss_roll from scikit-leard with noise=0, 25 and then only keeping dimensions (0,2). The generated samples were normalized by 7.5.
Base Models We trained a WGAN-GP model for both the datasets. The generator was fully- connected network with ReI.U non-linearities that mapped the discriminator was fully-connected network with ReLU non- noneriicassstes that mapped e R² R. We refer the reader to Gulrajani al. (2017) for the exact network structures. The gradient penalty factor was set to 10. The models were trained for 10K generator iterations with a batch size of 256 using the Adam optimizer with a learning rate of 10-4 31 0.5, and B1 0.9. We updated the discriminator 5 times for each generator iteration.
N(00, T2x2)
IR2 Similarly,
Hyperparameters We DOT for 100 steps and performed gradient descent using the Adam optimizer with a learning rate of 0.01 and 8 = (0., 0.9) as suggested by Tanaka (2019). DDLS was for 50 iterations with step-size of 0.01 and the Gaussian noise scaled by factor of 0.1 as suggested by Che et al. (2020). For DGflow. we set the step-size y 0.01, the number of steps 100, and the noise regularizer y = 0.01. We used the output from the WGAN-GP discriminator directly as logit for estimating the density ratio for DDLS and fG
Metrics We compared the different methods quantitatively on two metrics: % high quality samples and kernel density estimate (KDE) score. A sample is classified as a high quality sample if it lies within 4 standard deviations of its nearest Gaussian. The KDE score is computed by first estimating the KDE using generated samples and then computing the log-likelihood of the training samples under the KDE estimate. KDE was performed using sklearn. .ei neighbors. KernelDensis ty with Gaussian kernel and a kernel bandwidth of 0.1. The quantitative metrics were averaged over 10 runs with 5000 samples from each method.
D.2 IMAGE EXPERIMENTS
Datasets CIFARIO Krizhessky et al. 2009 is a dataset 60K natural RGB images of size
32 x 32 from 10 classes. STL.10 dataset of 100K natural RGB images of size 96 x 96 from 10
15

---

Published as a conference paper at ICLR 2021
Table 6: Network architectures used for MMDGAN and VAE models.
(a) Generator or Decoder
(b) Discriminator or Encoder
Input Shape: (b, d, 1, 1)
Input Shape: (b, 3, 32, 32)
Upconv
C6nv(64)
BatchNorm
LeakyReLU(O.22
ReLU
Conv(128)
Upconv (128)
BatchNorm
BatchNorm
LeakyReLU(0.2))
ReLU
Conv(256)
Upconv(64)
BatchNorm
BatchNorm
LeakyReLU(O.2))
ReLU
Conv(m)
Upconv(())
Output Shape: (b, m, 1, 1)
Tanh
Output Shape: (b, 3, 32, 32)
classes. We resized the STL10 (Coates et al., 2011) dataset to 48 48 for SNGAN and WGAN- GP, and to 32 x 32 for MMDGAN, OCFGAN-GP, and VAE since the respective base models were trained on these sizes.
Base Models for CIFAR10 We used the publicly available pre-trained models for WGAN-GP, SN-DCGAN (hi), and SN-DCGAN (ns). We refer the reader to Tanaka (2019) for exact details about these models. For SN-ResNet-GAN and OCFGAN-GP
used the pre-trained models from Miyato et al. (2018) and Ansari et al. (2020) respectively. We used the respective discriminators of SN-DCGAN (ns), SN-DCGAN (hi), and WGAN-GP for density-ratio estimation when refining their generators. For the SN-ResNet-GAN (hi) generator, we used SN-DCGAN (ns) discriminator as the non-saturating loss provides better density-ratie estimation than a discriminator trained using the hinge loss.
We trained our own models for MMDGAN, VAE, and Glow. We used the generator and dis- criminator architectures shown in Table 6 for MMDGAN with d = 32. VAE used the same architecture with d 64. Our Glow model was trained using the code available at https: //github. com/ y@as //aast/llow PyTorch with a batch size of 56 for 150 epochs. The density ratio correctors, D> (see section 3.2). were initialized with weights from the SN-DCGAN (ns) re- leased by Tanaka (2019). Dx was then fine-tuned on images from SN-DCGAN (ns)'s generator and the generator being improved (c.g., MMDGAN and OCFGAN-GP) using SGD with learning rate of 10 and momentum of 0.9. We fine-tuned D> for 10000 iterations with a batch size of 64.
Base Models for STL10 We used the publicly available pre-trained models TTnaka 2019 Ansari et al. (020) for WGAN-GB SN-DCGAN (hi), SN-DCGAN (ns), and OCFGAN-GP GP. We tramed our own models for MMDGAN and VAE with the same architecture and training details as CIFAR 10. We fine-tuned the density ratio correctors for STL10 for 5000 iterations with other details being the same as CIFARIO.
Hyperparameters We performed 25 updates of fG for CIFAR1O and STL10 with step size of 0.1 for models that do not require density ratio corrections. For STL10 models that require a density ratio correction, we performed 15 updates with a step size of 0.05. The noise regularizer (7), whenever used, was set to 0.01.
Metrics We used the Fréchet Inception Distance (FID) (Heusel et al., 2017) and Inception Score (IS) (Salimans et al. 2016) metrics to evaluate the quality of generated samples before and after re- finement. The denotes the confidence in classification of the generated samples using a pretrained Inception V3 network whereas the FID is the Fréchet distance between multivariate Gaussians fitted to the 2048 dimensional feature vectors extracted from the InceptionV3 V3 network for real and gen- erated data. Both the metrics were computed using 50K samples for all the models, except Glow
16

---

Published as a conference paper at ICLR 2021
Table 7: /diierrgences and their derivatives.
f-divergence
f
KL
r log r logr (r + lo =+1
log + log log(r +
JS
log D
(r + log(r + 1) 2log2 2
74T
where we used 10K samples. Following Tanaka (2019), we used the entire training and test set (60K images) for CIFARIO and the entire unlabeled set (100K images) for STL10 the set of real images used to compute FID.
D.3 CHARACTER LEVEL LANGUAGE MODELING
Dataset
We used the Billion Words dataset (Chelba al. 2013) which was pre-processed into 32-character long strings.
Base Model
Our generator was ID CNN which followed the architecture used by Gulrajani al.
2017
Hyperparameters
We performed 50 updates of fl with a step size of 0.1 and noise factor
= 0.
Metries
The IS-4 and JS-6 scores were computed using the code provided by Gulrajani et al. 2017) https://gt thub. cwm/igum/uu htggul//2/mmrroveeewg...ece from the models to compute the JS-4 score.
We used 10000 samples
E ADDITIONAL RESULTS
DGrlow
DGllow
DGflowKK [K:]
(log Dj
135)
Figure 4: Qualitative comparison of DGflow with different f-divergences on the 25Gaussians and 2DSisssroll datasets.
Fig. 4 shows the samples generated by WGAN-GP (leftmost, blue) and refined samples gener- ated using DGG with different f-divergences (red). Fig. 5 shows the deterministic compo- nent, (xx)(((p/)))x)))) of the velocity for different f-divergences on the datasets. Fig. (right) shows the latent space distribution recovered by DG flow when applied in the latent space for the 2D datasets. This latent space is same as the one derived by Che et al. 2022). i.c., (z(z) pz(z) cxp(d(go which is shown in Fig. (left) for both datasets.
Table 1I shows the comparison of DG flow with DOT terms of the inception score for the CI- FAR10 and STL10 datasets. DG flow outperforms DOT significantly for all the base GAN mod- els on both the datasets. Table compares different variants of f applied to MMDGAN, OCFGAN-GP, VAE, and Glow generators in terms of the inception score. DG leads
17

---

Published as a conference paper at ICLR 2021
Figure 5: A vector plot showing the deterministic component of the velocity, i.c., the drift - vxx (po for different f-divergences the 25Gaussians and 2DSwissroll dataset.
Table 8: Runtime comparison of DOT, DDLS, and DG flowkL. the 25Gaussians dataset. The runtime is averaged over 100 runs with standard deviation reported parentheses.
Method DOT
Runtime (s) per 5K samples
2.24 (0.18)
DDLS
2.23 (0.14)
flolow
2.22 (0.15)
significant improvement in the quality of samples for all the models. Tables 13 and 14 compare the deterministic variant of flo (y = 0) against DOT and DDLS. These results show that enhancement for flowo not necessity, and it outper-
the diffusion term only serves forms competing methods even without added noise. Table 15 shows the results of DG on MMDGAN, OCFGAN-GP, and VAE models when the SN-DCGAN (ns) discriminator is directly used as a density-ratio estimator without an additional density corrector. Figures 7 and I0 show the samples generated by the base model (left) and the refined samples (right) using flwlow for the CIFAR10 and STL10 datasets.
Runtime DG/low flow performs backward pass through do to compute the gradient of the density-ratio with respect the latent vector. This results in the same runtime complexity as that of DOT and DDLS. Table g shows a comparison the runtimes of DOT, DDLS, and, DGf on the 25Gaussians dataset under same condi- tions. As expected, these refinement methods have sim- ilar runtimes in practice. The wall-clock time required for DG to refine 100 samples from different base models the CIFAR10 and STL10 datasets is reported in tables g and 10
(a) Gaussians
(b) 2D Swissroll
Figure 6: Latent space recovered by
DG flow (right) for the 2D datasets same as the one derived by Che al. (2020) (left).
18

---

Published as a conference paper at ICLR 2021
Table 9: Runtime of DG flowar, for models that do not require density-ratio correction on a single GeForce
RTX 2080 Ti GPU. The runtime is averaged over 100 runs with standard deviation reported in parentheses.
Model
Runtime (s) per 100 samples
WGAN-GP
0.897 (0.017)
CIFARIO
SN-DCGAN (hi)
0.952 (0.008)
SN-DCGAN (ns)
0.952 (0.007)
SN-ResNee-GAN
1.982 (0.013)
WGAN-GP
1.376 (0.025)
STLIO
SN-DCGAN (hi)
1.413 (0.015)
SN-DCGAN (ns)
1.415 (0.013)
Table Runtime DG flowar, for models that require density-ratio correction on single GeForce RTX
2080 Ti GPU. The runtime is averaged over 100 runs with standard deviation reported in parentheses.
Model
Runtime (s) per 100 samples
CIFARIO
MMDGAN
1.192 (0.007)
OCFGAN-GP
1.186 (0.011)
VAE
1.186 (0.012)
MMDGAN OCFGAN-GP
1.036 (0,004)
STLIO
1.029 (0.010)
VAE
1.028 (0.011)
Table 11: Comparison of different variants of flow with DOT on the CIFARIO and STL10 datasets. Higher scores are better.
Inception Score
Model
Base Model
DOT
DG flowat)
DG
DG DGolowoog Di
WGAN-GP
6,51 (.02)
7.45
7.99 (.02)
7.71 (.02)
7.11 (.03)
CIFARIO
SN-DCGAN (hi)
7.35 (.03)
8.02
8.13 (.02)
7.98 (.01)
7.85 (.02)
SN-DCGAN (ns)
7.38 (.03)
7.97
8.14 (.03)
7.98 (.04)
7.94 (.01)
SN-ResNee-GAN
8.38 (.03)
9.35 (.03)
9.13 (.04)
9.05 (.03)
WGAN-GP
8.72 (.02)
9.31
10.41 (.02)
8.85 (.06)
9,80 (.03)
STLIO
SN-DCGAN (hi)
8.77 (.03)
9.35
9.74 (.04)
9.50 (.05)
9.41 (.07)
SN-DCGAN (ns)
8.61 (.04)
9.45
9.66 (.01)
9.49 (.03)
9.18 (.03)
Table 12; Comparison of different variants of DG flow applied to MMDGAN, OCFGAN-GP, VAE, and Glow models, Higher scores are better.
Inception Score
Model
Base Model
DG flolo
DG/lowas)
DG DG flowgog D)
MMDGAN
5.74 (.02)
6.27 (.05)
5,99 (.03)
6.02 (.01)
CIFARIO
OCFGAN-GP
6.52 (.02)
7.21 (.05)
6.93 (.03)
6.92 (.03)
VAE
3.20 (.01)
3.85 (.01)
3.21 (.02)
3.57 (.02)
Glow
3.64 (.02)
4.57 (.02)
3.91 (.03)
4.47 (.03)
CIFARIO
MMDGAN
6.07 (.02)
6.16 (.01)
6.12 (.03)
6.12 (.03)
OCFGAN-GP
7.09 (.01)
7.46 (.04)
7.10 (.03)
7,33 (.02)
VAE
3,25((0)
3.72 (.04)
3.27 (.01)
3.65 (.03)
19

---

Published as a conference paper at ICLR 2021
Table 13: Comparison of different variants of DG without diffusion (i.e., 7
the CIFARIO and
STL10 datasets. Lower scores are better.
Fréchet Inception Distance
Model
Base Model
DOT
DGflowau
DG
DG flowoog
CIFARIO
WGAN-GP
28.34 (.11)
24.14
24.64 (.13)
23.30 (.11)
24.42 (.19)
SN-DCGAN (hi)
20.67 (.09)
17.12
15.79 (.07)
16.79 (.09)
17.79 (.05)
SN-DCGAN (ns)
20.94 (.12)
15.78
15.47 (.11)
16.32 (.11)
16.97 (.08)
WGAN-GP
51.34 (.21)
44,45
38.96 (.08)
50.44 (.09)
39.35 (.12)
SILIO
SN-DCGAN (hi)
40.82 (.16)
34.85
35.18 (.09)
36.53 (.13)
36.75 (.13)
SN-DCGAN (ns)
41.83 (.20)
34.84
34.81 (.08)
35.75 (.10)
37.68 (.08)
Table 14: Comparison of DDLS with fGflow (with and without diffusion) on the CIFAR10 dataset. Higher
scores are better.
Model
Inception Score
NN-ReeNet-GAN (Miyato
2018
8.22 (.05)
SN-ResNet-GAN DDLS (cal) (Che al. 2020
9.09 (.10)
SN-ResNet-GAN (our evaluation)
8.38 (.03)
SN EN-RResNetGGAN + DG (lowix.) (7 = 0)
9.35 (.04)
SN Ri-ReeNet-GNNN + (lowwK (7 = 0.01)
9.35 (.03)
BigGAN
9.22
Table 15: Comparison of different variants of DG flow applied MMDGAN, OCFGAN-GP, and VAE models without density- ratio correction. Lower scores are better.
Fréchet Inception Distance
Model
Base Model
KL
JS
log D
CIFARIO
MMDGAN
42.03 (.06)
39.06 (.08)
39.68 (.06)
39.47 (.07)
OCFGAN-GP
31.95 (.07)
27.92 (.08)
29.25 (.06)
28.82 (.10)
VAE
129.49 (.19)
127.50 (.15)
128.24 (.11)
128.3 (.14)
Glow
100.7 (.14)
93.47 (.09)
97.50 (.11)
97.78 (.14)
MMDGAN
47.22 (.04)
45.75 (.10)
45.96
46.26 (.13)
STL1O
OCFGAN-GI GP
36.60 (.15)
34.17 (.18)
34.42 (.04)
34,99 (.07)
VAE
150.49 (.07)
151.76 (.01)
152.03 (.05)
151.88 (.11)
20

---

Published as a conference paper at ICLR 2021
WGAN-GP
(b) WGAN-GP + fG
(c) SN-DCGAN (hi)
(d) SN-DCGAN (hi) + DG DG flow
(c) SN-DCGAN (ns)
(f) SN-DCGAN (ns) + DG flow
(g) SN-ResNet-GAN
(h) SN-ResNet-GAN + DG
Figure 7: Samples from different models for the CIFAR10 dataset before (left) and after (right) refinement
using DGflow.
21

---

Published as a conference paper at ICLR 2021
(a) MMDGAN
(h) MMDGAN + DG flow
(c) OCFGAN-GP
(d) OCFGAN-GP + fl
(c) VAE
(f) VAE + DGflow
(g) Glow
(h) Glow + DG
Figure 8: Samples from different models for the CIFAR10 dataset before (left) and after (right) refinement
using DGflow.
22

---

Published as a conference paper at ICLR 2021
WGAN-GP
(b) WGAN-GP flow
SN-DCGAN (hi)
(d) SN-DCGAN (hi) DG flow
(e) SN-DCGAN (ns)
(l) SN-DCGAN (ns) DG
Figure 9; Samples from different models the STL10 dataset before (left) and after (right) refinement using DG flow.
23

---

Published as a conference paper at ICLR 2021
MMDGAN
(b) MMDGAN fG
(c) OCFGAN-GP
(d) OCFGAN-GP fl
VAE
(f) VAE DG low
Figure 10: Samples from different models STL10 dataset before (left) and after (right) refinement using DG
24


Reviews:
[{"review_id": "qbYU8eP82s8", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, authors have proposed to use Wasserstein gradient flows to update samples from deep generative models (DGM) to closer to the empirical data distribution which is expected to produce better generated samples. The Wasserstein gradient flows are induced from the entropy-regularized f-divergence functional between data the empirical data distribution and the to-be-learned distribution. The experimental results have shown improvement in the qualitative and quantitative results on real-world datasets of images and texts. \n\nThe writing is clear in general however the first paragraph of section 3.1 seems vague. Authors state that \"experiments show\nthat the stale estimate ...  provides a good enough approximation for the purpose of refining samples\" but then later they state \"refining directly in the data-space using the stale estimate poses problems\". It seems not consistent. Moreover, in the experimental sections, there is no result which refines \"directly in the data-space\". \n\nIt is not clear how to combine Eq. (8) with (11) to obtain the update on Eq. (12).  It will be more convinced if authors provide rigorous proof. \n\nIn the experimental section:\n   - In Table 2, there is no result of FID for the baseline DDLS. Similarly, there is no result of IS for the baseline DOT.\n   - Is it possible to use DOT/DDLS to apply for GANs with vector-valued critics (and/or VAE, Glow, etc.) by using auxiliary GDMs as with DGflow? If yes, what are their results in Table 4?\n\nFor images, KL divergence usually provides better results while  JS divergence is better for texts? Do authors have any insights?", "extra_scores": {}, "extra_text": {}}, {"review_id": "mPDGGP_pb8_", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Pros:\n* The proposed framework seems principled and practical. Propagating generated samples to follow the data distribution by simulating the gradient flow of the f-divergence to the data distribution is reasonable, and the required quantity for simulation, i.e. the density ratio between the sample and data distributions, is readily given by the discriminator in GAN training.\n* The presentation follows a clear logic flow, and related works are clearly connected. Experiment shows promising results.\n\nCons:\n* Some statements can be made more precise and rigorous, up to my knowledge.\n  - \"The discriminator is trained to maximize this distance\": although the discriminator is involved in a minimax optimization problem, it is the distributions-dependent optimal discriminator that defines a distance between two distributions. The minimax objective may not be a distance between two distributions given an arbitrary discriminator.\n  - Eq. (1). Up to my knowledge, on a metric space, there is no formal definition of __gradient__. Even the concept of tangent vector is not defined on a metric space. Formally, a tangent vector involves a differential structure, so the space is often required to be a manifold. To define gradient, the space is further required to be a Riemannian manifold. What can be defined on a metric space is __gradient flow__, as __curves__ holding the intuition to minimize a given function steepest, and there are several formal descriptions on this intuition, e.g. the minimizing movement scheme. But the curves cannot be described using tangent vectors $x'(t)$ and gradients, as presented in Eq. (1).\n  - Eq. (3) is specific to the 2-Wasserstein space.\n* On Lemma 3.2.\nThe result is based on the rule of change of variables, Eq. (25). But if $g$ is not required to be injective, the right hand side of Eq. (25) needs to be multiplicated by the number of $z$'s that make $x = g(z)$ [Federer, 1969, \"Geometric Measure Theory\", Thm. 3.2.5]. So the lemma may need to be adjusted accordingly.\n* It is favorable to cite the specific theorem/statement from works by Villani (2008) and Ambrosio et al. (2008) since they are huge books.\n* On the method.\nAlthough asymptotically the method guarantees that the sample distribution will converge to the data distribution, for each sample it may not converge and may traverse non-stop in the support of the data distribution. In other words, there exists nontrivial (non-zero) dynamics that keeps the data distribution stationary/invariant. Some examples in Fig. (2) already show this behavior to some extent, where all the samples along evolution seem realistic and differ in e.g. color or orientation. So is there a method to determine when to stop the evolution? Also, is it a problem that the evolution may change some attributes of the original sample?\n\n=== EDIT: post rebuttal ===\n\nThanks for the response and addressing the issues for a more serious research paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "dz0Xukf7sCu", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Summary\nThe paper introduces Wasserstein gradient flows for refining the samples of generative models. To achieve this, the authors demonstrate the gradient flow of divergences between two distribution in the Wasserstein space can be expressed as a stochastic differential equation whose dynamic mainly depends on the gradient of the divergence computed through the ratio between the two distributions. The authors observe the ratio between the generated distribution and the target distribution is provided by the discriminator and so the dynamic can be simulated via the stochastic Euler scheme. Instead of refining the distribution in the data space the authors suggest and show how to do it in the latent space. In addition, they expand their approach for non-GAN generative models, this setting requires to train (or just finetune)  another discriminator. They experimentally show that the proposed method improves the quality of the generated samples for a large panel of experimental settings. In particular, they slightly outperform other refining algorithms.\n# Major comments\n## Pros\nI really enjoyed reading the paper, even if I had to read it twice to understand everything! The paper is well written and pleasant to read. In addition, the idea presented is very nice and is new to me. The main hypothesis made (that are not always valid) are clearly expressed and discussed. The experimental section convinced me the method is good at doing the job it is made for!\n\n## Cons\nI do not have strong concerns regarding your work. However, I am not very familiar with refining techniques. It is why I give \"only\" a 7/10. \nI have however the following remarks:\n1) When applying your method for a model that did not use a classifier to be trained, do you really think retraining another discriminator between the two generative models is better than just training the discriminator? Or is it because you do not suppose that you have access to the dataset but only two the two generative models? This is not very clear to me why you would like to estimate the ratio indirectly. \n2) I would have liked to see the gradient flow in the 2D space fig 1. It would make a beautiful word and directly explicit how the method is working in this simple case. I don't know if this is easy to draw due to the randomness of the SDE though.\n3) Why don't you compare to DDLS in table 2? I think the results could be made more readable by merging tables 2 and 3 and keeping only what is important in these tables.\n4) I think discussing the intuition behind using different divergence and their impact on the refinement good also be interesting.\n5) I would also like to see something about the induce \"refinement\" time, I suppose this is not too heavy but it would be interesting to have the information about that somewhere.\n6) Experimenting on the violation of 3.2 would be interesting as well.\n\n# Minor comments\nfocussed -> focused\nWeiner -> Wiener\nof particle -> of the particles\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "MM0bnm3Fjco", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nThis paper proposes using gradient flows to improve samples from generative models. In practice, it uses the density ratio estimator from a GAN’s discriminator, in simulating the particle dynamics. The resulting algorithm updates latent variables based on a drift term, which is the gradient of the energy that combines the f-divergence and a negative entropy, and a diffusion term which adds a Gaussian noise. This method can be extended to other generative models such as VAE by training a density ratio corrector, similar to a discriminator.\n\nThe idea of introducing gradient flows for refining samples is theoretically very interesting, and the authors conducted a wide range of experiments to support the idea. However, I have major concerns regarding the actual algorithm and the evaluation. I would consider increasing my score if these can be well addressed.\n\nPros\n\n1. Gradient flow provides a solid theory for improving samples from generative models, a direction that has attracted increasing attention recently. The idea proposed in this paper can help provide a unifying view of several recently proposed methods.\n2. The paper is well-written, and it provides a good coverage of background materials.\n3. It provides a wide range of experiments, from toy examples to image and text generation.\n\nCons\n\n1. It is unclear to me why the proposed DGflow can outperform DDLS, which as the authors acknowledged, is equivalent to the DDLS when using the KL-divergence. More specifically how are DDLS and DGflow (KL) different in the experiments?\n2. In Table 2, the FID for baseline models seem very different from numbers reported elsewhere (e.g., for SN-DCGAN(ns), the FID from Miyato et al. is 29.3, but here says 20.9).  With this discrepancy, it is difficult to evaluate the improvement from the proposed method.\n\nOther comments:\n\n1. The criticism of DDLS, that p_Z(z) might be undefined, seems unfair. The problem with uniform prior, e.g., can be handled by projected gradient descent. In addition, is there a similar term from entropy regularisation? \n2. Consider discussing the relationship with using gradient flows for training as in [1]\n3. Consider citing [2], which achieves the state-of-the-art GAN scores from incorporating refining latent in training.\n\n[1] Deep Generative Learning via Variational Gradient Flow. Gao et al. 2019\n[2] LOGAN: Latent optimisation for generative adversarial networks. Wu et al. 2019\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "K1R5kb6BoQz", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \n---------------\nIn this paper, the authors present a technique for refining the output of generative models by using gradients from discriminator to update the latent variables from which the generator produces a sample. The technique is more versatile than previous work since it can be applied to vector-valued discriminators, and does not restrict itself to scalar discriminators. The technique itself isn’t particularly complicated, yet delivers impressive results.\n\nPros:\n-------\nThis work provides some sound theoretical foundations on which to build sample refinement, which isn’t specific to a single GAN architecture, but is applicable to a wide range of generative techniques. Although I’ll admit that I’m not familiar enough with stochastic differential equations to understand every aspect of the author’s theoretical justification, the idea of leveraging gradients from the discriminator to update not the generator (which lives in a high dimensional parameter space) but the latent variable (which lives in a much lower dimensional space) seems plausible.\n\nIn addition, one sees that the authors invested much thought and effort into the experiments section. I especially liked that they did language modeling, a task that shows a generative model works on domains other than just images. All told, after reading the experiments one is satisfied with the evidence of the effectiveness of the author’s technique.\n\nCons:\n--------\nThe only major concern I have is that by only sampling images generated by updated latent variables, one loses diversity of generated images. A common experiment in some GAN papers was to choose two latent vectors $z_1$ and $z_2$, and show how the generator is able to smoothly interpolates between these two generated samples $g(z_1)$ and $g(z_2)$ as one generates samples with $g(\\lambda z_1 + (1-\\lambda) z_2), \\lambda\\in[0,1]$. It would be interesting to see if the latent refinement technique presented here also allows such smooth interpolation, or if as one starts with $\\lambda z_1 + (1-\\lambda) z_2$ and then refines, if the generated samples fall into a few “modes” of good samples while not generating the samples between the modes.\n\nIn 5.2 and 5.3, I would appreciate it if the authors could specify, as the authors did in 5.1, how many times the authors did the train a base model and then refine pipeline in order to obtain the means and standard deviations reported.\n\nI would also appreciate it if the authors would report the JS-6 scores in the character level language modeling experiment. One gets the scores more or less for free, if runtime concerns are an issue, I can recommend this: <https://github.com/cseward/ngram_language_model> which does the same thing as the evaluation metric the authors used (<https://github.com/igul222/improved_wgan_training>) but is much faster by using C++ under the hood (there’s a python API) and saving the n-grams in a tree structure.\n\nOne last note: In our community, to get papers published one must generally demonstrate that a method works on some standard datasets, and authors are unfortunately yet understandably hesitant to discuss situations where a method fails to produce the desired results, as such a discussion could add arrows to the quiver of unduly critical reviewers. Such situations include datasets where the method doesn’t work, sensitivity to hyperparameters and  Yet it is exactly these counter-examples which often help further understanding of the method and pave the way for further advancements. Therefore I would humbly request that, if this paper is accepted, the authors also discuss any failure cases they may have found, as these would make the paper an even more compelling read.\n\nRoom for improvement:\n---------------------------------\nOne of your claims is that “By refining inferior samples, our technique avoids expensive sample rejection used by previous methods.” Please consider reporting the runtime (i.e. wallclock time) of your method vs the methods you’re comparing against (such a report is fine in the appendix).\n\nIt’s a fact of life that most readers don’t want to delve into the mathematical details in order to understand a method, but like a few pretty pictures that give them at least an idea of what’s going on. I think the author’s paper would gain more traction in the community if in the introduction they could come up with some “eye candy” showing how the method allows the gradient to flow all the way from a discriminator to the latent variable, resulting in a better good generated sample. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work improves deep generative models by applying Langevin dynamics to sample in the latent space. The authors test their method under different configurations (different loss functions) and various generative models (VAE, flow, besides GAN). Experimental results demonstrate the benefits of the proposed method in different generative tasks. 

I tend to accept this solid work. I just have two suggestions: 1) the authors should discuss the connections and the differences between the proposed method and the energy-based methods like (Arbel et al., 2020) in-depth; 2) it may be more suitable to replace "Wasserstein gradient flow" with "Discriminator gradient flow" in the title.
Decision: Accept (Poster)