Deep Generative Wasserstein Gradient Flows
Authors: Alvin Heng
Abdul Fatir Ansari
Harold Soh
Venue: Submitted to ICLR 2023 (2023)
Keywords: deep generative modeling
gradient flow

Abstract: Deep generative modeling is a rapidly-advancing field with a wealth of modeling choices developed in the past decades. Amongst them, Wasserstein gradient flows (WGF) are a powerful and theoretically rich class of methods.
However, their applications to high-dimensional distributions remain relatively underexplored. In this paper, we present Deep Generative Wasserstein Gradient Flows (DGGF), which constructs a WGF between two distributions by minimizing the entropy-regularized $f$-divergence. We demonstrate how to train a deep density ratio estimator that is required for the WGF and apply it to the task of generative modeling. Experiments demonstrate that DGGF is able to synthesize high-fidelity images of resolutions up to $128\times128$, directly in data space. We demonstrate that DGGF has an interpretable diagnostic of sample quality by naturally estimating the KL divergence throughout the gradient flow. Finally, we show DGGF's modularity by composition with external density ratio estimators for conditional generation, as well as for unpaired image-to-image translation with no modifications to the framework.

Under review as a conference paper at ICLR 2023
DEEP GENERATIVE WASSERSTEIN GRADIENT FLOWS
Anonymous authors
Paper under double-blind review
ABSTRACT
Deep generative modeling is appidly-advancing field with a wealth of modeling choices developed in the past decade. Amongst them, Wasserstein gradient flows (WGF) are powerful and theoretically rich class of methods. However, their applications to high nigh-dimensional distributions remain relatively underexplored. this paper, we present Deep Generative Wasserstein Gradient Flows (DGGF), which constructs WGF minimizing the hatropy-reulaaiie tween two distributions. We demonstrate how to train a deep density ratio estima- tor that is required for the WGF and apply it to the task of generative modeling. Experiments demonstrate that DGGF is able to synthesize high-fidelity images resolutions up 128 128, directly in data space. We demonstrate that DGGF has an interpretable diagnostic of sample quality by naturally estimating the KL divergence throughout the gradient flow. Finally, we show DGGF's modularity by composition with external density ratio estimators for conditional generation, as well as for unpaired image-to-image translation without modifications the underlying framework.
f-divergence be-
INTRODUCTION
Gradient flow methods are powerful and general class of techniques with diverse applications rang- ing from physics (Carrillo et al. 2019 Adams et al. 2011 and sampling (Bernton 2018) to neural network optimization (Chizat& & Bach 2018) and reinforcement learning (Richemond & Maginnis 2017 Zhang et al.] 2018). In particular, Wasserstein gradient flow (WGF) methods are a popular specialization that model the gradient dynamics on the space of probability measures with respect to the Wasserstein metric; these methods aim to construct the optimal path between two probabil- ity measures optimality refers to the path of steepest descent in Wasserstein space.
source distribution q(x) and target distribution p(x) where the notion of
The freedom in choosing q(x) and p(x) when constructing the WGF makes the framework a natural fit for a variety of generative modeling tasks. For data synthesis, we choose q(x) to be a simple distribution easy to draw samples from (e.g., Gaussian), and p(x) to be complex distribution which we would like to learn (e.g., the distribution of natural images). The WGF then constructs the optimal path from the simple distribution synthesize data resembling that from the complex distribution. Furthermore, we could choose both p(x) and gxx) to be distributions from different domains of the same modality (e.g., images from separate domains). The WGF then naturally performs domain translation.
However, despite this fit and the wealth of theoretical work established over the past decades (mm- brosio et al.) 2005; Santambrogio 2017). applications of WGFs to generative modeling of high- dimensional distributions remain under-explored and limited. A key difficulty is that the 2- Wasserstein distance and divergence functionals are generally intractable. Existing works rely on complex optimization schemes with constraints that contribute to model complexity, such as approx- imations of the 2- 2-Wasserstein distance with input convex neural networks (Mokrov al. [222I) dual variational optimization schemes with the Fenchel conjugate (Fan et al.) [021] or adopting a particle simulation approach, but amortizing sample generation auxiliary generators (Gao et al. 2019 2022).
In this work, we take step towards resolving the shortcomings of WGF methods for deep generative modeling. We propose Deep Generative Wasserstein Gradient Flows (DGGF), which is formulated using the gradient flow of entropy-regularte f-divergences (Fig. T. As this formulation involves

---

Under review as a conference paper at ICLR 2023
P2
Unconditional
p(x)
Image Generation
((x)
Density Retio Compoattion
Autometti
Fuo
Horen
(Class-conditional
Generation)
Unpaired Image-to-lmage
Transkation
q0 ~ Xo
q1 ~ x1
92~xx X2
9K
XX-1 QK X
(Cat2dog)
Sample
Diagrostic
dxc Vx (x/)
+1 2/ddwe
(KL Estimate)
Gracient stops
Figure I: Left: illustration of the generative gradient flow process using DGGF. The evolution of the gradient flow is governed by the SDE shown in the figure. We visualize intermediate samples of the LSUN Church dataset. Right: visualization of application domains of DGGF. At its core, DGGF is able to perform high-fidelity unconditional image generation. The unconditional model can be used for class-conditions Additionally, DGGF is able to perform unpaired image-to-image translation with no modifications needed to the framework. Finally, DGGF possesses an innate sample diagnostic by estimating the KL divergence over the flow, which decreases as sample quality is improved over the flow.
generation via density ratio composition with external pretrained classifiers.
density ratio estimation, we introduce a novel algorithm for training deep density ratio estimators and show experimentally for the first time that gradient flow methods scale to image dimensions as high as 128 x 128. Our gradient flow formulated entirely the data space, with no need for additional generator networks. The density ratio formulation allows DGGF to be composed with ternal density ratio estimators, which we show allows us to utilize pretrained external classifiers for class-conditional generation. In addition, we demonstrate that DGGF can be viewed estimating the KL divergence of samples over the flow, providing it with an innate diagnostic to evaluate sam- ple quality that also enhances model interpretability. We also show simple technique of leveraging data- detandepnndent priors to boost generative performance. Finally, by leveraging the freedom of choos- ing the source and target distributions, we show DGGF can be applied to unpaired image-to-image translation with no modifications to the framework.
BACKGROUND
In the following, we give a brief overview of gradient flows and density ratio estimation. For a more comprehensive introduction to gradient flows, please refer to Santambrogio (2017). A thorough
overview density ratio estimation can be found in Sugiyama et al. (2012a).
Wasserstein Gradient Flows. To motivate the concept of gradient flows, we consider Euclidean space equipped with the familiar L2 distance metric (X', || ll2). Given function F X - R, the curve (E()) that follows the direction of steepest descent is called the gradient flow of F:
x((t) = = -FF(x)()).
(1)
In generative modeling, we are interested in sampling from the probubility distribution of given dataset. Hence, instead of Fuclidean space, we consider the space of probubility measures with finite second moments equipped with the 2-Wasserstein metric (Pz(2), W2). Given functional F (22) + R in the 2-Wasserstein space, the gradient flow of the steepest descent curve of F. We call such curves Wasserstein gradient flows (WGF).
Density Ratio Estimation via Bregman Divergence. Let q(x) and p(x) be two distributions over X' E Tdd where we have access to i.i.d samples X, ~ q(x) and Xp ~ p(x). The goal of density ratio estimation (DRE) is to estimate the true density ratio r" rx()
g(x) based on samples x, and Xp-
p(x)
2

---

Under review as a conference paper at ICLR 2023
We will focus density ratio fitting under the Bregman divergence (BD), which is framework that unifies many existing DRE techniques (Sugiyama et al. 2012a b). Let g : R + IR be a twice continuously differentiable convex function with bounded derivative. The BD seeks to quantify the discrepancy between the estimated density ratio ry and the true density ratio r":
(r l|ro) Excolo(r" (x)) ((ro(x)) + Ög(ra (x)))o(()) (000)0g(roo
(2)
As we only have access samples, estimate the expectations in Eq. estimates:
2 using Monte Carlo
1 (ro N Σ (g((roxx (roo)x( ((ro(xx")))
N Σ [0g(ro(([")]
1
where drop the term Ep(g(r"(x)) (x))] it does not depend the model re during optimization.
The minimizer of Eq. 3 which we denote 0*, satifies ra- (x) = rx) = ((x)/p(x). For ease of
notation, we will use the hatted symbol F., refer to Monte Carlo estimates.
3 GENERATIVE MODELING WITH WASSERSTEIN GRADIENT FLOWS
This section describes our key contribution: Deep Generative Wasserstein Gradient Flows (DGGF), in which we show how to train a deep density ratio estimator for high-fidelity generative modeling. Let q(x) and p(x) be two distributions over X' E Rd. Assume that p(x) is the target distribution that we wish to learn. We choose (x) to be a known distribution that we can sample from, such a uniform
Gaussian distribution. Our goal is to construct a WGF starting from (xx). such that trajectories along the flow decrease some notion of distance between (xx) and p(x). This will allow to flow samples from qxx) to p(x); in our case, when q(x) is simple prior and p(x) is the distribution of natural images, we can perform generative modeling of images.
Formally, the functional F P2)) - P encodes the aforementioned notion of distance between q(x) and pxx). Following Ansari et al. (2021), we choose FF to be from the family of entropy- regularized f-divergences defined
Fb(()) = p(x))((((x///(x)))) + fo(x)) log g(x)dx,
q(x)
(4)
where / RR -> I. is a twice-differeeniable convex function with f(1) = 0. We can understand
the first term, the f-divergence, as measuring the discrepancy between gxx) and p(x). Popular f-divergences include the Kullback-Leibler (KL). Pearson-x² divergence and Jensen- (JS)
divergence. The first term of Eq. 4 ensures that the "distance" between q(x) and p(x) decreases along the gradient flow, while the second (differential entropy) term improves expressiveness and prevents the collapse of the gradient flow onto the data points (Ansari et al. 2021).
From Distributions to Particles. In this study, the form mingep(()) Ff. (q). One way to formulate this optimization is to construct a gradient flow in Wasserstein space. The gradient flow of the functional (p) in Wasserstein space is the curve of measures {q he which solves the following Fokker- -lanckk equation (FPE) (Ambrosio et al.
interested in functional optimization of
2005 Villani 2009
(xq((x)
+ + VZ
(5)
where div and V2 denote the divergence and Lapacian operators respectively, and f' denotes the first derivative of f. Eq. i a non-linear partial differential equation, which makes solving for the distribution ge challenging. Instead of attempting to obtain the WGF at the population density level by solving for qt, we can utilize the connection of FPEs with stochastic differential equations (SDE) and simulate the equivalent particle system described by the following SDE:
dx. V
(q (x, (xx))//x)))))d + + 2ydd
where dw, denotes the standard Wiener process. Eq. 6 describes the stochastic evolution of particle x,; in image modeling terms, x, represents an image sample as it is evolved through the SDE. Eq. 6 and Eq. 5 are equivalent in the sense that the marginal distribution ze of particles that
3

---

Under review as a conference paper at ICLR 2023
evolve under Eq. satisfies the FPE of Eq. 5 In other words, we able to obtain samples from go along the gradient flow of F. by first drawing samples X. ~ qu and then simulating the SDE in Eq. 6 Empirically, we simulate the discretized version of Eq. 6 using the Euler- Maler-Maruyama method
Xk+1 = X - V/
+ 22mEkk
(7)
where Ea ~ NN(,, I), 17 is the step size and the time interval [0, K] is discretized into equal intervals.
3.1 GRADIENT FLOW VIA DENSITY RATIO ESTIMATION
Simulating the gradient flow in Eq. requires an estimate of the density ratio qx(x)/p(xx) which is unavailable to us. We would like to leverage the Bregman divergence to train an estimator ro such that we can simulate the gradient flow numerically as
X++1 = X - V of (ra(x))) VEK.
(8)
where we combine the constants of the Gaussian noise term into a single hyperparameter How- ever, training such an estimator requires access to samples x; ~ (x(x). Unlike diffusion models (Ho) jet al., 2020 Song al. 2020), where the time-dependent ground truths x, ~ q(xx) Xx) can be ob- tained analytically, we do not have access to the ground truth samples along the gradient flow. In early experiments, we attempted to draw samples X3 along every step of the flow when simulating Eq. 8 during training. However, this resulted in poor performance that regularly diverges as the X1:K drawn changes at every iteration as parameters 0 are updated, resulting in the lack of stable learning signal that fixed ground truths would provide.
Instead of training the model on xI we propose to only draw samples xx by simulating Eq. for the full K steps. Consider a density ratio estimator with parameters O, at training iteration L. The х are drawn from the distribution qe (xxx given by
f qo (x( (XK (x)xx
ge (x<x)
(9)
where Mo(xx| (x) is the transition kernel of simulating Eq. 8 using ro, (x). We optimize ro, (x) for the & training iteration using the Bregman divergence
[().)= = Elllg(r,, (x))r(x) - g(r,, (x))] Eg [Og(ro, (x))
(10)
where expectation over p(x) can be estimated using samples drawn from the dataset. Similar to Ansari et al. 2202T) we are effectively approximating the time-dependent density ratio with a
stale estimate (x) ((x))/(()) As training progresses, the samples Xx (x() improve correspondingly with re, (x). We hypothesize that training ra(x) on samples of improving qual- ity allows it learn the density ratio across the gradient flow without the need for explicit time- dependence. We validate our hypothesis experimentally in Sec. 5.2 where we show that despite potentially time-indepenee -ndeeeedddnn formulation, our model does not collapse to single density ratio es- timate and has implicitly learned the density ratio over the flow. We find this result noteworthy and motivates further investigations into the necessity of explicit time embedding in relevant frameworks such as diffusion models.
Our training scheme bears resemblance to Short-Run EBM Nijkamp et al.] 2019), where the model draws samples from approximation of the true Langevin dynamics. At each training step, DGGF learns to refine its estimate of the density ratio re rx()) by looking at positive samples from the data distribution and negative samples drawn from its implicit distribution. However, despite the sim- ilarity, DGGF is fundamentally distinct from EBMs: DGGF is formulated as the path of steepest descent in Wasserstein space, while EBMs are derived from maximum likelihood. We further elab- orate on the salient distinctions from EBMs and other models in Sec. g Once DGGF is trained, sampling at test time is simply a matter of running Eq. 8 directly in data space. We provide training and sampling pseudocode in Algorithms and respectively.
Choices for f-divergences and Bregman divergence. DGGF allows for flexibility in the choice of f-divergence in the gradient flow, as well as the choice of g in the Bregman divergence objective. We consolidate a list of common f-divergences and their first derivatives that we study in this paper in Table in the appendix.
4

---

Under review as a conference paper at ICLR 2023
Certain forms for pair naturally with specific f-divergences for the gradient flow, they simplify the calculations needed. In our experiments, we utilize two forms of g: the first is when g(t) (t 1)², which corresponds to the Least-Squares Importance Fitting (LSIF) objective:
(LSIF(0) = E [x]ro(x)]² [ro(x)],
(11)
and the second g(t) = flogt (1 t) log(1 t), which corresponds to the Logistic Regression (LR) objective:
(LR( = -Rp log + rr())
R.g.
log
rx(x)
(12)
rx())
Due to numerical compatibility issues that we discuss in Appendix B we pair the LSIF objective with the Pearson-x2 divergence and the LR objective with the KL. JS and logD divergences. We abbreviate them as L.SIF- LR-KI., LR-JS and LR-logD respectively and study these pairings in our experiments in Sec.
3.2 DENSITY CHASM AND DATA-DEPENDER PRIORS
In principle, the gradient flow formulation does not place any restrictions on the prior distribution 4o (x). In the oo limit, the gradient flow is guaranteed to converge to p(x) (see Appendix D for a proof). However, empirically, we observe that when implementing the flow with a finite number of steps, the choice of prior distribution significantly affect model performance.
This observation may be attributed to the density chasm problem (Rhodes et al. 2020). Consider two densities qxx) and p(x) which differ greatly, for example measured by their KT. divergence. A binary classifier that is trained on distinguishing the two distributions, which is equivalent to density ratio estimation using the "density ratio trick", can obtain near perfect accuracy while learning relatively poor estimate of the density ratio (Ansari et al., 2021). A simple example would be two narrow Gaussians with means that are far apart. A classifier can learn a trivial boundary with near perfect accuracy, such as straight line between the two modes, without having to estimate the density ratio between the two distributions accurately.
We found that using common priors for (x) such as uniform distribution led to poorer sample quality due the large chasm between simple distribution and the complex multimodal data distribution. Inspired by generation from seed distributions with robust classifiers (Santurkar et al.) 2019), we leverage a data-dependens prior by fitting a simple multivariate Gaussian to the training dataset, and sampling Xo from the data-dependent prior
go = (uD, Ep), where pD Ep[x), Ep = Ed((x (x
(13)
where D represents the training dataset. Samples from this prior poor but this approach is sufficient to cross the density chasm; we visualize some samples from the prior in Fig. g and experimental results support this approach (Sec. 5.2).
Algorithm Training
repeat
Sample Xp ~ p(z), Xo ~ (x)
Algorithm Sampling
for + 1, K do
Sample Xg ~ -q((x)
Obtain JH from 2o by simulating Eq. end for Update A according to
for +- 1, do
Obtain .R from ao by simulating Eq.
end for return X
Volg' (oog'(ro(xx)) (x)))(r) ((ro(xp)) g((ro(xx)))
until converged
4 RELATED WORKS
Gradient flows are a general framework for constructing the steepest descent curve of a given func- tional, and consequently have been used in optimizing variety of distance metrics, ranging from
5

---

Under review as a conference paper at ICLR 2023
the f f-divergence (Gao et al. crepancy (Arbel et al. 2019 Mrouch & Nguyen 1222l1 b. Sobolev distance (Mrouch et al. 2019) and related forms of the Wasserstein distance (Liutkus et al. 2019). Recent works have also explored the interpretation of the FPE as a continuous normalizing Hlww for generative modeling (Xu et al.) 2022]. A well-known method to simulate WGFs at the population density level is using the Jordan, Kinderlehrer, and Otto (JKO) scheme (Jordan et al. 1998). which approximates the dynamics of the FPE through an iterative time-discretization update. The JKO scheme requires computation of the 2-Wasserstein distance and a free energy functional, which are generally intractable. Several works have been proposed to circumvent this problem: [Mokrov et al. (202T) leverages Brenier's theorem and convex neural networks to approximately estimate W2, while Fan et al. (2021) leverages the Fenchel conjugate to evaluate the free energy as /-divergences. Our work avoids the JKO scheme entirely by adopting a particle-based approach, where we use the Huler-Maruyama method to simu- late the gradient flow, parameterized by neural networks. In this way, we avoid the need to estimate W2. as well as the dual optimization needed in the variational formulation Fan et al. (202T).
2019 2022
Ansari et al. 2021 Fan et al. 2021), maximum mean dis-
More closely related to DGGF are other particle-based approaches, Luttuus et al. (2019) uses a non- parametric approach to optimize the Sliced Wasserstein distance, where the marginal distribution of the particles is computed empirically to approximate q; flol proposes to leverage pretrained GAN discriminators as density ratio estimators for sample refinement. DGGF can be seen as a gen- eral case of DG DG flow, where we recover DG if we fix the prior go to be the implicit distribution defined by the GAN generator. When qo is chosen as a simple prior, DGGF can perform genera- tion from scratch, as opposed to purely refinement. Similar to DGGF, VGrow and EPT train deep density ratio estimators using the Bregman divergence and apply them to the task of unconditional image generation. However, there are several key differences that distinguish DGGF from VGrow and EPT. As VGrow is specific instance of EPT with the logistic regression objective, we focus on EPT while noting that our discussion applies to both methods. EPT utilizes formalism that performs generation on a predetermined batch of particles during the training process. This causes the estimator to converge to a single density ratio estimate after training, preventing further sam- pling at test time due to zero gradients. In the same vein, EPT also has poor generative performance when formulated in the data space" It was necessary to amortize sample generation to auxiliary generator and formulate the WGF in latent space 2. By flowing z and matching Go(z) = x, the generator can be sampled at test time. We show from our results in Sec. 5 that generation can be done entirely in the data space, without the complexity of training an additional generator.
Our model also shares similarities with EBMs (LeCun al., 2006), which model the data likeli- hood using a Boltzmann distribution. The model can be sampled efficiently with Langevin dynam- ies (Welling & Teh 201T), which requires an infinitely long chain for proper mixing. Similar to DGGF, Nijkamp DGjk.NNijkmpp
et al. (2219) reinitializes the chain at every training iteration, effectively sampling from an approximation of the true Langevin dynamics. Fundamentally, the key difference is DGGF is formulated by finding the steepest descent curve in Wasserstein space, while EBMs are derived from maximum likelihood estimation. As shown in prior works (Jordan et al.. 1998 Liu et al. the KL divergence,
2019). Langevin dynamics can be derived from WGFs when f is chosen
which we reproduce in Appendix E The density ratio formulation of DGGF allows us the flexibility of composition with other density ratio estimators, as well as choosing the source and target distri- butions of the flow, which permits applications such as composition with pretrained classifiers (Sec. 5.3) and image-to-image translation (Sec. [5.4). These are not directly applicable with EBMs, and adaptations to such domains require significant theoretical and experimental modifications to the
generative framework (Zhao & Chen 202 Sasaki et 2021).
5 EXPERIMENTS
In this section, we present empirical results of the generative performance of DGGF on various com- mon image datasets across different resolutions. Our goal is to demonstrate that DGGF is able to generate diverse, high-quality images across various f-divergences and Bregman divergence objec- tives, as well as generalizes to other tasks such as class-conditional generation and image-to-image
Bassed on experiments of EPT in Sec. and on unpublished scores cited by the authors; please see https: //open review. net. forum?? /wt./feruuum//// ?1 www.ummmmaaaaaa BawawggJJJ9 Cq
Score as reported in Gao et al. (2020).

---

Under review as a conference paper at ICLR 2023
Figure 2: Samples from DGGF-DDP on CIFARIO 32 CelebA 642 and LSUN Church 128 using LSIF-X x2. More results using various BD objectives and f-divergences can be found Appendix.
translation. We will use DGGF-DDP to abbreviate experiments with the data detenden prior and DGGF-UP for ablation experiments with the uniform prior.
5.1 SETUP
We test DGGF with unconditional genera- tion on CIFARIO, CelebA and LSUN Church datasets, class-conditional generation on CI- FARI0 and image-to- to-image translation on the Cat2dog dataset. All pixel values are normal- ized to the range [-1, 1J. For CIFARIO, we keep the resolution of the images at 32x32, while for CelebA and LSUN Church we resize them to 64x64 and 128x 128, respectively. We use mod- ified ResNet architectures for all experiments in this study. See Appendix for more details.
Table 1: CIFAR10 and CelebA scores.
Model
FID
CIFARIO 32 x
(no outer loop)
Gao
2022
46.63
JKO-Flow Fan
2021
23.7
IGEBM (Du & Mordatch
2019
40.58
SNGAN Miyato et al.
2018)
21.7
PixeICNN (Van Oord al.
2016
65.93
NVAE Vahdat & Kautz
2020
51.67
NCSN (Song & Ermon
2019
25.32
DGGF-DDP (LSIF-X*)
28.12
5.2 IMAGE GENERATION
DGGF-DDP (I.R-KI)
28.80
DGGF-DDP (LR-JS)
29.92
In Fig. we show uncurated samples of DGGF. DDP on different combinations of g and f- divergences. Visually, our model is able to pro- duce high-quality samples on a variety datasets up to resolutions of 128 x 128, surpassing exist- ing gradient flow techniques (Gao et al., 2019: 2022]. More samples with other /ddivergences can be found Appendix|] []] In Table we show the FID scores of DGGF-DDP in comparison with relevant baselines utilizing different generative ap- proaches. On CIFARIO, our model performs com- parably with SNGAN and the score cooe-bbsed NCSN, while outperforming the baseline EBM, autore- gressive method PixeICNN. Our method strongly outperforms EPT without an auxiliary generator (no outer loop), which is the gradient flow baseline utilizing density ratio method. comparison to the WGF baselines, DGGF strongly outperforms EPT without an auxiliary generator (no outer loop), while performing comparably with JKO-Flow. For CelebA, our model is outperformed by the state-of-the-ant variational autoencoder NVAE, but outperforms NCSN and Short-Run EBM.
DGGF-DDP (LR-logD)
30.72
DGGF-UP (I.IIF-X?)
35.11
DGGF-UP (LR-KL)
39,90
CelebA 64 64
NCSN (ongg & Ermon
2019
26.89
NVAE Vahdat & Kautz
2020
14.74
EBM-SR (Nijkamp et al.
2019
23.0:
DGGF-DDI (LSIF-X*)
22.42
DGGF-DDP (LR-KL)
22.88
To provide intuition for the gradient flow process, provide intermediate samples for the LSUN Church dataset in Fig. 5 of the appendix, which visualizes how samples drawn from the data- dependent prior is evolved
high quality sample. In this scenario, the prior contains low level features such as the color the sky and rough silhouette the building. The gradient flow retains these coarse features, but generates the higher frequency details necessary create realistic im-
7

---

Under review as a conference paper at ICLR 2023
age. This matches the intuition that the gradient flow is the steepest descent curve, as the shortest path would mean changing as little of the image as possible to reach the target distribution. We also visualize samples obtained by interpolating in the prior space for CelebA in Fig. of the appendix. Despite the use of a relatively complex prior, the model is able to smoothly interpolate in the latent space, indicating the model has learnt a semantically relevant latent representation that is charac- teristic of a valid generative model. Finally, to verify that our model has not merely memorized the dataset, particularly due to concerns that the prior fitted from data, we show nearest neighbor samples of the generated images in the training set in Appendix HH] We can clearly see that samples produced by DGGF-DDI are distinct from the closest samples the training set, which tells us that DGGF-DDP is capable of generating new and diverse samples beyond the data it was trained on.
As our model outputs the density ratio ra (xx) = ((xx)/((xx) throughout the gradient flow, our model can be interpreted as estimating the KL di- vergence ( (g(xx))lp(xxx)
Model Estimate of KL. over Flow.
1.5
DDP
1.0
UP
= g(x+) (xx )dx; = Ex, (oogro To (xx)) where the expectation is taken over the batch of samples being evolved. We denote this estimate as DEL- We show that this estimate is valid in Fig. which plots DKL over the gradient flow for both DDP and UP (the UP results serve as ablations in the next paragraph). Focusing on the DDP results, we observe that Dxr. decreases monotonically over the flow, which agrees with the notion that g(x+) approaches p(xx), as seen in Fig. 5 This validates our hypothesis that de- spite the use of a stale estimate, the model did not collapse to a single density ratio and has learnt a valid density ratio over the flow. As evident from Fig. 5] the sample quality improves pro- gressively over the flow DKL decreases. As such, this pro- vides DGGF with an interpretable diagnostic of its own sample quality that comes innately with the density ratio formulation.
0.5
0.0
6.0
3.0
0.0
50
100
Gradient flow steps K
Figure 3: KL estimate DKL. of the gradient flow for CIFARIO generation for both DDP and UP for LSIF-x² (upper) and LR-KL (lower).
Ablations with Uniform Prior. We motivate the use of the data-dependent prior by conducting ablation experiments with go being uniform prior, x U[-1,1]. 1]. All hyperparameters are kept identical to DDP experiments to isolate the effects of the choice of prior distribution. We include qualitative samples of DGGF-UP in Fig. i8 the appendix. The quantitative results can be seen from the FID scores in Table Visually, DGGF-UP produces diverse and appealing samples even with uniform prior. However, when comparing quantitative scores we observe that the use of DDP improves results significantly. Support for the density chasm hypothesis can be found by comparing the DKL of DDP and UP in Fig. 3 For both LSIF-x² and LR-KL, DGGF-UP has a significantly larger D&L at the start of the flow as compared to DGGF- DDP. This corresponds to our intuition, as the uniform prior is 'farther' from the data distribution as compared to the data-dependent prior. As a result, the model has to push the particles over a larger density chasm, leading to poorer performance given a fixed computational budget.
5.3 CONDITIONAL GENERATION WITH ROBUST CLASSIFIERS
The density ratio framework allows us to compose different density ratio estimators together, there- fore allowing us to construct WGFs between distributions diffferent from those in training. To illus- trate this, consider a multiclass classifier which predicts the probability that a given image belongs to any one of N classes. We show in Appendix F that we can express such classifiers
density ratio ply = n n|x) = N "(xxly = n)/p(x)) We can thus obtain a conditional density ratio estimator ra (x:ly = n1) ge (x.) / p/)p ly n)) by composing our unconditional estimator rx(x)) with the classifier output (see Appendix
re re = n) =
N ro (o(x))p(y=nnp -1
(14)
When (x ly n) is used in simulating the WGF in Eq. 8 obtain class-conditionss generative model. This is conceptually similar to the idea proposed in Song et al. (2020), where an uncondi- tional score model Vx log (x(()) is composed with time-dependent classifier Vx log pe(((((()))
8

---

Under review as a conference paper at ICLR 2023
plane
automabile
60g
frog
4hia
(a)
(b)
Figure 4: (a) Class-conditional samples from composition with a robust classifier. (b) Image-to-
image translation process from cat to dog images using DGGF.
form class-condition conditioal model. However, whereas Song et al. (2200) requires the separate train-
ing of a üime-dependen classifier, our formulation allows us to use off-the-sheli pretrained classifiers with no further retraining. Inspired by earlier work on image synthesis with robust classifiers (San- turkar et al. 2019), we found that using pretrained adversarially saariallyy-rbuust classifier was necessary in obtaining useful gradients for the gradient flow. We show our results Fig. 4a where each row represents conditional samples of each class in the CIFARIO dataset.
5.4 UNPAIRED BAGGE-TO-MANE TRANSLATION
Our framework can also be directly applied unpaired cenee-tt--magg-rranntee (I2I). We simply fix the prior distribution go (x) to a source domain and p(x) to a target domain. We then train the model in exactly the same manner unconditional generation (see Algorithm
We test our 12I model on the Cat2dog dataset (Lee et al. 2018). From Fig. 4b DGGF able to smoothly translate images of cats to dogs while maintaining relevant semantic features of the image. For example, the background colors of the image and the pose of the cat are unchanged-a cat that facing certain direction is translated that the facial tones of the cat is preserved-aa a cat with light fur is translated to dog with light fur. CycleGAN (Zhu al. 2017) achieves better FID scores than DGGF (Table i in the appendix) but like many 121 methods (Lee et al. 2018 Choi et al.) 2020 Zhao & Chen 2021 Nie et al.) 2021,, CycleGAN incorporates specific inductive biases, such as dual generators and discriminators together with the cycle-consistence loss. Incorporating such inductive biases into the gradient flow process can improve the translation and would make for interesting future work.
dog that faces the same direction. We also observe
CONCLUSION
In this paper, we proposed DGGF, method simulate the Wasserstein gradient flow between two distributions that minimizes the entropy-regularized f-divergence. As constructing such a flow requires an estimate of the density ratio, we showed how to leverage the Bregman divergence train deep density ratio estimator that is able to synthesize diverse images of high quality. We showed that the modularity of DGGF allows for composition with external density ratio estimators, as well as direct application to tasks such unpaired image-to-image translation. Given the flexibility of choosing the source and target distribution, future work can investigate different choices of the two distributions that could correspond entirely applications. Another possible avenue is investigating how we can incorporate task-specific inductive biases into the gradient flow process, which should allow the WGF to perform better the problem at hand.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Stefan Adams, Nicolas Dirr, Mark A Peletier, and Johannes Zimmer. From a large-deviations prin- ciple to the wasserstein gradient flow: a new micro-macro passage. Communications in Mathe- matical Physics, (07(3):711-815) 2011.
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005.
Abdul Fatir Ansari, Ming Liang Ang. and Harold Soh. Refining deep generative models via dis- criminator gradient flow. In ICIR, 2021.
Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. Advances in Neural Information Processing Systems, 32, 2019.
Espen Bernton. Langevin monte carlo and jko splitting. In Conference On Learning Theory, pp- 1777-1798. PMLR, 2018.
José Antonio Carrillo, Katy Craig, and Francesco Patacchini. A blob method for diffusion. Cal- culus of Variations and Partial Differential Equations, 58(2):1-53, 2019.
Lenaic Chizat and Francis Bach. On the global convergence of gradient descent for over- parameterized models using optimal transport. Advances neural information processing sys- tems, 31, 2018.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings the recognition, pp. 8188-8197, 2020.
the IEEE/CVF conference on computer vision and pattern
Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903.0860 2019.
Logan Engstrom, Andrew Iyyas, Shibani Santurkar, and Dimitris Tsipras. Robustness (python li- brary), 2019. URL http:////ggg.uu.u...ooo/////////
Jiaojiao Fan, Amirhossein Taghvaei, and Yongxin Chen. Variational wasserstein gradient flow. arXiv preprint arXiv:2112.02422, 2021.
Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, and Diederik P Kingma. Learning energy-based models by diffusion recovery likelihood. arXiv preprint a5Xiv:2012.08125, 2020.
Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang, and Shunkang Zhang. Deep generative learning via variational gradient flow. In International Conference on Machine Learning, pp. 2093-2101. PMLR, 2019.
Yuan Gao, Jian Huang, Yuling Jiao, Jin Liu, Xiliang Lu, and Zhijian Yang. Deep generative learning via euler particle transport. In Mathematical and Scientific Machine Learning, pp. 336-368. PMLR, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851. 2020.
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker-
planck equation. SIAM journal on mathematical analysis, 29(1):1-17, 1998.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Huang. tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
Hsin- Ying Lee, Hung- Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Hsuan Yang. Diverse image-to-image translation via disentangled representations. In Proceedings of the European conference on computer vision (ECCV), pp. 35-51, 2018.
Chang Liu, Jingwei Zhuo, and Jun Zhu. Understanding meme dynamics as flows on the wasserstein space. In International Conference on Machine Learning, pp. 4093-4103. PMLR, 2019.
10

---

Under review as a conference paper at ICLR 2023
Antoine Liutkus. Umut Simsekli, Szymon Majewski, Alain Durmus, and Robian-Robert Stöter. Sisser-wassesttein flows: Nonparametric generative modeling via optimal transport and diffu- sions. In International Conference Machine Learning, pp. 4104 1104-4113. PMLR, 2019.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv: 05957, 2018.
Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, and Evgeny Bur- naev. Large-scale wasserstein gradient flows. Advances in Neural Information Processing Sys- tems, 14:15243-15256 2021.
Youssef Mroueh and Truyen Nguyen. On the convergence of gradient descent in gans: Mmd gan as a gradient flow. In International Conference on Artificial Intelligence and Statistics, pp. 1720- 1728. PMLR, 2021.
Youssef Mroueh, Tom Sercu, and Anant Raj. Sobolev descent, The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2976-29955 PMLR, 2019.
Weili Nie, Arash Vahdat, and Anima Anandkumar. Controllable and compositional generation with latent-space energy-based models. Advances in Neural Information Processing Systems, 34:13497-13510, 2021.
Erik Nijkamp. Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent non- persistent short-run meme toward energy-based model. Advances in Neural Information Process- ing Systems, 32, 2019.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. Ad- vances in neural information processing systems, 33:4905 2020.
Pierre H Richemond and Brendan Maginnis. On wasserstein reinforcement learning and the fokker- planck equation. arXiv preprint urXiv: 2017.
Filippo Santambrogio. (Euclidean, metric. and Wasserstein) gradient flows: an overview. Bulletin of Mathematical Sciences, 7(1):87-154, 2017.
Shibani Santurkar, Andrew Ilyss, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with single (robust) classifier. Advances in Neural Information Pro- cessing Systems, 32, 2019.
Hiroshi Sasaki, Chris G Willcocks. and Toby P Breckon. Unit-ddpm: Unpaired image translation with denoising diffusion probabilistic models. arXiv preprint arXiv:2104.055 053588, 2021.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33: 2438-12448, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. basre generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456. 2020.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012a.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the breg- man divergence: a unified framework of density-ratio estimation. Annals of the Institute of Sta- tistical Mathematics, 6405):1009-1044, 2012b.
Arash Vahdat and Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in Neural Information Processing Systems, 33: 33:19967-19679, 2020.
Aaron Van Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International conference machine learning, pp. 1757-1756. PMLR, 2016.

---

Under review as a conference paper at ICLR 2023
Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-I1), pp. 681 681-6 Citeseer, 2011.
Yilun Xu, Ziming Liu, Max Tegmark, and Tommi Jaakkola. Poisson flow generative models. arXiv preprint arXiv:2209. / 178, 2022.
Ruiyi Zhang, Changyou Chen, Chunyuan Li, and Lawrence Carin. Policy optimization stein gradient flows. International Conference on Machine Learning, pp. 5737-5746. PMLR, 2018.
Yang Zhao and Changyou Chen. Unpaired image-to-image translation via latent energy transport. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp-
19418-16422 2021.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
12

---

Under review as a conference paper at ICLR 2023
Figure 5: Illustration of the gradient flow process for LSUN Church, starting from a sample from the data-dependent prior the leftmost column.
Figure 6: Interpolation results between leftmost and rightmost samples with CelebA.
A TOY DATASETS
Real Samples
LSIF-x2
LR-KL
LR-JS
LR-log
Figure 7: Comparison of different DGGF pairings of Bregman and f-divergences the 2DSwiss-
roll dataset.
To affirm that samples generated by DGGF indeed converge to the target distribution p(x). train DGGF on the synthetic 2DSwissroll dataset. The density ratio estimator is parameterized by simple feedforward multilayer perceptron. We train the model to flow samples from the prior (x()) = (0, I) to the target distribution, which we sample from the make_sw ss_roll function scikit Pesit-eee re. We plot the results in Fig. 7 from which we can see that the model indeed converges to p(x) successfully for all combinations of f and
B BREGMAN DIVERGENCE AND ÍDIIVERGENCE PAIRING
When computing the LR objective Eq. 12 we find that we run into numerical stability issues when letting re (x) be the output of an unconstrained neural network and subsequently taking the required logarithms in Eq. 12 To circumvent this issue, we let rx(x) be expressed as the exponential of the neural network's output, i.e., the output of the neural network log rx().. This formulation naturally lends itself to the gradient flow of the KL, JS and logD divergences, whose first derivatives f' that is required in Eq. are also logarithmic functions of ro(x), as seen from Table. 2 We can thus utilize numerically stable routines in existing deep learning frameworks, avoiding the need for potentially unstable operations like exponentiations (see Appendix for details). As such, we pair LR with the aforementioned divergences and abbreviate the combinations LR-KL, LR-JS. LR- logD. We did not run into such stability issues for the LSIF objective (Eq. II as the model learns to automatically output non non-negative scalar over the course of training, hence for LSIF we allow the neural network to estimate (I() directly and pair it with the Pearson-x divergence. We abbreviate this pairing as LSIFxx2.
13

---

Under review as a conference paper at ICLR 2023
C STABLE COMPUTATION OF LR AND S /-DIVERGENCES
As mentioned in Sec. B computing the logarithm of unconstrained neural networks leads to in-
stabilities in the training process. This problem when computing the LR objective in Eq. 12 and the various first derivatives of f f-divergences. We can circumvent this problem by letting ro be expressed as the exponential of the neural network and use existing stable numerical routines to avoid intermediate computations that lead to the instabilities (for example, computing logarithms and exponentials directly). Let us express the neural network output as N (N) - logro(x).. The LR objective can then be rewritten as
log (x)
(LR( -R log + rx(x)
E.y
(15)
ro(
= E
E.g [logsigmoid((NN(O0)
(16)
where logsigmoid(z) log T+e which has stable implementations in modern deep learn-
ing libraries.
Similarly for the f-divergences whose first derivatives involve logarithms, we can calculate them stably as
fr((((x)) logr(x) + NNo (x)
(17)
2r(x) log 2 + logssigmoid(NNN((3)
frs(((x)) log 1+ r(x)
(18)
[ogD(((x)) = log(r 1) + = -1 logg Dogsigmoii((-NNNaae + 1.
(19)
D PROOF OF CONVERGENCE OF GRADIENT FLOW
We provide simple proof of the convergence of the Wasserstein gradient flow in the t limit.
Theorem 1. Let the functional F (g') be defined as
Ff (f + / (x) log (x)dx.
(20)
F (q") ) is non-increasing as a function of time and converges to the global minimum in the co limit.
Proof.
0F((0')
0g' 0t
(V (W,,
(21)
= F/
(22)
where in the first line we apply the chain rule, and in the second line we use the definition of the dg gradient flow in the Wasserstein space, of = V Ff (analogous to the gradient flow in Euclidean space Eq. I
E CONNECTIONS WITH LANGEVIN DYNAMICS
In this section we demonstrate the connection between WGFs and Langevin dynamics, which have also been studied in prior works such as Jordan et al. (1998) ; Liu al. (2019). Langevin dynamics a MCMC method that is able to produce samples from a probability density p(x) using only the score function Vx log pxx). Let e be the step size and xn n (xx) be initial sample drawn from prior distribution, Langevin dynamics iteratively updates X2 as
Xk+1 = X + 2
Vx lollooppxx + VeEx.
(23)
14

---

Under review as a conference paper at ICLR 2023
Table 2: f-divergences and their first derivatives f'.
f divergence
f
Pearson-x* x²
(r-1)
2(r 1)
KL
r rlogr
log r + 1
JS
Flogr (r + log ++1
2
log r+I
2r
log
(r + 1) log (r lo
log + 1) +1
where Çk ~ (0, I). As shown by Welling & Tch 2011). J. as - oo and 17 - O, then x ~ p(x)
under certain regularity conditions, Langevin dynamics are most notably utilized as method to
sample from EBMs, which model p(x) as a Boltzmann distribution po(x) = (xx((=Exx))//((1)
We show the connection WGFs by considering the discretized SDE Eq. 7 with f log + corresponding to the KL divergence:
X = X V (x)(((()))///(xxe)) + 2
(24)
Upon decomposing the logarithm term, we immediately see that the WGF gives us
X
X
V go (xx) + V log)(xx) + V2
(25)
prior dowawwighting
Langevin dynamics
We exactly recover Langevin dynamics in the last two terms if we set the hyperparameters 1 = and 7 = 1. The emergence of the term we call 'prior downweighting' can be interpreted as pushing the samples away from (xx) by reducing its log-likelihood. Hence. WGF with f corresponding to the KL divergence can be intuitively understood as an "enhanced" version of Langevin dynamics, where samples are pushed in the direction which not only increases log p(x), but also explicitly decreases log (xx).
F CLASSIFIERS ARE DENSITY RATIO ESTIMATORS
To perform class-conditional generation in the DGGF framework, would like to estimate the density ratio a certain class over the data distribution: p(xly n2)p(x). With Bayes rule, we can write this
p(xly n) p(x)
ply = px))p(xx/pp n)
(26)
p(x)
p(u = n|x)
(27)
ply=n) - n)
The denominator term p(y n) can be viewed as constant, e.g., assume the N classes equally distributed, then p(u = n) 1/N. Therefore, we have that the class probability given by the softmax output of classifier is actually density ratio:
Np(u n/x) = p(xly = n)
(28)
p(x)
We can use this equation to convert an unconditional DGGF to a class-conditional generator. Recall the gradient flow equation:
dx. = - (x===<(((((x))) + 22ddw,
(29)
We can multiply the inverse of the classifier output with rx(xx) qe (x:x) /xxx)//(xx)) to get a density ratio between qe (x+) and the conditional data distribution p(x:ly = n):
(Xx) Np(x))
qe (x:)
ne (x:)p(( = nlxx)) p(x.) p(x.ly n.) N
(30)
pxx,y==
That is, we took our unconditional model and converted to a conditional generative model by composing it with pretrained classifier. To get the correct class-conditional density ratio, we should therefore compute
1
r = n) = (x())))( = nlxx))
(31)
N
and use this conditional density ratio estimator in the WGF SDD..
15

---

Under review as a conference paper at ICLR 2023
Table 3: Network structures for the density ratio estimator ro(x).
LSUN Church 128
CIFARI0
CelebA 64
3x3 Conv2d, 64
3x3 Conv2d, 128
3x3 Conv2d, 64
Cat2dog 64
ResBlock Down 64
3 x ResBlock 128
ResBlock Down 64
3x3 Conv2d, 64
ResBlock Down 128
ResBlock Down 256
ResBlock Down 128
ResBlock Down 64
ResBlock Down 128
2 x ResBlock 256
ResBlock 128
ResBlock Down 128
ResBlock 128
ResBlock Down 256
ResBlock Down 256
Self Attention 128
ResBlock Down 256
2 x ResBlock 256
ResBlock 256
ResBlock Down 128
ResBlock 256
ResBlock Down 256
ResBlock Down 256
ResBlock Down 256
ResBlock Down 256
2 X ResBlock 256
ResBlock 256
Global Mean Pooling
ResBlock 256
Global Mean Pooling
Global Mean Pooling
Dense ->
Global Mean Pooling
Dense +
Dense ->
Dense +
G EXPERIMENTAL DETAILS
Unconditional Image Generation. For datasets, we perform random horizontal flip as form of data augmentation. For CelebA, we center crop the image to 140x 140 before resizing to 64 For LSUN Church, we resize the image to 128 128 directly. We use the same training hyperpa- rameters across all three datasets, which is as follows. We train all models with 120000 training steps with the Adam optimizer with a batch size of 64. We use a learning rate of 1 x 10 and decay by factor of 0.1 at training steps 100000 and 110000. We set the number of gradient flow steps to K = 100 at training time and K = 110 at test time as discussed Sec. 5.2 and use a 3 and noise factor
10-2 The specific residual architectures are given in Table 3 We update model weights using an exponential moving average (Song & Ermon 2020 given by
step size
e' < m00 + (1-m) m 0,, where 0, is the parameters of the model at the i-th training step, and 0' an independent copy of the parameters that we save and use for evaluation. We set m = 0.998. We use the LeakyRel.U activation with a negative slope of 0.2. We experimented with spectral normaliza- tion and self attention layers for unconditional image generation, but found that training was stable enough such that they were not worth the added computational cost. The FID results in Table [ are obtained by generating 50000 images from the data-dependent prior, and testing the results against the training set for both CIFARIO and CelebA.
Conditional Generation with Robust Classifier. The unconditional model used for conditional generation is the same model obtained from the section above. The pretrained robust classifier checkpoint obtained from the of library (Engstrom et al. 2019). It is based on a ResNet50 architecture and is trained with L2-norm perturbations of =
We choose the LR-KL variant for our results in Fig. 4a flow is given by
This means that our conditional gradient
X = =X,- 2x log (x(()))
N p </l==n(>>>> (1) + E
1
(32)
X
V (log ra (x(x)) - d p(y = n m(xx)) + Ura
(33)
where in the second line we introduce parameter that scales the magnitude of the classifier's gradients so they are comparable to the magnitude of DGGF's gradients. We use Ø = 0.1.
Unpaired Image-to-image Translation. The Cat2dog dataset contains 871 Birman cat images and 1364 Samoyed and Husky dog images. 100 of each are set aside test images. We first resize the images to 84x84 84 before center cropping to 64 x4. Due to the relatively small size of the dataset, we use shallower residual architecture as compared to CelebA 642 despite the same resolution (Table 3) to prevent overfitting. We also utilize spectral normalization and a self attention layer at the 128-channel level to further boost stability. We set K 100 during training and K 110
16

---

Under review as a conference paper at ICLR 2023
Table 4: FID scores for image-to-image translation with the cat2dog dataset.
Model
FID
DGGF
108.10
CycleGAN
51.79
(a)
(b)
(c)
Figure 8: Samples drawn from the data-dependent priors of (a) CIFARIO 32 (b) CelebA 642 and (c) LSUN Church 128?.
test time. As observed the model tends diverge late in training, limit the number of training steps to 40000, with a decay factor of 0.1 applied to the learning rate at steps 20000 and 30000. All other hyperparameters generation. We report results for LSIF-Pearson, although we have experimented with LR-KL and found performance be similar. The FID result Table. I obtained by translating the 100 test cat images, and testing the results against the 100 test dog images.
kept identical to the experiments
unconditional image
17

---

Under review as a conference paper at ICLR 2023
H
NEAREST NEIGHBORS
(a)
(b)
Figure 9: Nearest neighbor images for CIFARIO measured by La distance in (a) the feature space of an Inception V3 network pretrained on ImageNet and (b) data space. The column to the left of the red line are samples from DGGF LSIF-x The images the right of the line are the 10 nearest neighbors in the training dataset.
(b)
Figure 10: Nearest neighbor images for CelebA as measured by Le distance in (a) the feature space of Inception V3 network pretrained on ImageNet and (b) data space. The column the left of the red line are samples from DGGF LSIF-Pearson The images to the right of the line are the 10 nearest neighbors in the training dataset.
18

---

Under review as a conference paper at ICLR 2023
1
UNCURATED SAMPLES DGGF-DDP
Figure 11: Uncurated samples of CIFARIO LSIF-Pearson.
Figure 12: Uncurated samples of CIFARIO LR-KL.
Figure 13: Uncurated samples of CIFARIO LR-JS.
19

---

Under review as a conference paper at ICLR 2023
Figure 14: Uncurated samples of CIFARIO LR-logD.
Figure 15: Uncurated samples of CelebA LSIF-Pearson.
Figure 16: Uncurated samples of CelebA LR-KL.
20

---

Under review as a conference paper at ICLR 2023
Figure 17: Uncurated samples of LSUN Church LSIFF-Parsoo..
21

---

Under review as a conference paper at ICLR 2023
J
UNCURATED SAMPLES DGGF-UP
(a) LSIF-Pearson uniform prior.
(b) LR KL uniform prior.
Figure 18: Uncurated CIFAR10 samples with DGGF-UP.
22


Reviews:
[{"review_id": "t9q7LF5_9b", "reviewer": "Reviewer_NwTe", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work re-explores deep generative modeling by minimizing the entropy regularized f-divergence in the Wasserstein-2 space of probability measures and derives an algorithm based on an ambiguous discretization method of the relevant Wasserstein gradient flows. Some experimental results on image synthesis are presented to support the proposed method.", "summary_of_the_review": "The paper lacks of both theoritical and computational justification.", "strength_and_weaknesses": "The work is not theoretically sound, and the proposed algorithm lacks justifications of its correctness both intuitively and technically.\nIt is well-known that Wasserstein gradient flows can be utilized in deep generative modeling. There are a series of publications that have introduced Wasserstein gradient flows in deep generative modeling and explored their potential in analyzing or developing deep generative modeling algorithms, for example please see [1-9]. Most of these relevant papers are not mentioned in the related work. It is not novel to leverage the Wasserstein gradient flow of the entropy regularized f-divergence in deep generative modeling and to further simulate the gradient flow with a density ratio estimation procedure [8, 9]. The empirical evaluation of the proposed method seems reasonable.\n\nThe central part of the proposed method is to numerically simulate the relevant Wasserstein gradient flows with density ratio estimation between the pushforward distribution $q_t$ and the target distribution $p$. According to the authors on page 4, “The simulation of Eq. 8 requires a time-dependent estimate of the density ratio; however, we train the density ratio estimator $r_\\theta$ without explicit dependence on time”. As far as I’m concerned, the numerical simulation formula in Eq. 8 should explicitly depend on a time-dependent density ratio which is a basic result of Euler-Maruyama discretization. It is hard to capture the basic intuition of recasting this density ratio as a time-independent one, especially from a perspective of simulating a time-dependent Wasserstein gradient flow path. In practice, the proposed method does only estimate a time-independent density ratio and plugs it in Eq. 8 as indicated by the pseudocode of the training algorithm in the appendix. Furthermore, a natural question would be raised regarding the convergence properties of the proposed algorithm. Suppose that the training of the proposed model is completed, the pushforward distribution $q_{t_K}$ is approximately equal to the target distribution $p$ when $t_K$ is large enough. Accordingly, the estimated density ratio between $q_{t_K}$ and $t_K$ would be expected to approximate a constant 1 almost everywhere. This observation is basically inspired by the convergence behavior of Wasserstein gradient flows. Then a straightforward question is how to generate new samples at test time just using a time-independent density ratio while the estimated density ratio is approximately equal to 1. Due to this possible issue, it is recommended to justify the convergence property of the iterations in Eq. 8 when a claimed time-independent density ratio estimator is deployed.\n \n[1] Geometrical Insights for Implicit Generative Modeling, 2017.\n[2] Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions, ICML 2019.\n[3] Sobolev Descent, AISTATS 2019.\n[4] Maximum mean discrepancy gradient flow, NeurIPS 2019.\n[5] Deep generative learning via variational gradient flow, ICML 2019.\n[6] Refining deep generative models via discriminator gradient flow, ICLR 2021.\n[7] On the convergence of gradient descent in GANs: MMD GAN as a gradient flow, AISTATS 2021.\n[8] Reﬁning deep generative models via discriminator gradient ﬂow, ICLR 2021.\n[9] Deep generative learning via Euler particle transport, MSML 2022.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bQV9VleRaN3", "reviewer": "Reviewer_YDPG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper under consideration proposes a method (DGGF) for solving generative modeling related problems. The method is based on Wasserstein gradient flows with potential function given by entropy-regularized $f$-divergence. Essentially, the authors try to approximate density ratio function (DRE) via Bregman divergence combined with Euler-Maruyama simulation. The density ratio function is used at the inference stage allowing a practitioner to sample from target distribution.", "summary_of_the_review": "In spite of practical applicability of the proposed DGGF approach the theoretical interpretability should be significantly improved. For now there is a gap between Wasserstein gradient flows theory and proposed methodology. \n", "strength_and_weaknesses": "**Strength**\n\n* The paper provides a comprehensive list of practical applications and shows practical advantages of the method in several use cases. For example, the proposed density-ratio based method could be adapted for conditional generation on-the-fly. \n* DGGF approach seems to overcome the alternative methods based on density ratio function (EPT) and doesn’t require additional latent space to data space generators. The proposed Algorithm is easy to implement (compared to JKO-based Wasserstein gradient flows methods)\n\n**Weaknesses**\n* My main concern regarding the proposed method is the lack of theoretical interpretability. The theory of Wasserstein gradient flows is nice but it is not clear if the Algorithm 1 is indeed turning the function $r_{\\theta}$ to be the density ratio function $q_t(x)/p(x)$. At first, $r_{\\theta}$ does not depend on time which limits the interpretability from a theoretical point of view. There are cases when the prior distribution $q_0(x)$ is significantly different from target distribution $p(x)$ and one can expect that the density ratio function significantly changes along the gradient flow (for example, when the uniform distribution is used as the prior). Secondly, the convergence analysis of the Algorithm 1 itself is also unclear. Bregman divergence optimization is anticipated by stochastic procedure (EM simulation) and it is hard to see if the optimization converges to something relevant. Therefore, special attention should be paid to the convergence analysis of the Algorithm 1 and what is the physical meaning of $r_{\\theta^*}$ it converges to. How does $r_{\\theta^*}$ resemble the real density ratio function? How does it depend on $K$?\n* In light of my previous point I have some doubts concerning the  interpretation of  “Model Estimate of KL over Flow” section. The Figure 4 shows that the quantity $E_{x_{t_k}} \\log r_{\\theta}(x_{t_k})$ is indeed converges to zero, but if we can interpret the charts as KL convergence remains unclear for me.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1EyS9U3Rku", "reviewer": "Reviewer_yo3y", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores a unconditional learning generation model using Wasserstein Gradient Flow directly on the data space. To make the training feasible, the authors propose to use a deep density ratio estimator for learning in Wasserstein space.Authors demonstrate the proposed method on Cifar10 and CelebA, and the result shows that DGGF is able to generate high-quality images.\n\nFrom a conceptual point of view, the work differs from related approaches, in the sense that it tackles the problem of learning unconditional generative models directly on data space. However, from computational point of view, I found it quite simililar to Energy-based models.", "summary_of_the_review": "Overall, I tend to give a negative vote, but the score can still be adjusted if my concerns can be addressed. My main concern comes from the fact that the experimental results of the article are not strong, and there are still many aspects that could be discussed quantitatively (e.g. on a larger datasets, reporting the training speed of the algorithm, etc.), which makes the strengths of WGGF unclear to me.\n", "strength_and_weaknesses": "Strengths\n* [S1]: The paper addresses an important problem in an interesting way, based on a clear motivation.\n* [S2]: The method is sensible and clearly presented, I enjoyed reading it.\n\nWeaknesses\n*[W1] Insufficient empirical evidence; several experimental comparisons are absent and should be included (see below for further details).\n\n\nDetails:\n* Experiments(baselines):  Considering there are few related work that addressing the intractable chanlledge, some baseline method([1][2]) should be included for comparision.\n* Can you further investigate the algorithm's computational complexity considering the training is ? Is it possible to quantify the performance on larger data sets?\n* Can a pseudo-code be added to the training/inference procedures?\n* Can a discussion be included to this concurrent work? https://arxiv.org/pdf/2209.11178v4.pdf \n\n[1] Large-scale wasserstein gradient flows. \n[2] Variational wasserstein gradient flows.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject