Generative Learning With Euler Particle Transport
Authors: Yuan Gao
Jian Huang
Yuling Jiao
Jin Liu
Venue:  (2021)
Keywords: Deep density-ratio estimation
forward Euler method
Mckean-Vlasov equation
Monge-Ampere equation
residual map

Abstract: We propose an Euler particle transport  (EPT) approach for generative learning.  The proposed approach is motivated by the problem of finding the optimal transport map from a reference distribution to a target distribution characterized by the Monge-Ampere equation. Interpreting the infinitesimal linearization of the Monge-Ampere equation from the perspective of gradient flows in measure spaces leads to a stochastic McKean-Vlasov equation. We use the forward Euler method to solve this equation. The resulting forward Euler map pushes forward a reference distribution to the target. This map is the composition of a sequence of simple residual maps, which are computationally stable and easy to train. The key task in training is the estimation of the density ratios or differences that determine the residual maps. We estimate the density ratios (differences) based on the Bregman divergence with a gradient penalty using deep density-ratio (difference) fitting. We show that the proposed density-ratio (difference) estimators do not suffer from the ``curse of dimensionality" if data is supported on a lower-dimensional manifold. Numerical experiments with multi-mode synthetic datasets and comparisons with the existing methods on real benchmark datasets support our theoretical results and demonstrate the effectiveness of the proposed method.

Under review as a conference paper at ICLR 2021
GENERATIVE LEARNING WITH EULER PARTICLE TRANSPORT
Anonymous authors Paper under double-blind review
ABSTRACT
We propose an Euler particle transport (EPT) approach for generative learning. The proposed approach is motivated by the problem of finding the optimal transport map from a reference distribution to a target distribution characterized by the Monge-Ampere equation. Interpreting the infinitesimal linearization of the Monge- Ampere equation from the perspective of gradient flows in measure spaces leads to a stochastic McKean- McKeen-Vlasov equation. We use the forward Euler method to solve this equation. The resulting forward Euler map pushes forward reference distribution to the target. This map is the composition of a sequence of simple residual maps, which are computationally stable and easy to train. The key task in training is the estimation of the density ratios or differences that determine the residual maps. We estimate the density ratios (differences) based on the Bregman divergence with gradient penalty using deep density-ratio (difference) fitting. We show that the proposed density-ratio (difference) estimators do not suffer from the "curse of dimensionality" if data is supported on lower-dimensional manifold. Numerical experiments with multi-mode synthetic datasets and comparisons with the existing methods real benchmark datasets support our theoretical results and demonstrate the effectiveness of the proposed method.
INTRODUCTION
The ability to efficiently sample from complex distributions plays a key role in a variety of prediction and inference tasks in machine learning and statistics (Salakhutdinov 2015). The long-standing methodology for learning an underlying distribution relies on an explicit statistical data model, which can be difficult to specify in many applications such as image analysis, computer vision and natural language processing. In contrast, implicit generative models do not assume a specific form of the data distribution, but rather learn nonlinear map to transform a reference distribution to the target distribution. This modeling approach has been shown to achieve impressive performance in many machine learning tasks (Reed et al.) 2016 Zhu et al.) 2017). Generative adversarial networks (GAN) (Goodfellow et al. 2014),, variational auto-encoders (VAE) (Kingma & Welling, 2014) and slow-based methods (Rezende & Mohamed] 2015) are important representatives of implicit generative models.
In this paper, we propose an Euler particle transport (EPT) approach for learning a generative model by integrating ideas from optimal transport, numerical ODE, density-ratio estimation and deep neural networks. We formulate the problem of generative learning as that of finding nonlinear transform that pushes forward a reference to the target based on the quadratic Wasserstein distance. Since it challenging to solve the resulting Monge-Ampère equation, we consider the continuity equation derived from the linearization of the Monge-Ampen equation, which is gradient flows converging to the target distribution. We solve the Mckean-Vlan equation associated with the gradient flow using the forward Euler method. The resulting EPT that pushes forward the reference distribution to the target distribution is a composition a sequence simple residual maps, which are computationally stable and easy to train. The residual maps are completely determined by the density ratios between the distributions at the current iterations and the target distribution. We estimate density ratios based on the Bregman divergence with gradient regularizer using deep density-ratin fitting.
We establish bounds on the approximation errors due to lincarization the Monge-Ampèrt equation, Fuler discretization of the Mckean-Vlasov equation, and deep density-ratio estimation. Our result

---

Under review as a conference paper at ICLR 2021
the eтror rate for the proposed density-ratio estimators improves the minimax rate of nonparametric estimation via exploring the low-dimensional structure of the data and circumvents the "curse of dimensionalliy". Experimental results multi-mode synthetic data and comparisons with state- of-the-art GANs on benchmark data support our theoretical findings and demonstrate that EPT is computationally more stable and easier to train than GANs. Using simple ReLU ResNets without batch normalization and spectral normalization, we obtained results that are better than or comparable with those using GANs trained with such tricks.
EULER PARTICLE TRANSPORT
Let X R. be random vector with distribution v, and let Z be a random vector with distribution μ. We assume that μ has known and simple form. Our goal is to construct a transformation T such that (H = V, where T 1 denotes the push-forward distribution of y by T, that is, the distribution of (Z). Then we can sample from by first generating a Z ~ μ and calculate T(Z). In practice, unknown and only a random sample { {x i.i.d. v available. We must construct T based on the sample.
There may exist multiple transports T with TH = v. The optimal transport is the one that minimizes the quadratic Wasserstein distance between μ and y defined by
Wa(p, v) = inf +EI((u,))
E - 11)
(1)
where I'(H, v/ denotes the set of couplings of (H, v/) (Villani 2008 Ambrosio et al. 2008 Suppose that μ and have densities q and p with respect to the Lesbeque measure, respectively. Then the optimal transport map T such that T = V characterized by the Monge-Ampère equation (Brenier 1991, McCann 1995 Santambrogio (013). Specifically, the minimization problem in (I) admits a unique solution = (1,7) + with T = V μ-a.e., where 1 is the identity map and /V the gradient of the potential function V m + R. This function is convex and satisfies the Monge-Ampère equation
((2)
det dct(V" = p(V)(z)) ZER.
(2)
Therefore, to find the optimal transport T, it suffices to solve 2 for v. However, is challenging to solve this degenerate elliptic equation due to its highly nonlinear nature.
Below we describe the proposed EPT method for obtaining an approximate solution of the Monge- Ampère equation (2). It consists of the following steps: (a) lincarizing (2) via residual maps, (b) determining the velocity fields governing the stochastic McKean- Vlasov equation resulting from the linearization, (c) calculating the forward Euler particle transport map and, (d) training the EPT map by estimating the velocity fields from data. Since velocity fields are completely determined by density ratios, this step amounts to nonparametric density ratio estimation. We also provide bounds on the errors due to linearization, discretization and estimation. Mathematical details and proofs are given in the appendix.
Linearization via residual map A basic approach to addressing the difficulty due to nonlinearity is linearization. We use a linearization method based on the residual map
Tr V 1 + tb, t
where 4, m R is function be chosen such that the law of T, (Z) approaches as increases VVillani 2008). for details.
We give the specific form of & below, see Theorem B. in the appendix
This linearization scheme leads to the stochastic process X, RD + R. satisfying the McKean- Vlasov equation
dt xx) vx(, t2 O, with X. H4 /z- a.e. R
where v2 is the velocity vector field of X,. In addition, we have 2, V Thus v. also determines the residual map (3). The details of the derivation are given in Theorems B.2 and B.I in the appendix. Therefore, estimating the residual map (3) is equivalent to estimating tt.
2

---

Under review as a conference paper at ICLR 2021
The movement of X, along is completely governed by vt, given the initial value. We choose ta to decrease the discrepancy between the distribution X,, say ja,, at time t and the target v with respect to properly chosen measure. An equivalent formulation of (4) is through the gradient flow (ue)e2o with {v, 1220 as its velocity fields, see Proposition B.1 in the appendix. Computationally more convenient work with (4)
Determining velocity field The basic intuition that we should move in the direction that decreases the differences between jr and the target v. We use an energy functional Clu,] to measure such differences. An important energy functional Liue] is the f-divergence (Ali & Silvey 1966).
(1((z)
Clua] (D (1]
p(a))
da.
JR.
p(x))
where is the density of Pt, is the density of and f IR= - I is assumed to be a twice- differentiable convex function with f(1) = We choose P, such that C[ue] is minimized. We show in Theorem B.I in the appendix that (x) = (x)( and v2(x) = (x). Therefore,
gc(x))
ve (x(x) - (r where rr(x))
p(x)
For example, if we use the x2 -dvveeegee with f(c) = (c-1)²/ 1)2 then (z(x)= = Vr(( is simply the gradient of the density ratio. Other types of velocity fields can be obtained by using different energy functionals such as the Lebesgue norm of the density difference, i.e. Llu,] = =fgm 1 - p(xr)
see Section B.2 for details.
The forward Euler method Numerically, we need to discretize the McKean- Vlasov equation (4)
Let > 0 be a small step size. We use the forward Euler method defined iteratively by:
Ts = 1 + suk:
(6)
X (Xxx),,
(7)
14++1 = (Tx)#
(8)
M4, U2 is the velocity field at the kth step, k = 0, 1,... K for some large K. The particle process {X&} is discretized version of the continuous process [X,) in The final transport map is the composition of sequence of simple residual maps To. Ti..,, TK. i.e.,
where Xo ~
T= TK o T---.... To. This updating scheme is based on the forward Euler method for solving equation (4. This is the reason we refer to the proposed method as Euler particle transport (EPT).
Training EPT When the target v is unknown and only a random sample is available, natural to learn by first estimating the discrete velocity fields Uy at the sample level and then plugging the estimator of U2 in (6). For example, if we use the f-divergence as the energy functional, estimating
(x( ff" (Tx dynamically at each iteration k. Nonparametric density-ratio estimation using Bregman divergences and gradient regularizer are discussed in Section 4 below. Let 6g be the estimated velocity fields at
(rx boils down to estimating the density ratios (x Us (x(x)) (x)pp(z))
the lith iteration. The kth estimated residual map is T + sūg. Finally, the trained map
i fx πk-
To.
Theoretical guarantees We establish the following bound the approximation error due the linearization of the Monge-Ampère equation under appropriate conditions:
W2 v) = O(E-M))
(10)
for some > 0, see Proposition B. in the appendix. Therefore, μe converges to v exponentially fast as + co. For an integer K > and small > 0, let {uî t E [ks,, (k + 1)s), & = .....,KK be a piecewise constant interpolation between jıks and (4(++1)s: kt 0, 1..,. K. Under the assumption that the velocity fields v, are Lipschitz continuous with respect to (ax, p4),, the discretization error of uc can be bounded in finite time interval [)) follows:
K}
sup W2 m2) (s)) tE(O,T)
(11)
The proof of II] is given in Proposition B.2 in the appendix. The enror bounds 100 and (11) imply that the distributions of the particles X generated by the EPT map defined in 7 with a small s and sufficiently large & converges the target at the rate of discretization size s.
3

---

Under review as a conference paper at ICLR 2021
When training the EPT map, the deep neural networks to estimate the density ratios (density differences) with samples. In Theorem 4.1 we provide an estimation error bound that improves the minimax rate of deep nonparametric estimation via exploring the low-dimensional structure of data and circumvents the "curse of dimensionality." Thus this result of independent interest in nonparametric estimation using deep neural networks.
IMPLEMENTATION
We now described how to implement EPT and train the optimal transport T with an i.id. sample {X+ R from unknown target distribution
The EPT map is trained via the forward
Euler iteration (6- (6--8) with a small step size 8 > 0. The resulting map is composition of a sequence of residual maps, i.e., Tk TK- ... To for large K. As implied by Theorem [4.] in Section 4 each The O,..., K can be estimated with high accuracy by Tx
sbb; where
tg (x) = (" ) (z)). Here Rsg. is the density-ratin estimator defined in (44 below based
(Y R Qk and the data (X:) ~ p. Therefore, according the EPT map 19. the particles
((:)) Tx TK-10... io(Y)), i = -,, 1,...,n
serve as samples drawn from the target distribution v, where particles (Y:)= C from simple reference distribution μ.
IR are sampled
In many applications, high-dimensional complex data such as images, texts and natural languages, tend to have low- oow-iimessional latent features. To learn generative models with latent low-dimensional structures, it beneficial to have the option of first sampling particles {Z, }i=1 from a low-dimensional
reference distribution ji E P2(TR) with < d. Then we apply T to particles Y; = Go Go (Z:)). =
..., n, where we introduce another deep neural network Gg R > IR with parameter We can estimate Gø via fitting the pairs {Z,, Y.)] We describe the EPT algorithm below.
Outer loop for modeling low dimensional latent structure (optional)
) - Sample { C T from a low-dimensional reference distribution j and let V = G = .1.....
Inner loop for finding the push-forward map
If there no outer loops, sample Y; p,i I,...,
* Get 0(x) -f" (x)
via solving 14) below with Y; Y, Set
T= 1 + sú with small step size 8.
* Update the particles Y; = 7(.),
End inner loop
there are outer loops, update the parameter of Go(-) via solving
ming Σm (Z,) 1:13/nn
End outer loop
4 DEEP DENSITY-RATIS AND DENSITY-DIFFERENCE FITTING
The evaluation of velocity fields depends on the dynamic estimation of discrepancy between the push-forward distribution qi and the target distribution p. Density-ratio and density-differene fitting with the Bregman score provides a unified framework for such discrepancy estimation without
estimating each density separately (Gneiting & Raftery 2007 Dawid 2007 Sugiyama et al. 2012a b.
Kanamori & Sugiyama| 2014).
Let r(x) = qxx)) /p(x) be the density ratio between a given density q(x) and the target p(x). Let g R + IR be a differentiable and strictly convex function. The separable Bregman score with
the base probability density p for measuring the discrepancy between r and a measurable function R R + is
B(r, R) Ex Ex (R(X)) ((((()))g((())))) o(R(X)) Ex (R(X))
(RxX))
4

---

Under review as a conference paper at ICLR 2021
Here focus on the widely used least-squares density-ratio (LSDR) fitting with g(c) (c 1)? as a working example, i.e.,
B BLSDR Rt) = (xxxxx[[[XX] -
(12)
For other choice of g, such as g(c) = e log c - + 1) log(c + 1) corresponding to estimating r via the logistic regression (LR), and the scenario of density difference fitting will be presented in detail in Section B.3.
Gradient regularizer The distributions of real data may have low-dimensional structure with their support concentrated on low-dimensional manifold, which may cause the f-divergence to be ill-posed due non-overlapping supports. To exploit such underlying low-dimensional structures and avoid ill posseddsss,, we derive a simple weighted gradient regularizer "E, (R)) [R][ R [3], motivated
by recent works smoothing via noise injection (Sønderby al. 2017 Arjovsky & Bottou 2017).
This serves as regularizer for deep density-ratio fitting. For example, with g(c) = (c- the resulting gradient regularizer
E V R
(13)
which recovers the well-known squared Sobolev semi-norm in nonparametric statistics. Gradient regularization stabilizes and improves the long time performance EPT. The detailed derivation is presented in Section B33.2]
LSDR estimation with gradient regularizer Let {x,};=] and {Yi)==1 be two collections of i.i data from densities p(æ:) and q(æ:), respectively. Let H W..w.s.B be the set of ReLU neural networks Rg with parameter Φ depth 'D width W, size S, and | lo S B. We combine the least
squares loss (2] with the gradient regularizer (13) our objective function. The resulting gradient regularized LSDR estimator of r =p/g is given by
n Σ - + ta- RI
RS E arg min
2R
Σ n
(14)
where a: 2 0 is regularization parameter.
Estimation error bound We first show that the density ratio is identifiable through the objective function by proving that, at the population level, we can recover the density ratio r via minimizing
BLSDR(R) = BLSDR (r, R) + aEE VR + +C,
where BLSDR is defined in 12 and C = Ex~gl [r (xx) 1.
Lemma 4.1. For any 2 0, we have r E arg mina Bi RR). In addition, Bi (R) 2 0 for any R with Ex Ex= (x()) < co, and B BisDn (R)
R(xx) r(x) (q.p)-a.e. R
This identifiabiity result shows that the target density ratio is the unique minimizer of the population version of the empirical criterion in (14). This provides a the basis for establishing the convergence result of deep nonparametric density-ratio estimation.
Next bound the nonparametric estimation error [| - r r(( under the assumptions that the support of P is concentrated on a compact low-dimensional manifold and r is Lipsichiz continuous. Let 2t C [-c, cm be a Riemannian manifold (Lee) 2010 with dimension m. condition number
1 1/T, volume V, geodesic covering regularity R. and m < M = O (m (m(nm///TT) < m. Denote M. x E [-c, c)mT inf x yll y E 2) S < ,,,cc (0, 1).
Theorem 4.1. Assume supp(r) = M. and r(x) satisfies [r(=)] S. B for finite constant B > 0 and is Lipschitz continuous with Lipschitz constant L. Suppose the topological parameter of H.D.s... M-2
in 149 with a = 0 satisfies D = Olog n),, W = O(n (nnarii log S = =Onn*7 /log* n), and B 2B. Then,
RXX.Y. 11
- ( S C(B² cImM)nn 2/(2+M)
where C' is a universal constant.
The error bound established in Theorem 4.1 for the nonparametric deep density-ratio fitting is new. This result is of independent interest for nonparametric estimation with deep neural networks.
5

---

Under review as a conference paper at ICLR 2021
The above derived rate Onn 2T is faster than the optimal rate of convergence for nonpara- metric estimation of Lipschitz target in R", where the optimal rate is On 2+m Stone 1982 Schmidt-Hieber 2020) as long as the intrinsic dimension M of the data is much smaller than the ambient dimension m. Therefore, the proposed density-ratio estimators circumvent the "curse of dimensionality"" if data is supported on a lower-dimensional manifold.
RELATED WORK
We discuss connections between EPT and the existing related works. The existing generative models, such as VAEs, GANs and flow-based methods, parameterize a transform map with a neural network, say G, that solves
min DGG G
(15)
where D(.,-) an integral probability discrepancy. The original GAN (Goodfellow et al. 2014).
f-GAN (Nowozin et al. 2016) and WGAN (Arjovsky et al. (017)) solve the dual form of (5) by parameterizing the dual variable using another neural network with D as the JS-divergence, the f-divergence and the 1- 1-Wasserstein distance, respectively. Based on the fact that the 1- 1-Wasserstein distance can be evaluated from samples via linear programming (Sriperumbudur et al. 2012). Liu (et al. (2018) and (Genevay et al. (2018) proposed training the primal form of WGAN via two-stage method that solves the linear programm. SWGAN (Dehpandee et al. 2018) and MMDGAN (Li et al.
2017: Binkowski et al. 2018
the sliced quadratic Wasserstein distance and the maximum mean discrepancy (MMD) as D, respectively.
Vanilla VAE Kingma & Welling 2014) approximately solves the primal form of (5) with the KL-divergence loss under the framework of variational inference. Several authors have proposed methods that use optimal transport losses, such as various forms of Wasserstein distances between the distribution of learned latent codes and the prior distribution as the regularizer in VAE to improve performance. These methods include WAE (Tolstikhin al. 2018). Sliced WAE (Kolouri al. 2019) and Sinkhorn AE (Patrini et al., 2019).
Discrete time flow- methods minimize (15) with the KL divergence loss (Rezende & Mohamed 2015: Dinh et al. [2015) 2017: Kingma et al. 2016 Papamakarios et al. [017 Kingma & Dhariwal 2018). Grathwohl et al. 2019) proposed an ODE flow approach for fast training in such methods using the adjoint equation (Chen et al. 2018b). By introducing the optimal transport tools into maximum likelihood training, Chen et al. (201 Sa) and /Zaang et al. (0188) considered continuous time flow. Chen eet al. (201 8a) proposed a gradient flow in measure spaces in the framework of variational inference and then discretized it with the implicit movement minimizing scheme (De Giorgi 1993, Jordan ee al. 1998). Zhang et al. (2018) considered gradient flows in measure spaces with time invariant velocity fields. CFGGAN Johnson& & Zhang 2018) derived from the perspective of optimization in the functional space is special form of EPT with L[-) taken the KI divergence. SW flow Liutkus et al., (219)) and MMD flow (Arbel et al. 2019) are gradient flows in measure spaces. MMD flow can be recovered from EPT by first choosing CF as the Lebesgue norm and then projecting the corresponding velocity vector fields onto reproducing kernel Hilbert spaces, please see Appendix B.4 for a proof. However, neither SW flow nor MMD flow can model hidden low-dimensional structure with the particle sampling procedure.
SVGD (Liu 2017) and the proposed EPT are both particle methods based on gradient flow in measure spaces. However, the SVGD samples from an un-normalized density. while EPT focuses on generative leaning, i.e., learning the distribution from samples. At the population level, projecting the velocity fields of EPT with KL divergence onto reproducing kernel Hilbert Spaces will recover the velocity fields of SVGD. The proof is given in Appendix [B.5] Score-based methods (Song & Ermon 2019: 2020 Ho et al. 2020) are also particle methods based on unadjusted Langevin flow and deep score estimators. At the population level, the velocity fields of these score- -csredbased methods are random since they have Brownian motion term, while the velocity fields of EPT deterministic. At the sample level, these score-based methods need to learn a vector-valued deep score function. while in EPT we need to estimate the density ratios which are scalar functions.

---

Under review as a conference paper at ICLR 2021
6 EXPERIMENTS
The implementation details on numerical settings, network structures, SGD optimizers and hyper- parameters are given in the appendix. All experiments are performed using NVIDIA Tesla K80 GPUs. The PyTorch code of EPT is available at
com. anw.anoonnmmm/am/am
2D Examples. We EPT to learn 2D distributions adapted from Grathwohl al. (2019) with multiple modes and density ridges. The first row in Figure shows kernel density estimation (KDE) plots of 50k samples from target distributions including (from left to right) 8Gaussians, pinwheel, moons, checkerboard, 2spirals, and circles.
The second and third rows show the KDE plots of learned samples via EPT with f-divergencel Lebesgue norm (left six of the second/ third row), and surface plots of estimated density ratio/ difference after 20k iterations of EPT with f-divergencel Lebesgue norm (right six of the second/ third row), respectively. Clearly, the generated samples via EPT are nearly indistinguishable from those of the target samples and the estimated density-ration difference functions are approximately equal to 1/0, indicating the learnt distribution matches the target well.
We further visualize the transport maps learned with 5squares and large4goussians from 4squares and
respectively. We use 200 particles connected with grey lines to manifest the learned transport maps. As shown in the left two figures in Figures]] 2 the central squares 5squares were learned better with the gradient penalty, which is consistent with the result of the estimated density-ratee praiio right two figures in Figure 2 For large4gaussians, the learned transport map exhibited some optimality under quadratic Wasserstein distance due the obvious correspondence between the samples left two figures in Figure 2
Figure I: KDE plots of the target samples (the first row), KDE plots of the learned samples via EPT with f-divergence// Lebesgue norm (left six of the second/third row)) and surface plots of estimated density ratio/ difference after 20k iterations EPT with f-divergence/ Lebesgue norm (right six of the second/ third row).
Figure 2: Learned transport maps (left two) and estimated density ratio (right two) in learning 5squares from 4squares and learning large4gaussians from 3mall4guussians.
Results on Benchmark Image Data. We show the performance of applying EPT to benchmark data MNIST (LeCun et al.) 1998), CIFARIO (Krizhevsky & Hinton| 2009) and CelebA (Liu al. 2015) using ReLU ResNets without batch normalization and spectral normalization. The particle evolutions on MNIST and CIFAR1O without using outer loop are shown in Figure 3 Clearly, EPT can transport samples from multivariate normal distribution into a target distribution.
We further compare EPT using the outer loop with the generative models including WGAN, SNGAN and MMDGAN. We considered different f-divergences, including Pearson's 's x KI, JS and logl (Gao et al.) 2019) and different deep density-ratio fitting methods (LSDR and LR). Table shows FID (Heusel et al. 2017) evaluated with five bootstrap sampling of EPT with four divergences on CIFARIO. We can see that EPT using ReLU ResNets without batch normalization and spectral generative models. Comparisons of the real samples and learned samples on MNIST, CIFAR 10 and CelebA are shown in Figure 4 where high-fidelity learned samples are comparable to real samples visually.
normalization attains (usually better) comparable FID scores with the
7

---

Under review as a conference paper at ICLR 2021
$
7
3
2
7
/
K
f
3
7
7
6
/
2
1
9
5
9
3
7
7
/
0
2
9
Figure 3: Particle evolution of EPT MNIST and CIFAR10.
Table Mean (standard deviation) FID scores on CIFARIO. The FID score of NSCN is reported in Song & Ermon (019)) and results in the right table are adapted from Arbel et al. (0018)
Models
CIFARIO (50k)
Models
CIFARIO (50k)
EPT-LSDR-X*
24.9 (0.1)
WGAN-GP
31.1 (0.2)
EPT-LR-KL.
25.9 (0.1)
MMDGAN-GP-LA
31.4 (0.3)
EPT-LR-JS
25.3 (0.1)
SN-GAN
26.7 (0.2)
EPT-LR-logD
24.6 (0.1)
SN- SN-SMMDAAN
25.0 (0.3)
NCSN
25.3
0
9
9
9
4
5
2
9
4
6
8
1
0
9
4
4
5
7
7
1
8
4
7
3
4
6
3
9
3
1
7
8
6
6
7
9
8
5
7
4
0
5
6
3
2
7
8
5
1
/
1
O
6
7
4
4
0
7
5
Figure 4: Visual comparisons between real images (left 3 panels) and generated images (right panels) by EPT-LSDR->2 MNIST, CIFARIO and CelebA.
CONCLUSION
EPT is a new approach for generative learning via training transport map that pushes forward reference to the target. This approach uses the forward Euler method for solving the McKean-Vlan equation, which results from linearizing the Monge-Ampee equation that characterizes the optimal transport map. The EPT map is composition of sequence of simple residual maps. The key task in training is the estimation of density ratios that completely determine the residual maps. We estimate density ratios based on the Bregman divergence with gradient penalty using deep density-ratio fitting. establish bounds on the approximation errors due to linearization, discretization, and density-ratio estimation. Our results provide strong theoretical guarantees for the proposed method and ensure that the EPT map converges fast to the target. We also show that the proposed density (difference) estimators do not suffer from the "curse of dimensionality" if data is supported manifold. This is an interesting result in itself since density-rate ratio estimation is an important basic problem in machine learning and statistics. Because EPT easy to train, computationally stable, and
lower-dimensional
8

---

Under review as a conference paper at ICLR 2021
enjoys strong theoretical guarantees, expect it to be useful addition to the methods for generating learning.
The proposed EPT method is motivated from the Monge-Ampie equation that characterizes the optimal transport map. However, while the EPT map pushes forward a reference distribution to the target, it is not estimate of the optimal transport map itself. How consistently estimate the Monge-Ampére optimal map is challenging and open problem.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series (Methodological), 28 (1):131-142, 1966.
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008.
Martin Anthony and Peter Bartlett. Neural network learning: Theoretical foundations. cambridge university press, 2009.
Michael Arbel, Dougal Sutherland, Mikolaj Binkowski, and Arthur Gretton. On gradient regularizers for MMD GANs. In NIPS, 2018.
Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. NeurIPS, 2019.
Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In ICLR, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICML, 2017.
Vladimir Igorevich Amold. Geometrical methods in the theory of orlinary differential equations, volume 250. Springer Science Business Media, 2012.
Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, 2002.
Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mchrabian. Nearly-tight ve-dimension and pseudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20:1-17, 2019.
Mikolaj Binkowski, Dougal Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In ICLR, 2018.
Yann Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Commu- nications on Pure and Applied Mathematics, 44(4):375 1991.
Changyou Chen, Chunyuan Li, Liqun Chen, Wenlin Wang, Yunchen Pu, and Lawrence Carin Duke. Continuous-tim flows for efficient inference and density estimation. In JCML, 2018a.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differen- tial equations. In NIPS, 2018b.
Frank H Clarke. Optimization and nonsmooth analysis, volume 5. Siam, 1990.
A Philip Dawid. The geometry of proper scoring rules. Annals of the Institute of Statistical Mathematics, 59(1):77-93, 2007.
E De Giorgi. New problems on minimizing movements, boundary value problems for partial differential equations. Results Applied Mathematics, 29:81-98, 1993.
Ishan Deshpande, Ziyu Zhang, and Alexander Schwing. Generative modeling using the sliced wasserstein distance. In CVPR, 2018
Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-linear independent components estimation. In ICLR, 2015.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using Real NVP. In ICLR, 2017.
Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang, and Shunkang Zhang. Deep generative learning via variational gradient flow. In ICML., 2019.
10

---

Under review as a conference paper at ICLR 2021
Izrail Moiseevitch Gelfand and Sergei Vasilevich Fomin. Calculus of variations. Dover Publications, 2000.
Aude Genevay, Gabriel Peyre, and Marco Cuturi. Learning generative models with sinkhorn diver- gences. In ICML, 2018.
Tilmann Gneiting and Adrian Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-37, 2007.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems 27, pp. 2672-2680. Curran Associates, Inc., 2014.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. FFJORD: Free-form continuous dynamics for scalable reversible generative models. In ICLR, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge local nash equilibrium. In NIPS, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in
Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020.
Richard Holley and Daniel Stroock. Logarithmic sobolev inequalities and stochastic ising models.
Journal of Statistical Physics, 46: 119-119944 1987.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICMI,, 2015.
Rie Johnson and Tong Zhang. Composite functional gradient learning of generative adversarial models. In ICML, 2018
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the Fokker-Planck equation. SIAM Journal on Mathematical Analysis, 29(1): 1998.
Takafumi Kanamori and Masashi Sugiyama. Statistical analysis of distance estimators with density differences and density ratios. Entropy, 16(2):921 0422999222222 2014.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In NIPS, 2018.
Durk Kingma, Tim Salimans, Rafal Jozefowicz, Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. In NIPS. 2016.
Soheil Kolouri, Phillip Pope, Charles Martin, and Gustavo K Rohde. Sliced-Wasseren Wasserstein autoencoder: An embarrassingly simple generative model. In ICLR, 2019.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):228-2324, 1998.
John Lee. Introduction to Riemannian Manifolds. Springer, 2010.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMD GAN: Towards deeper understanding of moment matching network. In NIPS, 2017.
Huidong Liu, GU Xianfeng, and Dimitris Samaras. A two-step computation of the exact gan Wasserstein distance. In ICML, 2018
Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems, volume 30, pp. 3115-3123. Curran Associates, Inc., 2017.
11

---

Under review as a conference paper at ICLR 2021
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
Antoine Liutkus, Umut Simsekli, Szymon Majewski, Alain Durmus, Fabian-Robert Stöter, Kamalika Chaudhuri, and Ruslan Salakhutdinov. Sliced Wasserstein flows: Nonparametrie generative modeling via optimal transport and diffusions. In ICML, 2019.
Robert J. McCann. Existence and uniqueness of monotone measure-preserving maps. Duke Mathe- matical Journal, 80(2):309-33 -244, 1995.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.
Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. arXiv preprint arXiv:1610.03483, 2016.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization, In NIPS, 2016.
Otto and C. Villani. Generalization of inequality by talagrand and links with the logarithmic sobolev inequality. Journal of Functional Analysis, 173:261 1006,,400,,00 2000.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. In NIPS, 2017.
Giorgio Patrini, Samarth Bhargav, Rianne den Berg, Max Welling, Patrick Forré, Tim Genewein, Marcello Carioni, KFU Graz, Frank Nielsen, and CSL Sony. Sinkhorn autoencoders. In UAI, 2019.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In /CML, 2016.
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In ICML, 2015.
Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. In NIPS, 2017.
Ruslan Salakhutdinov. Learning deep generative models. Annual Review of Statistics and Its Application, 2:361-385, 2015.
Filippo Santambrogio. Optimal transport for applied mathematicians, Springer, 2015.
Johannes Schmidt-Hiehe, Nonparametric regression using deep neural networks with relu activation function. The Annals of Statistics, in press, 2020.
Zuowei Shen, Haizhao Yang, and Shijun Zhang. Deep network approximation characterized by number of neurons. arXiv preprint arXiv: 1906::/906.0449000 2019.
Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi, and Ferene Huszár. Amortised map inference for image super-resolution. In ICLR, 2017.
Yang Song and Stefano Ermon. Generative modeling estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020.
Bharath K. Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard SchIkopf, and Gert R. G Lanckriet. On the empirical estimation of integral probability metrics. Electronic Journal of Statistics, 6: 6:1550-1599, 2012.
Charles J. Stone. Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 1044):004- 1982.
12

---

Under review as a conference paper at ICLR 2021
Masashi Sugiyama, Takafumi Kanamori, Taiji Suzuki, Marthinus D Plessis, Song Liu, and Ichiro Takeuchi. Density-difference estimation. In NIPS, 2012a.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012b.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.
I Tolstikhin, O Bousquet. S Gelly, and Schölkopf. Wasserstein auto-encoders. In ICLR, 2018.
Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47, Cambridge university press, 2018.
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
Linfeng Zhang, Weinan E,, and Lei Wang. Monge-Ampère flow for generative modeling. arXiv preprint arXiv:1809. 2018.
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, 2017.
13

---

Under review as a conference paper at ICLR 2021
APPENDIX
In the appendix, we provide the implementation details on numerical settings, network structures, SGD optimizers, and pppar--rraeeterss in the paper. We show the numerical convergence of EPT with simulated datasets and compare the learning and inference of EPT with other generative models. We give detailed theoretical background and proofs of the results mentioned in the paper. We also provide proofs MMD flow and SVGD can be derived from EPT by choosing appropriate f-divergences.
A APPENDIX: NUMERICAL EXPERIMENTS
A. IMPLRMENTATION DETAILS, NETWORK STRUCTURES,
We provide the details of two versions the EPT algorithm, EPTvl in Algorithm | and EPTv2 in Algorithm 2 below. In Algorithm I we describe the algorithm without outer loops. In Algorithm 2
we describe the algorithm with a latent structure and outer loops.
A.1.1 2D EXAMPLES
Experiments on 2D examples in our work were performed with deep LSDR fitting and the Pearson x² divergence. We use the EPTv1 (Algorithm[D) [ without outer loops. In inner loops, only multilayer perceptron (MLP) was utilized for dynamic estimation of the density ratio between the model distribution Oo and the target distribution The network structure and hyper-parameter in EPT and deep LSDR fitting were shared
2D experiments. We adopt EPT to push particles from predrawn pool consisting of 50k i.i.d. Gaussian particles to evolve in 20k steps. We used RMSProp with the learning rate O. 0.0005 and the batch size 1k as the SGD optimizer. The details are given in Table and Table A2 We note that is the step size, is the number of particles, is the penalty coefficient, and T is the mini-batch gradient descent times of deep LSDR fitting or deep logistic regression in each inner loop hereinafter.
Table A1: MLP for deep LSDR fitting.
Layer
Details
Output size
Linear, ReLU
2
Linear, ReLU
64
3
Linear, ReLU
64
4
Linear
Table A2: Hyper-parameters in EPT on 2D examples.
Parameter
T
Value
0.005
50k
0 or 0.5
5
A.1.2 REAL IMAGE DATA
Datasets. We evaluated EPT on three benchmark datasets including two small datasets MNIST, CIFARIO and one large dataset CelebA from GAN literature. MNIST contains training set of 60k examples and a test set of 10k examples as 28 x 28 bilevel images which were resized to 32 x 32 resolution. There are a training set of 50k examples and test set of 10k examples 32 x 32 color images in CIFAR1O. We randomly divided the 200k celebrity images in CelebA into two sets for training and test according the ratio 9:1. We also pre-processed CelebA images by first taking 160 x 160 central crop and then resizing to the 64 x 64 resolution. Only the training sets are used to train models.
14

---

Under review as a conference paper at ICLR 2021
Algorithm 1: EPTvl: Euler particle transport
Input: K N' 0,
// maximum 1oop count, step size,
regulare aarizaation coeficient
X; U, Y pr 1,2, ,.,.,,, ;
real samples, initial particles
+0
while K do
Ro E arg minR, I
-2 via SGD
// determine the density ratio
t² (x) = I (
// approximate the velocity field
7h = 1 + sBt
// define the forward Buler map // update particles
yktl (}( 1,2, ..
h ++1
end
Output: Y uig, i 1,2,
// transported particles
Evaluation metrics. Fréchet Inception Distance (FID) (Heusel et al. 2017) computes the Wasserstein
distance W2 with summary statistics (mean μ and variance Σ) of real samples and generated samples ge in the feature space of the Inception-v3 model (Szegedy et al., 2016). i.e., FID = ||xx- /gg 11 + Tr(Ex Exg - 2xxx Eg)x ) Here, FID is reported with the TensorFlow implementation
and lower FID is better.
Network architectures and hyper-parameter settings. We employed the ResNet architectures used by et al. (2019) in our EPT algorithm. Especially, the batch normalization (Ioffe Szegedy Gao
2015) and the spectral normalization (Miyato et al. (018)) of networks were omitted for EPT-LSDR-
x To train neural networks, we set SGD optimizers as RMSProp with the learning rate 0.0001 and the batch size 100. Inputs {2 in EPTv2 (Algorithm
2) were vectors generated from 128-dimensional standard normal distribution on all three datasets. Hyper-parameters are listed in Table A3 where IL expresses the number of inner loops in each outer loop. Even without outer loops, EPTv1 (Algorithm|] [T can generate images on MNIST and CIFAR as well by making use of large set of particles. Table shshows shows the
Table A3: Hyper- parameters in EPT with outer loops on real image datasets.
Parameter
T
IT.
Value
128
0.5
1k
0
20
Table A4: Hyper- parametrrss in EPT without outer loops real image datasets.
Parameter
T
Value
0.5
4k
0 5
A.2 NUMERICAL. CONVERGENCE
We illustrate the convergence property of the learning dynamics of EPTvl synthetic datasets pinwheel, checkerboard and 2spirals. As shown in Figure on the three test datasets, the dynamics of both the estimated LSDR fitting losses in (14) with O and the estimated value of the gradient norms Ex-q, ||| [ (X)] demonstrate the estimated LSDR loss converges to the theoretical value
15

---

Under review as a conference paper at ICLR 2021
Algorithm 2: EPTv2: Euler particle transport with latent structure
Input: II, E N", 8 O, a > 0 // maximum inner loop count, maximum
outer Toop count, step size, regularization coeficient
X, ~ 1, v, =,,1=1,2,...
// real samples
G f Gg
// initialize the transport map
-+0
/+ outer 1ocp +/
while OL do
Z ~ 1.2, ,
// latent particles // intermediate particles
Y (Z))), 1,2, n
+
inner lcap */
while k < IL do
R E arg minn, L (alV 2R (Y+") via SGD
// determing the donsity ratio
(=))) ==)"(R((=)))
// approximate the velocity field // define the forward Euler map // update particles
7 =1+
yk+t i,
k k+1
end
G++1 E arg ming, E (Go(Z)) 8I71 via SGD // fit the transport
map
+ j+1
end
Output: GOL -R- Rd
// transport map with latent structure
-3 3390
chacoa board
Zremeb
L1052
Fatheel
303
150
200
2006
2300
3603
4006
5300
32000
14606
LO083
38000
20000
200
Checurtced
checia
checke
122
Pra tax
102
53
3008
2000
4008
32000
140.0
10003
380000
20000
Figure 5: The numerical convergence phenomenon of EPTvI on simulated datasets. First row: LSDR fitting loss (14) with v.s. iterations on pinwheel, checkerboard and 2spirals. Second row: Estimation of the gradient norm E Ex (X)]l2) v.s. iterations on pinwheel, checkerboard and 2spirals.
A.3 LEARNING AND INFERENCE
The learning process of EPT performs particle evolution via solving the McKean- Vlasov equation using forward Euler iterations. The iterations rely on the estimation of the density ratios (difference) between the pushforward distributions and the target distribution. To make the inference EPTv1 more amendable, we propose EPTv2 based on EPTv1. EPTv2 takes advantage of neural network to fit the pushforward map. The inference of EPTv2 is fast since the pushforward map parameterized neural network and only forward propagation is involved. These aspects distinguish EPTv2 from score-based generative models (Sng & Ermon 20099 2020) which simulate Langevin dynamics to generate samples.
16

---

Under review as a conference paper at ICLR 2021
B APPENDIX: THEORETICAL BACKGROUNDS AND PROOFS
B.1 THEORETICAL BACKGROUNDS FOR SECTION
For convenience, we first give the following notation be used in this section. Let Pa( denote the space of Borel probability measures on R" with finite second moments, and let P2 (IE denote the subset of (2((RT) in which measures are absolutely continuous with respec- t to the Lebesgue measure (all distributions are assumed to satisfy this assumption hereinafter). Tan,P2 (2R")) denotes the tangent space to P2 (2(" at μ.. Let A P2((x")) := {rr
P2( is absolutely continuous, | E (I), Rt). LiPioc denotes the set of func- tions that are Lipschitz continuous on any compact set of Rm For any & E (1, oo], we use I. (M, IE"")) (Le (4. R)) to denote the I, space of j-measurable functions on Rm (on any compact set of "..". With 1, det and tr, we refer to the identity map. the determinant and the trace. We use V. 42 and to denote the gradient or Jacobian operator, the Hessian operator and the Laplace operator, respectively.
We are now ready to describe the proposed method in a mathematically rigorous fashion and provide theoretical guarantees. Let ~ 9. X = T7,(()) and denote the distribution of X as q. With a small f, the map o is invertible according to the implicit function theorem, and have the change of variables formula
g(x)
det (=) (det(
(B-1)
((x)
where
4= Tr, Tr (z).
(B-2)
Using the fact 1== applying the first order Taylor expansion to (B-1) we have
det(A + (B) det [ | BB) AA, B Rmxm with A invertible, and
log ((æ) log (xx) (A)(()) o(t)
(B-3)
Let t - in B-2 and B-3)
obtain random process {æ:} and its law q satisfying
dæ;
V4(x;;, with To
(B-4)
dt
In qo (xx.) Ax;x;), with
(B-5)
dt
Equations and B35 resulting from linearization of the Monge-Ampère equation 2 can be interpreted as gradient flows in measure spaces (Ambrosio 2008). And thanks to this connection, we can resort to solving a continuity equation characterized by a type of McKean-VIad equation, an ODE system that is easier handle.
B.2 GRADIENT FLOWS Pg (R.)
For P (R") with density let
Clu] = / F(q(z))dz (R") - R U [0}
(B-6)
be an energy functional satisfying v E arg min Cl+). where F(.) : R R a difce-differentiable convex function. Among the widely used metrics Pa (IR" in implicit generative learning, the
following two are important examples of L (1) f-divergence given in Ali & Silvey 1966;; (2) Lebesgue norm of density difference:
|| vll vllE:((mm) = / la(x) 2dx.
(B-7)
We call {utHERS C (Rt P2(R")) gradient flow of the functional L|-), (pcheER C Pa (R a.c., t R+ and the velocity vector field v, E Tan,,, (PR") satisfies V. E E0C(m]] a.e. LE EER, where aC[:] is the subdifferential of L|-].
Definition.
The gradient flow (Nelea+ of C].) enjoys the following nice properties.
17

---

Under review as a conference paper at ICLR 2021
Proposition B.1. (i) The following continuity equation holds in the sense of distributions.
te = V (Huzv)) in R+ R with no = μ4,
(B-8)
(ii) Energy decay along the gradient flow: de Clu = -1 W2(pe, v) = O(exp=XX if Clu] is X-geodetically convex with X
a.e. LE RR In addition,
(iii) Conversely, if ( is the solution of continuity equation B-8 in (i) with (x) specified by B-9 in (ii), then {pe}e a gradient flow of Cl-].
Remark B.1. In part (ii) of Proposition B.1 for general f f-divergences, we assume the functional L to be X-geodesically convex for the convergence of b4 to the target in the quadratic Wasserstein distance. However, for the KI. divergence, the convergence can be guaranteed if v satisfies the log- Sobolev inequality(Oto & Villani 2000). In addition, the distributions that are strongly log-concave outside bounded region, but not necessarily log-concave inside the region satisfy the log-Sobolev inequality, see, for example, Holley & Stroock (9877). Here the functional L can even be nonconvex, an example includes the densities with double well potential.
Remark B.2. Equation (8.48) in Proposition 8.4.6 of and Ambrosio et al. (2008) shows the connec- tion (locally) of the velocity vt of the gradient flow hr and the optimal transport along Pt, i.e., let The be the optimal transport from jr to Pe+ho then T I + ho approximates the optimal transport map from Pt to Pr4h on [t, + hi for a small h.
+ hw; + o(h) in LP. So locally,
Proof. (i) The continuity equation B-8 follows from the definition of the gradient flow directly, see,
page 281 in (Ambrosio et al., 2008 ) (ii) The first equality follows from the chain rule and integration by part, see, Theorem 24.2 of Villani 20008) The second one linear convergence follows from Theorem 24.7 of Villani (2008), where the assumption on in equation (24.6) is equivalent to the A-geodetically convex assumption here. (iii) Similar to (i) see, page 281 in Ambrosio al. (2008).
Theorem B.1. (i) Representation of the velocity fields: if the density qi of pt is difserentiable, then ve(x)) = = (g(())) pr-.ee x E R
(B-9)
ii) If we let t be time-dependent in BB44
B-5 i.e., D, then the linearized Monge-Ampère equations B-5 are the same as the continuity equation (B-8) by taking (, = -F (" (2:)..
Proof. (i) Recall C(p] is a functional on PS (R"). By the classical results in calculus of variation
(Gelfand & Fomin 2000),
2c
(x) dt
Clg + tgl 1==0= F² (q(x)),
0g
where 8C4g) denotes the first order of variation of Cii at q, and 9.9 9 the densities of i and an Bi arbitrary g E (R"),, respectively. Let
==(=)==F'' = F(z) :R + RR.
Some algebra shows,
(lr(((=)) = V (g(x)).
Then, follows from Theorem 10.4.6 in (Ambrosio et al. 2008 that
F' 8 L(u),
We say that X-geodetically convex if there exists constant > such that for every fr. M2 (R""), there exists a constant speed geodestic : [0, 1] + (IR") such that 70 = p1, 1 = μ2 and
((>)) s (1 - s)(((m)) sL(p)) > s(1 s)d(m)) Vs [0, 1].
where d is metric defined on P (R.") such the quadratic Wasserstein distance.
18

---

Under review as a conference paper at ICLR 2021
where, (p) denotes the one in (p( with minimum length. The above display and the definition
of gradient flow implies the representation of the velocity fields v%.
(ii) The time dependent form of B
reads
dæ; (xt,,, with 20 ~ q.
dt
d In (xt)
(x,) with qo =
df
By chain rule and substituting the first equation into the second one, we have
dq dq: dæ
( dge di Vg. Vo.().,
- dt dæ, dt -
which implies,
dq: = -. - Va V.VV, = - (4: V.,).
dt
By B-9), the above display coincides with the continuity equation (B-8) with Vr FV
V
(gr (x)(
Theorem B.1 1 and Proposition B.1 imply that {1e)e. the solution of the continuity equation (B-8 with (x)) = - F' (x)), converges rapidly to the target distribution v. Furthermore, the continuity equation has the following representation under mild regularity conditions on the velocity fields.
(H.. E L (R++) and vel-) E LiPioc with upper bound B, and Lipschitz constant L. such that (B, + L.) E (BR)). Then the solution of the continuity equation B-8) can be represented as μr = (X1)#11, where X( R I + R. satisfies the McKean- Vlasov equation (4)
Theorem B.2. Assume
Proof. The Lipschitz assumption of vn implics the existence and uniqueness of the McKean -eKean-Vllsov equation 4 according to the classical results in ODE (Arnold, (012). By the uniqueness of the continuity equation, see Proposition 8.1.7 in Ambrosio et al.) (2008), it is sufficient to show that 14 (X satisfies the continuity equation (B-8) in weak sense. This can be done by the standard test function and smoothing approximation arguments, see, Theorem 4.4 in Santambrogio (2015) for details.
As shown in Lemma B.I below, the velocity fields associated with the f-divergence 5 and the Lebesgue norm (B-7) are determined by density ratio and density difference respectively. Lemma B.1. The velocity fields satisfy
(x)) -f (r(((())) Clu = DD((Hl)), where ri(x)) (a))
p(z)
2Vd(()), Clu] = =llia where dxxx) = =q.(x) p(x).
Proof. By definition,
F(q((x)) = ((a)s(()))) G[n] = DD((ul()), (p()) Llu] = || 1 11122
Direct calculation shows
f((((=) ), C[n] = (H(()))) Mz)
((()
(q) C[u) = || (R
Then, the desired result follows from the above display and B-9
Several methods have been developed to estimate density ratio and density difference in the literature. Examples include probabilistic classification approaches, moment matching and direct density-ratio (difference) fitting, see (Sugiyama et al.) (2012a]b), Kanamori & Sugiyama (2014); Mohamed & Lakshminarayanan| (016) and the references therein.
19

---

Under review as a conference paper at ICLR 2021
Proposition B.2. Suppose that the velocity fields Lipschitz continuous with respect to (a. that is, there exists a finite constant L. >> such that
(x) vz( S T x- abl W2(pe, M and 2, u E TR".
(B-10)
Then for any finite T the bound 11 on the discretization ervor holds:
sup x0(7.
Wa(pe, 14,) O()).
Remark B.3. If we take (
/2 Lemma B.1 then the velocity fields = (r)
where (x) gr (r)))/((x). In the proof of Theorem B.1 part (ii), it is shown that satisfies
dq:/dl V q: v.). Thus for this simple [-divergence function, the verification of the Lipschitz
condition B-10 amounts verifying that Vr( is Lipschitz in the sense of (B-10
Proof. Without loss of generality let K = - > be an integer. Recall {uz t E [ks,, (k + 1)s) is the piecewise constant interpolation between μal and 1x+11 defined
M (Tr WMks
where,
T. = 1 + (t ks)
[A is defined in (16)-(18) with v% 0, O,.., K - 1, 14 μ.. Under assumption (B-10) we can first show in way similar to the proof of Lemma 10 Arbel et al. (2019) that
uks i.e., the continuous velocity in (B-9 time ks,
Wa(KES Mx) = O(s).
(B-11)
Let be the optimal coupling between li and jkss, and (X, r. Let X, =T,t'((') and Y, be the solution of with Xo Y and & [ks, (k + 1)s). Then
X₁ ~ pe; Y ur
and
Y = L of(Y))d..
follows that
(2 (1e (kss)
s B[Ily; 11
(B-12)
E[l| L oz (x(
S B[( [((((\)ll2dd)))
ks
S Oss2).
where, the first inequality follows from the definition of W2 and the last equality follows from the the uniform bounded assumption of vy. Similarly.
W (pk, 1G)
<
[|]| - X, 1
= Elll (
S (
(B-13)
Then,
Wa(me, pi)
Wa(me, Wa(uess + W2( m4) O(s),
where the first incquality follows from the triangle inequality, see for example Lemma 5.3 in Santambrogio (2015), and the second one follows from (B-1I BB13 1-133-
20

---

Under review as a conference paper at ICLR 2021
B.3 DERIVATION AND PROOFS OF THE RESULTS SECTION 4
B.3.1 BREGMAN SCORE FOR DENSITY RATIO/DIFFERENCE
The separable Bregman score with the base probability measure measure the discrepancy between
a measurable function R th + R! and the density ratio r
B ratio (r, R) Ex Exxplg'(R(X))) (R(X) r(X)) g
Ex-plg' (
((R(X))) Ex=elg' (x<glg((((X))) (R(X))]
be verified that Bratio (r, R) B (ratio r). where the equality holds iff R
For deep density-difffernce fitting, a neural network IRm + RR_ utilized to estimate the density-difference d(x)) g(x) - p(x) between given density q and the target p. The separable Bregman score with the base probability measure w to measure the discrepancy between D and d can be derived similarly,
Bain (d, D) (xx)pp(X) (D(X)) - (
+ Ex 1 (X) - =(D(X))
Here, we focus the widely used least-squares density-ratio (LSDR) fitting with g (c 1)² as a working example for estimating the density ratio r. The LSDR loss function is
BLSDR - [R())] 2Ex [R(X)] +
B.3.2 GRADIENT PENALTY
We consider noise convolution form of Bratio (r, R) with Gaussian noise .N( Bratio (r, It) = Ex- [g' (R{( + +)) - E) g([(( + +))]] EX E.lg (R(( + e))]
Taylor expansion applied to gives
Eg R(x + c)] R(x) +R(x)++
Using equations (13)-(17) in Roth et al. (2017), we get
Ba Patio (r, R) xs Bratio (r: R) + Eplg" [R) VR
ie.. tiable.
serves regularizer for deep density-ratio fitting when g is twice differen-
B.3.3 PROOFS IN SECTION
4
Below prove Theorem 4. in Section 4
Lemma B.2. For given densities p(æ) and q(x), let (xx) = <(z)/p(x) with Ex)g1(X)]]11 <<
For any 2 define nonnegative functional
BLSDDR (R) = (LSDR((, R) + [|V|] + C, R is measurable.
Then, r E arg ming BLSDR (R). Moreover, 2a iff R R(x) r(x) = (g.p)-a.e. x ERTH.
Proof. By definition, is easy to check
BLSDR Bratio R) - B Bratio (r,r),
where B (r, R) is the Bregman score with the base probability measure p between and r. Then E arg milsmmaseeebbllee R 23 BD (It) follow from the fact Bratio (r, R) Bratio (r, r) and the equality
holds iff R r. Since
Ba BRR
B0 BLSDD. R) + aEpl V Z
Then,
Ba B²) =
iff
20 BLSDR (R) = 0 and E VR = 0,
which is further equivalent to
= constant (q.p)-a.e.,
and the constant since is density ratio.
21

---

Under review as a conference paper at ICLR 2021
Theorem B.3. Assume supp(r) = 2r, and r(x) is Lipschitz continuous with the bound B and the Lipschitz constant L. Suppose the topological parameter of ,,.. in (20) with a satisfies D O(logn) n), W On 364M log n)), S On /lg' n.), and 2B. Then.
E [x, R s C(B² + -
where C. is universal constant.
Proof. We use (R) to denote BLSDD C' for simplicity, i.c.,
B(R) E Ex-p [R(X²] 2Ex ~x [R())
(B-14)
Rewrite (20) with 0 as
R E arg
min Rs Noo.wss
B(R =
Σ n
Ro (X,) (Y:)).
(B-15)
By Lemma B.2 and Fermat's rule (Clarke 1990), we know 0 E @B) Then, AR direct calculation yields.
|R - rl rll2 B(R) (B(R)-Brr)) (0B(r), R r) B(R) Br).
(B-16)
VR E WD,wss,s we have,
[Ro - rll: B(Rx) B(r)
(B-17)
B(Rx) (BRRs) B (Rs)
(Rx)) B(R)) + B(R)) B(r)
-
sup ReH..w.s.s
(B(R) B(R) + R - r [2(3)*
where the inequality uses the definition of R3 R and B-16 We prove the theorem by upper bounding the expected value of the right hand side term in B-17)
To this end, we need the following
auxiliary results (B-18) B (B-20).
B [(,,) (sup (sup B(R) B(R) < 42, + 1)6(H),
(B-18)
R
where
(HH) = E[2,,0.
sup
Σ R(()) i=1
R....
is the Gaussian complexity of HDDW.S,B
Bartlett & Mendelson 2002)
Proof of 1- Let g(c) = = - == -x;=((,,)) E RT x RM
R(z) R)) R$ (x) R(y))
Denote Z = (X,Y), Z; (X,, . 1 1,..,, with X, X; i.i.d. p. Y,Y, ii.d ~ Let 2, be an i.i.d. copy of Zi, and (e:) i.i.d. Rademacher random (standard normal) variables that are independent of Z; and Z, Then,
B(R) -
and
B(R) 1 Σ R(Z.).
Denote
R(?() = E(Z
sup
R(()))
n
REHo,w.s.e
22

---

Under review as a conference paper at ICLR 2021
as the Rademacher complexity of S.s. (Brrlettt & Mendelson 2002). Then,
Ei, (sup (R) - [(R])] = B(z,y sup
EE (T((2.) - R(Z)))
R
1=1 n2 (2, (.,
E
sup
R(Z,) R(Z,)]
E (Z, (Z,, 7.
(su R
oi (R)( R(Z)))I
i=1
<
Bi
(sup R
Σ
o. (())l + - _
sup R
o. a Z,)
1=1
2g o H)
4 + L)9R(H)
(2B + 1)((H))
where, the first inequality follows from the Jensen's inequality, and the second equality holds since the distribution of or (R((2 - (()) and R(2.) Rt( Zz) are the same, and the last equality holds since the distribution of the two terms are the same, and last two inequality follows from the Lipschitz
contraction property where the Lipschitz constant of g on H.D W.S.B is bounded by 2B + and the relationship between the Gaussian complexity and the Rademacher complexity, see for Theorem 12 and Lemma in Bartlett & Mendelson 2002 respectively.
(HH) S C2B, V DS S
log
exp(- log"
(B-19)
DS log S
DS loggs S
Proof of (B-19). Since H. is negation closed,
n
((H) E(z,c)) sup
c,R(Z,)
REHo.w.s.e n
Ez, Be,
sup REHD.W.S.B TI
Conditioning on {4)) VR, R E HD,w.s,B it easy to check
n
d2 R)
G (R(()) R(Z)))
n i=1
vn
where, (R, Rt)
Vn V Em (R(Z.) R((Z,))2. Observing the diameter of HD,w,s,s under dz
most B, we have
C3 E
16/² 13
6(H) <
(2,),
lo,, 8yd6
Vn
Ca
B(z))
VlogN(H. dz. 8)
Vni 6-²
Ca
B
6Bn do,
V
dVCH
< C.B(
1/2 log(
exp( ( log"
))
VC#
VC#
VC#2
S CBB V
n
log
DS log S exp log²
n
7
DSlog S
DS S
where, the first inequality follows from the chaining Theorem 8.1.3 in Vershynin 2011). and the second inequality holds due dzt dHt. and in the third inequality used the relationship between
23

---

Under review as a conference paper at ICLR 2021
the matric entropy and the VC-dimension of the ReLU networks w.s. (Anthony & Bartlett 2009), i.e..
6Bn
log PggN(H, doo 8) S VCH log 8VCH
and the fourth inequality follows by some calculation, and the last inequality holds due to the upper bound of VC-dimension for the ReI.U network HD, W,S,8 satisfying
VCH S C,DS
see Bartlett et al. 2019
For any two integer M, N, there exists RS H.D.W.s.E with width W max(8MNN/M
4.M, 12N + 14] and depth = 9M + 12, and B = 2B, such that
||| RolliA CeeLmM((NN))
(B-20)
Proof of (B-20).
We use Lemma 4.1, Theorem 4.3, 4.4 and following the proof of Theorem 1.3 in Shen et al. 201 9)..
Let A be the random orthoprojector in Theorem 4.4, then to check A (2R,) A c) C [-oym, Vmicl/M. Let F be extension of the restriction of Mt. which is defined similarly as g on page 30 in Shen et al. (2019). Since we assume the target is Lipschitz continuous with the bound B and the Lipschitz constant L, let small enough, then by Theorem 4.3, there exist a ReLU
network Rs E H...Ws.B with width
W = max/&MN + +4M, 12N + 14),
and depth
D= = 9M + 12,
and B = 2B, such that
|| Roll 80cT. (0cT.m/M(NM)) 2/m
and
( S B + 3LcVmM.
where, A is a w- negligible set with v(N) can be arbitrary small. Define RS Rsg A. Then. following the proof after equation (4.8) in Theorem 1.3 Shen et al. (219), we get our (B-20) and
[ W) s 2B, || (1- S 2B 3cL MMM.
Let DS S n, combing the results (B-17) B-20p, we have
E(x,v): R. - r rl(?:
{8C 1)0(H)) + CacLm. 1/M
DS DSlog& S
< 18G
log
n
DS log
CaclmmM(NM)
s CCB2 + cLmM)in -200M))m22/22MM 2/2+.M)
M where, last inequality holds since we choose M = log n, N गलनश
M-2
log n, S = n MM++ 1oM++1log" /log* n.
i..., D 9log 12, W 12n 12n 2 /log n + 14.
B.4 THE RELATIONSHIP BETWEEN EPT AND MMD FLOW
Here show that MMD flow can be considered special case of EPT.
Proof. Let H be reproducing kernel Hilbert space with characteristic kernel K(x, z). Recall in
MMD flow,
C[p] = 2 || 1
24

---

Under review as a conference paper at ICLR 2021
and
dC[n] OF
K(x,, z)du(z) / ((tx
(x)
and the vector fields
a64m
mnd
V (x, 2))x(z) /
V (x, p)
By Lemma B.1 the vector fields corresponding the Lebesgue norm 1 11u 11lu-v1 2 = 1/2 SRm (g(()) ddx are defined as
Vp(x) (x).
Next, we will show the vector fields v is exactly by projecting the vector fields er on to the reproducing kernel Hilbert space 7 Hm By the definition of reproducing kernel we have,
p(x)) = (p((), ((x,, 1) = / K(x, 2)p(z)dz,
and Hence,
(<(x) (ac(-),, ((x,)))( = 2)
vx(x)) = Vp(x) - Vg((x)
/ (e <p(()) qe (z)))
0
mund (x).
This completes the proof.
B.5 PROOF OF THE RELATION BETWEEN EPT AND SVGD
Proof. Let f(u) = u log u in (). With this ff the velocity fields v. (r) = Vr((x) in Stein class associated with 4t-
Let
rx(x)
(Ve, g) H(m)
/ g(x)" Vr((x)
qe (x)dx
(x)
g(x)² T V log r (x)q (x)dx
Ex~q [g
v 4: q: (X) - g(x)" V log)(XX]
Ex [g v
log 9: X)) V -g(x)]
+ Ex (x) [ lo
g(x)
Ex (x) T g] + Ex (x) [Tg]
Ex [pg].
where the last equality is obtained by restricting g in a Stein class associated with i.e.,
Ex~g(xx Tg, = 0. This is exactly the velocity fields of SVGD LLu 2017).
25


Reviews:
[{"review_id": "VnTeGP_skk4", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Pros:\n* The exploration to use a flow/transport map for generative modeling is inspiring and worth encouraging.\n* The writing roughly follows a clear logic flow.\n* The proofs seem valid to me.\n\nCons:\n* Method.\n  - My major concern is on whether the method indeed implements the optimal transport map. As I understand it, optimal transport map and gradient flow are two independent concepts. The former is determined by two distributions, while the latter by a function(al) of distribution which gives the gradient. I do not understand why various gradient flows can all approximate the optimal transport map. Particularly, the optimal transport map does not depend on the choice of the function f in the f-divergence, while different choices of f give different gradient flows.\n\n    It makes sense to approach to the target distribution $\\nu$ by minimizing the f-divergence of the current distribution $\\mu_t$ to $\\nu$, which can be done by approximating the gradient flow of the f-divergence. But it does not seem to have a relation to optimal transport, which is the claimed motivation of the work.\n* Writing.\n  - Some theoretical results seem to be already well-established, e.g., results in Theorem B.1 are all covered by Villani (2008) and Ambrosio et al. (2008). The authors may need to rephrase these results as an introduction to the background knowledge of this area, so that the novel theoretical contributions can be highlighted.\n  - The main paper frequently refers to the appendix, which feels a little disturbing.\n  - It seems to me that $q(x)$ in Eq. (2) should be $q(z)$, since $q$ is the density of $\\mu$, which represents the easy-to-sample measure/distribution on the latent space, and $Z \\sim \\mu$.\n\n=== EDIT: post-rebuttal ===\n\nI thank the authors for their patient and detailed replies regarding my concerns. I noticed that the authors have addressed the concerns to some extent in the updated paper (e.g., the remark in Conclusion). I'm also aware that the method can achieve the machine-learning goal of transporting a reference distribution to the data distribution, regardless of my concern. So I raised my score by 1 point.\n\nNevertheless, I still feel uncomfortable to give a positive score. The current presentation of the motivation may confuse or even mislead the community. The authors present the Monge-Ampere Eq. (2), which solves for the _optimal_ transport from the reference distribution to the data distribution. But the method is constructed by simulating the gradient flow of f-divergence. Although the resulting transport serves for transforming the reference distribution to the data distribution (since the gradient flow minimizes their f-divergence), the transport is unnecessarily _optimal_ (there may be multiple transports to transform the reference distribution to the data distribution; the gradient-flow transport is one of them, which may not be the optimal one that solves the Monge-Ampere equation).\n\nFor the authors' reply \"Our method is based on a first-order approximation to the Monge-Ampere equation\", I think the \"Monge-Ampere equation\" therein is different from the original one (Eq. (2)). If the method, which is constructed from simulating the gradient flow of the f-divergence, is to be treated as a first-order approximation to some Monge-Ampere equation, then the Monge-Ampere equation is between two adjacent/neighboring distributions on the gradient flow curve, but Eq. (2) is between the two ending points of the curve (i.e., the reference distribution and the data distribution). The two Monge-Ampere equations have different solutions unless the gradient flow coincides with the geodesic on the Wasserstein space, which is unnecessarily the case.\n\nI agree that \"An important advantage of the proposed approach is that it allows general energy functional in constructing the gradient flow from the reference distribution to the target distribution\". But I think it does not have much to do with _optimal_ transport or the Monge-Ampere equation. This thought works since the gradient flow minimizes the energy functional, whose minimum is achieved only when the two distributions coincide (when the energy functional is a proper divergence/discrepancy).", "extra_scores": {}, "extra_text": {}}, {"review_id": "D_8YGn6WZH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "==== Summary ====\n\nThis paper tackles generative modeling (sampling, in particular) via finding the push forward functions T (equivalently, the velocity fields v) that iteratively moves particles from a reference distribution toward the target data distribution. The velocity fields are solved by minimizing the f-divergence between the particle density at iteration k and the target data density, which is shown to be in the form of gradient of density ratio. Based on this intuition, the training stage becomes estimating the density ratio via neural networks, for each iteration k=1,...,K. However, estimating the density ratio can be quite difficult when two densities have little overlapping support. Thus, the author proposes to add gradient regularizer to the density ratio estimating function. The experiment on real-world computer vision benchmarks demonstrate reasonable sampling quality, and the FID score on CIFAR-10 is comparable to some GAN baselines in the generative modeling literature.\n\n\n==== Pros and Cons ====\n\nPros:\n\n(1) Technical sounds and interesting insights for non-parametric density ratio estimation with low-dimensional assumptions. \n\n(2) Writing is clear, sections/paragraphs are well structured.\n\nCons:\n\n(1) The training and inference time seems way larger than GANs because of the iterative updating nature. The inference time is more close to score-based generative models.\n\n(2) Finding velocity fields and conduct inference is very related to score-based generative modeling [1,2,3], and the author should discuss the connections, see detailed below.\n\n(3) The experiments on real-world datasets are not very compelling, and lack some ablation studies, see detailed below.\n\n\n==== Technical Questions ====\n\nQ1:What are the connections to score-based generative models? For example, [1] also analyze the particle evolution with the Vlasov process, and interpreting it as a gradient flow for minimizing the KL divergence. [2,3,4] are state-of-the-art score-based generative models that iteratively move particles based on the velocity field (i.e., score functions). The experimental comparison is missing.\n\nQ2: The learning objective of decoder G_theta is not justified. Why squared loss? Is that just memorizing the Y_tilde? \n\nQ3: Lack of ablation study for real-world datasets. What is the FID score in CIFAR-10 for your method without the outer loop in Algo 2?  What is the number of inner/outer iterations? Also from the Table A3 and A4, it seems like the coefficient for gradient regularizer alpha = 0. Does this mean that the density ratio estimation problem does not suffer from the no overlapping support issue? If so, why introducing it in Section 4?\n\nQ4: What are the training and inference time for high-dimensional image datasets such as CIFAR-10 and CelebA? In Algorithm 2, how many density ratio estimators R_{phi}^{k} do you learn? Is it that for each outer-inner loop (j,k) pair, you need to learn one? If so, that seems like a lot of models.\n\n==== Additional Feedback / minor suggestions ====\n\n(1) Subplots in Figure 3 are too small. I can not see the figures’ legend.\n\n(2) The Pytorch code of EPT link is not working (page not found on github).\n\n(3) In the first paragraph of Section 3, the right-hand-side of velocity fields seems to have unbalanced parenthesis: f’’(R)(x)) => f’’(R(x)) ?\n\n==== Reference ====\n\n[1] Stein Variational Gradient Descent as Gradient Flow, NeurIPS 2017\n\n[2] Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019\n\n[3] Improved Techniques for Training Score-Based Generative Models, NeurIPS 2020\n\n[4] Denoising Diffusion Probabilistic Models, NeurIPS 2020\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "8MC1oRwt5VN", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers generative learning by discretizing a Wasserstein gradient with Euler methods. More precisely, some samples of a target distribution are given and the goal is to pushforward some samples of an initial distribution to the target distribution. The proposed method is obtained by  minimizing the f-divergence between the initial distribution and the target distribution, but considering the Wasserstein gradient flow of the f-divergence w.r.t. the target distribution (= the objective function). This Wasserstein gradient flow is discretized via Euler method to obtain the proposed algorithm. This Euler method involves the Wasserstein gradient of the objective, which is intractable. The authors describe a statistical methodology to compute this Wasserstein gradient based on the samples of the target distribution and samples from the current distribution. They also prove a bound for the estimated Wasserstein gradient wrt the true Wasserstein gradient. Finally, the paper presents relevant numerical experiments.\n\nOverall the approach is natural, the topic is interesting and paper is well written. The authors show how to approximate the Wasserstein gradient, which is the main technical point in the implementation of the ETP for f-divergences. They also prove a bound for it and the simulations are promising. \nHowever, some typos/approximative statements make the reading experience difficult (see below). \nEq (10). The linear convergence of the gradient flow to the target distribution is not something standard, since it usually requires the objection to be lambda-geo-convex (lambda > 0). It holds if the f-divergence is KL and the target strongly log concave but beyond this case I don't know. Could the author give other examples? Moreover, I don't believe that it is a good strategy to claim strong results like this without justifying.\nEq (11). I don't see why Eq B-10 holds here and I suspect that it does not. Can the authors prove that v_t is jointly Lipschitz? Otherwise, remove Eq (11). Moreover, Eq (11) does not imply convergence to the target distribution as claimed. The reason is that Eq (11) is a uniform convergence result over compact sets [0,T]. This does not imply convergence in the limit T = infty in general, as well documented in many works, see e.g. Kushner & Yin 2003. The situation is much more complicated (an additional stability result would be needed).\nTh 4.1. How reasonable is it to assume r Lipschitz. In the EPT, r depends on the current density, whose regularity is unknown. Moreover, How could be combine this bound in a study of the EPT itself. Can we study the convergence of the EPT? I understand that these questions are difficult and might not be answered by the authors now, but they should at least be raised as limitations of the paper. In the current form, the reader has the feeling that the authors just overlooked most of the fundamental questions.\nFinally, why the composition of transport maps constructed by the algorithm should approximately be the optimal one? This would depend on the objective function which is an f-divergence. I don't understand the linearization of Monge Ampere equation. \n\nOverall, the paper and the result are interesting but more rigor should be put on justifying the claims. Unjustified claims are not  unnoticed. Moreover, the fundamental questions should be raised. I rate the paper as marginally above, as I appreciate the bound on the Wasserstein gradient which is a first step toward analyzing the ETP (and is relevant from a statistical estimation point of view). The num exp also show interesting results for generative learning tasks.\n\nMINOR : \n\nETP vs EPT throughout the paper\nSection 2: \"Lebesque\"\nSection 3, line 6-7. typos (parentheses). Moreover, some notations are not yet defined\nMore details (like an intuitive explanation of Lemma 4.1) on the LSDR methodology would be appreciated in the main paper.\nAfter Lemma 4.1, \"Lipsichiz\"\nTh 4.1: \"with the bound B\" ??\nRelated works on discretized gradient flows : https://arxiv.org/pdf/2002.03035.pdf, https://arxiv.org/pdf/1704.07520.pdf, https://arxiv.org/abs/2006.02509, https://arxiv.org/abs/2006.09797 among others.\nCheck end of first paragraph of Page 7.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a discretization of Wasserstein gradient flow with an euler scheme, and propose a way to estimate each step of the euler scheme using ratio estimators from samples regularized with gradient penalties. Statistical bounds are given to bound the estimated flows from the wasserstein flow.  

Reviewers have raised concerns regarding the assumptions under which results present in the paper hold, this was clarified by the authors (goedesic lambda convexity, log sobolev constant for the target density . lipchitizity of velocity fields).  The paper needs a revision to incorporate that feedback and to be in shape for publication. 

Other concern were on earlier claims in the paper regarding the monge ampere equation and approximation of the optimal mapping this was addressed by the rebuttal. 

Other concerns were also on explaining the relation of the work to score based models and energy based models. 

Overall the paper needs to state in a clearer way the assumptions for the theoretical results and to acknowledge the limitations of those assumptions in analyzing the euler scheme. 


Decision: Reject